In today's in-person group discussions, smartphones are integrated as intelligent workstations; yet given their co-presence in such face-to-face interactions, whether and how they may enhance people's behavioral engagement with others remains underexplored. This work investigates how animating personal smartphones to move expressively, without compromising regular functions, can transform them into active embodied facilitators for co-located group interaction. In the four-stranger small-group discussion setting, guided by Tuckman's group-development theory, we conducted a design workshop (n=12) to identify problematic group-work circumstances and design expressive, attention-efficient animated phone facilitations. Subsequently, we developed AnimaStand, a movement-enabled phone stand that animates phones to deliver group facilitation cues according to conversation dynamics. In a between-subjects Wizard-of-Oz study (n=56) with four-stranger group discussions, where everyone's phone was on an AnimaStand, the facilitations re-engaged inactive members, enhancing group dynamics, task operation performance, and relationships. We finally discuss prospects for more adaptive and generalizabl
End-of-Life (EoL) phones significantly exacerbate global e-waste challenges due to their high production volumes and short lifecycles. Disassembly is among the most critical processes in EoL phone recycling. However, it relies heavily on human labor due to product variability. Consequently, the manual process is both labor-intensive and time-consuming. In this paper, we propose a low-cost, easily deployable automated and selective disassembly and sorting system for EoL phones, consisting of three subsystems: an adaptive cutting system, a vision-based robotic sorting system, and a battery removal system. The system can process over 120 phones per hour with an average disassembly success rate of 98.9%, efficiently delivering selected high-value components to downstream processing. It provides a reliable and scalable automated solution to the pressing challenge of EoL phone disassembly. Additionally, the automated system can enhance disassembly economics, converting a previously unprofitable process into one that yields a net profit per unit weight of EoL phones.
The popularity of touchscreen phones has been growing around the world since the iPhones and Android phones were released. More and more mobile phones with large touchscreen have been produced, however, the phones with small size displays are still in the majority of touch phones. The foremost interface on touch smartphones is the information input module using soft keyboards. Traditional input methods on touch phones have either too small key buttons (such as QWERTY) or too many functions (such as 3$\times$4 keyboard), which are inconvenient to use. Moreover, the conventional soft keyboards only use tapping to input texts while current touch smartphones allow various gestures on the touchscreen, such as sliding. In this paper, a novel soft keyboard called QWERT is proposed for touchscreen-based smartphones. The users can interact with phones via finger gestures of tapping or sliding when input text by using the QWERT. In doing so, the interactions between users and smartphones will be faster and easier. An experiment carried out on inexperienced human subjects shows that they can learn very fast due to their familiarities with QWERTY. A simulation experiment based on a cognitive a
With the popularity of smartphones and tablets, users have become accustomed to using different devices for different tasks, such as using their phones to play games and tablets to watch movies. To conquer the market, one app is often available on both smartphones and tablets. However, although one app has similar graphic user interfaces (GUIs) and functionalities on phone and tablet, current app developers typically start from scratch when developing a tablet-compatible version of their app, which drives up development costs and wastes existing design resources. Researchers are attempting to employ deep learning in automated GUIs development to enhance developers' productivity. Deep learning models rely heavily on high-quality datasets. There are currently several publicly accessible GUI page datasets for phones, but none for pairwise GUIs between phones and tablets. This poses a significant barrier to the employment of deep learning in automated GUI development. In this paper, we collect and make public the Papt dataset, which is a pairwise dataset for GUI conversion and retrieval between Android phones and tablets. The dataset contains 10,035 phone-tablet GUI page pairs from 5,5
Hearing research requires a calibrated sound source, traditionally as lab equipment. Online research is quicker and more inclusive, but most participants lack calibration equipment and their sound sources are uncalibrated and diverse. This article explains how the open-source EasyEyes.app calibrates loudspeakers online. A library of smartphone-microphone profiles allows EasyEyes to use the participant's phone to calibrate their computer's loudspeaker in three minutes. Participants select their phone model, which is verified by screen size. Calibration employs the Novak et al. nonsynchronous maximum-length-sequence (MLS) algorithm. The computer's loudspeaker is corrected by convolving its input with the inverse of its impulse response. Researchers can contribute to the open-access library by calibrating phones with a measurement microphone. In the library, each profile is linked back to the profile used to produce it, back to the manufacturer profile of a measurement microphone. Correction accuracy is such that playing the flat-spectrum MLS through the corrected loudspeaker produces a nearly flat spectrum, with standard deviation less than 3 dB. A survey shows that a library of 94 p
In the current landscape of pervasive smartphones and tablets, apps frequently exist across both platforms. Although apps share most graphic user interfaces (GUIs) and functionalities across phones and tablets, developers often rebuild from scratch for tablet versions, escalating costs and squandering existing design resources. Researchers are attempting to collect data and employ deep learning in automated GUIs development to enhance developers' productivity. There are currently several publicly accessible GUI page datasets for phones, but none for pairwise GUIs between phones and tablets. This poses a significant barrier to the employment of deep learning in automated GUI development. In this paper, we introduce the Papt dataset, a pioneering pairwise GUI dataset tailored for Android phones and tablets, encompassing 10,035 phone-tablet GUI page pairs sourced from 5,593 unique app pairs. We propose novel pairwise GUI collection approaches for constructing this dataset and delineate its advantages over currently prevailing datasets in the field. Through preliminary experiments on this dataset, we analyze the present challenges of utilizing deep learning in automated GUI development
We describe a comprehensive methodology for developing user-voice personalized automatic speech recognition (ASR) models by effectively training models on mobile phones, allowing user data and models to be stored and used locally. To achieve this, we propose a resource-aware sub-model-based training approach that considers the RAM, and battery capabilities of mobile phones. By considering the evaluation metric and resource constraints of the mobile phones, we are able to perform efficient training and halt the process accordingly. To simulate real users, we use speakers with various accents. The entire on-device training and evaluation framework was then tested on various mobile phones across brands. We show that fine-tuning the models and selecting the right hyperparameter values is a trade-off between the lowest achievable performance metric, on-device training time, and memory consumption. Overall, our methodology offers a comprehensive solution for developing personalized ASR models while leveraging the capabilities of mobile phones, and balancing the need for accuracy with resource constraints.
The failure mode of the phone falling has a lot to do with the ground material. At present, the research on ground material and mobile phone damage is generally carried out through experiments, which is extremely costly. This paper presents a method to identify the material of mobile phones falling on the ground. The method determines the material of the mobile phone falling to the ground according to the data of the mobile phone accelerometer and can obtain the ground material of the mobile phone falling through a large number of user data. By analyzing the physical process of mobile phone falling, the accelerometer data interval which can reflect the characteristics of falling is reasonably intercepted. And the data features that can reflect the collision are extracted. Finally, based on the fully connected neural network, the method of determining the material of mobile phones falling on the ground is developed. The experimental results show that the method has a high identification rate, and the average identification rate for different ground materials is 96.75%. Instead of relying on auxiliary devices, the identification process only relies on the phone's built-in acceleratio
The proliferation of mobile phone usage among learners from diverse socio-economic backgrounds has prompted school authorities to contemplate banning these devices within educational institutions. This research seeks to explore the motivations and usage patterns of high school learners in response to the proposed ban. Employing a mixed-methods approach, we conducted surveys and interviews with 262 students from three township schools in the Western Cape province of South Africa. Grounded in the Uses and Gratification Theory (UGT), our study examined four key categories: reasons for mobile phone use, usage patterns, purchasing influences, and behavioral factors. Our findings reveal a predominant opposition among students to the ban, despite a significant number opting to leave their phones at home due to concerns about theft and robbery in their neighborhoods. Financial constraints, specifically the inability to afford data bundles and airtime, also contribute to this behavior. Notably, 40% of the participants reported using their phones for more than five hours daily, a duration classified as overuse in existing literature. The primary motivations for mobile phone use among these l
With the increasing interconnection of smart devices, users often desire to adopt the same app on quite different devices for identical tasks, such as watching the same movies on both their smartphones and TVs. However, the significant differences in screen size, aspect ratio, and interaction styles make it challenging to adapt Graphical User Interfaces (GUIs) across these devices. Although there are millions of apps available on Google Play, only a few thousand are designed to support smart TV displays. Existing techniques to map a mobile app GUI to a TV either adopt a responsive design, which struggles to bridge the substantial gap between phone and TV or use mirror apps for improved video display, which requires hardware support and extra engineering efforts. Instead of developing another app for supporting TVs, we propose a semi-automated approach to generate corresponding adaptive TV GUIs, given the phone GUIs as the input. Based on our empirical study of GUI pairs for TVs and phones in existing apps, we synthesize a list of rules for grouping and classifying phone GUIs, converting them to TV GUIs, and generating dynamic TV layouts and source code for the TV display. Our tool
Learning phone types from phone instances has been a long-standing problem, while still being open. In this work, we revisit this problem in the context of self-supervised learning, and pose it as the problem of matching cluster centroids to phone embeddings. We study two key properties that enable matching, namely, whether cluster centroids of self-supervised representations reduce the variability of phone instances and respect the relationship among phones. We then use the matching result to produce pseudo-labels and introduce a new loss function for improving self-supervised representations. Our experiments show that the matching result captures the relationship among phones. Training the new loss function jointly with the regular self-supervised losses, such as APC and CPC, significantly improves the downstream phone classification.
Motion blur of fast-moving subjects is a longstanding problem in photography and very common on mobile phones due to limited light collection efficiency, particularly in low-light conditions. While we have witnessed great progress in image deblurring in recent years, most methods require significant computational power and have limitations in processing high-resolution photos with severe local motions. To this end, we develop a novel face deblurring system based on the dual camera fusion technique for mobile phones. The system detects subject motion to dynamically enable a reference camera, e.g., ultrawide angle camera commonly available on recent premium phones, and captures an auxiliary photo with faster shutter settings. While the main shot is low noise but blurry, the reference shot is sharp but noisy. We learn ML models to align and fuse these two shots and output a clear photo without motion blur. Our algorithm runs efficiently on Google Pixel 6, which takes 463 ms overhead per shot. Our experiments demonstrate the advantage and robustness of our system against alternative single-image, multi-frame, face-specific, and video deblurring algorithms as well as commercial products
This article proposes and documents a machine-learning framework and tutorial for classifying images using mobile phones. Compared to computers, the performance of deep learning model performance degrades when deployed on a mobile phone and requires a systematic approach to find a model that performs optimally on both computers and mobile phones. By following the proposed pipeline, which consists of various computational tools, simple procedural recipes, and technical considerations, one can bring the power of deep learning medical image classification to mobile devices, potentially unlocking new domains of applications. The pipeline is demonstrated on four different publicly available datasets: COVID X-rays, COVID CT scans, leaves, and colorectal cancer. We used two application development frameworks: TensorFlow Lite (real-time testing) and Flutter (digital image testing) to test the proposed pipeline. We found that transferring deep learning models to a mobile phone is limited by hardware and classification accuracy drops. To address this issue, we proposed this pipeline to find an optimized model for mobile phones. Finally, we discuss additional applications and computational co
Recent research has made great progress in realizing neural style transfer of images, which denotes transforming an image to a desired style. Many users start to use their mobile phones to record their daily life, and then edit and share the captured images and videos with other users. However, directly applying existing style transfer approaches on videos, i.e., transferring the style of a video frame by frame, requires an extremely large amount of computation resources. It is still technically unaffordable to perform style transfer of videos on mobile phones. To address this challenge, we propose MVStylizer, an efficient edge-assisted photorealistic video style transfer system for mobile phones. Instead of performing stylization frame by frame, only key frames in the original video are processed by a pre-trained deep neural network (DNN) on edge servers, while the rest of stylized intermediate frames are generated by our designed optical-flow-based frame interpolation algorithm on mobile phones. A meta-smoothing module is also proposed to simultaneously upscale a stylized frame to arbitrary resolution and remove style transfer related distortions in these upscaled frames. In addi
In this paper we present the preliminary measurement results of harvesting radio frequency(RF) energy from the mobile phones. The aim is to revolutionize the way mobile phones are being charged and paving a way of charging the future mobile phones through RF energy harvesting. In order to measure the amount of energy that can be harvested, mobile phones from two different manufactures namely Asus and Samsung have been used. It was shown that depending on the manufacturer it is possible to harvest 1.53 joules amount of energy per day.
Research in location determination for GSM phones has gained interest recently as it enables a wide set of location based services. RSSI-based techniques have been the preferred method for GSM localization on the handset as RSSI information is available in all cell phones. Although the GSM standard allows for a cell phone to receive signal strength information from up to seven cell towers, many of today's cell phones are low-end phones, with limited API support, that gives only information about the associated cell tower. In addition, in many places in the world, the density of cell towers is very small and therefore, the available cell tower information for localization is very limited. This raises the challenge of accurately determining the cell phone location with very limited information, mainly the RSSI of the associated cell tower. In this paper we propose a Hidden Markov Model based solution that leverages the signal strength history from only the associated cell tower to achieve accurate GSM localization. We discuss the challenges of implementing our system and present the details of our system and how it addresses the challenges. To evaluate our proposed system, we impleme
Human beings produce electronic waste (e-waste) at an unprecedented pace. Mobile phones and other inter-connected smart devices make a significant contribution to the generation of e-waste. Reverse logistics (RL) activities play an essential role in managing mobile phones during their end-of-life. However, remanufacturing and/or refurbishing of mobile phones might prove difficult not only from an operational point of view but also from a data management and privacy perspective (due to privacy-related regulatory frameworks like the EU General Data Protection Regulation directive). In this paper, we propose a distributed trustless and secure framework for electronic equipment RL activities based on blockchain technology. We consider the remanufacturing/refurbishing recovery option for mobile phones, and we develop an autonomous and effective back-end data sharing architecture based on smart contracts/blockchain technology for keeping track of all the remanufacturing/refurbishing processes. For demonstrating the applicability of our approach, we develop a functional set of smart contracts and a local private blockchain. The benefits of our framework are further discussed, along with f
Mobile phones play increasingly bigger role in our everyday lives. Today, most smart phones comprise a wide variety of sensors which can sense the physical environment. The Internet of Things vision encompasses participatory sensing which is enabled using mobile phones based sensing and reasoning. In this research, we propose and demonstrate our DAM4GSN architecture to capture sensor data using sensors built into the mobile phones. Specifically, we combine an open source sensor data stream processing engine called 'Global Sensor Network (GSN)' with the Android platform to capture sensor data. To achieve this goal, we proposed and developed a prototype application that can be installed on Android devices as well as a AndroidWrapper as a GSN middleware component. The process and the difficulty of manually connecting sensor devices to sensor data processing middleware systems are examined. We evaluated the performance of the system based on power consumption of the mobile client.
WiFi fingerprinting is one of the mainstream technologies for indoor localization. However, it requires an initial calibration phase during which the fingerprint database is built manually. This process is labour intensive and needs to be repeated with any change in the environment. While a number of systems have been introduced to reduce the calibration effort through RF propagation models or crowdsourcing, these still have some limitations. Other approaches use the recently developed iBeacon technology as an alternative to WiFi for indoor localization. However, these beacon-based solutions are limited to a small subset of high-end phones. In this paper, we present HybridLoc: an accurate low-overhead indoor localization system. The basic idea HybridLoc builds on is to leverage the sensors of high-end phones to enable localization of lower-end phones. Specifically, the WiFi fingerprint is crowdsourced by opportunistically collecting WiFi-scans labeled with location data obtained from BLE-enabled high-end smart phones. These scans are used to automatically construct the WiFi-fingerprint, that is used later to localize any lower-end cell phone with the ubiquitous WiFi technology. Hyb
Phones are becoming an important execution surface for general-purpose agents, but training open models for reliable phone use remains difficult because the environment that matters at deployment, real devices running real apps, is slow, stateful, side-effectful, and hard to reset or verify, while scalable mock environments only approximate real behavior. We present PhoneBuddy, a training recipe and open-model line for agentic phone use that combines a real-app environment with a mock-app environment, PhoneWorld, which reconstructs runnable mock apps from real GUI usage structure. PhoneBuddy first builds a shared supervised fine-tuning stage from trajectories collected in both environments, then compares real-app RL against mixed RL across both environments. Across a 150-task human evaluation on real phones spanning apps, mini-apps, and cross-app workflows, task success rate improves from 36.67\% after supervised fine-tuning to 40.67\% after real-app RL and 45.33\% after mixed RL. On AndroidWorld, the same progression rises from 60.3\% to 77.2\% to 83.2\%. These results show that mock-app training is not a replacement for real-app RL, but a complementary source of scalable, resetta