搜索结果：Text-to-Speech

Extended Reality (XR) technologies offer transformative potential for language education, yet current platforms largely neglect the accessibility needs of deaf and hard-of-hearing individuals. Existing solutions typically operate in single-language environments and lack integrated support for sign language and multimodal communication. There is a critical need for inclusive platforms that serve both deaf and hearing learners through cross-modal AI services embedded in immersive environments. This study presents a modular platform integrating six AI services: speech-to-text transcription (OpenAI Whisper), multilingual translation (Meta NLLB), text-to-speech synthesis (AWS Polly), sentiment analysis (RoBERTa), session summarisation (flan-t5-base-samsum), and International Sign (IS) translation via Google MediaPipe. An IS dataset of 750 gesture videos was processed to extract hand landmark coordinates mapped to 3D avatar animations within a Unity-based VR environment on Meta Quest 3 headsets. The system was validated through technical benchmarking of AI service performance, including comparative evaluation of text-to-speech services and multilingual translation models (NLLB-200 and EuroLLM 1.7B variants), load testing to assess platform. scalability, and end-to-end pipeline latency measurement for both the hearing and the deaf user pathways. The educational scenario was additionally evaluated in a companion pilot study, 50 which shares the same underlying AI services and provides complementary user-perception evidence. Technical benchmarking confirmed the platform's viability for real-time XR deployment. TTS benchmarking confirmed AWS Polly's lowest latency (50-100 ms first byte) at competitive cost. The EuroLLM 1.7B Instruct model achieved a BLEU score of 84.34, outperforming NLLB's 79.25. Load testing with 1,000 simulated concurrent users demonstrated average response times under 800 milliseconds with no critical failures. Avatar animation latency for IS sign rendering remained consistently under 300 milliseconds. End-to-end pipeline latency averaged 2.05 ± 0.31 s for the hearing pathway and 2.32 ± 0.34 s for the deaf (IS) pathway, both within accepted thresholds for conversational educational use. The companion pilot (N = 10) reported a mean overall experience rating of 4.6/5.0, 92% user satisfaction and unanimous (100%) demand for expanded language and sign-language support. 50. The results presented in this study focus on the technical feasibility of integrating cross-modal AI services within XR environments for accessible, multilingual language learning. The modular architecture enables independent scaling and adaptation to diverse contexts, laying the groundwork for equitable educational solutions aligned with EU digital accessibility objectives. Learning a new language can be challenging, and it is even more difficult for deaf individuals who rely on sign language. This study addresses the challenge by creating a virtual reality (VR) learning environment where a digital 3D character (avatar) can speak, translate, and perform sign language in real time. The system uses several artificial intelligence tools working together: one that converts speech into text, another that translates text between multiple languages, a third that converts text back into spoken language, and a fourth that translates text into International Sign Language gestures performed by the avatar. Users wear a VR headset and interact with the avatar in a virtual classroom where they can select their preferred language and receive immediate translations in both spoken and signed forms. The system’s technical performance was validated through benchmarking of AI translation models, text-to-speech services, end-to-end pipeline latency measurement, and scalability testing, confirming its suitability for real-time educational applications. Complementary user feedback gathered in a companion pilot study, which uses the same underlying AI services, is cross-referenced where relevant. 50 This research is a step toward creating virtual learning environments where language barriers and hearing limitations no longer prevent people from accessing education.

This study proposed the integration of a laser Doppler vibrometer sensing with a Variational Inference with adversarial learning for Text-to-Speech-based voice conversion system to enhance automatic speech recognition for individuals with dysarthria in noisy environments. The proposed framework combines the noise robustness of laser Doppler vibrometer and generative modeling capabilities of Variational Inference with adversarial learning for Text-to-Speech to transform dysarthric speech into intelligible acoustic outputs. Experimental results demonstrated significant gains in automatic speech recognition accuracy compared with conventional acoustic methods, even at low signal-to-noise ratios. These findings establish a foundation for future clinical applications of augmentative and alternative communication systems.

Effective patient education is crucial in preventing venous thromboembolism (VTE), improving patient outcomes, and reducing health care costs. However, traditional educational methods often lack engagement and fail to address individual patient needs comprehensively. This study aimed to develop and preliminarily validate an immersive, large language model-based patient education system for VTE designed to promote patient engagement and care adherence by delivering highly relevant, actionable, and patient-centered information. We developed ChatVTE, an interactive, intelligent patient education platform, by integrating a retrieval-augmented large language model (Qwen1.5-7B) with text-to-speech and lip-synch technologies. The system's performance was initially assessed through a comparative evaluation against ChatGPT. This involved using a standardized set of VTE-related questions, administered from December 10 to 31, 2024, with responses rigorously evaluated by 4 VTE domain experts using a 5-point Likert scale for accuracy, completeness, consistency, and safety. Subsequently, we consecutively enrolled a prospective cohort of 25 adult inpatients with VTE from the Departments of Pulmonary Vascular and Thrombotic Diseases and General Surgery at the Sixth Medical Center of the Chinese People's Liberation Army General Hospital between March 1 and May 31, 2025. These participants engaged with the ChatVTE system throughout their inpatient stay and completed postintervention assessments upon discharge. Expert evaluation demonstrated that ChatVTE significantly outperformed ChatGPT in accuracy, completeness, consistency (all P<.001, r>0.5), and safety (P=.01, r=0.327). Among the 25 enrolled patients (age: mean 55.4, SD 13.2 years), ChatVTE achieved high average scores (mean score >4.0/5.0) in 8 of the 9 experience dimensions evaluated but received a notably lower score in the emotional support domain (1.92/5.0). This study validates the feasibility of ChatVTE in the management of patients with VTE, demonstrating its potential to enhance the quality of patient-health care provider interaction and the efficacy of knowledge dissemination. These preliminary findings suggest that ChatVTE could be a valuable tool for improving patient education and facilitating shared clinical decision-making.

Navigating through everyday environments, like walking down a sidewalk, which many people often take for granted, is a difficult task for millions of people with vision impairments since it involves sophisticated object detection, depth perception, and situational awareness, all working seamlessly to guide a person through complex surroundings. Many current assistive devices for vision-impaired people are either expensive, information-overabundant, or missing critical information. This paper details our Vision Alarming System (VAS), which can improve the safety for blind and vision-impaired people by providing awareness of both positions and nature of nearby obstacles; thus, assisting users to make decisions to avoid collisions, reduce accidents and casualties, while enhance their experience, independence, and confidence when participating in traffic. VAS is an Artificial Intelligence/Internet-of-Things (AI/IoT)-powered system developed utilizing the cutting-edge Raspberry Pi 5, a Light Detection and Ranging (LiDAR) sensor, and an AI depth camera, operating as different containers in a Docker architecture, and leveraging a Robotic Operating System 2 (ROS 2) backbone. VAS communicates the obstacle detections to users via Bluetooth interface, using the neural Text-To-Speech (TTS) system, namely, Piper, and the Sound eXchange (SoX) technologies. Our proof-of-concept system proves that VAS can be a standalone, open-source, extremely low cost, low power consumption assistive device which can synergistically utilize the cutting-edge AI/IoT technologies to provide blind and vision-impaired users with an appropriate amount of critical information about their surrounding environments.

Background/Objectives: The objective of this scoping review was to map and critically describe emerging speech-in-noise assessment tools developed over the last decade for the evaluation of hearing loss beyond conventional audiological measures. Methods: This review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines. A comprehensive literature search was performed in the PubMed/MEDLINE, Scopus, and Embase databases. A comprehensive review of studies describing novel or emerging SIN-based assessment tools was conducted, with a particular emphasis on those including adult participants with normal hearing and hearing loss. Results: Nine studies met the inclusion criteria and were included in the review. The identified tools cover a range of methodological innovations, including advanced digits-in-noise paradigms, antiphasic and binaural presentation modes, optimized adaptive procedures, and digital or automated testing platforms. Several studies also incorporated artificial intelligence-based approaches, such as machine learning, text-to-speech, and automatic speech recognition, to enhance test development, administration, and hearing loss classification. Across all studies, SIN measures demonstrated the ability to reliably differentiate between normal hearing listeners and individuals with hearing loss and to provide complementary information beyond pure-tone audiometry. Conclusions: Emerging speech-in-noise tools show considerable potential to improve the functional assessment of hearing loss and to support more sensitive, accessible, and scalable approaches for hearing evaluation. Further research is required to assess their clinical integration and long-term impact on hearing screening and diagnostic pathways.

This research presents the development and empirical evaluation of an AI-enhanced interactive storytelling system designed specifically for children with cognitive disabilities. Addressing the critical gap in accessible educational technologies, our system integrates multiple artificial intelligence components - including speech recognition, text-to-speech conversion, and adaptive narrative generation - within a comprehensive accessibility framework. We employ a three-tier architecture leveraging Neo4j graph database technology for efficient management of complex branching narratives and user interaction data. Following an agile development methodology with Scrum framework, the platform was evaluated through a six-week study involving 45 children with diverse cognitive profiles. Quantitative analysis reveals significant improvements across key metrics: a 45% increase in engagement duration, 109% enhancement in interaction frequency, and 32-41% improvements in comprehension and narrative sequencing abilities compared to traditional storytelling methods. The system achieved 92% task completion success with 3.2% error rates, demonstrating both technical robustness and educational efficacy. Our findings contribute novel insights into the design of assistive technologies, demonstrating that AI-driven interactive storytelling can effectively address accessibility barriers while promoting cognitive development in children with disabilities. This research establishes evidence- based design principles and provides a validated framework for developing inclusive educational technologies.

Spoken Question Answering (SQA) extends machine reading comprehension to spoken content and requires models to handle both automatic speech recognition (ASR) errors and downstream language understanding. Although large-scale SQA benchmarks exist for high-resource languages, Vietnamese remains underexplored due to the lack of standardized datasets. This paper introduces ViSQA, the first benchmark for Vietnamese Spoken Question Answering. ViSQA extends the UIT-ViQuAD corpus using a reproducible text-to-speech and ASR pipeline, resulting in over 13,000 question-answer pairs aligned with spoken inputs. The dataset includes clean and noise-degraded audio variants to enable systematic evaluation under varying transcription quality. Experiments with five transformer-based models show that ASR errors substantially degrade performance (e.g., ViT5 EM: 62.04% [Formula: see text] 36.30%), while training on spoken transcriptions improves robustness (ViT5 EM: 36.30% [Formula: see text] 50.70%). ViSQA provides a rigorous benchmark for evaluating Vietnamese SQA systems and enables systematic analysis of the impact of ASR errors on downstream reasoning.

Voice phishing (vishing) has emerged as a major vector for social engineering, exploiting the emotional and persuasive aspects of speech to deceive victims. Existing public datasets are often monolingual and lack structured annotations of persuasive intent and interactional structure in telephony scenarios. VISHGUARD is a multilingual synthetic audio corpus designed to advance research on persuasion-sensitive vishing detection. It contains 3,000 simulated phone calls in English, French, and Modern Standard Arabic evenly distributed between fraudulent and legitimate categories. Each sample is annotated across several dimensions, including persuasion strategy, interactional markers, and emotional tone. The audio recordings were synthetically generated using text-to-speech synthesis and controlled noise mixing, with ambient sounds used to approximate office, street, and home-like acoustic variability rather than full telecommunication-channel effects. The dataset covers durations from 10 to 90&#x2009;seconds and maintains balanced linguistic and class distributions. VISHGUARD provides a reproducible, script-conditioned synthetic telephony corpus with multidimensional annotations to support controlled research on persuasion-aware vishing detection in multilingual settings. All data files are publicly available to ensure transparency and reuse.

Voice-based authentication is widely used today but remains vulnerable to spoofing attacks like text-to-speech and voice conversion, threatening privacy and its authenticity. To mitigate such threats, various detection techniques ranging from conventional machine learning models to contemporary deep learning frameworks have been used in the past. Yet, it is difficult to choose an ideal solution, as model performance typically varies with dataset attributes, computational power, and deployment environments.This paper is a comparative experimental analysis of four spoofing detection models as controllers: Support Vector Machine (SVM), Gaussian Mixture Model (GMM), ResNet, and Wav2Vec 2.0 with any transformer. The models are tested with one experiment through the usage of the automatic speaker verification spoof 2019 Logical Access dataset. Equal error rate&#xa0;and tandem detection cost function measures the effectiveness of the detection, whereas training and inference time measure the effectiveness of the computation. Results, under the standardized set up, are that ResNet is typified with a good detection, at a comparatively low computational cost, and Wav2Vec 2.0 gives good error rates, at a significantly larger cost of fine-tuning. The GMM and classical SVM models are lower in their cost of computation and greater in error rates, respectively.Overall, the results reveal that more complex models do not always improve, and&#xa0;using the detection performance and runtime analysis, the study gives appropriate guidance on how to choose the spoof detection models based on the deployment needs, not only the accuracy.

This study aims to extend the counterintuitive observation that Automatic Speech Recognition (ASR) errors can be beneficial for Alzheimer's Disease (AD) detection. Our objective is to conduct a large-scale investigation to validate this phenomenon and, more importantly, to elucidate the specific mechanisms by which ASR errors can serve as valuable diagnostic clues for distinguishing individuals with AD from Healthy Controls (HC). We employed 18 ASR models, in both their original and fine-tuned versions, to generate 36 sets of transcripts from the ADReSS dataset. We also synthesized speech from both manual and ASR transcripts using a text-to-speech (TTS) model. Knowledge-based features and pre-trained embeddings were extracted and fed into two proposed AD detection models : a self-attention model and a cross-attention-based interpretability model. To uncover the underlying mechanisms, we conducted a multi-faceted set of analyses, including examinations of ASR error types, words affected by ASR errors, linguistic comparisons, attention weight analysis, and case studies. We demonstrate that transcripts generated by certain ASR models achieve higher AD detection accuracy than gold-standard manual transcripts. This performance gain stems not from errors in general or a high Word Error Rate (WER), but from specific and asymmetric error patterns. Our analyses reveal that these patterns amplify some pre-existing linguistic deficits in AD speech (e.g., disfluencies), thereby increasing the feature-level divergence between the AD and HC groups. Furthermore, we show that these diagnostic clues are effectively preserved when speech is synthesized from ASR transcripts, holding significant implications for data augmentation strategies in AD research. The specific, asymmetric error patterns introduced by certain ASR models enhance the distinction between AD and HC groups by amplifying pathological linguistic deficits associated with AD. This work suggests a paradigm shift for clinical ASR development: optimizing models not merely for transcription accuracy, but for their downstream diagnostic utility.

Silent speech interfaces (SSIs) offer a viable alternative to traditional microphones in capturing clear audio in noisy environments. We propose a reconceptualized SSI that reproduces voice by monitoring continuous multiaxial strain maps induced by throat muscle movements. The system integrates a computer vision-based optical strain (CVOS) sensor with deep learning-based voice reconstruction, enabling clear alphabetic communication under extreme noise conditions. The CVOS sensor-comprising a soft silicone substrate with micromarkers and a tiny camera-achieves high-sensitivity marker detection and captures complex strain patterns with higher scalability and reliability compared to conventional wearable sensors. The inference pipeline of the CVOS-based SSI incorporates physics-based automated baseline calibration and content-adaptive temporal attention, enabling robust analysis of the captured strain patterns. Based on the inference results, a personalized text-to-speech model subsequently reconstructs the speaker's voice. These algorithmic features ensure robustness under dynamic conditions by employing real-time adaptive signal processing that compensates for inter- and intrasubject anatomical variability. Alphabet-based communication is achieved through the synergy between optimized algorithms and interface design. The performance of the CVOS-based SSI was validated in real-world noisy scenarios, confirming its practical applicability.

Prerecorded courses are increasingly used in medical education, and audio quality is known to influence learners' comprehension and engagement. Traditional audio recording, however, is time-consuming and may be uncomfortable for some educators. Advances in generative artificial intelligence (AI) now allow for realistic voice cloning, but its pedagogical value compared with conventional recording has not been assessed. This study aimed to evaluate the usefulness and perception of AI-based voice cloning for prerecorded courses in medical pedagogy compared with traditional audio recording. We conducted a randomized trial among fourth- and fifth-year medical students at a French university. Participants accessed four 10-minute prerecorded lectures on critical appraisal of medical research. The control group received lectures with audio recorded by the teacher, whereas the intervention group received audio generated from the teacher's cloned voice using a commercial AI text-to-speech system, with identical slides and scripts. The primary outcome was the total score on 2 online tests (17 multiple-choice questions on knowledge acquisition and 15 multiple-choice questions on knowledge application). Secondary outcomes included satisfaction ratings, course viewing metrics, and production time. A total of 88 students were randomized, and 64 (72.7%) watched at least 15 seconds of video and, thus, were included in the modified intention-to-treat population. Mean total test scores did not differ significantly between the AI voice cloning and audio recording groups (51.2, SD 4.9 vs 51.8, SD 6.9 out of 100; adjusted mean difference -0.9, 95% CI -2.7 to 4.4; P=.60). Satisfaction was high in both groups. Production time was shorter with AI (22.5, SD 6.45 vs 35, SD 7.07 minutes per video). We did not detect a significant difference in learning outcomes or satisfaction between AI voice cloning and conventional recording, whereas AI voice cloning reduced production time, making it a practical alternative for prerecorded medical courses. Nevertheless, some students may perceive synthetic voices as less authentic, representing a potential barrier to widespread adoption.