Transformer-based models have significantly advanced image captioning through self-attention mechanisms and parallel computation. However, existing methods typically adopt teacher-forcing strategies during training by conditioning the decoder exclusively on ground-truth tokens, whereas at inference, captions are generated autoregressively based solely on previously predicted tokens. Such discrepancy between training and inference conditions leads to a progressive accumulation of prediction errors, resulting in captions that deviate significantly from the visual content. While scheduled sampling strategies mitigate this issue, directly integrating them disrupts Transformer parallelism and overlooks visual saliency differences. To tackle these limitations, we present VMFormer, a multi-stage decoding Transformer featuring a Visual-Aware Scheduled Sampling (VASS) module that bridges this gap through two key innovations: 1) A two-stage decoding scheme where an initial self-study stage generates candidate tokens, followed by a hybrid stage dynamically blending ground-truth references with predictions via a visual clues controlled gate. 2) A cognitive-inspired prioritization mechanism that retains visual keywords (nouns/verbs/attributes) in early training phases before transitioning to linguistic refinements, mirroring human captioning patterns. Crucially, the VASS module preserves the parallel computational strengths of Transformer architectures and is designed as a plug-and-play component, readily adaptable to Transformer-based captioning models. Experiments on challenging MSCOCO achieve 142.2 CIDEr. We extend VMFormer to video captioning and demonstrate consistent improvements on MSRVTT and MSVD datasets.
This study combines modern source-separation techniques with interpretable machine learning methods to investigate how binary perceptual annotations of diphthongal and monophthongal /aɪ/ relate to measurable acoustic variation in sung vocal performance. Using a corpus of studio-recorded music processed with vocal-instrumental source separation, we extract F1 and F2 trajectories for 1004 /aɪ/ tokens, each perceptually categorized as either monophthongal or diphthongal. These trajectories were modeled using two approaches: (i) a gradient boosted decision tree trained on engineered acoustic features and (ii) a multilayer perceptron (MLP) trained directly on raw formant trajectories. Both models achieved high accuracy and Area Under the Receiver Operating Characteristic Curve, indicating that perceptual labels can reliably be predicted from both raw and engineered acoustic input. Moreover, a Shapley Additive Explanations-based feature importance analysis showed that features such as ΔF1/ΔF2, cubic spline coefficients, and trajectory derivatives captured systematic differences between perceived monophthongal and diphthongal tokens, highlighting the value of dynamic representations over static distance-based measures. The results indicate that perceptual annotations of diphthongal and monophthongal /aɪ/ correspond with quantifiable acoustic information and demonstrate how explainable machine learning can help map gradient vowel dynamics onto binary perceptual categories. The study further emphasizes how recent advances in source separation make sung performance a valuable new domain for phonetic research.
Millimeter-wave (mmWave) radar provides a privacy-preserving and illumination-robust sensing modality for contactless gesture recognition. However, sparse radar point clouds degrade substantially as sensing distance increases: the number of valid detections decreases, echo intensity attenuates, and Doppler-related motion cues become less reliable. Such range-induced degradation leads to a distribution shift between near-range training samples and far-field test samples, making it difficult for models trained at short distances to generalize to unseen longer distances. Existing point-cloud gesture recognition methods usually treat radar detections as generic sparse point sequences and rarely model distance-related point loss, echo attenuation, and physical-attribute unreliability explicitly. This work introduces RPT-Mamba, a range-aware physical token Mamba network for sparse mmWave radar point cloud sequences. RPT-Mamba constructs physical point tokens from spatial coordinates, Doppler velocity, echo intensity, point-level range, and sample-level range information. During training, a range-aware stochastic degradation strategy adaptively removes points and masks dynamic attributes according to the estimated sensing distance, while a context-guided attribute reconstruction objective recovers masked Doppler and intensity attributes from spatial and frame-level context. A bidirectional Mamba temporal encoder then models long-range gesture dynamics over frame tokens. On the public mTransSee dataset, RPT-Mamba achieves 92.09% accuracy and 92.04% Macro-F1 under the random split protocol, and 85.34% accuracy and 84.77% Macro-F1 under a challenging near-to-far protocol, exceeding point-cloud, radar-gesture, Transformer, and Mamba baselines.
Accurate and efficient multi-class classification for brain tumors from MRI images is still a significant problem, requiring not only local feature learning but also global context comprehension. We present a novel Query-Guided Cross-Attention Convolutional Neural Network (QGC-CNN), which incorporates convolutional learning into a transformer-style network with learnable query tokens. The interaction between query tokens and feature maps helps discover long-range relationships that are difficult for standard CNNs to capture. To improve reliability and robustness, an ensemble method combining Xception, EfficientNetB0, ResNet50, and QGC-CNN was constructed, where the predicted values were combined using a LightGBM meta-learner. The experiments show that the proposed framework achieved a test accuracy of 95.50% and an AUC of 0.99 for the "No Tumor" class. Importantly, the QGC-CNN model shows great efficiency, having just 0.83 GFLOPs, which is 10.9 times lower than that of Xception. The proposed framework demonstrates that combining query-guided cross-attention with ensemble learning can provide accurate and computationally efficient brain tumor classification from MRI images.
Despite the impressive generative capabilities of text-to-image (T2I) diffusion models, they remain vulnerable to implicit sexual prompts, where subtle cues disguised as benign terms or adversarial tokens unexpectedly generate the inappropriate content due to model biases or latent correlations in training data. Existing safety mechanisms face fundamental limitations: detection methods primarily identify explicit content and fail to capture implicit malicious intent, while mitigation approaches rely on static negative prompts inadequate for diverse implicit scenarios. To address these challenges, we propose UniNDM, a unified noise-driven framework that rethinks safety mechanisms through the lens of noise dynamics in diffusion processes. Our key insight is that early-stage predicted noise exhibits inherent separability between normal and sexually explicit content, which we theoretically demonstrates quadratically increasing semantic concentration with timestep. Leveraging this property, we develop a lightweight noise-based detector achieving superior accuracy with virtually no computational overhead. For mitigation, we introduce noise-enhanced adaptive negative guidance: dynamically generating context-specific negative prompts via large language models to handle diverse implicit content, while optimizing initial noise by suppressing attention concentration on explicit tokens to provide comprehensive protection. Besides the U-Net-based diffusion models, we further extend our framework to emerging Diffusion Transformer architectures through region-constrained semantic guidance tailored for their unified multimodal attention. Comprehensive experiments across U-Net models (SDv1.4, v1.5, v2.1, XL) and DiT models (SDv3) on both natural and adversarial datasets demonstrate substantial improvements over state-of-the-art methods, including SLD, UCE, Safree, etc. Our code is publicly available at https://github.com/Aries-iai/UniNDM.
The clinical utility of monitoring longitudinal changes in musculoskeletal trajectories, including bone mineral density (BMD), muscle strength, height, and weight for fracture prediction, remains underutilized, as current gold-standard tools such as the Fracture Risk Assessment Tool (FRAX) rely solely on cross-sectional baseline data. This study aimed to determine whether a deep learning model integrating individualized musculoskeletal trajectories improves fracture prediction accuracy compared to established static benchmarks. We developed the Hybrid Trajectory-Based model (HyTrax), a Transformer-based deep learning model that encodes sequential measurements of hip and spine BMD, grip strength, height, and weight as temporal tokens, incorporating subject-specific slopes derived from linear mixed-effects models. The model was trained and internally validated in 27,512 postmenopausal women from the Women's Health Initiative (WHI) and externally evaluated in 1,193 participants from the Framingham Heart Study (FHS). In the WHI validation set, the HyTrax + FRAX (BMD) ensemble model achieved a time-dependent Area Under the Curve (AUC) of 0.85 for Major Osteoporotic Fracture, outperforming both the longitudinal Transformer alone (AUC = 0.80) and the standard FRAX-BMD model (AUC = 0.82). The HyTrax + FRAX (BMD) ensemble model demonstrated favorable discrimination and improved risk stratification (Net Reclassification Improvement +26.5%) in WHI. Evaluation in the FHS cohort demonstrated the transportability of the longitudinal embeddings, with the HyTrax + Baseline 2 ensemble model (integrating longitudinal embeddings with clinical risk factors, BMD, and grip strength) achieving an AUC of 0.74. Explainability analyses identified early longitudinal weight fluctuations and overall height loss trajectories as important predictors of future fracture risk, alongside static factors such as age and genetic predisposition. By leveraging individualized trajectories through deep sequential modeling with baseline FRAX probability, the HyTrax + FRAX (BMD) ensemble model improved fracture discrimination over static assessments, offering a framework for incorporating repeated clinical measures into fracture prediction.
In Mandarin, the head noun in [modifier clause-de-N] encodes not only the arguments or adjuncts of the clausal verbs (e.g. "John/biscuits/plate" in the event of "John eating biscuits") but also non-argument, non-adjunct participants extended from the clausal event (e.g. "crumbs" resulting from the event of "John eating biscuits"). This article adopts a typological, multi-factorial analytic framework of noun-modifying clause constructions (NMCCs) and proposes a developmental hierarchy of three types of NMCCs: argument > adjunct > extended. We tested the hierarchy in two studies: an analysis of 728 tokens of NMCC produced by 369 children (1;2-6;9) in naturalistic corpora and a contextualised sentence repetition task conducted with 121 children (3;0-4;11) in China. Our findings show that NMCC emerges surprisingly early around 2;0 in Mandarin children. Argument NMCCs show developmental primacy in frequency and accuracy. Adjunct and extended NMCCs are acquired at a later stage, consistent with the proposed hierarchy.
Most hallucination mitigation for large language models (LLMs) operates post-hoc, leaving safety-critical clinical deployment without real-time warning capability. We present a calibrated hidden-state probing pipeline that enables token-time hallucination detection under explicit false-positive-rate (FPR) constraints, making it suitable for streaming clinical deployment. We formulate token-time hallucination detection as a constrained sequential decision problem separating non-circular supervision construction (using ROUGE-L, exact-choice, and exploratory NLI verifiers), FPR-constrained operating-point selection, and downstream intervention evaluation into a modular, reusable protocol. A lightweight two-layer hidden-state probe classifies each generated token into one of three risk states (Safe, AtRisk, or Hallucinating), where the intermediate AtRisk state captures pre-error instability. We validate across four medical QA benchmarks (Endoscopy, PubMedQA, MedHallu, MedQA-USMLE) spanning approximately 44,500 generation trajectories and three backbone LLMs (Qwen3-8B, Llama-3.1-8B-Instruct, BioMistral-7B). On the public biomedical QA benchmarks the probe yields viable detection across all backbone-dataset combinations: EDR@5 up to 0.390 (PubMedQA) and 0.352 (MedHallu) at the strict cap γ≤0.10, rising to 0.587 with 29%-59% hallucination reduction at the monitoring cap γ≤0.20. The trigger fires roughly 10-35 tokens before error onset, whereas a post-hoc check has zero lead time by construction. A single-institution Endoscopy corpus serves as a case study (EDR@5 0.613, Llama-3.1), treated as illustrative because its small confirmed-correct denominator (26-52 per backbone) yields wide confidence intervals. Probe latency is ∼10ms per answer versus ∼10s for multi-sample baselines. Hidden-state probing with FPR-constrained calibration provides a practical, low-latency solution for real-time hallucination monitoring in clinical LLM deployments. The modular pipeline separating supervision construction, probe training, and trigger calibration is directly reusable with alternative detectors or verifiers.
Fine-Grained Visual Classification (FGVC) remains a challenging task due to subtle inter-class differences and large intra-class variations. Existing approaches typically rely on feature-selection mechanisms or region-proposal strategies to localize discriminative regions for semantic analysis. However, these methods often fail to capture discriminative cues comprehensively while introducing substantial category-agnostic redundancy. To address these limitations, we propose $\text {H}^{3}$ Former, a novel token-to-region framework that leverages high-order semantic relations to aggregate local fine-grained representations with structured region-level modeling. Specifically, we propose the Semantic-Aware Aggregation Module (SAAM), which exploits multi-scale contextual cues to dynamically construct a weighted hypergraph among tokens. By applying hypergraph convolution, SAAM captures high-order semantic dependencies and progressively aggregates token features into compact region-level representations. Furthermore, we introduce the Hyperbolic Hierarchical Contrastive Loss (HHCL), which enforces hierarchical semantic constraints in a non-Euclidean embedding space. The HHCL enhances inter-class separability and intra-class consistency while preserving the intrinsic hierarchical relationships among fine-grained categories. Comprehensive experiments conducted on four standard FGVC benchmarks validate the superiority of our $\text {H}^{3}$ Former framework. Code is available at https://github.com/xiaozhangfangyang/H3Former.
Gamified digital training may support long-term engagement in aging populations, yet randomized evidence linking behavioral effects to convergent neuroimaging outcomes remains limited. This study tested whether a home-based digital mahjong-based cognitive engagement intervention enhanced cognition in community-dwelling older adults and explored brain changes using multimodal magnetic resonance imaging (MRI). We conducted a single-blind, parallel-group randomized controlled trial in Taipei, Taiwan (July 2020-November 2021). Community-dwelling adults aged 55 years and older were randomized (1:1) to a 6-month digital mahjong-based intervention (two 30-minute sessions per week via an iPad app) with an incentivized walking component (Mi Band step tracking with in-game tokens) or no intervention. Primary MRI outcomes were gray matter volume, regional homogeneity (ReHo), fractional amplitude of low-frequency fluctuations (fALFF), and global brain connectivity. Secondary outcomes included cognition (Montreal Cognitive Assessment), physical activity (International Physical Activity Questionnaire), mental health (Brief Resilience Scale and Mandarin version of the Demoralization Scale), quality of life (EQ-5D with a visual analog scale), physical fitness and body composition, and blood biomarkers. Per-protocol analyses were performed. Voxel-wise imaging analyses used analysis of covariance in SPM12 with family-wise error (FWE) correction at the cluster level (P<.05). Sensitivity analyses controlling for MRI delay duration were conducted to assess robustness of findings. In total, 60 participants were enrolled (n=30, 50% per group); 96.7% (29/30) in the intervention group and 90% (27/30) in the control group completed primary assessments. Compared with controls, the intervention group showed a greater improvement in Montreal Cognitive Assessment delayed recall (between-group difference=-0.79 points, 95% CI -1.538 to -0.042; P=.04). Neuroimaging revealed increased ReHo in the right insula (FWE-corrected P<.05) and decreased gray matter volume in the left frontal pole and orbitofrontal cortex, with reduced fALFF in the left frontal pole. Changes in delayed recall were positively associated with right insular ReHo and also showed significant associations with the right anterior cingulate gyrus (ReHo), left inferior frontal gyrus, and right paracingulate gyrus (fALFF; FWE-corrected P<.05 in all cases). In exploratory dose analyses, right insular ReHo correlated with total practice sessions (r=0.48; P=.009). Among physiological outcomes, the intervention group had a larger reduction in diastolic blood pressure (-3.50 mm Hg, 95% CI -6.911 to -0.091; P=.04), whereas the thyroid-stimulating hormone decreased but remained within the normal range. Sensitivity analyses confirmed that primary findings remained stable after controlling for MRI delay duration. This 6-month home-based digital mahjong intervention improved delayed recall and increased right insular ReHo. Integrating culturally familiar cognitive challenges with lifestyle engagement may be a feasible strategy associated with salience network-related functional changes and memory performance. Future large-scale trials should use active control conditions and factorial designs to rigorously isolate component-specific mechanisms. ClinicalTrials.gov NCT05808426; https://clinicaltrials.gov/study/NCT05808426.
Standard k-mer methods treat amino acids as categorical tokens without directly encoding physicochemical properties. Although physicochemical properties have been incorporated into various bioinformatics tasks, their potential as a direct, systematic alternative for the k-mer counting paradigm has not been fully evaluated. We present PhysioChem-K-mer, a framework that transforms protein sequences into physicochemical property-based feature spaces, serving as an alternative to conventional amino-acid-identity k-mer representations. Our main hypothesis is that property-based representations capture functional constraints more effectively than traditional amino acid-based methods. To test this hypothesis, we created a controlled benchmark comprising 1500 synthetic sequences spanning 10 diverse protein families. The dataset retained core functional motifs while deliberately excluding evolutionary patterns typically found in natural biological sequences. Notably, our hydropathy-based PhysioChem-K-mer achieved a classification accuracy of 81.33% on a controlled synthetic benchmark, representing an absolute gain of 44.33% points over standard 3-mer methods (37.00%). The framework was further evaluated using real UniProt/Swiss-Prot data, comprising 11,620 sequences across 10 families, to ensure practical generalizability. Based on real data, PhysioChem-Hydropathy achieved 64.63%, an absolute gain of 47.68% points over the standard 3-mer baseline (16.95%), while reducing features by 73.9% and training time by 81.6%. By directly integrating biochemical knowledge into feature representations as a primary design principle, PhysioChem-K-mer combines interpretability with computational efficiency. These results suggest that physicochemical properties offer a vital source of information for protein classification, validated here on both synthetic and real-world data.
Thyroid ultrasound diagnosis in clinical practice typically relies on both transverse and longitudinal views of the same lesion. However, most existing deep learning methods process these views independently or only perform simple feature fusion, which limits their ability to model cross-view semantic consistency and reduces diagnostic robustness. We propose an uncertainty-weighted mixture-of-experts (UMoE) framework with dual knowledge distillation for dual-view thyroid ultrasound classification. The proposed model is built on a shared Vision Transformer backbone that encodes both dual-view and single-view branches. Cross-view knowledge distillation is performed on uncertainty-refined patch tokens to align lesion-related representations across views, while cross-level knowledge distillation transfers holistic semantic knowledge from the dual-view branch to the single-view branches through the CLS token. Experiments on the in-house DTN5K dataset and an external public thyroid ultrasound dataset show that the proposed framework consistently outperforms recent single-view and dual-view baselines in accuracy, F1-score, and area under the receiver operating characteristic curve. Ablation studies further verify the effectiveness of the proposed dual-view training strategy and each major component. The proposed UMoE framework improves cross-view consistency during training and enhances the predictive ability of each single-view branch, leading to more accurate and robust thyroid ultrasound classification. This study provides a clinically relevant dual-view learning framework for thyroid ultrasound analysis and offers a practical strategy for improving the reliability of computer-aided thyroid nodule diagnosis.
Synthetic aperture radar (SAR) ship detection faces three persistent challenges: coherent speckle noise that obscures target boundaries, heterogeneous background clutter in coastal and harbor scenes, and ship targets whose spatial extent varies by more than an order of magnitude within the same image. To address these issues jointly, this paper proposes MSA-DET, an improved SAR ship detection network built upon YOLOv11. In the backbone, a Multi-Scale Cross-axis Attention module (MSCAttention) runs horizontal and vertical axial attention branches in parallel across multiple receptive-field scales, sharpening feature representations for ship targets that vary widely in size and orientation. In the neck, the standard C3k2 block is redesigned as C3k2_SSA by embedding sparse self-attention, which selectively focuses on the most discriminative spatial tokens while suppressing speckle interference and reducing computational overhead. An Adaptive Spatial Feature Fusion detection head (ASFF) replaces fixed pyramid-level aggregation with learned per-pixel blending weights, resolving gradient conflicts across scales and improving localization consistency for both small and large ships. On the HRSID dataset, MSA-DET achieves an mAP@0.5:0.95 of 63.6% and mAP@0.5 of 88.1%, representing gains of 4.0% and 1.6% over the YOLOv11n baseline; on SSDD, it reaches 69.6% and 97.7%, surpassing the baseline by 7.2% and 2.1%, respectively. These results demonstrate that coordinated multi-stage redesign-rather than isolated module substitution-is an effective strategy for SAR-oriented ship detection. The accuracy gains are accompanied by a moderate increase in model size (8.9 M parameters versus 2.6 M for YOLOv11n) and computational cost (9.6 G FLOPs versus 6.3 G), a trade-off that is justified by the substantial improvement in detection quality.
Infants acquire language with generalization from minimal experience, whereas large language models require billions of training tokens. What underlies efficient development in humans? We investigated this problem through experiments wherein robotic agents learn to perform actions associated with imperative sentences (e.g., push red cube) via curiosity-driven self-exploration. Our approach amortizes active inference using Q-learning, enabling intrinsically motivated developmental learning. The simulations reveal key findings corresponding to observations in developmental psychology. (i) Generalization improves markedly as the scale of compositional elements increases. (ii) Curiosity-driven exploration enables faster learning. (iii) Rote pairing of sentences and actions precedes compositional generalization. (iv) Exception handling induces U-shaped developmental performance, a pattern like representational redescription in child language learning. These results suggest that curiosity-driven active inference accounts for how intrinsically motivated sensorimotor-linguistic learning supports scalable compositional generalization and exception handling in humans and artificial agents.
Ambient AI scribes generate transcripts at scale, but routine quality assurance is constrained by the absence of human-verified reference transcripts in most deployment settings. We evaluated whether disagreement among heterogeneous automatic speech recognition (ASR) systems can serve as an informative signal for localizing transcription uncertainty, using a public English-language medical-speech corpus rather than clinical encounter recordings. Eight commercial and open-source ASR systems were applied to 50 medical-education audio clips (8 h 14 min). Multi-model outputs were aligned, and a leave-one-out consensus procedure was used to score per-model agreement while reducing circularity. Disagreement across models was sparse and localized: 72.1% of positions showed strong agreement (7-8 systems concordant), whereas only 2.5% were high-risk positions with minimal agreement (0-3 systems). Low-agreement regions were systematically enriched for meaning-bearing lexical differences, defined as lexical mismatches after excluding punctuation, contraction, numeric, and filler variation. A single-annotator human-corrected (HC) validation layer showed that transcription errors increased monotonically with decreasing agreement. At an illustrative post hoc threshold, flagging positions where six or fewer systems agreed selected 28.6% of tokens while recovering 93.7% of single-annotator HC-verified errors on this proxy corpus. These findings suggest that cross-model disagreement may help focus human review on a small number of likely error-prone transcript regions. However, agreement among all systems does not guarantee correctness, because shared errors may remain undetected by this approach. Validation on real clinical encounter data is required before operational deployment.
Synthesizing human motion from textual descriptions is essential for immersive digital applications, yet existing methods face a persistent trade-off between semantic fidelity and physical realism. Large language model (LLM)-based approaches can interpret diverse open-vocabulary instructions and compose high-level action plans, but they often generate motions that violate physical constraints. Physics-aware models improve realism through simulation or control, but they struggle with semantic complexity, fine-grained instructions, and novel concepts. To address this gap, we propose In-Context Model Predictive Generation (ICMPG), a framework that integrates language-model planning with inference-time physical feedback. ICMPG reformulates motion synthesis as a Model Predictive Control (MPC)-like process with two modules. The Context-Aware Motion Generation (CAMG) module uses an LLM as a planner to decompose textual commands and generate candidate motion sequences from motion tokens. The Model Predictive Generation (MPG) module evaluates these candidates through physical simulation and semantic alignment, estimates a composite reward, and selects the best sequence to guide subsequent generation steps. Unlike open-loop generation, this closed-loop refinement enables ICMPG to adapt motions to both the input semantics and the simulated physical environment without task-specific policy retraining. Extensive experiments across standard and zero-shot open-vocabulary settings show that ICMPG generalizes robustly to diverse commands and produces motions that are more physically plausible and semantically faithful than representative baselines on the evaluated benchmarks. The framework bridges semantic interpretation and physical simulation while remaining flexible enough to incorporate different LLM backbones, enabling more versatile and controllable text-driven motion synthesis.
Given a reference image, composed image retrieval aims to search for the target image from the image gallery with a modification sentence. The complicated semantic relationships within the composed query to comprehend the preserved visual context and modified semantics pose a challenge for the researchers. Most current approaches concentrate on the global or fragmental composition to combine the multi-modal query features and bridge the distance between the query and the target. However, global-level matching tends to ignore fine-grained components that distinguish the matched instance from hard negative instances. As for the fragment-level matching, the over-sensitivity to the local noise due to the lack of explicit local labels and excessive computational demands may have a detrimental impact on performance. In this paper, we propose Semantic Composition via Optimal Transport (SCOT) for composed image retrieval to address the aforementioned issues. To optimize the computational resources, SCOT first introduces Hybrid-modal Affinity Summarization (HAS) to compact salient relationships in the affinity matrix from the reference image patches and modification tokens. Furthermore, to mitigate the over-confidence in local noise, SCOT adaptively assigns optimal transport weights on the salient aligned fragments to Retain Prototype from Alignments (RAS) and guides discriminant semantics learning to Update Semantics from Enhanced Embeddings (USEE), while preserving the characteristics from global structures. Specifically, RAS module investigates the preserved semantics from the reference image based on the affinity matrix, meanwhile USEE module distills significant modification semantics. The proposed SCOT has demonstrated advanced experimental performances on widely-used datasets FashionIQ, CIRR, and Fashion200K. Ablative studies manifest the effectiveness of optimal transport guidance on preserved visual prototypes and updated semantics.
Test-time scaling has emerged as a promising method to enhance the reasoning capabilities of large language models (LLMs) and vision-language models (VLMs) during inference without additional training. While foundational studies established scaling paradigms in general domains, their applicability to the unique complexities of medical AI remains underexplored. This study aims to conduct a comprehensive investigation of test-time scaling in the medical domain. We evaluate the impact of scaling across different model sizes and task complexities. Furthermore, we seek to identify domain-specific bottlenecks and assess model robustness against user-driven perturbations, such as misleading clinical authority. This study evaluated a diverse set of general and medical-specific LLMs and VLMs. Experiments used five textual medical benchmarks comprising over 5500 questions and two multimodal benchmarks comprising 7000 samples. Performance was measured under three scaling conditions: increasing token budgets, iterative sequential scaling, and parallel scaling. Robustness was tested by embedding misleading hints with varying tones and levels of simulated clinical expertise into prompts. For nonreasoning LLMs, accuracy saturated quickly, with token usage often remaining under 500 tokens regardless of budget increases. Reasoning models demonstrated significant performance gains on complex tasks as token budgets increased. Notably, we identified distinct domain-specific behaviors. First, current VLMs showed a structural bottleneck in integrating visual clues and experienced limited benefit from token expansion. Second, medically fine-tuned LLMs excelled in clinical question answering but exhibited degraded scaling efficiency on calculation tasks compared to general-domain models. This reflects a disparity between qualitative clinical alignment and procedural logic. Third, while optimal scaling improved robustness, models exhibited a cognitive vulnerability by readily abandoning correct reasoning when confronted with misleading expert physician hints. Regarding scaling strategies, parallel scaling outperformed sequential scaling on easier tasks. Conversely, extended sequential scaling or increased budgets proved essential for complex problem-solving. Test-time scaling rules from general domains do not perfectly translate to medical AI. Longer reasoning is not universally beneficial. Concise reasoning with parallel scaling is optimal for simpler tasks. An extended chain of thought via sequential scaling or increased budgets is required for complex problems. Furthermore, safe clinical deployment requires addressing fundamental vision-language alignment, balancing clinical and procedural reasoning, and mitigating vulnerabilities to perceived clinical authority.
Existing image-text retrieval methods often compute cross-modal similarity using global single-vector representations. Although efficient for coarse semantic alignment, such compressed representations are limited when textual queries involve fine-grained semantics, including objects, attributes, relations, and their compositional structures. This paper focuses on fine-grained text-to-image retrieval and proposes Q-GrAM, a retrieval-oriented adaptation of the BLIP-2 Q-Former. Instead of treating Q-Former queries as a homogeneous set, Q-GrAM partitions a fixed query budget into semantically differentiated groups. A text-guided router assigns token-level semantic demands to query groups, while query conditional initialization modulates each group according to group-level textual summaries. The resulting grouped visual query features are matched with text tokens through a group-aware late interaction scorer, and auxiliary routing balance and inter-group diversity regularization are introduced to stabilize semantic specialization. Experiments on MS-COCO 5K, Flickr30K, and Flickr30K-CFQ show that Q-GrAM achieves strong text-to-image retrieval performance against both global embedding baselines and representative fine-grained image-text matching methods, while maintaining competitive bidirectional retrieval performance. These results demonstrate the effectiveness of structured, text-conditioned Q-Former query specialization for fine-grained text-driven image search.
Purpose. Radiation therapy (RT) treatment planning requires iterative, multi-day optimization workflows in which subjective planning strategies produce inter-planner variability in plan quality. Existing computational approaches automate isolated aspects of this workflow, yet none orchestrates an end-to-end pipeline from physician directive to deliverable plan. We developed a compound artificial intelligence (AI) platform for fully autonomous RT treatment planning that combines multi-agent large language model (LLM) orchestration with directive-conditioned three-dimensional (3D) dose prediction, natively integrated with a commercial treatment planning system (TPS). Methods. Seven specialized agents navigated the multi-objective optimization landscape through structured clinical reasoning, iteratively analyzing dose-volume histogram (DVH) metrics and spatial dose patterns, formulating trade-off strategies, and executing validated modifications through the TPS across five fully autonomous iterations per case. A directive-conditioned 3D dose prediction model supplied patient-specific DVH values from which initial optimization objectives were autonomously derived, eliminating the need for curated templates or manual initialization. A retrieval-augmented generation (RAG) system encoded institutional knowledge into the planning workflow. We evaluated 60 retrospective cases across brain, lung, and prostate sites, with 10 intensity-modulated RT (IMRT) and 10 volumetric modulated arc therapy (VMAT) plans per site spanning 20.0-79.2 Gy in 3-44 fractions, scored by the proportion of dosimetric criteria satisfied. Results. Across all 60 cases, AI plans achieved 89.8 ± 9.4% of dosimetric criteria versus 85.2 ± 10.8% for clinical reference plans ( p < 0.001 ). IMRT plans improved in 25 of 30 cases with none worsened (94.1 ± 6.7% vs 84.3 ± 11.8%, p < 0.001 ); VMAT plans showed no significant difference (85.6 ± 9.9% vs 86.1 ± 9.7%, p = 0.770 ). Each plan iteration completed in 20.2 ± 12.7 min, of which agent reasoning consumed 5.2 ± 1.7 min ( 114,429 ± 11,798 tokens, 0.43 ± 0.04 ). Conclusions. These results established the feasibility of end-to-end, fully autonomous, universal RT treatment planning through compound AI. Integrating dose prediction as an agent-invoked tool for objective initialization resolved the dependency on curated templates and manual specification that constrained prior LLM-based planning systems.