Homography estimation is a fundamental problem in geometric computer vision, yet reliable estimation remains highly challenging under adverse imaging conditions. In low-light, haze, rain, underwater, dynamic, and cross-modality scenarios, degradation and appearance discrepancy severely weaken feature correlations, making both hand-crafted matching pipelines and purely data-driven regression models prone to unstable correspondences. To address this limitation, we propose an optimization-driven homography estimation framework that reformulates robust alignment as a progressive energy minimization process. The proposed framework consists of complementary components with explicit optimization roles. First, a Robust Feature Initialization (RFI) strategy provides a static-plane-aware warm start by suppressing unreliable dynamic or degradation-corrupted regions. Second, an Alignment Fidelity Module (AFM) unfolds fidelity minimization into learnable residual update steps, progressively refining the deformation field through feature-space alignment. Third, a Perception Regularization Module (PRM) introduces task-driven semantic constraints during training, encouraging the learned deformation to preserve both geometric consistency and perception-level reliability. In addition, we develop a Degradation Constraint Learning (DCL) strategy, where an auxiliary restoration decoder encourages the shared encoder to learn degradation-robust intermediate representations. To facilitate evaluation under challenging conditions, we construct a comprehensive harsh-environment homography benchmark covering synthetic and real-world scenes. Extensive experiments demonstrate that the proposed method achieves state-of-the-art alignment accuracy, robust generalization across diverse degradations, and consistent improvements on five downstream perception tasks, while maintaining practical inference efficiency with an adaptive stopping criterion.
The evolution of large models has witnessed the emergence of In-Context Learning (ICL) capabilities. In Natural Language Processing (NLP), numerous studies have demonstrated the effectiveness of ICL. Inspired by the success of Large Language Models (LLMs), researchers have developed Large Multimodal Models (LMMs) with ICL capabilities. However, explorations of demonstration configuration for multimodal ICL remain preliminary. Additionally, the controllability of In-Context Examples (ICEs) provides an efficient and cost-effective means to observe and analyze the inference characteristics of LMMs under varying inputs. This paper conducts a comprehensive external and internal investigation of multimodal in-context learning on the image captioning task. Externally, we explore demonstration configuration strategies through three dimensions: shot number, image retrieval, and caption assignment. We employ multiple metrics to systematically and thoroughly evaluate and summarize key findings. Internally, we analyze typical LMM attention characteristics and develop attention-based metrics to quantify model behaviors. We also conduct auxiliary experiments to explore the feasibility of attention-driven model acceleration and compression. We further compare performance variations between LMMs with identical model design and pretraining strategies and explain the differences from the angles of pre-training data features. Our study reveals both how ICEs configuration strategies impact model performance through external experiments and characteristic typical patterns through internal inspection, providing dual perspectives for understanding multimodal ICL in LMMs. Our method of combining external and internal analysis to investigate large models, along with our newly proposed metrics, can be applied to broader research areas.
Infrared and visible image fusion is pivotal for robust visual perception across all weather conditions and scenes. Although deep learning-based methods have made notable progress, most either assume pre-aligned inputs or rely on implicit feature-space alignment, which fails to fundamentally address the amplification of registration errors and the loss of semantic structure in the fused results. To this end, we propose a universal representation and end-to-end framework for jointly registering and fusing unaligned infrared-visible image pairs, dubbed URMIF. Each image is mapped into modality-invariant (homogeneous) and modality-specific (heterogeneous) features: the invariant "structural skeleton" encodes geometry and semantics to stabilize alignment, while the specific "texture carrier" preserves thermal saliency and visible details to enable complementary fusion. Therefore, we propose a bi-directionally coupled registration-fusion module. This module performs hierarchical deformation estimation from coarse to fine, effectively mitigating visual mismatches caused by complex parallax in real-world scenes. Within this framework, the fusion component acts as the "evaluator" of registration, providing feedback regularization to update the deformation and suppress error accumulation. Furthermore, we introduce a dominant-plane prior as a scene-level constraint, seeding stable global and patch-wise homographies and reconciling cross-modal detail conflicts, to reinforce geometric consistency and semantic reliability. We also release a large-scale dataset comprising 1,500+ unaligned infrared/visible pairs with registration ground truth, spanning diverse illumination conditions and fields of view. Based on this dataset and additional benchmarks, extensive experiments validate that our framework achieves robust alignment and high-quality fusion on misaligned inputs, markedly reducing artifacts and improving the performance of downstream tasks such as detection and segmentation. Code and benchmark are available at https://github.com/ZengxiZhang/URMIF.
Visual diffusion models have revolutionized the field of creative AI, producing high-quality and diverse content. However, they inevitably memorize training images or videos, subsequently replicating their concepts, content, or styles during inference. This phenomenon raises significant concerns about privacy, security, and copyright within generated outputs. In this survey, we provide the first comprehensive review of replication in visual diffusion models, marking a novel contribution to the field by systematically categorizing the existing studies into unveiling, understanding, and mitigating this phenomenon. Specifically, unveiling mainly refers to the methods used to detect replication instances. Understanding involves analyzing the underlying mechanisms and factors that contribute to this phenomenon. Mitigation focuses on developing strategies to reduce or eliminate replication. Beyond these aspects, we also review papers focusing on its real-world influence. For instance, in the context of healthcare, replication is critically worrying due to privacy concerns related to patient data. Finally, the paper concludes with a discussion of the ongoing challenges, such as the difficulty in detecting and benchmarking replication, and outlines future directions including the development of more robust mitigation techniques. By synthesizing insights from diverse studies, this paper aims to equip researchers and practitioners with a deeper understanding at the intersection between AI technology and social good. We release this project at https://github.com/WangWenhao0716/Awesome-Diffusion-Replication.
Remote sensing (RS) images from multiple modalities and platforms exhibit diverse details due to differences in sensor characteristics and imaging perspectives. Existing visionlanguage research in RS largely relies on relatively homogeneous data sources. Moreover, they still remain limited to conventional visual perception tasks such as classification or captioning. As a result, these methods fail to serve as a unified and standalone framework capable of effectively handling RS imagery from diverse sources in real-world applications. To address these issues, we propose RingMo-Agent, a model designed to handle multimodal and multi-platform data that performs perception and reasoning tasks based on user textual instructions. Compared with existing models, RingMo-Agent 1) is supported by a largescale vision-language dataset named RS-VL3M, comprising over 3 million image-text pairs, spanning optical, SAR, and infrared (IR) modalities collected from both satellite and UAV platforms, covering perception and challenging reasoning tasks; 2) learns modality adaptive representations by incorporating separated embedding layers to construct isolated features for heterogeneous modalities and reduce cross-modal interference; 3) functions as an agent-based framework equipped with external tool-use capabilities, employing a reinforcement learning-optimized trajectory decoding mechanism that integrates visual grounding tools for long-horizon embodied navigation tasks. Extensive experiments on various RS vision-language tasks demonstrate that RingMo- Agent not only proves effective in both visual understanding and sophisticated analytical tasks, but also exhibits strong generalizability across different platforms and sensing modalities.
Analyzing hand-object interaction in egocentric vision facilitates VR/AR applications and human-robot policy transfer. Existing research has mostly focused on modeling the behavior paradigm of interactive actions (i.e., "how to interact"). However, the more challenging and fine-grained problem of capturing the critical moments of contact and separation between the hand and the target object (i.e., "when to interact") is still underexplored, which is crucial for immersive interactive experiences in mixed reality and robotic motion planning. Therefore, we formulate this problem as temporal interaction localization (TIL). Some recent works extract semantic masks as TIL references, but suffer from inaccurate object grounding and cluttered scenarios. Although current temporal action localization (TAL) methods perform well in detecting verb-noun action segments, they rely on category annotations during training and exhibit limited precision in localizing hand-object contact/separation moments. To address these issues, we propose a novel zero-shot approach dubbed EgoLoc to localize hand-object contact and separation timestamps in egocentric videos. EgoLoc introduces hand-dynamics-guided sampling to generate high-quality visual prompts. It exploits the vision-language model to identify contact/separation attributes, localize specific timestamps, and provide closed-loop feedback for further refinement. EgoLoc eliminates the need for object masks and verb-noun taxonomies, leading to generalizable zero-shot implementation. Comprehensive experiments on the public dataset and our novel benchmarks demonstrate that EgoLoc achieves plausible TIL for egocentric videos. It is also validated to effectively facilitate multiple downstream applications in egocentric vision and robotic manipulation tasks. Code and relevant data are released at https://github.com/IRMVLab/EgoLoc.
In this paper, we focus on Single-Domain Generalized Object Detection (Single-DGOD), aiming to transfer a detector trained on one source domain to multiple unknown domains. Existing methods typically rely on simulation-driven paradigms, such as discrete augmentation or static textual prompts, to expand the boundaries of the training distribution. However, finite simulations often fail to capture the infinite dynamic variations of real-world scenarios, which can lead to overfitting on synthetic styles and limit the model's ability to handle complex structural degradations. Inspired by the manifold hypothesis, we argue that despite diverse visual variations, semantic features inherently reside on a compact, low-dimensional manifold. Thus, the key to generalization lies in learning to rectify deviant samples back onto this stable manifold, rather than merely exhausting external perturbations. To this end, we propose a new framework, i.e., Manifold Regression with Visual-Text Dual Chain-of-Thought (MR-DCoT), which reformulates robust generalization as a manifold regression problem. Specifically, we design a Visual-Text Dual Chain-of-Thought module that couples VLM-guided global semantic evolution with diffusion-based local structural perturbations to generate structured off-manifold hard examples. Subsequently, a Class-Specific Prototype Anchoring mechanism is introduced to learn a robust rectification operator that guides deviant features back toward the source semantic manifold. By establishing a closed loop of simulation for outlier generation and regression for semantic correction, our method effectively bridges the distribution gap, significantly boosting generalization and robustness to unseen shifts. Extensive evaluations on three complementary benchmarks, covering diverse driving weather conditions, real-to-art generalization, and zero-shot semantic segmentation, demonstrate the superiority and versatility of our method in handling complex domain shifts.
In vision-language models (VLMs), visual tokens usually bear a significant amount of computational overhead despite sparsity of information in them when compared to text tokens. To address this, most existing methods learn a network to prune redundant visual tokens using certain training data. Differently, we propose a text-guided training-free token optimization mechanism dubbed SparseVLM without fine-tuning costs. Given that visual tokens complement text tokens in VLM's linguistic reasoning, we select relevant text tokens to rate the significance of visual tokens using self-attention matrices and, then, prune visual tokens using the proposed strategy to maximize sparsity while retaining information. In particular, we introduce a rank-based strategy to adaptively determine the sparsification ratio for each layer, alongside a token recycling method that compresses pruned tokens into more compact representations. However, the Attention Gravity and Attention Sink phenomena undermine the sparsification reference of SparseVLM: text-visual alignment. To address this, we introduce improved text-visual attention patterns to enhance the fidelity of query-aware vision token selection. First, we correct the Attention Gravity effect, where positional encodings induce disproportionately high attention to vision tokens positioned near textual tokens. We further mitigate the Attention Sink by selecting priority attention heads that suppress the excessive attention absorbed by semantically irrelevant vision tokens. The two parts refine the text-visual alignment from both the token and head perspectives. Finally, for video understanding tasks, we propose Temporal-Balanced Sparsification, complementing spatial relevance with temporal diversity to reduce cross-frame redundancy. Extensive experiments on 23 widely used image and video understanding tasks across 7 vision-language models demonstrate the effectiveness of our SparseVLM+ framework. For example, equipping LLaVA with SparseVLM+ yields a 57.1% reduction in FLOPs and a 38.4% decrease in CUDA latency, while preserving 99.6% of its original accuracy on average. Our code is available at https://github.com/Gumpest/SparseVLMs.
Parameter-Efficient Fine-Tuning (PEFT) methods enable adapting large pre-trained models to downstream tasks with minimal overhead. Current approaches predominantly rely on low-rank decomposition to reparameterize weight increment matrices, assuming that model updates follow low-rank patterns. However, weight updates during fine-tuning may exhibit more complex statistical properties. An alternative paradigm decomposes weight increment matrices into frequency-domain components, offering potentially superior expressivity through flexible frequency component combinations. In this work, we unify these approaches under a Triplet Matrix Decomposition framework and rigorously compare their expressivities. Our analysis reveals that frequency-domain methods can surpass low-rank approaches when optimal frequency components are selected, and this advantage stems from orthogonal transformation matrices and flexible basis vector combinations. Building on these insights, we propose Learnable Orthogonal Adaptation (LoTA), a novel PEFT method that learns task-specific transformations and adaptive basis combinations. LoTA employs cascaded Householder transformations to construct orthogonal matrices with minimal parameters while ensuring exploration of the complete orthogonal space. We address the discrete optimization challenge of basis selection through finite-difference gradient approximation, enabling end-to-end backpropagation. Extensive experiments including natural language understanding, mathematical reasoning, commonsense reasoning, computer vision and visual instruction tuning tasks demonstrate that LoTA achieves superior parameter efficiency and performance compared to existing PEFT methods.
Handwritten text recognition (HTR) in examination scenarios has gained increasing attention for its role in intelligent grading systems. However, existing studies have not systematically modeled the complex handwriting phenomena inherent in exam settings, hindering a comprehensive understanding of the recognition challenges and limitations of current methods. Specifically, handwriting artifacts pose significant challenges to recognition models in two complementary aspects: sequentially, they disrupt the reading order and lead to non-monotonic sequences, while visually, they distort character structures and induce attention drift. To enable systematic benchmarking of exam handwriting, we first construct BNU-Exam-HTR, a large-scale dataset of handwritten exam text, and establish BNU-Exam-Benchmark, a fine-grained evaluation framework defining 12 representative challenges observed in real exam handwriting. To overcome these challenges, we further propose EduOCR, a recognition model with a collaborative dual-branch decoder. The Sequential Symbol Module (SSM) uses autoregressive decoding to handle non-monotonic sequences, while the Permutation-Aware Prediction Head (PPH) simulates artifact perturbations to guide the shared encoder in distinguishing characters from noise, thus stabilizing attention and mitigating alignment errors. Extensive experiments show that EduOCR consistently outperforms state-of-the-art HTR models, OCR tools, and multimodal large language models across all 12 challenges, demonstrating superior robustness and adaptability.
Although Multi-Agent Reinforcement Learning (MARL) has achieved notable success in cooperative tasks, many existing methods struggle to generalize to heterogeneous scenarios, where diversified specialization is required. In such tasks, a common way is to disable parameter sharing among agents, but this alone fails to induce sufficient behavioral specialization. To address this limitation, sequential updating schemes have recently been proposed, where agents are updated one by one based on the behaviors of their predecessors. However, exploration strategies tailored to the sequential setting remain underexplored, leaving untapped potential in utilizing the information flow between agents. In this work, we propose to leverage the sequential access to predecessor information to explicitly enhance both exploration efficiency and policy heterogeneity. We introduce a novel MARL framework, Multi-Agent Divergence Policy Optimization (MADPO) with Mutual Policy Divergence Maximization (Mutual PDM). The proposed Mutual PDM consists of two key components, the intra-agent PDM and the inter-agent PDM. The former one encourages each agent to explore novel behaviors by deviating from its own previous policies across episodes, while the latter one promotes behavioral heterogeneity between the current agent and its predecessors. To improve the stability and directionality of the divergence signals, we employ the Conditional Cauchy-Schwarz (CCS) divergence for Mutual PDM. Further, we propose a new extension of CCS divergence for measuring policy divergence of more than two agents, the Generalized Conditional Cauchy-Schwarz (GCCS) divergence. Our GCCS divergence can guide agents with more informative incentives, and enhance the global heterogenization with multiple policies. We theoretically prove that the proposed two divergences can offer a stable entropy-guided incentive for exploration and heterogenization. We empirically demonstrate the connection between overall performance and policy heterogeneity in heterogeneous MARL. Extensive experiments on three challenging MARL benchmarks with over 25 heterogeneous scenarios demonstrate that MADPO consistently outperforms existing state-of-the-art sequential up dating methods in terms of both overall performance and agent heterogeneity.
Polarization imaging extends the capabilities of traditional vision systems by capturing rich multidimensional data essential for various tasks. However, color polarization cameras suffer from severe performance degradation under low-light conditions, which limits their use in real-world environments. Existing polarization-aware low-light enhancement (PLLE) methods mainly rely on supervised learning, which makes them sensitive to training data distributions and light-specific hyperparameters, with poor cross-domain generalization. To address these challenges, we propose ZeroPIE, the first zero-reference framework for PLLE trained exclusively on normal-light polarization images. Deviating from conventional data-driven paradigms, we derive four physically grounded illumination-invariant priors: Degree of Linear Polarization (DoLP), Angle of Linear Polarization (AoLP), geometric invariant, and chromatic invariant. These priors serve as robust, lighting-independent conditions that bridge low-light and normal-light polarization images. We further propose a recovery framework based on the Residual Denoising Diffusion Model (RDDM), where the proposed priors guide the reconstruction of normal-light polarization images without using real low-light data during training. Extensive experiments on both synthetic and real-world datasets demonstrate that ZeroPIE outperforms state-of-the-art methods in radiance restoration, polarization parameter reconstruction and downstream vision tasks, exhibiting strong robustness to unknown real-world degradations.
As an effective technique for network compression, sparsification-based pruning generally applies penalty terms to suppress the importance of dropped parameters, which is regarded as the suppressed sparsification paradigm. This paradigm weakens the dropped parameters, damaging the capacity of the dense network before pruning and thereby leading to performance degradation. To address the above issue, this paper comprehensively investigates the relative sparsity effect of emerging stimulative training (ST) and reveals the potential of the enhanced sparsification to alleviate performance degradation. Based on the relative sparsity effect, a structured pruning framework named STPP is proposed, utilizing an enhanced sparsification paradigm rather than a conventional suppressed one. STPP maintains the magnitude of dropped parameters and enhances the expressivity of kept parameters by self-distillation. To mitigate the suboptimality of subnet architecture exploration and severe distillation gap in vanilla ST, three key designs are introduced in STPP: (1) Efficient KD-guided exploration with architecture-aware metric broadcast. STPP couples enhanced sparsification and architecture exploration into a unified process guided by KD loss, and utilizes architecture-aware metric broadcast for efficient architecture exploration. (2) Progressive sparsification suitable for multi-dimensional space. STPP gradually reduces the width of subnets to transfer the expressivity smoothly and alleviate the distillation gap. (3) Subnet mutating expansion. STPP introduces a support subnet mutated from the sampled subnet for hierarchical knowledge distillation. Based on the above techniques, the potential of the novel enhanced sparsification paradigm in structured pruning is sufficiently unleashed. Specifically, without fine-tuning or other bells and whistles, STPP can reach a new Pareto frontier at different budgets compared to existing pruning methods, especially under extremely aggressive pruning scenarios, e.g., remaining 97.32% Top-1 accuracy (74.11% in 76.15%) while reducing 85% FLOPs for ResNet-50 on ImageNet.
Despite the impressive generative capabilities of text-to-image (T2I) diffusion models, they remain vulnerable to implicit sexual prompts, where subtle cues disguised as benign terms or adversarial tokens unexpectedly generate the inappropriate content due to model biases or latent correlations in training data. Existing safety mechanisms face fundamental limitations: detection methods primarily identify explicit content and fail to capture implicit malicious intent, while mitigation approaches rely on static negative prompts inadequate for diverse implicit scenarios. To address these challenges, we propose UniNDM, a unified noise-driven framework that rethinks safety mechanisms through the lens of noise dynamics in diffusion processes. Our key insight is that early-stage predicted noise exhibits inherent separability between normal and sexually explicit content, which we theoretically demonstrates quadratically increasing semantic concentration with timestep. Leveraging this property, we develop a lightweight noise-based detector achieving superior accuracy with virtually no computational overhead. For mitigation, we introduce noise-enhanced adaptive negative guidance: dynamically generating context-specific negative prompts via large language models to handle diverse implicit content, while optimizing initial noise by suppressing attention concentration on explicit tokens to provide comprehensive protection. Besides the U-Net-based diffusion models, we further extend our framework to emerging Diffusion Transformer architectures through region-constrained semantic guidance tailored for their unified multimodal attention. Comprehensive experiments across U-Net models (SDv1.4, v1.5, v2.1, XL) and DiT models (SDv3) on both natural and adversarial datasets demonstrate substantial improvements over state-of-the-art methods, including SLD, UCE, Safree, etc. Our code is publicly available at https://github.com/Aries-iai/UniNDM.
Significant progress has been made in the improvement and downstream applications of the Contrastive Language-Image Pre-training (CLIP) vision-language model, while less attention has been paid to the interpretation of CLIP. We propose a Gradient-based visual and textual Explanation method for CLIP (Grad-ECLIP), which interprets the matching result of CLIP for a specific input image-text pair. By decomposing the encoder's architecture and identifying the relationship between matching similarity and intermediate spatial features, Grad-ECLIP generates effective heat maps that reveal the impact of image regions or words on the CLIP results. Unlike previous Transformer interpretation methods that focus on utilizing self-attention maps, which are typically extremely sparse in CLIP, we produce high-quality visual explanations by applying channel and spatial weights to token features. Qualitative and quantitative evaluations verify the effectiveness and superiority of Grad-ECLIP compared with the state-of-the-art methods. Finally, a series of analyses are conducted based on our visual and textual explanation results, from which we explore the working mechanism of image-text matching, the strengths and limitations in attribution identification of CLIP, and the relationship between the concreteness/abstractness of a word and its usage in CLIP. The code of Grad-ECLIP is available here: https://github.com/Cyang-Zhao/Grad-Eclip.
Open-vocabulary 3D scene graph generation aims to predict 3D objects and their predicates beyond the annotated label space. Compared to closed-set 3D scene graph generation methods, the open-vocabulary approach is more general, practical, and less dependent on labor-intensive ground truth annotations. Existing open-vocabulary 3D scene graph generation methods rely on learning individual object and predicate features in the representation space while ignoring higher-level 3D scene representations, leading to overfitting and suboptimal performance. In this work, we propose a hyperbolic learning-based approach to address this problem by leveraging hyperbolic geometry to learn hierarchical 3D scene representations in the form of scene-region-instance, where the scene represents the complete 3D environment, a region contains related instances, and an instance corresponds to an individual object or predicate. Specifically, our method decomposes a 3D scene into a discrete hierarchy consisting of scene, region, and instance nodes, and embeds this hierarchy into a learned hyperbolic representation space. The learned hyperbolic embeddings are optimized in a bottom-up manner, where higher-level nodes are derived from their corresponding child nodes. The learned hierarchical 3D scene representations are incorporated as structural and semantic guidance for open-vocabulary 3D scene graph generation. We further observe that outliers in the form of erroneous hyperbolic embeddings can negatively impact hierarchical reasoning. To mitigate their negative impact, we present an enhancement strategy that learns an adaptive distance metric robust to the outliers over the learned hyperbolic representation space and subsequently improves overall performance. Extensive experiments on 3DSSG and ScanNet datasets demonstrate the effectiveness of our method in 3D scene graph generation under closed-set, open-vocabulary, and zero-shot settings.
Multi-view clustering (MVC) relies on consistency learning to align and fuse multi-view information for building clustering decision boundaries. However, mainstream methods adopt sample-to-sample/distribution/structure similarity smoothing for consistency alignment, which builds upon continuous cluster manifolds with semantic and geometric overlap and reliable paired priors of sample correspondences. They suffer from semantic- and instance-level view-unaligned problems inherent in the real-world data with discrete non-convex structures and unreliable cross-view correspondences. Misled by unaligned noise, they weaken discriminative semantic boundaries via similarity smoothing over false-positive pairs, and degrade discrete non-convex structures by mistakenly bridging distinct cluster manifolds via pseudo-semantic interpolation. Such consistency alignment pursues highly similar representations, distributions and structures, which deviates from the clustering logic of many-to-one partitioning as well as its learning goal for separable semantic boundaries. Guided by the many-to-one principle, we jointly formulate consistency alignment and clustering decision as a novel sample-to-cluster map, termed Multi-view Discrete Optimal Transport. Specifically, MvDOT is instantiated as a cluster-level transport matching framework that first adopts semi-discrete OT to learn a global OT barycenter via aggregating semantic-geometric information from all views, and then transports samples to barycenter-anchored consensus clusters under consistency constraints on semantic assignment and geometric measure. Even with discrete cluster manifolds and unreliable sample correspondences, MvDOT achieves cross-view consistent alignment while preserving inter-cluster semantic boundaries to uncover the underlying cluster structures. Extensive experiments show MvDOT outperforms 10 baselines with higher confidence and stronger robustness in complex MVC tasks.
Text-to-image diffusion models benefit artists with high-quality image generation. Yet their stochastic nature hinders artists from creating consistent images of the same subject. To address this challenge, we previously proposed a cluster-guided tuning paradigm, termed OneActor, which efficiently performs consistent subject generation via a learned semantic cluster guidance to bypass the laborious backbone tuning. However, along with the rapid development, there is a growing demand for the capability to synthesize images containing multiple consistent subjects from user-provided image references, which exceeds the competence of OneActor. For this issue, we present a novel consistent multi-subject generation paradigm, termed OneActor++, to render multiple subjects from both textual prompts and image references. To start with, we pioneer a new multi-target formulation of the cluster guidance theory, paving the way to multi-subject identity preservation. Leveraging this foundation, we reinvent multi-subject modulization with a Mixture of Projectors (MoP) architecture. This design orchestrates a holistic representation of multiple subjects aggregating their commonality and individualities. To refine the cluster guidance predictions, we introduce an enhanced tuning recipe featuring an anchored noising strategy to rectify guidance misalignment, coupled with a drift regulation strategy to prevent optimization collapse. To unlock the new ability of image-referenced generation, we devise a one-step inversion mechanism as the final piece of the overall paradigm. Comprehensive experiments demonstrate that OneActor++ outperforms a variety of baselines in generating multiple consistent subjects from both prompt and image inputs. With excellent subject consistency, superior prompt alignment, and high image quality, OneActor++ showcases powerful practical possibilities across a wide range of application scenarios.
Most existing multimodal sentiment analysis approaches assume access to complete multimodal inputs. However, real-world applications frequently encounter incomplete or corrupted modalities, posing a critical challenge. Although several methods have been proposed to tackle this issue, they mainly rely on data imputation and heuristic coordination constraints, which fail to effectively extract and leverage task-relevant information from the incomplete multimodal data. To address this challenge, we propose a unified framework termed Mutual Information Disentanglement with uncertainty-Aware fuSion (MIDAS), which effectively restructures multimodal representations under incomplete conditions. MIDAS adopts a variational modeling strategy to represent each modality with multivariate Gaussian latent variables and further decomposes them into shared and exclusive factors. To obtain reliable representations, we design a minimax objective that minimizes the mutual information between shared and exclusive spaces for stable disentanglement, while maximizing the mutual information among shared spaces across modalities to enhance semantic alignment. In addition, an uncertainty-aware fusion mechanism is introduced, where posterior variance is leveraged as a reliability indicator to adaptively weight latent features during fusion, ensuring robust integration even when modalities are incomplete. Extensive experiments on three widely used datasets show that MIDAS achieves strong and consistent performance gains over competitive baselines across a wide range of incomplete settings, demonstrating its effectiveness and robustness for incomplete data scenarios.
Rank minimization is a fundamental yet NP-hard problem with applications in recommendation systems, signal processing, and video denoising. Neurodynamic approaches provide a principled framework by modeling optimization as a continuous-time dynamical system, but they are often sensitive to discretization parameters and require a large number of iterations in practice. In this paper, we propose a unified data-driven framework that integrates continuous-time Matrix Neurodynamic Approaches (MNA) with deep unfolding networks, thereby combining the interpretability and convergence guarantees of neurodynamic approaches with the adaptability of learning-based models. Firstly, we discretize the MNA via the forward Euler scheme, deriving two discrete algorithms: the Fixed-Step Discrete Matrix Neurodynamic Approach (FDMNA) and the Variable-Step Discrete Matrix Neurodynamic Approach (VDMNA). For both approaches, we establish explicit optimality conditions and analytically characterize the interplay among iteration number, error, and step-size parameters. Secondly, to enable end-to-end optimization, we unfold FDMNA and VDMNA into trainable deep architectures in which all algorithmic parameters are learned directly from data. Remarkably, this data-driven training allows both networks to achieve fixed-step optimality while preserving theoretical interpretability. To ensure differentiable learning, we further introduce the Logistic-Threshold Function (LT-Func), which is a smooth surrogate for the non-smooth rank-$r$ projection operator, thus enabling adaptive singular value selection within the network. Finally, extensive experiments on low-rank signal recovery and high-resolution video compression and reconstruction verify both the effectiveness and scalability of the proposed framework, demonstrating superior performance over conventional approaches.