Learning to read involves the formation and tuning of letter representations, but it is unknown whether this orthographic tuning influences very early visual processing or only later processing. This study tested the hypothesis that experience increases the extraction of sensory information about letters by comparing the EEG activity elicited by upright and inverted letters. In a set of conventional univariate analyses, we found that inverted letters elicited larger P1 amplitudes (starting ca. 110 msec) and larger N170 amplitudes (starting ca. 160 msec) compared with upright letters. These larger amplitudes could reflect enhanced processing, but they might instead reflect degraded processing. We therefore performed multivariate pattern classification (decoding) to assess the amount of information about letter identity in the neural signal. Specifically, we decoded which individual letter was presented from the pattern of voltage across the scalp at each time point. We found that decoding accuracy was greater for upright letters than for inverted letters during the P1 latency range (starting ca. 90 msec), particularly in electrodes over the left hemisphere. This provides evidence for enhanced tuning for upright letters in early visual processing. By contrast, we found higher decoding accuracy for inverted letters than for upright letters during and after the N170 component (starting ca. 140 msec). These results demonstrate that massive experience with upright letters influences sensory processing, leading to enhanced feature extraction for highly familiar (upright) letter forms at an early stage, followed by enhanced neural discriminability for less familiar (inverted) letter forms at a later stage.
The human brain has inherent limitations in consciously processing visual information. When individuals monitor a rapid sequence of images for detecting two targets, they often miss the second target (T2) if it appears within a short time frame of 200-500 ms after the first target (T1), a phenomenon known as the attentional blink (AB). The neural mechanism behind the AB remains unclear, largely due to the use of simplistic visual items such as letters and digits in conventional AB experiments, which differ significantly from naturalistic vision. This study employs advanced multivariate pattern analysis (MVPA) of human electroencephalography (EEG) data (including 17 females and 18 males) to explore the neural representations associated with target processing within a naturalistic paradigm under conditions where AB does or does not occur. Our MVPA analysis successfully decoded the identity of target images from EEG data. Moreover, in the AB condition, characterized by a limited time between targets, T1 processing coincided with T2 processing, resulting in the suppression of late representational markers of both T1 and T2. Conversely, in the condition with longer inter-target interval, neural representations endured for a longer duration. These findings suggest that the AB can be attributed to the suppression of neural representations in the later stages of target processing.
Processing ordinality, i.e., the rank of an item in a series such as 1st, 2nd, 3rd, etc., is a fundamental skill shared by humans and animals. While humans often use symbolic sequences like numbers or letters, ordinality does not depend on language or symbols. Across species, ordinality plays a critical role in behaviors such as decision-making, foraging, and social organization. We hypothesize that ordinality perception is supported by neuronal tuning, i.e., neurons selectively responsive to specific ranks. Using ultrahigh-field 7 T fMRI and population receptive field (pRF) modeling in human participants (both female and male), we identified neural populations in parietal and premotor cortices that are tuned to nonsymbolic ordinal positions. Comparable with other sensory domains, tuning width increased with preferred ordinal rank, suggesting reduced precision and potentially lower perceptual accuracy for higher ranks. Additionally, pRF measurements revealed that cortical territory devoted to higher ordinalities decreased with rank, reinforcing that neural precision is greatest for early positions (e.g., 1st and 2nd) and declines with rank. These responses did not generalize to symbolic ordinality. Similar tuning to nonsymbolic ordinality emerged spontaneously in hierarchical convolutional neural networks trained on visual tasks. Together, these results suggest that the tuning properties of these neuronal populations support nonsymbolic ordinality perception and may reflect an inherent feature of neural processing.
Letters are the primitives of reading expertise. Single letter recognition relies on a hierarchy of processing stages, in which early visual features gradually evolve into abstract letter representations, but the temporal organization of these stages remains poorly understood. To address it, we applied multivariate pattern analysis (MVPA) to electroencephalography (EEG) data recorded while adult readers (n = 35) performed a one-back repetition detection task on single letters and pseudoletters. Traditional event-related potential (ERP) analyses revealed differences between letters and pseudoletters in the N1 (140-170 ms), P2 (210-270 ms), and P3 (300-500 ms) components. Multivariate temporal generalization analyses showed that neural patterns distinguishing letters from pseudoletters were highly generalizable from approximately 140 to 600 ms after stimulus onset. A spatiotemporal searchlight analysis indicated that, despite this temporal generalization, the topographic configuration of EEG channels contributing to classification changed along this window, suggesting that neural representations in later processing stages were transformed from earlier perceptual stages. These findings indicate that letter recognition unfolds as a cascade of continuous and interacting processes rather than via discrete stages. Early perceptual letter-specific activity, indexed by the N1 component, remains engaged throughout later, increasingly abstract, orthographic processing stages to jointly support letter identification.
Letter processing plays a key role in visual word recognition. However, word recognition models typically overlook or greatly simplify early perceptual processes of letter recognition. We suggest that optimal transport theory may provide a computational framework for describing letter shape processing. We use representational similarity analysis to show that optimal transport cost (Wasserstein distance) between pairs of letters aligns with neural activity elicited by visually presented letters <225 ms after stimulus onset, outperforming an existing approach based on shape overlap. We additionally show that optimal transport can capture the emergence of geometric invariances (e.g., to position or size) observed in letter perception. Finally, we demonstrate that Wasserstein distance predicts neural activity similarly well to features from artificial networks trained to classify images and letters. However, whereas representations in artificial neural networks emerge in a computationally unconstrained manner, our proposal provides a computationally explicit route to modeling the earliest orthographic processes.
Automaticity in decoding print is crucial for fluent reading. This process relies on associative memories between letters and speech sounds (LSS) that are overlearned through years of reading practice. While previous neuroimaging studies have identified neural correlates of LSS integration across different stages of reading development, the specific neural signatures underlying automatic LSS integration remain unclear. In the present study, we aimed to isolate neural components specifically associated with automatic LSS integration in literate adults. To this end, we developed an artificial script training paradigm in which adult native Finnish speakers were taught to associate unfamiliar foreign letters with familiar Finnish speech sounds. Using magnetoencephalography (MEG), we directly compared the audiovisual processing of newly learned and overlearned LSS associations within the same task, one day after training. Event-related fields (ERFs) and multivariate decoding revealed largely shared neural circuits of audiovisual integration for both types of LSS associations, as evidenced by multisensory interaction and congruency effects. Interestingly, the processing of congruent overlearned audiovisual associations uniquely recruited brain activity in the left medial parietal cortex during the 235-475 ms time window. Furthermore, temporal generalization analysis of the congruency effects revealed that while both newly learned and overlearned audiovisual associations engaged common neural mechanisms, the newly learned associations were processed systematically more slowly by a few hundred milliseconds. Our study identified the spatiotemporal neural signatures underlying automatized LSS processing, offering insights into neural markers that may help identify levels of reading proficiency.
Despite decades of intense study, the spatiotemporal processing of letters in visual word recognition has yet to be elucidated, with the debate largely focusing on whether individual letters are processed serially or in parallel. The present study investigated the processing of individual letters and letter combinations through time in visual word recognition using displays where signal-to-noise ratio (SNR) varied randomly throughout a 200 ms exposure duration. In Experiment 1, SNR varied either homogeneously across all letters or independently for each letter position (cf. heterogeneous sampling). Reading accuracy was substantially greater with homogeneous than heterogeneous sampling. Experiment 2 again used heterogeneous sampling and classification images (CIs) were calculated for individual letter positions or conjunctions thereof, reflecting processing efficiency according to time during target exposure. These CIs or their Fourier transforms were passed to a classifier to assess differences in the result patterns across individual letter positions or their conjunctions. Overall, the present results indicate the following: (1) significant parallel letter processing capacity throughout exposure duration; (2) dissociable processing mechanisms for each letter position; and (3) letter position-specific mechanisms for letter conjunctions that are distinct from those for individual letters. The results also provide evidence relevant to the neural code underlying the perceptual mechanisms that were uncovered.
Neural processes distinguishing romantic love from opposite-sex friendships remain a key challenge in neuroscience. Research on monogamous prairie voles has revealed that the nucleus accumbens (NAcc) is pivotal for partner-specific processing through plastic changes. However, it remains unclear in humans whether the NAcc differentiates a partner from opposite-sex friends, and how partner-related processing changes as the relationship matures. In a sample of 47 heterosexual male participants, we investigated the neural representations of a female partner, a female friend, and a male friend, in the NAcc, caudate nucleus and putamen. We collected fMRI data from participants during a social incentive delay task designed to elicit neural responses in anticipation of social approval from each of them. Classifier-based multivoxel pattern analysis (MVPA) demonstrated that neural activity patterns in all three regions distinguished the female partner from the female friend. Importantly, similarity-based MVPA revealed that, in the NAcc, the female friend was represented closer to the male friend than to the partner. Furthermore, exploratory analyses indicated that individuals in longer romantic relationships presented less distinguishable neural responses between the partner and the female friend in the NAcc. These findings suggest partner-specific processing in the NAcc, with this specificity diminishing as the relationship matures.
The classification of handwritten letters from invasive neural signals has lately been subject of research to restore communication abilities in people with limited movement capacities. This study explores the classification of ten letters (a,d,e,f,j,n,o,s,t,v) from non-invasive neural signals of 20 participants, offering new insights into the neural correlates of handwriting. Letters were classified with two methods: the direct classification from low-frequency and broadband electroencephalogram (EEG) and a two-step approach comprising the continuous decoding of hand kinematics and the application of those in subsequent classification. The two-step approach poses a novel application of continuous movement decoding for the classification of letters from EEG. When using low-frequency EEG, results show moderate accuracies of 23.1% for ten letters and 39.0% for a subset of five letters with highest discriminability of the trajectories. The two-step approach yielded significantly higher performances of 26.2% for ten letters and 46.7% for the subset of five letters. Hand kinematics could be reconstructed with a correlation of 0.10 to 0.57 (average chance level: 0.04) between the decoded and original kinematic. The study shows the general feasibility of extracting handwritten letters from non-invasively recorded neural signals and indicates that the proposed two-step approach can improve performances. As an exploratory investigation of the neural mechanisms of handwriting in EEG, we found significant influence of the written letter on the low-frequency components of neural signals. Differences between letters occurred mostly in central and occipital channels. Further, our results suggest movement speed as the most informative kinematic for the decoding of short hand movements.
This longitudinal study investigated the differential impacts of maternal speech on early socio-communicative development in infants at low likelihood (LL) and elevated likelihood (EL) of autism spectrum disorder (ASD). Using functional near-infrared spectroscopy, we measured cortical responses and connectivity in 6-month-old infants while they listened to their mother's voice and an unfamiliar female voice. LL infants exhibited extensive cortical activation and robust connectivity in temporal and frontal regions, particularly in areas associated with voice processing, reward, and language functions. In contrast, EL infants showed minimal activation and weaker connectivity in these regions. Specifically, LL infants demonstrated significant connectivity between the superior temporal gyrus and the inferior frontal gyrus on the left side and between the orbitofrontal cortex and language areas, facilitating language processing and reward-related responses to maternal speech. These neural patterns were absent in EL infants, highlighting a neural basis for subsequent language delays. Furthermore, many of these reward-related or language-related networks predicted subsequent language development. Our findings underscore the importance of neural sensitivity to familiar human voices, regarding them as rewards that will eventually facilitate the acquisition of speech.
Large neuronal networks demonstrate complex dynamics across multiple scales, ranging from single-neuron excitability and spike-train variability to mesoscopic rhythms and whole-brain activity. Different types of differential equation models have been developed to comprehend these phenomena, connecting deterministic, stochastic, and mean-field descriptions. At the deterministic level, ordinary differential equation (ODE) models, including conductance-based neuron models, neural-mass systems, and whole-brain networks, summarize neural behavior through a reduced set of macroscopic variables. At the population level, mean-field partial differential equation (PDE) models such as Fokker-Planck, age-structured, kinetic, and neural field equations describe the evolution of probability or population densities over membrane-potentials, synaptic states, and other kinetic variables. These PDEs link single-neuron mechanisms to population-level activity and allow one to analyze bifurcations, oscillations and other collective patterns. Stochastic differential equation (SDE) models and their extensions that include jump-diffusion processes and stochastic PDEs (SPDEs) are widely used to describe random membrane fluctuations, irregular spike trains, synaptic plasticity and large-scale variability in neural activity. These stochastic models are also applied to neural data analysis, for example to quantify noise in electro-physiological recordings and to infer latent neural dynamics. Because variability and noise are central in neural systems, we devote more space to stochastic models but always relate them back to the surrounding ODE and PDE frameworks. This hierarchy of ODE, PDE, and SDE-SPDE models shows that the versatility of differential-equation-based approaches in neuroscience offers unified tools for multiscale modeling, neural signal processing, cognitive modeling, and the analysis of noisy neural systems. We also discuss some known numerical and computational approaches, especially for stochastic models and conclude by outlining open challenges, such as multiscale inference, control-oriented formulations and the integration of differential-equation models with modern machine-learning methods.
Words with transposed letters-often referred to as jumbled words or transposed-letter nonwords-can still be read fluently. A comparable phenomenon occurs in Chinese, a logographic language in which words are composed of characters-square-shaped symbols that often correspond to morphemes. Consequently, most Chinese words are compound in nature. The present study investigated the neural mechanisms underlying the recognition of Chinese jumbled words-two-character compounds with transposed character positions-using visual event-related potentials (ERPs). Participants performed an implicit reading task (color detection) while viewing Chinese words (or jumbled words) and pseudowords presented under two conditions that manipulated character order within each item: a left-to-right Canonical condition and a right-to-left Jumbled condition. Robust ND250 and left anterior negativity (LAN) effects were observed in both conditions. The ND250, associated with orthographic whole-word processing, was significantly attenuated in the Jumbled condition relative to the Canonical condition, suggesting that jumbled words only partially activated the orthographic representations of their base forms. The LAN, interpreted as an index of morphological decomposition, was also elicited in both conditions and did not significantly differ between Canonical and Jumbled conditions, indicating that decomposition processes operate automatically and are insensitive to character order. Taken together, these findings demonstrate that both whole-word and decomposition processing routes are engaged in the recognition of Chinese jumbled words, offering novel electrophysiological evidence for the flexibility of visual recognition of Chinese compound words.
This study assessed the neural mechanisms and relative saliency of categorization for speech sounds and comparable graphemes (i.e., visual letters) of the same phonetic label. Given that linguistic experience shapes categorical processing, and letter-speech sound matching plays a crucial role during early reading acquisition, we hypothesized sound phoneme and visual grapheme tokens representing the same linguistic identity might recruit common neural substrates, despite originating from different sensory modalities. Behavioral and neuroelectric brain responses (ERPs) were acquired as participants categorized stimuli from sound (phoneme) and homologous letter (grapheme) continua each spanning a /da/-/ga/ gradient. Behaviorally, listeners were faster and showed stronger categorization of phoneme compared to graphemes. At the neural level, multidimensional scaling of the EEG revealed responses self-organized in a categorial fashion such that tokens clustered within their respective modality beginning ∼150-250 ms after stimulus onset. Source-resolved ERPs further revealed modality-specific and overlapping brain regions supporting phonetic categorization. Left inferior frontal gyrus and auditory cortex showed stronger responses for sound category members compared to phonetically ambiguous tokens, whereas early visual cortices paralleled this categorical organization for graphemes. Auditory and visual categorization also recruited common visual association areas in extrastriate cortex but in opposite hemispheres (auditory = left; visual = right). Our findings reveal both auditory and visual sensory cortex supports categorical organization for phonetic labels within their respective modalities. However, a partial overlap in phoneme and grapheme processing among occipital brain areas implies the presence of an isomorphic, domain-general mapping for phonetic categories in dorsal visual system.
This study investigated the neural effects of perceptual (caused by blur processing) and sexual ambiguity (induced by morphing manipulation) in facial images via event-related potentials and time-frequency analysis. Gaussian blur and morphing of male and female faces were used to create gradually ambiguous face images, and participants completed a sexual dimorphism judgment task. Results revealed that the N170 was strongly affected by blur, which suggested that it disrupted the structural encoding of faces. The early posterior negativity exhibited a different response compared with the N170, which indicated that it may reflect both bottom-up processing due to lack of visual cues and top-down processing driven by the sexual dimorphism judgment task. Additionally, late posterior potential amplitude increased under conditions of explicit perceptual and sexual faces, which confirmed that ease of interpretation in sexual dimorphism judgments influenced neural responses. Furthermore, the time-frequency analysis revealed that high-frequency gamma activity at approximately 200 ms was associated with extracting and evaluating facial features, whereas activity after 600 ms reflected processes related to retaining facial information. These findings suggest that visual processing and semantic evaluation of faces rely on complex mechanisms influenced by both the clarity of physical cues and task context.
Recently, the biologically inspired intelligent artificial visual neural system has aroused enormous interest. However, there are still significant obstacles in pursuing large-scale parallel and efficient visual memory and recognition. In this study, we demonstrate a 28 × 28 synaptic devices array for the artificial visual neuromorphic system, within the size of 0.7 × 0.7 cm2, which integrates sensing, memory, and processing functions. The highly uniform floating-gate synaptic transistors array were constructed by the wafer-scale grown monolayer molybdenum disulfide with Au nanoparticles (NPs) acting as the electrons capture layers. Various synaptic plasticity behaviors have been achieved owing to the switchable electronic storage performance. The excellent optical/electrical coordination capabilities were implemented by paralleled processing both the optical and electrical signals the synaptic array of 784 devices, enabling to realize the badges and letters writing and erasing process. Finally, the established artificial visual convolutional neural network (CNN) through optical/electrical signal modulation can reach the high digit recognition accuracy of 96.5%. Therefore, our results provide a feasible route for future large-scale integrated artificial visual neuromorphic system.
Subjective cognitive decline (SCD) is a preclinical stage of mild cognitive impairment (MCI). Although dance training has been shown to be beneficial for mental health, cognitive function, and neural activity in older adults with MCI, its effect on SCD remains unclear. This study aimed to examine the effects of dance training on the aforementioned factors and on oxytocin secretion in older adults with SCD. Participants (aged 65-84 years) were assigned to either the intervention group (n = 22) with a 12-week dance training program or the control group without any alternative training (n = 22). Apathy, depression, Montreal Cognitive Assessment scores, urinary oxytocin levels, and resting-state functional magnetic resonance imaging indices, including amplitude of low-frequency fluctuations (ALFF) and functional connectivity (FC), were evaluated pre- and post-intervention. Compared to the control group, the intervention group exhibited significantly higher urinary oxytocin levels and significantly higher ALFF in the left medial orbitofrontal cortex post-intervention. Moreover, the intervention group showed more enhanced FC between the left medial orbitofrontal cortex and the left precuneus post-intervention than the control group. However, mental health or cognitive performance was not significantly different between the groups. Our results are particularly important in light of previous findings that older adults with SCD show a reduced FC between the medial orbitofrontal cortex and the precuneus, and that oxytocin levels are positively associated with the prefrontal-amygdala oxytocinergic circuit in socioemotional processing. Thus, dance training may contribute to socioemotional resilience-related neural and molecular adaptations in SCD.
This study investigated the behavioral and neural responses underlying affective and cognitive control during emotional interference and their association with sleep disturbance in insomnia disorder (ID) patients. Forty-three ID patients and 43 aged- and sex-comparable healthy controls (HC) completed an emotional interference task (EIT) while undergoing functional magnetic resonance imaging (fMRI) scanning. Sleep disturbance was assessed using self-report questionnaires. Analyses of variances (ANOVAs) were conducted on behavioral measures and brain activations. Correlations were used to examine the relationships between sleep disturbance and the significantly activated brain regions. Behaviorally, after controlling for trait anxiety, the ANOVA on reaction times (RTs) revealed a significant group × attention × emotion interaction. Specifically, compared with HC, ID patients exhibited significantly slower RTs when attending to neutral faces and ignoring fearful faces. For accuracy, a significant main effect of group indicated lower accuracy in ID patients than HC. Whole-brain ANOVA analysis revealed that under the ignore-fear condition, ID patients showed increased right dorsolateral prefrontal cortex (DLPFC) activation and reduced caudate nucleus activation, whereas under the attend-fear condition they showed reduced DLPFC and heightened caudate responses related to HC. Exploratory analyses indicated that, in ID patients, greater sleep disturbance was associated with less right caudate nucleus deactivation under the ignore-fear condition. These findings suggest attention-dependent alterations in prefrontal control and striatal processing of emotional salience during emotional interference in ID, which may help explain how sleep disturbance is related to disrupted cognitive-emotional regulation in this condition.
According to psycholinguistic theories, during language processing, spoken and written words are first encoded along independent phonological and orthographic dimensions, then enter into modality-independent syntactic and semantic codes. Non-invasive brain imaging has isolated several cortical regions putatively associated with those processing stages, but lacks the resolution to identify the corresponding neural codes. Here, we describe the firing responses of over 1000 neurons, and mesoscale field potentials from over 1400 microwires and 1500 iEEG contacts in 21 awake neurosurgical patients with implanted electrodes during written and spoken sentence comprehension. Using forward modeling of temporal receptive fields, we determined which sensory or abstract dimensions are encoded. We observed a double dissociation between superior temporal neurons sensitive to phonemes and phonological features and previously unreported ventral occipito-temporal neurons sensitive to letters and orthographic features. We also discovered novel neurons, primarily located in middle temporal and inferior frontal areas, which are modality-independent and show responsiveness to higher linguistic features. Overall, these findings show how language processing can be linked to neural dynamics, across multiple brain regions at various resolutions and down to the level of single neurons.
Letter recognition is assumed to involve several levels of analysis, including coarse tuning for category and novelty and more fine tuning for specific features, related to letter orientation. We employed an oddball fast periodic visual stimulation (FPVS) paradigm with magnetoencephalography (Elekta VectorView, 306 sensors) to study neural discrimination responses in the source space. Using contrasts between native and foreign letters, digits, or inverted native letters, we aimed to isolate the neural responses to visual novelty, category, and orientation during character analysis. The study was conducted with a cohort of 25 adults. The response topography demonstrated bilateral organization, including language-related brain regions such as the ventral occipitotemporal cortex, inferior parietal cortex, and middle temporal areas. Comparing conditions, we revealed right lateralized parietal clusters, associated with novelty tuning, and left lateralized occipitotemporal clusters exhibiting higher activity for letters among digits discrimination, supporting the role of this area in letter processing. No distinct spatial patterns specific to orientation tuning were observed in comparison to novelty and category tuning. We propose that expertise-dependent orientation-specific tuning mechanisms may operate within an embedded neural framework that spatially overlaps with coarse tuning systems, but are characterized by specific spatiotemporal patterns.
The recognition of handwritten Arabic characters offerings a multifaceted challenge that holds fundamental standing across domains such as document digitization, human-computer interaction, and assistive technologies. Arabic script's cursive form combined with positional character variations and diverse handwriting styles creates substantial obstacles for traditional machine learning techniques. In this study, we propose a deep Convolutional Neural Network (CNN) architecture tailored for the classification of isolated handwritten Arabic letters. The dataset includes 28 classes representing the Arabic alphabet, with balanced samples preprocessed and augmented for robust training. The proposed CNN achieved a high classification accuracy of 96.8%, significantly outperforming Support Vector Machine (SVM) and K-Nearest Neighbors (KNN), which recorded 85.3% and 82.1%, respectively. Performance was evaluated using cross-validation, confusion matrix analysis, and statistical testing. A paired t-test yielded a p-value < 0.01, confirming the statistical significance of the CNN's superiority. This work possesses significant potential for practical deployment in areas including postal address reading bank check processing educational tools and historical manuscript digitization. The model's architectural design allows extension to Persian and Urdu cursive-based languages which enables multilingual handwriting recognition. Future directions include scaling the system to support connected script and word-level recognition as well as integration into mobile or web-based OCR systems for broader accessibility and real-time use. The proposed CNN architecture adheres to traditional deep learning design principles yet demonstrates its unique contribution through specialized application to isolated Arabic handwriting by employing a meticulously balanced dataset alongside an extensive augmentation pipeline and conducting statistically validated comparisons with classical methods. The study also provides a reproducible framework benchmarked on real handwriting variations.