Reinforcement learning methods have recently been very successful at performing complex sequential tasks like playing Atari games, Go and Poker. These algorithms have outperformed humans in several tasks by learning from scratch, using only scalar rewards obtained through interaction with their environment. While there certainly has been considerable independent innovation to produce such results, many core ideas in reinforcement learning are inspired by phenomena in animal learning, psychology and neuroscience. In this paper, we comprehensively review a large number of findings in both neuroscience and psychology that evidence reinforcement learning as a promising candidate for modeling learning and decision making in the brain. In doing so, we construct a mapping between various classes of modern RL algorithms and specific findings in both neurophysiological and behavioral literature. We then discuss the implications of this observed relationship between RL, neuroscience and psychology and its role in advancing research in both AI and brain science.
Neuroscience and AI have an intertwined history, largely relayed in the literature of both fields. In recent years, due to the engineering orientations of AI research and the monopoly of industry for its large-scale applications, the mutual expansion of neuroscience and AI in fundamental research seems challenged. In this paper, we bring some empirical evidences that, on the contrary, AI and neuroscience are continuing to grow together, but with a pronounced interest in the fields of study related to neurodegenerative diseases since the 1990s. With a temporal knowledge cartography of neuroscience drawn with advanced document embedding techniques, we draw the dynamical shaping of the discipline since the 1970s and identified the conceptual articulation of AI with this particular subfield mentioned before. However, a further analysis of the underlying citation network of the studied corpus shows that the produced AI technologies remain confined in the different subfields and are not transferred from one subfield to another. This invites us to discuss the genericity capability of AI in the context of an intradisciplinary development, especially in the diffusion of its associated metro
Reinforcement learning (RL) has a rich history in neuroscience, from early work on dopamine as a reward prediction error signal (Schultz et al., 1997) to recent work proposing that the brain could implement a form of 'distributional reinforcement learning' popularized in machine learning (Dabney et al., 2020). There has been a close link between theoretical advances in reinforcement learning and neuroscience experiments throughout this literature, and the theories describing the experimental data have therefore become increasingly complex. Here, we provide an introduction and mathematical background to many of the methods that have been used in systems neroscience. We start with an overview of the RL problem and classical temporal difference algorithms, followed by a discussion of 'model-free', 'model-based', and intermediate RL algorithms. We then introduce deep reinforcement learning and discuss how this framework has led to new insights in neuroscience. This includes a particular focus on meta-reinforcement learning (Wang et al., 2018) and distributional RL (Dabney et al., 2020). Finally, we discuss potential shortcomings of the RL formalism for neuroscience and highlight open q
Land use expansion is linked to major sustainability concerns including climate change, food security and biodiversity loss. This expansion is largely concentrated in so-called frontiers, defined here as places experiencing marked transformations due to rapid resource exploitation. Understanding the mechanisms shaping these frontiers is crucial for sustainability. Previous work focused mainly on explaining how active frontiers advance, in particular into tropical forests. Comparatively, our understanding of how frontiers emerge in territories considered marginal in terms of agricultural productivity and global market integration remains weak. We synthesize conceptual tools explaining resource and land-use frontiers, including theories of land rent and agglomeration economies, of frontiers as successive waves, spaces of territorialization, friction, and opportunities, anticipation and expectation. We then propose a new theory of frontier emergence, which identifies exogenous pushes, legacies of past waves, and actors anticipations as key mechanisms by which frontiers emerge. Processes of abnormal rent creation and capture and the built-up of agglomeration economies then constitute k
This paper provides a perspective on applying the concepts of information thermodynamics, developed recently in non-equilibrium statistical physics, to problems in theoretical neuroscience. Historically, information and energy in neuroscience have been treated separately, in contrast to physics approaches, where the relationship of entropy production with heat is a central idea. It is argued here that also in neural systems information and energy can be considered within the same theoretical framework. Starting from basic ideas of thermodynamics and information theory on a classic Brownian particle, it is shown how noisy neural networks can infer its probabilistic motion. The decoding of the particle motion by neurons is performed with some accuracy and it has some energy cost, and both can be determined using information thermodynamics. In a similar fashion, we also discuss how neural networks in the brain can learn the particle velocity, and maintain that information in the weights of plastic synapses from a physical point of view. Generally, it is shown how the framework of stochastic and information thermodynamics can be used practically to study neural inference, learning, and
Neuroscience research publications encompass a vast wealth of knowledge. Accurately retrieving existing information and discovering new insights from this extensive literature is essential for advancing the field. However, when knowledge is dispersed across multiple sources, current state-of-the-art retrieval methods often struggle to extract the necessary information. A knowledge graph (KG) can integrate and link knowledge from multiple sources. However, existing methods for constructing KGs in neuroscience often rely on labeled data and require domain expertise. Acquiring large-scale, labeled data for a specialized area like neuroscience presents significant challenges. This work proposes novel methods for constructing KG from unlabeled large-scale neuroscience research corpus utilizing large language models (LLM), neuroscience ontology, and text embeddings. We analyze the semantic relevance of neuroscience text segments identified by LLM for building the knowledge graph. We also introduce an entity-augmented information retrieval algorithm to extract knowledge from the KG. Several experiments were conducted to evaluate the proposed approaches. The results demonstrate that our me
In recent years, deep learning revolutionized machine learning and its applications, producing results comparable to human experts in several domains, including neuroscience. Each year, hundreds of scientific publications present applications of deep neural networks for biomedical data analysis. Due to the fast growth of the domain, it could be a complicated and extremely time-consuming task for worldwide researchers to have a clear perspective of the most recent and advanced software libraries. This work contributes to clarify the current situation in the domain, outlining the most useful libraries that implement and facilitate deep learning application to neuroscience, allowing scientists to identify the most suitable options for their research or clinical projects. This paper summarizes the main developments in Deep Learning and their relevance to Neuroscience; it then reviews neuroinformatic toolboxes and libraries, collected from the literature and from specific hubs of software projects oriented to neuroscience research. The selected tools are presented in tables detailing key features grouped by domain of application (e.g. data type, neuroscience area, task), model engineeri
The exponential growth of neuroscience literature presents a significant challenge for researchers seeking to efficiently access and utilize relevant information. To address this issue, we introduce the Brain Knowledge Engine (BrainKnow), an automated system designed to extract, link, and synthesize neuroscience knowledge from scientific publications. BrainKnow constructs a comprehensive knowledge graph encompassing 3,626,931 relationships across 37,011 neuroscience concepts, derived from 1,817,744 articles. This vast repository of knowledge is accessible through a user-friendly web interface, facilitating efficient navigation and data retrieval. BrainKnow employs advanced graph network algorithms, specifically Node2Vec, to enhance knowledge recommendation and visualization. This enables users to explore semantic relationships between concepts, predict potential new relationships, and gain a deeper understanding of the interconnectedness within neuroscience. Additionally, BrainKnow ensures real-time updates by synchronizing with PubMed, providing researchers with access to the most current information. BrainKnow serves as a valuable resource for neuroscience researchers, offering a
In recent years, Artificial Intelligence (AI) shows a spectacular ability of insertion inside a variety of disciplines which use it for scientific advancements and which sometimes improve it for their conceptual and methodological needs. According to the transverse science framework originally conceived by Shinn and Joerges, AI can be seen as an instrument which is progressively acquiring a universal character through its diffusion across science. In this paper we address empirically one aspect of this diffusion, namely the penetration of AI into a specific field of research. Taking neuroscience as a case study, we conduct a scientometric analysis of the development of AI in this field. We especially study the temporal egocentric citation network around the articles included in this literature, their represented journals and their authors linked together by a temporal collaboration network. We find that AI is driving the constitution of a particular disciplinary ecosystem in neuroscience which is distinct from other subfields, and which is gathering atypical scientific profiles who are coming from neuroscience or outside it. Moreover we observe that this AI community in neuroscienc
Language models encode text as subword tokens, raw bytes, or rendered pixels, but these encodings are usually compared under modeling constraints that expose different amounts of linguistic content to models across different languages. We instead ask what each encoding preserves when both the content and the downstream capacity are controlled. Using verified parallel sentences across thirteen languages and five scripts, we compare tokens, bytes, and pixels through a shared bottleneck whose width is swept to trace rate-utility frontiers. This separates three quantities that are often conflated: the number of input positions an encoding creates, the latent capacity available after encoding, and the task-relevant information that survives compression. We evaluate three utilities: surface form preservation, cross-lingual sentence alignment, and topic classification. No encoding dominates across tasks or capacity regimes. Pixels preserve surface form best, bytes preserve cross-lingual alignment best, especially in same-script multilingual settings, and tokens support topic prediction best. These performances are not explained by sequence length alone. Short inputs can discard useful mea
Neuroscience has long informed the development of artificial neural networks, but the success of modern architectures invites, in turn, the converse: can modern networks teach us lessons about brain function? Here, we examine the structure of the cortical column and propose that the transformer provides a natural computational analogy for multiple elements of cortical microcircuit organization. Rather than claiming a literal implementation of transformer equations in cortex, we develop a hypothetical mapping between transformer operations and laminar cortical features, using the analogy as an orienting framework for analysis and discussion. This mapping allows us to examine in greater depth how contextual selection, content routing, recurrent integration, and interlaminar transformations may be distributed across cortical circuitry. In doing so, we generate a broad set of predictions and experimentally testable hypotheses concerning laminar specialization, contextual modulation, dendritic integration, oscillatory coordination, and the effective connectivity of cortical columns. This proposal is intended as a structured hypothesis rather than a definitive account of cortical computa
Like many scientific disciplines, neuroscience has increasingly attempted to confront pervasive gender imbalances within the field. While much of the conversation has centered around publishing and conference participation, recent research in other fields has called attention to the prevalence of gender bias in citation practices. Because of the downstream effects that citations can have on visibility and career advancement, understanding and eliminating gender bias in citation practices is vital for addressing inequity in a scientific community. In this study, we sought to determine whether there is evidence of gender bias in the citation practices of neuroscientists. Using data from five top neuroscience journals, we find that reference lists tend to include more papers with men as first and last author than would be expected if gender were not a factor in referencing. Importantly, we show that this overcitation of men and undercitation of women is driven largely by the citation practices of men, and is increasing over time as the field becomes more diverse. We develop a co-authorship network to assess homophily in researchers' social networks, and we find that men tend to overci
Within the field of Humanities, there is a recognized need for educational innovation, as there are currently no reported tools available that enable individuals to interact with their environment to create an enhanced learning experience in the humanities (e.g., immersive spaces). This project proposes a solution to address this gap by integrating technology and promoting the development of teaching methodologies in the humanities, specifically by incorporating emotional monitoring during the learning process of humanistic context inside an immersive space. In order to achieve this goal, a real-time emotion detection EEG-based system was developed to interpret and classify specific emotions. These emotions aligned with the early proposal by Descartes (Passions), including admiration, love, hate, desire, joy, and sadness. This system aims to integrate emotional data into the Neurohumanities Lab interactive platform, creating a comprehensive and immersive learning environment. This work developed a ML, real-time emotion detection model that provided Valence, Arousal, and Dominance (VAD) estimations every 5 seconds. Using PCA, PSD, RF, and Extra-Trees, the best 8 channels and their r
This article discusses how to create an interactive virtual training program at the intersection of neuroscience, robotics, and computer science for high school students. A four-day microseminar, titled Swarming Powered by Neuroscience (SPN), was conducted virtually through a combination of presentations and interactive computer game simulations, delivered by subject matter experts in neuroscience, mathematics, multi-agent swarm robotics, and education. The objective of this research was to determine if taking an interdisciplinary approach to high school education would enhance the students learning experiences in fields such as neuroscience, robotics, or computer science. This study found an improvement in student engagement for neuroscience by 16.6%, while interest in robotics and computer science improved respectively by 2.7% and 1.8%. The curriculum materials, developed for the SPN microseminar, can be used by high school teachers to further evaluate interdisciplinary instructions across life and physical sciences and computer science.
The proliferation and refinement of affordable virtual reality (VR) technologies and wearable sensors have opened new frontiers in cognitive and behavioral neuroscience. This chapter offers a broad overview of VR for anyone interested in leveraging it as a research tool. In the first section, it examines the fundamental functionalities of VR and outlines important considerations that inform the development of immersive content that stimulates the senses. In the second section, the focus of the discussion shifts to the implementation of VR in the context of the neuroscience lab. Practical advice is offered on adapting commercial, off-theshelf devices to specific research purposes. Further, methods are explored for recording, synchronizing, and fusing heterogeneous forms of data obtained through the VR system or add-on sensors, as well as for labeling events and capturing game play.
Defining interdisciplinary physics today requires first a reformulation of what is physics today, which in turn calls for clarifying what makes a physicist. This assessment results from my forty year journey arguing and fighting to build sociophysics. My view on interdisciplinary physics has thus evolved jumping repeatedly to opposite directions before settling down to the following claim: today physics is what is done by physicists who handle a problem the "physicist's way". However the training of physicists should stay restricted to inert matter. Yet adding a focus on the universality of the physicist approach as a generic path to investigate a topic. Consequently, interdisciplinary physics should become a cabinet of curiosities including an incubator. The cabinet of curiosities would welcome all one shots papers related to any kind of object provided it is co-authored at least by one physicist. Otherwise the paper should uses explicitly technics from physics. In case a topic gets many papers, it would be moved to the incubator to foster the potential emergence of a new appropriate subfield of physics. A process illustrated by the subsection social physics in Frontiers in physic
Endowing brain anatomy, dynamics, and function with a network structure is becoming standard in neuroscience. In its simplest form, a network is a collection of units and relationships between them. The pattern of relations among the units encodes numerous properties which have been shown to have a profound effect on networked systems' dynamics and function. In an effort to strike a balance between idealization and detail, network neuroscience studies typically involve simplifying assumptions at both neural and network modeling levels. However, the extent to which existing neural models depend on such approximations is as yet poorly understood. Here, we discuss whether and how increasing neurophysiological detail and generalizing the basic simple network structure often adopted in network neuroscience may help improve our understanding of brain phenomenology and function.
Recent advances and reflections on reproducible human neuroscience, especially brain-wide association studies (BWAS) leveraging large datasets, have led to divergent and sometimes opposing views on research practices and priorities. The debates span multiple dimensions. Shifts along these axes have fractured consensus and further fragmented an already heterogeneous field of cognitive neuroscience. Here, we sketch a holistic and integrative response grounded in population neuroscience, organized around a closed-loop "design-analysis-interpretation" research cycle that aims to build consensus while bridging these divides. Our central claim is that population neuroscience offers a unique population-level vantage point for identifying general principles, characterizing inter-individual variabilities, and benchmarking intra-individual changes, thereby providing a supportive framework for small-scale, mechanism-focused studies at the individual level and allowing them to co-evolve with population-level studies. Population neuroscience is not simply about providing larger N for BWAS; its deeper goal is to accumulate a family of cross-scale priors and shared infrastructures that can suppor
Noninvasive medical neuroimaging has yielded many discoveries about the brain connectivity. Several substantial techniques mapping morphological, structural and functional brain connectivities were developed to create a comprehensive road map of neuronal activities in the human brain -namely brain graph. Relying on its non-Euclidean data type, graph neural network (GNN) provides a clever way of learning the deep graph structure and it is rapidly becoming the state-of-the-art leading to enhanced performance in various network neuroscience tasks. Here we review current GNN-based methods, highlighting the ways that they have been used in several applications related to brain graphs such as missing brain graph synthesis and disease classification. We conclude by charting a path toward a better application of GNN models in network neuroscience field for neurological disorder diagnosis and population graph integration. The list of papers cited in our work is available at https://github.com/basiralab/GNNs-in-Network-Neuroscience.
Benchmarking the performance of complex systems such as rail networks, renewable generation assets and national economies is central to transport planning, regulation and macroeconomic analysis. Classical frontier methods, notably Data Envelopment Analysis (DEA) and Stochastic Frontier Analysis (SFA), estimate an efficient frontier in the observed input-output space and define efficiency as distance to this frontier, but rely on restrictive assumptions on the production set and only indirectly address heterogeneity and scale effects. We propose Geometric Manifold Analysis (GeMA), a latent manifold frontier framework implemented via a productivity-manifold variational autoencoder (ProMan-VAE). Instead of specifying a frontier function in the observed space, GeMA represents the production set as the boundary of a low-dimensional manifold embedded in the joint input-output space. A split-head encoder learns latent variables that capture technological structure and operational inefficiency. Efficiency is evaluated with respect to the learned manifold, endogenous peer groups arise as clusters in latent technology space, a quotient construction supports scale-invariant benchmarking, and