Multimodal conversational generative AI has shown impressive capabilities in various vision and language understanding through learning massive text-image data. However, current conversational models still lack knowledge about visual insects since they are often trained on the general knowledge of vision-language data. Meanwhile, understanding insects is a fundamental problem in precision agriculture, helping to promote sustainable development in agriculture. Therefore, this paper proposes a novel multimodal conversational model, Insect-LLaVA, to promote visual understanding in insect-domain knowledge. In particular, we first introduce a new large-scale Multimodal Insect Dataset with Visual Insect Instruction Data that enables the capability of learning the multimodal foundation models. Our proposed dataset enables conversational models to comprehend the visual and semantic features of the insects. Second, we propose a new Insect-LLaVA model, a new general Large Language and Vision Assistant in Visual Insect Understanding. Then, to enhance the capability of learning insect features, we develop an Insect Foundation Model by introducing a new micro-feature self-supervised learning wi
Insect neuroethology provides a compelling biological template for efficient autonomous navigation. We draw an analogy between the formal embodied AI visual point-goal navigation task and the ability of insects to discover, learn, and refine visually guided paths around obstacles between a discovered food location and their nest. We develop a novel integrative model of mushroom body and central complex, two insect brain structures, that have been implicated, respectively, in associative learning and path integration. We demonstrate the mushroom body learning triggered by collisions results in adaptive obstacle avoidance and consequently optimised paths to the goal, corroborating the hypothesis of recent behavioural work that an insect can learn continuously as they travel. The embodied insect-inspired model achieves success rates comparable to recent state-of-the-art models at many orders of magnitude less computational cost in the standardised Habitat point-goal navigation benchmark. Testing in a more realistic simulated environment validates its robustness to perturbations.
Automatic recognition of insect sound could help us understand changing biodiversity trends around the world -- but insect sounds are challenging to recognize even for deep learning. We present a new dataset comprised of 26399 audio files, from 459 species of Orthoptera and Cicadidae. It is the first large-scale dataset of insect sound that is easily applicable for developing novel deep-learning methods. Its recordings were made with a variety of audio recorders using varying sample rates to capture the extremely broad range of frequencies that insects produce. We benchmark performance with two state-of-the-art deep learning classifiers, demonstrating good performance but also significant room for improvement in acoustic insect classification. This dataset can serve as a realistic test case for implementing insect monitoring workflows, and as a challenging basis for the development of audio representation methods that can handle highly variable frequencies and/or sample rates.
Insect-pests significantly impact global agricultural productivity and quality. Effective management involves identifying the full insect community, including beneficial insects and harmful pests, to develop and implement integrated pest management strategies. Automated identification of insects under real-world conditions presents several challenges, including differentiating similar-looking species, intra-species dissimilarity and inter-species similarity, several life cycle stages, camouflage, diverse imaging conditions, and variability in insect orientation. A deep-learning model, InsectNet, is proposed to address these challenges. InsectNet is endowed with five key features: (a) utilization of a large dataset of insect images collected through citizen science; (b) label-free self-supervised learning for large models; (c) improving prediction accuracy for species with a small sample size; (d) enhancing model trustworthiness; and (e) democratizing access through streamlined MLOps. This approach allows accurate identification (>96% accuracy) of over 2500 insect species, including pollinator (e.g., butterflies, bees), parasitoid (e.g., some wasps and flies), predator species (e
By leveraging their high mobility and small size, insects have been combined with microcontrollers to build up cyborg insects for various practical applications. Unfortunately, all current cyborg insects rely on implanted electrodes to control their movement, which causes irreversible damage to their organs and muscles. Here, we develop a non-invasive method for cyborg insects to address above issues, using a conformal electrode with an in-situ polymerized ion-conducting layer and an electron-conducting layer. The neural and locomotion responses to the electrical inductions verify the efficient communication between insects and controllers by the non-invasive method. The precise "S" line following of the cyborg insect further demonstrates its potential in practical navigation. The conformal non-invasive electrodes keep the intactness of the insects used while controlling their motion. With the antennae, important olfactory organs of insects preserved, the cyborg insect, in the future, may be endowed with abilities to detect the surrounding environment.
In this work, an individual-based model of forest insect outbreaks is presented. The results obtained show that the outbreak is an emerging feature of the system. It is a common product of the characteristics of insects, the environment in which the insects live, and the way insects behave in it. The outbreak dynamics is an effect of scale. In a sufficiently large forest regardless of the density of trees and their spatial distribution, provided that the range of insect dispersion is large enough, it develops in the form of an outbreak. In very small forests, the dynamics becomes more chaotic. It loses the outbreak character and, especially in the forest with random tree distribution, there is a possibility that the insect population goes extinct. The local dynamics of the number of insects on one tree in a forest, where the dynamics of all insects has the character of outbreak, is characterized by a rapid increase in number and then a rapid decrease until the extinction of the local population. It is the result of the influx of immigrants from neighboring trees. The type of tree distribution in the forest becomes visible when the density of trees becomes low and/or the range of in
Insects represent half of all global biodiversity, yet many of the world's insects are disappearing, with severe implications for ecosystems and agriculture. Despite this crisis, data on insect diversity and abundance remain woefully inadequate, due to the scarcity of human experts and the lack of scalable tools for monitoring. Ecologists have started to adopt camera traps to record and study insects, and have proposed computer vision algorithms as an answer for scalable data processing. However, insect monitoring in the wild poses unique challenges that have not yet been addressed within computer vision, including the combination of long-tailed data, extremely similar classes, and significant distribution shifts. We provide the first large-scale machine learning benchmarks for fine-grained insect recognition, designed to match real-world tasks faced by ecologists. Our contributions include a curated dataset of images from citizen science platforms and museums, and an expert-annotated dataset drawn from automated camera traps across multiple continents, designed to test out-of-distribution generalization under field conditions. We train and evaluate a variety of baseline algorithms
Identification of insects in flight is a particular challenge for ecologists in several settings with no other method able to count and classify insects at the pace of entomological lidar. Thus, it can play a unique role as a non-intrusive diagnostic tool to assess insect biodiversity, inform planning, and evaluate mitigation efforts aimed at tackling declines in insect abundance and diversity. While species richness of co-existing insects could reach tens of thousands, to date, photonic sensors and lidars can differentiate roughly one hundred signal types. This taxonomic specificity or number of discernible signal types is currently limited by instrumentation and algorithm sophistication. In this study we report 32,533 observations of wild flying insects along a 500-meter transect. We report the benefits of lidar polarization bands for differentiating species and compare the performance of two unsupervised clustering algorithms, namely Hierarchical Cluster Analysis and Gaussian Mixture Model. We demonstrate that polarimetric properties could be partially predicted even with unpolarized light, thus polarimetric lidar bands provide only a minor improvement in specificity. Finally, w
Camera traps, combined with AI, have emerged as a way to achieve automated, scalable biodiversity monitoring. However, the passive infrared (PIR) sensors that trigger camera traps are poorly suited for detecting small, fast-moving ectotherms such as insects. Insects comprise over half of all animal species and are key components of ecosystems and agriculture. The need for an appropriate and scalable insect camera trap is critical in the wake of concerning reports of declines in insect populations. This study proposes an alternative to the PIR trigger: ultra-lightweight convolutional neural networks running on low-powered hardware to detect insects in a continuous stream of captured images. We train a suite of models to distinguish insect images from backgrounds. Our design achieves zero latency between trigger and image capture. Our models are rigorously tested and achieve high accuracy ranging from 91.8% to 96.4% AUC on validation data and >87% AUC on data from distributions unseen during training. The high specificity of our models ensures minimal saving of false positive images, maximising deployment storage efficiency. High recall scores indicate a minimal false negative rat
Bees are among the master navigators of the insect world. Despite impressive advances in robot navigation research, the performance of these insects is still unrivaled by any artificial system in terms of training efficiency and generalization capabilities, particularly considering the limited computational capacity. On the other hand, computational principles underlying these extraordinary feats are still only partially understood. The theoretical framework of reinforcement learning (RL) provides an ideal focal point to bring the two fields together for mutual benefit. In particular, we analyze and compare representations of space in robot and insect navigation models through the lens of RL, as the efficiency of insect navigation is likely rooted in an efficient and robust internal representation, linking retinotopic (egocentric) visual input with the geometry of the environment. While RL has long been at the core of robot navigation research, current computational theories of insect navigation are not commonly formulated within this framework, but largely as an associative learning process implemented in the insect brain, especially in the mushroom body (MB). Here we propose spec
Stick insect stepping patterns have been studied for insights about locomotor rhythm generation and control, because the underlying neural system is relatively accessible experimentally and produces a variety of rhythmic outputs. Harnessing the experimental identification of effective interactions among neuronal units involved in stick insect stepping pattern generation, previous studies proposed computational models simulating aspects of stick insect locomotor activity. While these models generate diverse stepping patterns and transitions between them, there has not been an in-depth analysis of the mechanisms underlying their dynamics. In this study, we focus on modeling rhythm generation by the neurons associated with the protraction-retraction, levitation-depression, and extension-flexion antagonistic muscle pairs of the mesothoracic (middle) leg of stick insects. Our model features a reduced central pattern generator (CPG) circuit for each joint and includes synaptic interactions among the CPGs; we also consider extensions such as the inclusion of motoneuron pools controlled by the CPG components. The resulting network is described by an 18-dimensional system of ordinary differ
In precision agriculture, the detection and recognition of insects play an essential role in the ability of crops to grow healthy and produce a high-quality yield. The current machine vision model requires a large volume of data to achieve high performance. However, there are approximately 5.5 million different insect species in the world. None of the existing insect datasets can cover even a fraction of them due to varying geographic locations and acquisition costs. In this paper, we introduce a novel "Insect-1M" dataset, a game-changing resource poised to revolutionize insect-related foundation model training. Covering a vast spectrum of insect species, our dataset, including 1 million images with dense identification labels of taxonomy hierarchy and insect descriptions, offers a panoramic view of entomology, enabling foundation models to comprehend visual and semantic information about insects like never before. Then, to efficiently establish an Insect Foundation Model, we develop a micro-feature self-supervised learning method with a Patch-wise Relevant Attention mechanism capable of discerning the subtle differences among insect images. In addition, we introduce Description Co
Approximately half of the existing winged-insect species are of very small size (wing length about 0.3-4 mm); they are referred to as miniature insects. Yet until recently, much of what we know about the mechanics of insect flight was derived from studies on relatively large insects, such as hoverflies, honey bees and hawkmoths. Because of their very small size, many miniature insects fly at a Reynolds number (Re) on the order of 10 or less. At such a low Re, the viscous effect of the air is very large: A miniature insect moves through the air as would a bumble bee move through mineral oil. Miniature insects must use new flapping mode and new aerodynamic mechanisms to fly. Over the past decade, much work has been done in the study of the mechanics of flight in miniature insects: novel flapping modes have been discovered and new mechanisms of aerodynamic force generation have been revealed; progress has also been made on the fluid-mechanics related flight problems, such as flight power requirements and flight dynamic stability. This article reviews these developments and discusses potential future directions.
The sterile insect technique controls mosquito-borne diseases such as malaria, dengue, and yellow fever through either eradication or depressing the associated vector population. We formulate a three-dimensional delayed mosquito population suppression model with a saturated release rate to explore the interactive dynamics between wild, sterile, and non-sterile mosquitoes, focusing on the delay and residual fertility in the interactive dynamics among insects. We investigate the stability of the positive equilibrium and derive the Hopf bifurcation conditions. We establish the stability conditions for the positive equilibrium and examine how the time delay ($τ$) and residual fertility affect the non-sterile insects' dynamics. Below the critical values of the delay, the system remains stable, while beyond that, the Hopf bifurcation is guaranteed under certain circumstances. However, analysis shows a clear band of non-sterile insect population values as residual fertility varies within a very narrow range. This suggests that within this interval, the system exhibits sensitive dependence on the fertility parameter, likely due to underlying nonlinear dynamics. Numerical simulations are pr
Accurate insect pest recognition plays a critical role in agriculture. It is a challenging problem due to the intricate characteristics of insects. In this paper, we present DeWi, novel learning assistance for insect pest classification. With a one-stage and alternating training strategy, DeWi simultaneously improves several Convolutional Neural Networks in two perspectives: discrimination (by optimizing a triplet margin loss in a supervised training manner) and generalization (via data augmentation). From that, DeWi can learn discriminative and in-depth features of insect pests (deep) yet still generalize well to a large number of insect categories (wide). Experimental results show that DeWi achieves the highest performances on two insect pest classification benchmarks (76.44\% accuracy on the IP102 dataset and 99.79\% accuracy on the D0 dataset, respectively). In addition, extensive evaluations and ablation studies are conducted to thoroughly investigate our DeWi and demonstrate its superiority. Our source code is available at https://github.com/toannguyen1904/DeWi.
Insects rely on their hearing in order to communicate, identify and locate potential mates, and avoid predators. Due to their small sizes, many insect species are not able to utilize the interaural time and intensity differences employed by vertebrates for the localization of sound, but have instead evolved other mechanisms to perform this task. One such mechanism is the antenna, which provides directionally sensitive acoustic information. In the current work, we discuss the physical limitations imposed by the Gabor limit and the nature of acoustic radiation as small length scales. We then propose mechanisms that antennal insects may use in order to localize sound and extract precise frequency information from transient signals, thereby circumventing these physical limitations.
Sterile Insect Technique (SIT) is a promising control method against insect pests and insect vectors. It consists in releasing males previously sterilized in laboratory, in order to reduce or eliminate a specific wild population. We study in this paper the implementation by feedback control of SIT-based elimination campaign of Aedes mosquitoes. We provide state-feedback and output-feedback control laws and establish their convergence, as well as their robustness properties. In this design procedure, a pivotal role is played by the average number of secondary female insects produced by a single female insect, called basic offspring number, and by the use of properties of monotone systems. Illustrative simulations are provided.
Deep learning-based approaches have produced models with good insect classification accuracy; Most of these models are conducive for application in controlled environmental conditions. One of the primary emphasis of researchers is to implement identification and classification models in the real agriculture fields, which is challenging because input images that are wildly out of the distribution (e.g., images like vehicles, animals, humans, or a blurred image of an insect or insect class that is not yet trained on) can produce an incorrect insect classification. Out-of-distribution (OOD) detection algorithms provide an exciting avenue to overcome these challenge as it ensures that a model abstains from making incorrect classification prediction of non-insect and/or untrained insect class images. We generate and evaluate the performance of state-of-the-art OOD algorithms on insect detection classifiers. These algorithms represent a diversity of methods for addressing an OOD problem. Specifically, we focus on extrusive algorithms, i.e., algorithms that wrap around a well-trained classifier without the need for additional co-training. We compared three OOD detection algorithms: (i) Ma
Insects as pollinators play a crucial role in ecosystem management and world food production. However, insect populations are declining, calling for efficient methods of insect monitoring. Existing methods analyze video or time-lapse images of insects in nature, but the analysis is challenging since insects are small objects in complex and dynamic scenes of natural vegetation. In this work, we provide a dataset of primary honeybees visiting three different plant species during two months of the summer period. The dataset consists of 107,387 annotated time-lapse images from multiple cameras, including 9,423 annotated insects. We present a method pipeline for detecting insects in time-lapse RGB images. The pipeline consists of a two-step process. Firstly, the time-lapse RGB images are preprocessed to enhance insects in the images. This Motion-Informed-Enhancement technique uses motion and colors to enhance insects in images. Secondly, the enhanced images are subsequently fed into a Convolutional Neural network (CNN) object detector. The method improves the deep learning object detectors You Only Look Once (YOLO) and Faster Region-based CNN (Faster R-CNN). Using Motion-Informed-Enhanc
The question of whether insects experience pain has long been debated in neuroscience and animal behavior research. Increasing evidence suggests that insects possess the ability to detect and respond to noxious stimuli, exhibiting behaviors indicative of pain perception. This study investigates the relationship between pain stimuli and physiological responses in crickets (Gryllidae), focusing on heart rate (ECG) and brain wave (EEG) patterns. We applied a range of mechanical, chemical, thermal, and electrical stimuli to crickets, recording ECG and EEG data while employing a deep learning-based model to classify pain levels. Our findings revealed significant heart rate changes and EEG fluctuations in response to various stimuli, with the highest intensity stimuli inducing marked physiological stress. The AI-based analysis, utilizing AlexNet for EEG signal classification, achieved 90% accuracy in distinguishing between resting, low-pain, and high-pain states. While no social sharing of pain was observed through ECG measurements, these results contribute to the growing body of evidence supporting insect nociception and offer new insights into their physiological responses to external