Japan Agency for Marine-Earth Science and Technology (JAMSTEC) has made available the JAMSTEC Earth Deep-sea Image (J-EDI), a deep-sea video and image archive (https://www.godac.jamstec.go.jp/jedi/e/index.html). This archive serves as a valuable resource for researchers and scholars interested in deep-sea imagery. The dataset comprises images and videos of deep-sea phenomena, predominantly of marine organisms, but also of the seafloor and physical processes. In this study, we propose J-EDI QA, a benchmark for understanding images of deep-sea organisms using a multimodal large language model (LLM). The benchmark is comprised of 100 images, accompanied by questions and answers with four options by JAMSTEC researchers for each image. The QA pairs are provided in Japanese, and the benchmark assesses the ability to understand deep-sea species in Japanese. In the evaluation presented in this paper, OpenAI o1 achieved a 50% correct response rate. This result indicates that even with the capabilities of state-of-the-art models as of December 2024, deep-sea species comprehension is not yet at an expert level. Further advances in deep-sea species-specific LLMs are therefore required.
Deep-sea cold seep stage assessment has traditionally relied on costly, high-risk manned submersible operations and visual surveys of macrofauna. Although microbial communities provide a promising and more cost-effective alternative, reliable inference remains challenging because the available deep-sea dataset is extremely small ($n = 13$) relative to the microbial feature dimension ($p = 26$), making purely data-driven models highly prone to overfitting. To address this, we propose a knowledge-enhanced classification framework that incorporates an ecological knowledge graph as a structural prior. By fusing macro-microbe coupling and microbial co-occurrence patterns, the framework internalizes established ecological logic into a \underline{\textbf{G}}raph-\underline{\textbf{R}}egularized \underline{\textbf{M}}ultinomial \underline{\textbf{L}}ogistic \underline{\textbf{R}}egression (GRMLR) model, effectively constraining the feature space through a manifold penalty to ensure biologically consistent classification. Importantly, the framework removes the need for macrofauna observations at inference time: macro-microbe associations are used only to guide training, whereas prediction r
It may be important to precisely know heights of moored oceanographic instrumentation. For example, moorings can be closely spaced or accidentally be located on small rocks or in small gullies. Height variations O(1 m) will yield registration of different values when conditions such as small-scale density stratification vary strongly. Such little height variations may prove difficult to measure in the deep sea, requiring high-accuracy pressure sensors preferably on all instruments in a mooring-array. In this paper, an alternative method for relative height determination is presented using high-resolution temperature sensors moored on multiple densely-spaced lines in the deep Western Mediterranean. While it was anticipated that height variations between lines could be detected under near-homogeneous conditions via adiabatic lapse rate O(0.0001degrC m-1) by the 0.00003degrC-noise-level sensors, such was prevented by the impossibility of properly correcting for short-term bias due to electronic drift. Instead, a satisfactory height determination was found during a period of relatively strong stratification and large turbulence activity. By band-pass filtering data of the highest-resol
Benchmarking multi-object tracking and object detection model performance is an essential step in machine learning model development, as it allows researchers to evaluate model detection and tracker performance on human-generated 'test' data, facilitating consistent comparisons between models and trackers and aiding performance optimization. In this study, a novel benchmark video dataset was developed and used to assess the performance of several Monterey Bay Aquarium Research Institute object detection models and a FathomNet single-class object detection model together with several trackers. The dataset consists of four video sequences representing midwater and benthic deep-sea habitats. Performance was evaluated using Higher Order Tracking Accuracy, a metric that balances detection, localization, and association accuracy. To the best of our knowledge, this is the first publicly available benchmark for multi-object tracking in deep-sea video footage. We provide the benchmark data, a clearly documented workflow for generating additional benchmark videos, as well as example Python notebooks for computing metrics.
In deep-sea exploration and surgical robotics scenarios, environmental lighting and device resolution limitations often cause high-frequency feature attenuation. Addressing the differences in frequency band sensitivity between CNNs and the human visual system (mid-frequency sensitivity with low-frequency sensitivity surpassing high-frequency), we experimentally quantified the CNN contrast sensitivity function and proposed a wavelet adaptive spectrum fusion (WASF) method inspired by biological vision mechanisms to balance cross-frequency image features. Furthermore, we designed a perception frequency block (PFB) that integrates WASF to enhance frequency-domain feature extraction. Based on this, we developed the FE-UNet model, which employs a SAM2 backbone network and incorporates fine-tuned Hiera-Large modules to ensure segmentation accuracy while improving generalization capability. Experiments demonstrate that FE-UNet achieves state-of-the-art performance in cross-domain tasks such as marine organism segmentation and polyp segmentation, showcasing robust adaptability and significant application potential. The code will be released soon.
To address non-linear disturbances and uncertainties in complex marine environments, this paper proposes a disturbance-resistant controller for deep-sea cranes. The controller integrates hierarchical sliding mode control, adaptive control, and neural network compensation techniques. By designing a global sliding mode surface, the dynamic coordination between the driving and non-driving subsystems is achieved, ensuring overall system stability. The subsystem surfaces reduce oscillations and enhance tracking accuracy. Adaptive control dynamically adjusts system parameters, enhancing robustness against external uncertainties, while the neural network compensates for time-varying disturbances through real-time learning. The stability of the control scheme is verified on the basis of Lyapunov theory. The simulation results demonstrate that, compared to traditional PID control, the proposed controller exhibits significant advantages in trajectory tracking accuracy, response speed, and disturbance rejection.
As terrestrial resources become increasingly depleted, the demand for deep-sea resource exploration has intensified. However, the extreme conditions in the deep-sea environment pose significant challenges for underwater operations, necessitating the development of robust detection robots. In this paper, we propose an advanced path planning methodology that integrates an improved A* algorithm with the Dynamic Window Approach (DWA). By optimizing the search direction of the traditional A* algorithm and introducing an enhanced evaluation function, our improved A* algorithm accelerates path searching and reduces computational load. Additionally, the path-smoothing process has been refined to improve continuity and smoothness, minimizing sharp turns. This method also integrates global path planning with local dynamic obstacle avoidance via DWA, improving the real-time response of underwater robots in dynamic environments. Simulation results demonstrate that our proposed method surpasses the traditional A* algorithm in terms of path smoothness, obstacle avoidance, and real-time performance. The robustness of this approach in complex environments with both static and dynamic obstacles hig
Continuous and reliable underwater monitoring is essential for assessing marine biodiversity, detecting ecological changes and supporting autonomous exploration in aquatic environments. Underwater monitoring platforms rely on mainly visual data for marine biodiversity analysis, ecological assessment and autonomous exploration. However, underwater environments present significant challenges due to light scattering, absorption and turbidity, which degrade image clarity and distort colour information, which makes accurate observation difficult. To address these challenges, we propose DEEP-SEA, a novel deep learning-based underwater image restoration model to enhance both low- and high-frequency information while preserving spatial structures. The proposed Dual-Frequency Enhanced Self-Attention Spatial and Frequency Modulator aims to adaptively refine feature representations in frequency domains and simultaneously spatial information for better structural preservation. Our comprehensive experiments on EUVP and LSUI datasets demonstrate the superiority over the state of the art in restoring fine-grained image detail and structural consistency. By effectively mitigating underwater visual
The performance of underwater crawling and adjustment of the body posture for underwater manipulating of the deep-sea crawling and swimming robot (DCSR) is directly influenced by the dynamic performance of the underwater mechanical legs (UWML), as it serves as the executive mechanism of the DCSR. Compared with the mechanical legs of legged robots working on land, the UWML of the DCSR not only possesses the characteristics of the land used mechanical legs, but is also affected by the influence of the deep-sea underwater working environment (i.e., the hydrodynamic force, viscous resistance and dynamic seal resistance). To reduce these influence, firstly, the hydrodynamic force of the UWML were researched based on theory and experiment, and the hydrodynamic model was established with the fitted hydrodynamic parameters. Secondly, the oil viscous resistance and the dynamic seal resistance were studied experimentally, and the change laws of both with respect to the joint speed and the ambient pressure (depth of operation) were obtained. The results provide a basis for the subsequent research on the structure optimization and high performance control of the UWML and DCSR.
While modelling the galactic chemical evolution (GCE) of stable elements provides insights to the formation history of the Galaxy and the relative contributions of nucleosynthesis sites, modelling the evolution of short-lived radioisotopes (SLRs) can provide supplementary timing information on recent nucleosynthesis. To study the evolution of SLRs, we need to understand their spatial distribution. Using a 3-dimensional GCE model, we investigated the evolution of four SLRs: Mn-53, Fe-60, Hf-182, and Pu-244 with the aim of explaining detections of recent (within the last $\approx$1-20 Myr) deposition of live Mn-53, Fe-60, and Pu-244 of extrasolar origin into deep-sea reservoirs. We find that core-collapse supernovae (CCSNe) are the dominant propagation mechanism of SLRs in the Galaxy. This results in the simultaneously arrival of these four SLRs on Earth, although they could have been produced in different astrophysical sites, which can explain why live extrasolar Mn-53, Fe-60, and Pu-244 are found within the same, or similar, layers of deep-sea sediments. We predict that Hf-182 should also be found in such sediments at similar depths.
Since its discovery, the deep-sea glass sponge Euplectella aspergillum has attracted interest in its mechanical properties and beauty. Its skeletal system is composed of amorphous hydrated silica and is arranged in a highly regular and hierarchical cylindrical lattice that begets exceptional flexibility and resilience to damage. Structural analyses dominate the literature, but hydrodynamic fields that surround and penetrate the sponge have remained largely unexplored. Here we address an unanswered question: whether, besides improving its mechanical properties, the skeletal motifs of E. aspergillum underlie the optimization of the flow physics within and beyond its body cavity. We use extreme flow simulations based on the 'lattice Boltzmann' method, featuring over fifty billion grid points and spanning four spatial decades. These in silico experiments reproduce the hydrodynamic conditions on the deep-sea floor where E. aspergillum lives. Our results indicate that the skeletal motifs reduce the overall hydrodynamic stress and support coherent internal recirculation patterns at low flow velocity. These patterns are arguably beneficial to the organism for selective filter feeding and s
Visual localization plays an important role in the positioning and navigation of robotics systems within previously visited environments. When visits occur over long periods of time, changes in the environment related to seasons or day-night cycles present a major challenge. Under water, the sources of variability are due to other factors such as water conditions or growth of marine organisms. Yet it remains a major obstacle and a much less studied one, partly due to the lack of data. This paper presents a new deep-sea dataset to benchmark underwater long-term visual localization. The dataset is composed of images from four visits to the same hydrothermal vent edifice over the course of five years. Camera poses and a common geometry of the scene were estimated using navigation data and Structure-from-Motion. This serves as a reference when evaluating visual localization techniques. An analysis of the data provides insights about the major changes observed throughout the years. Furthermore, several well-established visual localization methods are evaluated on the dataset, showing there is still room for improvement in underwater long-term visual localization. The data is made public
The performance of a large-scale water Cherenkov neutrino telescope relies heavily on the transparency of the surrounding water, quantified by its level of light absorption and scattering. A pathfinder experiment was carried out to measure the optical properties of deep seawater in South China Sea with light-emitting diodes (LEDs) as light sources, photon multiplier tubes (PMTs) and cameras as photon sensors. Here, we present an optical simulation program employing the Geant4 toolkit to understand the absorption and scattering processes in the deep seawater, which helps to extract the underlying optical properties from the experimental data. The simulation results are compared with the experimental data and show good agreements. We also verify the analysis methods that utilize various observables of the PMTs and the cameras with this simulation program, which can be easily adapted by other neutrino telescope pathfinder experiments and future large-scale detectors.
Increasing interest in the acquisition of biotic and abiotic resources from within the deep sea (e.g. fisheries, oil-gas extraction, and mining) urgently imposes the development of novel monitoring technologies, beyond the traditional vessel-assisted, time-consuming, high-cost sampling surveys. The implementation of permanent networks of seabed and water-column cabled (fixed) and docked mobile platforms is presently enforced, to cooperatively measure biological features and environmental (physico-chemical) parameters. Video and acoustic (i.e. optoacoustic) imaging are becoming central approaches for studying benthic fauna (e.g. quantifying species presence, behaviour, and trophic interactions) in a remote, continuous, and prolonged fashion. Imaging is also being complemented by in situ environmental-DNA sequencing technologies, allowing the traceability of a wide range of organisms (including prokaryotes) beyond the reach of optoacoustic tools. Here, we describe the different fixed and mobile platforms of those benthic and pelagic monitoring networks, proposing at the same time an innovative roadmap for the automated computing of hierarchical ecological information of deep-sea ecos
A pressure sensor, located for 4 months in the middle of a 1275-m long taut deep-sea mooring in 2380 m water-depth above a seamount with sub-surface top-buoys and seafloor anchor-weight, demonstrates deterministic spectral peaks at sub-harmonics of the local near-inertial frequency. None of these frequencies can be associated with oceanographic motions. No corresponding peaks are found in spectra of other observables like water-flow (differences), temperature, and pressure in the top-buoys of the mooring. The mid-cable pressure sensor was mounted on a nearly 1-kN weighing non-swiveled frame. Its data seem to reflect a resonant mechanical oscillation of the tensioned mooring cable under repeated short-scale Strouhal vibrations induced by water-flow. Equivalent vertical motions of 0.5-m amplitudes are found dominant at frequency f*/4, with monotonically decreasing peaks at f*/2, 3f*/4 and f* = 1.083f, where f denotes the local inertial frequency of Earth rotation. The observations provide, physically, proof of the Earth rotation and, oceanographically, of the relative vortex-rotation dzeta = 0.083f induced by (sub-)mesoscale eddies associated with a nearby seamount, so that effective
There is a wealth of data on live, undecayed 60Fe ($t_{1/2} = 2.6 \ \rm Myr$) in deep-sea deposits, the lunar regolith, cosmic rays, and Antarctic snow, which is interpreted as originating from the recent explosions of at least two near-Earth supernovae. We use the 60Fe profiles in deep-sea sediments to estimate the timescale of supernova debris deposition beginning $\sim 3$ Myr ago. The available data admits a variety of different profile functions, but in all cases the best-fit 60Fe pulse durations are $>1.6$ Myr when all the data is combined. This timescale far exceeds the $\lesssim 0.1$ Myr pulse that would be expected if 60Fe was entrained in the supernova blast wave plasma. We interpret the long signal duration as evidence that 60Fe arrives in the form of supernova dust, whose dynamics are separate from but coupled to the evolution of the blast plasma. In this framework, the $>1.6$ Myr is that for dust stopping due to drag forces. This scenario is consistent with the simulations in Fry et. al (2020), where the dust is magnetically trapped in supernova remnants and thereby confined around regions of the remnant dominated by supernova ejects, where magnetic fields are low
Each year, underwater remotely operated vehicles (ROVs) collect thousands of hours of video of unexplored ocean habitats revealing a plethora of information regarding biodiversity on Earth. However, fully utilizing this information remains a challenge as proper annotations and analysis require trained scientists time, which is both limited and costly. To this end, we present a Dataset for Underwater Substrate and Invertebrate Analysis (DUSIA), a benchmark suite and growing large-scale dataset to train, validate, and test methods for temporally localizing four underwater substrates as well as temporally and spatially localizing 59 underwater invertebrate species. DUSIA currently includes over ten hours of footage across 25 videos captured in 1080p at 30 fps by an ROV following pre planned transects across the ocean floor near the Channel Islands of California. Each video includes annotations indicating the start and end times of substrates across the video in addition to counts of species of interest. Some frames are annotated with precise bounding box locations for invertebrate species of interest, as seen in Figure 1. To our knowledge, DUSIA is the first dataset of its kind for de
Arctic sea ice plays a critical role in regulating Earth's climate system, significantly influencing polar ecological stability and human activities in coastal regions. Recent advances in artificial intelligence have facilitated the development of skillful pan-Arctic sea ice forecasting systems, where data-driven approaches showcase tremendous potential to outperform conventional physics-based numerical models in terms of accuracy, computational efficiency and forecasting lead times. Despite the latest progress made by deep learning (DL) forecasting models, most of their skillful forecasting lead times are confined to daily subseasonal scale and monthly averaged values for up to six months, which drastically hinders their deployment for real-world applications, e.g., maritime routine planning for Arctic transportation and scientific investigation. Extending daily forecasts from subseasonal to seasonal (S2S) scale is scientifically crucial for operational applications. To bridge the gap between the forecasting lead time of current DL models and the significant daily S2S scale, we introduce IceBench-S2S, the first comprehensive benchmark for evaluating DL approaches in mitigating the
Turbulence is indispensable to redistribute nutrients for all life forms larger than microbial, on land and in the ocean. Yet, the development of deep-sea turbulence was not studied in three dimensions to date. As a disproportionate laboratory, an array of nearly 3000 high-resolution temperature sensors had been installed for three years on the flat 2500-m deep bottom of the Mediterranean Sea. The time series from the half-cubic hectometer mooring-array allows for the creation of unique movies of deep-sea water motions. Although temperature differences are typically 0.001degrC, variable convection-turbulence is observed as expected from geothermal heating through the flat seafloor. During about 40% of the time, an additional turbulence, 3 times stronger in magnitude, is observed from slantwise advected warmer waters to pass in turbulent clouds. Besides turbulent clouds and seafloor heating, movies also reveal weakly turbulent interfacial-wave breakdown that commonly occurs in the open ocean far away from boundaries.
Atmospheric muons are important probes for studying primary cosmic rays and extensive air showers. Additionally, they constitute a significant background for many underground and deep-sea neutrino experiments, such as TRopIcal DEep-sea Neutrino Telescope (TRIDENT). Understanding the muon flux at various depths in the deep sea is essential for validating TRIDENT simulations and guiding the development of optimized trigger strategies. This paper introduces a novel device based on plastic scintillalors and silicon photomultipliers (SiPMs) named MuonSLab, which is designed to measure muon flux in the deep sea and has the potential to be extended to other atmospheric muon property measurements. We discuss the design and instrumentation of MuonSLab and present results from several muon flux measurements, demonstrating its sensitivity to muon detection and its stability during operations across multiple locations.