共找到 8 条结果
Implementing target trial emulation (TTE) studies as standardized, reproducible analytic workflows is technically demanding. We developed Text-guided Health-study Estimation and Specification Engine Using Strategus (THESEUS), which uses large language models (LLMs) to translate free-text study descriptions into structured analytic specifications and Strategus R scripts within the Observational Health Data Sciences and Informatics (OHDSI) ecosystem. THESEUS executes 2 steps: an LLM maps study descriptions to a JavaScript Object Notation (JSON) schema, and validated specifications are converted into Strategus R scripts through rule-based logic. For standardization evaluation, we compared specifications generated by 8 LLMs using 15 OHDSI-based TTE studies and 15 non-OHDSI studies under primary-analysis and full-analyses settings. Under the primary-analysis setting, overall standardization accuracy ranged from 0.93 to 0.97 across models in OHDSI studies and from 0.82 to 0.95 in non-OHDSI studies. Gemini-3.1-Pro achieved the highest overall accuracy in OHDSI studies, while Gemini-3.1-Pro and Gpt-5.5 jointly achieved the highest overall accuracy in non-OHDSI studies. Under the full-analyses setting, field-level sensitivity ranged from 0.83 to 0.97 in OHDSI studies, with 0.07-0.80 false positives (FPs) per study, and from 0.77 to 0.89 in non-OHDSI studies, with 0.53-1.20 FPs per study. Gpt-5.5 performed best at the field level. THESEUS was implemented as a web application and coding-agent tools. Pairing a standardized data model with a structured analysis framework enables reliable LLM-assisted interpretation of study descriptions and deterministic workflow construction in observational research. THESEUS supports translation of natural language study descriptions into executable, shareable code in standardized observational research settings.
Coding agents powered by large language models (LLMs) have gained traction for automating code generation through iterative problem-solving with minimal human involvement. Despite the emergence of various frameworks, e.g., LangChain, AutoML, and AIDE, ML scientists still struggle to effectively review and adjust the agents' coding process. The current approach of manually inspecting individual outputs is inefficient, making it difficult to track code evolution, compare coding iterations, and identify improvement opportunities. To address this challenge, we introduce a visual analytics system designed to enhance the examination of coding agent behaviors. Focusing on the AIDE framework, our system supports comparative analysis across three levels: (1) Code-Level Analysis, which reveals how the agent debugs and refines its code over iterations; (2) Process-Level Analysis, which contrasts different solution-seeking processes explored by the agent; and (3) LLM-Level Analysis, which highlights variations in coding behavior across different LLMs. By integrating these perspectives, our system enables ML scientists to gain a structured understanding of agent behaviors, facilitating more effective debugging and prompt engineering. Through case studies using coding agents to tackle popular Kaggle competitions, we demonstrate how our system provides valuable insights into the iterative coding process.
Effective management of Water Distribution Networks (WDNs) is essential to ensure efficient and reliable water supply in cities. However, many management tasks require complex system modelling and optimization approaches, which heavily rely on specialized domain expertise and human resources. Recent advancements in Large Language Models (LLMs) offer promising opportunities to automate complex hydraulic decision-making tasks. This study presents an LLM-based agent framework to automate WDN management tasks. Two tasks are considered to evaluate the feasibility and limitations of LLM agents: hydraulic model calibration and pump operation optimization. The key component of the proposed framework is an Orchestrating Agent that interprets tasks and system states, generates update strategies or executable code, and interacts with three specialized agents to carry out implementation: a Knowledge Agent performing reasoning based on hydraulic principles, a Modelling Agent that interfaces with hydraulic simulation tool EPANET, and a Coding Agent that executes code and returns output feedback. To assess the capabilities of these agents, the framework was systematically tested on two benchmark WDNs - Net2 and Anytown. The results indicate that the reasoning capability demonstrated through interaction with the Knowledge Agent effectively replicates expert-level hydraulic thinking, though it lacks numerical precision. In contrast, the Modelling Agent, which integrates external simulation tools, enhances reliability, although interpreting and enforcing numerical constraints expressed in natural language remain challenging, particularly in looped networks such as Anytown where the agent often converged to suboptimal solutions. Furthermore, the Coding Agent, where code for optimization algorithms is iteratively generated and executed, delivers the most consistent and accurate performance across both networks, underscoring its practical potential. These findings highlight the potential of LLM-based agents for automated, accurate hydraulic optimization, and represent a significant step toward LLM-driven multi-agent frameworks for hydraulic decision-making. This work establishes a foundation for future advancements in specialized, domain-focused LLM applications in complex hydraulic management scenarios.
The rapid adoption of multi-agent frameworks for automated code generation has significantly enhanced software development efficiency, yet simultaneously introduced critical security challenges that remain largely unexplored. While extensive research has investigated jailbreaking vulnerabilities in single-agent large language models, existing studies have overlooked the unique security risks arising from collaborative dynamics in multi-agent systems, where distributed decision-making and social interactions may amplify rather than mitigate adversarial threats. To address this gap, we propose the first comprehensive security assessment framework for multi-agent code generation, introducing Implicit Multi-agent Attack (IMA), a novel jailbreaking strategy that exploits social engineering and collaborative reinforcement within agent networks. Our evaluation encompasses four prominent frameworks (MetaGPT, CrewAI, AutoGen, and ChatDev) using the established RMCBench benchmark (Resistance to Malicious Code Benchmark) with 282 malicious code generation tasks across text-to-code, function-level, and block-level completion scenarios. Compared to traditional explicit attacks and single-agent baselines, IMA demonstrates substantially higher effectiveness, achieving an average attack success rate of 89.01% and revealing collaborative harm amplification factors up to 114.9%. The results expose fundamental vulnerabilities in current multi-agent architectures, with defense mechanisms showing alarmingly low detection rates below 30%. Crucially, by isolating the final coding agent via a direct jailbreak baseline (SADJ), we demonstrate that multi-agent collaboration itself amplifies attack success by an average of 11.4% (CADA), confirming that the vulnerability lies in the architecture rather than the underlying model alone.
Large language models (LLMs) show promise for clinical decision support but often struggle with case-specific reasoning. We present Ophtimus-V2-Tx, an 8-billion-parameter ophthalmology-specialized LLM fine-tuned on more than 10,000 case reports. Evaluation is conducted on a pre-collected dataset. Alongside text metrics (ROUGE-L, BLEU, METEOR) and a semantic similarity score, we use CliBench to map outputs to standardized codes (ICD-10-CM, ATC, ICD-10-PCS) and compute hierarchical F1 (L1-L4 and Full), with code mapping used strictly as an evaluation tool. Ophtimus-V2-Tx is competitive with a state-of-the-art general model and stronger in several settings. It improves text metrics (ROUGE-L 0.40 vs. 0.18; BLEU 0.26 vs. 0.05; METEOR 0.45 vs. 0.29) with comparable semantic similarity. On CliBench, it attains a higher full-code score for secondary diagnosis and ties or leads at selected granular levels for primary diagnosis, while medication and procedure results are close with overlapping confidence intervals. Relative to other ophthalmology-tuned baselines, it shows consistently higher text-generation scores. These findings indicate that a compact, domain-adapted model can approach-or in targeted settings, exceed-large general LLMs on clinically grounded outputs while remaining feasible for on-premise use. We also describe an auditable evaluation pipeline (frozen coding agent, identical prompts, hierarchical metrics) to support reproducibility and future benchmarking.
Humans have believed in gods and spirits since the earliest days of the Holocene, and many people still believe in them today. Although the existence of religious belief has been a human constant, the nature and prevalence of religion has changed dramatically throughout human history. Here we describe the emerging science of religious change. We first outline a multilevel framework for studying religious change drawn from theories of socioecological psychology and cultural evolution. We illustrate this framework with four case studies featuring two ancient religious changes (the rise of punitive religions and doctrinal rituals) and two modern religious changes (the rise of atheism and nontraditional religions). We then review useful methods for examining religious change, including ethnographic coding, agent-based modeling, and time-series analysis. Next, we explore future directions, highlighting the need for predictive forecasts, nonlinear models, and non-Western samples. We also outline ten key questions that need to be answered for a fuller understanding of religious change. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
This paper focuses on identifying structural features responsible for resolution of heavy isotope coded peptides during reversed-phase chromatography. This was achieved by using labeled coding agents that varied in structure, number of deuterium atoms, placement of deuterium in the coding agent, and the functional group targeted by the reagent. Six coding agents were examined. Deuterated versions of the coding agents studied included succinic anhydride-2H4, acetic acid 2,5-dioxopyrrolidin-1-yl ester-2H3, propionic acid 2,5-dioxopyrrolidin-1-yl ester-2H5, pentanoic acid 2,5-dioxopyrrolidin-1-yl ester-2H9, [3-(2,5-dioxopyrrolidin-1-yloxycarbonyl)-propyl]-trimethylammonium chloride-2H9, and the commercial ICAT-2H8 reagent. It was found that these labeling agents vary widely in both their absolute and relative contribution to the chromatographic isotope effect. Relative effects were evaluated by normalizing resolution for the number of deuterium atoms in the derivatized peptide. The single, most dominant effect was the placement of deuterium atoms relative to hydrophilic functional groups in the coding agent. It was concluded that the probability of a deuterium atom interacting with the stationary phase of a reversed-phase chromatography (RPC) column and impacting resolution is greatly diminished by placing it adjacent to a hydrophilic group, as explained by solvophobic theory. But peptide size and coding agent size were also seen to correlate inversely with the magnitude of the isotope effect. This effect was explained as being due to the relative size of the coding agent versus that of the coding agent-peptide conjugate.
This paper reports studies comparing the relative degree of sialylation among human serum glycoproteins carrying complex biantennary N-linked, hybrid, and high-mannose oligosaccharides. Comparisons were made by coupling lectin affinity selection with stable isotope coding of peptides from tryptic digests of serum. After proteolysis, samples were split and differentially acetylated with stable isotope coding agents according to either origin or the separation method by which they would be fractionated. A lectin column prepared from Sambucus nigra agglutinin (SNA) was used to select and compare the concentration of sialic acid containing glycopeptides. The relative standard deviation in quantification using this method was 4%. Using this method the concentration of sialic acid containing glycoproteins from a normal individual were compared to those in a pooled serum sample from a large number of normal individuals. It was found that sialylation varied less than 2-fold in all but four or five glycoproteins. Further studies were done on the degree of sialylation within glycoproteins. Samples labeled with the light isoform of the coding agent were applied to a set of serial lectin columns consisting of a concanavalin A (Con A) column coupled to an SNA column for selecting sialic acid appended to glycopeptides with complex biantennary N-linked, hybrid, and high-mannose glycans. In contrast, samples labeled with the heavy isoform of the coding agent were applied to a Con A lectin column alone to select glycopeptides containing complex biantennary N-linked, hybrid, and high-mannose glycans, without regard to sialylation. Glycopeptides thus selected were mixed, deglycosylated by PNGase F, and fractionated by reversed-phase chromatography (RPC). The RPC fractions were then analyzed by ESI-MS. The relative standard deviation of the method was 4%. All glycopeptides identified contained sialic acid except one. Peptides in which the relative abundance of isotopic isoforms was equal were considered to indicate that the protein parent was fully sialylated at that specific glycosylation site.