搜索结果：Billions

The v-myb oncogene of the avian myeloblastosis virus has led to the discovery of a large and growing family of myb-related genes in a wide variety of eukaryotes including animals, plants, fungi and slime molds. The Myb-related proteins contain a highly conserved sequence, often present in multiple tandem repeats which constitute a DNA-binding domain. These proteins generally function in the regulation of cell growth and differentiation, often by coregulating gene expression along with DNA-binding proteins of other classes. This review focuses on the evolution of the myb gene family and the role of these genes in development.

Recent work in language modeling demonstrates that training large transformer models advances the state of the art in Natural Language Processing applications. However, very large models can be quite difficult to train due to memory constraints. In this work, we present our techniques for training very large transformer models and implement a simple, efficient intra-layer model parallel approach that enables training transformer models with billions of parameters. Our approach does not require a new compiler or library changes, is orthogonal and complimentary to pipeline model parallelism, and can be fully implemented with the insertion of a few communication operations in native PyTorch. We illustrate this approach by converging transformer based models up to 8.3 billion parameters using 512 GPUs. We sustain 15.1 PetaFLOPs across the entire application with 76% scaling efficiency when compared to a strong single GPU baseline that sustains 39 TeraFLOPs, which is 30% of peak FLOPs. To demonstrate that large language models can further advance the state of the art (SOTA), we train an 8.3 billion parameter transformer language model similar to GPT-2 and a 3.9 billion parameter model similar to BERT. We show that careful attention to the placement of layer normalization in BERT-like models is critical to achieving increased performance as the model size grows. Using the GPT-2 model we achieve SOTA results on the WikiText103 (10.8 compared to SOTA perplexity of 15.8) and LAMBADA (66.5% compared to SOTA accuracy of 63.2%) datasets. Our BERT model achieves SOTA results on the RACE dataset (90.9% compared to SOTA accuracy of 89.4%).

This paper presents a study of semi-supervised learning with large convolutional networks. We propose a pipeline, based on a teacher/student paradigm, that leverages a large collection of unlabelled images (up to 1 billion). Our main goal is to improve the performance for a given target architecture, like ResNet-50 or ResNext. We provide an extensive analysis of the success factors of our approach, which leads us to formulate some recommendations to produce high-accuracy models for image classification with semi-supervised learning. As a result, our approach brings important gains to standard architectures for image, video and fine-grained classification. For instance, by leveraging one billion unlabelled images, our learned vanilla ResNet-50 achieves 81.2% top-1 accuracy on the ImageNet benchmark.

Tree boosting is a highly effective and widely used machine learning method. In this paper, we describe a scalable end-to-end tree boosting system called XGBoost, which is used widely by data scientists to achieve state-of-the-art results on many machine learning challenges. We propose a novel sparsity-aware algorithm for sparse data and weighted quantile sketch for approximate tree learning. More importantly, we provide insights on cache access patterns, data compression and sharding to build a scalable tree boosting system. By combining these insights, XGBoost scales beyond billions of examples using far fewer resources than existing systems.

Abstract Proteins are essential to life, and understanding their structure can facilitate a mechanistic understanding of their function. Through an enormous experimental effort 1–4 , the structures of around 100,000 unique proteins have been determined 5 , but this represents a small fraction of the billions of known protein sequences 6,7 . Structural coverage is bottlenecked by the months to years of painstaking effort required to determine a single protein structure. Accurate computational approaches are needed to address this gap and to enable large-scale structural bioinformatics. Predicting the three-dimensional structure that a protein will adopt based solely on its amino acid sequence—the structure prediction component of the ‘protein folding problem’ 8 —has been an important open research problem for more than 50 years 9 . Despite recent progress 10–14 , existing methods fall far short of atomic accuracy, especially when no homologous structure is available. Here we provide the first computational method that can regularly predict protein structures with atomic accuracy even in cases in which no similar structure is known. We validated an entirely redesigned version of our neural network-based model, AlphaFold, in the challenging 14th Critical Assessment of protein Structure Prediction (CASP14) 15 , demonstrating accuracy competitive with experimental structures in a majority of cases and greatly outperforming other methods. Underpinning the latest version of AlphaFold is a novel machine learning approach that incorporates physical and biological knowledge about protein structure, leveraging multi-sequence alignments, into the design of the deep learning algorithm.

We propose a simple method to extract the community structure of large networks. Our method is a heuristic method that is based on modularity optimization. It is shown to outperform all other known community detection methods in terms of computation time. Moreover, the quality of the communities detected is very good, as measured by the so-called modularity. This is shown first by identifying language communities in a Belgian mobile phone network of 2 million customers and by analysing a web graph of 118 million nodes and more than one billion links. The accuracy of our algorithm is also verified on ad hoc modular networks.

Continuing population and consumption growth will mean that the global demand for food will increase for at least another 40 years. Growing competition for land, water, and energy, in addition to the overexploitation of fisheries, will affect our ability to produce food, as will the urgent requirement to reduce the impact of the food system on the environment. The effects of climate change are a further threat. But the world can produce more food and can ensure that it is used more efficiently and equitably. A multifaceted and linked global strategy is needed to ensure sustainable and equitable food security, different components of which are explored here.

Freshwater scarcity is increasingly perceived as a global systemic risk. Previous global water scarcity assessments, measuring water scarcity annually, have underestimated experienced water scarcity by failing to capture the seasonal fluctuations in water consumption and availability. We assess blue water scarcity globally at a high spatial resolution on a monthly basis. We find that two-thirds of the global population (4.0 billion people) live under conditions of severe water scarcity at least 1 month of the year. Nearly half of those people live in India and China. Half a billion people in the world face severe water scarcity all year round. Putting caps to water consumption by river basin, increasing water-use efficiencies, and better sharing of the limited freshwater resources will be key in reducing the threat posed by water scarcity on biodiversity and human welfare.

Fish provides more than 4.5 billion people with at least 15 % of their average per capita intake of animal protein. Fish’s unique nutritional properties make it also essential to the health of billions of consumers in both developed and developing countries. Fish is one of the most efficient converters of feed into high quality food and its carbon footprint is lower compared to other animal production systems. Through fish-related activities (fisheries and aquaculture but also processing and trading), fish contribute substantially to the income and therefore to the indirect food security of more than 10 % of the world population, essentially in developing and emergent countries. Yet, limited attention has been given so far to fish as a key element in food security and nutrition strategies at national level and in wider development discussions and interventions. As a result, the tremendous potential for improving food security and nutrition embodied in the strengthening of the fishery and aquaculture sectors is missed. The purpose of this paper is to make a case for a closer integration of fish into the overall debate and future policy about food security and nutrition. For this, we review the evidence from the contemporary and emerging debates and controversies around fisheries and aquaculture and we discuss them in the light of the issues debated in the wider agriculture/farming literature. The overarching question that underlies this paper is: how and to what extent will fish be able to contribute to feeding 9 billion people in 2050 and beyond?

Global poverty is falling rapidly, but in around fifty failing states, the world's poorest people face a tragedy that is growing inexorably worse. This bottom billion live on less than a dollar a day and while the rest of the world moves steadily forward, this forgotten billion is left further and further behind with potentially serious consequences not only for them but for the stability of the rest of the world. Why do the states these people live in defy all the attempts of the international aid community to help them? Why does nothing seem to make a difference? In The Bottom Billion, Paul Collier pinpoints the issues of corruption, political instability and resource management that lie at the root of the problem. He describes the battle raging in these countries between corrupt leaders and would-be reformers and the factors such as civil war, dependence on the export of natural resources and lack of good governance that trap them into a downward spiral of economic and social decline. Collier addresses the fact that conventional aid has been unable to tackle these problems and puts forward a radical new plan of action including a new agenda for the G8 which includes more effective anti-corruption measures, preferential trade policies and where necessary direct military intervention. All of these initiatives are carefully designed to help the forgotten bottom billion, one of the key challenges facing the world in the twenty first century.

Kathryn Yusoff examines how the grammar of geology is foundational to establishing the extractive economies of subjective life and the earth under colonialism and slavery. She initiates a transdisciplinary conversation between feminist black theory, geography, and the earth sciences, addressing the politics of the Anthropocene within the context of race, materiality, deep time, and the afterlives of geology.

The U.S. Department of Energy (DOE) and the U.S. Department of Agriculture (USDA) are both strongly committed to expanding the role of biomass as an energy source. In particular, they support biomass fuels and products as a way to reduce the need for oil and gas imports; to support the growth of agriculture, forestry, and rural economies; and to foster major new domestic industries--biorefineries--making a variety of fuels, chemicals, and other products. As part of this effort, the Biomass R&D Technical Advisory Committee, a panel established by the Congress to guide the future direction of federally funded biomass R&D, envisioned a 30 percent replacement of the current U.S. petroleum consumption with biofuels by 2030. Biomass--all plant and plant-derived materials including animal manure, not just starch, sugar, oil crops already used for food and energy--has great potential to provide renewable energy for America's future. Biomass recently surpassed hydropower as the largest domestic source of renewable energy and currently provides over 3 percent of the total energy consumption in the United States. In addition to the many benefits common to renewable energy, biomass is particularly attractive because it is the only current renewable source of liquid transportation fuel. This, of course, makes it invaluable in reducing oil imports--one of our most pressing energy needs. A key question, however, is how large a role could biomass play in responding to the nation's energy demands. Assuming that economic and financial policies and advances in conversion technologies make biomass fuels and products more economically viable, could the biorefinery industry be large enough to have a significant impact on energy supply and oil imports? Any and all contributions are certainly needed, but would the biomass potential be sufficiently large to justify the necessary capital replacements in the fuels and automobile sectors? The purpose of this report is to determine whether the land resources of the United States are capable of producing a sustainable supply of biomass sufficient to displace 30 percent or more of the country's present petroleum consumption--the goal set by the Advisory Committee in their vision for biomass technologies. Accomplishing this goal would require approximately 1 billion dry tons of biomass feedstock per year.

Bayesian optimization techniques have been successfully applied to robotics, planning, sensor placement, recommendation, advertising, intelligent user interfaces and automatic algorithm configuration. Despite these successes, the approach is restricted to problems of moderate dimension, and several workshops on Bayesian optimization have identified its scaling to high-dimensions as one of the holy grails of the field. In this paper, we introduce a novel random embedding idea to attack this problem. The resulting Random EMbedding Bayesian Optimization (REMBO) algorithm is very simple, has important invariance properties, and applies to domains with both categorical and continuous variables. We present a thorough theoretical analysis of REMBO. Empirical results confirm that REMBO can effectively solve problems with billions of dimensions, provided the intrinsic dimensionality is low. They also show that REMBO achieves state-of-the-art performance in optimizing the 47 discrete parameters of a popular mixed integer linear programming solver.

Abstract Stellar distances constitute a foundational pillar of astrophysics. The publication of 1.47 billion stellar parallaxes from Gaia is a major contribution to this. Despite Gaia’s precision, the majority of these stars are so distant or faint that their fractional parallax uncertainties are large, thereby precluding a simple inversion of parallax to provide a distance. Here we take a probabilistic approach to estimating stellar distances that uses a prior constructed from a three-dimensional model of our Galaxy. This model includes interstellar extinction and Gaia’s variable magnitude limit. We infer two types of distance. The first, geometric, uses the parallax with a direction-dependent prior on distance. The second, photogeometric, additionally uses the color and apparent magnitude of a star, by exploiting the fact that stars of a given color have a restricted range of probable absolute magnitudes (plus extinction). Tests on simulated data and external validations show that the photogeometric estimates generally have higher accuracy and precision for stars with poor parallaxes. We provide a catalog of 1.47 billion geometric and 1.35 billion photogeometric distances together with asymmetric uncertainty measures. Our estimates are quantiles of a posterior probability distribution, so they transform invariably and can therefore also be used directly in the distance modulus ( ). The catalog may be downloaded or queried using ADQL at various sites (see http://www.mpia.de/~calj/gedr3_distances.html ), where it can also be cross-matched with the Gaia catalog.

Abstract For the vast majority of stars in the second Gaia data release, reliable distances cannot be obtained by inverting the parallax. A correct inference procedure must instead be used to account for the nonlinearity of the transformation and the asymmetry of the resulting probability distribution. Here, we infer distances to essentially all 1.33 billion stars with parallaxes published in the second Gaia data release. This is done using a weak distance prior that varies smoothly as a function of Galactic longitude and latitude according to a Galaxy model. The irreducible uncertainty in the distance estimate is characterized by the lower and upper bounds of an asymmetric confidence interval. Although more precise distances can be estimated for a subset of the stars using additional data (such as photometry), our goal is to provide purely geometric distance estimates, independent of assumptions about the physical properties of, or interstellar extinction toward, individual stars. We analyze the characteristics of the catalog and validate it using clusters. The catalog can be queried using ADQL at http://gaia.ari.uni-heidelberg.de/tap.html (which also hosts the Gaia catalog) and downloaded from http://www.mpia.de/~calj/gdr2_distances.html .