As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or inference budgets remains challenging. In this work, we propose a method to pre-train a smaller general-purpose language representation model, called DistilBERT, which can then be fine-tuned with good performances on a wide range of tasks like its larger counterparts. While most prior work investigated the use of distillation for building task-specific models, we leverage knowledge distillation during the pre-training phase and show that it is possible to reduce the size of a BERT model by 40%, while retaining 97% of its language understanding capabilities and being 60% faster. To leverage the inductive biases learned by larger models during pre-training, we introduce a triple loss combining language modeling, distillation and cosine-distance losses. Our smaller, faster and lighter model is cheaper to pre-train and we demonstrate its capabilities for on-device computations in a proof-of-concept experiment and a comparative on-device study.
We present Rsubread, a Bioconductor software package that provides high-performance alignment and read counting functions for RNA-seq reads. Rsubread is based on the successful Subread suite with the added ease-of-use of the R programming environment, creating a matrix of read counts directly as an R object ready for downstream analysis. It integrates read mapping and quantification in a single package and has no software dependencies other than R itself. We demonstrate Rsubread's ability to detect exon-exon junctions de novo and to quantify expression at the level of either genes, exons or exon junctions. The resulting read counts can be input directly into a wide range of downstream statistical analyses using other Bioconductor packages. Using SEQC data and simulations, we compare Rsubread to TopHat2, STAR and HTSeq as well as to counting functions in the Bioconductor infrastructure packages. We consider the performance of these tools on the combined quantification task starting from raw sequence reads through to summary counts, and in particular evaluate the performance of different combinations of alignment and counting algorithms. We show that Rsubread is faster and uses less memory than competitor tools and produces read count summaries that more accurately correlate with true values.
In their important work on international comparisons of national incomes and of comparative price structure, Kravis, Heston and Summers (i982, p. 8) have noted that 'services are much cheaper in the relative price structure of a typical poor country than in that of a rich country'. This phenonomen has been documented now fairly systematically by the data, gathered under their guiding hand, of the United Nations International Comparison (ICP) which covers 34 countries. Table I reproduced from their work (I982), and Fig. I based on rows 3 and II-13, indeed show this tendencyfor the relationship between relative service prices and real per capita GDP in this Kravis-Heston-Summers 34-country 6-group sample. The tendency is strongly evident except for the intermediate groups III and IV.
暂无摘要（点击查看原文获取完整内容）
暂无摘要（点击查看原文获取完整内容）
Light-weight convolutional neural networks (CNNs) are specially designed for applications on mobile devices with faster inference speed. The convolutional operation can only capture local information in a window region, which prevents performance from being further improved. Introducing self-attention into convolution can capture global information well, but it will largely encumber the actual speed. In this paper, we propose a hardware-friendly attention mechanism (dubbed DFC attention) and then present a new GhostNetV2 architecture for mobile applications. The proposed DFC attention is constructed based on fully-connected layers, which can not only execute fast on common hardware but also capture the dependence between long-range pixels. We further revisit the expressiveness bottleneck in previous GhostNet and propose to enhance expanded features produced by cheap operations with DFC attention, so that a GhostNetV2 block can aggregate local and long-range information simultaneously. Extensive experiments demonstrate the superiority of GhostNetV2 over existing architectures. For example, it achieves 75.3% top-1 accuracy on ImageNet with 167M FLOPs, significantly suppressing GhostNetV1 (74.5%) with a similar computational cost. The source code will be available at https://github.com/huawei-noah/Efficient-AI-Backbones/tree/master/ghostnetv2_pytorch and https://gitee.com/mindspore/models/tree/master/research/cv/ghostnetv2.
A significant fraction of software failures in large-scale Internet systems are cured by rebooting, even when the exact failure causes are unknown. However, rebooting can be expensive, causing nontrivial service disruption or downtime even when clusters and failover are employed. In this work we separate process recovery from data recovery to enable microrebooting -- a fine-grain technique for surgically recovering faulty application components, without disturbing the rest of the application. We evaluate microrebooting in an Internet auction system running on an application server. Microreboots recover most of the same failures as full reboots, but do so an order of magnitude faster and result in an order of magnitude savings in lost work. This cheap form of recovery engenders a new approach to high availability: microreboots can be employed at the slightest hint of failure, prior to node failover in multi-node clusters, even when mistakes in failure detection are likely; failure and recovery can be masked from end users through transparent call-level retries; and systems can be rejuvenated by parts, without ever being shut down.
Human linguistic annotation is crucial for many natural language processing tasks but can be expensive and time-consuming. We explore the use of Amazon's Mechanical Turk system, a significantly cheaper and faster method for collecting annotations from a broad base of paid non-expert contributors over the Web. We investigate five tasks: affect recognition, word similarity, recognizing textual entailment, event temporal ordering, and word sense disambiguation. For all five, we show high agreement between Mechanical Turk non-expert annotations and existing gold standard labels provided by expert labelers. For the task of affect recognition, we also show that using non-expert labels for training machine learning algorithms can be as effective as using gold standard annotations from experts. We propose a technique for bias correction that significantly improves annotation quality on two tasks. We conclude that many large labeling tasks can be effectively designed and carried out in this method at a fraction of the usual expense.
Deploying convolutional neural networks (CNNs) on embedded devices is difficult due to the limited memory and computation resources. The redundancy in feature maps is an important characteristic of those successful CNNs, but has rarely been investigated in neural architecture design. This paper proposes a novel Ghost module to generate more feature maps from cheap operations. Based on a set of intrinsic feature maps, we apply a series of linear transformations with cheap cost to generate many ghost feature maps that could fully reveal information underlying intrinsic features. The proposed Ghost module can be taken as a plug-and-play component to upgrade existing convolutional neural networks. Ghost bottlenecks are designed to stack Ghost modules, and then the lightweight GhostNet can be easily established. Experiments conducted on benchmarks demonstrate that the proposed Ghost module is an impressive alternative of convolution layers in baseline models, and our GhostNet can achieve higher recognition performance (e.g. 75.7% top-1 accuracy) than MobileNetV3 with similar computational cost on the ImageNet ILSVRC-2012 classification dataset. Code is available at https://github.com/huawei-noah/ghostnet.
BACKGROUND: A hypothetical ideotype is presented to optimize water and N acquisition by maize root systems. The overall premise is that soil resource acquisition is optimized by the coincidence of root foraging and resource availability in time and space. Since water and nitrate enter deeper soil strata over time and are initially depleted in surface soil strata, root systems with rapid exploitation of deep soil would optimize water and N capture in most maize production environments. • THE IDEOTYPE: Specific phenes that may contribute to rooting depth in maize include (a) a large diameter primary root with few but long laterals and tolerance of cold soil temperatures, (b) many seminal roots with shallow growth angles, small diameter, many laterals, and long root hairs, or as an alternative, an intermediate number of seminal roots with steep growth angles, large diameter, and few laterals coupled with abundant lateral branching of the initial crown roots, (c) an intermediate number of crown roots with steep growth angles, and few but long laterals, (d) one whorl of brace roots of high occupancy, having a growth angle that is slightly shallower than the growth angle for crown roots, with few but long laterals, (e) low cortical respiratory burden created by abundant cortical aerenchyma, large cortical cell size, an optimal number of cells per cortical file, and accelerated cortical senescence, (f) unresponsiveness of lateral branching to localized resource availability, and (g) low K(m) and high Vmax for nitrate uptake. Some elements of this ideotype have experimental support, others are hypothetical. Despite differences in N distribution between low-input and commercial maize production, this ideotype is applicable to low-input systems because of the importance of deep rooting for water acquisition. Many features of this ideotype are relevant to other cereal root systems and more generally to root systems of dicotyledonous crops.
Unbiased Value Estimates for Environmental Goods: A Cheap Talk Design for the Contingent Valuation Method by Ronald G. Cummings and Laura O. Taylor. Published in volume 89, issue 3, pages 649-665 of American Economic Review, June 1999
Although the classic Saussurean conception of language segregates the linguistic sign from the material world, this paper shows linguistic phenomena playing many roles in political economy. Linguistic signs may refer to aspects of an exchange system; differentiated ways of speaking may index social groups in a social division of labor; and linguistic “goods” may enter the marketplace as objects of exchange. These aspects of language are not mutually exclusive, but (instead) may coincide in the same stretch of discourse. Illustrations are drawn primarily from a rural Wolof community in Senegal. It is argued that linguistic signs are part of a political economy, not just vehicles for thinking about it. Only a conception of language as multifunctional can give an adequate view of the relations between language and the material world, and evade a false dichotomy between “idealists” and “materialists.”[language, political economy, sociolinguistics, semiotic theory, Senegal]
Abstract Conventionally, Apartheid is regarded as no more than an intensification of the earlier policy of Segregation and is ascribed simplistically to the particular racial ideology of the ruling Nationalist Party. In this article substantial differences between Apartheid and Segregation are identified and explained by reference to the changing relations of capitalist and African pre-capitalist modes of production. The supply of African migrant labour-power, at a wage below its cost of reproduction, is a function of the existence of the pre-capitalist mode. The dominant capitalist mode of production tends to dissolve the pre-capitalist mode thus threatening the conditions of reproduction of cheap migrant labour-power and thereby generating intense conflict against the system of Segregation. In these conditions Segregation gives way to Apartheid which provides the specific mechanism for maintaining labour-power cheap through the elaboration of the entire system of domination and control and the transformation of the function of the pre-capitalist societies.
暂无摘要（点击查看原文获取完整内容）
Economists often ask how private information is shared through markets, costly signaling, and other mechanisms. Yet most information sharing is done through ordinary, informal talk. Economists are inconsistent in their view of such ‘cheap talk’: sometimes it is supposed that communication generally leads to efficient equilibria; other times it is supposed that since ‘talk is cheap,’ it is never credible. The authors think both views are wrong. In this paper, they describe what some recent research in game theory teaches about when people will convey private information by cheap talk.
Contents Acknowledgments Introduction 1. The Homosocial World of Working-Class Amusements 2. Leisure and Labor 3. Putting on Style 4. Dance Madness 5. The Coney Island Excursion 6. Cheap Theater and the Nickel Dumps 7. Reforming Working Women's Recreation Conclusion Notes Index
This article examines the methodological implications of the fact that what people say is often a poor predictor of what they do. We argue that many interview and survey researchers routinely conflate self-reports with behavior and assume a consistency between attitudes and action. We call this erroneous inference of situated behavior from verbal accounts the attitudinal fallacy. Though interviewing and ethnography are often lumped together as “qualitative methods,” by juxtaposing studies of “culture in action” based on verbal accounts with ethnographic investigations, we show that the latter routinely attempts to explain the “attitude–behavior problem” while the former regularly ignores it. Because meaning and action are collectively negotiated and context-dependent, we contend that self-reports of attitudes and behaviors are of limited value in explaining what people actually do because they are overly individualistic and abstracted from lived experience.
The idea that women's subordinate position stems from a lack of job opportunities, and can be ended by the provision of sufficient job opportunities, is deeply rooted and held by a wide spectrum of opinion, from international development agencies, government bureaux and mainstream Marxists to many women's organizations. Our work in the Workshop on the Subordination of Women in the process of development at the Institute of Development Studies, Sussex University, has led us to reject this perspective as a starting point. We do not accept that the problem is one of women being left out of the development process. Rather, it is precisely the relations through which women are 'integrated' into the development process that need to be problematized and investigated. For such relations may well be part of the problem, rather than part of the solution. Our starting point, therefore, is the need to evaluate world market factories from the point of view of the new possibilities and the new problems which they raise for Third World women who work in them.
Introduction Part I. The Broadside Ballad: 1. Small and popular music 2. A Godly ballad to a Godly tune 3. The 1642 Stock Part II. The Broadside Picture: 4. Idols in the frontispiece 5. Stories for walls 6. Godly tables for good householders Part III. The Chapbook: 7. The development of the chapbook trade 8. Penny books and marketplace theology Conclusion.
A large body of literature suggests willingness‐to‐pay is overstated in hypothetical valuation questions as compared to when actual payment is required. Recently, “cheap talk” has been proposed to eliminate the potential bias in hypothetical valuation questions. Cheap talk refers to process of explaining hypothetical bias to individuals prior to asking a valuation question. This study explores the effect of cheap talk in a mass mail survey using a conventional value elicitation technique. Results indicate that cheap talk was effective at reducing willingness‐to‐pay for most survey participants; however, consistent with previous research, cheap talk did not reduce willingness‐to‐pay for knowledgeable consumers.