Large language models (LLMs) use data to learn about the world in order to produce meaningful correlations and predictions. As such, the nature, scale, quality, and diversity of the datasets used to train these models, or to support their work at inference time, have a direct impact on their quality. The rapid development and adoption of LLMs of varying quality has brought into focus the scarcity of publicly available, high-quality training data and revealed an urgent need to ground the stewardship of these datasets in sustainable practices with clear provenance chains. To that end, this technical report introduces Institutional Books 1.0, a large collection of public domain books originally digitized through Harvard Library's participation in the Google Books project, beginning in 2006. Working with Harvard Library, we extracted, analyzed, and processed these volumes into an extensively-documented dataset of historic texts. This analysis covers the entirety of Harvard Library's collection scanned as part of that project, originally spanning 1,075,899 volumes written in over 250 different languages for a total of approximately 250 billion tokens. As part of this initial release, th
Historical photographic plate archives anchor a growing body of time-domain astronomy, but time-domain claims drawn from them are vulnerable to plate-sensitivity variations correlated with environmental modulators that can mimic real astrophysical signals. I present a simple, broadly applicable protocol for testing such artifact hypotheses: regress catalog-aggregate reference-population metrics (stellar detection counts or plate limiting magnitudes) against the suspected modulator. Under the artifact hypothesis, the reference metric varies systematically; under the null hypothesis, it does not. I apply the protocol to test whether geomagnetic storm activity, measured by the planetary Kp index, modulates plate sensitivity at two independent observatories. At Harvard College Observatory, the DASCH DR7 archive provides limiting magnitudes for 12,510 exposures across 500 sky positions: no significant trend across Kp bins (Spearman rho = -0.011, p = 0.234). At Palomar, the MAPS Catalog of POSS-I records stellar detection counts for 638 fields: no significant trend (Spearman rho = 0.017, p = 0.673). Plate sensitivity is invariant across the full range of geomagnetic activity at both site
We report the results of identical pre- and post-event surveys given to attendees of a talk, two-sided conversation, and Q&A centered around the book If Anyone Builds It, Everyone Dies at Harvard University in March 2026, covering perceived probability of AI-caused extinction or severe disempowerment resulting from unimpeded AI development, confidence in those estimates, and global priority. Among the 89 matched participants, the post-event median estimate of the probability of existential risk from advanced AI was 70%, and 96% agreed that mitigating AI existential risk should be a global priority. Although these self-selected respondents' pre-event views were already high (50% and 93%, respectively) relative to results of similar surveys that were previously administered to experts and the general public, the event produced increases on all measures when considering the respondents in aggregate. The magnitudes of increases in risk probability were negatively correlated with prior familiarity with the topic: among attendees with little prior familiarity, 60% shifted upward and none shifted downward, whereas among self-described experts, no respondents shifted upward and 20% shi
How has generative AI impacted the experiences of college students? We study the influence of AI on the study habits, class choices, and career prospects of Harvard undergraduates (n=326), finding that almost 90% of students use generative AI. For roughly 25% of these students, AI has begun to substitute for attending office hours and completing required readings. Half of students are concerned that AI will negatively impact their job prospects, and over half of students wish that Harvard had more classes on the future impacts of AI. We also investigate students' outlook on the broader social implications of AI, finding that half of students are worried that AI will increase economic inequality, and 40% believe that extinction risk from AI should be treated as a global priority with the same urgency as pandemics and nuclear war. Around half of students who have taken a class on AI expect AI to exceed human capabilities on almost all tasks within 30 years. We make some recommendations to the Harvard community in light of these results.
Fairness (also known as equity interchangeably) in machine learning is important for societal well-being, but limited public datasets hinder its progress. Currently, no dedicated public medical datasets with imaging data for fairness learning are available, though minority groups suffer from more health issues. To address this gap, we introduce Harvard Glaucoma Fairness (Harvard-GF), a retinal nerve disease dataset with both 2D and 3D imaging data and balanced racial groups for glaucoma detection. Glaucoma is the leading cause of irreversible blindness globally with Blacks having doubled glaucoma prevalence than other races. We also propose a fair identity normalization (FIN) approach to equalize the feature importance between different identity groups. Our FIN approach is compared with various the-state-of-the-art fairness learning methods with superior performance in the racial, gender, and ethnicity fairness tasks with 2D and 3D imaging data, which demonstrate the utilities of our dataset Harvard-GF for fairness learning. To facilitate fairness comparisons between different models, we propose an equity-scaled performance measure, which can be flexibly used to compare all kinds o
The US higher education system concentrates the production of science and scientists within a few institutions. This has implications for minoritized scholars and the topics with which they are disproportionately associated. This paper examines topical alignment between institutions and authors of varying intersectional identities, and the relationship with prestige and scientific impact. We observe a Howard-Harvard effect, in which the topical profile of minoritized scholars are amplified in mission-driven institutions and decreased in prestigious institutions. Results demonstrate a consistent pattern of inequality in topics and research impact. Specifically, we observe statistically significant differences between minoritized scholars and White men in citations and journal impact. The aggregate research profile of prestigious US universities is highly correlated with the research profile of White men, and highly negatively correlated with the research profile of minoritized women. Furthermore, authors affiliated with more prestigious institutions are associated with increasing inequalities in both citations and journal impact. Academic institutions and funders are called to creat
Surprise Machines is a project of experimental museology that sets out to visualize the entire image collection of the Harvard Art Museums, intending to open up unexpected vistas on more than 200,000 objects usually inaccessible to visitors. Part of the exhibition Curatorial A(i)gents organized by metaLAB (at) Harvard, the project explores the limits of artificial intelligence to display a large set of images and create surprise among visitors. To achieve such a feeling of surprise, a choreographic interface was designed to connect the audience's movement with several unique views of the collection.
Glaucoma is the number one cause of irreversible blindness globally. A major challenge for accurate glaucoma detection and progression forecasting is the bottleneck of limited labeled patients with the state-of-the-art (SOTA) 3D retinal imaging data of optical coherence tomography (OCT). To address the data scarcity issue, this paper proposes two solutions. First, we develop a novel generalization-reinforced semi-supervised learning (SSL) model called pseudo supervisor to optimally utilize unlabeled data. Compared with SOTA models, the proposed pseudo supervisor optimizes the policy of predicting pseudo labels with unlabeled samples to improve empirical generalization. Our pseudo supervisor model is evaluated with two clinical tasks consisting of glaucoma detection and progression forecasting. The progression forecasting task is evaluated both unimodally and multimodally. Our pseudo supervisor model demonstrates superior performance than SOTA SSL comparison models. Moreover, our model also achieves the best results on the publicly available LAG fundus dataset. Second, we introduce the Harvard Glaucoma Detection and Progression (Harvard-GDP) Dataset, a multimodal multitask dataset t
Innovation is a major driver of economic and social development, and information about many kinds of innovation is embedded in semi-structured data from patents and patent applications. Although the impact and novelty of innovations expressed in patent data are difficult to measure through traditional means, ML offers a promising set of techniques for evaluating novelty, summarizing contributions, and embedding semantics. In this paper, we introduce the Harvard USPTO Patent Dataset (HUPD), a large-scale, well-structured, and multi-purpose corpus of English-language patent applications filed to the United States Patent and Trademark Office (USPTO) between 2004 and 2018. With more than 4.5 million patent documents, HUPD is two to three times larger than comparable corpora. Unlike previously proposed patent datasets in NLP, HUPD contains the inventor-submitted versions of patent applications--not the final versions of granted patents--thereby allowing us to study patentability at the time of filing using NLP methods for the first time. It is also novel in its inclusion of rich structured metadata alongside the text of patent filings: By providing each application's metadata along with
Harvard architecture CPU design is common in the embedded world. Examples of Harvard-based architecture devices are the Mica family of wireless sensors. Mica motes have limited memory and can process only very small packets. Stack-based buffer overflow techniques that inject code into the stack and then execute it are therefore not applicable. It has been a common belief that code injection is impossible on Harvard architectures. This paper presents a remote code injection attack for Mica sensors. We show how to exploit program vulnerabilities to permanently inject any piece of code into the program memory of an Atmel AVR-based sensor. To our knowledge, this is the first result that presents a code injection technique for such devices. Previous work only succeeded in injecting data or performing transient attacks. Injecting permanent code is more powerful since the attacker can gain full control of the target sensor. We also show that this attack can be used to inject a worm that can propagate through the wireless sensor network and possibly create a sensor botnet. Our attack combines different techniques such as return oriented programming and fake stack injection. We present impl
Comparison of the old observations of Cepheids in the Small Magellanic Cloud from the Harvard data archive, with the recent OGLE and ASAS observations allows an estimate of their period changes. All of matched 557 Cepheids are still pulsating in the same mode. One of the Harvard Cepheid, HV 11289, has been tentatively matched to a star which is now apparently constant. Cepheids with log P > 0.8 show significant period changes, positive as well as negative. We found that for many stars these changes are significantly smaller than predicted by recent model calculations. Unfortunately, there are no models available for Cepheids with periods longer than approximatelly 80 days, while there are observed Cepheids with periods up to 210 days.
In this work we evaluate a neural based speech intelligibility booster based on spectral shaping and dynamic range compression (SSDRC), referred to as WaveNet-based SSDRC (wSSDRC), using a recently designed Greek Harvard-style corpus. The corpus has been developed according to the format of the Harvard/IEEE sentences and offers the opportunity to apply neural speech enhancement models and examine their performance gain for Greek listeners. wSSDRC has been successfully tested for English material and speakers in the past. In this paper we revisit wSSDRC to perform a full scale evaluation of the model with Greek listeners under the condition of equal energy before and after modification. Both normal hearing (NH) and hearing impaired (HI) listeners evaluated the model under speech shaped noise (SSN) at listener-specific SNRs matching their Speech Reception Threshold (SRT) - a point at which 50 % of unmodified speech is intelligible. The analysis statistics show that the wSSDRC model has produced a median intelligibility boost of 39% for NH and 38% for HI, relative to the plain unprocessed speech.
Digital Access to a Sky Century @ Harvard (DASCH) is a project to digitize the collection of ~500,000 glass photographic plates held at Harvard College Observatory. The collection spans the time period from 1880 to 1985, during which time every point on the sky has been observed approximately 500 to 1000 times. In this paper we describe the results of the DASCH commissioning run, during which we developed the data-reduction pipeline and fine-tuned the digitzer's performance and operation. This initial run consisted of 500 plates taken from a variety of different plate-series, all containing the open cluster Praeseppe (M44). We report that accurate photometry at the 0.1mag level is possible on the majority of plates, and demonstrate century-long light-curves of various types of variable stars in and around M44.
The last decade has seen a proliferation of mentoring programs that provide high-school students authentic research experiences. Such programs expose students to front-line research, equip them with basic research skills (including coding skills), and introduce them to scientist role models. Mentors in such programs range from undergraduate students to faculty members. Here, I describe the founding and first two years of operation of the Harvard Science Research Mentoring Program (SRMP). This program specifically recruits advanced graduate students and postdoctoral scholars to serve as mentors. By mentoring high-school students over a long timescale (September to May), early-career scientists gain hands-on experience in the skills required to advise students|skills that are often required of them in future academic positions yet seldom taught by academic institutions. Finally, I invite directors of existing and prospective SRMPs to join the Global SPHERE Network, through which directors of SRMPs around the world can share their experiences, best practices, and questions.
A biographical profile of the theoretical physicist, Julian Schwinger, and a discussion of "The Memories of Julian" Centennial Celebration (held at Harvard University on February 12, 2018) is presented herein.
The astrophysical parameters of four unstudied open star cluster candidates - Harvard 9, Ivanov 2, Ivanov 7, and Ivanov 9 - have been estimated for the first time using the PPMXL database. The stellar density distributions and color-magnitude diagrams for each cluster are used to determine the geometrical structure (cluster center, limited radius, core and tidal radii, the distances from the Sun, from the Galactic center and from the Galactic plane). Also, the main photometric parameters (age, distance modulus, color excesses, membership, total mass, relaxation time, luminosity and mass functions) are estimated.
In this paper, we demonstrate and discuss results of our mining the abstracts of the publications in Harvard Business Review between 1922 and 2012. Techniques for computing n-grams, collocations, basic sentiment analysis, and named-entity recognition were employed to uncover trends hidden in the abstracts. We present findings about international relationships, sentiment in HBR's abstracts, important international companies, influential technological inventions, renown researchers in management theories, US presidents via chronological analyses.
Scientists have created a silicon chip that can write dozens of DNA sequences simultaneously using electricity and water-based enzymes, offering a cleaner alternative to conventional DNA manufacturing。 The breakthrough could eventually support portable DNA-writing devices and even massive DNA data storage, although new chemistry will be needed to s
The Harvard College Observatory was the preeminent astronomical data center of the early 20th century: it gathered and archived an enormous collection of glass photographic plates that became, and remains, the largest in the world. For nearly twenty years DASCH (Digital Access to a Sky Century @ Harvard) actively digitized this library using a one-of-a kind plate scanner. In early 2024, after 470,000 scans, the DASCH project finished. Now, this unique analog dataset can be integrated into 21st-century, digital analyses. The key DASCH data products include ~200 TB of plate images, ~16 TB of calibrated light curves, and a variety of supporting metadata and calibration outputs. Virtually every part of the sky is covered by thousands of DASCH images with a time baseline spanning more than 100 years; most stars brighter than B ~ 15 have hundreds or thousands of detections. DASCH Data Release 7, issued in late 2024, represents the culmination of the DASCH scanning project.
The nutritional quality of diets has significantly deteriorated over the past two to three decades, a decline often underestimated by the people. This deterioration, coupled with a hectic lifestyle, has contributed to escalating health concerns. Recognizing this issue, researchers at Harvard have advocated for a balanced nutritional plate model to promote health. Inspired by this research, our paper introduces an innovative Image-Based Dietary Assessment system aimed at evaluating the healthiness of meals through image analysis. Our system employs advanced image segmentation and classification techniques to analyze food items on a plate, assess their proportions, and calculate meal adherence to Harvard's healthy eating recommendations. This approach leverages machine learning and nutritional science to empower individuals with actionable insights for healthier eating choices. Our four-step framework involves segmenting the image, classifying the items, conducting a nutritional assessment based on the Harvard Healthy Eating Plate research, and offering tailored recommendations. The prototype system has shown promising results in promoting healthier eating habits by providing an acce