Unlocking Earth’s History: How AI and the Biodiversity Heritage Library Are Rescuing 19th-Century Ecological Data

Executive Overview

As the global community confronts the compounding, existential threats of rapid climate change and catastrophic biodiversity loss, scientists and researchers are looking backward just as eagerly as they are looking forward. Historical archives—long sitting quietly in climate-controlled museum basements, university libraries, and private collections—contain vital baseline data about the natural world before the heavy hand of modern industrialization fundamentally altered it.

For two decades, the Biodiversity Heritage Library (BHL) has stood at the vanguard of this archival resurrection. Celebrating its 20th anniversary, the BHL has grown into the world’s largest open-access digital library for biodiversity literature and archives. Boasting an astonishing corpus of over 64 million pages, the platform is far more than a digital bookshelf; it is a sprawling, high-resolution repository of historical ecological data. These pages hold the secrets of past ecosystems, waiting to be extracted, aggregated, and deployed to model anthropogenic effects on climate, the environment, and life on Earth.

Now, a pioneering project bridging Harvard University’s Ernst Mayr Library (EML) and the BHL has taken a massive leap forward. By leveraging state-of-the-art Artificial Intelligence (AI) tools, researchers have successfully digitized, transcribed, and integrated the lost ornithological field journals, notebooks, and bird lists of Walter Faxon—a 19th-century Harvard zoologist and field naturalist. This initiative not only rescues fragile handwritten observations compiled between 1886 and 1919 but also paves the way for seamless data ingestion into the Global Biodiversity Information Facility (GBIF). This fusion of historical natural history and cutting-edge technology exemplifies how the past can illuminate the path toward ecological survival.

Collections as data: Transcribing the ornithological observations of Walter Faxon

Detailed Chronology: From Harvard Fieldwork to AI-Driven Transcription

To understand the magnitude of the Faxon project, one must trace its journey across more than a century—from the muddy fields of Massachusetts in the late 1800s to the algorithmic pipelines of modern machine learning laboratories.

The Life and Legacy of Walter Faxon (1848–1920)

Born in Massachusetts on February 4, 1848, Walter Faxon dedicated his life to the rigorous documentation of the natural world. Serving as an Assistant Professor of Zoology at Harvard University from 1872 to 1886 and as the Curator of the Invertebrate Department at Harvard’s prestigious Museum of Comparative Zoology (MCZ) for 46 years (1874–1920), Faxon was primarily a specialist in crustaceans and mollusks.

However, ornithology was his lifelong avocation. Faxon approached the study of birds with the exact same unyielding scientific rigor, discipline, and skepticism that governed his institutional curatorial work. A Fellow of the American Ornithologists’ Union from 1901 to 1904, Faxon spent nearly 60 years in the field. Contemporary peers noted that his alertness of eye, ear, and mind, combined with a scrupulous aversion to error, made him one of the most reliable field ornithologists of his era. When Walter Faxon identified a bird, his record was never questioned.

Collections as data: Transcribing the ornithological observations of Walter Faxon

Preservation and Digitization (2025–2026)

Faxon’s physical legacy—comprising roughly 2,200 pages of manuscript field journals, bird lists, field notebooks, correspondence, and intricate pencil drawings of arthropods and crustaceans—was safely housed within the Ernst Mayr Library. Yet, paper manuscripts remain locked silos unless they are digitized and made searchable.

The turning point came in 2025, when the collection was prioritized for digitization with the vital financial and operational support of the Harvard Library Collections Digitization Program, falling squarely under its institutional theme of Climate Change.

Following digitization, the institutional challenge shifted to transcription. Handwritten 19th-century cursive, marginalia, and shorthand notations are notoriously difficult and labor-intensive for human transcribers to process at scale. To solve this bottleneck, the EML appointed a Pforzheimer Fellow in mid-2026: Mohit Mandal, a Harvard PhD candidate in Anthropology.

Collections as data: Transcribing the ornithological observations of Walter Faxon

Tasked with investigating the viability of AI-powered transcription tools from June 1 to August 31, 2026, Mandal embarked on an empirical evaluation of leading language models. After rigorous testing, he selected ChatGPT due to its superior flexibility with follow-up prompting and training consistency, supplementing its output with cross-verification via Claude to maximize accuracy. By the conclusion of his fellowship in late August 2026, Mandal and the EML team had achieved their primary operational objective: transcribing nearly 2,000 manuscript pages of Faxon’s ecological observations and seamlessly integrating them into the BHL platform.


Supporting Context & Metrics: Unlocking the Data Pipeline

The Faxon collection in the Ernst Mayr Library may be physically modest, but its data density is extraordinary. Of the collection’s 2,200 total pages, 1,640 pages consist strictly of field journals, bird lists, and field notebooks compiled continuously over a 33-year span from 1886 to 1919.

The Value of Historical Species Occurrence Data

Within these 1,640 pages lie thousands of individual bird observations. Each entry captures precise parameters:

Collections as data: Transcribing the ornithological observations of Walter Faxon
  • Species Identification: Scrupulously verified scientific and common names.
  • Temporal Metrics: Exact dates and seasonal timelines of sightings.
  • Spatial Data: Specific geographic localities across Massachusetts and New England.
  • Demographic & Behavioral Notes: Number of individuals seen, sex, life-stage markers, and auditory behaviors such as vocalizations and song activity.

When these longitudinal records are aggregated with global datasets through platforms like GBIF, they provide conservation scientists with an invaluable baseline. They allow researchers to answer critical ecological questions: How have migration windows shifted over the last 130 years in response to rising temperatures? Which bird populations have contracted geographically, and which have expanded? Without historical field notes like Faxon’s, modern climate science lacks the historical baseline required to measure the true velocity of contemporary ecological collapse.

The Technological Hurdle: AI and Handwritten Manuscripts

Despite the successes of the 2026 Pforzheimer Fellowship, deploying AI against 19th-century handwritten manuscripts revealed distinct technological boundaries. Mandal and the EML team encountered several recurring transcription challenges:

  1. Faded and Smudged Pencil Notes: Much of Faxon’s field recording was done on the fly in pencil, leading to surface smudges that frequently confused optical recognition layers.
  2. Unconventional Margins: Marginal text scrawled at sharp angles along page borders was frequently skipped or misconstrued by AI models.
  3. Interlinear Notations: Faxon frequently inserted tiny numbers and symbols between the primary lines of text to denote flock sizes, singing behavior, or counts. Standard word-processing line spacing constraints initially forced AI models to drop these critical numerical modifiers entirely. Mandal ultimately bypassed this by training models to format interlinear text explicitly as superscript.
  4. AI "Hallucinations": On certain pages, language models exhibited generative errors. For example, on page 65928110 of Faxon’s Field Journal v.1, the species Sialia sialis (Eastern Bluebird) was erroneously transcribed as Hylocichla ustulata (Swainson’s Thrush). Similarly, in a field notebook entry, Merula migratoria (historical nomenclature for the American Robin) was mis-transcribed as Mniotilta varia (Black-and-white Warbler).

These errors underscore a vital reality: the human element remains fundamentally irreplaceable. Artificial intelligence serves as an extraordinarily powerful force multiplier, but human oversight, archival expertise, and editorial curation are mandatory to ensure scientific data integrity.

Collections as data: Transcribing the ornithological observations of Walter Faxon

Official Statements and Institutional Perspectives

Reflecting on the milestone, institutional leaders have emphasized the profound synergy between historical library collections and the urgent demands of modern environmental science.

Joe deVeer, Co-Secretary of the BHL Executive Committee and Digital Projects Manager at the Ernst Mayr Library—where he has served since 2006—noted the library’s long-standing commitment to global accessibility. As a founding member of the BHL, the EML has spent two decades working alongside international colleagues to build what is now a 64-million-page digital commons. The integration of the Faxon collection represents a direct fulfillment of BHL’s founding mission: rescuing physical print and manuscript treasures from obscurity and placing them directly into the hands of global researchers.

Speaking on the operational realities of the Pforzheimer Fellowship, Mohit Mandal articulated the core philosophical takeaway of the project:

Collections as data: Transcribing the ornithological observations of Walter Faxon

"The human element remains central to guide the goals and direction of the project, without which AI tools—at least at this stage of their development—often falter in accuracy, especially when handling complex page layouts and difficult handwriting."

Furthermore, leadership at the Ernst Mayr Library and the broader Harvard Library system have highlighted that this project acts as a blueprint for future institutional endeavors. By aligning archival digitization with pressing contemporary themes like climate change, academic libraries are transforming from passive storehouses of paper into active engines of planetary research.


Future Outlook: The Next Era of Biodiversity Informatics

As the Biodiversity Heritage Library marks its 20th anniversary, the completion of the Walter Faxon transcription project points toward an ambitious, data-rich future.

Collections as data: Transcribing the ornithological observations of Walter Faxon

Modernizing the BHL Platform

The next phase of BHL’s evolution involves a comprehensive modernization of its digital infrastructure. Upcoming platform upgrades are specifically designed to ingest, index, and expose AI-generated transcript files natively. This will allow researchers anywhere in the world to perform deep, granular text searches across millions of pages of handwritten manuscripts—a capability that was previously restricted to printed books and monographs.

Feeding the Global Biodiversity Information Facility (GBIF)

The ultimate destination for Faxon’s newly transcribed ornithological observations is GBIF. By exporting cleaned, structured transcript files from BHL into GBIF, historical observations made in the woods of Massachusetts in the late 19th century will sit side-by-side with contemporary eBird submissions and satellite telemetry data. This seamless bridge between Victorian-era field naturalism and 21st-century big data creates a continuous, centuries-long timeline of ecological observation.

A Sustainable Blueprint for Archives Worldwide

The success of deploying ChatGPT and Claude to process Faxon’s notebooks demonstrates that AI-assisted transcription is no longer a speculative sci-fi concept; it is an operational reality. While human editorial review remains vital to catch model hallucinations and correctly interpret complex marginalia, the sheer speed of AI processing unlocks archival collections that would have otherwise taken decades to transcribe manually.

Collections as data: Transcribing the ornithological observations of Walter Faxon

As climate change accelerates and species face unprecedented pressures, the race to rescue data from the past has never been more urgent. Through the collaborative efforts of the Ernst Mayr Library, the Harvard Library Collections Digitization Program, visionary fellows like Mohit Mandal, and the enduring global infrastructure of the Biodiversity Heritage Library, Walter Faxon’s 130-year-old field notes are finally speaking again—providing the vital historical context humanity needs to understand, protect, and restore the living world.

Leave a Comment

Your email address will not be published. Required fields are marked *