By Joseph deVeer Digital Projects Manager, Ernst Mayr Library, Harvard University, and Co-Secretary of the BHL Executive Committee
Executive Overview
As the global community confronts the compounding crises of rapid climate change and unprecedented biodiversity loss, scientists, conservationists, and policymakers are looking deeper into the past to forecast the future. Understanding how ecosystems responded to environmental pressures over a century ago provides a vital baseline for measuring modern anthropogenic impacts.
For two decades, the Biodiversity Heritage Library (BHL) has stood at the vanguard of this effort. Marking its 20th anniversary as the world’s largest open-access digital library for biodiversity literature and archives, the BHL has amassed a staggering corpus of over 64 million pages. Yet, a vast proportion of historical ecological data remains locked within handwritten manuscripts, field journals, and notebooks tucked away in archival vaults.
Recently, a groundbreaking initiative led by Harvard University’s Ernst Mayr Library (EML) and integrated into the BHL has begun to bridge this gap. By combining archival preservation with cutting-edge artificial intelligence, researchers have successfully digitized and transcribed the extensive ornithological field notes of Walter Faxon—a prominent 19th-century Harvard zoologist. Spanning 33 years of meticulous ecological observations (1886–1919), this collection of over 2,200 pages represents a goldmine of species occurrence data.
This project not only highlights the immense potential of AI-powered transcription tools like ChatGPT and Claude in handling historical manuscripts, but it also underscores the irreplaceable role of human oversight. Ultimately, this initiative paves the way for exporting rich, historical datasets to the Global Biodiversity Information Facility (GBIF), empowering global efforts to model biodiversity trends and mitigate accelerating species decline.
Detailed Chronology: From 19th-Century Fieldwork to 21st-Century AI Integration
The journey of Walter Faxon’s records from physical ink on paper to a globally accessible digital dataset spans over 150 years, defined by key milestones in science, institutional preservation, and technological innovation.
1848–1920: The Life and Legacy of Walter Faxon
Born on February 4, 1848, in Massachusetts, Walter Faxon established himself as a towering figure in American zoology. Serving as an Assistant Professor of Zoology at Harvard University from 1872 to 1886 and subsequently as the Curator of the Invertebrate Department at Harvard’s Museum of Comparative Zoology (MCZ) for 46 years (1874–1920), Faxon dedicated his professional career primarily to the study of crustaceans and mollusks.
However, ornithology was his lifelong avocation. Pursued with the same scrupulous scientific rigor as his curatorial duties, Faxon’s field observations earned him widespread renown. Recognized as a Fellow of the American Ornithologists’ Union from 1901 to 1904, contemporaries noted that his nearly 60 years of field experience, combined with an exceptional eye, ear, and meticulous avoidance of error, made him one of the most accurate field ornithologists of his era. When Faxon identified a bird, his record was never questioned—a testament to the high integrity of the data he left behind.
2025: Digitization Under the Climate Change Framework
Recognizing the critical value of these historical observations in tracking ecological shifts over time, the Ernst Mayr Library undertook the physical digitization of Faxon’s ornithological field journals, notebooks, and bird lists in 2025. This phase was made possible through the support of the Harvard Library Collections Digitization Program, specifically aligned with its institutional priority on climate change research.
June – August 2026: The Pforzheimer Fellowship and AI Experimentation
To transform these newly digitized images into searchable, machine-readable text, the EML hosted a 2026 Pforzheimer Fellow, Mohit Mandal, a PhD candidate in Anthropology at Harvard University. Working intensively from June 1 to August 31, 2026, Mandal spearheaded an experimental workflow leveraging generative artificial intelligence to tackle the notoriously difficult task of transcribing historical handwriting.
Mandal’s methodology began with a rigorous comparative evaluation of leading AI models, specifically ChatGPT and Google Gemini, to determine their aptitude for processing historical manuscripts. While Gemini demonstrated exceptional accuracy, its utility was hampered by strict file upload limitations and a lack of cross-platform flexibility, restricting its operations strictly to Google Workspace ecosystems. In contrast, ChatGPT offered greater adaptability, robust follow-up prompting capabilities, and the capacity to maintain contextual consistency through customized model training.
To maximize precision, Mandal adopted a multi-model approach: initial transcriptions were generated via ChatGPT and subsequently verified using Claude, drawing upon the complementary strengths of both architectures to catch errors and elevate overall transcription fidelity.
Supporting Context & Metrics: The Scale and Challenges of the Collection
The Collection’s Anatomy
The Faxon collection housed within the Ernst Mayr Library is remarkably lean yet dense with ecological data. Totaling just over 2,200 manuscript pages, it includes:
Field journals and field notebooks (totaling 1,640 pages)
Detailed bird lists
Historical correspondence
Eight original pencil drawings of Arthropoda and Crustacea
Miscellaneous archival items
Furthermore, the EML holds extensive related collections, including the papers of William Brewster, which feature 145 letters exchanged between Brewster and Faxon.
The crown jewel of Faxon’s collection is its 33-year span of species occurrence data (1886–1919). Within these pages lie thousands of individual bird observations categorized meticulously by:
Species name
Exact date
Geographic locality
Number of individuals observed
Biological metadata (such as sex, life stage, and behavioral vocalizations)
When aggregated with parallel historical datasets worldwide, these records serve as a powerful lens through which scientists can reconstruct past ecosystems and measure long-term biodiversity trajectories.
Technical Hurdles in AI Transcription
Despite the triumphs of the Pforzheimer Fellowship—successfully generating transcripts for nearly 2,000 manuscript pages—the project exposed significant technical hurdles inherent to processing 19th-century handwriting with modern language models.
Among the primary obstacles were physical artifacts of the manuscripts:
Pencil Smudges and Faded Ink: Notes jotted down hastily in the field often suffered from degradation over time.
Marginalia at Angles: Text crammed into margins or written at skewed angles was frequently ignored or misparsed by AI vision encoders.
Interlinear Notations: One of the most intricate challenges involved capturing Faxon’s tiny interlinear numbers and symbols. Placed between the primary lines of text, these notations recorded critical details such as population counts, singing behaviors, and flocking dynamics. Standard word processors (like Microsoft Word and Google Docs) presented line-spacing constraints that initially caused AI models to drop these markers entirely. Mandal ultimately resolved this by training models to format interlinear text using superscript conventions.
AI "Hallucinations": On certain pages, models occasionally substituted incorrect species names due to cursive ambiguities—such as transcribing Sialia sialis (Eastern Bluebird) as Hylocichla ustulata (Swainson’s Thrush), or Merula migratoria (American Robin) as Mniotilta varia (Black-and-white Warbler).
Official Statements and Insights
Reflecting on the conclusion of his fellowship, Mohit Mandal emphasized a vital reality regarding the current state of technology in digital humanities and archival science:
"The human element remains central to guide the goals and direction of the project, without which A.I. tools—at least at this stage of their development—often falter in accuracy, especially when handling complex page layouts and difficult handwriting."
This sentiment underscores the cooperative nature of modern digital archiving. While artificial intelligence provides the necessary scaling power to process thousands of pages in a fraction of human time, human editorial oversight, context-aware prompt engineering, and cross-model verification are indispensable for ensuring scientific integrity.
Joseph deVeer, Digital Projects Manager at the EML and Co-Secretary of the BHL Executive Committee, noted the broader institutional significance of the work:
"With the rapid evolution of AI models, and the upcoming modernization of the BHL platform, we are moving closer to a sustainable means of extracting biodiversity data from handwritten field observations."
Future Outlook: Bridging Archives to Global Biodiversity Frameworks
The completion of the Faxon transcription project marks not an end, but a beginning. The overarching objective of this endeavor is the seamless integration of historical data into the Global Biodiversity Information Facility (GBIF).
By leveraging the Biodiversity Heritage Library’s robust infrastructure—which hosts transcript files, indexes scientific names, and renders historical texts fully searchable—the data extracted from Faxon’s notebooks is transitioned from isolated library archives into active, global scientific circulation.
As the BHL celebrates its 20th milestone, initiatives like the digitization and AI-assisted transcription of the Walter Faxon collection illuminate the path forward. By rescuing centuries-old observations from paper and translating them into interoperable digital formats, institutions like Harvard’s Ernst Mayr Library and the BHL are equipping researchers with the empirical depth required to understand, track, and ultimately protect life on Earth for generations to come.
Acknowledgements
The success of this project was made possible through the generous support of the Harvard Library, via the Harvard Library Collections Digitization Program and the 2026 Pforzheimer Fellowships. Special recognition is extended to Mohit Mandal for his exceptional execution, vision, and dedication throughout his fellowship.
To support the ongoing mission of the Biodiversity Heritage Library in preserving historical literature and unlocking vital ecological data, consider contributing to the BHL 20th Anniversary Fund.
