Collections
History Lab has collected many different collection of declassified government documents as well as documents from international organizations. This page describes the collections as well as the natural language processing tools used on them.
U.S. Government
Foreign Relations of the United States, colloquially known as FRUS, is published by the Office of the Historian of the State Department and represents the official diplomatic history of the United States. It is comprised of over 450 volumes containing official documents from various government agencies, personal papers of Presidents and other key diplomatic players, transcripts of recorded telephone conversations, and, in more recent volumes, declassified documents. The collection spans a long period, with volumes beginning in 1861. Our processed collection currently contains volumes from 1861-1988, with most documents covering the 20th century.
Increasingly overburdened with the sheer number of diplomatic telegrams and cables, the State Department began to digitize its communications in the early 1970s. Our processed collection of these cables comes from the Central Foreign Policy Files at the National Archives, and includes cables, records of cables withdrawn for classification reasons, and so-called “P-Reel” records corresponding to communications sent physically via diplomatic pouch. We currently hold all available digital records from this collection, which spans 1973-1979.
The Kissinger collection is a series that comes to us from the State Department’s FOIA Reading Room. It is made up of transcriptions of recorded phone conversations involving Henry Kissinger, from 1973 to 1976, which includes part his tenure as Secretary of State. The collection includes nearly five thousand individual conversations throughout the covered period.
The Clinton E-Mail collection consists of 54,149 individual messages covering her tenure as Secretary of State from 2009 to 2012. They touch on important moments in contemporary US foreign policy, like Clinton’s response to Wikileaks, the Arab Spring, and negotiations with Iran. You can pull up individual e-mail, or view the original pdfs showing the series of messages to which it belongs, as well as the parts that officials decided to redact.
The daily briefings a U.S. president receives from intelligence agencies has gone through many iterations over the years. Before it settled under its current President's Daily Brief name in 1961, it was known as the Current or Central Intelligence Briefing from 1951 to 1961. The briefing was originally named Daily Summary, a name which it held from 1946 to 1951. The CIA has released the entirety of the Daily Summaries and the Current/Central Intelligence Briefings as well as all of the President's Daily Briefs between 1961 and 1977. We have collected and processed the text for 9,680 of these documents, making them all available in the same place. As with our other collections, we have run topic modeling and Named Entity Recognition on the documents to make them easier to search.
The CIA collection is our second largest collection, consisting of 935,716 documents and nearly 12.4 million pages acquired from the CIA Reading Room. The FOIArchive provides access to the CIA records made available via the automatic declassification provisions of Executive Order 13526, which requires declassifying nonexempt historically valuable records 25 years or older. It provides researchers with a superior search interface to the CIA Records Search Tool (CREST) and allows them to query our other corpora in concert with the CREST collection. We have run OCR on the PDFs to make the text of the documents searchable. Because of unevenness in the quality of the PDFs, the quality of the text varies.
Other Governments
The UK Cabinet Papers collection is comprised of 42,539 documents spanning most of the 20th century. These are the official records of the most important decision-making body within the United Kingdom. The documents cover a range of topics, from the world wars and international security, to empire and decolonization, to the welfare state and the domestic economy. The collection is predominantly divided into conclusions and memoranda from 1907 to 1990
The Azeredo da Silveira collection is comprised of the personal papers of the Brazilian foreign minister during the presidency of Ernesto Geisel. The collection contains 10,279 digitized documents which include letters, memos, and other correspondence. Themes in the collection range from international governance, to nuclear issues, to US-Brazilian relations. The collection further enriches History Lab’s primary source coverage of the 1970s, with documents spanning 1974 to 1979. The papers are provided to us by The Center for Research and Documentation of the Contemporary History of Brazil (CPDOC ) at Fundação Getulio Vargas (FGV), where the original paper documents are housed. Note that OCR results are not optimal for some documents, especially those that were hand-written. We therefore provide links to the CPDOC site that features the original pdfs.
International Organizations
We have worked with the World Bank Group Archives to make some of their collections available on our website. The largest subcollection consists of the records of Robert Strange McNamara. McNamara served as President of the World Bank for 13 years, from 1968 to 1981. Over the course of his presidency, McNamara brought considerable change to all aspects of Bank operations. In addition to the dramatic increase in the volume of loans, he refocused Bank lending to go beyond infrastructure to basic human needs and poverty reduction. The Bank diversified into sectors such as rural development, family planning and pollution control, which were previously overlooked by both the Bank and the development community. The Bank also began to play a more active and, at times, critical role in many developing economies. The records provide a very full account of his long and active presidency. They span the entirety of McNamara’s presidency and consist of speeches, daily schedules, travel briefings, minutes of meetings, correspondence and other records. Other subcollections include the papers of other World Bank Presidents as well as Records of the Office of the Treasurer and Central Files.
The United Nations collection contains the digital records from Kofi Annan's tenure as Secretary General of the UN (1997 to 2006) and from Ban Ki-Moon's time as Secretary-General (2007 to 2016).
The NATO collection contains 46,000 records, the majority of which are documents that have been declassified through the NATO’s Public Disclosure Program. The digital collection provides a documentary record of the first thirty years of the alliance system, with particularly extensive records on the Alliance’s formation and earliest operations. NATO’s Security Policy stipulates a set of protocols for the continuous release of
formerly classified documents -- included in this is the systematic review of secret documents after thirty years, as well as an ad-hoc review process for the expedited release of select documents. Since the opening of the NATO Archives in 1999 -- following decades of calls by
scholars and citizens for research access -- declassified NATO documents have been used alongside U.S. and British documents on a wide range of new studies on the dynamics of intra-alliance diplomacy.