WWW https://www.math.utah.edu/~beebe

The TUG bibliography archive

Last top-level updates: Fri Oct 11 18:54:28 2024 [but the bibliography archive files are updated often!]

This is a large archive of freely distributable bibliographic data in BibTeX format; see the sections below for more information on its contents, and how to mirror, and use, the data. As of 11 October 2024, there are more than 2.00 million BibTeX entries in the combined archives at this site.

If you are unfamiliar with BibTeX, or bibliographic markup systems, and would like to learn more, visit this tutorial . It discusses many of the issues that are important for bibliographic work, and describes numerous software tools that can make such work easier and more productive.

You can jump from here directly to the TUG bibliography archive download or mirror sections.

In December 2021, in collaboration with scientists at U.S. National Oceanic and Atmospheric Administration (NOAA), we released a new archive of bibliographies of journals on the subject of fisheries research. That collection is expected to grow over the next several months.

Project history

The TeX User Group (TUG) bibliography archive began in the early 1990s (before the World Wide Web!) as a modest project to record bibliographic data for the maintainer's large collection of books and journals. The data are in BibTeX format, which is the most extensible, portable, and powerful format for markup of bibliographic information, and has the great advantage that even people who have never seen a BibTeX entry can easily understand the meaning of its various fields.

When the bibtex program is available, as it is in all standard TeX and LaTeX distributions since the mid-1980s, entries can be referenced in a document with \cite{lab-1,lab-2,…,lab-n} commands, which causes those citation labels to be written to an auxiliary file when TeX typesets the document. When BibTeX later reads that file, it searches a user-supplied list of bibliographic data files for matching entries, and then formats them into the user-chosen bibliographic style, of which hundreds are available. The next time that TeX typesets the document, the selected references are displayed as a bibliography section at the point where the \bibliography{file-1,file-2,…,file-k} command appears. When bibliographic entries cross-reference other entries, it may be necessary to run TeX and BibTeX a few more times in order to collect all of the needed cross references. Their log files record requests for still-missing BibTeX entries.

The great beauty of that approach is that the horrid and tedious job of formatting reference-list data is handled completely by a computer program, instead of a poor human, and provided that the BibTeX entries are correct, the bibliography will be too. Cross references embedded in BibTeX entries ensure that related documents, such as comments, errata, notes, and remarks are automatically included as well, without the author having to remember to cite them. The ease of bibliography production afforded by BibTeX encourages authors to do a thorough job of citing prior work, rewarding the authors of those works, and helping the reader to find related documents.

As the collection grew, so did its value, and it became evident that it was an effort worth sharing with others, because once a correct BibTeX entry has been created for a document, no one else has to do that job again. In addition, as new data become available, such as Web locations of documents, and standard document numbers, the BibTeX entries can be updated, and new citations of those entries then automatically include the new material, provided that a bibliography style that knows about the new fields has been used.

The growth of the World Wide Web allows sharing of data with others around the globe, and made it worthwhile to develop software tools to convert bibliographic data from a wide variety of formats to BibTeX form. That has been particularly important for automatically recovering reusable bibliographic data from publisher Web pages for journal-issue contents, which is likely to be the most reliable source of such data. Some of those tools are documented in a journal article, A Bibliographer's Toolbox, and the conversion to relational databases is described in another article, BibTeX meets relational databases. The most important tools for BibTeX file creation and maintenance are described in yet another article, Bibliography prettyprinting and syntax checking.

The TUG bibliography archive covers scores of subjects, and hundreds of journals, with more than one million BibTeX entries. Having them available in multiple SQL databases has made it possible to automate the job of identifying new entries that are candidates for inclusion in author- or subject-specific bibliographies, as well as to run numerous sanity checks on the entire corpus of data. As a result, both the quality and the quantity of the bibliographic data continue to grow.

Several publishers, including ACM, AIP, AMS, APS, arXiv, Elsevier (ScienceDirect), EMS, EPS, IEEE, SAO/NASA, SIAM, Springer, and Wiley, have recognized the value of BibTeX markup, and can return literature database search results in BibTeX form. However, most of those results still require a certain amount of manual editing to provide missing fields, to add protecting braces around proper nouns in titles, to supply missing authors, and to correct botched mathematical markup. Some of those publishers have even adopted the BibTeX citation-label syntax that was developed for the TUG bibliography and BibNet Project archives. None of them yet offers complete journal coverage in BibTeX form, and often, there are severely low limits on how many entries can be retrieved in response to a single search.

Project content highlights

Note: In the tables in the rest of this document, you can replace .html with .bib in any bibliography link to get the original BibTeX file. With similar changes, you can get DVI files (.dvi), LaTeX wrappers (.ltx), PDF files (.pdf), compressed PostScript files (.ps.gz or .ps.xz), spelling dictionaries (.sok), and titleword cross-reference files (.twx).

Related bibliographies

The BibNet Project contains bibliographies on selected authors in numerical analysis and quantum theory, and on selected subjects in those areas. If names like Niels Bohr, Albert Einstein, Werner Heisenberg, Robert Oppenheimer, Wolfgang Pauli, Erwin Schrödinger, and Edward Teller are possibly familiar to you, then you may find their bibliographies useful, because they record not only all (or nearly all) the known publications of those authors, but also publications about them and their work.

Quantum theory radically changed the history of the Twentieth Century, so even if you didn't know about those scientists before, you certainly use electronic devices, including computers, GPS systems, the Internet, and mobile telephones, that could not exist without the understanding of quantum theory and Relativity that they were key figures in developing. Nuclear weapons created in the Manhattan Project in the Second World War (1939–1945), and the Cold War that followed it, are also a legacy of quantum theory, but so is nuclear power, which provides 13% of the world's electrical supply, and in some countries, more than half the electricity.

Two other large BibTeX-format bibliography archives of note are the Karlsruhe Collection of Computer Science Bibliographies (7 million entries in early 2021), and the Universität Trier DBLP Computer Science Bibliography (5.5 million entries in early 2021).

The Karlsruhe archive mirrors the Utah archives, possibly with some rearrangement into subject-specific directories.

Archive file types

Each BibTeX bibliography has the standard file extension .bib. It is accompanied by a LaTeX file with extension .ltx that is used to typeset all of the entries in the BibTeX file to demonstrate that they are free of TeX-markup errors, and show how they might appear in a reference list in one particular bibliography style. Those two files are the only ones created by humans. The remaining files for each bibliography are created by software, and are automatically updated as new versions of the bibliography are released on the TUG bibliography archive Web site. Their file extensions are:

• .dvi (TeX DeVice-Independent output file),
• .html (HyperText Markup Language file),
• .pdf (Portable Document Format file),
• .ps.gz (PostScript file with gzip compression),
• .ps.xz (PostScript file with xz compression),
• .sok (spelling exception list), and
• .twx (title-word cross reference list).

In a Web browser, the .bib and .html files should be visually identical, allowing cut-and-paste operations from either, but the HTML file is enriched with hypertext links that in many cases lead to online documents. The BiBTeX file is the critical file, and is needed if you wish to incorporate multiple references from a given bibliography file in your document.

Downloading archive files

You can find top-level indexes of TUG bibliography archive files in one giant index or, if you know, or can guess, the BibTeX filename, in smaller separate index sections with initial-file-letter indexes in a,    b,    c,    d,    e,    f,    g,    h,    i,    j,    k,    l,    m,    n,    o,    p,    q,    r,    s,    t,    u,    v,    w,    x,    y,    z.

Mirroring the archive

If you are willing, and have adequate disk space (about 11GB), we strongly urge you to consider mirroring the project archive from its home site to your site, either for local-use only, or made available to the public at your Web site. Librarians have a good acronym for that practice: LOCKSS (Lots of Copies Keeps Stuff Safe).

If you succeed in creating a stable up-to-date mirror that you believe will be able to exist for a long time, please send e-mail to the maintainer with a request for it to be added to a list of TUG bibliography archive mirrors.

One brute-force way to pull the entire archive to your system is a recursive retrieval with either of two popular Unix utilities:

% ncftpget -R ftp://ftp.math.utah.edu/pub/tex/bib/

% wget --recursive ftp://ftp.math.utah.edu/pub/tex/bib/

A better way is to exploit the fact that the master host FTP server can return entire directory trees in any of several archive formats:

% wget ftp://ftp.math.utah.edu/pub/tex/bib.jar

% curl -o bib.tar.gz ftp://ftp.math.utah.edu/pub/tex/bib.tar.gz

% ncftpget ftp://ftp.math.utah.edu/pub/tex/bib.tar

% wget ftp://ftp.math.utah.edu/pub/tex/bib.tar.bz2

% wget ftp://ftp.math.utah.edu/pub/tex/bib.zip

% wget ftp://ftp.math.utah.edu/pub/tex/bib.zoo

You can use those same URLs in most Web browsers, and then unpack the just-downloaded archive file in a suitable location. The unpacking normally preserves file protections and file timestamps.

The preferred way, however, is to use the rsync utility, which uses a clever algorithm on both sides of the connection to transfer only the changes between files, dramatically reducing transfer times when the two archives have similar contents.

# Find out what collections are available to rsync:
% rsync rsync://ftp.math.utah.edu/
CTAN            all of ftp://ctan.tug.org/ (huge)
bib             TeX User Group bibliography archive (large)
bibnet          BibNet Project bibliography archive
historic       	archive of past TeX distributions and files
texlive         all of ftp://tug.org/texlive/ (huge)
TUHS           	The Unix Heritage Society (TUHS) mirror

# Fetch one of them (the -a option preserves important timestamp
# information, and the -z option turns on compression to reduce
# network traffic; add the -v option for verbose output):
% rsync -a -z rsync://ftp.math.utah.edu/bib .

# See how long a subsequent update might take
% time rsync -a -z rsync://ftp.math.utah.edu/bib .
0.004u 0.013s 0:00.34 2.9%      0+0k 0+0io 0pf+0w

rsync can be used to populate an initial copy of a mirror

The rsync utility should now be standard in most Unix distributions, but if your machine does not have it, you can find it at http://rsync.samba.org/. There is a separate project that wraps the command-line version in a graphical user interface for common Unix, Mac OS X, and Microsoft Windows systems: http://www.opbyte.it/grsync/. Prebuilt versions of grsync are installable from some Unix package distributions. The grsync program remembers your settings, so once you have used it to configure and run a mirror update, you can run it manually from time to time and get updates with a single click.

Once you have a copy of the archive on your system, use a regularly scheduled cron job to keep your copy up-to-date. We recommend at least weekly updates if your copy is for local use only, and nightly (our time: GMT/UCT - 7 hours) if your copy is a mirror on a public Web site.

There are public mirrors of the TeX User Group bibliography archives at Oak Ridge National Laboratory (Oak Ridge, TN, USA) and at Sandia National Laboratory (Albuquerque, NM, USA).

Searching archive files

There are several ways to search the archive files, apart from Web search engines whose own copies of the data are likely to be several weeks out of date. If you know which bibliography has the entry you want, then just visit the file in your favorite text editor and use its search commands.

The Unix grep command-line utility family is one common approach to search in multiple files:

% grep -B 4 '^ *title *= .*Einstein.*Berlin' *.bib

Its limitation is that it is line based, and search strings must match a single line.

The bibsearch utility provides a much faster way, and it eliminates the line-boundary constraint because each BibTeX entry is treated as a single block of text:

% bibsearch
> title & einstein & berlin & 2003

A more powerful way to search is first to convert the data to SQL (Structured Query Language) with bibtosql, and then to use the bibsql front end, or the sqlite3 program directly, and enter SQL commands for selective searching and display of specified fields, or even entire BibTeX entries:

# create the SQLite3 database (once only)
% bibtosql --create -database sqlite *.bib | sqlite3 tugbib.db

# search the SQLite3 database
% sqlite3 tugbib.db

-- how many BibTeX entries are in the database?

sqlite> select count(*) from bibtab;
926188

-- which entries are about Einstein's years in Berlin?

sqlite> select filename, label from bibtab
        where (title like '%Einstein%Berlin%')
        order by filename, year, label;
bohr-niels.bib|Hendry:1986:BRJ
bohr-niels.bib|Hendry:1986:BRW
...
einstein.bib|Treder:1966:ESE
einstein.bib|Kirsten:1979:AEB
einstein.bib|Nelkowski:1979:ESB
...
einstein.bib|vanDongen:2012:MIM
...

-- get the most recent entry about Einstein in Berlin

sqlite> select entry from bibtab
     where (label = 'vanDongen:2012:MIM');

@Article{vanDongen:2012:MIM,
  author =       "Jeroen van Dongen",
  title =        "Mistaken Identity and Mirror Images: {Albert and Carl
                 Einstein}, {Leiden} and {Berlin}, {Relativity} and
                 Revolution",
...
}

The sqlite3 program is public-domain software. It is extremely portable, and its database files do not depend on the host operating system or the host CPU's memory byte order; once created, those files can be copied and used everywhere. Prebuilt versions are available for common desktop platforms, and even for some mobile telephones!

Because most BibTeX entries in the archives carry a time stamp field that records when the entry was created or modified, you can use that field to find recently added material:

-- change output format to aligned column

sqlite> .mode columns

-- find the most recent Einstein entries

sqlite> select label, bibtimestamp, substr(title, 1, 40) from bibtab
        where (filename = 'einstein.bib')
          and (bibtimestamp > '2013.01.01 00:00:00 AAA')
        order by bibtimestamp;
Lanouette:1994:AS  2013.01.11 06:50:11 ???  Atomic Spies
Walker:1997:PUD    2013.01.11 09:28:10 MST  Prompt and utter des
Buchwald:2001:HEB  2013.01.11 12:08:33 MST  Histories of the Ele
Walker:2004:TMI    2013.01.11 12:17:42 ???  Three Mile Island: N
Thackray:1977:BRB  2013.01.12 11:56:22 MST  Book Review: booktit
...

The paper at the bibsql Web site gives numerous examples of how the data can be mined in many more ways that are simply infeasible without the added structure of SQL fields.