David H. Bailey
"Computo ergo sum."
Updated: 04 Dec 2014
This site is owned and operated by David H. Bailey. Material on this site is provided for research purposes only, and does not necessarily reflect the views or policies of the Lawrence Berkeley National Lab, the University of California, Davis, the U.S. Department of Energy or any other organization.
Bailey has two affiliations for his professional research work:
- Senior Scientist (retired), Computational Research Department, Lawrence Berkeley National Laboratory. Bailey officially retired from LBNL effective 27 June 2013, but continues as an active researcher.
- Research Fellow, Department of Computer Science, University of California, Davis (February 2013 - present).
High performance computing. Bailey is a leading figure in the field of high-performance scientific computing, with research ranging from numerical algorithms to supercomputer performance studies. He is author or co-author of one book and over 100 papers in this field. His paper "The NAS Parallel Benchmarks" (co-authored with several colleagues at NASA Ames Research Center) is widely cited in performance studies of scientific computer systems. His paper "FFTs in External or Hierarchical Memory" presented a technique for performing the fast Fourier transform (FFT) on parallel and hierarchical memory computers that is the now basis of many FFT implementations on modern computer systems. He has received both the Sidney Fernbach Award from the IEEE Computer Society and the Gordon Bell Prize from the Association for Computing Machinery.
Computational and experimental mathematics. Bailey is also a leading figure in the field of computational and experimental mathematics, applying high performance computing to problems in research mathematics. He is author or co-author of five books and over 100 papers in this field, many of them in conjunction with his long-time collaborator Jonathan M. Borwein of the University of Newcastle, Australia. Bailey is also a co-author of two widely used high-precision computation software packages. His best-known paper in this area, "On the Rapid Computation of Various Polylogarithmic Constants," co-authored with Peter Borwein (Jonathan Borwein's brother) and Simon Plouffe, describes a new formula for pi that permits one to directly calculate binary digits of pi beginning at an arbitrary starting position (this formula was discovered using Bailey's computer implementation of the PSLQ algorithm). In two more recent papers, Bailey and the late Richard Crandall demonstrated a connection between these formulas and a fundamental question about digit randomness. Bailey has received the Chauvenet Prize and the Merten Hesse Prize from the Mathematical Association of America.
Financial mathematics. Bailey, together with his colleagues Jonathan Borwein (of the University of Newcastle, Australia), Marcos Lopez de Prado (of Guggenheim Partners) and Jim Qiji Zhu (of Western Michigian University), have written a series of papers on mathematical finance. Their best-known paper in this area, "Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance," has attracted considerable interest in the field (see Press reports for details).
Other activities. Bailey and his colleague Jonathan Borwein jointly operate a website devoted to experimental mathematics. They also write articles on mathematics and science for the Math Drudge blog and contribute articles to the Huffington Post and the Conversation. More recently, Bailey, Borwein, Marcos Lopez de Prado and Jim Qiji Zhu have instituted a website and blog devoted to financial mathematics and the misuse of mathematics in the field.
See Papers directory for a full list of over 200 technical papers.
Recent press reports
For additional press reports, see the Press reports page.
- 27 Jan 2015: Priyanjana Bengani, a writer for the publication Automated Trader, discussed our paper Pseudo-mathematics and financial charlatanism at length, with comments by several analysts. Bailey was quoted saying, "when a statistical test is applied multiple times, the probability of obtaining false positives increases as a function of the number of trials."
- 04 Dec 2014: Shawn Langlois, a MarketWatch.com columnist, quoted from our article How have 2014 market prophets fared in his column This 'hideously expensive' market has given the 'show horse' pundits fits.
- 03 Dec 2014: Brett Steenbarger, author of several books and a news column on market trading, mentioned our article How have 2014 market prophets fared in his article Do market prophets bring market profits? Steenbarger lamented that "the performance of funds has largely fallen short of benchmarks; the doomsayers have brought doom to their followers; and even those forecasters who beat the market over a prior ten year period are likely to underperform the following year."
- 29 Nov 2014: Frank Acland featured our Huffington Post article on LENR in his article in E-Cat World, a news and information forum for low energy nuclear reaction research.
- 30 Oct 2014: Michael P. Regan, columnist for Bloomberg News, commented on our research in his article Beware Overfitting Models Even if They Win Baseball Bets. He mentioned our paper Statistical Overfitting and Backtest Performance and our new online tool demonstrating backtest overfitting.
- 28 Jun 2014: Jason Zweig, columnist for the Wall Street Journal, commented our our research in his article Huge Returns at Low Risk? Not So Fast. He first mentioned two humorous examples of investment strategies that would have been hugely successful if implemented over the past 20 years or so, but only by statistical accident. Then in reference to a recent claim of a strategy that has a "100% ... probability of outperformance," he quotes Bailey as responding, tongue in check, "Popes have been trying to achieve infallibility for 2000 years, and these people have finally done it."
- 10 May 2014: John Rekenthaler, Vice President of Research for Morningstar, presented a synopsis of our "Pseudo-Mathematics" paper. He relayed our recommendation that it would "behoove the investment community to adopt a similar policy" to one now being promoted in the pharmaceutical industry, namely to make all test results public.
- 28 Apr 2014: Bailey was mentioned in a Pacific Standard article by Ryan Jacobs. It quotes Bailey saying, "What you end up doing is that the models that you derive or you select tend to just focus on idiosyncrasies of the data, and don't have any real fundamental forward predictive power."
- 23 Apr 2014: After the talk by Bailey and Marcos Lopez de Prado at the "Battle of the Quants" meeting in New York City in March (see below), Bailey was invited to do a video interview for Institutional Investor Journals. This video is now available online Here.
- 17 Apr 2014: Bailey was featured in a Barron's article by Brendon Conway. Bailey was quoted as saying, "The higher the number of configurations tried, the greater is the probability that the backtest is overfit."
- 16 Apr 2014: Bailey was featured in a Financial Times article by Stephen Foley. He summarizes the recent paper by Bailey and others by saying, "The authors' argument is that, by failing to apply mathematical rigour to their methods, many purveyors of quantitative investment strategies are, deliberately or negligently, misleading clients." If you cannot view the article due the pay wall, a PDF copy is available Here.
- 11 Apr 2014: Bailey was featured in a Bloomberg News article. It quotes us as saying, "We strongly suspect that such back-test overfitting is a large part of the reason why so many algorithmic or systematic hedge funds do not live up to the elevated expectations generated by their managers."
- 10 Apr 2014: The paper by Bailey and three co-authors, "Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance" was published by the American Mathematical Society. It is available free of charge from the AMS website, or from the SSRN website (preprint version).
- 26 Mar 2014: On Tuesday March 26, 2014, Marcos Lopez de Prado and Bailey jointly presented a talk How to spot backtest overfitting at the Battle of the Quants meeting in New York City.
Bailey has written one book on performance science, four books on computational and experimental mathematics, a CD-ROM reference. All of these books are available at Amazon.com or directly from the respective publishers. Further information on the experimental math books is available in the Books section of the experimental math website. The performance tuning book is available Here.
Bailey and his colleague Jonathan Borwein have authored articles for The Conversation, an international forum of academic research and discussion based in Melbourne, Australia. A listing of these articles is available here:
Experimental Mathematics website
Bailey has published numerous research studies in the area of "experimental" (computer-assisted) mathematics, which establish that modern high-performance computer technology can be effectively utilized as a tool for mathematical research. Here is a website with additional information:
Financial Mathematics website
Bailey and Borwein, together with their colleagues Marcos Lopez de Prado of Hess Energy Trading Co. and Qiji Jim Zhu of Western Michigan University, have recently written a series of papers in mathematical finance, with the objective of helping researchers and investors distinguish mathematically sound techniques from the unfortunately much larger body of questionable techniques that sadly pervade the finance community and financial news. Here is a website with additional information:
High-precision software library
Bailey is a co-author of several software libraries for high-precision computation. These libraries include translation facilities so that one can use, with minor modifications, ordinary Fortran or C++ programs to perform high-precision calculations:
Huffington Post articles
Bailey and his colleague Jonathan Borwein have authored articles for the Huffington Post, a popular news and information based in the U.S. that was recently named the world's most influential blog/news site in a U.K. Guardian article. A listing of these articles is available here:
Math Drudge blog
Bailey and his collaborator Jonathan Borwein, jointly operate the "Math Drudge" blog, which contains essays, news and other material in the realm of mathematics, computing and scientific research:
Mathematical Investor blog
Bailey and Borwein, together with their colleagues Marcos Lopez de Prado and Qiji Jim Zhu, operate the "Mathematical Investor" blog, which contains essays, news and other material in the realm of financial mathematics and related disciplines:
Online copies of over 180 of Bailey's technical papers are available here:
Online copies of many of Bailey's recent lectures are available here:
Some articles, papers, photos and other materials not directly related
to Bailey's scientific research work are available at this website:
In 1996, Peter Borwein (brother of Jonathan Borwein), Simon Plouffe and Bailey co-authored a paper that presents a new formula for pi:
This formula, now known as the "BBP formula for pi", permits one to compute the n-th binary or hexadecimal digit of pi, without computing the first n-1 digits, by means of a simple scheme that requires very little memory. It was discovered by Simon Plouffe using a computer program written by Bailey that implements a simplified version of Helaman Ferguson's "PSLQ" algorithm. More recently, Richard Crandall and Bailey have shown that there is a connection between the new pi formula and the centuries-old question of normality (ie, statistical randomness of digits in a certain sense) of pi and various other math constants.
Some additional information on pi:
- Bailey's pi website:
- Science News article (April 24, 2004):
PDF (5 Mbyte)
- Scientific American article (May 2003):
- See if your name is coded in the first four billion binary digits of pi:
- Fax to Bailey from "The Simpsons" TV show:
Note: The 40,000th digit of pi was provided to the show by Bailey, and this was aired in the show on 6 May 1993 -- search for "Marge in Chains" at
Simpson math website.
Here are some press reports mentioning Bailey:
Bailey's detailed curriculum vitae (resume), including a list of publications, is
This blog, contains essays, news and other material in the realm of modern science and religion.
This contains over 80 articles on modern science and religion, discussing many issues involved and underscoring the pointlessness of a "war" between the two disciplines.
Websites of interest