Round 2... 10 Trillion Digits of Pi

Same program, same computer, just a longer wait...

By Alexander J. Yee & Shigeru Kondo

(Last updated: September 22, 2016)

December 28, 2013: The record has been improved to 12 trillion digits.

For anyone who is interested, this was recently the topic of a Stack Overflow question that I answered.

Although not as exciting as the first time around, we finally managed to pull off a second run using same program and mostly the same hardware. This time, it took about a year due numerous hardware failures along the way...

1415926535 8979323846 2643383279 5028841971 6939937510 : 50
5820974944 5923078164 0628620899 8628034825 3421170679 : 100

2597691971 6538537682 7963082950 0909387733 3987211875 : 4,999,999,999,950
6399906735 0873400641 7497120374 4023826421 9484283852 : 5,000,000,000,000
0874753286 1800417105 0417234177 3440660835 6303291247 : 5,000,000,000,050
0494788787 3350953767 0283816394 7603993291 8259328455 : 5,000,000,000,100

4974817456 4378784878 2784442724 8702748355 5412518205 : 6,000,000,000,000
3820441376 6006093732 4607915193 1761308978 1570965271 : 6,000,000,000,050

1248405812 4259631936 5227923837 0799098542 1979648580 : 7,000,000,000,000
1792125328 5414275137 5682812824 6229793359 0061969684 : 7,000,000,000,050

6636168271 2799313869 3626137973 6405361590 7572836055 : 8,000,000,000,000
6695601471 2802778702 1660409717 9930049024 4234355462 : 8,000,000,000,050

5399802500 0358265067 6788634714 4962622491 5222520345 : 9,000,000,000,000
7793366495 1565215134 1259428770 6232189125 1045517921 : 9,000,000,000,050

9544408882 6291921295 9268257225 1615742394 7483010753 : 9,999,999,999,950
9804871001 5982157822 2070871138 6966940952 1989228675 : 10,000,000,000,000
4392476662 7656619000 2124460557 5531593458 4820611421 : 10,000,000,000,050

Shortcuts:

Numberworld Home

Computation Statistics - All times are Japan Standard Time (JST).

Just like last time, the full computation statistics are listed below. Likewise, the final screenshot is shown. However, due to the numerous times that the computation had been interrupted, the screenshot only shows the final leg of the computation as opposed to start -> finish.

The main computation took 371 days on Shigeru Kondo's desktop. The computer was dedicated for this task.

Over the course of the computation, there were multiple hard drive failures. Each of which required us to roll the computation back to a previous checkpoint. The result was approximately 180 days of lost time.

Due to the size of this computation, a tremendous amount of memory was needed:

Roughly 44 TB* of disk was needed to perform the computation.
Another 7.6 TB of disk was needed to store the compressed output of decimal and hexadecimal digits.

If the digits were stored in an uncompressed ascii text file, the combined size of the decimal and hexadecimal digits would be 16.6 TB.

*All units used in this article are binary:

GB = 2³⁰ bytes

TB = 2⁴⁰ bytes

Pi - Computation
371 days

Start : 10:34 AM (JST) October 10, 2010
Finish: 3:14 PM (JST) October 16, 2011

Pi - Verification
1.86 days (Primary - Bellard's BBP)
4.94 days (Secondary - Plouffe's BBP)
Start : See Detailed Timeline
Finish: See Detailed Timeline

Computation Step	Time
Series Summation:	157 days
Square Root:	5.57 days
Final Multiply:	3.57 days
Verify Multiply:	8.29 hours
Base Conversion:	18.0 days
Verify Base Conversion:	94.6 hours
Time Lost to HW Failures:	180 days
Total Computation Time:	191 days
Total Real Time:	371 days

Validation File: Validation - Pi - 10,000,000,000,050.txt

Note that the multicore efficiency % is inaccurate due to the numerous times that the computation has been interrupted.

For a detailed timeline of events related to this computation, see here.

Purpose

This computation was basically a followup to our computation to 5 trillion digits. So the motivation and purpose is the same.

To see how far can that limit of personal computing be pushed...

As we will show, reaching 10 trillion digits of Pi is much more difficult than 5 trillion digits using our current methods and 2010 computer hardware.

Hardware: Shigeru Kondo's Desktop

The machine we used is mostly the same as the previous computation. The only main difference is that there are more hard drives.

Processor: 2 x Intel Xeon X5680 @ 3.33 GHz - (12 physical cores, 24 hyperthreaded)
Memory: 96 GB DDR3 @ 1066 MHz - (12 x 8 GB - 6 channels)
Motherboard: Asus Z8PE-D12
Hard Drives: 1 TB SATA II (Boot drive)
5 x 2 TB SATA II (Store Pi Output)
24 x 2 TB SATA II (Computation) - various models
Raid Controller: 3 x LSI MegaRaid SAS 9260-8i
Operating System: Windows Server 2008 R2 Enterprise x64
Built By: Shigeru Kondo
Pictures
: Click to enlarge.

Software: y-cruncher - A Multi-Threaded Pi Program/Benchmark

As with the hardware, the software side hasn't had any changes since our previous computation.

Software for Computation:

The program that was used for the main computation is y-cruncher v0.5.5.

Multiple builds were used between the different times that the computation was interrupted.

See main page: y-cruncher - A Multi-Threaded Pi Program

Version v0.5.5 is virtually identical to v0.5.4 as it consists only of a few bug fixes and adds a few improvements that are irrelevant to this computation.

Originally, our plan was to use v0.6.1 which is expected to be a significantly faster than v0.5.5. However, it was never completed in time for this computation. (and it still isn't complete...)

Software for Verification:

The program that was used for the verification is y-cruncher BBP v1.0.119.

See main page: y-cruncher BBP

This is the exact same binary that we used to verify our previous computation. There have been no changes at all.

This program implements the digit-extraction algorithm for Pi using the BBP formulas. It's sole purpose was to verify the main computation.

Both programs are written by me (Alexander J. Yee) and are available for download from their respective pages.

Methods/Formulas Used

Nothing has really changed from last time. See here for details.

10 Trillion Digits - Difficulties of this computation

More details coming later... after I finish my midterms...

Detailed Timeline (With Screenshots)

Date	Event	Screenshot
First Attempt
October 10, 2010	Start of Computation	View
October 16, 2010	Week 1: Series summation is past 6% complete.	View
October 23, 2010	Week 2: Series summation is past 13% complete.	View
October 30, 2010	Week 3: Series summation is past 17% complete.	View
November 6, 2010	Week 4: Series summation is past 22% complete.	View
November 14, 2010	Week 5: Series summation is past 28% complete.	View
November 20, 2010	Week 6: Series summation is still past 28% complete.	View
November 27, 2010	Week 7: Series summation is past 36% complete.	View
December 4, 2010	Week 8: Series summation is still past 36% complete.	View
December 9, 2010	Hard Drive Failure: Computation Failed A backup was scheduled for when the 36% - 47% block was completed. But a hard drive failed just before we got there...	View

Second Attempt
December 12, 2010	Start of Computation	View
December 18, 2010	Week 1: Series summation is past 6% complete.	View
December 25, 2010	Week 2: Series summation is past 13% complete.	View
January 1, 2011	Week 3: Series summation is past 17% complete.	View
January 8, 2011	Week 4: Series summation is past 22% complete.	View
January 15, 2011	Week 5: Series summation is past 28% complete.	View
January 22, 2011	Week 6: Series summation is still past 28% complete.	View
	Checkpoint 4: Manual backup at 36%.
January 29, 2011	Week 7: Series summation is past 36% complete.	View
February 5, 2011	Week 8: Series summation is still past 36% complete.	View
February 6, 2011	Hard Drive Read Error: Roll back to Checkpoint 4.	View
February 11, 2011	Hard Drive Write Error: Roll back to Checkpoint 4. All hard drives of the same model as the failed drive are replaced.
February 12, 2011	Week 9: Series summation is past 36% complete.	View
February 19, 2011	Week 10: Series summation is still past 36% complete.	View
February 26, 2011	Week 11: Series summation is still past 36% complete.	View
	Checkpoint 3 Reached: Manual backup at 47%.
March 5, 2011	Week 12: Series summation is past 47% complete.	View
March 11, 2011	The Earthquake... For a moment I pause to grasp what has happenned in Japan... And the relief to have felt when I finally got that email from Shigeru Kondo telling me that's he's okay. Makes us wonder why we were still running this...
March 12, 2011	Week 13: Series summation is still past 47% complete. If you're wondering how this was still running through all the power outages, Japan is split into two power grids. Only the eastern grid (50 Hz) was affected by the earthquake. The western grid (60 Hz) was not affected.	View
	Hard Drive Read Error: Roll back to Checkpoint 3. (17 days lost)
March 19, 2011	Week 14: Series summation is past 47% complete.	View
March 27, 2011	Week 15: Series summation is still past 47% complete.	View
	Hard Drive Write Error: Roll back to Checkpoint 3.
April 2, 2011	Week 16: Series summation is past 47% complete.	View
April 10, 2011	Hard Drive Failure: Roll back to Checkpoint 3. (again...)
April 16, 2011	Week 18: Series summation is past 47% complete.	View
April 23, 2011	Week 19: Series summation is still past 47% complete.	View
April 30, 2011	Week 20: Series summation is still past 47% complete.	View
May 2, 2011	Hard Drive Read Error: The program is able to recover and continue...	View
May 7, 2011	Hard Drive Read Error: Unrecoverable this time, roll back to Checkpoint 3. All hard drives of the same model as the failed drive are replaced.	View
May 14, 2011	Week 22: Series summation is past 47% complete.	View
May 21, 2011	Week 23: Series summation is still past 47% complete.	View
	Checkpoint 2 Reached: Manual backup at 60%.
May 28, 2011	Week 24: Series summation is past 60% complete.	View
June 4, 2011	Week 25: Series summation is still past 60% complete.	View
June 12, 2011	Week 26: Series summation is still past 60% complete.	View
June 18, 2011	Week 27: Series summation is still past 60% complete.	View
June 25, 2011	Checkpoint 1 Reached: Manual backup at 77%.	View
June 25, 2011	Week 28: Series summation is 77% complete.	View
July 2, 2011	Week 29: Series summation is past 77% complete.	View
July 10, 2011	Week 30: Series summation is still past 77% complete.	View
July 16, 2011	Week 31: Series summation is still past 77% complete.	View
July 23, 2011	Week 32: Series summation is still past 77% complete.	View
July 30, 2011	Week 33: Series summation is still past 77% complete.	View
August 4, 2011	Checkpoint 0 Reached: Manual backup at end of Series Summation.
August 6, 2011	Week 34: Final Division step of Series Summation.	View
August 13, 2011	Week 35: Final Multiplication step of Series Summation.	View
August 17, 2011	Series Completion Checkpoint: Manual backup at end of Series Computation
August 20, 2011	Week 36: Iterating Square Root.	View
August 26, 2011	Computation of Pi Complete: Output 8.3 trillion hexadecimal digits of Pi. The digits match the BBP results!!!	View View
August 27, 2011	Week 37: Base Conversion...	View
September 3, 2011	Week 38: Base Conversion...	View
September 10, 2011	Hard Drive Read Error: Roll back to end of series computation.	View
September 18, 2011	Week 40: Final Multiply...	View
	Pi Computation Checkpoint: Manual backup at end of Pi computation.
September 25, 2011	Week 41: Base Conversion...	View
October 1, 2011	Week 42: Base Conversion...	View
October 8, 2011	Week 43: Base Conversion...	View
October 11, 2011	Base Conversion Complete: Output 10 trillion decimal digits of Pi. The digits match our previous 5 trillion digit computation.	View View View
October 16, 2011	Verification Complete: End Computation of 10 trillion digits of Pi!!!	View

Questions or Comments

Alexander Yee - [email protected]
Shigeru Kondo - [email protected]

You can also find me on XtremeSystems Forums under the username: poke349

Back To:

Numberworld Home

Aug	OCT	Nov
	01
2017	2018	2019