The Agile Architect

Scientists use Big Data Techniques to Solve for Pi

Big Data analysis is being used to solve problems never before attempted. Our Agile Architect reports on a new technique that is groundbreaking in the breadth of its application.

• By Mark J. Balbes, Ph.D.
• April 1, 2022

Big Data is everywhere these days. It’s become ubiquitous with modern software development. Using techniques developed originally by Google, Big Data allows for rapid analysis of large, unstructured data sets by bringing compute power to the data rather than bringing the data to the computer. Free from the restrictions of structured data sets, Big Data can solve problems previously undreamed of.

This has never been truer than today when scientists at Saisho Tsuki University in Haifa, Israel announced that they are using Big Data techniques to calculate the final digit of Pi. “We are almost there,” wrote data scientist Robin Mosby in the research journal Some Other Times.

Calculating the final digit of Pi

I was quite taken with the article and decided to contact Mosby directly to see if she would agree to be interviewed. To my surprise, she was quite familiar with "The Agile Architect," and quickly agreed to a video chat with her entire team.

Mosby started by explaining that her big breakthrough came a year ago.

“I was walking my dogs when I had the first of a series of epiphanies," she told me. "At the time, our team was using Big Data techniques to analyze oceanic forensics. By instrumenting the Atlantic Ocean with a multiplicity of cheap compute devices connected to sensors, we could ask a question of our system, fan it out to our devices, and construct the answer from the results almost instantly. I was thinking about this amazing achievement when I thought how wonderfully everything could be accomplished just by manipulating the zeros and ones in a suite of cheap, modern compute devices.”

Researcher April Tauschen, a member of Mosby's team, continued, shared her personal reaction to her team leader's realization.

“When Robin shared her thoughts with me the next day," she said, "I was stunned. All the answers to all the questions in the universe can be found in the zeros and ones that are already in our compute devices, if only there were some way to mine the data for the answers.”

“And, of course, there is!" said Fred Mertz, emeritus researcher who runs the lab next to Robin’s. "We use Big Data techniques all the time to mine large data sets. So, we set out, determined to solve a big problem by mining the zeros and ones in the computer. We chose to solve for Pi exactly. Everyone thinks it is impossible, but we realized we could use Robin’s new technique to approach the problem differently. Rather than calculating from the most significant digit down, we decided to start from the last digit and work our way up.”

Mosby concluded our interview with a surprising revelation.

“I am very pleased to tell you that we have, in fact, calculated an upper limit on the last digit of Pi," she said. "Pretty amazing feat when you consider that Pi is infinite, right? But here’s the important part. While Pi is infinite in length, we have calculated with no uncertainty that the last digit of Pi is a member of a finite set with elements {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}.

"And that, my dear Agile Architect and all of your wonderful readers,” Mosby added with a flourish, “shows the tremendous power of our groundbreaking technique!”

Final Thoughts

Robin sent me a short note a few days after our chat. It cryptically teased at another major breakthrough. “By transforming the problem from Base 10 to Base 2," the note read, "we believe we can significantly narrow the solution set. Stay tuned!”

As technology progresses, scientists will discover new and innovative ways to solve problems that were once thought impossible. With this new technique for data mining the zeros and ones already in our computers, solving for the last digit of Pi will seem like child’s play. We would be fools to think otherwise.