From Ancient Algorithms to Modern AI: The Evolution of Numerical Methods

Imagine trying to predict the complex flow of water through an irregular riverbed, or calculating the exact heat distribution across a massive engine component. While standard textbook problems offer neat, easily solvable equations, the real world is fundamentally messy. Friction, fluid dynamics, continuous variables, and irregular geometries make finding a "perfect," exact mathematical formula nearly impossible in most practical scenarios.

This is exactly where numerical methods step in. When traditional analytical mathematics hits a wall and cannot provide a closed-form exact solution, we turn to the science of approximation. Numerical methods are essentially step-by-step algorithms that break down incredibly complex continuous mathematics into manageable, discrete arithmetic operations. They do not give us an absolutely perfect answer, but they provide an approximate solution that is accurate enough to construct massive bridges, predict weather patterns, and program realistic 3D physics simulations. They act as the essential bridge between abstract mathematical theory and real-world problem-solving.

It is a common misconception that numerical methods are an entirely modern or Western invention. In reality, the roots of algorithmic thinking lie deep within Ancient Indian Mathematics. Long before the advent of calculators, Indian mathematicians were developing step-by-step iterative methods to solve complex geometrical and astronomical problems.

The earliest evidence of this algorithmic thinking is found across ancient Indian knowledge systems. Jain mathematicians, while mapping the incredibly complex and vast dimensions of the universe—such as the Jambudweep model in Karnanuyog—extensively used numerical approximations. For instance, early Jain texts utilized $\sqrt{10}$ as an established numerical approximation for $\pi$ to calculate massive circular circumferences where exact measurement was impossible. Parallel to these cosmological calculations, geometric approximations were also recorded in the Sulba Sutras, where mathematicians calculated the approximate value of $\sqrt{2}$ up to five decimal places for constructing specific structures.

However, as we explored in our previous article, The Genesis of Zero, none of these advanced numerical calculations would have been practically possible without the Indian invention of the decimal place-value system and zero. Imagine trying to run a repetitive algorithmic calculation using Roman numerals—it is practically impossible!

The pinnacle of this era was reached by the Kerala School of Mathematics, particularly Madhava of Sangamagrama (14th century). Centuries before the European Renaissance, Madhava developed infinite series to calculate the highly accurate value of $\pi$. To achieve this numerically, Madhava geometrically divided the circumference of a circle into continuously smaller arcs and calculated their chord lengths recursively. He identified a mathematical pattern that formed an infinite series:

$\frac{\pi}{4} = 1 - \frac{1}{3} + \frac{1}{5} - \frac{1}{7} + \dots$

What makes Madhava a true pioneer of numerical analysis is that he didn't just stop at the infinite series. Knowing that it is impossible to calculate up to infinity, he introduced a mathematical "correction term" (remainder). This allowed him to truncate the calculation after a few steps and still compensate for the error, achieving the value of $\pi$ accurate to 11 decimal places. Instead of looking for an exact, closed-form equation, Madhava used this iterative numerical approximation—proving that the foundational principles of today's computational mathematics were established in India centuries ago.

As centuries passed, the mathematical landscape experienced a seismic shift in the 17th century with the simultaneous invention of Calculus by Isaac Newton and Gottfried Wilhelm Leibniz. This era marked the dawn of analytical exactness. Suddenly, mathematicians were no longer limited to approximations or iterative steps; they had a profound new framework to calculate instantaneous rates of change (derivatives) and exact accumulated areas (integrals).

Calculus gave scientists the confidence to model the physical world with absolute precision. Through differential equations, Newton could elegantly describe the exact two-body gravitational motion of planets using simple, closed-form analytical formulas. This spectacular success created a paradigm shift in scientific thinking. A belief took root that the entire universe was governed by deterministic mathematical laws, and for every physical problem, an "Exact Analytical Solution" could be derived if one just worked hard enough on the algebra.

During this golden era of exactness, the ancient art of numerical approximation was pushed slightly into the background. Why settle for an approximate answer when calculus promised a mathematically perfect, universally true equation? The pursuit of pure, theoretical exactness became the ultimate goal of mathematics.

The dream of finding an exact analytical solution for every physical phenomenon was elegant, but it inevitably clashed with the chaotic reality of nature. The fundamental difference between analytical and numerical methods lies in their end goal: analytical methods seek a perfect, closed-form mathematical formula (like solving $x^2 - 4 = 0$ to get exactly $x = \pm 2$), whereas numerical methods seek a highly accurate calculated number through repetitive arithmetic.

Calculus is flawless in theory and works perfectly for idealized, highly controlled scenarios—such as calculating the gravitational interaction between exactly two celestial bodies. However, add just one more planetary body to the mix (the infamous 3-Body Problem), and the analytical equations become chaotic and practically impossible to solve. The same roadblock appears in advanced applied physics and aerospace engineering. If you attempt to model the turbulent airflow over an aircraft wing or solve the Navier-Stokes equations to understand fluid dynamics, you will quickly find that no exact mathematical formula exists to give you a direct answer.

When pure analytical mathematics hits this definitive wall, Numerical Methods make their grand return. Instead of fruitlessly searching for an impossible exact equation, numerical methods take the complex, continuous problem and discretize it—breaking it down into millions of tiny, manageable calculations (often visualised as meshes or grids). It is the ultimate engineering realization: in the real world, a robust, highly accurate approximation is infinitely more valuable than a non-existent perfect formula.

When the limitations of pure analytical mathematics became undeniably apparent, the very pioneers who championed exact equations realized they had to embrace approximations. During the European mathematical renaissance, the focus shifted back to algorithmic and iterative methods to solve the problems that calculus equations alone could not handle.

Interestingly, the foundational tool for finding the roots of complex algebraic equations—the Newton-Raphson method—arose from this era. Isaac Newton derived an early version of the iterative algorithm in 1669, which was later refined and simplified by mathematician Joseph Raphson in 1690. Similarly, Leonhard Euler introduced Euler's Method to approximate solutions for complex differential equations step-by-step, while Carl Friedrich Gauss formalized Gaussian Elimination, a systematic numerical procedure for solving large sets of linear equations.

However, these brilliant classical methods came with a massive practical bottleneck: they required hundreds or even thousands of repetitive arithmetic calculations (iterations). Before the invention of electronic computers, these loops had to be performed entirely by hand. Mathematicians and human "computers" spent weeks, months, or even years manually calculating tables, filling massive notebooks with numbers just to solve a single engineering or astronomical problem. The theoretical algorithms were perfect, but humanity simply lacked the computational speed to unlock their true potential.

The grueling manual calculations of the classical era finally found their perfect match in the mid-20th century with the invention of electronic computers. The digital revolution did not just assist numerical methods; it fundamentally transformed the entire discipline. Machines do not get fatigued, they do not make manual copying errors, and they can execute millions of repetitive loops (iterations) in a mere fraction of a second.

The breakthrough came with the integration of "floating-point arithmetic," a digital architecture that allowed computers to handle both astronomically large and infinitesimally small fractional numbers with incredible precision. Algorithms that were once considered theoretical curiosities because they were simply too tedious to calculate by hand—such as large-scale matrix inversions or the complex Runge-Kutta integration methods—suddenly became everyday practical tools. Numerical methods and computers became a match made in mathematical heaven. Calculations that would have taken a room full of human mathematicians a lifetime to solve were now executed by silicon chips in a matter of seconds, paving the way for modern software engineering.

Fast forward to today, and numerical methods are no longer just an academic subject; they are the invisible engines powering our modern digital world. Every advanced technology we interact with today is fundamentally built upon these iterative mathematical algorithms.

Consider Machine Learning and Artificial Intelligence. When a complex Neural Network is trained, the primary goal is to minimize error. The absolute backbone of this learning process is an algorithm called 'Gradient Descent'. Despite the buzzwords, Gradient Descent is, at its core, a classic numerical optimization method designed to iteratively find the lowest point (minimum error) in a multi-dimensional mathematical landscape.

The same principle applies to computer graphics and interactive educational tools. Whether you are rendering a complex 3D physics animation in Blender or building a 2D interactive canvas for optics simulations in a Next.js web application, the computer does not solve exact, closed-form analytical physics equations. Instead, it relies on numerical integration (like the Verlet or Euler methods) to calculate light tracing, object collisions, and kinematics frame-by-frame.

Even macro-level global challenges rely entirely on this science. Modern weather forecasting involves massive supercomputers processing billions of non-linear fluid dynamics equations simultaneously. Because an exact analytical solution for the Earth's atmosphere simply does not exist, Numerical Weather Prediction (NWP) discretizes the atmosphere into 3D grids, applying numerical approximations to predict tomorrow's rain or storm with remarkable accuracy.

From the vast cosmological approximations of ancient Jain texts and Madhava’s infinite series to the sophisticated neural networks running on modern silicon chips, the journey of numerical methods has come full circle. The foundational algorithmic thinking laid down by ancient Indian mathematicians—the philosophy of breaking down seemingly unsolvable problems into iterative, manageable steps—is exactly what powers today’s artificial intelligence and computational physics.

The ultimate takeaway from this historical evolution is a profound shift in how we view mathematics. Mathematics is far more than just memorizing static textbook formulas or endlessly hunting for an elusive, "perfect" exact equation. It is a dynamic, continuous journey of practical computation. When pure theory reaches its limits in the face of real-world complexities, it is the art of numerical approximation that steps forward, proving that the true essence of mathematics lies in finding robust, actionable solutions for the world around us.