The theory of chaos
In 1974 Mitchell Feigenbaum was working on a deep problem – chaos.
At Los Alamos, a Center for Nonlinear Studies was established to coordinate work on chaos and related problems.
Chaos poses problems that defy accepted ways of working in science. It makes strong claims about the universal behavior of complexity.
The twentieth-century science will remember only three things: relativity, quantum mechanics, and chaos.
Relativity eliminated the Newtonian illusion of absolute space and time; quantum theory eliminated the Newtonian dream of a controllable measurement process; and the chaos eliminates the Laplacian fantasy of deterministic predictability.
The modern study of chaos began with the creeping realization in the 1960s that quite simple mathematical equations could model systems every bit as violent as a waterfall. Tiny difference in input could quickly become overwhelming differences in output – a phenomenon given the name “sensitive dependencies on initial conditions”. The Butterfly Effect.
The Butterfly Effect
Edward Lorenz’s created a toy weather in 1960 that succeeded in mesmerizing his colleagues.
Understand the laws and you understands the universe. That was the philosophy behind the modeling weather on a computer.
The fathers of modern computing always has Laplace in mind, and in the 1950s, VonNeumann recognized that weather modeling could be an ideal task for computer. Given an approximate knowledge of a system’s initial condition and an understanding of natural law, one can calculate the approximate behavior of the system.
Two technologies were maturing together, the digital computers and the space satellite.
Von Neumann recognized that a complicated dynamical system could have points of instability – critical points where small push can have a large consequences.
Weather forecasting was the beginning but hardly the end of the business of using computers to model complex systems. But beyond two or three days the world’s best forecasts were speculative, and beyond six or seven days they were worthless. The Butterfly Effect was the reason.
Lorenz turned his attention more and more to the mathematics of systems that never found a steady state.
The Butterfly Effect acquired a technical name: sensitive dependence on initial conditions.
In science as in life, it is well known that a chain of events can have a point of crisis that could magnify small changes.
Lorenzo put the weather aside and looked for even simpler ways to produce this complex behavior. He found one in a system of just three equations. They were nonlinear.
In fluid dynamics, everything boils down to one canonical equation, the Navier-Stoker equation.
Lorenz took a set of equations for convection and stripped it to the bone. Almost nothing remained of the original model, but he did leave the nonlinearity.
Lorenz’s paper Deterministic: Nonperiodic Flow was hidden on the page 130 of volume 20 of the Journal of the Atmospheric Sciences. The mysterious curve depicited at the end, the double spiral that became known as the Lorenz attractor.
Revolution
Under normal conditions the research scientists is not an innovator but a solver of puzzles and the puzzles upon which he concentrates are just those which he believes can be both stated and solved within the existing scientific tradition. Then there are revolutions.
Those who recognized chaos in the early days agonized over how to shape their thoughts into publishable paper.
Topology studies the properties that remains unchanged when shapes are deformed by twisting or stretching or squeezing. Topology is geometry on rubber sheets. Both subjects, topology and dynamical systems, went back to Henri Poincare, who saw them as two sides of one coin. He was the first to understand the possibility of chaos.
Differential equations describe the way systems change continuously over time.
Any set of equations describing a dynamical system allows certain parameters to be set at the start.
Chaos and instability, concepts only beginning to acquire formal definitions, were not the same at all. A chaotic system could be stable if particular brand of irregularity persisted in the face of small disturbances. The chaos Lorenz discovered, with all its unpredictability, was as stable as a marble in a bowl. It was locally unpredictable, globally stable.
Stephen Smale is known for Smale’s Horseshoe. This topological transformation provided a basis for understanding the chaotic properties of dynamical systems. A space is stretched in one direction, squeezed in another, and then folded.
Life’s Ups and Downs
Mathematically inclined biologists of the twentieth century build a discipline, ecology, that stripped away the noise and color of real life and treated populations as dynamical systems. In the emergence of chaos as a new science in the 1970s, ecologists were destined to play a special role.
Differential equations describe processes that change smoothly over time, but differential equations are hard to compute. Simpler equations – difference equations – can be used for processes that jump from state to state.
Malthusian version: xnext = rx (1-x). By the 1950s several ecologists were looking for at variations of that particular equation, known as the logistic difference equation.
People would say that James Yorke discovered Lorenz and given the science of chaos its name. The second part was actually true.
Yorke gave Lorenz’s paper to Smale.
Nonlinear systems with real chaos were rarely taught and rarely learned.
Enrico Fermi once exclaimed: “It does not say in the Bible that all laws of nature are expressible linearly.”
Robert May was biologist working on the simplest ecological questions of how single populations behave over time. What happens when a population growth rise above certain points? Beyond a certain point, the “point of accumulation”, periodically gives way to chaos, fluctuations that never settle down at all.
The blossoming of chaos in the US and Europe has inspired a huge body of parallel work in the Soviet Union. Yasha Sinai and others assembled a powerful working group of physicists in Gorki.
Chaos brought an astonishing message: simple deterministic models could produce what looked like random behavior. The behavior actually had an exquisite fine structure, yet any piece of it seemed indistinguishable from noise.
A Geometry of Nature
Benoit Mandelbrot was a mathematical jack-of-all-trade. He was in Hendrik Houthakker’s office when he recognized the picture of reality. He helped Houthakker with cotton prices.
In the history of chaos Mandelbrot made his own way. He is best understood as a refugee.
Bourbaki began as a club of mathematicians. It began as reaction to Poincare. The group stressed the primacy of mathematics among sciences.
Mandelbrot joined the IBM. He ventured into mathematical linguistics, explaining a law of the distribution of words. He investigated game theory.
The Noah Effect means discontinuity: when a quantity changes, it can change almost arbitrarily fast. The Joseph Effect means persistence. There comes seven years of great plenty throughout the land of Egypt.
Since Euclidean measurements – length, depth, thickness – failed to capture the essence of irregular shapes, Mandelbrot turned to a different idea, the idea of dimension. Dimension is a quality with a much richer life for scientists than for non-scientists. He moved beyond dimensions 0, 1, 2, 3, … to a seeming impossibility: fractional dimensions. Fractional dimensions becomes a way of measuring qualities that otherwise have no clear definition: the degree of roughness or brokenness or irregularity in an object.
The claim was that the degree of irregularity remains constant over different scales.
The Koch curve has some interesting features. It is a continuous loop, never intersecting itself. It is infinitive long. Helge von Koch, the Swedish mathematicians first described it in 1904.
For Koch curve Mandelbrot could characterize the fractional dimension precisely. It was 1.2618.
Self-similarity is symmetry across scale.
Cristopher Scholz was working on form and structure of solid earth and came across Mandelbrot and fractal techniques.
Not immediately, but a decade after Mandelbrot published his psychological speculations, some theoretical biologists began to find fractal organization controlling structures all through the body.
Scientists could hardly avoid the word fractal, but if they wanted to avoid Mandelbrot’s name they could speak of fractional dimension as Hausdorff-Besicovitch dimension.
Mandelbrot found his most enthusiastic acceptance among applied scientists working with oil or rock or metals, particularly in corporate research centers.
Scaling became part of a movement in physics that led more directly than Mandelbrot’s own work, to the discipline known as chaos.
Strange Attractors
Turbulence was a problem with pedigree.
In the 1930s A. N. Kolmogorov put forward a mathematical description that gave some feeling on how these eddies work. He imagined the whole cascade of energy down through smaller and smaller scales until finally a limit is reached, when the eddies become so tiny that the relatively larger effects of viscosity take over.
Closely related, but quite distinct, was the question of what happens when turbulence begins. How does a flow cross the boundary from smooth to turbulent? This orthodox paradigm came from Lev D. Landau.
In high energy physics glory goes to the theorists, while experimenters have become highly specialized technicians, managing expensive and complicated equipment.
Harry Swinney was experimenting with stuff. He designed an apparatus to measure how well carbon conducted heat around the critical point where it turned from vapor to liquid. Swinney found that conductivity changed by a factor of 1.000.
Other experimenters were Jerry Gollub, Gunther Ahlers, Reynolds and Rayleigh.
David Ruelle wrote a paper together with Floris Takens, On the Nature of Turbulence. They proposed that just three independent motions would produce the full complexity of turbulence.
Every orbit must eventually end up at the same place, the center: position 0, velocity 0. This central fixed point “attracts” the orbits. Instead of looping around forever, they spiral inward. One advantage of thinking of states as points in space is that it makes changes easier to watch.
The simple equations Rober May studied were one-dimensional – a single number was enough. Lorenz’s stripped-down system of fluid convection was three-dimensional. It took three distinct numbers to nail down the state of the fluid at any instant.
Mathematicians had to accept the fact that systems with infinitely many degrees of freedom required a phase space of infinite dimensions.
The attractor was stable, low-dimensional, and nonperiodic.
The Poincare map removes a dimension from an attractor and turns a continuous line into a collection of points.
The most illuminating strange attractor, because it was simple, came from an astronomer, Michael Henon. He was working on the three-body problem. Two-body problem is easy, solved by Newton. Each body – the earth and the moon, travels in the perfect ellipse around the system’s joint center of gravity. Add another body it gets complicated.
Henon heard about strange attractors from Lorenz and Ruelle. Where Lorenz and others had stuck to differential equations, he turned to difference equations, discrete in time.
Universality
Mitchell Feigenbaum was the next one working on the chaos. Peter Carruthers hired him to Los Alamos National Laboratory. He also hired Keneth Wilson. He studied the phase transition of matter. He did the work that brought the whole theory together under the rubric of renormalization group theory.
Renormalization had entered physics in the 1940s as a part of quantum theory that made it possible to calculate actions of electrons and photons. It was devised by Richard Feynman, Julian Schwinger and Freeman Dyons.
Lorenz described another kind of behavior called “almost-intransitivity”. An almost-intransitive system displays one sort of average behavior for a very long time, fluctuating withing certain bounds. Then, for no reason whatsoever, it shifts into a different sort of behavior, still fluctuating but producing a different average.
The Ice Age may simply be a byproduct of chaos.
Certain region was like a mysterious boundary between smooth flow and turbulence in a fluid. But it was unexpected regularity hidden in this system. Feigenbaum calculated the ratio of convergence to the finest precision possible. He came up with a number, 4.669. Robert May realized later that he too had seen this geometric convergence.
With the first period-doubling, the attractor splits in two, like a dividing cell.
Zeroing in on chaos. A simple equation repeated many times over: Mitchel Feigenbaum focused on straightforward functions, taking one number as input and producing another as output.
Feigenbaum had created an universal theory. Universality made the difference between beautiful and useful.
There is a fundamental presumption in physics that the way you understand the world is that you keep isolating its ingredients until you understand the stuff that you think is truly fundamental. The assumption is that there are a small number of principles that you can discern by looking at things in their pure state.
The Experimenter
Albert Libchaber and Jean Maurer set out in 1977 to build an experiment that would reveal the onset of turbulence.
Libchaber plan was to create convection in the liquid helium by making the bottom plate warmer than the top plate. It was exactly the convection model described by Lorenz. The classical system known as Rayleigh-Benard convection. Libchaber was not aware of Lorenz, not yet. Nor had he any idea of Mitchell Feigenbaum’s theory.
Flow was shape plus change, motion plus form. Flow was Platonic idea, assuming that change in systems reflected some reality independent of the particular instant.
Fluid motion, from smooth flow to turbulence, is thought of as motion through space.
Libchaber froze the space so that he could play with the time. Once the experiment began, the helium rolling inside the cell inside the vacuum container inside the nitrogen bath.
So far, Libchaber was reproducing a well-known experiment in fluid mechanics. It also happened to be precisely the flow that Lorenz had modeled with his system of three equations.
Feigenbaum’s scaling theory predicted not only when and where the new frequencies would arrive but also how strong they would be – their amplitudes.
It was Pierre Hohenberg of AT&T that brough theory and experiment together. He connected Libchaber and Feigenbaum.
The leap from maps to fluid flow seemed so great that even those most responsible sometimes felt it was like a dream.
The experimenters’ discoveries helped set in motion the era of computer experimentation.
Simulations break reality into chunks, as many as possible but always too few.
according to the new theory, the bifurcations should have produced a geometry with precise scaling, and that was just what Libchaber saw, the universal Feigenbaum constants turning in that instant from a mathematical ideal to a physical reality, measurable and reproducible.
Images of Chaos
Michael Barnsley met Feigenbaum in 1979. He had an idea where the numbers, Feigenbaum’s sequences come from. He had a context, the numerical territory known as complex plane. Each number is composed of two parts. A real part, corresponding to east-west longitude, and an imaginary part, corresponding to north-south latitude.
David Ruelle told Barnsley that he is talking about Julia sets and that he should get in contact with Mandelbrot.
The Mandelbrot set is the most complex object in mathematics, its admirers like to say.
Many fractal shapes can be formed by iterated process in the complex plane, but there is just one Mandelbrot set.
Julia sets were class of shapes invented and studied by Gaston Julia and Pierre Fatou.
In 1979 Mandelbrot discovered that he could create one image in the complex plane that would serve as a catalogue of Julia sets, a guide to each and every one. The Mandelbrot set is a collection of points.
Julia, Fatou, Hubbard, Barnsley, Mandelbrot – these mathematicians changed the rules how to make geometrical shapes.
The boundary is where a Mandelbrot set program spend most of its time and makes all of its compromises.
Barnsley turned to randomness as the basis for a new technique of modeling natural shapes. He called it “the chaos game”. Barnsley’s central insight was this: Julia sets and other fractal shapes, though properly viewed as the outcome of a deterministic process, has a second valid existence as the limit of a random process. In Barnsley’s technique chance serves only as a tool. The results are deterministic and predictable.
The Dynamical Systems Collective
The education of physicists depends on a system of mentors and proteges. In 1977 chaos offered no mentors.
Edward A. Spiegel knew Lorenz personally, and he had known about chaos since the 1960s.
At Santa Cruz the group of researchers called itself the Dynamical Systems Collective. They were grouped around Robert Stetson Shaw. Doyne Farmer was the group’s most articulate spokesman. There were also Norman Packard and James Crutchfield.
Farmer said that the same thing drew them all, the notion that you could have determinism but not really.
The Russian conception, the Lyapunov exponent provided a way of measuring the conflicting effects of stretching, contracting, and folding in the phase space of an attractor. An exponent greater than zero meant stretching – nearby points would separate. An exponent smaller than zero meant contraction. A strange attractor, it turned out, had to have at least one positive Lyapunov exponent.
In the late 1940s, Claude Shannon invented an information theory. Hardware determined the shape of the theory. Bits became the basic measure of information.
In terms of Shannon’s information theory, ordinary language contain greater than fifty percent redundancy in the form of sounds or letters that are not strictly necessary to conveying a message. Redundancy is a predictable departure from the random.
To Robert Shaw, strange attractors were engines of information. They gave a challenging twist to the question of measuring a system’s entropy. They created unpredictability. They raised entropy. And as Shaw saw it, they created information where none existed.
The information theory framework allowed the Santa Cruz group to adopt a body of mathematical reasoning that had been well investigated by communication theorists.
Crutchfield, Farmer and Packard started collaborating with established physicists and mathematicians: Huberman, Swinney, Yorke.
Fractal dimension, Hausdorff dimension, Lyapunov dimension, information dimension – the subtleties of these measures of a chaotic system were best explained by Farmer and York.
Inner Rhythms
“The sciences do not try to explain, they hardly even try to interpret, they mainly make models. By a model is meant a mathematical construct which, with he addition of certain verbal interpretations describes observed phenomena.” John Neumann[1]
The choice is always the same. You can make your model more complex and more faithful to reality, or you can make it simpler and easier to handle.
David Ruelly had strayed from formalism to speculate about chaos in the heart – a dynamical system of vital interest to every one of us – as he wrote.
Irregularities in the heartbeat has long since been discovered, investigated, isolate and categorized.
The heart will not stop fibrillating on its own. This brand of chaos is stable. Only a jolt of electricity from a defibrillation device can return the heart to its steady state. Determining the size and shape of that jolt was something Arthur T. Winfree worked on.
When you reach an equilibrium in biology you’re dead according to Arnold Mandell.
Schrodinger’s view was unusual. That life was both orderly and complex was a truism. In Schrodinger’s day, neither mathematics nor physics provided any genuine support for the ideas. There were no tools for analyzing irregularity as a building block of life. Now those tools exist.
Chaos and Beyond
Simple systems behave in simple ways.
Complex behaviors implies complex causes.
Different systems behave differently.
Now all that has changed. In the intervening twenty years, physicists, mathematicians, biologists and astronomers have created an alternative sets of ideas. Simple systems give rise to complex behaviors. Complex systems give rise to simple behaviors. And more important, the laws of complexity hold universally.
The second law of thermodynamics. Everything tends towards disorder. The universe is a one-way street. Entropy must always increase in the universe and in any hypothetical isolated system within it.
Nature forms patterns. Some are orderly in space but disorderly in time, others orderly in time but disorderly in space. Some patterns are fractal, exhibiting structures self-similar in scale. Others give rise to steady states or oscillating ones.
Joseph Ford said that evolution is chaos with feedback. As a reply to Einstein he also said that god plays dive with the universe, but they’re loaded dice.
[1] In the book on page 273
