The theory that escaped Einstein


Rodney A. Brooks





Epsilon Publishers

2075 Whippet Way

Sedona, AZ 86336






In this final chapter we present a summary and overview of Quantum Field Theory. We will describe its structure and its achieve­ments. Out of fairness, we will also describe the gaps in the theory. We will start with the three pillars on which this amazing edifice rests.




The field principle. The first pillar is the assumption that nature is made of fields. A field is a set of physical properties that exist at every point of space. However, the concept of a field as a property of space does not come easily. It eluded the great Newton, who could not accept action-at-a-distance. It wasn’t until 1845 that Faraday, inspired by patterns of iron filings, conceived the idea of fields, and it took another 50 years before the concept was accepted without invoking an imaginary ether. The use of colors is my attempt to make the field picture more palatable.

Once you are comfortable with the field picture of nature, I think you will find it more satisfying than the particle picture. While particles are more natural to our thinking, they raise the question: what’s inside them? What is an electron made of? There must be something inside it. The concept of a point particle – a singularity in space – is even more disturbing. I hope you will agree with me that space with properties makes more sense than particles placed in space.

The quantum principle (discretization). The quantum principle was intro­duced in 1900 by Max Planck, who showed that EM radiation is emitted from hot objects in discrete units called quanta. A further step in discretization was made in 1922 by Otto Stern and Walther Gerlach. Their classic experiment (Fig. 10-1) showed that the angular momentum (or spin) of the electron can have only two values — nothing in between.



Fig. 10-1. The Stern-Gerlach experiment. A beam of atoms is deflected by a non-uniform magnetic field. Classically one would expect that the atom’s magnetism (which arises from the spin of its electrons) could have any magnitude and that the beam would be deflected into a continuous band. Instead, the beam separates into two distinct parts, corresponding to discrete spin values of +½ and -½ Planck units.



The relativity principle. The third pillar is the assumption that the field equations are the same for all uniformly-moving observers. This is Ein­stein’s Principle of Relativity, enunciated in 1905. QFT is the only theory that successfully combines the relativity and quantum principles.

Occam’s Razor. I’m tempted to add another pillar, but it’s really more of a wish than a rule. I’m referring to Occam’s razor: “No more things should be presumed to exist than are absolutely necessary”, attributed to William of Occam (1285-1349). In other words, all other things being equal, the simplest explanation is the best. Einstein put it somewhat differently: “A physical theory should be as simple as possible, but no simpler.” The last phrase is important because, as Julian Schwinger said, “nature does not always select what we, in our ignorance, would judge to be the most symmetrical and harmonious possibility”. If the theory were as simple as possible, there would be just one field (or perhaps none!), and the world would be very uninteresting — not to mention uninhabitable. I think it can be said that the equations of QFT are indeed about as simple as possible, but no simpler.


The move from a particle description to a field description will be especially fruitful if the fields obey simple equations, so that we can calculate the future values of fields from the values they have now… Maxwell’s theory of electromagnetism, general relativity, and quantum chromodynamics all have this property. Evidently, Nature has taken the opportunity to keep things relatively simple by using fields. – F. Wilczek (W2008, p. 86)





On these pillars rests the most successful theory ever constructed – a theory that explains every­thing from the tiniest atomic nucleus to the most remote star. We will now describe its essential features.

The fields. There are seven fields in QFT — five force fields and two matter fields. The force fields include gravity, electromagnetic forces, strong and weak nuclear forces, and the recently-discovered Higgs field. The matter fields include the lepton and baryon fields.[1] Each of these fields is actually a group of interrelated “sub-fields” that share the same equations. For example, the EM field consists of electric and magnetic fields that are described by Maxwell’s equations.

Spin. In QFT there are no particles, so there is nothing to spin on its axis. Thus the concept of spin is more abstract and not easy to picture. Because of this, the term helicity is sometimes used instead. Helicity is a mathematical concept related to the number of field components and how they change when viewed from different angles. Despite its abstract definition, helicity gives rise to angular momentum just as does a spinning particle. The force fields have integer spin or helicity: 0 for the strong field and Higgs field, 1 for the EM and weak fields, and 2 for the gravitational field. The matter fields (lepton and baryon) have a spin of ½, and this half-integer spin leads to the Exclusion Principle (see below). This is why quanta of the matter fields act more like particles than do quanta of the force fields.

Hilbert algebra. In QFT the concept of discretization is extended to field strength. Even though the values are continuous, field strength is treated mathematically as the limit of increasingly finer discrete values. This means that quantum fields are not described by ordinary numbers. They are described by vectors in what mathematicians call Hilbert space, and their dynamics are described by operators in Hilbert space that obey partial differential equations. Because of this, we do not talk about field intensity as a simple number; we talk about its expectation value. How­ever, you do not need to understand Hilbert algebra, or even to know the field equations, to grasp the basic concepts of QFT. Just remember that Hilbert space is not real; it is a mathematical tool and Hilbert operators are not to be con­fused with the physical fields that exist in real three-dimensional space.

Quanta. Hilbert algebra, in turn, leads to field quantization. The fields of QFT are made of units called quanta. Each quantum is a separate, indivisible chunk of field that lives a life and dies a death of its own. For example, the photon is a quantum of the EM field and protons and neutrons are quanta of the baryon field. Quanta are sometimes called excitations in a field, but that term doesn’t do them justice. Excitations can have any magnitude. They can diminish as they travel and slowly die away like water waves, whereas quanta have a fixed energy that doesn’t change, no matter how spread-out the quantum is. Quanta may be free and travel through space on their own, or they may be bound, as an electron in an atom, but each quantum keeps its own identity. If it is absorbed or changes its spin state, it does so as a unit. This all-or-nothing behavior, of course, is reminiscent of particles, making it difficult for many physicists to give up their belief in particles.

Self-fields. Quanta are not the only kind of field in QFT; there are also self-fields. Self-fields do not have a life of their own; they are created by a source and remain permanently attached to that source. Examples of self-fields are the gravitational field of the earth (Fig. 2-5), the electric field around an electron (Fig. 6-7), and the strong field around a nucleon (Fig. 4-2).

The vacuum field. Finally, there is the vacuum field. The equations of quantum field theory do not permit the field strength to be zero. Even in regions where there are no quanta or self-fields, there is a background field called the vacuum field. The vacuum field is especially important in the case of the Higgs mechanism (see below).

Quantum collapse. If fields evolved only as described by the field equations, nothing of significance would happen; these equations don’t describe the transfer of energy or momentum. For example, they don’t describe how a photon transfers its energy to a photoreceptor in your eye. The process by which such transfers are made is called quantum collapse because after a quantum transfers its energy it must disappear from all space. It can’t continue to exist with no energy. Although we have no theory to describe quantum collapse, it is an essential part of QFT.[2] As Art Hobson wrote, referring to the two-slit experiment:

The entire spread-out field… must deposit its quantum of energy all at once, in a single instant, because the field cannot carry some fraction of one quantum – it must always contain either exactly one or exactly zero quanta of energy. When the field deposits its quantum of energy on the viewing screen, the entire spread-out field must instant­aneously lose this much energy. — A. Hobson (H2007, p. 303)

Quantum collapse is similar to collapse of the wave-function in QM, but the concepts are very different. The QM wave-function describes the proba­bility that a particle is in a given place or that the system is in a given state, and when a measurement is made the probabilities “collapse” into a certainty. In QFT the collapse is a physical process that actually happens. It may not be what we expected, but neither did we expect that the earth is round or that matter is made of atoms. Just as we learned to live with those concepts, we can learn to live with quantum collapse. As we saw in Chapter 9, quantum collapse offers a simple solution to both the measurement problem and the entanglement problem.

Quarks and gluons. Quarks and gluons are the basic fields that consti­tute the strong and baryon fields. However, because of the Principle of Confinement (see below), they do not exist in free form. Indeed, we would not have known about them if not for the large number of hadrons (quanta of the strong and baryon fields) that were discovered in the latter half of the 20th century – the so-called “subatomic zoo”. In 1961 Yuval Ne’eman (an Israeli physicist) and Murray Gell-Mann saw a pattern in the “zoo” that led Gell-Mann to predict a new particle (the omega-minus). Three years later Gell-Mann and George Zweig showed that this pattern would result if hadrons were made of more basic fields that Gell-Mann called quarks, a word taken from Ulysses by James Joyce.[3] Later gluons (again Gell-Mann’s term) were introduced to hold the quarks together. The field theory that describes quarks and gluons was given the name (by guess whom) quantum chromodynamics (QCD), because arbitrary colors are used to describe different kinds of quarks.[4] While QCD has its own name, it still is part of Quantum Field Theory.

Mass. In classical physics mass is a measure of inertia, but in QFT it is a number that appears in the field equations. The effect of the mass term is to slow down the speed at which a field evolves and propagates, so mass plays the same inertial role in QFT that it does in clas­sical physics. But this is not all that mass does. This same term also causes the fields to oscillate, and the greater the mass, the higher the frequency.[5]

Energy. In classical physics, energy means the ability to do work, and work is defined as force exerted over a distance. This definition, however, doesn’t provide much of a picture, so in classical physics energy is a rather abstract concept.  In QFT, on the other hand, the energy of a quantum is represented by oscillations in its field intensity as described by Planck’s famous equation (p. 54): the more energy, the faster the oscillations. Using our color analogy, we might say that the oscillations cause the color to “shimmer”, and the faster the shimmer, the greater the energy of the field.




We will now look at some of the achievements of QFT – achievements that emerge from the theory as easily as raindrops falling from the clouds, or better yet, like presents appearing under the Christmas tree. Some of these have already been mentioned and some will be pleasant surprises. We will start with the most famous equation in physics history.

E = mc2. I know, I promised there would be no equations and, except for a few footnotes, I’ve kept my promise. But I think you will forgive me for making an exception for the world’s most famous equation — the only equation to have its biography written (B2000). And the thing is this: E = mc2 pops right out of QFT. Einstein had to work hard to derive it; it was published in a sepa­rate paper, following his first paper on relativity theory in 1905. In QFT, this equation follows as a simple consequence of the two previous re­sults. Since both mass and energy are represented by oscillations in the field intensity, it doesn’t take an Einstein to see that there is a relationship between the two. In fact, any schoolboy can combine the two equations and find (big drum roll, please) E = mc2.[6] Not only does this equation tumble right out of QFT, its meaning is seen physically in the oscillations of the fields caused by both mass and energy. (This simple derivation only occurred to me as I was writing this book; I have not seen it elsewhere.)

The subatomic zoo. It was recently shown that the field equations for quarks and gluons account for the mass and spin of all the hadrons in the sub-atomic zoo.[7] To start the calculation, three of the measured masses were used to fix the parameters of the theory. Using these fixed parameters, stable and quasi-stable excitations of the fields were then calculated. Fourteen such excitations were found with mass and spin in close agreement with the fourteen known hadrons, ranging from the proton and neutron to the exotic charmonium. Equally important is that there were no excitations corresponding to the quarks and gluons themselves, thereby providing a theoretical basis for the Principle of Confinement. As Frank Wilczek wrote:

Through difficult calculations of merci­less precision that call upon the full power of modern computer technology, [we have] shown that unbendable equations… account convincingly and in quantitative detail for the existence of protons and neutrons, and for their properties… I believe this is one of the greatest scientific achievements of all time. — F. Wilczek (W2008, p. 127)

An achievement, I might add, that very few people are aware of.

Exclusion principle. When the exclusion principle was introduced by Wolfgang Pauli in 1925, it was only a guess — an empirical postulate. It is only in QFT that this important principle is given a theoretical basis.

In my original paper I stressed the circumstance that I was unable to give a logical reason for the exclusion principle or to deduce it from general assumptions… If we search for a theo­retical explana­tion of this law, we must pass to the discussion of relativistic wave mechanics. – W. Pauli (Nobel lecture, 1945)

The theoretical explanation follows from the spin-statistics theorem of QFT, which states that two quanta with half-integer spin (called fermions after Enrico Fermi) cannot be in the same quantum state. Not only does this theorem explain why electrons in an atom must be in different states of energy, angular momentum and spin (as per Pauli’s original conjecture), it also explains why matter quanta do not accumulate like force fields but are always seen as separate entities. And, as we saw in Chapter 4, it explains why there is a limit to the number of neutrons in a nucleus.

Classical limit. Because the exclusion principle does not apply to force fields, which have integer spin, quanta of force fields can over­lap and build up. Even though each quantum acts individually, the effect is like that of a classical field.

The Higgs field. The Higgs field interacts with all other fields and, because it has a large vacuum expectation value, it generates the effective mass of these fields. As we saw in Chapter 5, the Higgs field was the final ingredient that made electroweak unification possible. It is the newest field in QFT, the Higgs boson being detected in 2012.

Dark Matter. As early as 1933, astronomical observations showed that there is more matter in the universe than was previously thought. Astronomers now believe that ordinary matter constitutes only 5% of the total mass of the universe, while something called dark matter makes up 27%. (The rest is made of a related substance called dark energy, which we won’t go into.) This conclusion is based on its apparent gravitational influence on other objects.[8] Dark matter is found primarily around galaxies and is believed to be a million times less dense than normal matter. It is not “seen” because it doesn’t emit or absorb light — that is, it doesn’t interact (or interacts feebly) with the EM field.

QFT offers two possible field explanations for dark matter. One possibility was suggested by Steven Weinberg and Frank Wilczek: a new field with quanta called axions.

I’m very fond of axions, in part because I got to name them. I used that opportunity to fulfill a dream of my youth. I’d noticed that there was a brand of laundry detergent called “Axion”, which sounded to me like the name of a particle. So when theory produced a hypothetical particle that cleaned up a problem with an axial current, I sensed a cosmic convergence. The problem was to get it past the notoriously conservative editors of Physical Review Letters. I told them about the axial current, but not the detergent. It worked. – F. Wilczek (W2008, p. 203)

Attempts have been made to detect axions with an apparatus called DAMA (for DArk MAtter.) that is buried under the Gan Sasso mountain in Italy. According to the DAMA scientific team, a signal was detected that is consistent with dark matter, but other scientists have treated this finding with skepticism and new experiments are underway.

The other possible explanation is a field suggested by a new approach to QFT called supersymmetry.  Either of these hypothesized fields would be sufficiently stable and interact feebly enough with normal matter to qualify.




Now we will take a look at what is not covered by QFT — the gaps in the theory.

Renormalization. The most obvious gap is that QFT does not describe the interaction of a quantum with its self-field—an interaction that Richard Feynman called a “silly” idea (Chapter 6). The problem is that according to QFT the result of this interaction turns out to be infinite, which is obviously not correct. The problem was solved for EM fields by the simple expedient of replacing the infinite quantities by their experimen­tally-determined values. However, the fact that the infinities occur in the first place shows that some­thing is going on at the single quantum level that is not described by the QFT equations.

Quantum collapse. Quantum collapse is not described by the field equations, but this doesn’t mean that it doesn’t happen. There are two aspects of quantum collapse that bother many physicists. One is that the collapse is instantaneous and occurs at the same time at widely separated points. Physicists call such a process non-local because it involves communication faster than the speed of light. This was particularly bothersome to Einstein, who called it “spooky action at a distance”. If an emitted photon spreads out over light-years and then is absorbed by an atom, how, Einstein asked, could part of the field that is many light-years away “know” this instantly? [9] Non-locality is especially bothersome in the case of two entangled photons.   Now the equations of QFT do not permit non-local effects, but quantum collapse is not described by these equations, so there is no theoretical reason that it can’t occur. In fact, quantum collapse is necessary if quanta are to act as indivisible units. In any case, non-locality has been experimentally documented and it does not lead to any paradoxes or inconsistencies. To me, the real paradox is why physicists can’t accept it.

The other problem is that, so far as we know, quantum collapse is random. QFT does not explain when or how it happens. All we know is that the probability of collapse is related to the field intensity at a given point. The idea of randomness was also troubling to Einstein:

I find the idea quite intolerable that an electron exposed to radiation should choose, of its own free will, not only its moment to jump off, but also its direction. In that case, I would rather be a cobbler, or even an employee in a gaming house, than a physicist. – A. Einstein (letter to Max Born, 1924)

However, 25 years later Einstein softened his stand. In a 1950 speech to the International Congress of Surgeons, after describing the “over­whelming evidence” for giving up causality, he concluded by saying:

Will this credo survive forever? It seems to me a smile is the best answer. – A. Einstein (Physics Today, June 2005, p. 47-48)

While the problem of randomness is not solved by QFT, at least it has been pinned down to a specific event. It is no longer a vague phenomenon related to the role of the observer, as in QM; it is a physical event that happens with or without an observer. Maybe someday we will have a theory to describe it, but even if it is truly random there is nothing inherently contradictory about that. It may not be what we expected but, like Einstein, we can always smile.

A speculation. Quantum collapse and renormalization both involve some­thing that happens at the single quantum level – something not described by QFT. Renormalization is needed because QFT doesn’t explain how a quantum interacts with its self-field. Quantum collapse is a mystery because QFT doesn’t describe when or how a quantum transfers energy or momentum. It is possible that these two problems are related, and that if one gap is filled, the other will also be filled.

The next two gaps are not unique to QFT, but are present in every theory.

Whys and Wherefores. QFT does not explain why the numbers that appear in the equations have the values they do. The most famous example is the so-called fine structure constant that describes the interaction between matter fields and the EM field. This constant was once thought to have a value of 1/137, and this, as you might imagine, led to some numerological attempts to explain why nature had chosen this particular integer — and such an unusual one at that. Sir Arthur Eddington claimed that the number could be obtained by “pure deduction”, but these attempts were abandoned when more precise measurements showed that the actual value is 1/137.04.

Many physicists still wonder why the masses and coupling constants are what they are, and there are attempts to find explanations that are more sophisticated than the 1/137 saga, including something called superstring theory. There are also less sophisti­cated attempts, such as the so-called anthropic principle, which states that if the values were different from what they are, the human race could not exist. QFT does not supply answers, nor does any other theory. QFT does an amazing job of explaining the world we live in, but why the constants are what they are is, in my opinion, a teleological, if not theological, question.

Consciousness. Finally we come to the grand-daddy of mysteries. How dare physicists talk about “theories of everything” when they can’t explain what goes on behind their very noses!

As humans, we can identify galaxies light years away, we can study particles smaller than an atom.  But we still haven’t unlocked the mystery of the three pounds of matter that sits between our ears.  – B. Obama (remarks by the president on the BRAIN Initiative, April 2, 2013)

But please understand, by consciousness I don’t mean simple information processing, such as can be done by any computer. I mean the sense of awareness, the sensations, the feelings that human and other minds experience every day – from the color red to the beauty of a Mozart sonata or the pain of a toothache. Such sensations are known as qualia. Most physicists don’t want to be bothered with the question, and it is left to philosophers like Charlie Chaplin to worry about it:

Billions of years it’s taken to evolve human consciousness… The miracle of all existence… More important than anything in the whole universe. What can the stars do? Nothing but sit on their axis! And the sun, shoot­ing flames 280,000 miles high. So what? Wasting all its natural resources. Can the sun think? Is it conscious? — C. Chaplin (film “Limelight”, 1952)

I see consciousness as a more pressing problem than the question of why the field constants have the values they do. My concern began when I was thinking about pain. Pain surely cannot be explained by fields or particles or relativity, or even quantum field theory. Nor can pleasurable sensations, like the enjoyment of music or the intense pleasure of an orgasm.

Then one day we had a visitor – a young computer hot shot. As we were sitting down for dinner I asked him “Do you think a computer can ever experience a sexual orgasm?” Well this young fellow began to tell me how you could create an orgasm by putting 0’s and 1’s into the right memory banks. Of course this was nonsense, so I told him he had flunked the test and couldn’t have any dinner. I didn’t really; we fed him, but it gave me the idea to write a book about consciousness with the title “Do Robots Have Orgasms?”  (The illustration shows the cover designed by my nephew Roger L. Brooks.) I eventually abandoned the book because I figured that a book that boiled down to one word (“no”) wouldn’t sell. However, I had already written or sketched chapters with proposed explanations for consciousness, such as artificial intel­ligence, religion, and Quantum Mechanics. As I was working on the Quantum Mechanics chapter, I realized that all the quantum mechanical ex­planations ignored QFT. I then became aware that QFT is ignored almost everywhere — as if it never existed. And that’s why I wrote the book that I wrote.



Is there any prospect that the consciousness mystery will ever be solved? Ambrose Bierce didn’t think so. To quote from his Devil’s Dictionary:

Mind, n. A mysterious form of matter secreted by the brain. Its chief activity consists in the endeavor to ascertain its own nature, the futility of the attempt being due to the fact that it has nothing but itself to know itself with. – Ambrose Bierce (B1958, p. 87)


Among physicists there are different approaches to understanding real­ity. Some physicists believe that reality consists of particles, despite the many inconsistencies and absurdities that accompany that picture – not to mention questions like what the particles are made of. Others believe in wave-particle duality, which is neither fish nor fowl, and some just don’t worry about reality (see Norsen and Hawking quotes in Chapter 1).

It is truly surprising how little difference all this makes.  Most physicists use quantum mechanics every day in their working lives without needing to worry about the fundamental prob­lem of its interpretation. Being sensible people with very little time to follow up all the ideas and data in their own specialties and not having to worry about this fundamental problem, they do not worry about it. — S. Weinberg, (W1992, p. 84

But for those who believe there is a reality and who want to understand it, the choice was described this way by Robert Oerter:

Wave or particle?  The answer: Both, and neither.  You could think of the electron or the photon as a particle, but only if you were willing to let particles behave in the bizarre way described by Feynman: appearing again, interfering with each other and cancelling out.  You could also think of it as a field, or wave, but you had to remember that the detector always registers one electron, or none — never half an electron, no matter how much the field has been split up or spread out.  In the end, is the field just a calculational tool to tell you where the particle will be, or are the particles just calculational tools to tell you what the field values are?  Take your pick. — R. Oerter (O2006, p. 128)

But before you take your pick, let us take a look at some of the things QFT has accomplished:

  • QFT explains why Feynman’s detector always register one electron or none.
  • QFT provides a simple derivation of E = mc2 and gives it a meaning.
  • QFT explains the Pauli Exclusion Principle.
  • QFT explains why matter quanta act like particles.
  • QFT explains why the number of neutrons in a nucleus is limited.
  • QFT explains why force fields act like classical fields in the limit of many quanta.
  • QFT explains the Higgs mechanism.
  • QFT offers an explanation for dark matter.
  • QFT explains the subatomic zoo – “one of the greatest scientific achieve­ments of all time”.
  • QFT explains the paradoxes of special relativity.
  • QFT is compatible with general relativity. Gravity is a force field, not curvature of space-time.
  • QFT explains gravity waves.
  • QFT resolves wave-particle duality (there are no particles, there are only fields).
  • QFT explains the Uncertainty Principle.
  • QFT offers a solution to the measurement problem.
  • QFT explains entanglement (Einstein’s “spooky action at a distance”).
  • QFT resolves the Ehrenfest paradox.

And so, dear reader, I hope that, like Frank Wilczek, Steven Weinberg, Art Hobson, Julian Schwinger (and me), you will choose QFT: the only theory that offers a picture of reality that is understandable and makes sense. And perhaps one day the physics community will finally abandon the QM ship made of particles floating on a sea of paradox for smoother sailing on the seas of quantum fields.


Click here to learn more about Fields of Color.


[1] However, two of these fields (strong and baryon) are effective fields that are made of more basic but “invisible” fields called quarks and gluons.

[2] The reader should be warned that many physicists do not accept quan­tum collapse. They believe that a superposition of states continues until a later time. For example, Roger Penrose suggested that collapse occurs when the gravitational energy exceeds a certain amount (P1994, p. 339).

[3] Zweig had used the word ace to describe these new quanta, but Gell-Mann’s “quark” prevailed.

[4] These colors are not to be confused with the equally arbitrary colors I have used to help visualize fields.

[5] It is straightforward math to show that the frequency of oscillation is given by f = mc2/h, where h is Planck’s constant.

[6] Planck’s Law says that the energy of a quantum is given by E = hf, where f is frequency and h is Planck’s constant. Combining this with the equation in footnote 5 gives E = mc2.

[7] S. Aoki et al., “Quenched Light Hadron Spectrum”, Phys Rev Lett 84(2): pp. 238-41. (2000)

[8] The planets Neptune and Pluto were discovered for the same reason.

[9] When I asked Dirk Bouwmeester (Physics Dept., UCSB) a similar question, his reply was “In for a penny, in for a pound.” In other words, it doesn’t matter if the distance is a nanometer or a light-year, the prin­ciple is the same. If you can accept one, you should be able to accept the other.