Yesterday I planned to write a review of Richard Panek’s The 4 Percent Universe (Houghton Mifflin Harcourt, 2011), a fascinating look at dark matter and dark energy and the current state of our research into them. Panek is an excellent writer with an eye for detail and the human touch. He gets you into the thick of scientific controversy and brings out not only the issues but the personalities involved — the good news is that the personalities, particularly in the case of dark energy, didn’t seem to matter, because the major players reached the same conclusion.

But as I worked on the review, I found myself focusing on the dark energy side of the book, especially the question of how dark energy findings could be supported by other evidence. So while Panek spends an equal amount of time with dark matter, and runs through everything from dark matter candidates (WIMPs, MACHOs, etc.) to attempts to use gravitational lensing to constrain the population of dark objects (not to mention the ‘Bullet cluster’ analysis that did so much to show us dark matter’s impact within galactic clusters), I’ll spend my time where his own energy is the greatest, on the whirlwind controversy over an accelerating expansion.

The Cosmos As It Used to Be

Back in the 1920s, Edwin Hubble laid the groundwork for an expanding universe. Hubble used the period/luminosity relation in Cepheid variables (this was Henrietta Swan Leavitt’s work) to find the distances to nearby galaxies. He took the redshift for those galaxies as equivalent to their velocities. Panek explains how he graphed the distances he found against the galactic velocities, concluding they were directly proportional to each other. The relation seemed straightforward: The greater the distance, the greater the velocity. That was a relationship, one-to-one, that you could plot on a graph as a straight line on a 45-degree angle. Assume a constant rate of expansion and the straight line would continue no matter how far you looked.

But of course we also know that the universe is full of matter, and that matter attracts other matter by virtue of gravity. So it’s perfectly understandable to think that the expansion of the universe should be slowing, and the question becomes, what is the density of matter, and can we, by showing how fast the expansion is slowing, see what lies ahead for the cosmos? Can we push that straight line graph until it bends by using distant supernovae as our standard candles?

These are sound questions, but the scientists studying them had no reason to believe that by 1998, they would have changed our view of the cosmos. It was a breakthrough the likes of which astronomy has rarely seen, with two teams of passionate, committed scientists — working with their own agendas, each battling feverishly to get their work out ahead of the other guys — coming to the same startling conclusion. The straight line on the graph was bending the wrong way, and that meant the expansion wasn’t slowing. Type 1a supernovae at great distances were dimmer than they ought to be at their particular redshift, and thus further away than our theories said. The expansion of the universe was, in fact, speeding up.

And what do you do when you run into a result like that? Panek excels in following up the question, and if you ever wanted to see science at the level of everyday head-banging research, theories butting against each other, observations failing at critical, hair-pulling moments only to be replaced by serendipitous discoveries, this is the book for you. It says something that two combative teams working with mostly independent data sets and relying on different methods of analysis arrived at a conclusion that neither team had expected. Yet this is precisely what the High-z Supernova team and the Supernova Cosmology Project proceeded to do in 1998.

The Dark Energy Follow-Up

Re dark energy, the answer to the question of what to do next is, you set out to prove the effect doesn’t exist. Two possibilities immediately surfaced, the first being a new kind of dust. Astronomers already know how to correct for the dust within galaxies that makes light from them redder, but perhaps there was a different kind of dust — call it ‘gray dust,’ even if no one has a clue what it is — and posit that it exists between the galaxies. Or consider another possibility: Maybe Type 1a supernovae in the early universe were intrinsically fainter. Perhaps the supernova process was slightly different then, creating astronomical events that make us think they were more distant, when we are actually looking at a slightly different kind of object.

What to do? Step back from the question and consider that if the supernova evidence is correct, then the time we live in is one in which dark energy is dominant over matter, winning its battle with gravity and thus accounting for the universe’s continued expansion. What that means is that in the earlier universe, the further back you go, the smaller and denser the cosmos and the greater the gravitational influence of matter. Thus there would be a time when matter, not dark energy, was dominant, the expansion would have been decelerating, and supernovae would thus appear brighter than they should. That’s an observational test for dark energy alternatives.

Assuming you can find the right, incredibly distant supernova. And therein is the observational crux — ‘gray dust’ couldn’t account for such a supernova, nor could changes to how supernovae functioned in the early universe. The trick would be to find a supernova distant enough that it would have exploded in that far earlier era, and then to study it. Panek explains:

You would need a supernova that had exploded before the expansion of the universe ‘turned over’ — before the universe had made the transition from deceleration to acceleration, back when matter, not energy, was winning the tug of war. You would expect that supernova to be brighter than it ‘should’ be. Plot it on the Hubble diagram — way out there, far beyond the nearby supernovae from Calán/Tololo, beyond the high-redshift supernovae that the two teams had discovered — and the upward deviation from the 45-degree straight line that High-z and SCP had graphed would ‘turn over,’ too, just like the universe. It would dip down.

And if it didn’t, you’d have to rethink dark energy.

Being Dealt the Right Hand

Did I mention luck in all this? You can’t see supernovae this far across the universe with a ground-based telescope, but the Hubble Space Telescope can do the job, and in late 1997 it had been used to find such objects in the Hubble Deep Field image, which contained about three thousand galaxies. Adam Riess, working with the High-z collaboration while at the Space Telescope Science Institute, kept the two supernovae Hubble had found in mind. SN 1997ff and SN 1997fg needed follow-up, returning to the same field to make multiple reference images. The two supernovae were far enough out to test the scenario of a universe that was still slowing down rather than expanding, but only if later images could be found for comparison.

But there had been no follow-ups. It finally occurred to Riess that HST images from some unrelated observation might by utter chance contain one of the two supernovae. He surveyed Hubble’s work in the relevant time frame and, astoundingly enough, found that SN 1997ff had appeared in Hubble test images, made in 1997 when Shuttle astronauts had installed the Near Infrared Camera and Multi-Object Spectrometer. Using them, Riess was able to establish that SN 1997ff had exploded 10.2 billion years ago, far earlier than the time the expansion of the universe should have started to accelerate. At a spring Space Telescope Science Institute symposium in 2001, Riess produced his results using a transparency showing a chart of redshift against brightness of the supernovae found by both the High-z and SCP teams.

Panek’s treatment of the moment is worth lingering over, as Riess, covering part of the transparency, begins to reveal the data points showing the averages of supernovae at the various redshifts:

Riess revealed the first three dots on the transparency: here, and here, and here, the averages of the nearby supernovae from the Calán/Tololo survey.

Then, moving to the right, the next three dots: here, and here, and here, the averages of the distant supernovae from the SCP and High-z searches.

The dots were beginning to describe the now-familiar gentle departure from the straight line, the upward turn toward the dimmer. In six dots Riess had taken his audience from a few hundred million light-years across the universe, to a billion, then two billion, three, four. Now, he said, he had the point that represented SN 1997ff. He had determined its redshift to be about 1.7, the farthest supernova to date by a long shot, a distance of about eleven billion light years.

They knew what they were going to see, but the hundred or so astronomers in the auditorium couldn’t help themselves. They shifted in their seats. Leaned forward. Held back. Crossed arms.

There: SN 1997ff.

A gasp.

The gentle upward curve was gone. In its place was a sharp downward pivot. The supernova was twice as bright as you would naïvely expect it to be at that distance. The universe had turned over, allright.

While Riess went on to explain that the result ruled out the hypothetical effects of exotic gray dust or a change in the nature of supernovae at a confidence level greater than 99.99 percent, the evidence continued to loom on the screen behind him. His audience couldn’t take their eyes off it. For the astronomers of the invisible, it was something to see.

An Open-Ended Story

Thus Panek on a remarkable moment in our history, but then, there seem to be more and more remarkable moments these days. Now and then I hear astronomers comment on their own good luck to be living in a time when such monumental results are being discussed. For it’s not as if we have closed the book on dark matter and dark energy. As Panek shows throughout this provocative volume, and with the same verve he demonstrates above, we’re in a story that emphatically says ‘to be continued,’ one that is posing as many questions as it answers. His discussion of dark matter charts the problems of observing something that is by its nature unseen, and brings to life the personalities deeply involved in working out potential solutions.

This is a story about astronomers who found what they didn’t expect. “They weren’t looking for dark matter. They weren’t looking for dark energy. And when they found the evidence for dark matter and dark energy, they didn’t believe it.” Just how the evidence accumulated for both, and how scientists intend to probe more deeply into their mysteries, is what this book chronicles. It’s an intellectual feast that will provoke discussion and insight while acknowledging that we are only at the beginning of a new era in astronomy, one where the universe is in many ways something less than meets the eye. What a time to be alive and studying the heavens!