How supernovae are helping uncover the mysteries of dark energy

The largest component of the universe is something we know almost nothing about. 

The best and most accurate observations that cosmologists have gathered over decades show that all the matter around us, every single atom we see anywhere in the cosmos, makes up just 5 percent of all that exists. Another 27 percent is dark matter, which holds galaxies together. And everything else — a staggering 68 percent of the universe — is dark energy, a force that is responsible for the expansion of the universe. 

Without dark energy, the rate of expansion would be slowing down over time. But it’s very clear that’s not the case, and the rate of expansion is actually increasing. There must be some kind of force pushing that expansion, and that unknown force is what we call dark energy. 

It’s the largest constituent of the universe, and it’s a mystery. But for a certain type of scientist, that makes studying it an irresistible challenge. 

At a meeting of the American Astronomical Society earlier this month, researchers presented data a decade in the making from the largest and most uniform sample of supernovae ever collected. The data was part of the Dark Energy Survey, an international collaboration of more than 400 astronomers working together to unpick the mysteries of dark energy.

The analysis focused on a variety of supernovae called Type 1a. These are particularly useful to astronomers because they have a highly predictable brightness, making them invaluable as mile markers that can be used to accurately measure distance. By using these supernovae to calculate the distance to far-off galaxies, scientists can measure how fast the universe is expanding and hopefully learn more about the strange stuff of dark energy.

Dark energy might make up a huge portion of the universe, but its effects are subtle. To detect its influence, researchers must look at huge datasets that show the movements of galaxies on a large scale. It requires very precise tools to be able to detect the kind of widespread effects that dark energy has on the movements of galaxies.

“To make these super precise measurements, you need the best cameras and the best telescopes available, on the ground or in space,” explained Maria Vincenzi of Duke University, who co-led the cosmological analysis of the DES supernova sample. “Building these kinds of instruments is such a monumental effort that it’s something that cannot be done by a single group or by the resources of a single university.”

Most previous research into dark energy using supernovae was done using a technique called spectroscopy, in which light from a supernova is split into wavelengths. By looking for the wavelengths of light that are absent, scientists can infer which wavelengths have been absorbed — which tells you the composition of an object.

That’s extremely useful for getting detailed information from an object, but it’s also a very expensive and time-consuming process that requires the use of a specialist telescope such as the James Webb Space Telescope.

The recent research took a different approach. “We tried to do things in a completely different way,” Vincenzi said. They used a technique called photometry, in which they observed light from objects and tracked how the brightness changed over a period of a few weeks, producing data called a light curve. 

They then fed these light curves into a machine learning algorithm, which was trained to identify the particular supernovae they wanted — the Type 1a supernovae.

The machine learning aspect was key because differences between the light curves of supernova types can be subtle. “The machine learning algorithm can see things that even a very well trained eye might not be able to see,” Vincenzi said, as well as being much faster. 

That enabled the group to identify a huge sample of around 1,500 of these supernovae across the five-year dataset, collected from a single instrument called the Dark Energy Camera mounted on the Víctor M. Blanco Telescope in Chile.

With this impressive dataset, the researchers were able to understand more about the expansion of the universe than ever before, and the findings support a widely held model of the universe that is truly bizarre.

The weirdness is all about dark energy’s density. To understand why that’s important, it helps to start off by thinking about something more familiar: matter.

“As the universe expands, the volume of the universe is increasing. But the amount of matter is not. It’s a constant of total matter. So if the volume is increasing and the matter is constant, the density will decrease,” explained Dillon Brout of Boston University, who co-led the cosmological analysis.

So far, so good. But dark energy isn’t like that — it has constant density over time. “As the universe expands, the density does not decrease. You get a correspondingly larger total amount of dark energy,” Brout said.

That means that dark energy seems to be a property of space itself, which is why it is also sometimes known as the energy of the vacuum. “If you get more space, you get more dark energy. If the universe increases in size, you get just the right amount of dark energy, because it’s a property of space itself,” Brout said.

Dark energy is unlike anything else we know of in nature, so some people are skeptical of the theory and believe there must be some other explanation for the rate of the expansion of the universe, such as something about general relativity being wrong or incomplete.

But increasingly, cosmologists agree that this theory of dark energy’s constant density over time, called Lambda cold dark matter, is the best explanation we have for the observations we’ve made. The new research doesn’t definitively prove this theory is true, but it is consistent with it. 

“This has been mind blowing for everybody who works in the field for the last twenty years,” Vincenzi said. “Because it’s a form of energy that’s very difficult to reconcile with any previous knowledge of energy and the forces we are used to thinking about in physics.”

Dark energy can be thought of as one side of a cosmological coin, with dark matter being the other. The two forces counteract each other: one pushing things apart and the other pulling them together.

“Matter and dark matter influence the universe with their gravity. So dark matter has the tendency to slow the expansion of the universe, whereas dark energy has the tendency to speed it up,” Brout said. “So it’s really like a tug of war between the dark matter with the pull of gravity, and the repulsiveness of dark energy.”

This model means that as time passes and the universe expands, there is more and more dark energy. At earlier points in the universe’s history, its physics was dominated by dark matter because its size was smaller and matter’s density was higher. As the universe has gotten bigger, dark energy has come to dominate.

“Dark energy dominates in the parts of the universe that are mostly empty, over the vast distances between galaxies which are mostly filled with empty space. In the regions of the galaxy that are filled with a lot more matter or dark matter, like in a galaxy or in the solar system, we don’t feel or see the effects of dark energy,” Brout explained. 

That’s part of the reason that dark energy is so difficult to study: researchers need to look at the large-scale movements of galaxies to see its effects.

If all of this seems counterintuitive and strange, then buckle up, because there’s even more weirdness to be uncovered in this story. 

Even though scientists know that there’s an awful lot of dark energy in the universe, its effects are relatively small. Even though it’s driving the expansion of the universe, which is hardly inconsequential, there is a long-standing problem in cosmology where its effects are weaker than theory predicts they should be — much weaker.

In fact, the predictions of quantum mechanics, the most widely held theory of how matter operates at the atomic scale, state that dark energy should be orders of magnitude stronger than it is.

“If dark energy is the energy of the vacuum, the value that we find is 120 orders of magnitude off the theoretical expectation from quantum mechanics. And that’s just bonkers,” Brout said. “It’s sometimes called the largest discrepancy between theory and observations in all of science.”

But if dark energy were as powerful as quantum mechanics predicts, then it would have scattered material in the early universe, preventing early galaxies from forming. The development of life as we know it is arguably dependent on the relative weakness of dark energy.

This discrepancy in the apparent value of the cosmological constant, which is part of general relativity, is a major question for cosmology. It has even been described as physics’ “most embarrassing problem.” 

For dark energy researchers, however, that bewildering discrepancy is what makes the subject so compelling and critical to study.

“We’re measuring dark matter and dark energy, which make up 95 percent of the universe,” Brout said. “And boy, if we don’t understand 95 percent of the universe, we have to go looking and try to understand it.”

The largest component of the universe is something we know almost nothing about. 

The best and most accurate observations that cosmologists have gathered over decades show that all the matter around us, every single atom we see anywhere in the cosmos, makes up just 5 percent of all that exists. Another 27 percent is dark matter, which holds galaxies together. And everything else — a staggering 68 percent of the universe — is dark energy, a force that is responsible for the expansion of the universe. 

Without dark energy, the rate of expansion would be slowing down over time. But it’s very clear that’s not the case, and the rate of expansion is actually increasing. There must be some kind of force pushing that expansion, and that unknown force is what we call dark energy. 

It’s the largest constituent of the universe, and it’s a mystery. But for a certain type of scientist, that makes studying it an irresistible challenge. 

At a meeting of the American Astronomical Society earlier this month, researchers presented data a decade in the making from the largest and most uniform sample of supernovae ever collected. The data was part of the Dark Energy Survey, an international collaboration of more than 400 astronomers working together to unpick the mysteries of dark energy.

The analysis focused on a variety of supernovae called Type 1a. These are particularly useful to astronomers because they have a highly predictable brightness, making them invaluable as mile markers that can be used to accurately measure distance. By using these supernovae to calculate the distance to far-off galaxies, scientists can measure how fast the universe is expanding and hopefully learn more about the strange stuff of dark energy.

Dark energy might make up a huge portion of the universe, but its effects are subtle. To detect its influence, researchers must look at huge datasets that show the movements of galaxies on a large scale. It requires very precise tools to be able to detect the kind of widespread effects that dark energy has on the movements of galaxies.

“To make these super precise measurements, you need the best cameras and the best telescopes available, on the ground or in space,” explained Maria Vincenzi of Duke University, who co-led the cosmological analysis of the DES supernova sample. “Building these kinds of instruments is such a monumental effort that it’s something that cannot be done by a single group or by the resources of a single university.”

Most previous research into dark energy using supernovae was done using a technique called spectroscopy, in which light from a supernova is split into wavelengths. By looking for the wavelengths of light that are absent, scientists can infer which wavelengths have been absorbed — which tells you the composition of an object.

That’s extremely useful for getting detailed information from an object, but it’s also a very expensive and time-consuming process that requires the use of a specialist telescope such as the James Webb Space Telescope.

The recent research took a different approach. “We tried to do things in a completely different way,” Vincenzi said. They used a technique called photometry, in which they observed light from objects and tracked how the brightness changed over a period of a few weeks, producing data called a light curve. 

They then fed these light curves into a machine learning algorithm, which was trained to identify the particular supernovae they wanted — the Type 1a supernovae.

The machine learning aspect was key because differences between the light curves of supernova types can be subtle. “The machine learning algorithm can see things that even a very well trained eye might not be able to see,” Vincenzi said, as well as being much faster. 

That enabled the group to identify a huge sample of around 1,500 of these supernovae across the five-year dataset, collected from a single instrument called the Dark Energy Camera mounted on the Víctor M. Blanco Telescope in Chile.

With this impressive dataset, the researchers were able to understand more about the expansion of the universe than ever before, and the findings support a widely held model of the universe that is truly bizarre.

The weirdness is all about dark energy’s density. To understand why that’s important, it helps to start off by thinking about something more familiar: matter.

“As the universe expands, the volume of the universe is increasing. But the amount of matter is not. It’s a constant of total matter. So if the volume is increasing and the matter is constant, the density will decrease,” explained Dillon Brout of Boston University, who co-led the cosmological analysis.

So far, so good. But dark energy isn’t like that — it has constant density over time. “As the universe expands, the density does not decrease. You get a correspondingly larger total amount of dark energy,” Brout said.

That means that dark energy seems to be a property of space itself, which is why it is also sometimes known as the energy of the vacuum. “If you get more space, you get more dark energy. If the universe increases in size, you get just the right amount of dark energy, because it’s a property of space itself,” Brout said.

Dark energy is unlike anything else we know of in nature, so some people are skeptical of the theory and believe there must be some other explanation for the rate of the expansion of the universe, such as something about general relativity being wrong or incomplete.

But increasingly, cosmologists agree that this theory of dark energy’s constant density over time, called Lambda cold dark matter, is the best explanation we have for the observations we’ve made. The new research doesn’t definitively prove this theory is true, but it is consistent with it. 

“This has been mind blowing for everybody who works in the field for the last twenty years,” Vincenzi said. “Because it’s a form of energy that’s very difficult to reconcile with any previous knowledge of energy and the forces we are used to thinking about in physics.”

Dark energy can be thought of as one side of a cosmological coin, with dark matter being the other. The two forces counteract each other: one pushing things apart and the other pulling them together.

“Matter and dark matter influence the universe with their gravity. So dark matter has the tendency to slow the expansion of the universe, whereas dark energy has the tendency to speed it up,” Brout said. “So it’s really like a tug of war between the dark matter with the pull of gravity, and the repulsiveness of dark energy.”

This model means that as time passes and the universe expands, there is more and more dark energy. At earlier points in the universe’s history, its physics was dominated by dark matter because its size was smaller and matter’s density was higher. As the universe has gotten bigger, dark energy has come to dominate.

“Dark energy dominates in the parts of the universe that are mostly empty, over the vast distances between galaxies which are mostly filled with empty space. In the regions of the galaxy that are filled with a lot more matter or dark matter, like in a galaxy or in the solar system, we don’t feel or see the effects of dark energy,” Brout explained. 

That’s part of the reason that dark energy is so difficult to study: researchers need to look at the large-scale movements of galaxies to see its effects.

If all of this seems counterintuitive and strange, then buckle up, because there’s even more weirdness to be uncovered in this story. 

Even though scientists know that there’s an awful lot of dark energy in the universe, its effects are relatively small. Even though it’s driving the expansion of the universe, which is hardly inconsequential, there is a long-standing problem in cosmology where its effects are weaker than theory predicts they should be — much weaker.

In fact, the predictions of quantum mechanics, the most widely held theory of how matter operates at the atomic scale, state that dark energy should be orders of magnitude stronger than it is.

“If dark energy is the energy of the vacuum, the value that we find is 120 orders of magnitude off the theoretical expectation from quantum mechanics. And that’s just bonkers,” Brout said. “It’s sometimes called the largest discrepancy between theory and observations in all of science.”

But if dark energy were as powerful as quantum mechanics predicts, then it would have scattered material in the early universe, preventing early galaxies from forming. The development of life as we know it is arguably dependent on the relative weakness of dark energy.

This discrepancy in the apparent value of the cosmological constant, which is part of general relativity, is a major question for cosmology. It has even been described as physics’ “most embarrassing problem.” 

For dark energy researchers, however, that bewildering discrepancy is what makes the subject so compelling and critical to study.

“We’re measuring dark matter and dark energy, which make up 95 percent of the universe,” Brout said. “And boy, if we don’t understand 95 percent of the universe, we have to go looking and try to understand it.”