At 3 a.m. on a crisp May night in Chile, all seemed well with the world’s largest digital camera. Until it didn’t.
Inside the new Vera C. Rubin Observatory, Sandrine Thomas was running tests. As project scientist, her job is to keep the facility running. But suddenly, a line showing the telescope camera’s temperature changed. It had been flat. Now, it started to spike.
“That looks bad,” thought Thomas. And she was right.
Worried scientists quickly shut down the telescope.
I arrived a few hours later. I was jet-lagged but eager to get my first glimpse of the observatory. It sits on a high, flat-topped mountain called Cerro Pachón. From that perch, it aims a uniquely sharp eye on the cosmos.
With a wide and deep view of the sky, Rubin can see some of the slowest processes in the universe. The assembly of galaxies, for instance. Or the expansion of the cosmos. Rubin also maps the entire southern sky every couple of nights. That allows it to track some of the fastest events out there, such as stellar explosions.
Over the next 10 years, Rubin plans to take 2 million images. These will capture more of the cosmos than any other telescope. “For the first time in history, the number of cataloged celestial objects will exceed the number of living people!” astronomers wrote in 2019.
One of those astronomers was Željko Ivezić, at the University of Washington in Seattle. He’s been directing Rubin’s construction, which has taken decades.
The universe holds so many mysteries. “To answer them, you need something like Rubin,” Ivezić says. “There is no competition.”
But first, Thomas and her team had to get its camera back online.
From dark matter to asteroids
The idea to build Rubin came during another 3 a.m. vigil. This was in 1996. It happened on the next mountain over from Cerro Pachón.
Astronomer Tony Tyson and his colleagues had just brought a new camera to a telescope atop Cerro Tololo. This camera used what was then a fairly new technology: charge coupled devices, or CCDs. These chips convert particles of light, or photons, into electrons. The electrons can then be turned into an image of the light source.
Several CCDs arranged like the patches of a quilt act as one large camera. The bigger the camera, the more clearly those images can be resolved.
Back then, Tyson’s camera was the most powerful in the world. It was made up of four CCDs. He and a colleague had built it to map dark matter. This mysterious stuff is thought to make up 80 percent of the mass of the universe. Astronomers don’t know what it is. But they think it exists because its gravity affects objects we can see. (One of those effects was discovered by astronomer Vera Rubin, the new observatory’s namesake.)
As Tyson and some other astronomers sat in the telescope control room one night, Tyson got an idea. “Guys, we can do better than this,” he said. They could, in theory, build a bigger quilt of CCDs to create a much more powerful telescope camera.
Computers were getting better and faster all the time. They could keep up with such a growing flood of data.
Tyson made this new observatory his pet project.
“I had called it the Dark Matter Telescope,” he says. But what he envisioned could do a lot more than map dark matter. It also could explore the “universe of things that move and explode,” Tyson says. Think asteroids zooming by, stars pulsating and black holes scarfing up matter. The telescope could map millions of objects in our solar system — and farther out, billions of galaxies.
In 2010, the astronomy community put the project at the top of its wish list to be funded by the U.S. government. And they got their wish. That observatory was now a go.
Record setter
The Rubin Observatory has what’s now the largest digital camera ever built. It weighs about 3,000 kilograms (6,600 pounds). At 1.65 meters (5.4 feet) wide, it has 189 CCDs. Its light sensor has roughly the same number of pixels as 260 smartphone cameras.
Rubin also has a huge, unusual set of mirrors. The telescope starts out the way most do: A primary mirror 8.4 meters (27.5 feet) wide collects lots and lots of light. That mirror reflects the light onto a secondary mirror. It’s 3.5 meters (11.5 feet) wide. A third mirror fixes any distortions in the collected light.
The car-sized digital camera is suspended in the middle of the secondary mirror. By the time light bounces there, every point looks needle-sharp.
Rubin will observe the entire night sky visible to the Southern Hemisphere every three to four days. Its camera’s shutter will open for 30 seconds per picture, taking 1,000 images a night — every night for 10 years.
Normally, in the control room, you can hear the shutter clicking all night long.
Thomas finds this sound soothing. “When you can’t hear anything,” she says, “something might be wrong.”
Fun-house mirrors
To get to Cerro Pachón, I had flown into Chile’s seaside city of La Serena. From there, a local driver took me up into the clay-colored mountains. As the ear-popping drive wound higher, I kept my eyes on telescope domes glinting in the distance. I couldn’t stop smiling.
A high, dry spot far from city lights is the ideal home for a telescope. Up on that ridge, the air was so dry I could feel it parching my nostrils and throat. The air was so clear I could see for miles in every direction. The landscape was dotted with rocks and scrubby plants — plus the occasional wild horse or viscacha. (That’s a local rodent that Thomas described as a bunny with a squirrel tail.)
Since the observatory was still under construction, we had to wear reflective yellow vests and helmets. Some of the crew had plastered their helmets with stickers — including custom-made ones of Vera Rubin.
For almost a year while planning this visit, I had looked forward to seeing the massive telescope in action. It had collected its first light about a month earlier and taken data every night since. I was supposed to see the telescope take some of its earliest complete images.
But I had arrived eight hours after the camera’s temperature reading had gone haywire. The telescope was now shut down. When Thomas took me for a tour, the whole structure was motionless.
We passed the camera team on our way up to the dome. “Is my camera moving yet?” Thomas asked the team cheerfully. “Make it work!” (Then she turned to me: “We try to have a positive attitude, but we are all very bummed.”)
The silver lining was that I had an excellent view of the unusual primary mirror. Staring into it was like looking at a fun-house reflection. I swayed back and forth, then crouched down and slowly stood up to see how shapes changed. It was dizzying.
Keeping it cool
The mystery of the camera glitch led Thomas and her team to investigate a key aspect of the telescope’s design: temperature control.
The camera has to be kept cold. Heat can trigger CCDs to release electrons, mimicking light signals from objects in space.
A -123˚ Celsius (-189.4˚ Fahrenheit) metal “cryoplate” backs the detector. Another “cold” plate at -40 ˚C (-40 ˚F) sits behind that. Refrigeration lines snake cooling liquids through the camera. Even the outside of the sparkling dome is designed to reflect warming sunlight away from the telescope.
Thomas and her team were anxious to find out why at 3 a.m. the cryoplate had suddenly warmed. Rubin had been working well for the past month. This was its first crisis.
But the effects could be worse than just detecting phony photons. As the frigid case that holds the CCDs warms, the pressure rises too. Materials in the camera may then release gases that could damage the system.
The likelihood of this “is fairly low,” Sean MacBride told me during my visit. “But the consequence is pretty serious.” This, he said, “is on the top-five list of scariest things that could happen to the camera.”
Based at the University of Zurich in Switzerland, MacBride is a commissioning scientist for Rubin. That job involves testing all the pieces of the telescope and understanding how everything works before it starts collecting data.
By the afternoon, the camera seemed to have gone back to normal all on its own. That was a clue, said Kevin Fanning. This commissioning scientist for Rubin is based in Chile, where he works for the SLAC National Accelerator Laboratory.
Winter in Chile was just beginning. On the night of the incident, it had dropped to 5 ˚C (41 ˚F) outside for the first time since the camera was installed. “Today’s warmer, and it seems to have recovered,” Fanning said.
Maybe the issue had been related to the cold. But why would that make the cryoplate warm up? And why was the critical temperature around 5˚ C? Because few things “change state at that temperature,” Fanning said, it was puzzling.
Fanning proposed an experiment: Cool the dome to 5 ˚C and see if the cryoplate glitched again. The team would wait for it to get colder outside. Then, they’d open the dome a little to let some cold air in. Until then, the team took out a pack of Uno cards.
Do you have a science question? We can help!
Submit your question here, and we might answer it an upcoming issue of Science News Explores
Eyes on the sky
“I’m feeling personally disrespected by the weather, right now,” Fanning said. The next morning, though, he was in a good mood. The cryoplate had kept its cool. That hinted the failure had been triggered by the cold outside.
Maybe the material in the refrigeration lines had thickened and couldn’t cool the cryoplate as normal. Maybe some water got trapped in a pipe and froze, causing a clog. If the team could figure out where cold was messing with the system, they could wrap it in more insulation.
The crew ended up turning the camera back on that night. By the next night, they were back to normal observations. They’re still investigating the issue of the glitch, Fanning told me. In the meantime, researchers plan to add some insulation and pump extra heat into the dome.
“It was a difficult weekend, but I am very pleased by the progress we made and how the team got together to pivot back to [observations] so quickly,” Fanning said by email.
In June, the telescope hit a big milestone. The public got to see Rubin’s first images. The view included 10 million galaxies and more than 2,000 newly found asteroids. As Rubin views the same spots over time, more faint objects will pop out from the dark.
About 90 percent of Rubin’s time will be spent on a wide and deep survey of the sky. But it also will be able to point at things quickly. For instance, when another telescope detects a supernova, Rubin can pivot to look at it.
Anyone will be able to go to the telescope website and play with Rubin data. That includes students and amateur astronomers. “It’s really your ideas and your knowledge and your persistence that determine the science you can do,” Ivezić says.
Waking the dragon
About an hour before I headed down from the mountain back in May, the crew decided to turn on the telescope. Everyone hustled upstairs into the dome to watch. When we entered, the dome was rotating. It felt like the floor beneath us was moving instead.
The dome was like a cathedral, cavernous and round. But nothing echoed. The telescope filled most of the space. The dome walls were also covered with material to absorb stray light, which also soaked up much of the sound.
Seated in a desk chair with a laptop, Fanning directed the telescope through a series of moves to test its range of motion: Look up. Pan from low to high. Spin in a half circle.
Rubin in motion was like a dragon waking up. It moved with surprising elegance and speed. It leaned its head back, shook out its shoulders and turned its face to the sky, ready to open its eyes.