The James Webb Space Telescope reveals the universe as you’ve never seen or heard it before

Not only can we see in space and time billions of years ago. The magic is that we can see everything.

Although its predecessor, the Hubble Space Telescope, offered some amazing views, Webb, which was developed in partnership with NASA and the Canadian and European space agencies, is able to look even further back in time and show us more details about what which lies beyond planet Earth.

Take the recent release of the Pillars of Creation, first imaged in 1995 by Hubble. In the original image from the region believed to be the star-forming part of the galaxy, pillars of gas clouds that look like long fingers stretch skyward.

What we couldn’t see before, and what is now being revealed by the Webb telescope, are all the stars hidden behind the gas.

That’s because Webb sees infrared light that is normally invisible to humans.

By capturing infrared light, Webb can see objects that are so far away that the light they emit takes more than 13.5 billion years to reach Earth. This means that Webb is also like a time machine, as it can see what the universe looked like when the earth and sun were formed.

However, what Webb sends back is invisible to humans because we can’t see infrared light.

So it’s the job of Joe DePasquale and Alyssa Pagan, science visualization developers at the Space Telescope Science Institute in Baltimore, to translate the information from the Webb into something visible.

Joe DePasquale, senior science visual materials developer, creates images from the James Webb Space Telescope.

“We can’t see in the infrared. So there has to be some level of translation here. But we use a physical meaning like real physical science to represent color,” Pagan told Global’s The New Reality.

With the help of NASA scientists, Pagan and DePasquale separated the images into wavelengths. “We apply color according to these wavelengths. So the shortest wavelength filters that we have, we use blue for them. And as we move to longer and longer wavelengths, we go to the greens and then to the reds,” says DePasquale.

The end result is stunning images like the mountain-looking space rocks of the Carina Nebula, captured by Webb.

“What we see when we look at these images is the raw material for life,” says DePasquale.

“We understand the universe. We understand ourselves. It’s so intriguing to get that new perspective, that bigger picture. A lot of people might say, “Oh, that makes me feel small,” but I think for a lot of people, it actually makes you feel united, connected, part of something that’s so big and so beautiful. So you are part of something that is great.

On their own, these images are remarkable, but a Canadian scientist is now adding another level of emotion to it all.

Matt Russo, a University of Toronto physicist and sound specialist, worked with musician and friend Andrew Santaguida to add sound to the universe.

“The whole process felt really natural because we’re combining things we’re passionate about: music, astronomy, math, computer programming, science, communication — all these things wrapped into one package,” Russo says.

Matt Russo, a University of Toronto physicist and sonification specialist, created sounds for Webb’s images.

Their first attempt at sounding out an image was of the Trappist-1 solar system, first imaged by NASA’s Spitzer Space Telescope in 2017.

“[It] is an amazing solar system with seven Earth-sized planets. But they also found themselves locked into a musical pattern called orbital resonance. And that made it really natural to translate their movements into musical rhythms and pitches,” Russo says.

They did the Trappist sonification for pure fun – then NASA took notice.

“We kind of on our own, (we started) sounding out different things that (NASA) had put out and we were sending them and they were just starting to put it out there themselves. And in the end, that’s what made us work professionally for them.”

Some of the sonifications were met with skepticism by the public, such as when they made a sound for a black hole.

“There is a real sound wave detected in space in a galaxy cluster. And we were able to see the waves in the image, which means we can extract them and re-synthesize sound,” Russo says.

“Some outlets would say that this is actually recorded sound from a black hole, as if you had a microphone in space, which we know won’t work for several reasons. So it’s important that when we do sonification, we present it as exactly what it is: that it’s data collected in sound.”

Russo and Santaguida are now working on the latest images from the James Webb Telescope.

They take the spectacular images that DePasquale and Pagan have created and run them through a software system designed by Russo.

According to Rousseau, sometimes the sound of data can be a pleasant surprise. Other times, they have to get a little more creative to figure out how best to represent something in the image. Rousseau says they always try to be as scientifically accurate as possible.

“When we have a bit more musical input, we have to decide, for example, which musical instrument will be triggered by stars,” he adds. “People seem to have an intuition that the stars would make a kind of ringing or bell-like sound.”

Their sonifications of the Webb images now allow people to see — and hear — the universe.

Sonifications provide those living with visual impairments the chance to experience new insights into what is out there.

“The whole point is to convey these interesting features in the image through sound,” Russo says.

Christine Malek, a member of Toronto’s visually impaired community and an arts and culture consultant, says the sonifications by Russo and Santaguida allow her to conceptualize the images from the telescope, even though she is unable to see them.

“I never imagined experiencing astronomy in this way,” she tells The New Reality.

Christine Malek is a member of the visually impaired community and is helping NASA make the Webb images more accessible.

“When I experienced sonification for the first time, I felt it in a way that was not intellectual; it was sensory and visceral. So sometimes I wonder if sighted people experience this when looking at the night sky,” says Malek.

She now works regularly with Russo, Santaguida, and NASA to help translate the Webb images to the best of their abilities for the benefit of people living with visual impairments.

Malec is excited about the future of space exploration and hopeful for the future of accessible science content.

“I wonder if I was a kid now and came across things like sonification and image descriptions and astronomical stuff, would a career in STEM make more sense? Would it be more attractive? And I think the answer to that is yes. So I think this cause is really good for blind and partially sighted kids today to grow up with that as normal, I think it’s incredibly valuable.”