Seeing is believing: AR and VR bring the future of space exploration to life in 3D

Across industries, augmented and virtual reality are adding a 21st-century twist on standard operations and redefining business as usual. This ranges from mixed reality manufacturing to a more literal take on modern medicine, but AR and VR are also being implemented well beyond our Earthly confines, as NASA and JPL tap these capabilities to reimagine space exploration and mission procedures. Sometimes with these tools, as the saying goes, seeing is believing, as engineers and project scientists take a dive into the digital ether to bring tomorrow’s spacecraft to life in 3D. But, as it turns out, some old manual habits are hard to break and apparently spacefaring specialists make mistakes just like the rest of us.

SEE: IT expense reimbursement policy (TechRepublic Premium)

Space exploration with a virtual twist

In a recent post, NASA detailed a number of ways the agency taps AR and VR capabilities aboard the space station, ranging from remotely controlling robots to AR maintenance procedures. At the same time, JPL is using mixed reality to create an immersive Martian experience, allowing scientists to take a virtual stroll on our cosmic neighbor.

In one particular project, dubbed Sidekick, astronaut Scott Kelly used a Microsoft HoloLens headset to conduct ISS training and future mission prep. During these tests, a member of mission control on Earth could stream Kelly’s field of view via the headset and also draw visuals the astronaut could visualize in 3D on the HoloLens display, explained Benjamin Nuernberger, an AR specialist at JPL.

“So instead of just talking about something spatially, just verbally, now you can give some more spatial 3D instructions drawing into his field of view,” he said.

Situationally, Nuernberger discussed the benefits of these VR and AR capabilities alongside NASA’s detailed step-by-step iPad instructions the astronauts have at their disposal. While he gave the agency its props, stating NASA does a “great job” of providing detailed procedures, in some instances, such as “unforeseen repairs” or if the instructions are particularly complicated, “it’s hard to write up these procedures.”

“This kind of technology can help, especially in those situations and unforeseen circumstances, because you could see their perspective, and you can draw on the screen, and those screen drawings can then be displayed in 3D for the astronaut,” he said.

These astronauts may be conducting mission operations on an orbiting space laboratory hundreds of miles overhead, but the conundrum associated with implementing a plan or following a set of instructions isn’t necessarily so different from a familiar mundane experience here on Earth.

“Just imagine building IKEA furniture at home. You read the instructions, and you try to build it. Many times we mess up on a certain step,” Nuernberger said. “Astronauts, they’re extremely smart, and they know how to transfer the instructions from the paper or the iPad into the spatial realm, but still everyone’s human in the end.”

Scandinavian furniture snafus aside, he said these AR and VR capabilities can help save time, reduce risk and decrease the “cognitive load” on astronauts.

Over time, this project transformed into the ProtoSpace project Nuernberger currently works on, an undertaking that transforms spacecraft designs into 3D interactive visuals using uploaded CAD files.

“Basically, we want to visualize the spacecraft before it’s built, while it’s being built [and] maybe even after it’s built,” he said.

Mission prep past and present

To understand the real value-add of these spatial visualizations, it’s helpful to take a look at the way scientists and engineers would approach design and assembly before the days of AR and VR headsets. Prior to ProtoSpace, Nuernberger said this planning phase could involve building mockups with costs pushing “tens of thousands of dollars,” lots of cabling to support the craft and several day’s time putting it all together.

“Now, you can virtually see that almost instantaneously,” he said. “You have a CAD model. You upload it with those cables, and then you put on the headset and you’re ready to go.”

During the design process, Nuernberger said members of the team would “go back and forth over email or show PowerPoints to each other for a long time,” adding that he’s heard anecdotes where these scientists will put on one of the AR/VR headsets and “almost immediately” come up with the answer to a design or assembly question.

SEE: Juggling remote work with kids’ education is a mammoth task. Here’s how employers can help (free PDF) (TechRepublic)

A lot of these AR and VR-enabled capabilities and new efficiencies comes back to one critical point: NASA builds very specific spacecraft for a very specific environment and purpose. This means the team is in-essence reinventing the roving wheel with each new mission; essentially creating an exploration recipe from scratch with each go-around.

“We’re not like a car manufacturer where we’re doing the same thing over and over and over again,” he said. “We’re building one rover, maybe two […] and if we get something wrong, the risk is very high, especially, in the assembly phase because there’s eventually a launch window.”

Interestingly, there are also AR and VR workplace safety benefits at play. While NASA may routinely tackle extraterrestrial feats and maneuvers on distant planetary bodies, the agency must still abide by OSHA standards here on terra firma.

In one instance, Nuernberger said the NASA team was planning spacecraft and support equipment configurations when a Microsoft HoloLens revealed a critical flaw in their approach.

“They realized that if [they arranged the craft] in this configuration, someone’s going to have to be working under a suspended load, which is a very risky and dangerous thing, which is not allowed under OSHA standards,” Nuernberger said.

A surreal mixed-reality mashup

When it comes to the advantages of these tools, Nuernberger said the engineers have “seen that it’s not just a toy.” As the headset slides on and the visual capabilities take hold, there’s often this moment of realization that these AR and VR tools have real functionality, he explained.

“Typically it’s when someone puts on the headset for the first time and they look at these massive spacecraft, not just like a little augmented reality experience on your tabletop, but they look at this full-scale spacecraft and they’re like, ‘Oh my goodness, how big is this?!'”

Self-admittedly, this experience even surprises Nuernberger sometimes, as he slides on the headset and sees the Mars Rover in a living room or conference room and realizes just how large the craft is in a stereoscopic augmented reality as opposed to a pixelated vehicle framed on a computer screen.

Watching the NASA and JPL videos highlighting this technique can be a somewhat surreal experience: A team of JPL employees huddled around a seemingly massive yet-invisible object, each donning virtual reality headsets, manipulating components within a multi-million dollar craft that currently exists as mere digital ether in the middle of it all.

Situationally, sometimes the virtual and the physical worlds blend for the members of the team in this mixed reality mashup and old manual habits can be hard to break.

“We’ve had people come in with physical tools and put the physical tool on the virtual model, which is kind of cool,” Nuernberger said.

Best of the Week Newsletter

Our editors highlight the TechRepublic articles, downloads, and galleries that you cannot miss to stay current on the latest IT news, innovations, and tips.
Fridays

Sign up today

Also see