For many, the term ‘virtual reality’ will surely conjure up visions of newfangled headsets and how it’s being used in fields like gaming, but the technology in fact goes back decades, and an organisation that has been at the forefront of its development is NASA.
As explained by Matthew Noyes, a mechanical and software engineer at NASA, some of the organisation’s earliest research into virtual reality began as far back as the 1980s.
Led by Director of Virtual Environment Workstation (VIEW) Scott Fisher at the Ames Research Centre, VR was used to train astronauts in space station maintenance, with this work helping to define the form factor of the head-mounted displays, glove interfaces, and a form of binaural 3D audio we are familiar with today.
The Johnson Space Center then in the mid-1990s began developing its own hardware and software for VR, including head-mounted displays and custom in-house rendering engines. Among other things, this tech has been used to qualify astronauts on jetpack training, practice maneuvering around the exterior hull of the International Space Station (ISS), and to teleoperate humanoid robots.
“The technology has even been directly deployed to the ISS using an entire laptop screen as a VR display, allowing currently-flying astronauts to be briefed on maintenance procedures on-orbit. It is the first instance of VR in space,” Noyes explained.
Noyes will be taking Pause 2017 attendees through NASA’s work with VR during a keynote on Tech Day, looking at how its use of VR has developed and how this, in turn, has helped push the organisation forward to keep innovating and achieving its goals.
The hub of NASA’s VR work is now the Hybrid Reality Lab at Johnson Space Center, created in late 2015 to incorporate into the organisation’s work tech developed by the interactive entertainment industry.
“We are leveraging the HTC Vive VR headset with its innovative Lighthouse tracking system and Unreal Engine 4 together to provide a new capability we call ‘Hybrid Reality’, a mix of Virtual and Physical Reality to provide greater immersion at low cost,” Noyes said.
As he explained, Hybrid Reality as it’s used by NASA refers to 3D printing plastic models of tools such as drills and tracking them in 3D space as they’re manipulated by the user. Costing significantly less than what it would to manufacture the real tools, a plastic model is weighted to match what the real tool would in the operational environment as closely as possible.
Photorealistic graphics are then overlaid on top of the tracked model inside the Vive headset so, to the user, it looks like they’re using a real tool. VR then is used to fill out the background details of a scene, such as the mountains of Mars.
“It can simulate physical behavior, like the motion of objects in microgravity, or the way flames would burn and spread during an emergency fire egress drill, in ways that cannot be currently achieved in purely physical reality. By adding in physical components to the VR scene, we don’t lose the development of muscle memory or the effects of fatigue from training scenarios, yet still gain the benefits of VR,” Noyes said.
NASA is also working to incorporate Hybrid Reality into its Active Response Gravity Offload System (ARGOS), a robotic harness that offloads one’s body weight to simulate the effects of moving your body in reduced gravity.
“This will help better prepare astronauts for reduced gravity, and will help allay simulator sickness,” Noyes explained.
Through the work of the Hybrid Reality Lab, NASA is not only aiming to prepare astronauts for spaceflight more effectively than it has ever been able to, but at reduced cost.
“It is extremely challenging to prepare humans for spaceflight. The conditions in space are very different from those on Earth and are hard to replicate, which means it’s hard to evaluate how someone will actually react to those conditions; yet it’s imperative we simulate those conditions as best as possible; even the slightest mistake could cost you your life,” Noyes said.
Australia has had a part to play in this work, with NASA collaborating over the last year with Opaque Media Group. The Aussie company has been developing a video game capturing “what it’s really like to be a modern-day astronaut”, Noyes said, including the feeling of navigating around the ISS structure on a spacewalk.
For the company, the chance to receive feedback on the accuracy of their work from NASA astronauts, engineers, and operations personnel to make the experience more authentic is invaluable.
NASA’s work at the Hybrid Reality Lab blends together the three elements that define Pause 2017: creative, tech, and business.
“We seek to incorporate creative, artistic methodologies into astronaut training, everything from evoking emotional and stress responses from photorealistic environments to leveraging narrative storytelling techniques as a means to transfer knowledge to trainees or as a means to establish crew cohesion,” Noyes said.
“We integrate and develop the latest technologies in VR rendering, additive manufacturing, object tracking, and human-computer interaction to redefine how software, hardware, and wetware can work in synergy for NASA’s mission.
“Finally, we work closely with the world of business by sharing and incorporating ideas with entrepreneurs like Opaque Media Group and FusionVR, so that we may together not just define a new future of human spaceflight, but also to inspire new generations of scientists and engineers.”
Though Pause 2017 attendees might not be working on preparing astronauts for space, Noyes will be sharing key lessons in his keynote that can be taken on board by any entrepreneur or creator.
Pause Fest 2017 will take place at Federation Square on the 8th-10th of February 2017. Tickets can be purchased here.