CAPE CANAVERAL — Astronauts working in low-Earth orbit will soon receive Microsoft’s HoloLens technology during a cargo resupply mission to the International Space Station (ISS) via the Orbital ATK Cygnus spacecraft. A pair of the augmented reality headsets were previously destroyed while aboard SpaceX’s Dragon capsule when the Falcon 9 rocket exploded during a cargo run last summer.
NASA will apply the HoloLens technology to its newly-developed ‘Project Sidekick’ as a way for the crew of the ISS to expedite day-to-day operations and research while increasing overall efficiency. The headsets will enable Astronauts to conference with experts here on Earth in real time and provide a holographic instruction manual for designated scientific experiments.
HoloLens is the first fully untethered holographic computer ever developed and NASA is the first external organization to partner with Microsoft in utilizing the headsets. This partnership is the first step in the space agency’s long-term plan to pair holographic computing with space exploration. What the technology essentially does is fabricate objects in the real world using light instead of matter creating what is commonly referred to in science fiction as a hologram.
“Now we can place these holograms right into your world and right into your life with more reality than ever before. Now this mixed reality of having holograms existing/being pinned in the real world, really allow you to create brand new experiences” explained Microsoft Technology Fellow, Alex Kipman. “We’re incredibly excited to have Sidekick be one of those first experiences.”
The ISS crew have always depended on voice and written guidance when performing repairs or experiments. The first of the two Sidekick functions will revolutionize the way these tasks are handled by using “Remote Expert Mode.” This will utilize Microsoft’s familiar Skype technology to allow ground operators a shared first-person perspective with an ISS crew member while they provide instructions to tasks and real time updates.
Along with providing virtual aid, Sidekick can augment and facilitate individual tasks using its second function; “Procedure Mode.” This capability will display animated holographic illustrations on the surface of material or objects that a crew member is interacting with. This aspect of Sidekick’s technology will reduce overall training of future ISS Astronauts and NASA believes this mode will be invaluable for future manned missions to deep space where the delay in communications can slow routine operations and emergency repairs.
“HoloLens and other virtual and mixed reality devices are cutting edge technologies that could help drive future exploration and provide new capabilities to the men and women conducting critical science on the International Space Station,” said Director of the ISS program, Sam Scimemi. “This new technology could also empower future explorers requiring greater autonomy on the journey to Mars.”
[protected-iframe id=”d0d43bd68eba56ee2196a79518d87418-35584880-75321627″ info=”https://www.youtube.com/embed/S1IS8Kbzxos” width=”560″ height=”315″ frameborder=”0″ allowfullscreen=””]“As you saw in that video [embedded above] we’ve been hard at work for some time developing the Project Sidekick system and testing it” said Project Lead at NASA’s Jet Propulsion lab, Jeff Norris . “The video was us testing it in microgravity with the help of NASA’s Reduced Gravity Office on board the Weightless Wonder C-9 jet.”
More testing of Project Sidekick will begin when the crew of the International Space Station unbox their new set of HoloLens. This first pair of devices will be primarily used to evaluate hardware and software with NASA hoping to get a second pair for day-to-day use delivered to the ISS on a future mission.
Earlier this year, NASA and Microsoft agreed to collaborate on the development of the OnSight software which will enable scientists to conduct research on the surface of Mars using the HoloLens technology.
The OnSight software is being created at NASA’s Jet Propulsion Laboratory in Pasadena, California and will enable experimentation on the red planet with the help of the Mars Curiosity rover. “OnSight gives our rover scientists the ability to walk around and explore Mars right from their offices,” said program executive for the Mars Science Laboratory mission at NASA, Dave Lavery. “It fundamentally changes our perception of Mars, and how we understand the Mars environment surrounding the rover.”
Robin Seemangal focuses on NASA and advocacy for space exploration. He was born and raised in Brooklyn, where he currently resides. Find him on Instagram for more space-related content: @not_gatsby.
