IE 11 is not supported. For an optimal experience visit our site on another browser.

Video games help NASA control robots, rovers

NASA's Jeff Norris speaks at the Penny Arcade Expo
Twitch.TV
Jeff Norris and Matt Clausen at PAX
NASA's Jeff Norris and Matt Clausen talk about how gaming and space exploration go hand in hand during a presentation at the Penny Arcade Expo in Seattle.Twitch.TV

It may not seem like the serious interplanetary science being done by NASA and the entertainment provided by video games would have much in common, but a team from NASA's Jet Propulsion Laboratory says that, more and more these days, the two go hand in hand. 

During the Penny Arcade Expo this weekend, a team from NASA told a packed audience of video game enthusiasts that the Jet Propulsion Laboratory is not only using game technology to help the rest of us earthlings get an up close look at the latest mission to Mars, but it's also using game tech to better control NASA's robots and rovers.

In fact, Jeff Norris, manager of the Planning and Execution Systems Section at NASA's Jet Propulsion Laboratory, says he believes video games are key to how humans will explore space in the future.

"We are using the technology behind games and games as a platform themselves to really revitalize and change the way we are exploring space and sharing it with everyone,"  Norris said.

The Penny Arcade Expo (aka PAX) is a convention that celebrates all things gaming and draws some 70,000 gamers to Seattle each year. The team from NASA -- gamers themselves -- kicked off their PAX presentation with a demonstration of the new Xbox 360 game called "Mars Rover Landing" which, as the name suggests, challenges players to try to land a virtual version of the Curiosity rover on the red planet using a Kinect sensor and the movements of their body. (Read more about the game here.)

"This is a pretty big step for our agency," Norris said of the game. "We're embaracing the idea that games have become a really important medium for communicating our mission to everybody."

Norris and his team also showed off  other game-like experiences they've designed to give us earthbound explorers a closer look at Mars from the comforts of our own home computers.

The "Gale Crater" interactive experience -- which was created using real satellite data from Mars -- lets players get an an up-close view of the Red Planet locale where the rover will spend the years to come. And those who want to see Curiosity's rocker-bogie suspension system at work can check out the "Free Drive" game, which lets players run a virtual version of the bare-bones rover throughout Gale Crater's terrain. (These games can be played for free right here.) 

But more than just using games as a communication and education tool , NASA has turned to gaming technology as a means of controlling their robots and even their rovers.

NASA uses Kinect to control Robonaut
A look at the experimental interface NASA created to control its Robonaut robot. The Kinect game camera reads a person's body movements and feeds them to the robot which then mimics the same movements.NASA/JPL

"We're trying to find easier and more natural ways to control robots," explained Matt Clausen, who had been working on an game called "The Misadventures of P.B. Winterbottom" when he was hired by NASA JPL to work in the Human Interfaces group. 

His team has experimented with the Wii controller and with the Xbox controllers and are also experimenting with touch screens to manipulate various robots.

"A lot of money gets invested into video game technologies so we can just piggy back on what they've done," he explained.

Clausen's team has created an experimental Kinect-driven interface to control the highly dexterous humanoid robot called Robonaut (which is currently living on the International Space Station). At PAX, Clausen and his team members showed off how the Kinect game camera can read a person's body movements and feed them to the robot which then mimics each of the same movements one at a time.

However, "It's one thing to control a humanoid robot with Kinect -- that's an easy no brainer," Clausen said. "So how can we control a six-limbed robot that has no head, no direction, nothing very anthropomorphic at all?"

He was referring to the six-limbed robotic rover called ATHLETE (which stands for All-Terrain Hex-Limbed Extra-Terrestrial Explorer).  Clausen's team was able to map the robot's controls to human body movements using Kinect. At PAX they demonstrated how a person could walk in place to move ATHLETE forward or raise their arms to raise up two of the robot's arms (like a marionette being controlled by a puppeteer).  

ATHLETE
At NASA JPL's Human Interfaces group they are working on ways to control the ATHLETE rover using the Xbox 360 Kinect game controller.NASA JPL

But perhaps most interestingly, Norris pointed out that it is the kind of virtual reality experiences that video games offer that could be the future of space exploration.

For starters, he believes that rovers like Curiosity  and robots like Robonaut act like avatars for us humans -- taking us to places we are not well adapted to go ourselves. He also believes that building a holodeck (yes the kind found in "Star Trek") is within our grasp.  (If you doubt him, just check out Project Holodeck and the Oculus Rift head-mounted display projects that are in the works.)

"This is going to revolutionize so many things about our world and the ways that we work and the ways that we play but also the ways that we explore," he said. "There is, in my opinion, no greater tool for human exploration than an environment like this that can put you on those distant planets."

Norris said it's for this reason he has become passionate about bringing space exploration and the game industry together to build these technologies and systems now.

"They will enable us to enact a vision of  exploration that's just not been available to us before," he said.

If you'd like to watch the entire NASA presentation at PAX, follow this link to Twitch.TV.

Winda Benedetti writes about video games for NBC News. You can follow her tweets about games and other things on Twitter here @WindaBenedetti, and you can follow her on Google+. Meanwhile, be sure to check out the IN-GAME FACEBOOK PAGE to discuss the day's gaming news and reviews.