Artificial intelligence isn’t born brilliant. Machine learning algorithms learn to master tasks through trial and error, just like humans. But instead of having a fleet of clueless robots clumsily breaking things in the real world, computer scientists are training their A.I. with simulations of the physical world, aka video games.
A team of researchers led by the Massachusetts Institute of Technology are teaching the future brains of robots how to make and serve coffee, pick up a book and read it, and work a household computer all inside of a The Sims-inspired “VirtualHome” simulator. This lets them break down tasks into simpler steps that even a bucket of bolts could understand.
The study was presented at the Computer Vision and Pattern Recognition (CVPR) conference in June, and could be the first step toward creating the robot butlers science fiction movies has made us yearn for.
This is #15 on Inverse’s list of the 20 Ways A.I. Became More Human in 2018.
The researchers first crowdsource a large knowledge base of household tasks (top). Each task has a high level name, and a natural language instruction. They then collected "programs” for these tasks, (middle left). Where the annotators “translate” the instruction into simple code. They implemented the most frequent interactions in a 3D simulator, called VirtualHouse, allowing them to drive an agent to execute tasks defined by programs. They proposed methods to generate programs automatically from text (top) and video (bottom) thus driving an agent via language and a video demonstration.
“You can imagine a setting where robots assist with chores at home and can eventually anticipate personalized wants and needs, or impending action,” explains lead author Xavier Puig in a statement. “This could be especially helpful as an assistive technology for the elderly, or those who may have limited mobility.”
However, turning machine learning algorithms that were taught to navigate The Sims into functioning robots will take some high-tech hardware. This process could be accelerated through partnerships with Boston Dynamics and SoftBank, who would be among the most likely candidates for translating this Sims-based A.I. into physical assistants.
However, the authors said this study could be see immediate use by augmenting smart speakers’ ability. This tech could help further tailor the likes of Siri, Alexa, and the Google Assistant to individual customer needs.
Now you can confidently email your middle school teacher who confiscated your GameBoy to say that video games are actually incredibly useful for the future.
