Researchers have trained an AI to create basic tools by combining objects

The work, which uses a new capability to reason about shape, function, and attachment of unrelated parts, is a significant step toward enabling intelligent agents to devise more advanced tools that could prove useful in hazardous—and potentially life-threatening—environments.

The concept may sound familiar. It’s called “MacGyvering,” based on the name of a 1980s—and recently rebooted—television series. In the series, the title character, known for his unconventional problem-solving ability, uses different resources available to him. For years, computer scientists and others have been working to provide robots with similar capabilities.

In this latest work, researchers trained a robot with the new approach, then gave it a set of optional parts and told it to make a specific tool. Much like its human counterparts, the robot first examines the shapes of each part and how one might attach to another.

Macgyvering new tools

Using machine learning, the robot learns to match form to function—which object shapes facilitate a particular outcome—from numerous examples of everyday objects. For example, by learning that the concavity of bowls enables them to hold liquids, it uses this knowledge when making a spoon. Similarly, researchers taught the robots how to attach objects together from examples of materials that it could pierce or grasp.

In the study, researchers successfully created hammers, spatulas, scoops, squeegees, and screwdrivers.

“The screwdriver was particularly interesting because the robot combined pliers and a coin,” says Lakshmi Nair, a PhD student in the School of Interactive Computing at Georgia Institute of Technology. “It reasoned that the pliers were able to grasp something and says that the coin sort of matched the head of a screwdriver. Put them together, and it creates an effective tool.”

Currently, the robot is limited only to the shape and attachment. It cannot yet effectively reason about particular material properties, a crucial step in advancing to a real-world scenario.

“People reason that hammers are sturdy and strong, so you wouldn’t make a hammer out of foam blocks,” Nair says. “We want to reach that level of reasoning in our work, which is something we’re working on now.”

Space inspiration

The inspiration for the work comes from Apollo 13, the doomed seventh crewed flight of the Apollo space program. After an oxygen tank in the ship’s service module exploded two days into the mission, crew members needed to make makeshift modifications to the carbon dioxide removal system.

Despite a dangerously tight window of time and extremely high tension among all aboard and at mission control, the rescue proved successful. Nair and collaborators hope this research will serve as a foundation for future robotics technology that could reason faster and without the burden of stress.

“They were able to make this filter, but the solution took a long time to come up with,” Nair says. “We want to make robots that can assist humans in these kinds of scenarios to take the pressure off of them to come up with innovative solutions and potentially save their lives.”

The researchers presented their work at the 2019 Robotics: Science and Systems conference.

Support for the work came, in part, from the National Science Foundation and the Office of Naval Research.