Robot, object, action!

Robotic demonstrators developed by European researchers produce compelling evidence that ‘thinking-by-doing’ is the machine cognition paradigm of the future. Robots act on objects and teach themselves in the process.

European researchers at the PACO-PLUS project have developed robotic demonstrators to illustrate a new approach to robotic cognition. The team uses a concept called ‘object-action complexes’ (OACs), pronounced oaks.

These complexes represent a combination of perception of and action upon any given object. The OACs are recorded by the robot every time it performs an action. These recorded actions can be saved and exchanged with other robots, and over time the team hope it will lead to entire libraries of OACs that can be exchanged between researchers.

But that is just a major bonus of this particular system. The primary thrust was to show that robots could teach themselves simple actions, leading to more complex actions and, hopefully, leading to abstract thought in the future.

The PACO-PLUS team performed a large number of demonstrations with humanoid robots and the results so far are very promising.

Autonomous exploration

With autonomous exploration, a robot can patiently explore thousands of potential ways to interact with an object it has no prior knowledge of – learning over time that grasping it in one way is better than another, that it balances better in one direction than another, etc.

“The first thing a robot will do when it encounters an object for the first time is to lift it before its eyes and then rotate the object so the robot gets a look at it from all angles. After that, [the robot] will be able to identify the object from any angle,” explains Tamim Asfour, leader of the Humanoids Research Group at the Institute for Anthropomatics at the Karlsruhe Institute of Technology (KIT) in Germany and co-coordinator of the PACO-PLUS project.

One can see how complex actions could arise over time, as a robot progressively teaches itself how to interact with a cup, jug, handles and doors. Similarly, as the robot’s experience with different objects expands, its options expand, too, and it could learn to use a key with a hole in a door to see if that action will lead to any result.

This is the core of the OAC concept, that interaction with the environment leads to ever-more sophisticated strategies over time, and ultimately gives rise to ‘intelligence’.

Of course, the autonomous exploration is extremely time consuming, unsurprising when one recalls that it took hundreds of millions of years to evolve intelligent life. So another method for teaching robots is learning from human observation and human coaching, and PACO-PLUS developed a large range of successful demonstrators with this method as well.

Human coaching

With coaching, robots can watch humans perform an action, for example wiping a table, and then they can imitate it. It is astonishing how easy it is for robots to mimic complex actions like this.

Similarly, a coach can guide the robots through a complex task, for example putting a cup in a dishwasher. At each stage the coach tells the robot ‘open the dishwasher’, ‘take this object/cup’, ‘put it in the dishwasher’. Over time, the robot learns these sorts of actions.

This is a phenomenally sophisticated demonstration of robotic learning, and it was achieved, using a new approach, in the space of just four years. As such it stands as a remarkable testament to the work ethic of the PACO-PLUS team and the power of its approach.

“Of course, we did a lot of other work too,” notes Asfour. “For example, we looked at some situations with more traditional industrial robots, to see how our work could be applied there.”

The work has been very well received in the wider robotic community, and some partners are looking at how they can integrate some of the results into other projects. In the meantime, PACO-PLUS is currently pursuing further research funding to perfect the work achieved.

But already PACO-PLUS has dramatically advanced the field of robotic cognition and helped to set the research and development agenda for the next decade.

This is the second of a two-part PACO-PLUS feature. Part 1.