Overview

Affordances are inherent properties of objects and environments that enable certain actions to be executed. It is a concept originating from psychology that has gained increased attention in robotics as it provides a way to interpret environments and optimise planning and execution of actions. At the same time, reinforcement learning has become a crucial part of problem solving in robotics as well.

In this paper, we explore the limitations of reinforcement learning in the context of affordances. We specifically focus on the physical execution of affordances and show that they provide rather complex situations—proving to be quite difficult for reinforcement learning in certain cases. We also argue that the challenges can be overcome by splitting affordances into simpler components and propose a taxonomy of these components to accommodate this concept.

Finally, we explore whether or not virtual environments are suited to simulate real world affordances in the first place. We come to the conclusion that, while they are not inherently ill-suited, virtual environments do encounter some limitations that need to be accounted for when it is employed for real world simulation.

Our Approach

For the experiments we selected various affordances and implemented them in two environment. The first is an OpenAI Gymnasium environment with Mujoco and PyBullet as the physics engine, along with an implementation of DQN and PPO for learning algorithm. The other is an Unreal Environment in virtual reality, as well as an motion tracking setup, via the use of OptiTrack Technology.

We used the OpenAI environment to explore what kind of affordance properties may cause issues in regards to the reinforcement learning process itself but also in relation to the physics engine used. Based on our observations we propose a taxonomy for affordance components which are required to achieve the desired end state for fulfilling an affordance.

The Unreal Environment is used to compare the accuracy of the physics engine to the real world and evaluate whether or not simulated environments are inherently ill suited to certain kinds of task and therefore training a system in these task would be unreasonable without real world data.