Paper review of SEDRo: A Simulated Environment for Developmental Robotics
Introduction
Most current AI models focus on one certain kind of abilities, such as language understanding, vision, or motor control etc. Although remarkable progress have been made through these researches, the learned capabilities are hard to be generalized to other tasks.
The authors in this paper claim some issues in previous researches:
· Targeting a single skill rather than diverse skills (the main issue)
· Use of refined and focused datasets rather than diverse and noisy datasets (the first common pattern)
· Relying on explicit rewards rather than on other mechanisms (the second common pattern)
· Too many necessary components rather than a sufficient set of the learning mechanism (the third common pattern)
They believe that these issues lead to models for single tasks in overfitting solutions that cannot be generalized to multiple tasks.
The authors claim that the key to solve the problem is regularization. Specifically, we need to regularize by enforcing the use of the same learning mechanism to conduct multiple tasks. In order to help solve the problem, they propose their ongoing effort to build a Simulated Environment for Developmental Robotics (SEDRo).
Simulated Environment for Developmental Robotics (SEDRo)
The proposed simulated environment is comprised of two main components, the learning agent and the simulated environment. Within the simulated environment, there are a caregiver character, surrounding objects in the environment and the body of the agent. The agent will interact with the simulated environment by controlling the muscles in its body according to the sensor signals. There are no explicit reward from the environment, instead the agent is responsible for generating rewards itself.
SEDRo provides diverse experiences similar to that of human infants from the stage of a fetus to 12 months of age. The structure of the proposed environment is shown in the following figure.
In SEDRo, the input-output signal changes according to the development of the agent based on the fact that human infants develop in a curriculum which scaffolds the involved sensory and motor capabilities. The sensory input in SEDRo consists of touch, vision, acceleration, gravity, and proprioceptors.
Social interaction is important for human development. So the authors attempt to build scenarios for social interaction in SEDRo although it can be time consuming.
Evaluation Framework for Non-verbal Agents
Because the environment cannot provide sufficient language exposure beyond the first 12 months, the agent cannot acquire advanced language beyond the first few words. So the developmental progress of the agent cannot be evaluated based on their ability to follow verbal instruction. They overcome this challenge by using studies from developmental psychology. The evaluation framework is developed by simulating established experiments from developmental psychology. There are multiple developmental milestones in multiple skill domains. The authors leverage such known developmental milestones to develop suites of simulated experiments for evaluating the development of the artificial agent. The evaluation will conduct multiple experiments and compare the results with those of the human participants.
Limitations and conclusion
As the authors mentioned in the paper, this work is still ongoing, so there are inevitably some limitations. A major limitation of the work as the authors point out is the lack of back and forth interactive conversation between the caregivers and the infant agent.
Besides that, it’s not very clear how the proposed method and environment resolve the issues about previous methods. For example, the third issue mentioned by the authors is that Too many necessary components rather than a sufficient set of the learning mechanism. But it’s not very clear how the authors pick the set of sufficient mechanism in the proposed environment.
In all, the proposed environment provide possibilities for researchers in the community to explore the learning mechanism for artificial general intelligence.
