Joseph Modayil

Info .... Research .... Papers

Research

The goal of my research is to understand how useful forms of knowledge can be learned directly from a robot's sensor and motor experience. Appropriate machine learning algorithms can reduce the gap between the forms of knowledge that people use to reason about the world (high-level concepts of space, objects, and human activities) and the data interfaces available on robots (low-level sensors and motors).

Topics:

Reinforcement learning for robots

Two forms of knowledge that are empirically verifiable are predictions (expectations about the future) and controls (behaviour policies that achieve a goal). Predictions and controls can be expressed using approximate value-functions, and a robot can learn these directly from its sensorimotor experience.
  • AAAI 2011 Workshop on Workshop on Lifelong Learning from Sensorimotor Experience


  • Horde: a scalable real-time architecture for learning knowledge from unsupervised sensorimotor experience
    Richard S. Sutton, Joseph Modayil, Michael Delp, Thomas Degris, Patrick M. Pilarski, Adam White, and Doina Precup. In Proc. of 10th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2011).
    [pdf]

  • Acquiring a Broad Range of Empirical Knowledge in Real Time by Temporal-Difference Learning
    Joseph Modayil, Adam White, Patrick M. Pilarski, Richard S. Sutton. In Proceedings of the 2012 IEEE International Conference on Systems, Man, and Cybernetics. (SMC 2012)
    [pdf] An earlier version appears at the workshop ERLARS 2012.

  • Scaling Life-long Off-policy Learning
    Adam White, Joseph Modayil, Richard S. Sutton. In Second Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-EpiRob). Also presented at the European Workshop on Reinforcement Learning 2012.
    ArXiv preprint 1206.6262.

  • Multi-timescale Nexting in a Reinforcement Learning Robot
    Joseph Modayil, Adam White, and Richard S. Sutton. In Proceedings of the International Conference on Adaptive Behaviour. SAB 2012. Also an ArXiv preprint 1112.1133
    [pdf]

  • Scaling up Knowledge for a Cognizant Robot
    Thomas Degris, Joseph Modayil. In the AAAI 2012 Spring Symposium Designing Intelligent Robots: Reintegrating AI.
    [pdf]

  • Multi-timescale Nexting in a Reinforcement Learning Robot
    Joseph Modayil, Adam White, and Richard S. Sutton. To appear in Adaptive Behavior
    [pdf]

Representational Development

Good representations are required for effective learning and planning on hard problems. A key research question is to understand how a robot can autonomously develop better representations from its experience.
  • Discovering Sensor Space: Constructing Spatial Embeddings That Explain Sensor Correlations
    Joseph Modayil. International Conference on Development and Learning (ICDL 2010). pages 120--125.
    [ pdf ]

Activity Recognition

By using RFID sensors to detect the sequence of objects an individual is touching, computers can infer what activities they are performing. This research has the potential to assist individuals with cognitive impairments.
  • Improving the recognition of interleaved activities
    Joseph Modayil, Tongxin Bai, Henry Kautz. In Ubicomp 2008.
    [ pdf]
  • Integrating sensing and cueing for more effective activity reminders
    Joseph Modayil, Rich Levinson, Craig Harman, David Halper, Henry Kautz. In the 2008 AAAI Fall Symposium on AI in Eldercare.
    [ pdf]

Bootstrap learning an object ontology

My thesis examined how a robot can autonomously discover representations for physical objects. The robot was able to plan with its learned representations to move an object to a specified goal location.
  • Bootstrap learning for object discovery
    Joseph Modayil and Benjamin Kuipers. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-04), pages 742--747.
    [pdf] [abstract] [Earlier symposium version]


  • Autonomous shape model learning for object localization and recognition
    Joseph Modayil and Benjamin Kuipers. In IEEE International Conference on Robotics and Automaton (ICRA-06), pages 2991--2996.
    [pdf] [abstract]

  • Bootstrap learning of foundational representations
    Benjamin Kuipers, Patrick Beeson, Joseph Modayil, and Jefferson Provost. Connection Science, 18(2), June 2006, pages 145-158.
    [pdf][abstract][Earlier workshop version]

  • Where do actions come from? Autonomous robot learning of objects and actions
    Joseph Modayil and Benjamin Kuipers. In AAAI Spring Symposium Series 2007, Control Mechanisms for Spatial Knowledge Processing in Cognitive / Intelligent Systems.
    [pdf][abstract]

  • Autonomous Development of a Grounded Object Ontology by a Learning Robot
    Joseph Modayil and Benjamin Kuipers. In the Proceedings of the Twenty-Second Conference on Artificial Intelligence (AAAI-07).
    [pdf][abstract]

  • Robot Developmental Learning of an Object Ontology Grounded in Sensorimotor Experience
    Joseph Modayil. Doctoral dissertation, Computer Sciences Department, University of Texas at Austin. [pdf][abstract]

  • The initial development of object knowledge by a learning robot
    Joseph Modayil and Benjamin Kuipers. Robotics and Autonomous Systems. Volume 56, Issue 11, pages 879--890.

Robot Mapping and Navigation

This work shows how to construct robot maps with combinations of topological and metrical information. Robots can use maps for navigation, and to represent regions that are safe for travel.
  • Local metrical and global topological maps in the Hybrid Spatial Semantic Hierarchy
    Benjamin Kuipers, Joseph Modayil, Patrick Beeson, Matt MacMahon, and Francesco Savelli. In IEEE International Conference on Robotics and Automation (ICRA-04).
    [pdf] [abstract]

  • Using the topological skeleton for scalable global metrical map-building
    Joseph Modayil, Patrick Beeson and Benjamin Kuipers. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-04), pages 1530--1536.
    [pdf] [abstract]


  • Building local safety maps for a wheelchair robot using vision and lasers
    (Best student paper!)
    Aniket Murarka, Joseph Modayil, and Benjamin Kuipers. In Canadian Conference on Computer and Robot Vision (CRV-06).
    [pdf] [abstract]


  • Integrating multiple representations of spatial knowledge for mapping, navigation, and communication
    Patrick Beeson, Matt MacMahon, Joseph Modayil, Aniket Murarka, Benjamin Kuipers, and Brian Stankiewicz. In the Proceedings of the Symposium on Interaction Challenges for Intelligent Assistants. AAAI Spring Symposium Series, March 2007
    [pdf][abstract]


  • Factoring the mapping problem: mobile robot map-building in the Hybrid Spatial Semantic Hierarchy
    Patrick Beeson, Joseph Modayil, and Benjamin Kuipers. International Journal of Robotics Research, 2009.
    (in press)