Carnegie Mellon University Personal Robotics Lab

RI 16-662: Robot Autonomy Spring 2017

Siddhartha Srinivasa (siddh AT
Katharina Muelling (kmuelling AT

Gilwoo Lee (gilwool AT
Rosario Scalise (rscalise AT
Please Contact TAs at: or preferably, post on Piazza!
Office Hours: T 1:30pm-2:30pm, W 11am-12pm
Personal Robotics Lab, NSH 4502

Lectures: M/W, 3:00-4:20, GHC 4307


Robot autonomy delves into the interplay between perception, manipulation, navigation, and learning required to develop fully autonomous systems. We will focus on application domains like the home, retail, and healthcare and identify common themes and key bottlenecks. We will discuss the state of the art algorithms, their computational and hardware requirements, and their limitations. An end-to-end system often requires mixing and matching various algorithms and you will learn some tried and true methods for making systematic decisions. Students will learn how to address clutter and uncertainty in manipulation tasks, develop robust object recognition algorithms in real-world scenes, navigate safely in human spaces, and build behavior engines for high-level tasks, among many other topics.

Important Dates

January 18
HW0 out
January 23
Tutorial on Python/OpenRAVE
Jan 27
HW0 due; HW1 out
Feb 10
HW1 due; HW2 out
Feb 20
Project Proposals due
Feb 24
HW2 due; HW3 out
Mar 10
HW3 due; HW4 out
Mar 27
Project Mid-term Report due
Apr 5
HW4 due
Apr 26 - May 3
Final Project Presentations
May 5
Project Final Report due

Homework Policy

You are required to work in groups of 3-4 for all homeworks (except HW0, which you must submit individually). Each person has 2 slip days per semester that can be used for any homework, but not project deadlines. After that, your team will lose 50% of the total grade for that assignment for every day you are late.


Sign up!

Each class 3 students will volunteer to take notes for the class. Every student must scribe at least once per semester, and this will count for 5% of your final grade. Each scribe group must submit one document, typed, and emailed to the TAs as a PDF (if using latex please also submit the .tex file). Scribe document is due before class one week following the class the notes were taken. Please include the names of all group memebers.

Scribe Notes Example

Class Projects

Your project is worth 40% of your total grade, and must be done in groups of 4-5 of your choosing. They must include a real robotic system. We've gathered a few ideas from different groups, but you are free to contact professors on your own and pick a different project.

Old Projects

Title Team
Towers of Hanoi Darukhanavala, Lycas, Mittal, Suresh
The Tick-bot Hussain, Zhang, Joseph, Chanda
HERB the Builder Hamm, Komoroski, Roberts, Viola
Shooting hoops Bajaj, Hu, Kumar, Thomas
Picasso-bot Coyne, Rao, Subramanian

Software Setup for Assignments

OpenRAVE will be used to complete course assignments.

Windows/Mac/Non-Ubuntu Linux

If you use Windows or Mac OSX you should use our Linux Virtual Machine image available for download here. This includes all the software and configuration necessary to use OpenRAVE in Ubuntu 14.04 LTS. You will need to install Oracle VirtualBox to run the VM. After installing VirtualBox select "Import Appliance" from the File menu and select the (unzipped) VM image. The root password for the account within VM is bb8robot.

After logging into the VM, do the following:

$ sudo apt-get update && sudo apt-get dist-upgrade


The VM currently requires ~4GB of hard disk space, though could take up to 30GB if additional software is installed. It is pre-configured to use 4GB of RAM, 1 CPU, and 64MB of video memory. If you have 4GB of total RAM available or less, set that value to 50% of your total RAM during installation. Similarly with GPU memory. If you have a quad-core CPU, you may increase CPUs used to 2.

Ubuntu Linux 14.04

If you run Ubuntu 14.04 natively, you can choose to use the provided VM image, or install the software yourself. Do NOT install the OpenRAVE version available on the website. Instead, follow the instructions under Ubuntu Packages here. Then run:

$ sudo apt-get update && sudo apt-get install pr-openrave

We recommend using either UXTerm or urxvt instead of the default gnome-terminal to avoid an ugly character rendering bug with IPython and OpenRAVE.


Day Date Topic Reading Notes
W 18 Jan Introduction OpenRAVE. URL Slide
M 23 Jan Tutorial on Python/Openrave
W 25 Jan Grasping Theory Grasp quality measures, Suarez et al, 2006. PDF
Handey: A robot system that recognizes, plans, and manipulates, Lozano-Perez et al, ICRA 1987. PDF
Automatic Grasp Planning Using Shape Primitives, Miller et al, ICRA 2003. PDF
Slide Note Images Scribe
M 30 Jan Quality Metrics and Grasp Tables Analysis of Multifingered Hands, Kerr and Roth, IJRR 1986. PDF
Task-Oriented Optimal Grasping by Multifingered Robot Hands, Li and Sastry, 1988. PDF
Knowledge-Based Prehension: Capturing Human Dexterity, Iberall et al., 1988. PDF
Data-driven grasping, Goldfeder and Allen, 2011. PDF
Images Scribe
W 1 Feb Guest Lecture by Michael Koval: Physics-Based Grasping under Uncertainty Slide Scribe
M 6 Feb Configuration Spaces Spatial Planning: A Configuration Space Approach, Lozano-Perez, 1980. PDF Notes
W 8 Feb Motion Planning, Piano Movers Problem Real-Time Configuration Space Transforms for Obstacle Avoidance, Newman and Branicky, 1991. PDF
Computation of Configuration-Space Obstacles Using the Fast Fourier Transform, Kavraki, 1995 PDF
Notes Scribe
M 13 Feb Randomized Search Analysis of Probabilistic Roadmaps for Path Planning, Kavraki, Kolountzakis, and Latombe, 1998. PDF
The Gaussian sampling strategy for probabilistic roadmap planners, Boor Etal 1999. PDF
A Randomized Algorithm for Robot Path Planning Based on Lazy Evaluation, Bohlin Etal 2001. PDF
On the Relationship Between Classical Grid Search and Probabilistic Roadmaps, Lavalle Etal 2010. PDF
Notes(PRM) Notes(RRT) Scribe
W 15 Feb Discrete Search RRT-Connect: An Efficient Approach to Single-Query Path Planning, Kuffner and Lavalle, 1999. PDF Notes
M 20 Feb Planning with Constraints, Kinodynamic Planning Creating High-quality Paths for Motion Planning, Geraerts and Overmars, 2007. PDF Notes Scribe
W 22 Feb Planning with Costs Approaches for Heuristically Biasing RRT Growth, Urmson and Simmons, 2003. PDF
Anytime RRTs, Ferguson and Stentz, 2006. PDF
Notes Scribe
M 27 Feb Planning with Constraints Task Space Regions: A Framework for Pose-Constrained Manipulation Planning, Berenson Etal 2011. PDF Notes Scribe
W 1 Mar Motion Planning: Incremental densification Scribe
M 6 Mar Hybrid systems Scribe
M 8 Mar Guest Lecture by Shushman Choudhury: Pareto-Optimal Search over Configuration Space Beliefs for Anytime Motion Planning Densification Strategies for Anytime Motion Planning over Large Dense Roadmaps PDF
Pareto-Optimal Search over Configuration Space Beliefs for Anytime Motion Planning PDF
Slides Scribe
M 20 Mar Closed Loop Control Slides
W 22 Mar Kalman Filter Slides Scribe
M 27 Mar Localization Slides(Localization) Slides(Local Planning) Scribe
W 29 Mar MDP Slides
M 3 Apr Learning Slides Scribe
W 5 Apr Imitation Learning Slides Scribe
M 10 Apr Learning Slides
W 12 Apr Task Planning FF-Plan FFRob Slides