CSCE 439/839: Robotics: Algorithms and Applications
Spring 2022


Dr. Carrick Detweiler
Computer Science and Engineering
220 Schorr Center
carrick _at_
Office Hours: see Canvas page

Teaching Assistant

Adam Plowcha
aplowcha _at_
Office Hours: see Canvas page

Course Information

Lecture: MWF 10:30pm - 11:20pm in Avery 110
Lab: F 11:30pm - 12:20pm in Avery 110
Additional information can be found on Canvas

The primary source of information for this course is the Canvas Page for this class. This website a public version of the course materials, but see the course website for full details.

For detailed information on this course please see the course syllabus.

Course Description

Robots play an increasingly important role in our lives, from assembling our cars and cell phones to vacuuming our rugs and flying recon missions. To create systems that work in the real world, the field of robotics requires robust theory and algorithms that are tightly integrated with hardware that is designed with engineering expertise.

This will be a hands-on, lab-based course where you will implement the algorithms you learn about in class on a small balancing robot that you will build in lab. You will learn the fundamentals of robotics, as well as the cutting-edge in robotics research. Topics covered will include: control, navigation and path planing, obstacle avoidance, manipulation and grasping, and robotic vision processing. By the end of the course you will know why robots are not yet folding our clothes and driving our cars, but you will also learn what is needed to make these possible in the future.

You will interface with the embedded system that controls the robot by using learning to program in ROS (Robot Operating System). Programming will be in either C++ or Python, with an emphasis on C++. By the end of the course you will have a deep understanding of the design, programming, and interfacing of robot systems. This will prepare you for cutting edge careers in industry and research.


See the course schedule below for reading assignments and the general topics covered in the course. Unless otherwise noted, assignments are due via canvas before the start of class on the due date.

Course Survey
HW 1 (pdf,
Docker, ROS, and Arduino Introduction
In class Lab 1 (pdf)
Balboa Introduction
Lab 1 (pdf, ROS Code, Balboa Code)
HW 2 (pdf,
ROS, PID, Kalman Filters
Lab 2 (pdf)
mapping and sensors
Lab 3 (pdf,,, (optional))
Raspberry Pi and Ball Detection
Project (pdf,
Final Project

*Start dates are estimated for future assignments and are subject to change. Additional assignments will be added.
**Note html versions of assignments may lack some formatting or pictures found in the pdf version.

Course Schedule

Below is an approximate schedule of course topics. These are subject to change, assignments will be announced in class.

Week Topics
Week 1: Jan 17, starting Weds Introduction, Docker, ROS
Week 2: Jan 24 ROS, Balboa Robot, Sensors
Additional Readings: Example in class of communicaiton protocol

How not to write code

Balboa Robot Resources:
Balboa 32U4 Balancing Robot User's and Assmebly Guide
Balboa Arduino library
Balboa buying on Pololu
Other Balboa resources (pololu)
Balboa gear ratio chart
Balboa pinout
Balboa schematic
5-Channel reflectance sensor array

C Programming Resources:
Learn C The Hard Way, Zed Shaw (textbook on Amazon)
A wiki-style C "text book" alternative
Another online C text book
C Frequently Asked Questions from the days of Usenet

Python Programming Resources (I have not gone through all these courses, some may be better than others):
Free Python Course
Python for Beginners
Python Tutorial
More python resources

Information on Sensors:
IMUs (wikipedia)
Math behind IMU data fusion Overview of Micro-Electro-Mechanical (MEMs) systems
Accelerometer (wikipedia)
How an accelerometer works (sparkfun)
Short bit on how MEMs accelerometers work
More on MEMs accelerometers
More on accelerometers
Gyroscope (wikipedia)
Gyroscope (sparkfun)
MEMs Gyroscopes
Magnetometer (wikipedia)
Types of magnetometers
Converting 3-axis magnetometer reading to compass

Arduino-related things that might be useful for HW1:
Arduino Introduction
How to read a schematic
Arduino Uno R3 schematic
How to read a datasheet
Atmega328 datasheet
Arduino-function digital I/O
Arduino using buttons
Another button tutorial
Parallel versus Serial, UART (sparkfun)
Serial cummunication (sparkfun)
Serial communication (arduino)
Arduino UART serial communication
Week 3: Jan 31
ROS, PID Control
Week 4: Feb 7 PID Control, Sensors
Additional Readings: Example PID Matlab code shown in class

PID Control
Wikipedia PID control
PID Control with Matlab example (similar idea to what we covered in class)
Another PID description (with some tuning heuristics)
PID tuning heuristic (my preference is to start with PD ctrl first and not PI as discussed in this article)
Wikipedia Ziegler-Nichols tuning method
Longish video on Ziegler-Nichols method
Tuning balancing robot (not Balboa)
Good video discussing PID with autonomous cars

Configuration Space
Wikipedia Configuration Space (so-so)
2D Configuration space robot sim (cool if it works)
Video from the above website
Textbook chapter with formal description of C-space (LaVelle)
Week 5: Feb 14 Sensors, Kalman Filters
Additional Readings: Example Kalman Filter Matlab code shown in class

Kalman Filters
Wikipedia Kalman Filter
G. Welch and G. Bishop. An Introduction to the Kalman Filter
Kalman Filter math and pictorial explanation
Kalman Filter explanation and tutorial with Matlab
Wikipedia Extended Kalman Filter
A book in Python on all sorts of filters (long)

Particle Filters
Wikipedia Particle Filter
ROS Particle Filter
Bayesian Filtering Library (PF, KF, EKF, etc)
Particle Filter lecture notes (MIT: Hsiao, Plinval-Salgues, Miller)
Week 6: Feb 21 Data Fusion (Kalman, Markov, Monte Carlo Techniques)
Week 7: Feb 28 Fundamental Problems in Robotics, Reactive Control, Motion Planning
Week 8: March 7 Robot Localization
Week 9: March 14 NO CLASS, Spring Break
Week 10: March 21 Computer Vision
Additional Readings:
Color and Light
Wikipedia Color Vision
Wikipedia "The dress" Gold or Blue (human color perception example)

Color Spaces
HSV Color space
RGB, CMKY, HSV Color model description (youtube)
Wikipedia HSV color space

Color Blob and Feature Detection
HSV Color Detection in Matlab
Blob detection and connected components algorithm
Wikipedia Sobel Detector (edge detection)
Edge Detectors (Sobel and Laplacian)
Feature detection with OpenCV (good background as well as some openCV code)
Penn State lecture on correspondence matching
Wikipedia Harris Corner Detector
Wikpedia Kanade-Lucas-Tomasi feature tracker
CMU KLT lecture notes
Week 11: March 28 Visual Servo
Additional Readings:
Visual Servo
Hutchinson, Seth, Gregory D. Hager, and Peter I. Corke. A tutorial on visual servo control. IEEE transactions on robotics and automation 12, no. 5 (1996): 651-670.
Wikipedia visual servoing
Visual Servo Lecture Notes (Kemp, Georgia Tech)
Week 12: Apr 4 Feature and Landmark Detection
Additional Readings:
Camera Models and Calibration
Wikipedia Pinhole Camera
Rolling and global shutter
Toilet paper roll pinhole camera
Stanford Course Notes on Camera Models, Lenses, Calibration
MathWorks Camera Calibration Description
Matlab Camera Calibration Tool Reference
Matlab Camera Calibration (youtube)

Stereo Vision
Stereo vision tutorial
UNR Stereo vision tutorial
Penn State lecture notes on stereo vision
Wikipedia Triangulation (main technique for traditional stereo vision)
Stereo vision calibration in Matlab
Stereo Vision in ROS
Wikipedia Structure from motion
Stanford lecture on active stereo and more
Active Stereo

Feature and Landmark Detection
Wikipedia feature detection
Wikipedia Hough Transform
Hough transform basics
More on Hough transform
U. Washington Lecture ones on feature detection
Article on SIFT features
More details on SIFT features (above article was based on this one)
Matlab computer vision toolkit for feature detection
April tags
D. Scharstein and A. Briggs. Real-time recognition of self-similar landmarks. Image and Vision Computing, 19(11):763-772, September 2001.
Week 13: Apr 11 Final Project Topics
Week 14: Apr 18 Final Project Topics
Week 15: Apr 25 Final Project Topics
Week 16: May 2 Final Project Topics