CSCE 439/839: Robotics: Algorithms and Applications
Spring 2020 (COVID-19 Edition)

Instructor

Dr. Carrick Detweiler
Computer Science and Engineering
220 Schorr Center
carrick _at_ cse.unl.edu
Office Hours:
carrick.youcanbook.me and by appointment.

Teaching Assistant

Ajay Shankar
ashankar _at_ cse.unl.edu
Office Hours:
by appointment.

Course Information

Lecture: MWF 3:30pm - 4:20pm in Avery 119
Lab: M 4:30pm - 5:20pm in Avery 119
Online on Canvas

For detailed information on this course please see the new COVID-19 course syllabus (original here).

COVID-19 and Moving to Online Format

Given COVID-19 and the move to an online course format, things will be different for all of us. Please bear with me as we adjust. I will do my best to communicate any changes and my expectations. Please do the same in return and ask if you have questions or concerns. I will primarily use Canvas for communication, but make sure to check your university email regularly as well.

The class will be run asynchronously with videos and reading materials posted in canvas. We will hold whole-class office hours during some of the regularly scheduled class times. This will allow us to review some of the topics in more depth based on student questions. Dates for whole-class office hours will be posted on Canvas. In addition, you can schedule time to meet with me individually at carrick.youcanbook.me.

The major change to the course content and assessment is that there will be no more lab group assignments and no final project. Instead, there will be an additional homework (the plan is for two more this semester) and there will be some short online activities and quizzes to help assess participation.

If you have difficulty accessing the online materials due to internet problems, sickness, family care, or anything else, please let me know and we will figure out a solution. Finally, if I do not respond to your messages within 24 hours, please reach out to Ajay in Canvas (or feel free to copy him on any messages).

Course Description

Robots play an increasingly important role in our lives, from assembling our cars and cell phones to vacuuming our rugs and flying recon missions. To create systems that work in the real world, the field of robotics requires robust theory and algorithms that are tightly integrated with hardware that is designed with engineering expertise.

This will be a hands-on, lab-based course where you will implement the algorithms you learn about in class on a small balancing robot that you will build in lab. You will learn the fundamentals of robotics, as well as the cutting-edge in robotics research. Topics covered will include: control, navigation and path planing, obstacle avoidance, manipulation and grasping, and robotic vision processing. By the end of the course you will know why robots are not yet folding our clothes and driving our cars, but you will also learn what is needed to make these possible in the future.

You will interface with the embedded system that controls the robot by using learning to program in ROS (Robot Operating System). Programming will be in either C++ or Python, with an emphasis on C++. By the end of the course you will have a deep understanding of the design, programming, and interfacing of robot systems. This will prepare you for cutting edge careers in industry and research.

Assignments

See the course schedule below for reading assignments and the general topics covered in the course. Unless otherwise noted, assignments are due via canvas before the start of class on the due date.

Assignment Start Date* Due Date
Course Survey Monday, Jan 13th Tuesday, Jan 14th midnight
HW 1 (pdf, read_serial.zip)
ROS and Arduino Introduction
Monday, Jan 13th Checkoff 1: Tuesday, Jan 21st
Due: Friday, Jan 31st
In-Class Lab 1 (pdf) Monday, Jan 27 Wednesday, Feb 5
Lab 1 (pdf, ROS Code, Balboa Code)
Monday, Feb 3 Checkoff 1: Mon, Feb 10
Checkoff 2: Mon, Feb 17
Due: Mon, Feb 24
Lab 2 (pdf)
Monday, Feb 24 Checkoff 1: Weds, March 4
Checkoff 2: Mon, March 9
Due: Mon, March 16
HW 2 (pdf, hw2Code.tgz)
ROS, PID, Kalman Filters
Mon, March 30 Due: 3:30 CT, Weds, April 15
HW 3 (pdf, ball_detector.tgz, landmark_self_sim.tgz , bag-files)
ROS, PID, Kalman Filters
Thurs, April 16 Due: Midnight, Tues, May 5

*Start dates are estimated for future assignments and are subject to change. Additional assignments will be added.
**Note html versions of assignments may lack some formatting or pictures found in the pdf version.

Course Books

Suggested: G. Dudek and M. Jenkin, Computational Principles of Mobile Robotics, Cambridge University Press, 2nd Edition, 2010.

Course Schedule

Below is an approximate schedule of course topics. These are subject to change, assignments will be announced in class.


Week Topics Reading
Week 1: Jan 13 Introduction, ROS ros.org tutorials
Week 2: Jan 20 ROS, Balboa Robot, Sensors CPMR 4.1-4.8
Additional Readings:
Additional Readings: Serial protocol from class example

How not to write code

Balboa Robot Resources:
Balboa 32U4 Balancing Robot User’s and Assmebly Guide
Balboa Arduino library
Balboa buying on Pololu
Other Balboa resources (pololu)
Balboa gear ratio chart
Balboa pinout
Balboa schematic
5-Channel reflectance sensor array

C Programming Resources:
Learn C The Hard Way, Zed Shaw (textbook on Amazon)
A wiki-style C "text book" alternative
Another online C text book
C Frequently Asked Questions from the days of Usenet

Python Programming Resources (I have not gone through all these courses, some may be better than others):
Free Python Course
Python for Beginners
Python Tutorial
More python resources

Information on Sensors:
IMUs (wikipedia)
Math behind IMU data fusion Overview of Micro-Electro-Mechanical (MEMs) systems
Accelerometer (wikipedia)
How an accelerometer works (sparkfun)
Short bit on how MEMs accelerometers work
More on MEMs accelerometers
More on accelerometers
Gyroscope (wikipedia)
Gyroscope (sparkfun)
MEMs Gyroscopes
Magnetometer (wikipedia)
Magnetometers
Types of magnetometers
Converting 3-axis magnetometer reading to compass

Arduino-related things that might be useful for HW1:
Arduino Introduction How to read a schematic
Arduino Uno R3 schematic
How to read a datasheet
Atmega328 datasheet
Arduino-function digital I/O
Arduino using buttons
Another button tutorial
Parallel versus Serial, UART (sparkfun)
Serial cummunication (sparkfun)
Serial communication (arduino)
Arduino UART serial communication
Week 3: Jan 27
ROS, Sensors, PID Control CPMR 3.1.3
Week 4: Feb 3 PID Control, Configuration Space CPMR 6.1-6.2
Additional Readings: Example Matlab code shown in class

PID Control
Wikipedia PID control
PID Control with Matlab example (similar idea to what we covered in class)
Another PID description (with some tuning heuristics)
PID tuning heuristic (my preference is to start with PD ctrl first and not PI as discussed in this article)
Wikipedia Ziegler-Nichols tuning method
Longish video on Ziegler-Nichols method
Tuning balancing robot (not Balboa)
Good video discussing PID with autonomous cars

Configuration Space
Wikipedia Configuration Space (so-so)
2D Configuration space robot sim (cool if it works)
Video from the above website
Textbook chapter with formal description of C-space (LaVelle)
Week 5: Feb 10 Fundamental Problems in Robotics CPMR 2.1-2.6
Week 6: Feb 17 Reactive Control, Motion Planning CPMR 6.3
Week 7: Feb 24 Data Fusion (Kalman, Markov, Monte Carlo Techniques) CPMR 4.9
Week 8: March 2 Robot Localization CPMR 8.1-8.7
Additional Readings: Kalman example code from class

Bug Algorithms
CMU Bug algorithm lecture notes
Bug 2 with ROS example
I. Kamon, E. Rimon, and E. Rivlin, TangentBug: A Range-Sensor-Based Navigation Algorithm, The International Journal of Robotics Research, vol. 17, no. 9, pp. 934-953, 1998.

Kalman Filters
Wikipedia Kalman Filter
G. Welch and G. Bishop. An Introduction to the Kalman Filter
Kalman Filter math and pictorial explanation
Kalman Filter explanation and tutorial with Matlab
Wikipedia Extended Kalman Filter
A book in Python on all sorts of filters (long)

Particle Filters
Wikipedia Particle Filter
ROS Particle Filter
Bayesian Filtering Library (PF, KF, EKF, etc)
Particle Filter lecture notes (MIT: Hsiao, Plinval-Salgues, Miller)
Week 9: March 9 Computer Vision CPMR 5.1-5.4
Week 10: March 16: NO CLASS, COVID-19: Virtual Class from now
Week 11: March 23: NO CLASS, Spring Break
Week 12: March 30 Computer Vision CPMR 5.1-5.4
Additional Readings:
Color and Light
Wikipedia Color Vision
Wikipedia "The dress" Gold or Blue (human color perception example)

Camera Models and Calibration
Wikipedia Pinhole Camera
Rolling and global shutter
Toilet paper roll pinhole camera
Stanford Course Notes on Camera Models, Lenses, Calibration
MathWorks Camera Calibration Description
Matlab Camera Calibration Tool Reference
Matlab Camera Calibration (youtube)

Color Spaces
HSV Color space
RGB, CMKY, HSV Color model description (youtube)
Wikipedia HSV color space
Week 13: Apr 6 Feature Detection, Stereo Vision CPMR 5.5-5.8
Additional Readings:
Color Blob and Feature Detection
HSV Color Detection in Matlab
Blob detection and connected components algorithm
Wikipedia Sobel Detector (edge detection)
Edge Detectors (Sobel and Laplacian)
Feature detection with OpenCV (good background as well as some openCV code)
Penn State lecture on correspondence matching
Wikipedia Harris Corner Detector
Wikpedia Kanade-Lucas-Tomasi feature tracker
CMU KLT lecture notes

Stereo Vision
Stereo vision tutorial
UNR Stereo vision tutorial
Penn State lecture notes on stereo vision
Wikipedia Triangulation (main technique for traditional stereo vision)
Stereo vision calibration in Matlab
Stereo Vision in ROS
Wikipedia Structure from motion
Stanford lecture on active stereo and more
Active Stereo
Week 14: Apr 13 Feature and Landmark Detection, Visual Servo
Additional Readings:
Feature and Landmark Detection
Wikipedia feature detection
Wikipedia Hough Transform
Hough transform basics
More on Hough transform
U. Washington Lecture ones on feature detection
Article on SIFT features
More details on SIFT features (above article was based on this one)
Matlab computer vision toolkit for feature detection
April tags
D. Scharstein and A. Briggs. Real-time recognition of self-similar landmarks. Image and Vision Computing, 19(11):763-772, September 2001.

Visual Servo
Hutchinson, Seth, Gregory D. Hager, and Peter I. Corke. A tutorial on visual servo control. IEEE transactions on robotics and automation 12, no. 5 (1996): 651-670.
Wikipedia visual servoing
Visual Servo Lecture Notes (Kemp, Georgia Tech)
Week 15: Apr 20 Higher Level Control and Planning
Additional Readings:
Higher Level Control and Planning
Wikipedia subsumption architecture
Rod Brooks "Bottom-Up Robotics" (youtube)
CMU lecture notes on robot motion planning, visibility graphs, Voronoi, etc.
Columbia lecture notes on visibility graphs
Wikipedia general Voronoi info
Voronoi path planning
Week 16: Apr 27 Additional Topics