CSCE 439/839: Robotics: Algorithms and Applications
Spring 2022

Instructor

Dr. Carrick Detweiler
Computer Science and Engineering
220 Schorr Center
carrick _at_ cse.unl.edu
Office Hours: see Canvas page

Teaching Assistant

Adam Plowcha
aplowcha _at_ gmail.com
Office Hours: see Canvas page

Course Information

Lecture: MWF 10:30pm - 11:20pm in Avery 110
Lab: F 11:30pm - 12:20pm in Avery 110
Additional information can be found on Canvas

The primary source of information for this course is the Canvas Page for this class. This website a public version of the course materials, but see the course website for full details.

For detailed information on this course please see the course syllabus.

Course Description

Robots play an increasingly important role in our lives, from assembling our cars and cell phones to vacuuming our rugs and flying recon missions. To create systems that work in the real world, the field of robotics requires robust theory and algorithms that are tightly integrated with hardware that is designed with engineering expertise.

This will be a hands-on, lab-based course where you will implement the algorithms you learn about in class on a small balancing robot that you will build in lab. You will learn the fundamentals of robotics, as well as the cutting-edge in robotics research. Topics covered will include: control, navigation and path planing, obstacle avoidance, manipulation and grasping, and robotic vision processing. By the end of the course you will know why robots are not yet folding our clothes and driving our cars, but you will also learn what is needed to make these possible in the future.

You will interface with the embedded system that controls the robot by using learning to program in ROS (Robot Operating System). Programming will be in either C++ or Python, with an emphasis on C++. By the end of the course you will have a deep understanding of the design, programming, and interfacing of robot systems. This will prepare you for cutting edge careers in industry and research.

Assignments

See the course schedule below for reading assignments and the general topics covered in the course. Unless otherwise noted, assignments are due via canvas before the start of class on the due date.

Assignment

Course Survey

HW 1 (pdf, 439-839-docker-hw1.zip read_serial.zip)
Docker, ROS, and Arduino Introduction

In class Lab 1 (pdf)
Balboa Introduction

Lab 1 (pdf, ROS Code, Balboa Code)

HW 2 (pdf, hw2-code.zip)
ROS, PID, Kalman Filters

Lab 2 (pdf)
mapping and sensors

Lab 3 (pdf, ball-detector.zip, bag-files.zip, docker-camera.zip (optional))
Raspberry Pi and Ball Detection

Project (pdf, landmark_self_sim.zip)
Final Project

*Start dates are estimated for future assignments and are subject to change. Additional assignments will be added.
**Note html versions of assignments may lack some formatting or pictures found in the pdf version.

Course Schedule

Below is an approximate schedule of course topics. These are subject to change, assignments will be announced in class.

Week	Topics
Week 1: Jan 17, starting Weds	Introduction, Docker, ROS
Week 2: Jan 24	ROS, Balboa Robot, Sensors
Additional Readings:	Example in class of communicaiton protocol How not to write code Balboa Robot Resources: Balboa 32U4 Balancing Robot User's and Assmebly Guide Balboa Arduino library Balboa buying on Pololu Other Balboa resources (pololu) Balboa gear ratio chart Balboa pinout Balboa schematic 5-Channel reflectance sensor array C Programming Resources: Learn C The Hard Way, Zed Shaw (textbook on Amazon) A wiki-style C "text book" alternative Another online C text book C Frequently Asked Questions from the days of Usenet Python Programming Resources (I have not gone through all these courses, some may be better than others): Free Python Course Python for Beginners Python Tutorial More python resources Information on Sensors: IMUs (wikipedia) Math behind IMU data fusion Overview of Micro-Electro-Mechanical (MEMs) systems Accelerometer (wikipedia) How an accelerometer works (sparkfun) Short bit on how MEMs accelerometers work More on MEMs accelerometers More on accelerometers Gyroscope (wikipedia) Gyroscope (sparkfun) MEMs Gyroscopes Magnetometer (wikipedia) Magnetometers Types of magnetometers Converting 3-axis magnetometer reading to compass Arduino-related things that might be useful for HW1: Arduino Introduction How to read a schematic Arduino Uno R3 schematic How to read a datasheet Atmega328 datasheet Arduino-function digital I/O Arduino using buttons Another button tutorial Parallel versus Serial, UART (sparkfun) Serial cummunication (sparkfun) Serial communication (arduino) Arduino UART serial communication
Week 3: Jan 31	ROS, PID Control
Week 4: Feb 7	PID Control, Sensors
Additional Readings:	Example PID Matlab code shown in class PID Control Wikipedia PID control PID Control with Matlab example (similar idea to what we covered in class) Another PID description (with some tuning heuristics) PID tuning heuristic (my preference is to start with PD ctrl first and not PI as discussed in this article) Wikipedia Ziegler-Nichols tuning method Longish video on Ziegler-Nichols method Tuning balancing robot (not Balboa) Good video discussing PID with autonomous cars Configuration Space Wikipedia Configuration Space (so-so) 2D Configuration space robot sim (cool if it works) Video from the above website Textbook chapter with formal description of C-space (LaVelle)
Week 5: Feb 14	Sensors, Kalman Filters
Additional Readings:	Example Kalman Filter Matlab code shown in class Kalman Filters Wikipedia Kalman Filter G. Welch and G. Bishop. An Introduction to the Kalman Filter Kalman Filter math and pictorial explanation Kalman Filter explanation and tutorial with Matlab Wikipedia Extended Kalman Filter A book in Python on all sorts of filters (long) Particle Filters Wikipedia Particle Filter ROS Particle Filter Bayesian Filtering Library (PF, KF, EKF, etc) Particle Filter lecture notes (MIT: Hsiao, Plinval-Salgues, Miller)
Week 6: Feb 21	Data Fusion (Kalman, Markov, Monte Carlo Techniques)
Week 7: Feb 28	Fundamental Problems in Robotics, Reactive Control, Motion Planning
Week 8: March 7	Robot Localization
Week 9: March 14 NO CLASS, Spring Break
Week 10: March 21	Computer Vision
Additional Readings:	Color and Light Wikipedia Color Vision Wikipedia "The dress" Gold or Blue (human color perception example) Color Spaces HSV Color space RGB, CMKY, HSV Color model description (youtube) Wikipedia HSV color space Color Blob and Feature Detection HSV Color Detection in Matlab Blob detection and connected components algorithm Wikipedia Sobel Detector (edge detection) Edge Detectors (Sobel and Laplacian) Feature detection with OpenCV (good background as well as some openCV code) Penn State lecture on correspondence matching Wikipedia Harris Corner Detector Wikpedia Kanade-Lucas-Tomasi feature tracker CMU KLT lecture notes
Week 11: March 28	Visual Servo
Additional Readings:	Visual Servo Hutchinson, Seth, Gregory D. Hager, and Peter I. Corke. A tutorial on visual servo control. IEEE transactions on robotics and automation 12, no. 5 (1996): 651-670. Wikipedia visual servoing Visual Servo Lecture Notes (Kemp, Georgia Tech)
Week 12: Apr 4	Feature and Landmark Detection
Additional Readings:	Camera Models and Calibration Wikipedia Pinhole Camera Rolling and global shutter Toilet paper roll pinhole camera Stanford Course Notes on Camera Models, Lenses, Calibration MathWorks Camera Calibration Description Matlab Camera Calibration Tool Reference Matlab Camera Calibration (youtube) Stereo Vision Stereo vision tutorial UNR Stereo vision tutorial Penn State lecture notes on stereo vision Wikipedia Triangulation (main technique for traditional stereo vision) Stereo vision calibration in Matlab Stereo Vision in ROS Wikipedia Structure from motion Stanford lecture on active stereo and more Active Stereo Feature and Landmark Detection Wikipedia feature detection Wikipedia Hough Transform Hough transform basics More on Hough transform U. Washington Lecture ones on feature detection Article on SIFT features More details on SIFT features (above article was based on this one) Matlab computer vision toolkit for feature detection April tags D. Scharstein and A. Briggs. Real-time recognition of self-similar landmarks. Image and Vision Computing, 19(11):763-772, September 2001.
Week 13: Apr 11	Final Project Topics
Week 14: Apr 18	Final Project Topics
Week 15: Apr 25	Final Project Topics
Week 16: May 2	Final Project Topics

CSCE 439/839: Robotics: Algorithms and Applications Spring 2022