CSCE 439/839: Robotics: Algorithms and Applications
Spring 2022
Instructor
Dr. Carrick Detweiler
Computer Science and Engineering
220 Schorr Center
carrick _at_ cse.unl.edu
Office Hours: see Canvas page
Teaching Assistant
Adam Plowcha
aplowcha _at_ gmail.com
Office Hours: see Canvas page
Course Information
Lecture: MWF 10:30pm - 11:20pm in Avery 110
Lab: F 11:30pm - 12:20pm in Avery 110
Additional information can be found on Canvas
The primary source of information for this course is the Canvas Page for this class. This website a public version of the course materials, but see the course website for full details.
For detailed information on this course please see the course syllabus.
Course Description
Robots play an increasingly important role in our lives, from
assembling our cars and cell phones to vacuuming our rugs and flying
recon missions. To create systems that work in the real world, the
field of robotics requires robust theory and algorithms that are
tightly integrated with hardware that is designed with engineering
expertise.
This will be a hands-on, lab-based course where you will
implement the algorithms you learn about in class on a small
balancing robot that you will build in lab. You will learn the
fundamentals of robotics, as well as the cutting-edge in robotics
research. Topics covered will include: control, navigation and path
planing, obstacle avoidance, manipulation and grasping, and robotic
vision processing. By the end of the course you will know why robots
are not yet folding our clothes and driving our cars, but you will also
learn what is needed to make these possible in the future.
You will interface with the embedded system that controls
the robot by using learning to program in ROS (Robot Operating
System). Programming will be in either C++ or Python, with
an emphasis on C++. By the end of the course you will have a deep
understanding of the design, programming, and interfacing of robot
systems. This will prepare you for cutting edge careers in industry
and research.
Assignments
See the course schedule below for reading assignments and the general
topics covered in the course. Unless otherwise noted, assignments are
due via canvas before the start of class on the due date.
Assignment
|
Course Survey
|
HW 1 (pdf, 439-839-docker-hw1.zip read_serial.zip)
Docker, ROS, and Arduino Introduction
|
In class Lab 1 (pdf)
Balboa Introduction
|
Lab 1 (pdf, ROS Code, Balboa Code)
|
HW 2 (pdf, hw2-code.zip)
ROS, PID, Kalman Filters
|
Lab 2 (pdf)
mapping and sensors
|
Lab 3 (pdf,
ball-detector.zip,
bag-files.zip,
docker-camera.zip (optional))
Raspberry Pi and Ball Detection
|
Project (pdf,
landmark_self_sim.zip)
Final Project
|
*Start dates are estimated for future assignments and are subject to change. Additional assignments will be added.
**Note html versions of assignments may lack some formatting or pictures found in the pdf version.
Course Schedule
Below is an approximate schedule of course topics. These are subject to
change, assignments will be announced in class.
Week |
Topics |
Week 1: Jan 17, starting Weds |
Introduction, Docker, ROS |
Week 2: Jan 24 |
ROS, Balboa Robot, Sensors |
Additional Readings: |
Example in class of communicaiton protocol
How not to write code
Balboa Robot Resources:
Balboa 32U4 Balancing Robot User's and Assmebly Guide
Balboa Arduino library
Balboa buying on Pololu
Other Balboa resources (pololu)
Balboa gear ratio chart
Balboa pinout
Balboa schematic
5-Channel reflectance sensor array
C Programming Resources:
Learn C The Hard Way, Zed Shaw (textbook on Amazon)
A wiki-style C "text book" alternative
Another online C text book
C Frequently Asked Questions from the days of Usenet
Python Programming Resources (I have not gone through all these courses, some may be better than others):
Free Python Course
Python for Beginners
Python Tutorial
More python resources
Information on Sensors:
IMUs (wikipedia)
Math behind IMU data fusion
Overview of Micro-Electro-Mechanical (MEMs) systems
Accelerometer (wikipedia)
How an accelerometer works (sparkfun)
Short bit on how MEMs accelerometers work
More on MEMs accelerometers
More on accelerometers
Gyroscope (wikipedia)
Gyroscope (sparkfun)
MEMs Gyroscopes
Magnetometer (wikipedia)
Magnetometers
Types of magnetometers
Converting 3-axis magnetometer reading to compass
Arduino-related things that might be useful for HW1:
Arduino Introduction
How to read a schematic
Arduino Uno R3 schematic
How to read a datasheet
Atmega328 datasheet
Arduino-function digital I/O
Arduino using buttons
Another button tutorial
Parallel versus Serial, UART (sparkfun)
Serial cummunication (sparkfun)
Serial communication (arduino)
Arduino UART serial communication
|
Week 3: Jan 31
|
ROS, PID Control |
Week 4: Feb 7 |
PID Control, Sensors |
Additional Readings: |
Example PID Matlab code shown in class
PID Control
Wikipedia PID control
PID Control with Matlab example (similar idea to what we covered in class)
Another PID description (with some tuning heuristics)
PID tuning heuristic (my preference is to start with PD ctrl first and not PI as discussed in this article)
Wikipedia Ziegler-Nichols tuning method
Longish video on Ziegler-Nichols method
Tuning balancing robot (not Balboa)
Good video discussing PID with autonomous cars
Configuration Space
Wikipedia Configuration Space (so-so)
2D Configuration space robot sim (cool if it works)
Video from the above website
Textbook chapter with formal description of C-space (LaVelle)
|
Week 5: Feb 14 |
Sensors, Kalman Filters |
Additional Readings: |
Example Kalman Filter Matlab code shown in class
Kalman Filters
Wikipedia Kalman Filter
G. Welch and G. Bishop. An Introduction to the Kalman Filter
Kalman Filter math and pictorial explanation
Kalman Filter explanation and tutorial with Matlab
Wikipedia Extended Kalman Filter
A book in Python on all sorts of filters (long)
Particle Filters
Wikipedia Particle Filter
ROS Particle Filter
Bayesian Filtering Library (PF, KF, EKF, etc)
Particle Filter lecture notes (MIT: Hsiao, Plinval-Salgues, Miller)
|
Week 6: Feb 21 |
Data Fusion (Kalman, Markov, Monte Carlo Techniques) |
Week 7: Feb 28 |
Fundamental Problems in Robotics, Reactive Control, Motion Planning |
Week 8: March 7 |
Robot Localization |
Week 9: March 14 NO CLASS, Spring Break |
|
Week 10: March 21 |
Computer Vision |
Additional Readings: |
Color and Light
Wikipedia Color Vision
Wikipedia "The dress" Gold or Blue (human color perception example)
Color Spaces
HSV Color space
RGB, CMKY, HSV Color model description (youtube)
Wikipedia HSV color space
Color Blob and Feature Detection
HSV Color Detection in Matlab
Blob detection and connected components algorithm
Wikipedia Sobel Detector (edge detection)
Edge Detectors (Sobel and Laplacian)
Feature detection with OpenCV (good background as well as some openCV code)
Penn State lecture on correspondence matching
Wikipedia Harris Corner Detector
Wikpedia Kanade-Lucas-Tomasi feature tracker
CMU KLT lecture notes
|
Week 11: March 28 |
Visual Servo |
Additional Readings: |
Visual Servo
Hutchinson, Seth, Gregory D. Hager, and Peter I. Corke. A tutorial on visual servo control. IEEE transactions on robotics and automation 12, no. 5 (1996): 651-670.
Wikipedia visual servoing
Visual Servo Lecture Notes (Kemp, Georgia Tech)
|
Week 12: Apr 4 |
Feature and Landmark Detection |
Additional Readings: |
Camera Models and Calibration
Wikipedia Pinhole Camera
Rolling and global shutter
Toilet paper roll pinhole camera
Stanford Course Notes on Camera Models, Lenses, Calibration
MathWorks Camera Calibration Description
Matlab Camera Calibration Tool Reference
Matlab Camera Calibration (youtube)
Stereo Vision
Stereo vision tutorial
UNR Stereo vision tutorial
Penn State lecture notes on stereo vision
Wikipedia Triangulation (main technique for traditional stereo vision)
Stereo vision calibration in Matlab
Stereo Vision in ROS
Wikipedia Structure from motion
Stanford lecture on active stereo and more
Active Stereo
Feature and Landmark Detection
Wikipedia feature detection
Wikipedia Hough Transform
Hough transform basics
More on Hough transform
U. Washington Lecture ones on feature detection
Article on SIFT features
More details on SIFT features (above article was based on this one)
Matlab computer vision toolkit for feature detection
April tags
D. Scharstein and A. Briggs. Real-time recognition of self-similar landmarks. Image and Vision Computing, 19(11):763-772, September 2001.
|
Week 13: Apr 11 |
Final Project Topics |
Week 14: Apr 18 |
Final Project Topics |
Week 15: Apr 25 |
Final Project Topics |
Week 16: May 2 |
Final Project Topics |