CSCE 439/839: Robotics: Algorithms and Applications
Spring 2020 (COVID-19 Edition)
Instructor
Dr. Carrick Detweiler
Computer Science and Engineering
220 Schorr Center
carrick _at_ cse.unl.edu
Office Hours:
carrick.youcanbook.me and by appointment.
Teaching Assistant
Ajay Shankar
ashankar _at_ cse.unl.edu
Office Hours:
by appointment.
Course Information
Lecture: MWF 3:30pm - 4:20pm in Avery 119
Lab: M 4:30pm - 5:20pm in Avery 119
Online on Canvas
For detailed information on this course please see the new COVID-19 course syllabus (original here).
COVID-19 and Moving to Online Format
Given COVID-19 and the move to an online course format, things will be different for all of us. Please bear with me as we adjust. I will do my best to communicate any changes and my expectations. Please do the same in return and ask if you have questions or concerns. I will primarily use Canvas for communication, but make sure to check your university email regularly as well.
The class will be run asynchronously with videos and reading materials posted in canvas.
We will hold whole-class office hours during some of the regularly scheduled class times. This will allow us to review some of the topics in more depth based on student questions. Dates for whole-class office hours will be posted on Canvas. In addition, you can schedule time to meet with me individually at carrick.youcanbook.me.
The major change to the course content and assessment is that there will be no more lab group assignments and no final project. Instead, there will be an additional homework (the plan is for two more this semester) and there will be some short online activities and quizzes to help assess participation.
If you have difficulty accessing the online materials due to internet problems, sickness, family care, or anything else, please let me know and we will figure out a solution.
Finally, if I do not respond to your messages within 24 hours, please reach out to Ajay in Canvas (or feel free to copy him on any messages).
Course Description
Robots play an increasingly important role in our lives, from
assembling our cars and cell phones to vacuuming our rugs and flying
recon missions. To create systems that work in the real world, the
field of robotics requires robust theory and algorithms that are
tightly integrated with hardware that is designed with engineering
expertise.
This will be a hands-on, lab-based course where you will
implement the algorithms you learn about in class on a small
balancing robot that you will build in lab. You will learn the
fundamentals of robotics, as well as the cutting-edge in robotics
research. Topics covered will include: control, navigation and path
planing, obstacle avoidance, manipulation and grasping, and robotic
vision processing. By the end of the course you will know why robots
are not yet folding our clothes and driving our cars, but you will also
learn what is needed to make these possible in the future.
You will interface with the embedded system that controls
the robot by using learning to program in ROS (Robot Operating
System). Programming will be in either C++ or Python, with
an emphasis on C++. By the end of the course you will have a deep
understanding of the design, programming, and interfacing of robot
systems. This will prepare you for cutting edge careers in industry
and research.
Assignments
See the course schedule below for reading assignments and the general
topics covered in the course. Unless otherwise noted, assignments are
due via canvas before the start of class on the due date.
Assignment
| Start Date*
| Due Date
|
Course Survey
| Monday, Jan 13th
| Tuesday, Jan 14th midnight
|
HW 1 (pdf, read_serial.zip)
ROS and Arduino Introduction
| Monday, Jan 13th
| Checkoff 1: Tuesday, Jan 21st
Due: Friday, Jan 31st
|
In-Class Lab 1 (pdf)
| Monday, Jan 27
| Wednesday, Feb 5
|
Lab 1 (pdf, ROS Code, Balboa Code)
| Monday, Feb 3
| Checkoff 1: Mon, Feb 10
Checkoff 2: Mon, Feb 17
Due: Mon, Feb 24
|
Lab 2 (pdf)
| Monday, Feb 24
| Checkoff 1: Weds, March 4
Checkoff 2: Mon, March 9
Due: Mon, March 16
|
HW 2 (pdf, hw2Code.tgz)
ROS, PID, Kalman Filters
| Mon, March 30
| Due: 3:30 CT, Weds, April 15
|
HW 3 (pdf, ball_detector.tgz, landmark_self_sim.tgz
, bag-files)
ROS, PID, Kalman Filters
| Thurs, April 16
| Due: Midnight, Tues, May 5
|
*Start dates are estimated for future assignments and are subject to change. Additional assignments will be added.
**Note html versions of assignments may lack some formatting or pictures found in the pdf version.
Course Books
Suggested: G. Dudek and M. Jenkin, Computational Principles of Mobile Robotics, Cambridge University Press, 2nd Edition, 2010.
Course Schedule
Below is an approximate schedule of course topics. These are subject to
change, assignments will be announced in class.
Week |
Topics |
Reading |
Week 1: Jan 13 |
Introduction, ROS |
ros.org tutorials |
Week 2: Jan 20 |
ROS, Balboa Robot, Sensors |
CPMR 4.1-4.8 |
Additional Readings: |
|
Additional Readings: |
Serial protocol from class example
How not to write code
Balboa Robot Resources:
Balboa 32U4 Balancing Robot User’s and Assmebly Guide
Balboa Arduino library
Balboa buying on Pololu
Other Balboa resources (pololu)
Balboa gear ratio chart
Balboa pinout
Balboa schematic
5-Channel reflectance sensor array
C Programming Resources:
Learn C The Hard Way, Zed Shaw (textbook on Amazon)
A wiki-style C "text book" alternative
Another online C text book
C Frequently Asked Questions from the days of Usenet
Python Programming Resources (I have not gone through all these courses, some may be better than others):
Free Python Course
Python for Beginners
Python Tutorial
More python resources
Information on Sensors:
IMUs (wikipedia)
Math behind IMU data fusion
Overview of Micro-Electro-Mechanical (MEMs) systems
Accelerometer (wikipedia)
How an accelerometer works (sparkfun)
Short bit on how MEMs accelerometers work
More on MEMs accelerometers
More on accelerometers
Gyroscope (wikipedia)
Gyroscope (sparkfun)
MEMs Gyroscopes
Magnetometer (wikipedia)
Magnetometers
Types of magnetometers
Converting 3-axis magnetometer reading to compass
Arduino-related things that might be useful for HW1:
Arduino Introduction
How to read a schematic
Arduino Uno R3 schematic
How to read a datasheet
Atmega328 datasheet
Arduino-function digital I/O
Arduino using buttons
Another button tutorial
Parallel versus Serial, UART (sparkfun)
Serial cummunication (sparkfun)
Serial communication (arduino)
Arduino UART serial communication
|
Week 3: Jan 27
|
ROS, Sensors, PID Control |
CPMR 3.1.3 |
Week 4: Feb 3 |
PID Control, Configuration Space |
CPMR 6.1-6.2 |
Additional Readings: |
Example Matlab code shown in class
PID Control
Wikipedia PID control
PID Control with Matlab example (similar idea to what we covered in class)
Another PID description (with some tuning heuristics)
PID tuning heuristic (my preference is to start with PD ctrl first and not PI as discussed in this article)
Wikipedia Ziegler-Nichols tuning method
Longish video on Ziegler-Nichols method
Tuning balancing robot (not Balboa)
Good video discussing PID with autonomous cars
Configuration Space
Wikipedia Configuration Space (so-so)
2D Configuration space robot sim (cool if it works)
Video from the above website
Textbook chapter with formal description of C-space (LaVelle)
|
Week 5: Feb 10 |
Fundamental Problems in Robotics |
CPMR 2.1-2.6 |
Week 6: Feb 17 |
Reactive Control, Motion Planning |
CPMR 6.3 |
Week 7: Feb 24 |
Data Fusion (Kalman, Markov, Monte Carlo Techniques) |
CPMR 4.9 |
Week 8: March 2 |
Robot Localization |
CPMR 8.1-8.7 |
Additional Readings: |
Kalman example code from class
Bug Algorithms
CMU Bug algorithm lecture notes
Bug 2 with ROS example
I. Kamon, E. Rimon, and E. Rivlin, TangentBug: A
Range-Sensor-Based Navigation Algorithm, The International Journal
of Robotics Research, vol. 17, no. 9, pp. 934-953, 1998.
Kalman Filters
Wikipedia Kalman Filter
G. Welch and G. Bishop. An Introduction to the Kalman Filter
Kalman Filter math and pictorial explanation
Kalman Filter explanation and tutorial with Matlab
Wikipedia Extended Kalman Filter
A book in Python on all sorts of filters (long)
Particle Filters
Wikipedia Particle Filter
ROS Particle Filter
Bayesian Filtering Library (PF, KF, EKF, etc)
Particle Filter lecture notes (MIT: Hsiao, Plinval-Salgues, Miller)
|
Week 9: March 9 |
Computer Vision |
CPMR 5.1-5.4 |
Week 10: March 16: NO CLASS, COVID-19: Virtual Class from now |
|
|
Week 11: March 23: NO CLASS, Spring Break |
|
|
Week 12: March 30 |
Computer Vision |
CPMR 5.1-5.4 |
Additional Readings: |
Color and Light
Wikipedia Color Vision
Wikipedia "The dress" Gold or Blue (human color perception example)
Camera Models and Calibration
Wikipedia Pinhole Camera
Rolling and global shutter
Toilet paper roll pinhole camera
Stanford Course Notes on Camera Models, Lenses, Calibration
MathWorks Camera Calibration Description
Matlab Camera Calibration Tool Reference
Matlab Camera Calibration (youtube)
Color Spaces
HSV Color space
RGB, CMKY, HSV Color model description (youtube)
Wikipedia HSV color space
|
Week 13: Apr 6 |
Feature Detection, Stereo Vision |
CPMR 5.5-5.8 |
Additional Readings: |
Color Blob and Feature Detection
HSV Color Detection in Matlab
Blob detection and connected components algorithm
Wikipedia Sobel Detector (edge detection)
Edge Detectors (Sobel and Laplacian)
Feature detection with OpenCV (good background as well as some openCV code)
Penn State lecture on correspondence matching
Wikipedia Harris Corner Detector
Wikpedia Kanade-Lucas-Tomasi feature tracker
CMU KLT lecture notes
Stereo Vision
Stereo vision tutorial
UNR Stereo vision tutorial
Penn State lecture notes on stereo vision
Wikipedia Triangulation (main technique for traditional stereo vision)
Stereo vision calibration in Matlab
Stereo Vision in ROS
Wikipedia Structure from motion
Stanford lecture on active stereo and more
Active Stereo
|
Week 14: Apr 13 |
Feature and Landmark Detection, Visual Servo |
|
Additional Readings: |
Feature and Landmark Detection
Wikipedia feature detection
Wikipedia Hough Transform
Hough transform basics
More on Hough transform
U. Washington Lecture ones on feature detection
Article on SIFT features
More details on SIFT features (above article was based on this one)
Matlab computer vision toolkit for feature detection
April tags
D. Scharstein and A. Briggs. Real-time recognition of self-similar landmarks. Image and Vision Computing, 19(11):763-772, September 2001.
Visual Servo
Hutchinson, Seth, Gregory D. Hager, and Peter I. Corke. A tutorial on visual servo control. IEEE transactions on robotics and automation 12, no. 5 (1996): 651-670.
Wikipedia visual servoing
Visual Servo Lecture Notes (Kemp, Georgia Tech)
|
Week 15: Apr 20 |
Higher Level Control and Planning |
|
Additional Readings: |
Higher Level Control and Planning
Wikipedia subsumption architecture
Rod Brooks "Bottom-Up Robotics" (youtube)
CMU lecture notes on robot motion planning, visibility graphs, Voronoi, etc.
Columbia lecture notes on visibility graphs
Wikipedia general Voronoi info
Voronoi path planning
|
Week 16: Apr 27 |
Additional Topics |
|