Spring 2019 | Selective Core Course for Final Year Undergrad
Time and Place
- Lecture: Every Wednesday 10:20-12:10, Location @ #208, 2 Lychee Park
- Lab: Every Friday 10:20-12:10, Location @ #607, 7 Innovation Park
Instructor Team
- Lead Instructor: Prof. Song Chaoyang, Dr. Wan Fang (Invited)
- Teaching Assistant: Mr. Liu Xiaobo
- Office: Room 606, 7 Innovation Park
Course Objectives
- This course is intended for students advancing in the study of robotic engineering. The focus is on the problems of how a robot can learn to perceive the physical world well enough to act in it and make reliable plans. Subjects covered by this course include robotic collaboration, kinematics, Robot Operating System (ROS), robotic vision, calibration, RGB-D sensing, object recognition, artificial intelligence (AI) and deep learning (DL). Specific projects will be carried out throughout this course regarding the simulation of robot picking using fundamental kinematics and robot vision, an AI robot to play tic-tac-toe game, and a DL robot to play arcade claw game.
- To teach students how to conduct the basic kinematic formulation of a robotic system in simulation.
- To teach students how to use robotic vision, including algorithm, hardware, and software, in simulation.
- To teach students how to program artificial intelligence into a robot hardware performing interactive tasks.
- To teach students how to use deep learning methods to program a robot hardware to perform advanced tasks.
- To reinforce students’ team skills through various team project, including problem formulation, problem solutions and written reporting of results.
- To reinforce students’ visualization and hands-on skills through project virtual prototyping and/or physical construction exercises.
Learning Outcomes
- As an elective course for robotic engineering major, this course lays the foundation for students to use widely adopted Robot Operating System (ROS) to perform advanced robot control including basic mathematical formulation, hardware usage, and intelligence integration. The following learning outcomes are expected for students taking this course:
- Given functional and environmental requirement, utilize concepts generation methods within a team setting to achieve a consensus for a robot design concept.
- Design and develop functional robot programs from the perspectives of function, hardware, algorithm, and physical environment.
- Apply basics of disciplines including mechanical engineering, electrical engineering, applied mathematics, computer science to understand the use of robots in action.
- Communicate engineering decisions, justification for those decisions, designs, programming, and test results in multi-media presentation and report writing.
Pre-requisites
- MEE5101 Introduction to Robotics and Automation
- MEE5104 Robotic Modeling and Control
Grading
- 60%: project marking
- 15% per project for the first four projects, including 10% code submission and 5% video presentation.
- 10%: individual marking
- 30%: final project marking, including
- 10% final paper
- 10% final video demo
- 5% final poster
- 5% code submission
Robotic Engineering Project
- Project 1: Simulate picking using kinematics [Github]
- Simulate a robot arm picking objects from a shelf and placing them in a bin.
- Similar to Amazon Picking Challenge, but much simplified in ROS;
- Without using any image data, focus mainly on inverse kinematics for planning;
- Focus on the understanding of analytical method for common robotic grasping;
- To be completed using the students’ own laptop.
- Simulate a robot arm picking objects from a shelf and placing them in a bin.
- Project 2: Simulate picking using robotic vision [Github]
- Simulate a robot arm processing RGB-D images to grasp objects from a bin.
- Similar to industrial bin-picking scenario, but much simplified in ROS;
- Process simulated images in ROS to pick up simple object autonomously;
- Focus on the use of robotic vision techniques for common robotic grasping;
- To be completed using the students’ own laptop.
- Simulate a robot arm processing RGB-D images to grasp objects from a bin.
- Project 3: Program an AI robot to play Tic-Tac-Toe game [Github]
- Use AI to control a real robot to play tic-tac-toe with a human player.
- Similar to the Alpha-Go scenario, but much simplified using a real desktop robot;
- Design and implement an AI algorithm on a robot for tic-tac-toe;
- Focus on the design of AI algorithms to control a real robot arm autonomously;
- To be completed using a real desktop robot hardware (in progress).
- Use AI to control a real robot to play tic-tac-toe with a human player.
- Project 4: Program a DL robot to play arcade claw game [Github]
- Use Deep Learning to control a real robot to play arcade claw machine game.
- Similar to the DeepClaw scenario, but much simplified using a real desktop robot;
- Design and implement a deep neural network to play the game for data acquisition;
- Focus on the use of DL techniques on a real robot for data-driven robotics;
- To be completed using a real desktop robot hardware (in progress).
- Use Deep Learning to control a real robot to play arcade claw machine game.
- Project 5: Autonomous Robot Manipulation Competition [Github]
- Design a competitive and autonomous picking robot
- Design a robot system to autonomously play a board game competitively;
- In the final class, each team will present your robot system design in details by
- Writing a full paper describing your robot system in technical details;
- Submit a video demo of your robot system design;
- Submit a posted to present your robot system design;
- Open-source your codes by uploading on GitHub;
- Compete live and win! (Good luck)
- To be completed using a real desktop robot hardware.
- Design a competitive and autonomous picking robot
- Due to the limitation of the robotic hardware, a total of 9 students will be limited to this course.
Lecture & Lab Content
How to Video Record Your Presentation using PowerPoint
- There is a simple solution in Microsoft PowerPoint, as a recording function is built-in the software.
- The content of the video should revolve around your presentation, but you are welcome to be creative about how to present it.
- This is a Team assignment, and the question about who to present makes no sense at all.
- You need to prepare and submit a PDF version of the slides at this [link].
Late Homework Policy
- Each student is granted four unpenalized late days for the semester. Homework can be handed in no more than four days late and will receive a 25% penalty for each day late (excluding unpenalized late days used). Homework are due at 3 PM on the due date, and each late day extends the deadline by exactly 24 hours. All homework must be done to pass the course.
Course Materials
- Required:
- Introduction to Robotics – Mechanics and Control (4th Edition) by John Craig. [PDF]
- Optional:
- Robotics, Vision & Control – Fundamental Algorithms in MATLAB® by Peter Corke.
- Artificial Intelligence – A Modern Approach (3rd Edition) by Stuart Russel, and Peter Norvig.
- Mastering ROS for Robotics Programming by Lentin Joseph. [Website]
- A Mathematical Introduction to Robotic Manipulation by Richard M. Murray, Zexiang Li, and S. Shankar Sastry. [PDF]
Academic Integrity
- This course follows the SUSTech Code of Academic Integrity. Each student in this course is expected to abide by the SUSTech Code of Academic Integrity. Any work submitted by a student in this course for academic credit will be the student’s own work. Violations of the rules (e.g., cheating, copying, non-approved collaborations) will not be tolerated.
Course Schedule & Lecture Notes
Register your Office 365 account using SUSTech Email first, otherwise you won’t be able to access the files.
| Week | Date | Day | Class | Topic | Subject |
|---|---|---|---|---|---|
| 01 | February 20 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Course Introduction | Course Introduction |
| 01 | February 22 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Programming Basics | Linux and ROS |
| 02 | February 27 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part I Robot Foundations | Mathematical Foundations |
| 02 | March 01 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project I Simulate Picking Kinematics | ROS Basics |
| 03 | March 06 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part I Robot Foundations | Kinematics & Jacobian |
| 03 | March 08 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project I Simulate Picking Kinematics | ROS Simulation |
| 04 | March 13 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part I Robot Foundations | Dynamics & Control |
| 04 | March 15 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project I Simulate Picking Kinematics | Seminar [Hu Chunxu] ROS Picking Project I Submission |
| 05 | March 20 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part II Robotic Vision | Seminar [Dr. Qiu Qiang] |
| 05 | March 22 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project II Simulate Picking with Vision | Camera & Images RealSense with ROS |
| 06 | March 27 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part II Robotic Vision | Calibration & Servoing |
| 06 | March 29 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project II Simulate Picking with Vision | Seminar [Dr. Gao Lei] |
| 07 | April 03 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part II Robotic Vision | Features & Filters |
10:20-12:10 | #607, 7 Innovation Park | Cancelled due to holiday Project II Submission |
|||
| 08 | April 10 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part III Artificial Intelligence | Rational Agents |
| 08 | April 12 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project III Program a Tic-Tac-Toe Robot | Seminar [Liu Sicong] |
| 09 | April 17 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part III Artificial Intelligence | Search Problems |
| 09 | April 19 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project III Program a Tic-Tac-Toe Robot | Seminar [Zhang Min] |
| 10 | April 24 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part III Artificial Intelligence | Adversarial Search |
| 10 | April 26 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project III Program a Tic-Tac-Toe Robot | Seminar [Dr. Liang Conghui] Project III Submission |
10:20-12:10 | #208, 2 Lychee Park | Cancelled due to holiday | |||
| 11 | May 05 | Sunday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project IV Program an Arcade Claw Robot | Neural Networks Tensorflow Basics |
| 12 | May 08 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part IV Deep Learning | Network Tuning |
| 12 | May 10 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project IV Program an Arcade Claw Robot | Learning Picking |
| 13 | May 15 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part IV Deep Learning | Seminar [Liu Guoqing] |
| 13 | May 17 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project IV Program an Arcade Claw Robot | Robot Learning Project IV Submission |
| 14 | May 22 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part V Special Topics | Shareability & Reproducibility |
| 14 | May 24 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project V Collaborative Robot Learning | [Seminar] Li Miao |
| 15 | May 29 | Wednesday 10:20-12:10 | Lecture @ #208, 2 Lychee Park | Part V Special Topics | Industrial vs. Collaborative |
| 15 | May 31 | Friday 10:20-12:10 | Lab @ #607, 7 Innovation Park | Project V Collaborative Robot Learning | Seminar [Albert Causo] Final Presentation Project V Submission |