How can we use computers to recognize objects, people, actions, animals, places, etc from images? This seemingly trivial task that people perform without much effort has remained one of the core problems in Computer Vision. In this class we will study, play with, and implement algorithms for computational visual recognition using machine learning and deep learning. The class sessions will be a mix of classic lectures for the most foundational topics, and student-led paper review sessions for more advanced topics. We will also be reading recent research papers on computational visual recognition ranging from classifying images, to detecting and outlining every object in an image. In summary, after successful completion of this course you should be able to teach a robot how to distinguish dogs from cats.
Prerrequisites: This course requires no previous background in computer vision or machine learning but knowledge in either of those will be helpful. You need to know about matrices, calculating derivatives, and probabilities (bayes rule). You will also need to be a proficient programmer in a high-level language. There will be two homework assignments designed to be accessible to almost anybody fitting the previous criteria. These assignments will show you the basics of a modern general visual recognition system, and will give you the tools for implementing more advanced ones. There will also be a series of quizzes directly related to the assignments and material covered during class. Finally, we will have a class project where you will be able to work on something beyond your assignments and where you will have more freedom to pursue a focused problem that is of your interest and better matches your background. Finally we will be using Lua/Torch in the lecture notes, so running the following tutorial before the first day of class is recommended [html] [notebook].
Grading: Labs: 25pts, Paper presentation: 15pts, Paper summaries: 10pts, Project: 50pts.Date | Topic | ||
Wed, August 24th | Lecture: Introduction [slides]
|
Instructions for running iTorch from Docker: [here] (Useful if you have trouble installing dependencies or if you are a Windows user) | |
Mon, August 29th | Lecture: Image Processing Basics & Introduction to Computer Vision [slides]
| Reading: Szeliski Book, Chapter 3. | |
Wed, August 31st | Lecture: Machine Learning for Vision I [slides]
| For more into Train, Validation, Test splitting watch Andrew Ng's lecture on model selection [video] | |
Mon, September 5th | No classes this day -- Work on Linear Softmax + SGD Lab. | ||
Wed, September 7th | Lecture: Machine Learning for Vision II [slides]
| Useful related material: [Linear regression (Nando de Freitas/Oxford)] [Softmax vs Max-margin Loss (Andrej Karpathy/Stanford)] | |
Mon, September 12th | Lecture: Deep Learning for Vision I [slides]
| ||
Wed, September 14th | Paper Review: Image Classification | Presenters: Vicente [slides] Vicente [slides] |
|
Submit a 1 or 2 page project proposal in PDF (Deadline September 19th). | |||
Mon, September 19th | Student Paper Review: Scene Recognition and Sense of Place. | Presenters: Leandra, Ian [slides] Luke, Vijay [slides] |
|
Wed, September 21st | Student Paper Review: Object Detection.
| Presenters: Tom, Nick [slides] Xuwang [slides] |
|
Meet with the instructor this week to define your Final Project Proposal (Monday 26th office hours will be from 2pm to 5pm for this purpose) | |||
Mon, September 26th | Student Paper Review: Image Parsing and Semantic Segmentation | Presenters: Haina [slides] Hao |
|
Wed, September 28th | Lecture: Adversarial Examples and Generative Adversarial Networks. | Vicente [slides] Torch code to train a generative adversarial network [link], and to obtain Deep Dream images [link] |
|
Mon, October 3rd | Fall Break - no classes this day. | ||
Wed, October 5th | Student Paper Review: Recurrent Neural Networks | Presenters: Tianlu, Yin [slides] Weiqiang [slides] |
|
Mon, October 10th | Student Paper Review: Action Recognition from Videos.
| Presenters: Luyao, Jonathan [slides] Yujia, Sharon [slides] |
|
Wed, October 12th | Student Paper Review: Topic of your choice | Presenters: Siva [slides] Divya [slides] |
|
Mon, October 17th | Student Paper Review: Topic of your choice | Presenters: Abhimanyu [slides] Gautam [slides] |
|
Wed, October 19th | Guest Lecture
| ||
Mon, October 24th | Student Paper Review: Unsupervised learning of Deep Neural Networks. | Presenters: Colin, Joel [slides] Di Fang, Jeff [slides] |
|
Wed, October 26th | Student Paper Review: Multi-label Image Classification.
| Presenters: Jiayu, Xiao [slides] Jiankun, Xueying [slides] |
|
Submit a 3-page progress report of your Project (Monday October 31st) | |||
Mon, October 31st | Lecture: Revisiting the Principles of Categorization [slides].
| [Rosch,Lloyd,1978] [Jolicoeur,Gluck,Kosslyn,1984] |
|
Wed, November 2nd | Student Paper Review: Pushing the Limits in Visual Recognition | Presenters: Jingyun [slides] Tianyi, Haoran [slides] |
|
Mon, November 7th | Student Paper Review: Vision and Location.
|
Presenters: Yutong, Zheyuan [slides] Minghua [slides] |
|
Wed, November 9th | Brainstorming Session for New Tasks & Methods in Visual Recognition | ||
Mon, November 14th | No classes this day -- Please use this time to work on your projects. | ||
Wed, November 16th | Student Paper Review: Efficient Deep Models.
|
Presenters: Naveen [slides] Sihang [slides] |
|
Mon, November 21st | Last Lecture: Course Recap [slides].
| ||
Wed, November 23rd | Day Before Thanksgiving - no classes this day. | ||
Mon, November 28th | Project Presentations | ||
Wed, November 30th | Project Presentations | ||
Mon, December 5th | Project Presentations | ||
Project Deadline including Report or Technical Blog post (Mon, December 5th) |