CS 6501-004: Deep Learning for Visual Recognition

Instructor: Vicente Ordonez (vicente at virginia.edu)
Teaching Assistant: Ziyan Yang (zy3cx at virginia.edu) -- Hours: 11am to 12:30pm Tues and Thurs at Rice 436
Teaching Assistant: Tianlu Wang (tw8cb at virginia.edu) -- Hours: 10am to noon on Weds at Rice 436
Class Time: Mondays & Wednesdays between 3:30PM and 4:45PM, at Thornton Hall E303.
Discussion Forum: piazza.com/virginia/spring2019/cs6501004/home

Course Description: How can we use computers to recognize objects, people, actions, animals, places, etc from images? This seemingly trivial task that people perform without much effort has remained one of the core problems in Computer Vision. Recent advances in representation learning using multiple layers of abstraction (deep learning) have demonstrated to be an important aspect for designing artificial systems for visual recognition. In this class we will study, conceive, and implement deep learning models and learning algorithms for computational visual recognition. After this class you will be able to understand, design, implement, and assess the impact of deep learning techniques for a diverse range of visual recognition tasks.

Learning Objectives: (a) Develop intuitions between aspects in human vision and computer vision, (b) Understanding foundational concepts for representation learning using neural networks, (c) Become familiar with state-of-the-art models for tasks such as image classification, object detection, image segmentation, scene recognition, etc, and (d) Obtain practical experience in the implementation of visual recognition models using deep learning.

Syllabus

Date     Topic 
Mon, January 14th Introduction: Why is Visual Recognition Hard? [pptx] [pdf] Suggested Reading:
Szeliski Book, Chapter 3: Image Processing.
Wed, January 16th Image Processing and Image Manipulation: Convolutions [pptx] [pdf] Suggested Reading:
[What the Frog's Eye Tells the Frog's Brain]
Assignment on Image Processing and Manipulation [Colab]. Due January 28th 5pm EST.
Mon, January 21st MLK Holiday -- no class this day  
Wed, January 23rd Image Features: Gradients, HoG, SIFT, GIST, Bag-of-Features [pptx] [pdf] Suggested Reading:
[Lowe's SIFT Features. ICCV 1999]
[Csurka's Bag of Keypoints. ECCV 2004]
Assignment on Image Classification [Colab]. Due February 12th 5pm EST.
Mon, January 28th Machine Learning for Vision I: Introduction to Machine Learning [pptx] [pdf]  
Wed, January 30th Machine Learning for Vision II: Softmax Classifier [pptx] [pdf]  
Mon, February 4th Machine Learning for Vision III: Stochastic Gradient Descent, Regularization [pptx] [pdf]  
Wed, February 6th Introduction to Neural Networks (Deep Learning) [pptx] [pdf] Suggested Reading:
[Andrew Ng's Lecture Notes]
Assignment on Deep Learning Basics [Colab]. Due February 19th 5pm EST.
Mon, February 11th Neural Networks: Backpropagation -- Continuation [pptx] [pdf]  
Wed, February 13th Convolutional Neural Networks (CNNs) [pptx] [pdf] Suggested Reading:
[LeCun's Documnt. Recognition. Proc. of IEEE 1998]
Assignment on Convolutional Neural Networks [Colab]. Due March 5th 5pm EST.
Mon, February 18th CNN Architectures I: LeNet, AlexNet, VGG [pptx] [pdf] [VGG-Net slides] Required Reading:
[Krizhevsky, Sustkever, Hinton, NeurIPS 2012]
Wed, February 20th No Class this Day: Weather Warning Suggested Reading:
[Cognitive Psychology Paper on CNNs]
Mon, February 25th CNN Architectures II: GoogLeNet, ResNet, DenseNet and Intro to Object Detection [pptx] [pdf] Required Reading:
[Girshick et al CVPR 2014]
Wed, February 27th Object Detection: RCNN, Fast-RCNN, Faster-RCNN, YOLO, SSD: [pptx] [pdf].
Guest Lecture by Tianlu Wang 		Required Reading:
[Redmon et al CVPR 2016]
Mon, March 4th Object Detection II: Guest Lecture by Mahyar Najibi (PhD Student at UMD College Park). SNIPER: Efficient Multi-Scale Training. NeurIPS 2018. [pdf]. Required Reading:
[Najibi et al NeurIPS 2018]
Wed, March 6th Image Segmentation: Fully-Convolutional Networks. [pptx] [pdf] Required Reading:
[Shelhamer, Long & Darrell CVPR 2015]
Mon, March 11th Spring recess -- no class this day  
Wed, March 13th Spring recess -- no class this day  
Mon, March 18th Image Synthesis: Generative Adversarial Networks (GANs) -
Adversarial Examples, Style Transfer [pptx] [pdf]
  
Assignment on Advanced Topics: Adversarial Examples, Feature Inversion, RNNs [Colab]. Due April 4th 5pm EST.
Wed, March 20th Guest Lecture by Jacalyn (Jackie) Huband (ARCS) - Rivanna and Pytorch [pptx] [pdf] Required Reading:
[Gatys et al CVPR 2016]
Mon, March 25th Variational Auto-Encoders (VAEs): Guest Lecture by Tom Fletcher (Prof. at UVA's ECE and CS departments, expert in medical image analysis and computer vision) [slides]
Wed, March 27th Recurrent Neural Networks (RNNs) I Required Reading:
[Vinyals et al CVPR 2015]
Mon, April 1st Recurrent Neural Networks (RNNs) II [pptx] [pdf]  
Wed, April 3rd Text-to-Image Synthesis: Guest Lecture by Fuwen Tan (PhD Student at UVA). :
Text2Scene: Generating Compositional Scenes from Textual Descriptions. https://arxiv.org/abs/1809.01110 [pptx] [pdf]. 		 
Mon, April 8th
Speaker: Prof. David A. Forsyth (University of Illinois at Urbana-Champaign)
Talk: The Materials Objects are Fashioned From
Location: Rotunda Dome: 2:15pm to 3:30pm.
More information: Prof. Forsyth, obtained his PhD at the University of Oxford, and has been professor at the University of Illinois at Urbana Champaign and the University of California Berkeley. His talk will be about material recognition for fashion items, with applications to image segmentation and colorization. See more details in attached poster here. Invite more people!
 
Wed, April 10th Pedestrian Detection and Human Pose Estimation [pptx] [pdf]  
Mon, April 15th Fairness in AI Group Discussion: Group-1: Gender Shades [pdf], Group-2: Men Also Like Shopping [arxiv], Group-3: Women Also Snowboard [arxiv]. [pptx] [pdf]  
Wed, April 17th Fairness in Vision: Guest Lecture by Tianlu Wang (PhD Student at UVA). :
Adversarial Removal of Gender from Deep Image Representations. https://arxiv.org/abs/1811.08489. [pptx] [pdf] 		 
Mon, April 22nd Video Recognition + Image Grounded Question Answering [pptx] [pdf]  
Wed, April 24th Course Recap and What's Next? [pptx] [pdf]  
Mon, April 29th Final Project Poster Presentation  

Disclaimer: The professor reserves to right to make changes to the syllabus, including assignment due dates. These changes will be announced as early as possible.

Prerrequisites: This course requires no previous background in computer vision or machine learning but knowledge in either of those will be helpful. You need to know about matrices, calculating derivatives, and probabilities (bayes rule). You will also need to be at least a moderately proficient programmer in python. There will be several lab assignments. These assignments will show you the basics of modern general visual recognition algorithms and models, and will give you the tools for implementing more advanced ones. Finally, we will have a class project where you will be able to work on something beyond your assignments and where you will have more freedom to pursue a focused problem that is of your interest and better matches your background. Finally we will be using python/pytorch in the lecture notes, so being proficient in Python by completing a few projects in this language before the class starts is helpful. You should install python, jupyter, and pytorch, and complete the following notebook before our first day of class [pytorch_tensors].

Grading: Assignments: 50% (5 programming assignments: 10% + 10% + 10% + 10% + 10%), Class Project: 40%, Participation / Required Reading Summaries: 10%.

Late Submission Policy: You will lose 10% for each day late for the programming projects. This means the first 24 hours after the due date count as 1 day, up to 48 hours is two and 72 for the third late day. However, you will have three late days for the whole course to address any unexpected traveling, other commitments, interviews, hackathons, etc. These three late day grace period will be accounted at the end of the semester. Please do not send the professor or the TAs emails about submitting late assignments, you do not need to explain the reason for submitting a late assignment, please just submit the late assignment. You can always submit late assignments, and you have a grace period of 3 days for the semester. If you don't use more than three late days, you will not be penalized in any way (but this will be accounted at the end of the semester). Note: the 3-day grace period doesn't apply to the course project, and it doesn't count fractions of a day, only full days (if it shows late on Collab, it is late, and any fraction of a day counts as a full day). Note: Assignments on Collab will not accept submissions indefinitely, I can only guarantee that they will accept submissions during the next three days following the deadline. In summary, if this late policy is confusing, please just don't submit late assignments, this is a large class and any emails regarding this matter are not productive use of time for the staff.

Academic Integrity Statement: "The School of Engineering and Applied Science relies upon and cherishes its community of trust. We firmly endorse, uphold, and embrace the University’s Honor principle that students will not lie, cheat, or steal, nor shall they tolerate those who do. We recognize that even one honor infraction can destroy an exemplary reputation that has taken years to build. Acting in a manner consistent with the principles of honor will benefit every member of the community both while enrolled in the Engineering School and in the future. Students are expected to be familiar with the university honor code, including the section on academic fraud."

Accessibility Statement: "The University of Virginia strives to provide accessibility to all students. If you require an accommodation to fully access this course, please contact the Student Disability Access Center (SDAC) at (434) 243-5180 or sdac@virginia.edu. If you are unsure if you require an accommodation, or to learn more about their services, you may contact the SDAC at the number above or by visiting their website at https://www.studenthealth.virginia.edu/student-disability-access-center/about-sdac."

Other similar courses or courses with useful related material:

Department of Computer Science, University of Virginia, Spring 2019.