• Instructor: Anshumali Shrivastava (anshumali AT rice.edu)
• Class Timings: Monday 3pm-4:30pm
• Location: AEL A121
• Office Hours: Monday 4:30pm - 5:30pm, Duncan Hall 3118

Structure

This research seminar is intended to discuss recent advances and trends in machine learning. We will be presenting and discussing 1-2 recent technical papers each week. The focus will be on recent papers from Top Venues such as NIPS, ICML, KDD, etc. The aim is to understand the the key ideas and concepts with an aim of generalizing them and stimulating research. Whenever necessary, some concepts will be introduced for clarification and to make connections.

Grading and Logistics

Class participation and one presentation for 1 credit. In addition students can undergo a semester long project for 3 credits.

Prerequisite

Familiarity with basics in linear algebra, probability, and machine learning is required.

Paper Assignment

Paper Assignments

Schedule

• 08/22 : Logistics. Intro to Streaming Algorithm, Median Trick, etc.
  
• 08/29 : Counting on Streams: Bloom Filters, Count Sketches and Adaptive Sketches (Class Slides)
  
• Network Applications of Bloom Filters pdf
• An Improved Data Stream Summary: The Count-Min Sketch and its Applications pdf
• Time Adaptive Sketches (Ada-Sketches) for Summarizing Data Streams pdf
• 09/05 : NO CLASS LABOR DAY
  
• 09/12 : Randomize SVDs and Matrix Sketching
  
• Simple and Deterministic Matrix Sketching pdf
• Randomized block krylov methods for stronger and faster approximate singular value decomposition. pdf
• 09/19 : Random Kernel Features, Nystrom Approximations, etc
  
• Random Features for Large-Scale Kernel Machines. pdf
• Nystrom Method vs Random Fourier Features: A Theoretical and Empirical Comparison. pdf
• 09/26 : Variants of Gradient Descent, Frank Wolfe and Nestrov Accelerated Descent
  
• Chapter 3 from Convex Optimization: Algorithms and Complexity pdf
• Adaptive Subgradient Methods for Online Learning and Stochastic Optimization pdf
  
• 10/03 : Efficient Partition Function Estimation
  
• Noise-contrastive estimation: A new estimation principle for unnormalized statistical models pdf
• Approximate Weighted Model Counting
• 10/10 : NO CLASS MIDTERM RECESS
  
• 10/17 : Project Proposals
  
• 10/24 : Advanced MCMC
  
• A Split-Merge Markov chain Monte Carlo Procedure for the Dirichlet Process Mixture Model pdf
• Hamiltonian MCMC
• 10/31 : Variational Inference
  
• Variational Inference: A Review for Statisticians pdf
  
• Jason Eisner's Tutorial link (Additional Material)
• 11/07 : Advanced Topics 1: Submodular Function Maximization
  
• Andreas Krause's Tutorial pdf
• 11/14 : Advanced Topics 2: Differential Privacy pdf (Only first 3 chapters)
  
• 11/21 : Application : Question Answering
  
• Towards ai-complete question answering: A set of prerequisite toy tasks pdf
• Learning to Compose Neural Networks for Question Answering pdf
• 11/28 : Final Project Presentations
  

Students with Disability

If you have a documented disability that may affect academic performance, you should: 1) make sure this documentation is on file with Disability Support Services (Allen Center, Room 111 / adarice@rice.edu / x5841) to determine the accommodations you need; and 2) meet with me to discuss your accommodation needs.