ECE 6504: Advanced Topics in Computer Vision, Spring 2013

Electrical and Computer Engineering Department, Virginia Tech

Meets: TR 5:00 pm to 6:15 pm in McBryde Hall (MCB) 216

Instructor: Devi Parikh
Email: parikh@vt.edu
Office: 440 Whittemore Hall
Office hours: by appointment (via email)

Announcements Course overview Pre-requisite Requirements Important dates Schedule Resources

Announcements:

*** Best Presentation Prize: Xiao Lin

*** Best Project Prize: Neelima Chavali

*** Best Discussion Participant Prize: Stanislaw Antol

Two paper reviews are due by 12:00 pm (noon) the day of the class.

The schedule of presenters is available. Please remember to meet the instructor 3 days before your presentation for a dry-run with a complete set of slides. Email the instructor to set up an appointment a couple of days ahead of time.

By Thursday January 24th: Email 6 preferences (dates and topics) for when you would like to present in class. See the schedule for the semester.

Welcome to 6504!

Course overview:

This is a graduate course in computer vision. The focus of this course is to survey and critique current and state-of-the-art approaches in computer vision. We will read and analyze the strengths and weaknesses of research papers on a variety of important topics pertaining to visual recognition and identify open research questions. See the schedule for a list of topics we will cover.

Pre-requisite:

An introduction to computer vision or equivalent course.  A machine learning or pattern recognition course may be beneficial.

Requirements:

Following are the requirements to successfully complete this course:

Discussion Paper reviews Presentations Project

• Discussion (15% of your grade): Students will be required to read the assigned papers before each class and actively participate in discussions in class.

• Paper reviews (25% of your grade): Students will be required to write a review of any 2 of the assigned papers in each class. Each review should be less than one page (a typical length would be about half to three-quarters of a page). Each review should summarize the paper in 2-3 sentences, and clearly identify the main contribution of the paper. The review should describe the strengths and weaknesses of the paper. Comments on how convincing the experiments were, if the material was well presented, and how the paper can be improved and extended should be included. A good review also comments on how the paper relates to other papers we have read or you know of. Finally, identify any interesting open research questions or applications that arise from reading the paper. The reviews should be emailed to the instructor by 12:00 pm (noon) the day of the class (i.e. on Tuesdays and Thursdays). You can use upto 3 late days over the course of the semester. Beyond that, your submission will not be accepted. You need not submit paper reviews during the classes where you are presenting the papers (see below). 

• Presentations (25% of your grade): Each student will be asked to present the topic associated with a class 2 or 3 times over the course of the semester (number will depend on class enrollment). Each presentation should be 45 minutes long. Students should practice their talks ahead of time to make sure they are 45 minutes long -- not shorter by more than a few minutes, and certainly not longer. The talks should be well organized and polished. Following will be required to prepare the presentation.

• Papers (65% of the presentation grade): The student should read the assigned papers and other background papers to gain a good perspective on the topic as a whole. For each paper, students should clearly state the problem statement, and motivate why the problem is interesting and important. The key technical ideas should be presented. The student should describe the experimental set-up and present the results obtained. The strengths and weaknesses of the paper should be discussed. The student should discuss how the different papers relate to each other (similarities and differences). Finally, interesting open research questions should be identified. The slides should be made as visual (with videos, images, animations) and clear as possible. The student should look at the links provided next to each paper, as well as the authors webpages for extra material such as slides, videos, extra results, etc. Please clearly cite the source of each slide that is not your own. Even if you use slides made by the author, it is your responsibility to make sure your presentation as a whole flows well.

• Experiments (35% of the presentation grade): The student should also conduct some (small-scale) experiments on at least one of the papers to analyze an interesting and meaningful aspect of the approach that the paper has not analyzed (e.g. different datasets, sensitivity to any parameters in the approach, etc.) to gain a more complete understanding of the paper. A distilled demo-version of the main idea of the approach presented in the paper should be implemented. The goal is not to regenerate the results already present in the paper. You may implement it yourself, or download code if available. Again, the experimental setup, any non-trivial implementation choices you made, results obtained and conclusions one can draw from them should be described in the presentation (within the 45 minutes). Please cite any existing code or data you use for your experiments.

• Timetable: Meet with the instructor three days before your presentation (i.e. Fridays or Mondays) with a complete set of slides for a dry run. Please email the instructor at least a couple of days in advance to set up an appointment to meet. The instructor will give you feedback to be incorporated before your presentation in class. The day of the class, please send the instructor your final set of slides (to be posted on the class webpage). Abiding to this timetable is very important to not lose points.

• Announcement: please email the instructor 6 topics preferences (along with the associated date) for your presentations by the end of the day Thursday, January 24th. See the schedule for the semester.

• Project (35% of your grade): Your project can be about extending a technique we studied in class, or empirically analyzing it. Comparisons between two approaches are also welcome. It is wonderful if you design and evaluate a novel approach to an important vision problem. Look at the schedule to get ideas on what topics might be of interest to you for your project. Perhaps the experiments part of your presentation above might give you project ideas. Be creative! If you need help with ideas for your project, or would like to work with a partner, please come talk to me (please take an appointment via email). The following are deliverables for your project.

• Proposal (10% of your project grade, due February 14th): A 1-page description that describes the following:
• Problem statement: Clearly state the goal of your project.
• Related work: Briefly describe existing related work (with citations) and what your project brings to the table that these other works do not. The most relevant papers may not necessarily be papers listed on the schedule, so be sure to also look beyond the list.
• Approach: Describe the technical approach you plan to employ
• Experiments and results: Describe the experimental setup you will follow, which datasets you will use, which existing code you will exploit, what you will implement yourself, and what you would define as a success for the project. If you plan on collecting your own data, describe what data collection protocol you will follow. Specify if you plan on experimentally analyzing different characteristics of your approach, or if you will compare to existing techniques. Provide a list of experiments you will perform. Describe what you expect the experiments to reveal, or what is uncertain about the potential outcomes. If you have any preliminary results, please summarize those as well.

• Mid-semester presentation (20% of your project grade, in class on March 19th and 21st): A 10-minute presentation that describes the same points as the proposal in as much detail as 10 minutes allow. Describe any challenges you are facing. Clearly state what has already been accomplished, and what remains for the end of the semester. 

• Final presentation (35% of your project grade, in class between April 25th and May 7th): A 20-minute presentation describing the same points as above. Any assumptions of your approach should be clearly stated. Any insights on future extensions of this project should be discussed.

• Final report (35% of your project grade, due May 15th): A 8-page self-contained document describing your project. It should follow the same format and quality of presentation as conference papers you have been reading in this class. Thoroughly describe related work and explain what your work brings to the table that existing work did not. Any student in the class should be able to clearly understand your report such that he/she can implement it. The reader should not have to read another paper to understand your approach. Justify any design choices or judgment calls you made in your approach. When describing the experiments, describe why you conducted each experiment, and what you can conclude from the results. Include clear figures and tables, as well as illustrative qualitative examples if appropriate. If your project has a new idea and promising results, this report could serve as a first draft for a future conference submission! Please use the CVPR latex template (http://www.pamitc.org/cvpr13/files/cvpr2013AuthorKit.zip) to write your report.

• The class will vote on best paper presentation, best project and best discussion-participant! 

• Feedback is very welcome. If you have any questions or concerns about the class or the requirements, please be sure to discuss them with the instructor early on. Please email the instructor to set up an appointment.

• No laptops, cell phone or other distractions in class please.

• January 24th: Email the instructor 6 topic preferences (along with the associated dates) you would like to present in class. Please see the schedule for the semester.
• February 14th: Project proposals due
• March 19th and 22nd: Mid-semester project presentations in class
• April 25th-May 7th: Final project presentations in class
• May 15th: Final project report due

Schedule (topic and papers)

Resources

Other code and data:

• Visual Object Recognition synthesis lecture by Grauman and Leibe (short book on object recognition methods)
• Compiled list of recognition datasets
  
• OpenCV (open source computer vision library)
• Weka (Java data mining software)
• Netlab (Matlab toolbox for data analysis techniques, written by Ian Nabney and Christopher Bishop)
• CV Online
• Annotated Computer Vision Bibliography
• Oxford group interest point software
• Andrea Vedaldi's VLFeat code, including SIFT, MSER, hierarchical k-means.
• INRIA LEAR team's software, including interest points, shape features
• FLANN - Fast Library for Approximate Nearest Neighbors.  Marius Muja et al. 
• Google Goggles
• Kooaba
• LSH homepage
• Code for downloading Flickr images, by James Hays
• UW Community Photo Collections homepage
• INRIA Holiday images dataset
• NUS-WIDE tagged image dataset of 269K images
• MIRFlickr dataset
• LIBPMK feature extraction code, includes dense sampling
• LIBSVM library for support vector machines
• PASCAL VOC Visual Object Classes Challenge
  
• Fast SLIC superpixels
  
• Greg Mori's superpixel code
• Berkeley Segmentation Dataset and code
• Pedro Felzenszwalb's graph-based segmentation code
• Mean-shift: a Robust Approach Towards Feature Space Analysis [pdf]  [code, Matlab interface by Shai Bagon]
• David Blei's Topic modeling code
• Berkeley 3D object dataset (kinect)
• Labelme Database 
• Stanford Event Dataset
  
• SUN Scene and object dataset
• ImageNet dataset of 15K objects and ImageNet challenge
• Animals with Attributes dataset
• aYahoo and aPascal attributes datasets
• Attribute discovery dataset of shopping categories
• Public Figures Face database with attributes
• Relative attributes data
• WhittleSearch relative attributes data
• SUN Scenes attribute dataset
• Cross-category object recognition (CORE) dataset
• Leeds Butterfly Dataset
• FaceTracer database from Columbia
• Caltech-UCSD Birds dataset
• Database of human attributes
  
• Face detection code in OpenCV
• Gallagher's Person Dataset
• Face data from Buffy episode, from Oxford Visual Geometry Group
• CALVIN upper-body detector code
• UMass Labeled Faces in the Wild
• Ivan Laptev's Space-Time Interest Points code
• Hollywood activity dataset
• Stanford 40 Actions still image dataset
• Stanford People Playing Musical Instrument dataset
  
• UCF activity datasets
• PASCAL VOC action recognition taster challenge
• TRECVID video retrieval challenge
• UMich Collective Activity dataset
• Egovision workshop at CVPR 2012
• Amazon Mechanical Turk
• Using Mechanical Turk with LabelMe
• Point Cloud Library
• Robot Operating System
• KITTI Benchmark
  

• Frontiers in Computer Vision Workshop
• Scene Understanding Symposium SUnS
• Como Workshop on Category-level Object Recognition (2008)
• IMA Visual Learning and Recognition Workshop (2006)
• MSRI Visual Recognition Workshop (2006)
• ENS/INRIA Visual Recognition and Machine Learning Summer School
• International Computer Vision Summer School
• Vision and Sports Summer School
• Computer vision conferences
• ICCV 2005 / CVPR 2007 / ICCV 2009 Short Course on Recognition
• AAAI 2008 Tutorial on Recognition
• ICML 2008 Tutorial on Recognition
• Greg Mori and Ivan Laptev's tutorial on action recognition at ECCV 2010
• CVPR 2009 Workshop on Visual Place Categorization

Similar courses:

This course has been inspired by the following two courses:

• Visual Recognition (Kristen Grauman, Texas-Austin, Fall 2012)
• Learning-Based Methods in Vision (Alyosha Efros, CMU, Spring 2012) 

• Grounding Object Recognition and Scene Understanding (Antonio Torralba, MIT, Fall 2011)
• Visual Scene Understanding (Derek Hoiem, UIUC, Spring 2009)
• Statistical Models for Visual Recognition (Deva Ramanan, UCI, Winter 2009)
• Object Recognition and Scene Understanding (Antonio Torralba, MIT, Fall 2008)
• Scene Understanding Seminar (Aude Oliva, MIT, Fall 2006)
• Selected Topics in Vision & Learning (Serge Belongie, UCSD, Spring 2011)
• Learning and Inference in Vision (Bill Freeman, MIT)
• Cutting Edge of Computer Vision (Fei-Fei Li, Stanford)
• Recognizing People, Objects, and Scenes (Jitendra Malik, Berkeley)
• Recognition Problems in Computer Vision (Greg Mori, SFU)
• Vision and Learning (Jianbo Shi, UPenn)