Processamento de Imagem e Visão 

Project  2

Objectives - 
The second (and worthy of the name) PROJECT,  must be the realization of an idea which solves some problem or enables some classes of solutions. In short, think about a problem where images (3D, 2D, color, ultrasound, anything we call an image as described in the first class) are used to achieve some task.
How to write the proposal
You should sketch and PLAN the implementation of your idea. We will help you with the software (the idea is the thing!). We will supply you with software to do most of the tasks and advise on which path to follow. One example:

Ideia : A system to detect and track books exposed on an exhibitor (table). People grab the book and the software should detect  the motion,  search for a book and identify it.

Main tasks:

- 1) Detect motion in images (segment aggregate pixels belonging to moving objects)
- 2) Detect a book-like surface on the segmented image
- 3) Recognize the image pattern of the region and  compare it with a database.

Procedures:
These tasks translate into procedures/methods. So, you must  translate that into "math".
Assumption: the kinect is calibrated therefore you know the relation between the depth image and the rgb image.

Tasks
1) Can be done by  background subtraction : Estimate the background image,and then subtract it to the current image.  If the difference is "large" than classify a pixel as "moving". Run connected componentes and label regions candidate to be labeled as book
2) Check which pixels in the regions of 1) are explained by a plane (use RANSAC ). Check that the plane is conformal with book dimensions.
3) Correct the perspective (homography correction) and use color and image features to compare the book: for example, color histograms and/or keypoints in certain locations (SIFT, SURF, HARRIS, etc).
- There is software to do all these subparts. Everything you need is to iterate between sketching an idea, talk to me and go back to  planning and organizing!
Phases 1), 2) and 3) are illustrated in the video below, done by Hélder Miranda in his Master Thesis. His code is available to you (most of it  is part of the toolboxes already displayed in the software page).

Book segmentation