Object proposals using Non-parametric Label Transfer
Neelima Chavali Spring 2014 ECE 6504 Probabilistic Graphical Models: Class Project Virginia Tech
Fig 1.
Goal
We present a novel nonparametric, category-independent approach for generating
bounding box proposals which are likely to contain objects in an image. Given a
query image our system finds its nearest neighbours from a large databse
containing images annotated with bounding boxes. Our system then establishes a
dense correspondence between the query image and each of the nearest neighbours
using the SIFT flow algorithm. Based on the correspondences, the bounding boxes
in the neighbours are warped to the query image to produce candidate bounding
boxes. When compared to the same number of proposals, our approach outperforms
the state-of-the-art technique of Selective Search.
Approach
The core idea of our approach is object location-by-matching. Fig.1 shows the
pipeline of our approach, which involves the following steps:
1) Nearest Neighbour retrieval: Given a query image, retrieve a set of k nearest
neighbors from a given dataset in a particular feature space. We experiment with
the DeCAF, GIST and HOG.
2) Dense image alignment: Establish it dense pixelwise correspondence between
the query image and each of the retrieved nearest neighbors. Use the SIFT flow
algorithm for calculating correspondences.
3) Bounding box transfer: Warp the bounding box annotations from the nearest
neighbors to the query image according to the estimated dense correspondence.
Perform non-maximal suppression on the transferred bounding boxes to get the
bounding boxes of the query image.
Results
We evaluate the performance of our method as follows. For a given query image,
we calculate k=5,10,15,20 nearest neighbors using DeCAF, GIST, HOG. We compare
the Average Best Overlap (ABO) values in each case with ABO of the top n=25, 50,
75, 100 proposals of the Selective Search (SS), which is widely considered the
state of art technique for generating bounding box proposals. We use the
publicaly available implementation of SS. We sorted the selected search bounding
boxes with respect to the score reported by SS. Figure 2 shows the ABO achieved
by NPBT on PASCALVOCtest as a function of the number of bounding boxes
predicted.
We can see that NPBT-DeCAF @ 67 bounding box proposals outperforms SS @ 100
proposals by 16%, We note that SS and other techniques typically report results
at ~2k bounding boxes. However, we argue that in order for recognition to scale,
we must produce high ABO at small number of proposals.