Reading 26 : Shape Context

Citation:
Oltmans, Michael. Envisioning sketch recognition: a local feature based approach to recognizing informal sketches. Diss. Massachusetts Institute of Technology, 2007.
Publication Link 

Summary:
This paper focuses on recognizing sketches based on their visual appearance rather than stroke information. However, comparing pixels based on perceptual similarity will lead to inaccurate results. Hence, the sketches are represented as parts, and these parts are compared. The main challenges in this include over-tracing,  variations in signals, as well as segmentation issues.

Discussion:

The main takeaway from this paper is the bullseye representation used for images. In this representation shapes are described as collections of visual parts and their allowable conceptual variations. A visual part represents the appearance of a region of the shape. In contrast to a conventional tile based matching system, a bullseye representation is selected.   Each of the subdivisions of the circular region is a histogram bin that measures how many pixels of ink are contained in that region. A part is represented as a vector formed from the counts in each bin. To form the representation, we compute first calculate the visual parts at a sampling of locations that cover the sketched symbol. Then a standard codebook of parts is used to identify each individual part. This codebook is preprocessed for common parts in the domain(eg. resistors). Finally, the representation of the sketched symbol is calculated as a vector of distances that indicates the degree to which each of the codebook parts appears in the sketched symbol. The match vector representation is used to train a classifer to learn the differences between shape classes and to learn the allowable variations within a shape class. 

The second task is to localize of shapes in complete sketches, which is done by identifying candidate locations and training the classifier on them.