Ink Features for Diagram Recognition

Authors: Rachel Patel, Beryl Plimmer, John Grundy, Ross Ihaka

Comments:
1. Daniel's blog

Summary:
This paper proposes an algorithm for text vs shape recognition. This paper builts a classification tree based on certain features to classify text from shape. The results are then compared to Microsoft's divider and Ink Kit.

Feature Selection:
Set of 46 shapes were selected and then important features were selected based on analysis of these features on the following data. Set of 9 shapes were recognized and samples were collected from 26 people. The paper uses rpart function to find a classification tree. The aim is to find the most optimal position for a split to be made so that there are a minimal amount of misclassified strokes. If this is done for all features in the feature set, using the observations in the dataset, then the features that most accurately split the data into text and shape stroke groups, with the least amount of misclassified strokes, will be identified as the significant features for division of text and shape strokes.

The final set of features are Time till next stroke, Speed till next stroke, Distance from last stroke.Distance to next stroke, Bounding box width, Perimeter to area, Amount of ink inside, Total angle.

* The size of shape strokes is much larger than text strokes reflected by the use of bounding box width, perimeter to area and amount of ink inside features.
* Curvature is relevant for differentiating joined up letters from shapes
* Inter stroke distance is used to find words. This feature is slow for strokes in a word. Faster speed but a high inter stroke distance suggests next word.

Future work - first step would be to replace classification tree with more robust classifiers with the same features which allows variability.

Discussion:
Samples of cases where the algorithm failed would have been useful to understand more about the algorithm.
I think the sample sketches used give more idea about some symbols which fall under difficult cases for classification. The check box and the musical notes are some difficult symbols to distinguish from the text.