Authors: Brandon Paulson, Tracy Hammond
Comments:
1. Daniel's blog
2. Ben's Blog
Summary:
MARQS - Media Albums Retrieved by a Query Sketch - an application to which can store music/ pictures and retrieve them with the help of sketch based queries.Applications uses 2 recognizers - simple classifier and a linear classifier (similar to Rubine). The classifiers use set of global features that describe the sketch rather than single stroke. This helps the classifier to be extended to multi stroke sketches.
Sketches are first rotated about an major axis ( line between farthest points on sketch) such that the axis lies horizontal.Features - Bounding box aspect ratio,Pixel Density, Average Curvature and The number of perceived corners across all strokes of the sketch. Simple classifier is used for sketch with one example. It calculates the absolute differences between the feature of sketch and template (Errors), then normalized by dividing them by the value of feature in query sketch.The errors are then summed up and the result is published based on the lowest errors.
Linear classifier (similar to rubine) is applied to sketches with more than 2 samples.
Discussion:
This algorithm is an interesting approach. it is not affected by the direction of stroke/ scale/ change in local orientation. (None of the features depend on these factors). I do not think pixel density is an important feature- I would have to look into how this feature affects sketch recognition. Anyways it would be taken care by the weights calculated by the linear classifier.
I would like to know if there are features which can fit in this algorithm.
Sunday, August 31, 2008
Specifying Gestures by example
Author: Dean Rubine
Comment:
1. Daniel's blog
Summary:
This paper describes GRANDMA, a toolkit for adding gestures to click and drag interfaces and about the single stroke gesture recognizer used by the toolkit. The single stroke gesture recognition is chosen in order to avoid segmentation problems of multi stroke gesture based systems and to use eager recognition. Single stroke gestures also contribute to better usability of the UI.
Designing Gestures with GRANDMA
GRANDMA is a Model/View/Controller(MVC) framework. Gesture Designer must first determine the view classes and the associated gestures. A controller is associated with a view class and all its instances and instances of all the subclass. To add a gesture to the system, a gesture handler is created. Then a new gesture class is created with 15 training examples per class.
Semantics of each gesture has 3 actions: recog, manip and done. Recog is evaluated when gesture is recognized(when the mouse stops moving -0.2 seconds time out), Manip is evaluated on subsequent mouse points and Done is evaluated when the mouse button is released. Gesture drawn over multiple gesture handling views are recognized with the priority given to the topmost view.
Sketch recognition is done based on the statistical analysis on the vector of features(extracted from input gesture). Features are chosen based on constant computation time, ability work efficiently on large/small gestures and ability to differentiate between gestures. Author suggests not to use rejection when 'undo' can be used. Eager Recognition allows the system to recognise the gesture even before its completed without any explicit indication from the user.Multi Finger gesture recognition can be done by applying single recognition on each stroke and applying global features to discriminate between different multi path gestures.
Discussion:
First, thing i donot understand is the purpose of views. The next thing, I don't understand the significance of associating views with gestures.
Note: Rubine's features does not take care of scaling/ direction of stroke. So a large gesture may not be recognized if the training data were small in size.
Comment:
1. Daniel's blog
Summary:
This paper describes GRANDMA, a toolkit for adding gestures to click and drag interfaces and about the single stroke gesture recognizer used by the toolkit. The single stroke gesture recognition is chosen in order to avoid segmentation problems of multi stroke gesture based systems and to use eager recognition. Single stroke gestures also contribute to better usability of the UI.
Designing Gestures with GRANDMA
GRANDMA is a Model/View/Controller(MVC) framework. Gesture Designer must first determine the view classes and the associated gestures. A controller is associated with a view class and all its instances and instances of all the subclass. To add a gesture to the system, a gesture handler is created. Then a new gesture class is created with 15 training examples per class.
Semantics of each gesture has 3 actions: recog, manip and done. Recog is evaluated when gesture is recognized(when the mouse stops moving -0.2 seconds time out), Manip is evaluated on subsequent mouse points and Done is evaluated when the mouse button is released. Gesture drawn over multiple gesture handling views are recognized with the priority given to the topmost view.
Sketch recognition is done based on the statistical analysis on the vector of features(extracted from input gesture). Features are chosen based on constant computation time, ability work efficiently on large/small gestures and ability to differentiate between gestures. Author suggests not to use rejection when 'undo' can be used. Eager Recognition allows the system to recognise the gesture even before its completed without any explicit indication from the user.Multi Finger gesture recognition can be done by applying single recognition on each stroke and applying global features to discriminate between different multi path gestures.
Discussion:
First, thing i donot understand is the purpose of views. The next thing, I don't understand the significance of associating views with gestures.
Note: Rubine's features does not take care of scaling/ direction of stroke. So a large gesture may not be recognized if the training data were small in size.
Saturday, August 30, 2008
Visual Similarity of Pen Gestures
Comments:
1. Nabeel's Blog
Summary:
Gestures, a powerful feature in Pen based UI, are commands issued with a pen. Factors that affect selection of gestures are similarity among gestures, difficulty in learning gestures and wrong recognition by the computer.A pair of experiments is conducted to measure the similarity among gestures and derive an algorithm for computing the similarity.To analyze the date collected, Multi-Dimensional scaling technique is used. This technique reduces the number of dimensions in the data set so the patterns can be seen by viewing a plot of the data (2/3 dimensions). The gesture set chosen in the first experiment were based on 2 criteria: span a wide range of gestures and to include variation in orientation. Data was collected from 21 people. Analysis of the plots of gestures generated by MDS helped author to determine the geometric properties which influenced the users perceived similarity. Regression analysis helped the author develop a model of gesture similarity which can predict how people perceive 2 gestures. Similarity of the gestures were given by the euclidean distance between their features. Smaller distance means greater similarity. Regression analysis produced weights for different features in determining the similarity.The authors were able to derive a model which predicts gesture similarities with 0.74 correlation with reported data.Second experiment had 3 gesture sets with each set used to check the effect of variation of particular features on similarity perception.Though this goal could not be completely achieved since some features co varied with others.The derived model predicted reported gesture similarity with 0.71 correlation. The analysis of first gesture set (effects of absolute angle and aspect) gave 3 dimension with MDS plot. It showed that absolute angles in the set co varied greatly that it was difficult to determine if it was significant.Analysis of second gesture set shows that neither length nor area and significant in similarity judgement.Analysis of third gesture set showed that gestures whose lines were horizontal and vertical were perceived similar.
Discussion:
In both the experiments, the users were unaware of the operations a gesture is mapped to. It would interesting to find if there is a link between the context in which the gesture is used( application with which an user is interacting), operations performed by the gestures and the way user perceives a gesture.
1. Nabeel's Blog
Summary:
Gestures, a powerful feature in Pen based UI, are commands issued with a pen. Factors that affect selection of gestures are similarity among gestures, difficulty in learning gestures and wrong recognition by the computer.A pair of experiments is conducted to measure the similarity among gestures and derive an algorithm for computing the similarity.To analyze the date collected, Multi-Dimensional scaling technique is used. This technique reduces the number of dimensions in the data set so the patterns can be seen by viewing a plot of the data (2/3 dimensions). The gesture set chosen in the first experiment were based on 2 criteria: span a wide range of gestures and to include variation in orientation. Data was collected from 21 people. Analysis of the plots of gestures generated by MDS helped author to determine the geometric properties which influenced the users perceived similarity. Regression analysis helped the author develop a model of gesture similarity which can predict how people perceive 2 gestures. Similarity of the gestures were given by the euclidean distance between their features. Smaller distance means greater similarity. Regression analysis produced weights for different features in determining the similarity.The authors were able to derive a model which predicts gesture similarities with 0.74 correlation with reported data.Second experiment had 3 gesture sets with each set used to check the effect of variation of particular features on similarity perception.Though this goal could not be completely achieved since some features co varied with others.The derived model predicted reported gesture similarity with 0.71 correlation. The analysis of first gesture set (effects of absolute angle and aspect) gave 3 dimension with MDS plot. It showed that absolute angles in the set co varied greatly that it was difficult to determine if it was significant.Analysis of second gesture set shows that neither length nor area and significant in similarity judgement.Analysis of third gesture set showed that gestures whose lines were horizontal and vertical were perceived similar.
Discussion:
In both the experiments, the users were unaware of the operations a gesture is mapped to. It would interesting to find if there is a link between the context in which the gesture is used( application with which an user is interacting), operations performed by the gestures and the way user perceives a gesture.
Sketchpad by Ivan Sutherland
Comments:
1. Nabeel's blog
Summary:
This paper is a description of the Sketchpad system. This system allows the user to draw a shapes with the help of a pen and combination of keys. Author presents an example of drawing an hexagon to explain the concept of subpicture, constraints and copying.Changing the base hexagon changes the appearance of all the shapes that used the copies of the hexagon. A circular list of pointers is used to store the information on subpicture/shape. Garbage collection is performed over the list to remove free blocks. An hierarchy is maintained while storing new shapes. All shapes are grouped in a ring structure under generic. The blocks are further divided into super generic and generic generic blocks. For example: shape related to lines goes under Line with specific features. All the shapes in sketchpad are generated from lines, arcs and circles. These lines and circles are generated using difference equations.Features like Recursive merging, recursive deleting and expanding instances increases scope of sketchpad applications. Sketchpad allows users to add constraints to the system. Each constraint is added as a ring under generic block. Constraint satisfaction is done by a one pass method ,which tries to reduce the errors to zero by adjusting the constraint variables. When it fails, a slow and reliable relaxation method is used.
Discussion:
It would interesting to know the criteria used by the garbage collection algorithm in the system.
This system is similar to the FLUID framework in using the Geometric constraints.While this system uses the geometric constraints to create new shapes and use them to create complex pictures, the FLUID system uses these constraints to identify the ink drawn by user.
1. Nabeel's blog
Summary:
This paper is a description of the Sketchpad system. This system allows the user to draw a shapes with the help of a pen and combination of keys. Author presents an example of drawing an hexagon to explain the concept of subpicture, constraints and copying.Changing the base hexagon changes the appearance of all the shapes that used the copies of the hexagon. A circular list of pointers is used to store the information on subpicture/shape. Garbage collection is performed over the list to remove free blocks. An hierarchy is maintained while storing new shapes. All shapes are grouped in a ring structure under generic. The blocks are further divided into super generic and generic generic blocks. For example: shape related to lines goes under Line with specific features. All the shapes in sketchpad are generated from lines, arcs and circles. These lines and circles are generated using difference equations.Features like Recursive merging, recursive deleting and expanding instances increases scope of sketchpad applications. Sketchpad allows users to add constraints to the system. Each constraint is added as a ring under generic block. Constraint satisfaction is done by a one pass method ,which tries to reduce the errors to zero by adjusting the constraint variables. When it fails, a slow and reliable relaxation method is used.
Discussion:
It would interesting to know the criteria used by the garbage collection algorithm in the system.
This system is similar to the FLUID framework in using the Geometric constraints.While this system uses the geometric constraints to create new shapes and use them to create complex pictures, the FLUID system uses these constraints to identify the ink drawn by user.
About me
Email Address: manoj.prasad@neo.tamu.edu
Graduate Standing: 1st year Masters
Why am i taking Sketch Recognition class? I am interested in this field.This is the field which has kept me interested for more than 1 year.
What experience do you bring to this class? During my internship in HP labs, i had built an IM which can connect with gmail account and send Ink across clients. Other applications i had built were xml parsing applications (INKML parsing) and converters (ISF <--> INKML converters).
What do you expect to be doing in 10 years? I dont know.
What do you think will be the next biggest technological advancement in computer science?
quantum computers generating real random numbers.
What was your favorite course in undergrad (CS or otherwise)?
Computer Architecture
If you could be another animal, what would it be and why?
Ants because they are workaholics. I like to become one but am not able to.
What is your favorite motto or slogan?
What is your favorite movie? Rope
Name some interesting fact about yourself.
I am lazy.
I like to know about things which i dont know.
Graduate Standing: 1st year Masters
Why am i taking Sketch Recognition class? I am interested in this field.This is the field which has kept me interested for more than 1 year.
What experience do you bring to this class? During my internship in HP labs, i had built an IM which can connect with gmail account and send Ink across clients. Other applications i had built were xml parsing applications (INKML parsing) and converters (ISF <--> INKML converters).
What do you expect to be doing in 10 years? I dont know.
What do you think will be the next biggest technological advancement in computer science?
quantum computers generating real random numbers.
What was your favorite course in undergrad (CS or otherwise)?
Computer Architecture
If you could be another animal, what would it be and why?
Ants because they are workaholics. I like to become one but am not able to.
What is your favorite motto or slogan?
What is your favorite movie? Rope
Name some interesting fact about yourself.
I am lazy.
I like to know about things which i dont know.
Thursday, August 28, 2008
Introduction to Sketch Recognition by Tracy Hammond and Kenneth Mock
Comment:
1. Ben's Blog
Summary:
This paper is an overview of almost all the pen based technology available today. Sketchpad the first pen based system was created by Ivan sunderland in 1963. This system which utilized a pen and set of keys as mode of input to create simple graphics. From this primitive form , the pen based evolved over 40 years to tablet PCs, convertibles, whiteboards, USB connected Pen tablets, ... which are cheaper and affordable.These pen based systems use Digitizers to capture Pen based activities. These digitizers are of 2 type - Active and Passive. Passive digitizers detect touch, so even finger would be recognised as pen. The drawback of these digitizers are that they recognise the touch from palm and other things which are in contact with it. This causes vectoring. These digitizers usually have low accuracy and resolution. Active digitizers on the otherhand, recognise electromagnetic waves to detect pen activities(only special pens) and it is more precise.
This paper also discusses about the various applications of Pen based systems. Pen based systems have found themselves useful in teaching. Application like One Note, Annotations on Powerpoint, ... can be used to make presentation more dynamic and user defined. With the pen based systems, the field of sketch recognition has also grown into an important field. This field has application in diverse areas. ChemPad , an application which can recognise sketched diagran into a 3D view of the molecular structure. Hand Sketch, an application which recognises hand drawn symbols and plays music accordingly. This paper also discusses about a Framework(FLUID), which can used to create sketch interfaces. This framework uses ,LADDER and GUILD. LADDER, a language for defining objects that has to be identified by the application.
GUILD, a recognition system which recognises shapes based on the LADDER definition. The output of this system can be fed to Back end system , which would do domain specific jobs.
Discussion:
- Pen based systems have grown upon application where mouse cannot be used. Writing, sketching and gestures where using mouse would be difficult.So there raises the question - Can pen replace mouse? I think this would be difficult because we are used to mouse for so long that any change would cause discomfort to the users. This would require change in UI to either accomodate mouse activities( the way it is done now) or create a new UI which would not need mouse.
1. Ben's Blog
Summary:
This paper is an overview of almost all the pen based technology available today. Sketchpad the first pen based system was created by Ivan sunderland in 1963. This system which utilized a pen and set of keys as mode of input to create simple graphics. From this primitive form , the pen based evolved over 40 years to tablet PCs, convertibles, whiteboards, USB connected Pen tablets, ... which are cheaper and affordable.These pen based systems use Digitizers to capture Pen based activities. These digitizers are of 2 type - Active and Passive. Passive digitizers detect touch, so even finger would be recognised as pen. The drawback of these digitizers are that they recognise the touch from palm and other things which are in contact with it. This causes vectoring. These digitizers usually have low accuracy and resolution. Active digitizers on the otherhand, recognise electromagnetic waves to detect pen activities(only special pens) and it is more precise.
This paper also discusses about the various applications of Pen based systems. Pen based systems have found themselves useful in teaching. Application like One Note, Annotations on Powerpoint, ... can be used to make presentation more dynamic and user defined. With the pen based systems, the field of sketch recognition has also grown into an important field. This field has application in diverse areas. ChemPad , an application which can recognise sketched diagran into a 3D view of the molecular structure. Hand Sketch, an application which recognises hand drawn symbols and plays music accordingly. This paper also discusses about a Framework(FLUID), which can used to create sketch interfaces. This framework uses ,LADDER and GUILD. LADDER, a language for defining objects that has to be identified by the application.
GUILD, a recognition system which recognises shapes based on the LADDER definition. The output of this system can be fed to Back end system , which would do domain specific jobs.
Discussion:
- Pen based systems have grown upon application where mouse cannot be used. Writing, sketching and gestures where using mouse would be difficult.So there raises the question - Can pen replace mouse? I think this would be difficult because we are used to mouse for so long that any change would cause discomfort to the users. This would require change in UI to either accomodate mouse activities( the way it is done now) or create a new UI which would not need mouse.
Subscribe to:
Comments (Atom)
