The visual feature tracker detects and tracks features in the view of the cellphone camera. The tracks of individual features are shown as tails to the current location of the feature. The tails is colored according to how old the respective position is relative to the current position: From very recent observations in yellow to old positions in red.
The application has the purpose of demonstrating some of the state of the art Computer Vision and to be fun to play around with (How long can you keep a feature tracked while moving your cellphone around?).
Get the app for free on Google Play! If you like it and would like to support me consider donating an Euro or two via PayPal!
How to use
There is two modes: feature tracking and a simple camera view. In both modes touching the screen saves the image to the external storage. You can switch between them using the menu.
For each incoming frame from the camera, the Visual Tracker performs three steps:
- extract so called FAST corners
- associate this frames set of FAST corners with the set from the previous frame based on distance in the image and sum of squared differences (SSD) between the patches around two corners
- plot the traces of all tracked corners – traces across multiple frames are build up from frame-to-frame matches
FAST corners are pixels that fulfill a certain set of conditions that are selected such that the pixel is salient i.e. it has a high chance of being identifiable in the next frame and from other viewing angles and scales too. This is important since we want to track these corner or also feature points as long as possible – no matter the scale and view change of the camera.
The association of FAST corners between two frames underlies to assumptions: (1) the corner cannot move more than a few pixels per frame and (2) the image patch around a corner will be very similar between two frames – this means the distortion of the patch due to translation and rotation of the camera is small.
Assumption (1) allows us to prune most possible corner-corner associations. This is important, since the computation of the sum of squared differences (SSD) between two image patches gets quickly expensive with too many putative matches.
The frame-to-frame associations of corners thus obtained are then used to build up a longer history of the position of one specific corner in the image. This trace is plotted in on top of the current camera image. To depict the age of the trace, the oldest known position for a given corner gets more and more red the longer it has been tracked.