Studierstube Tracker

Availability

This page is an overview of our in-house technology, which is not available to the public. For an overview of available AR frameworks from our group check this page: Framework availability

Overview

Studierstube Tracker is a computer vision library for detection and pose estimation of 2D fiducial markers. It is a successor to the well known ARToolKitPlus library. Its concept is very similar to that of ARToolKit (ARTK), ARToolKitPlus (ARTK+) and ARTag, but its code base is completely different. Studierstube Tracker has been written from scratch with high performance for PCs as well as mobile phones in mind.

How does it compare to ARToolKitPlus?

Studierstube Tracker has been developed from scratch after 4 years of experience with ARToolKitPlus. It was designed from ground up to support mobile phones as well as PCs. Hence, its memory requirements are very low (100KB, 5-10% of ARTK+) and processing is very fast (about twice as fast as ARTK+ on mobile phones, ~1ms per frame on a PC). While ARTK+ follows a monolithic approach, requiring configuration at compile time, Studierstube Tracker is highly modular: Developers can extend Studierstube Tracker in any way, creating new features for it. Other than ARTK+, Studierstube Tracker is not open source though. Studierstube Tracker does not share any source code with ARToolKitPlus. It is a separate project and has no legal connections to ARToolKit or ARToolKitPlus.

Feature Overview

Studierstube Tracker's main features over other solutions are:
  • Fully class-based API
  • Many different marker types
    • Template markers (ARTK-style)
    • ID-markers (simple-id & BCH)
    • DataMatrix markers (see example 2 below)
    • Frame markers (see examples 3 & 4 below)
    • Split markers (see example 5)
    • Grid markers (see example 6)
  • Highest performance on low-end devices
    Studierstube Tracker is the fastest solution for tracking on mobile phones. It is about twice as fast as ARTK+, which used to be the fastest marker tracking library so far. At the same time tracking on phones is much more stable (less jitter) than with ARTK+.
    Benchmarks show that up to 185 images per second can be tracked on a 312Mhz smartphone (see below).
  • Low memory consumption
     The memory footprint of Studierstube Tracker mostly depends on the actual camera resolution (1 byte per pixel required). For a typical phone setup with a video stream of 320x240 pixels in YUV12 format the memory usage is below 100KB. This is only 5-10% of the memory usage of ARTK+.
  • Small Executable
     Studierstube Tracker.dll for Windows CE devices is only ~270KB storage.
  • Up to 4096 id-based markers
     The BCH marker system allows up to 4096 markers that are reliably detected at no speed penalty due to large number of markers. No markers need to be trained.
  • Up to 4 million frame markers
     Our new frame markers (FrameMarker2) can encode 22 bits of data and correct up to 3 wrong bits.
  • Camera pixel-formats (RGB24, RGB32, RGB565, YUV12)
    Studierstube Tracker natively supports pixel formats that are common on mobile phones such as RGB565 or YUV12.
  • Variable marker border width
     The marker border with can be modified freely giving more design choices.
  • High-quality pose estimation gives more stable tracking (less jitter).
    Studierstube Tracker includes an implementation of "Robust Planar Pose" by G. Schweighofer and A. Pinz
    This mode is only available on the PC version of Studierstube Tracker.
  • Support for MATLAB camera calibration toolbox
    Usage of the MATLAB camera calibration toolbox gives high accuracy camera calibration and works with any device (PC, mobile phone, etc).
  • Automatic thresholding
    Adapts fully automatic to changing lighting conditions.
    Our new adaptive thresholding algorithm can correctly binarize even extremely uneven light images. See Example 8.
  • Fully configurable at runtime
    All aspects of the tracking pipeline can be configured at runtime.
  • No limits to video resolution
    There are no limits to the size of input images (except for memory consumption; 1 byte per pixel). Arbitrarily sized images can be used for tracking. Images with 8 Megapixels and 2000 markers have been successfully tested.
  • Easy to extend
     Studierstube Tracker's was designed from ground up to be easily extensible. Every step of the tracking pipeline is configurable and can be exchanged with custom code - even when using the binary distribution of Studierstube Tracker.
  • Highly portable
    Studierstube Tracker's code base is highly portable since it does not depend on any external libraries except our own Studierstube Core & Math libraries. It further obeys restrictions specific to mobile phones for optimal portability.

 

Platform Support

Studierstube Tracker currently runs on
  • Windows XP
  • Windows CE & Windows Mobile
  • Symbian, Linux
  • MacOS
  • iPhone.

Extendibility

Studierstube Tracker is completely modular. Every part of the pipeline is implemented as a separate class, called a "Feature". The following features make up the complete tracking pipeline:
  • Thresholding
  • Fiducial detection
  • Marker detection
  • Corner filtering
  • Pose estimation
  • Pose filtering

What does Studierstube Tracker not do?

Studierstube Tracker is a marker tracking library. It analyzes camera images and reports the relative pose to markers found in the image. It does NOT
  • Read images from a camera
  • Render anything
  • Track natural features
  • Depend on or support any hardware units

Examples


Example 1 : Benchmark image. Tracked at a speed of
185 images/sec on a Motorola Q phone with 312 Mhz.


Example 2 : Tracking a DataMatrix Marker.
(encoded message: "http://www.imagination.at/")


Example 3: Tracking and augmenting a Frame-Marker.


Example 4: Tracking from a Frame-Marker with image content.


Example 5: Tracking a Split-Marker. FPS include complete AR workflow (most time goes into rendering...)


Example 6: Studierstube Tracker tracking from a regular map (extended with small dots).


Example 7: Studierstube Tracker detecting 383 BCH-markers (marked with crosses) in a 640x480 image.
Crosses are not centered exactly on the marker due to lens undistortion.


Example 8:Extreme unbalanced lighting. On the left edge of
the marker the white color is three times darker than the black color
on the right edge. StbTracker correctly detects this marker.