ARToolKit Documentation

http://www.hitl.washington.edu/artoolkit/documentation/

Beginner

[Getting Started]

Introduction to ARToolKit
ARToolKit is a C and C++ language software library that lets programmers easily develop Augmented Reality applications. Augmented Reality (AR) is the overlay of virtual computer graphics images on the real world, and has many potential applications in industrial and academic research. One of the most difficult parts of developing an Augmented Reality application is precisely calculating the user's viewpoint in real time so that the virtual images are exactly aligned with real world objects. ARToolKit uses computer vision techniques to calculate the real camera position and orientation relative to marked cards, allowing the programmer to overlay virtual objects onto these cards. The fast, precise tracking provided by ARToolKit should enable the rapid development of many new and interesting AR applications. This guide contains a complete description of the ARToolKit library, how to install it, and how to use its functionality in AR applications. Several simple sample applications are provided with ARToolKit to enable the programmer to get started immediately. ARToolKit includes the tracking libraries and complete source code for these libraries enabling programming to port the code to a variety of platforms or customize it for their own applications. ARToolKit currently runs on the SGI IRIX, PC Linux, Mac OS X, and PC Windows (95/98/NT/2000/XP) operating systems. The last version of ARToolKit is completly multi-platform.. The functionality of each version of the toolkit is the same, but the performance may vary depending on the different hardware configurations. The current version of ARToolKit supports both video and optical see-through augmented reality. Video see-through AR is where virtual images are overlaid on live video of the real world. The alternative is optical see-through augmented reality, where computer graphics are overlaid directly on a view of the real world. Optical see-through augmented reality typically requires a see-through head mounted display and has more complicated camera calibration and registration requirements. Comments and questions about ARToolKit and any bug reports are actively invited. News about new releases of ARToolKit, comments and questions and applications people are working on is broadcast on the ARToolKit mailing list, and there is an active community forum (see community webpage). Bug reports and fixes, patches, and feature requests are also actively sought. These should be posted using the tracker on the ARToolKit SourceForge project pages, linked above. IMPORTANT NOTICE REGARDING YOUR RIGHT TO USE ARTOOLKIT: ARToolKit is made available freely for non-commercial use under the GNU General Public License. Commercial licenses to a professional implementation of ARToolKit are available for users for whom the GPL is not suitable, or who require a higher level of support, or who require customization or other specialist modifictions. Commercial licenses are administered by ARToolworks, Inc., Seattle, WA, USA. Click here for more information on ARToolKit licenses and your usage options.

 Setting up ARToolKit
Introduction
At its core, ARToolKit is a collection of software libraries, designed to be linked into application programs. For this reason, ARToolKit is distributed as sourcecode, and you must compile it on your specific operating system and platform. You will need a development environment for your operating system. Free environments are available on all platforms. Build files are also supplied to enable building on Windows in the non-free Microsoft Visual Studio platform. Although ARToolKit offers similar functions across multiple platforms, installation differs according to your operating system. In this tutorial we will guide you through the configuration process for your specific installation. Your machine, operating system and platform must satisfy some basic requirements. Your hardware must be able to acquire a video stream, and have spare CPU to handle the tasks of video processing and display. There are also some basic software dependencies - important to avoid compiler and linker errors. The main packages are cross-platform (e.g. OpenGL, GLUT), and others are related to the specific video library for your machine (DirectShow, V4L, QuickTime). The software dependencies for each supported operating system are outlined below along with installation instructions.

Building
Building on Windows
Prerequisites
Prerequisite Development environment DSVideoLib-0.0.8bwin32 Instructions Microsoft Visual Studio 6 and Microsoft Visual Studio .NET 2003 are supported, but it is also possible to build the toolkit using free development environments (e.g. Cygwin, http://www.cygwin.com/) On Windows, DSVideoLib is used to handle communication with the camera driver. DSVideoLib-0.0.8b or later is required for ARToolKit 2.71. A source + binary package of DSVideoLib is included on the ARToolKit downloads page on sourceforge. GLUT Verify that GLUT runtime and SDK is installed. If not, you can download a binary package containing GLUT for Windows from http://www.xmission.com/~nate/glut.html Verify that you have the GLUT runtime installed in your system directory.

e.g. c:\windows\system32 glut32.dll

Verify that GLUT SDK is installed in your Visual C++ installation: Include\gl\glut.h Lib\glut32.lib

DirectX Runtime Video input device

Verify that DirectX runtime is installed: with Windows XP it is installed by default. You need to check your version; it must be 9.0b or later. Plug your camera or video input into your PC and install any necessary drivers. Verify that your camera has a VFW or WDM driver by running the program amcap.exe (this can be found in the DirectX SDK, in Samples\C++\DirectShow\Bin). Check for your camera in the Devices menu (see Figure 1).

Figure 1: amcap Screenshot

(Optional, for VRML A source + binary package of OpenVRML is included on the ARToolKit renderer only) downloads page on sourceforge. OpenVRML-0.14.3-win32

Building ARToolKit
You can now build ARToolKit. Build steps: 1. Unpack the ARToolKit zip to a convenient location. This location will be referred to below as {ARToolKit}. 2. Unpack the DSVideoLib zip into {ARToolKit}. Make sure that the directory is named "DSVL". 3. Copy the files DSVL.dll and DSVLd.dll from {ARToolKit}\DSVL\bin into {ARToolKit}\bin.

4. Install the GLUT DLL into the Windows System32 folder, and the library and headers into the VS platform SDK folders. 5. Run the script {ARToolKit}\Configure.win32.bat to create include/AR/config.h. 6. Open the ARToolKit.sln file (VS.NET) or ARToolkit.dsw file (VS6). 7. Build the toolkit. The VRML renderering library and example (libARvrml & simpleVRML) are optional builds: 8. Unpack the OpenVRML zip into {ARToolKit}. 9. Copy js32.dll from {ARToolKit}\OpenVRML\bin into {ARToolKit}\bin. 10. Enable the libARvrml and simpleVRML projects in the VS configuration manager and build.

Linux / SGI Irix.

Prerequisites
Prerequisite OpenGL and GLUT Instructions Verify that OpenGL and GLUT RunTimes are installed: You need to find opengl.so, glu.so, and glut.so, generally in /usr/X11R6/lib, /usr/lib, or /usr/local/lib. Verify that OpenGL and GLUT SDKs are installed: You need to find opengl.h, glu.h and glut.h, generally in /usr/X11R6/include, /usr/include, or /usr/local/include. Video Library Verify that your Video Library is installed: On Linux, V4L RunTime and SDK installed, or dv1394 and dv1394 RunTime and SDK installed. On SGI, ARToolKit has only been tested with SGI O2 models, but it should work with any SGI machine with similar video input hardware. (Optional, for VRML A source package of OpenVRML is included on the ARToolKit downloads renderer only) page on sourceforge. OpenVRML-0.14.3

Building ARToolKit
You can now build ARToolKit. Open a console application. Unpack the ARToolKit to a convenient location. The root of this location will be referred to below as {ARToolKit}:

tar zxvf ARToolKit-2.71.tgz

Configure and build. The Linux builds support video input using either Video4Linux, an IIDC-compliant or DV camera connected via IEEE-1394, or a Sony EyeToy camera connected via USB. You will be prompted as to which of the four Linux video drivers you wish to use at the Configure step.

cd {ARToolKit} ./Configure

Linux Setup Screenshot

make
Following a successful build, to run a binary such as simpleTest:

cd {ARToolKit}/bin ./simpleTest
The VRML renderering library and example (libARvrml & simpleVRML) are optional builds:

cd {ARToolKit}/lib/SRC/ARvrml make cd {ARToolKit}/examples/simpleVRML make cd {ARToolKit}/bin ./simpleVRML

Mac OS X
Prerequisites
Prerequisite Apple Developer Tools Instructions XCode Tools install CD is included with Mac OS X, or disk image available free from Apple, http://developer.apple.com/tools/xcode/ Mac OS X 10.3 and later include an inbuilt IIDC and DV FireWire camera driver. USB camera drivers (and Firewire camera drivers for pre-10.3 systems) can be obtained from ioXperts at http://www.ioxperts.com/ VRML Install X11 off the Mac OS X install CD 3 or download an install disk image Apple from http://www.apple.com/downloads/macosx/apple/x11formacosx.html. Download Fink (and optionally, FinkCommander) from

Camera driver

(Optional, for renderer only) X11 and Fink

http://fink.sourceforge.net/download/.

Building ARToolKit
Building using the XCode IDE: 1. Unpack the archive to a convenient location using StuffIt Expander, and open the ARToolKit.xcodeproj. 2. Builds include a script target "Configure" which enables accelerated and rectangular texturing by default. If you wish to change these defaults, manually run the ./Configure script from Terminal as for a command-line build (below). 3. Executables are built as bundled applications into ARToolKit/bin, with the Data/ directory copied into the application bundle so that they may be moved from this location to other locations. The VRML renderering library and example (libARvrml & simpleVRML) are optional builds: 4. Using FinkCommander, do a binary install of mozilla-dev, followed by an install of openvrml4dev and openvrml-gl5-dev. 5. Select the ARToolKit extensions target, and build. Alternately, ARToolKit can be built from the Terminal, using the Unix makefiles. Drop the ARToolKit into a convenient location, e.g. your Desktop, then open a Terminal window and type:

cd ~/Desktop tar zxvf ARToolKit-2.71.tgz

Configure and build

cd ~/ARToolKit ./Configure make

Following a successful build, to run a binary such as simpleTest, add these commands:

cd bin ./simpleTest
The VRML renderering library and example (libARvrml & simpleVRML) are optional builds:

fink -b install mozilla-dev fink install openvrml4-dev openvrml-gl5-dev cd ~/Desktop/ARToolKit/lib/SRC/ARvrml make cd ~/Desktop/ARToolKit/examples/simpleVRML make

cd ~/Desktop/ARToolKit/bin ./simpleVRML

Running the utilities and examples

Current versions of ARToolKit aren't designed to be installed in any standard location; thethe current compilation directory should be used to build your ARToolKit application. As ARToolKit builds only static libraries, these will be linked into any executable you build. For using ARToolKit you need to have some markers available: some default markers use with the sample applications are presented on the patterns directory. You can open it (with your pdf reader) and print all of them, it will be useful for your next experiments. ARToolKit needs mainly rigid markers, so a simple solution is to glue them to cardboard (see hardware page for more information). After that you need to test if the graphics part of ARToolKit work with your application. For that run graphicsTest on the bin directory. if it's working a 3D window will be displayed with a rotated teapot:

Graphics Test Screenshot

This test confirms that your camera support ARToolKit graphics module with OpenGL. A minimum frame rate of 15 fps is recommended, below performance of ARToolKit rendering module will be too restricted (see FAQ). You need also to test if the video input works and video can be displayed. You need to run the videoTest on the bin directory. if it's working (no error) a window video will be displayed (see figure below).This test confirms that your camera supports ARToolKit video module and ARToolKit graphics module.

Video Test Screenshot

If nothing is displayed on the window, the problem is most likely related to your default camera configuration. (see FAQ).

Your First ARToolKit Example

Introduction
Once ARToolKit has been installed there is a sample program, your ARToolKit version, in the

simpleTest or simple according

bin directory that can be run to show the capabilities of ARToolKit. In order to run the code you need to print out the hiroPatt.pdf paper fiduccial marker that is contained in the directory patterns . Best performance is achieved if this is glued to a piece of
cardboard to keep it flat. In each case the simple program output is the same, so the final section describes how the simple program looks when it is running in different platforms.

Running ARToolKit
In each platform you have generally two choices: click on the program from your OS explorer or starting from the command line: the last choice is better since it give you the error and standard output stream (and ARToolKit used it a lot). Each platform offer a dialog box to setup the video before starting the main AR loop. Windows On a Windows PC double click on the simple.exe icon in the bin directory from your windows explorer. A dos console window will open and when the camera is detected the follow dialog will open. Otherwise start the Command Prompt program from the Start Bar (In all programs->accessories-> Command Prompt on winXP, english version), go the bin directory and run simple.exe.

Windows Camera Config

In this video dialog you can change the output size of the video or the video format acquired from the camera. You need to notice that better the format is similar to ARToolKit tracking format, faster is the acquisition (RGB more efficient). In some camera driver you can also change the frame rate. By default this dialog is associate to your first video input so you can have some problems (see the FAQ). Linux

simple icon in the bin directory from your gnome or kde explorer (notice than you will not have error or output stream display). Otherwise start a terminal, go the bin directory and runsimple. If you have V4L this dialog will be appear (development in progress for
On a Linux double click on the firewire dialog support).

Linux Camera Config (with V4L)

In this video dialog you can change the output size of the video or the video format acquired from the camera. You need to notice that better the format is similar to ARToolKit tracking format, faster is the acquisition (RGB more efficient). By default this dialog is associate to your first video input so you can have some problems (see the FAQ). MacOS On MacOS X double click on the simple icon in the bin directory from your mac explorer. A console window will open and when the camera is detected the follow dialog will open. Otherwise start the Terminal program, go the bin directory and run simple.

MacOS Camera Config

The default Mac video dialog is really advanced so you can change a lot of elements, like video input, format or setup the video parameters (saturation, brightness). Furthermore, a preview window is displayed that help for the configuration. You need to notice that better the format is similar to ARToolKit tracking format, faster is the acquisition (RGB more efficient). SGI On SGI you need to use a terminal, go to the bin directory, and run

simple.

simpleTest Output
If successful, a video window will appear on the screen (otherwise report you on the FAQ, mailing list information or the documentation). When you point the camera at the black square with HIRO printed in it you should see a virtual blue cube appear attached to the marker. Figure 5 shows a screen snapshot of the program running. As the real marker is moved the virtual object should move with it and appear exactly aligned with the marker.

The simple program running - a virtual block aligned with a real marker

In order for the virtual object to appear the entire black square border of the marker must be in view and also the pattern contained within the square border must be visible at all times. If the virtual image does not appear, or it flickers in and out of view it may be because of lighting conditions. This can often be fixed by changing the lighting threshold value used by the image processing routines. If you hit the t' key on the keyboard you will be prompted to enter a new threshold value. This should be a value between 0 and 255; the default is 100. Hitting the d' key will show the thresholded video

image below the main video window. Possible tracking patterns found in the input image are marked by a red square in the thresholded image. This will help you check the effect of lighting and threshold values on the video input. Figure 6 shows simple.exe with the threshold image on.

Thresholded Video Image with Identified Pattern

The Esc' key quits the program and prints out frame rate information. The simple program shows how an application based on the ARToolKit software is able to calculate the camera viewpoint in real time and use this information to precisely overlay virtual images on a real world object. In the next section we explain how this works.

How does ARToolKit work?

Basic Principles
ARToolKit applications allow virtual imagery to be superimposed over live video of the real world. Although this appears magical it is not. The secret is in the black squares used as tracking markers. The ARToolKit tracking works as follows: 1. The camera captures video of the real world and sends it to the computer. 2. Software on the computer searches through each video frame for any square shapes. 3. If a square is found, the software uses some mathematics to calculate the position of the camera relative to the black square. 4. Once the position of the camera is known a computer graphics model is drawn from that same position. 5. This model is drawn on top of the video of the real world and so appears stuck on the square marker. 6. The final output is shown back in the handheld display, so when the user looks through the display they see graphics overlaid on the real world. The figure below summarizes these steps. ARToolKit is able to perform this camera tracking in real

time, ensuring that the virtual objects always appear overlaid on the tracking markers.

Limitations
There are some limitations to purely computer vision based AR systems. Naturally the virtual objects will only appear when the tracking marks are in view. This may limit the size or movement of the virtual objects. It also means that if users cover up part of the pattern with their hands or other objects the virtual object will disappear. There are also range issues. The larger the physical pattern the further away the pattern can be detected and so the great volume the user can be tracked in. Table 1 shows some typical maximum ranges for square markers of different sizes. These results were gathered by making maker patterns of a range of different sizes (length on a side), placing them perpendicular to the camera and moving the camera back until the virtual objects on the squares disappeared. Pattern Size (inches) 2.75 3.50 4.25 7.37 Usable Range (inches) 16 25 34 50

This range is also affected somewhat by pattern complexity. The simpler the pattern the better. Patterns with large black and white regions (i.e. low frequency patterns) are the most effective. Replacing the 4.25 inch square pattern used above, with a pattern of the same size but much more complexity, reduced the tracking range from 34 to 15 inches. Tracking is also affected by the marker orientation relative to the camera . As the markers become more tilted and horizontal, less and less of the center patterns are visible and so the recognition becomes more unreliable. Finally, the tracking results are also affected by lighting conditions. Overhead lights may create reflections and glare spots on a paper marker and so make it more difficult to find the marker square. To reduce the glare patterns can be made from more non-reflective material. For example, by gluing black velvet fabric to a white base. The 'fuzzy' velvet paper available at craft shops also works very well.

Table 1: Tracking range for different sized patterns.

You will find more informations on computer vision principle at this page, or more informations on performance at this page.

Camera Calibration
Introduction
In the current ARToolKit software, default camera properties are contained in the camera parameter file camera_para.dat, that is read in each time an application is started. The parameters should be sufficient for a wide range of different cameras. However using a relatively simple camera calibration technique it is possible to generate a separate parameter file for the specific cameras that are being used. In a video-see through AR interface, if the camera parameters are known then the video image can be warped to remove camera distortions. This page presents how use utility programs included with ARToolKit to calibrate your video camera. ARToolKit provides two calibration approaches: Two Step Calibration Approach Difficult to use, but results in better accuracy (better for 3D measurement). One Step Calibration Approach Easy to use, giving accuracy good enough for image overlay.

The next section introduces different steps for calibrating your camera through these two approachs.

Two Step Calibration

In this section we describe how calibrate a camera using ARToolKit's two step method. In order to use this method the pattern files calib_cpara.pdf and calib_dist.pdf need to be printed out. These are found in the patterns directory. The calib_cpara.pdf pattern is a grid of lines and should be scaled so that the lines are exactly 40 mm apart. The calib_dist.pdf pattern contains a 6 x 4 dot pattern and should be scaled so that the dots are 40 mm apart. Once the pattern files are printed out they should be attached to pieces of cardboard so they lie flat and rigid. The figure below shows these patterns as seen through the camera.

Figure 1: The calib_dist pattern and the calib_cpara pattern

The important camera properties that must be measured include the center point of the camera image, the lens distortion and the camera focal length. The program calib_dist is used to measure the image center point and lens distortion, while calib_param produces the other camera properties. Both of these programs can be found in the bin directory and their source is in the utils/calib_dist and utils/calib_cparam directories. The calib_dist program should be run first and then calib_cparam, since calib_cparam uses the output of calib_dist. In the remainder of this section we explain how to run each of these programs.

Running calib_dist
calib_dist uses the calib_dist.pdf image of a pattern of 6 x 4 dots spaced equally apart. When viewed through the camera lens, lens distortion causes a pin cushion effect that produces uneven spacing between the dots in the camera image. The calib_dist program measures the spacing between the dots and uses this information to calculate the lens distortion. Run the calib_dist program from the command prompt. You will obtain this output in your terminal:

> calib_dist Image size (x,y) = (720,486) ----------Mouse Button Left : Grab image. Right : Quit. ----------A window will appear showing live video. Point the camera at the calibration pattern so that all the dots are in view and click the left mouse button. This freezes the video image, as shown in Figure 2. Now click and drag on the image with the left mouse button to draw a black rectangle over each dot. Start with the dot closest to the top left hand corner of the image and continue until all the dots are found. The dots must be covered in the following order:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
After each rectangle is drawn image processing software will find the dot enclosed by the rectangle and place a red cross at its center. If a red cross does not appear redraw the rectangle until the dot is found. Figure 2 shows a user drawing a rectangle over one of the last dots.

Figure 2: User marking the calibration dots.

While each dot is being found the following will appear on-screen:

----------Mouse Button Left : Rubber-bounding of feature. (6 x 4) Right : Cansel rubber-bounding & Retry grabbing. ----------# 1/24 # 2/24 # 3/24 # 4/24 # 5/24 # 6/24 # 7/24 # 8/24 # 9/24 # 10/24 # 11/24 # 12/24 # 13/24 # 14/24 # 15/24 # 16/24 # 17/24 # 18/24 # 19/24 # 20/24 # 21/24 # 22/24 # 23/24

# 24/24

Once all 24 dots in the image have been found click the left mouse button again. This will store the position of the dots and unfreeze the video image.

----------Mouse Button Left : Save feature position. Right : Discard & Retry grabbing. ----------### No.1 ### 1, 1: 125.01, 102.84 2, 1: 198.73, 96.19 3, 1: 283.00, 94.30 4, 1: 369.78, 99.93 5, 1: 448.78, 110.33 6, 1: 514.39, 123.37 1, 2: 118.84, 173.96 2, 2: 192.13, 171.33 3, 2: 277.61, 171.27 4, 2: 366.40, 175.28 5, 2: 446.74, 181.88 6, 2: 512.50, 189.64 1, 3: 119.86, 246.72 2, 3: 191.37, 248.83 3, 3: 274.59, 251.42 4, 3: 361.36, 253.61 5, 3: 440.32, 255.61 6, 3: 505.38, 257.05 1, 4: 127.78, 313.80 2, 4: 196.05, 319.71 3, 4: 272.48, 327.11 4, 4: 355.03, 325.72 5, 4: 430.25, 324.01 6, 4: 493.18, 320.03
You should now take another image and repeat the process for 5-10 images from various angles and positions. The more images taken the more accurate the calibration. The figures below show typical sample images.

Once you have taken 5-10 images, click the right mouse button to stop the image capture and start calculating the camera distortion values.

----------Mouse Button Left : Grab next image. Right : Calc parameter. ----------[360.0, 243.0, 174.0] 596.018289 [360.0, 243.0, 174.0] 596.018289 [360.0, 243.0, 174.0] 596.018289 [360.0, 243.0, 174.0] 596.018289 [360.0, 243.0, 174.0] 596.018289 [360.0, 243.0, 174.0] 596.018289 [360.0, 243.0, 174.0] 596.018289 [330.0, 223.0, 201.0] 590.288659 [330.0, 228.0, 201.0] 486.692482 [330.0, 233.0, 201.0] 400.390511 [325.0, 238.0, 201.0] 330.137494 [325.0, 243.0, 201.0] 276.447160 [325.0, 248.0, 201.0] 241.422442 [325.0, 253.0, 201.0] 227.895132 [325.0, 253.0, 201.0] 227.895132 [325.0, 253.0, 201.0] 227.895132 [325.0, 253.0, 201.0] 227.895132 [325.0, 253.0, 201.0] 227.895132 [325.0, 253.0, 201.0] 227.895132 [325.0, 253.0, 201.0] 227.895132 [325.0, 253.0, 201.0] 227.895132 [325.0, 253.0, 201.0] 227.895132 [325.0, 253.0, 201.0] 227.895132 [325.0, 253.0, 201.0] 227.895132 [325.0, 253.0, 201.0] 227.895132

[324.0, 253.5, 201.0] 227.334239 -------------Center X: 324.000000 Y: 253.500000 Dist Factor: 201.000000 -----------------------Mouse Button Left : Check fittness. Right : 1/10. ----------It may take a while to calculate these parameters, please be patient. The center x and y values and distortion factor are the final key values generated by the calib_dist code. These values will be different for every camera and should be written down for use in the calib_cparam program. In order to check that these parameters are correct click the left mouse button again. This will show the first image grabbed with red lines drawn through the calibration dots. These lines should pass through the center of each of these dots (see Figure 3). Each time the left mouse is clicked the next grabbed image will be shown.

Figure 3: Two Sample Calibrated Image Result.

Once you are satisfied with the results from calib_dist hit the right mouse button to quit and run the calib_cparam code.

Running calib_cparam
calib_cparam is used to find the camera focal length and other parameters. It uses the pattern contained in calib_cparam.pdf, a grid pattern of 7 horizontal lines and 9 vertical lines (see Figure 2). This pattern should be printed out and glued to a piece of cardboard or other rigid board. calib_cparam is run from the console like calib_dist. Type calib_cparam at the command prompt and input the center coordinates and distortion ratio found from calib_dist:

> ./calib_cparam Input center coordinates: X = 324 : Y = 253 Input distotion retio: F = 201 Image size (x,y) = (720,486)
1) A live video window will appear. 2) Put the calibration board in front of the camera so the board is perpendicular to the camera, all of the grid lines are visible and the grid is as large as possible (see Figure 2). 3) Click the left mouse button to grab the image. A white horizontal line will appear overlaid on the image. 4) Move the white line to overlay the top black grid line as close as possible. The line is moved up and down using the up and down arrow keys, while it is rotated clockwise and anticlockwise using the left and right arrow keys. Once the white line is aligned with the top grid line hit the enter key. This line will now turn blue and another white line will appear (see Figure 4). This process should then be repeated for all the horizontal lines. Once the last horizontal line has been placed a vertical white line will appear and the process should be repeated for the vertical lines. The first vertical white line should be placed over the left most grid lines and the other lines placed from left to right (see Figure 5). The ordering of lines is very important. They should be placed from top to bottom and then from left to right until all 16 lines have been drawn on the screen.

Figure 4: Horizontal Line Placement

Figure 5: Vertical Line Placement

5) Once this process has been completed for one image, the grid pattern should be moved 100mm away from the camera (keeping the camera perpendicular to the pattern) and the process repeated again. Figure 6 shows the last line placement for one of these captured images.

Figure 6: Final Line Placement

6) Repeat the process five times, moving the calibration pattern a total of 500mm away from the camera. After the fifth calibration step the program will automatically calculate the camera parameters. You will be prompted for a file name to store these parameters in:

point_num = 315 -------------------------------------SIZE = 720, 486 Distotion factor = 324.000000 253.000000 201.000000 372.97979 -24.50134 248.86941 0.00000 0.00000 327.03507 122.42421 0.00000 0.00000 0.00000 1.00000 0.00000 -------------------------------------Input filename: wideAngleKeyence.dat
Once the values are stored in a camera data file hit the right mouse button to quit. By changing the name of this camera data file to camera_para.dat and placing it in the bin/Data directory it can be used immediately in the ARToolKit sample programs. Calibrating your camera should give improved tracking results. The distance between the grid lines in the calib_cparam.pdf pattern is exactly 40mm, while the pattern has to be moved back 100mm for each measurement and the measurements have to be repeated five times. These values are all fixed in the source code calib_cparam_sub.c in the util/calib_cparam directory. If you want to change the distance between grid lines the following source code should be modified:

inter_coord[k][j][i+7][0] = 40.0*i; inter_coord[k][j][i+7][1] = 40.0*j; inter_coord[k][j][i+7][2] = 100.0*k;

where 40.0 is the current distance, and the 100.0 is the distance the pattern should be moved back from the camera each time. The number of measurements that need to be taken can be modified by changing the variable:

*loop_num = 5;

One Step Calibration

The one step operation is exactly the same of the calib_dist. You need to print the calib_dist.pdf and run calib_camera2:

> calib_cparam2 Input the length between each markers: 40 ----------Mouse Button Left : Grab image. Right : Quit. -----------

After that repeat the same instruction of calib_dist program.

Beginner

[Developers Guide]

Development Principles
There are two parts to developing applications that use ARToolKit; writing the application, and training image-processing routines on the real world markers that will be used in the application. Writing an application with ARToolKit is very easy: a simple outline is used for creating an AR application. We base on it for writing a new application. Similarly, training pattern phase is largely simplified with the use of simple tool. The following steps must be taken in your application main code: Initialization Main Loop 1. Initialize the video capture and read in the marker pattern files and camera parameters. 2. Grab a video input frame. 3. Detect the markers and recognized patterns in the video input frame. 4. Calculate the camera transformation relative to the detected patterns.

5. Draw the virtual objects on the detected patterns.

Shutdown

6. Close the video capture down.

Steps 2 through 5 are repeated continuously until the application quits, while steps 1 and 6 are just performed on initialization and shutdown of the application respectively. In addition to these steps the application may need to respond to mouse, keyboard or other application specific events. The next page will describe you in details this different steps, and the following how training patterns (and deal with more than one pattern).

Developing your First Application, Part 1

Introduction
To demonstrate in detail how to develop an ARToolKit application, we will step through the source code for an existing example program: simpleTest (or simple according to your ARToolKit version). This program is found in the directory examples/simple/.

The SimpleTest Program

The file we will be looking at is simpleTest.c (or simple.c). This program simply consists of a main routine and several graphics drawing routines. The functions which correspond to the six application steps previously described (see here) are shown in Table 1. The functions corresponding to steps 2 through 5 are called within the mainLoop function. ARToolKit Step 1. Initialize the application 2. Grab a video input frame 3. Detect the markers 4. Calculate camera transformation 5. Draw the virtual objects 6. Close the video capture down Functions

init arVideoGetImage (called in mainLoop) arDetectMarker (called in mainLoop) arGetTransMat (called in mainLoop) draw (called in mainLoop) cleanup draw and cleanup. In

Table 1: Function calls and code that corresponds to the ARToolKit applications steps.

The most important functions in the program are main, init, mainLoop, the remainder of this section we will explain these function calls in detail.

main

The main routine of simple is shown below:

main(int argc, char *argv[]) { init(); arVideoCapStart(); argMainLoop( NULL, keyEvent, mainLoop );

}
The init initialization routine contains code for starting the video capture, reading in the marker and camera parameters, and setup of the graphics window. This corresponds to step 1 in the development method. Next, we enter in the real-time state, with a call to the video starting function arVideoCapStart. Following, the argMainLoop function is called which starts the main program

keyEvent with any keyboard events and mainLoop with the main graphics rendering loop. The definition of argMainLoop is contained in the file gsub.c.
loop and associates the function

init
The init routine is called from the main routine and is used to initialize the video capture and read in the initial ARToolKit application parameters. At first, the video path is opened and the video image size found:

/* open the video path */

if( arVideoOpen( vconf ) < 0 ) exit(0);

/* find the size of the window */

if( arVideoInqSize(&xsize, &ysize) < 0 ) exit(0); printf("Image size (x,y) = (%d,%d)\n", xsize, ysize);
The variable vconf contains the initial video configuration and is defined at the top of simple.c. In function of your platform the content can be really different: see video configuration page. For each platform a default string is defined, that generally open the first available video stream in your architecture. After, we need to initialize the ARToolKit application parameters. The key parameters for an ARToolKit application are: the patterns that will be used for the pattern template matching and the virtual objects these patterns correspond to. the camera characteristics of the video camera being used.

These are both read in from file names that can either be specified on the command line or by using default hard-coded file names. So, the camera parameters are read in with the default camera parameter file name Data/camera_para.dat:

/* set the initial camera parameters */

if( arParamLoad(cparaname, 1, &wparam) < 0 ) { printf("Camera parameter load error !!\n"); exit(0); }
Next, the parameters are transformed for the current image size, because camera parameters change depending on the image size, even if same camera is used.

arParamChangeSize( &wparam, xsize, ysize, &cparam );

The camera parameters are set to those read in, the camera parameters printed to the screen:

arInitCparam( &cparam ); printf("*** Camera Parameter ***\n"); arParamDisp( &cparam );

After that we read the pattern definition with the default pattern file

Data/patt.hiro:

if( (patt_id=arLoadPatt(patt_name)) < 0 ) { printf("pattern load error !!\n"); exit(0); } patt_id is an identified pattern identification.
Finally a graphics window is opened:

/* open the graphics window */

argInit( &cparam, 1.0, 0, 0, 0, 0 );
The second argument of argInit defined a zoom function, set to 1.0 for matching video image format, set to 2.0 for double the size (e.g have a VGA AR display with a 320x240image input).

mainLoop
This is the routine where the bulk of the ARToolKit function calls are made and it contains code corresponding to steps 2 through 5 of the required application steps. First a video frame is captured using the function

arVideoGetImage:

/* grab a video frame */

if( (dataPtr = (ARUint8 *)arVideoGetImage()) == NULL ) { arUtilSleep(2); return; }
The video image is then displayed on screen. This can either be an unwarped image, or an image warped to correct for camera distortions. Warping the image produces a more normal image, but can result in a significant reduction in video frame rate. In this example the image is non-unwarped:

argDrawMode2D(); argDispImage( dataPtr, 0,0 );

Then the function arDetectMarker is used to search the video image is searched for squares that have the correct marker patterns:

if( arDetectMarker(dataPtr, thresh, &marker_info, &marker_num) < 0 ) { cleanup(); exit(0); });

The number of markers found is contained in the variable marker_num, while marker_info is a pointer to a list of marker structures containing the coordinate information and recognition confidence values and object id numbers for each of the markers. The marker_info structure is detailled in the API documentation. At this moment, The video image have been displayed and analyzed. So we don't need to use it more: we can ask the frame grabber to start a new grabbing in the same time we operates new functions. For this you have just to call arVideoCapNext:

arVideoCapNext();
Remark: when you called this function, using the last video image buffer can produce a bad result (dependant of your plaform). Be sure that you finish to deal with the video image buffer. Next, all the confidence values of the detected markers are compared to associate the correct marker id number with the highest confidence value:

/* check for object visibility */

k = -1; for( j = 0; j < marker_num; j++ ) { if( patt_id == marker_info[j].id ) { if( k == -1 ) k = j; else if( marker_info[k].cf < marker_info[j].cf ) k = j; } } if( k == -1 ) { argSwapBuffers(); return; }
The transformation between the marker cards and camera can then be found by using the arGetTransMat function:

/* get the transformation between the marker and the real camera */
arGetTransMat(&marker_info[k], patt_center, patt_width, patt_trans);
The real camera position and orientation relative to the marker object i are contained in the 3x4 matrix, patt_trans. Finally, the virtual objects can be drawn on the card using the draw function:

draw(); argSwapBuffers();
Remark: a simple optimization step is done if no pattern are found ( k==-1), we can directly swap the buffer without a call to

draw and return:

if( k == -1 ) { argSwapBuffers(); return; }

draw
The draw functions is divided in initialize rendering, setup matrix, render object. You initialize a 3D rendering by ask ARToolKit to do rendering of 3D object and setup minimal OpenGL state:

argDrawMode3D(); argDraw3dCamera( 0, 0 ); glClearDepth( 1.0 ); glClear(GL_DEPTH_BUFFER_BIT); glEnable(GL_DEPTH_TEST); glDepthFunc(GL_LEQUAL);

You need after that to convert the computed transformation (3x4 matrix) to an openGL format (array of 16 values), using the function call argConvGlpara. These sixteen values are the position and orientation values of the real camera, so using them to set the position of the virtual camera causes any graphical objects to be drawn to appear exactly aligned with the corresponding physical marker.

/* load the camera transformation matrix */

argConvGlpara(patt_trans, gl_para); glMatrixMode(GL_MODELVIEW); glLoadMatrixd( gl_para );
The virtual camera position is set using the OpenGL function glLoadMatrixd(gl_para).The last part of the code is the rendering of 3D object, in this example a cube with a blue color under a white color light:

glEnable(GL_LIGHTING); glEnable(GL_LIGHT0); glLightfv(GL_LIGHT0, GL_POSITION, light_position); glLightfv(GL_LIGHT0, GL_AMBIENT, ambi); glLightfv(GL_LIGHT0, GL_DIFFUSE, lightZeroColor); glMaterialfv(GL_FRONT, GL_SPECULAR, mat_flash); glMaterialfv(GL_FRONT, GL_SHININESS, mat_flash_shiny); glMaterialfv(GL_FRONT, GL_AMBIENT, mat_ambient); glMatrixMode(GL_MODELVIEW); glTranslatef( 0.0, 0.0, 25.0 ); glutSolidCube(50.0);

And to finish, you reset to default some OpenGL variables:

glDisable( GL_LIGHTING ); glDisable( GL_DEPTH_TEST );

The steps mentioned above occur every time through the main rendering loop. While the program is running mouse events are handled by the mouseEvent function and keyboard events by the keyEvent function.

cleanup
The cleanup function is called to stop the video processing and close down the video path to free it up for other applications:

arVideoCapStop(); arVideoClose(); argCleanup();

Developing your First Application, Part 2

Using Other Patterns
The simpleTest program uses template matching to recognize the Hiro pattern inside the marker squares. Squares in the video input stream are matched against pre-trained patterns. These patterns are loaded at run time and are contained in the Data directory of the bin directory. In this directory, we find for example patt.hiro used in your last application. This file contain the pattern format, simply a sample image. In order to change the pattern recognized in simpleTest you need to modify your code and create a new template file. You can modify the simpleTest.c file by changing the loaded filename:

char *patt_name = "Data/patt.hiro";

with

char *patt_name = "Data/patt.yourpatt"; mk_patt and is contained in the bin directory. The source code for mk_patt is in the mk_patt.c file in the util directory.
The program to create your new template file is called To create a new template pattern, first print out the file blankPatt.gif found in the patterns directory. This is just a black square with an empty white square in the middle. Then create a black and white or color image of the desired pattern that fits in the middle of this square and print it out.

The best patterns are those that are asymmetric and do not have fine detail on them. Figure 1 shows some possible sample patterns. Attach the new pattern to the center of the blank square.

Figure 1: Sample Patterns.

Once the new pattern has been made, change to the bin directory and run the mk_patt program (in console mode only). You will be prompted to enter a camera parameter filename. Enter the filename camera_para.dat. This is the default camera parameter file.

ARToolKit2.32/bin/mk_patt Enter camera parameter filename: camera_para.dat

The program will then open up a video window as shown in Figure 2.

Figure 2: mk_patt Video Window

Place the pattern to be trained on a flat surface in similar lighting conditions as will exist when the recognition application will be running. Then hold the video camera above the pattern, pointing directly down at the pattern, and turn it until a red and green square appears around the pattern (figure 7). This indicates that the mk_patt software has found the square around the test pattern. The camera should be rotated until the red corner of the highlighted square is the top left hand corner of the square in the video image, as shown in Figure 7. Once the square has been found and

oriented correctly hit the left mouse button. You will then be prompted for a pattern filename. Enter for example patt.yourpatt. Once a filename has been entered a bitmap image of the pattern is created and copied into this file. This is then used for the ARToolKit pattern matching. In order to use the new pattern files they need to be copied to the bin/Data directory. You can recompile simpleTest and now use your own pattern! Once one pattern has been trained, others can be trained simply by pointing the camera at new patterns and repeating the process, or the right mouse button can be hit to quit the application.

Using Multiple Patterns

We want now use more than one pattern, with multiple patterns tracked associate with different 3D object. For that we will step through the source code for the simplem program in the directory

examples/simplem/). You will find two sources, simplemTest.c and object.c This program
can detect multiple markers and display a different object shape on each marker (cone, cube, sphere). The main differences with the

simple program are:

loading of a file with the declaration of multiple pattern. a new structure associate to the patterns that imply a different checking code and transformation call in your program. a redefinition of the syntax and the definition of the draw function. And the rest of the code keep the same! For loading multiple patterns in ARToolKit a specific function is proposed on

object.c name

read_ObjData. With this function the loading of the marker is now done with: if( (object=read_ObjData(model_name, &objectnum)) == NULL ) exit(0); printf("Objectfile num = %d\n", objectnum); object is a pointer to an ObjectData_T structure, a specific structure to manage a list of patterns. the model_name defined now not a pattern definition filename (here with the value Data/object_data), but a specific multiple pattern definition filename (WARNING: this format is different of a multiple pattern tracking filename !!!). The text file object_data specifies which
marker objects are to be recognized and the patterns associated with each object. The object_data file begins with the number of objects to be specified and then a text data structure for each object. Each of the markers in the object_data file are specified by the following structure: Name Pattern Recognition File Name Width of tracking marker Center of tracking marker

For example the structure corresponding to the marker with the virtual cube is:

#pattern 1 cone Data/patt.hiro

80.0 0.0 0.0

Note that lines beginning with a # character are comment lines and are ignored by the file reader. ARToolKit can now try to identify multiple patterns in the arDetectMarker routine. Since we now detect multiple markers, we need to maintain a visibility state for each object and modify the check step for known patterns. Furthermore, we need also to maintain specific translation for each detected markers.

/* check for object visibility */

for( i = 0; i < objectnum; i++ ) { k = -1; for( j = 0; j < marker_num; j++ ) { if( object[i].id == marker_info[j].id ) { if( k == -1 ) k = j; else if( marker_info[k].cf < marker_info[j].cf ) k = j; } } if( k == -1 ) { object[i].visible = 0; continue; } object[i].visible = 1; arGetTransMat(&marker_info[k], object[i].marker_center, object[i].marker_width, object[i].trans); }
So you have a visiblity flag associated with each marker, and a new transformation if the marker have been detected. For drawing your object you now call the draw function with the ObjectData_T structure in parameter and the number of objects:

/* draw the AR graphics */

draw( object, objectnum );
The draw function remains simple to understand: Traverse the list of object, if is visible use it position and draw it with the associated shape. You can compile simplem and be sure that all necessary files are on the displayed figure 3.

Data directory. The result is

Figure 3: simplem Video Window

You can modify the

object_data file and work with your own patterns!

ARToolKit Framework
Introduction
ARToolKit is a software ToolKit like GLUT. It furnishes predefined functions that you need to call in a specific order for developing an AR program . But you can also use different parts of the ToolKit seperately. ARToolKit supports multiple platforms, while attempting to minimise librairy dependencies without sacrificing efficiency. ARToolKit uses OpenGL for the rendering part, GLUT for the windows/event handler aspect and hardware-dependent video library and standard API on each platform (e.g. win32 in Windows). The API is in C and a lot of samples provide good starting points for creating new applications (skeleton framework). Figure 1 summarises the relationship between your Application, ARToolKit and dependent librairies.

Figure 1: ARToolKit Architecture.

Structure
This section provides a better description of the main elements of ARToolKit. The ARToolKit library consists of four modules: AR module: core module with marker tracking routines, calibration and parameter collection. Video module: a collection of video routines for capturing the video input frames. This is a wrapper around the standard platform SDK video capture routines. Gsub module: a collection of graphic routines based on the OpenGL and GLUT libraries. Gsub_Lite module: replaces GSub with a more efficient collection of graphics routines, independent of any particular windowing toolkit.

The next figures show the hierarchical structure of ARToolKit and relation with dependencies libraries.

Figure 2a: Hierarchical structure of ARToolKit using Gsub Module

Figure 2b: Hierarchical structure of ARToolKit using Gsub_Lite Module

For further requirements we advise you to use gsub lite, as gsub is now deprecated. The modules respect a global pipeline metaphor (video->tracking->display), so the user can easily replace any module with another (like gsub with Open Inventor renderer).

Figure 3: Main ARToolKit pipeline

Data Types
ARToolKit manipulates a lot of different kinds of variable. Internally, it uses global variables, that restrict re-entrant part of the code. Otherwise, standard multi-argument interface are used based on a data-flow approach. ARToolKit uses different image formats between different modules. Figure 4 summarises all the different formats supported. Some formats are only available on certain platforms or with certain hardware.

Figure 4: ARToolKit data-flow.

Information about the detected markers is contained in the in the include directory.

ARMarkerInfo structure defined in ar.h

Augmented Reality Functions

All ARToolKit functions generally begin with the prefix (arVideo for video, arg for graphics, reference documentation.

ar, and a specific prefix for each module

ar for main core). You will find more information in the API

Beginner
Tutorial 1 : Tracking Stability
Introduction

[Tutorials]

We have seen before how to create a simple ARToolKit program. Now we want introduce an important feature of ARToolKit: the history function. Go to the bin directory and run the simpleTest2 program. You will see something similar to the screenshot in Figure 1.

Figure 1: The simpleTest2 Program

You will notice no immediate difference with the simpleTest program. However, if you move the pattern close to the camera (as in Figure 2) and press 'c', a difference appears. In one case the cube seems very stable and in the other case it seems to jitter around slightly. In the former case we are using a history function, in the latter not.

Figure 2: A closer view reveals jittering when the history function is not used.

In this tutorial introduce how to use the history functions of ARToolKit.

Using the History Functions

Open the simpleTest2.c program in examples/simple2. In the mainLoop function you will be find this new function call:

/* get the transformation between the marker and the real camera */
if( mode == 0 || contF == 0 ) { arGetTransMat(&marker_info[k], patt_center, patt_width, patt_trans); } else { arGetTransMatCont(&marker_info[k], patt_trans, patt_center, patt_width, patt_trans); } contF = 1;
The arGetTransMatCont function uses information from the previous image frame to reduce the jittering of the marker. With arGetTransMat, only the information from the current image frame is used to compute the position of the marker. When using the history function, the result will be less accurate because the history information increases performance at the expense of accuracy. ARToolKit provides another function based on history, but for the detection phase. We have already introduced this function, it is arDetectMarker. The corresponding function that does not use a history if arDetectMarkerLite. As before, using a history will reduce the accuracy, but offer more stability and is a little faster. Replace the arDetectMarker call with this one:

/* detect the markers in the video frame */

if( arDetectMarkerLite(dataPtr, thresh, &marker_info, &marker_num) < 0 ) { cleanup();

exit(0); }
Recompile simpleTest2 and position the marker so that it faces the camera (as in Figure 3). You will notice than the cube does more and more "popping".

Figure 3: A closer view reveals jittering when the history function is not used.

The popping effect is due to the lost information from the last frame, and the bad orientation of the marker is not sufficient for the detection step. Even if template matching was not done successfully, if there is a tracked marker with almost same size and same position in previous frame, the marker is regarded as a same marker of previous frame.

Tutorial 2 : Camera and Marker Relationships

Introduction
ARToolKit gives the position of the marker in the camera coordinate system, and uses OpenGL matrix system for the position of the virtual object. We will describe more precisely these different elements and their relationship in this tutorial.

Coordinate Systems
We will begin with the

simpleTest program. Open it and add this line after arGetTransMat:

printf("%f %f %f\n",patt_trans[0][3],patt_trans[1][3],patt_trans[2][3]);
Recompile and start the program. Look at the output values. If you move the marker to the left, the first value increases, move to the top the second value decreases and move to the front the last value increases also. This is the Coordinate System (CS) used by ARToolKit:

Figure 1: ARToolKit Coordinate Systems (Camera and Marker)

The output values correspond to these coordinate systems. The marker CS has the same orientation of the OpenGL CS, so any transformation applied to the object associated with the marker needs to respect OpenGL transformation principles. For example, if you don't want to display the cube at the center of the marker CS, but sitting on the top you would need to replace:

glTranslatef(0.0,0.0,25.0);
with

glTranslatef(0.0,20.0,25.0);
And you will be obtain:

Figure 2: simpleTest with a translation of virtual object

It's also more intuitive to consider the marker static and the real camera moving. That is to say, we want to obtain the position of the camera in the marker's coordinate system. Go to the bin directory and run exview. This program will display the view shown in Figure 3, and show text output of the camera position in the marker CS. Another view presents a 3D model of the camera and the marker.

Figure 3: exview screenshot. in the top image, you can see the marker CS (Blue Z, Green Y, Red X). The lower picture shows the camera position in the marker coordinate system.

You can move your mouse in the 3D external view and manipulate it to feel this relation (other examples in Figure 4).

Figure 4: Different position of the camera.

Open exview.c from the examples/exview directory. Look at the method getResultRaw:

if( arGetTransMat(marker_info, target_center, target_width, target_trans) < 0 ) return; if( arUtilMatInv(target_trans, cam_trans) < 0 ) return; sprintf(string," RAW: Cam Pos x: %3.1f y: %3.1f z: %3.1f", cam_trans[0][3], cam_trans[1][3], cam_trans[2][3]);
You will see the same call of arGetTransMat and a call of arUtilMatInv that give you the inverted position. The relationship between Marker and Camera is important, and switching between them gives the possibility to work with multiple coordinate systems. Based on these coordinate system, you can also obtain the transformation between the different coordinate systems. Open relationTest.c in examples/relation directory. Like simplem you can see multiple objects. The main difference is this added code:

if( object[0].visible >= 0 && object[1].visible >= 0 ) { double wmat1[3][4], wmat2[3][4]; arUtilMatInv(object[0].trans, wmat1); arUtilMatMul(wmat1, object[1].trans, wmat2);

for( j = 0; j < 3; j++ ) { for( i = 0; i < 4; i++ ) printf("%8.4f ", wmat2[j][i]); printf("\n"); } printf("\n\n"); }
This code computes the relative transformation between two markers. transformation of marker 1 in the camera CS.

object[0].trans is the

object[1].trans is the transformation of the marker 2 in the camera CS. So the inversion of object[0].trans multiplied by object[1].trans will give you the transformation of marker 2 in marker 1 CS. Run the program
and put the markers in these position:

Figure 5: relationTest (rendering view and console output).

The last display column gives the relative transformation of marker 2 in marker 1 CS. The cone is on the left of the sphere (142mm in X direction), above marker 1 (27 mm on Y direction), and they are nearly in the same plane (7mm on Z direction). Try to move marker 1 and marker 2 and observe the output values to understand these relationships. Try to do the reverse operation by modifying the code of relationTest (relative transformation of marker 1 in marker 2 CS). In the next tutorial we will see another kind of transformation between markers by "associating" multiple markers in the same CS.

 Tutorial 3 : Multi-Marker Tracking

Introduction
ARToolKit can give you the position of multiple markers as a function of the camera CS. You can also have a set of markers (e.g glue to a cardboard plane) that define only one position. ARToolKit can do that with a specific set of functions based on the multiMarker module.

Using Multiple Markers

Print the

pattMulti.pdf pattern shown in Figure 1.

Figure 1: multiMarker Pattern

Run multiTest from the bin directory.

Figure 2: multiTest: multiMarker tracking (blue cube correspond to detected markers, red cube to not detected markers).

Try to turn the pattern in different directions and observe the color of the individaul cubes. You will notice that red cubes are displayed where a marker is occluded. How is this possible? We use the multiMarker tracking principle: a set of markers are defined based on their relative positions. When at least one marker is visible we can compute the position of the marker set in the camera CS. We now look at the code of multiTest to understand how to program this feature. Open multiTest.c from the examples/multi directory.

if( (config = arMultiReadConfigFile(config_name)) == NULL ) { printf("config data load error !!\n"); exit(0); } config (a ARMultiMarkerInfoT structure) and config_name (a specific file, here: Data/multi/marker.dat). The MultiMarker Configuration File contains the list of all patterns and
We now have their exact positions within the fixed CS (this can be with respect to the corner of one marker, the middle of the plane of the set of markers, or an arbitrary position).

Figure 3: multiMarker configuration file.

The detection step remains the same, but the computing of the transformation is different:

if( (err=arMultiGetTransMat(marker_info, marker_num, config)) < 0 ) { argSwapBuffers(); return; }

The arMultiGetTransMat function doesn't give the position of each marker in the camera CS, but the position of the fixed CS in the camera CS, and the relative position of each marker in the fixed CS. You can employ the resulting transformations to directly display object on any part of the set (for a flat sheet, in any part of that sheet). In the multiTest example, the draw function used the position of the fixed CS ( config->trans), and also the position of marker (config->marker[i].trans) to display the cube.

for( i = 0; i < config->marker_num; i++ ) { if( config->marker[i].visible >= 0 ) draw( config->trans, config->marker[i].trans, 0 ); else draw( config->trans, config->marker[i].trans, 1 ); }
You can easily combine this multiMarker tracking with distinct marker tracking to realize more complex applications.

Intermediate
proximited based (soon)

[Detailed Tutorials]

Local Interaction : range based and

Global Interaction : multimarker with paddle (soon)

Working with gsubLite (soon)

Coordinate Systems
ARToolKit defines different coordinate systems mainly used by the computer vision algorithm and the rendering. You need to precisely understand relations between them to prevent reverse image or bad position object display.

Computer Vision Coordinate System

This sketch designed by Kato summarises the main coordinate systems used in ARToolKit:

You need to notice that arGetTransMat give the position of the marker in the camera coordinate system (not the reverse). If you want the position of the camera in the marker coordinate system you need to inverse this transformation (arMatrixInverse()).

Rendering Coordinate System

When you use ARToolKit with openGL, remember that openGL is a right-handed coordinate system, with Z coming to you (i.e. the camera is facing in the direciton of -Z). So they can easily coincide together.

OpenGL Programming Guide. Addison-Wesley. Copyright.

ARToolKit uses a calibrated camera perspective that typically results in an off-axis projection matrix for OpenGL. Such a projection can not be created by calling gluPerspective, but requires the extra parameters of glFrustum. Rather than decomposing ARToolKit's projection into parameters to pass to glFrustum, we instead directly load the OpenGL projection matrix by setting glMatrixMode(GL_PROJECTION_MATRIX); and calling glLoadMatrix If you use gsub_lite[apidoc] for the rendering, this off-axis projection is retreived by calling the function arglCameraFrustum[apidoc].

You will find more information on OpenGL coordinate systems in the OpenGL Programming Guide (The Red book)

 Hardware
Developping applications with ARToolKit is low cost. For example, a typical ARToolKit system can be: Item Pentium IV 2Ghz Hauppauge WinTV capture card Marshall Board CCD Camera Sony Glastron PLM-A35 VGA to NTSC converter Total Cost In this page we introduce some of the hardware usable with ARToolKit. Price $1000 $50 $250 $400 $100 ~ US$1950

Camera
The choice of a camera is the most important. Resolution, frame-rate, optics distorsion, are some of the main parameters for selected your camera. But supported platforms, parameters of the driver (like auto-contrast) can also influence your choice. Some cameras enable by default auto-contrast that decrease your performance, or offers 25Hz frame-rate with a lost of image quality (distorsion, latency lag). The most efficient solution keep a video capture card with a PAL or NTSC camera that offers a large choice and a really good quality camera (but a PAL camera can be really expensive according its characteristics). The signal deliver is RGB image (color palette) that reduce hardware or software palette conversion cost. The traditional choice is USB or Firewire camera. You can notice that frame-rate or color palette (RGB, YUV, YUV compressed) depend mainly on the bandwidth of the technology. USB Camera used generally compressed format transmission like YUV:4:2:2 YUV:4:1:1 (lossy compression). Firewire Cameras offers better solution, but camera with full RGB color palette are generally expensive (a compressed format in VGA remains a good choice). USB1.1 IEEE1394a max 1.5 MByte/s most low cost solution. 60 Mbyte/s good solution with a standardized camera protocol.

PCI32Bit1.0 (33Mhz) 125.89 MByte/s PCI64Bit2.0 (33Mhz) 251.77 MByte/s USB2.0 IEEE1394b max 50 MByte/s badly supported on Linux. 100 Mbyte/s few cameras support this protocol.

PCI32Bit2.1 (66Mhz) 251.77 MByte/s PCI64Bit2.1 (66Mhz) 503.54 MByte/s And the bandwidth of a video image in function of the palette format, framerate, and image size (SIF is 320x240):

SIF RGB 15 fps SIF RGB 30 fps SIF YUV 4:1:1 15 fps SIF YUV 4:1:1 30 fps VGA RGB 15 fps VGA RGB 30 fps VGA YUV 4:1:1 15 fps VGA YUV 4:1:1 30 fps

27 MBit/s (3.37 MByte/s) 55 MBit/s (6.87 MByte/s) 13 MBit/s (1.62 MByte/s) 27 MBit/s (3.37 MByte/s) 106 MBit/s (13.25 MByte/s) 221 MBit/s (26.37 MByte/s) 53 MBit/s (6.63 MByte/s) 106 MBit/s (13.25 MByte/s)

New USB 2.0 or IEEE1394b cameras offer a good future choice for developing ARToolKit applications. Some people have also experimented with wireless camera image transmission. (lot of measurements come from this webpage )

HMD
ARToolKit uses computer vision techniques for image-based recognition and tracking. Since it uses only a single camera, a self-contained head tracking system can be developed if this camera is fixed to a head mounted display (HMD). In this way AR applications can be developed which use head mounted displays. Figure 1 shows two different HMDs with cameras attached, the Virtual i-O i-glasses and the Olympus EyeTrek displays. It is not necessary to have a head mounted display to use the ARToolKit, a camera connected to a computer is the only requirement. Without an HMD ARToolKit applications can be viewed on a computer monitor, with an HMD a more immersive experience can be created.

Figure 1a: Virtual i-O i-glasses with camera

Figure 1b: Olympus EyeTrek with camera

The ARToolKit programs shown so far have used video see-through augmented reality where the computer graphics are overlaid on video of the real world. Both the i-glasses and EyeTrek support NTSC video input, so a simple way to achieve video see-through AR is to connect the head mounted camera to the computer running the ARToolKit application, display the AR application on-screen and then use a VGA-NTSC converter to convert the monitor output to a signal that can be displayed back in the HMDs. The SGI O2 computer has NTSC input and output so these HMDs can be connected directly to the computer without the need for a converter.

ARToolKit also supports optical see-through augmented reality. In this case an optical see-through HMD is used and only the virtual images are shown in the HMD. Since the HMD is see-through the effect is that the virtual images are overlaid directly on the real world. The i-glasses are semitransparent and so can be used for optical see-through AR, however the EyeTrek displays are fully occlusive. There are advantages and disadvantages of optical versus video see-through AR. Optical see-through AR allows you to see virtual objects stereoscopically, rather than with bioccular viewing as with a single camera video see-through system. The real world is also perceived at the resolution of the eyes rather than the resolution of the display. The main drawback with optical see-through AR is the lag in the system. In a video see-through application the video frame of the real world that is shown in the HMD is the same video frame that is used to calculate the head position and orientation so the virtual image will appear exactly overlaid on the real world scene. This is achieved by not showing the video frame until the image processing is completed. However in an optical see-through application the view of the real world cannot be delayed, so the delay introduced into the system by the graphics and image processing is perceived by the user. This results in virtual images that may not appear attached to the real objects they are supposed to be registered with, or that "swim" around. These lag effects are particularly noticeable with rapid head motions or moving the object the virtual image is supposed to be attached to. For a full discussion of the advantages and disadvantages of optical and video see-through AR, please see Ron Azuma's Siggraph 95 course notes on Augmented Reality. To finish, we provide you with a short list of HMD solutions. A lot of interesting low-cost HMD product have been discontinued these last years, the market being too small to sell enough of this technology (like the famous Sony Glasstron). You can find a lot of second-hand products at VR resellers or on specific forums. Optical See-Through HMD SAAB Advisor 150 MicroOptical SV Viewer

Video See-Through HMD Trivisio ARVision 3D MR Lab Coastar

Standard HMD (for build-it-yourself See-Through HMD) I-glasses SVGA Olympus Eye-Trek (discontinued) Daeyang Cy-Visor Article on budget-level HMD complete list in the stereo3d website Build your video/optical HMD: example at UNC

Advanced
Computer Vision Algorithm
Introduction
ARToolKit is based on a basic corner detection approach with a fast pose estimation algorithm. We discuss more precisely of this algorithm in this page. In complement, we discuss on the accuracy of ARToolKit and the limitation of the actual camera calibration utility.

Principles
The next figures summarises the principle of the algorithm.

a. Original image

b. Thresholded image

c. Connected components

d. Contours

e. Extracted marker edges and corners

f. Fitted square

 Benchmarks

Performance
some value of measured accuracy of ARToolKit, jitter problems, CPU dependencies,etc... (from Kato or Billinghurst course notes).

Pattern Size Parameter

Figure 1: Tracking Error with Pattern Size

Range Parameter

Figure 2: Tracking Error with Range

Angle Parameter

Figure 3: Tracking Error with Angle

Camera Comparison
test with different cameras on different platforms.

Modify/Extend ARToolKit: brief design guidelines (soon)

Further Information
History
A complete review of history of ARToolKit since its creation. ARToolKit was developed in 1999 when Hirokazo Kato arrived at the HITLab. The first demonstration have been at SIGGRAPH 1999 for the shared space project. The support of Windows DirectShow Video have certainly help to a large scale use for a lot of exhibition. Large modification have been made from the last years (multiplatform, better tracking algorithm). We try to identify the main improvements of each versions in this page (intermediate version without official release are not described). The name below the version is mainly related to the people that deliver the distribution, since the original creators of ARToolKit remained H. Kato and M. Billinghurst. The bold versions are main official release.

 Feature List
A simple framework for creating real-time augmented reality applications A multiplatform library (Windows, Linux, Mac OS X, SGI) Overlays 3D virtual objects on real markers ( based on computer vision algorithm) A multi platform video library with: multiple input sources (USB, Firewire, capture card) supported multiple format (RGB/YUV420P, YUV) supported multiple camera tracking supported GUI initalizing interface A fast and cheap 6D marker tracking (real-time planar detection) An extensible markers patterns approach (number of markers fct of efficency) An easy calibration routine A simple graphic library (based on GLUT) A fast rendering based on OpenGL A 3D VRML support A simple and modular API (in C) Other language supported (JAVA, Matlab) A complete set of samples and utilities A good solution for tangible interaction metaphor OpenSource with GPL license for non-commercial usage

Road Map
Performance
This page resumes next steps on the ARToolKit library development. If you have an idea, a wish or a specific requirement, you can send an email at XXX@toto.com

Develpment Areas
General

Stereo tracking Radiometric support (lighting, shadow)

Tracking Video
Unify Linux Driver (with multilibrary Video Dialog) Multithread Linux Driver

Rendering Utilities
Radiometric utils

Samples Applications and Advanced Samples Package Distribution

Windows installer

Similar Projects
Real-Time Augmented Reality Project at Computational Video Group, Ottowa Matlab ARToolKit at Mixed Reality Lab, Singapore OSGAR: ARToolKit and OpenScene Graph at Augmented Environments Laboratory, Atlanta ARToolKit on Pocket PC at Interactive Media System Group, Vienna DART at Augmented Environments Laboratory, Atlanta

API
-a

[All]

Here is a list of all documented functions, variables, defines, enums, and typedefs with links to the documentation:

ar2VideoCapNext() : video.h ar2VideoCapStart() : video.h ar2VideoCapStop() : video.h ar2VideoClose() : video.h ar2VideoDispOption() : video.h ar2VideoGetImage() : video.h ar2VideoInqSize() : video.h ar2VideoOpen() : video.h AR_PIXEL_FORMAT : ar.h arActivatePatt() : ar.h arDeactivatePatt() : ar.h arDebug : ar.h arDetectMarker() : ar.h arDetectMarker2() : ar.h arDetectMarkerLite() : ar.h ARELEM0 : matrix.h ARELEM1 : matrix.h arFittingMode : ar.h arFreePatt() : ar.h argCleanup() : gsub.h argConvGLcpara() : gsub.h argConvGlpara() : gsub.h argDispHalfImage() : gsub.h argDispImage() : gsub.h argDraw2dLeft() : gsub.h argDraw2dRight() : gsub.h argDraw3dCamera() : gsub.h argDraw3dLeft() : gsub.h argDraw3dRight() : gsub.h argDrawMode : gsub.h argDrawMode2D() : gsub.h argDrawMode3D() : gsub.h argDrawSquare() : gsub.h arGetAngle() : ar.h arGetCode() : ar.h arGetContour() : ar.h arGetImgFeature() : ar.h arGetInitRot() : ar.h arGetLine() : ar.h arGetMarkerInfo() : ar.h arGetNewMatrix() : ar.h arGetPatt() : ar.h

arGetRot() : ar.h arGetTransMat() : ar.h arGetTransMatCont() : ar.h arGetVersion() : ar.h argInit() : gsub.h argInqSetting() : gsub.h ARGL_CONTEXT_SETTINGS_REF : gsub_lite.h arglCameraFrustum() : gsub_lite.h arglCameraFrustumRH() : gsub_lite.h arglCameraView() : gsub_lite.h arglCameraViewRH() : gsub_lite.h arglCleanup() : gsub_lite.h arglDispImage() : gsub_lite.h arglDispImageStateful() : gsub_lite.h arglDistortionCompensationGet() : gsub_lite.h arglDistortionCompensationSet() : gsub_lite.h arglDrawModeGet() : gsub_lite.h arglDrawModeSet() : gsub_lite.h argLineSeg() : gsub.h argLineSegHMD() : gsub.h argLoadHMDparam() : gsub.h arglPixelFormatGet() : gsub_lite.h arglPixelFormatSet() : gsub_lite.h arglSetupForCurrentContext() : gsub_lite.h arglTexmapModeGet() : gsub_lite.h arglTexmapModeSet() : gsub_lite.h arglTexRectangleGet() : gsub_lite.h arglTexRectangleSet() : gsub_lite.h argMainLoop() : gsub.h argSwapBuffers() : gsub.h argTexmapMode : gsub.h argUtilCalibHMD() : gsubUtil.h arImage : ar.h arImageProcMode : ar.h arImXsize : ar.h arImYsize : ar.h arInitCparam() : ar.h arLabeling() : ar.h arLabelingCleanup() : ar.h arLoadPatt() : ar.h arMalloc : ar.h arMatchingPCAMode : ar.h arMatrixAlloc() : matrix.h arMatrixAllocDup() : matrix.h arMatrixAllocInv() : matrix.h arMatrixAllocMul() : matrix.h arMatrixAllocTrans() : matrix.h arMatrixAllocUnit() : matrix.h arMatrixDet() : matrix.h arMatrixDisp() : matrix.h arMatrixDup() : matrix.h arMatrixFree() : matrix.h arMatrixInv() : matrix.h arMatrixMul() : matrix.h

arMatrixPCA() : matrix.h arMatrixPCA2() : matrix.h arMatrixSelfInv() : matrix.h arMatrixTrans() : matrix.h arMatrixUnit() : matrix.h arModifyMatrix() : ar.h arMultiActivate() : arMulti.h arMultiDeactivate() : arMulti.h arMultiFreeConfig() : arMulti.h arMultiGetTransMat() : arMulti.h arMultiReadConfigFile() : arMulti.h arParam : ar.h arParamChangeSize() : param.h arParamDecomp() : param.h arParamDecompMat() : param.h arParamDisp() : param.h arParamGet() : param.h arParamIdeal2Observ() : param.h arParamLoad() : param.h arParamObserv2Ideal() : param.h arParamSave() : param.h arSavePatt() : ar.h arTemplateMatchingMode : ar.h arUtilMat2QuatPos() : ar.h arUtilMatInv() : ar.h arUtilMatMul() : ar.h arUtilQuatPos2Mat() : ar.h arUtilSleep() : ar.h arUtilTimer() : ar.h arUtilTimerReset() : ar.h arVecAlloc() : matrix.h arVecDisp() : matrix.h arVecFree() : matrix.h arVecHousehold() : matrix.h arVecInnerproduct() : matrix.h arVecTridiagonalize() : matrix.h arVideoCapNext() : video.h arVideoCapStart() : video.h arVideoCapStop() : video.h arVideoClose() : video.h arVideoDispOption() : video.h arVideoGetImage() : video.h arVideoInqSize() : video.h arVideoOpen() : video.h

-t

TRUE : gsub_lite.h

API
ARToolKit Data Structures
Here are the data structures with brief descriptions:
ARMarkerInfo ARMarkerInfo2 ARMat ARMultiEachMarkerInfoT ARMultiMarkerInfoT ARParam arPrevInfo ARVec Main structure for detected marker

[Data Structures]

Internal structure use for marker detection Matrix structure Multi-marker structure Global multi-marker structure Camera intrinsic parameters Structure for temporal continuity of tracking Vector structure

ARMarkerInfo Struct Reference

main structure for detected marker. More...

#include <ar.h>

Data Fields
int int int double double double double area id dir cf pos [2] line [4][3] vertex [4][2]

Detailed Description
main structure for detected marker. Store information after contour detection (in idea screen coordinate, after distorsion compensated). Remarks: lines are represented by 3 values a,b,c for ax+by+c=0 Parameters:
area id dir
number of pixels in the labeled region marker identitied number Direction that tells about the rotation about the marker (possible values are 0, 1, 2 or 3). This parameter makes it possible to tell about the line order of the detected marker (so which line is the first one) and so find the first vertex. This is important to compute the transformation matrix in arGetTransMat(). confidence value (probability to be a marker) center of marker (in ideal screen coordinates) line equations for four side of the marker (in ideal screen coordinates)

cf pos line

vertex edge points of the marker (in ideal screen coordinates)

The documentation for this struct was generated from the following file:

ar.h

ARMarkerInfo2 Struct Reference

internal structure use for marker detection. More...

#include <ar.h>

Data Fields
int double int int int area pos [2] coord_num x_coord [AR_CHAIN_MAX] y_coord [AR_CHAIN_MAX]

int

vertex [5]

Detailed Description
internal structure use for marker detection. Store information after contour detection (in observed screen coordinate, before distorsion correction). Parameters:
area pos x_coord y_coord vertex
number of pixels in the labeled region position of the center of the marker (in observed screen coordinates)

coord_num numer of pixels in the contour.

x coordinate of the pixels of contours (size limited by AR_CHAIN_MAX). y coordinate of the pixels of contours (size limited by AR_CHAIN_MAX). position of the vertices of the marker. (in observed screen coordinates) rem:the first vertex is stored again as the 5th entry in the array for convenience of drawing a line-strip easier.

The documentation for this struct was generated from the following file:

ar.h

ARMat Struct Reference

matrix structure. More...

#include <matrix.h>

Data Fields
double * int int m row clm

Detailed Description

matrix structure. Defined the structure of the matrix type based on a dynamic allocation. The matrix format is : <---- clm ---> [ 10 20 30 ] ^ [ 20 10 15 ] | [ 12 23 13 ] row [ 20 10 15 ] | [ 13 14 15 ] v Parameters:
m
content of matrix

row number of lines in matrix clm number of column in matrix

The documentation for this struct was generated from the following file:

matrix.h

ARMultiEachMarkerInfoT Struct Reference

multi-marker structure More...

#include <arMulti.h>

Data Fields
int double double double double double int int patt_id width center [2] trans [3][4] itrans [3][4] pos3d [4][3] visible visibleR

Detailed Description
multi-marker structure Structure for multi-marker tracking really similar to ARMarkerInfo Parameters:
patt_id identification of the pattern width trans itrans pos3d visible
width of the pattern (in mm)

center center of the pattern (in mm)

estimated position of the pattern relative position of the pattern final position of the pattern boolean flag for visibility

visibleR last state visibility

The documentation for this struct was generated from the following file:

arMulti.h

ARMultiMarkerInfoT Struct Reference

global multi-marker structure More...

#include <arMulti.h>

Data Fields
ARMultiEachMarkerInfoT * int double int double marker marker_num trans [3][4] prevF transR [3][4]

Detailed Description

global multi-marker structure Main structure for multi-marker tracking. Parameters:

marker trans prevF transR
list of markers of the multi-marker pattern

marker_num number of markers used

position of the multi-marker pattern (more precisely, the camera position in the multi-marker CS) boolean flag for visibility last position

The documentation for this struct was generated from the following file:

arMulti.h

ARParam Struct Reference

camera intrinsic parameters. More...

#include <param.h>

Data Fields
int int double double xsize ysize mat [3][4] dist_factor [4]

Detailed Description
camera intrinsic parameters. This structure contains the main parameters for the intrinsic parameters of the camera representation. The camera used is a pinhole camera with standard parameters. User should consult a computer vision reference for more information. (e.g. ThreeDimensional Computer Vision (Artificial Intelligence) by Olivier Faugeras).

Parameters:
xsize ysize mat
length of the image (in pixels). height of the image (in pixels). perspective matrix (K). of distortion dist_factor[2]=distortion factor dist_factor[3]=scale factor

dist_factor radial distortions factor dist_factor[0]=x center of distortion dist_factor[1]=y center

The documentation for this struct was generated from the following file:

param.h

arPrevInfo Struct Reference

structure for temporal continuity of tracking More...

#include <ar.h>

Data Fields
ARMarkerInfo int marker count

Detailed Description
structure for temporal continuity of tracking History structure for arDetectMarkerLite and arGetTransMatCont

The documentation for this struct was generated from the following file:

ar.h

ARVec Struct Reference

vector structure. More...

#include <matrix.h>

Data Fields
double * int v clm

Detailed Description
vector structure. The vector format is : <---- clm ---> [ 10 20 30 ] Defined the structure of the vector type based on a dynamic allocation. Parameters:
m
content of vector

clm number of column in matrix

The documentation for this struct was generated from the following file:

matrix.h

API
ARToolKit File List
Here is a list of all documented files with brief descriptions:
ar.h arMulti.h gsub.h gsub_lite.h gsubUtil.h matrix.h param.h video.h ARToolKit subroutines ARToolkit multi pattern subroutines ARToolkit OpenGL subroutines A collection of useful OpenGL routines for ARToolKit ARToolkit OpenGL utilities subroutines ARToolkit algebric mathematics subroutines ARToolkit global structure (parameters) subroutines ARToolkit video subroutines

[Files]

ar.h File Reference

ARToolKit subroutines. More...

#include #include #include #include

<stdio.h> <malloc.h> <AR/config.h> <AR/param.h>

Data Structures
struct ARMarkerInfo main structure for detected marker. More... ARMarkerInfo2 internal structure use for marker detection. More... arPrevInfo structure for temporal continuity of tracking More...

struct

struct

Defines
#define arMalloc(V, T, S) allocation macro function

Typedefs
typedef char typedef short typedef int typedef unsigned char typedef unsigned short typedef unsigned int typedef int ARInt8 ARInt16 ARInt32 ARUint8 ARUint16 ARUint32 AR_PIXEL_FORMAT ARToolKit pixel-format specifiers.

Functions
ARUint32 arGetVersion (char **versionStringRef) Get the ARToolKit version information in numberic and string format. arInitCparam (ARParam *param) initialize camera parameters. arLoadPatt (const char *filename) load markers description from a file arDetectMarker (ARUint8 *dataPtr, int thresh, ARMarkerInfo **marker_info, int *marker_num) main function to detect the square markers in the video input frame. arDetectMarkerLite (ARUint8 *dataPtr, int thresh, ARMarkerInfo **marker_info, int *marker_num) main function to detect rapidly the square markers in the video input frame. arGetTransMat (ARMarkerInfo *marker_info, double center[2], double width, double conv[3][4]) compute camera position in function of detected markers. arGetTransMatCont (ARMarkerInfo *marker_info, double prev_conv[3][4], double center[2], double width, double conv[3][4]) compute camera position in function of detected marker with an history function. arGetTransMat2 (double rot[3][3], double pos2d[][2], double pos3d[][2], int num, double conv[3][4]) arGetTransMat3 (double rot[3][3], double ppos2d[][2], double ppos3d[][2], int num, double conv[3][4], double *dist_factor, double cpara[3][4]) arGetTransMat4 (double rot[3][3], double ppos2d[][2], double ppos3d[][3], int num, double conv[3][4]) arGetTransMat5 (double rot[3][3], double ppos2d[][2], double ppos3d[][3], int num, double conv[3][4], double *dist_factor, double cpara[3][4]) arFreePatt (int patt_no)

int

int

int

int

double

double

double double double double int

remove a pattern from memory. int arActivatePatt (int pat_no) activate a pattern on the recognition procedure. int arDeactivatePatt (int pat_no) desactivate a pattern on the recognition procedure. int arSavePatt (ARUint8 *image, ARMarkerInfo *marker_info, char *filename) save a marker. int arUtilMatInv (double s[3][4], double d[3][4]) Inverse a non-square matrix. int arUtilMatMul (double s1[3][4], double s2[3][4], double d[3][4]) Multiplication of two matrix. int arUtilMat2QuatPos (double m[3][4], double q[4], double p[3]) extract a quaternion/position of matrix. int arUtilQuatPos2Mat (double q[4], double p[3], double m[3][4]) create a matrix with a quaternion/position. double arUtilTimer (void) get the time with the ARToolkit timer. void arUtilTimerReset (void) reset the internal timer of ARToolkit. void arUtilSleep (int msec) sleep the actual thread. ARInt16 * arLabeling (ARUint8 *image, int thresh, int *label_num, int **area, double **pos, int **clip, int **label_ref) extract connected components from image. arLabelingCleanup (void) clean up static data allocated by arLabeling. arGetImgFeature (int *num, int **area, int **clip, double **pos) XXXBK. arDetectMarker2 (ARInt16 *limage, int label_num, int *label_ref, int *warea, double *wpos, int *wclip, int area_max, int area_min, double factor, int *marker_num) XXXBK. arGetMarkerInfo *marker_num) information on (ARUint8 *image, ARMarkerInfo2 *marker_info2, int

void

void

ARMarkerInfo2 *

ARMarkerInfo *

int

arGetCode (ARUint8 *image, int *x_coord, int *y_coord, int *vertex, int *code, int *dir, double *cf) XXXBK. arGetPatt (ARUint8 *image, int *x_coord, int *y_coord, int *vertex, ARUint8 ext_pat[AR_PATT_SIZE_Y][AR_PATT_SIZE_X][3]) Get a normalized pattern from a video image. arGetLine (int x_coord[], int y_coord[], int coord_num, int vertex[], double line[4][3], double v[4][2]) estimate a line from a list of point. arGetContour (ARInt16 *limage, ARMarkerInfo2 *marker_info2) int *label_ref, int label, int clip[4],

int

int

int

XXXBK. double arModifyMatrix (double rot[3][3], double trans[3], double cpara[3][4], double vertex[][3], double pos2d[][2], int num) XXXBK. arGetAngle (double rot[3][3], double *wa, double *wb, double *wc) extract euler angle from a rotation matrix. arGetRot (double a, double b, double c, double rot[3][3]) create a rotation matrix with euler angle. arGetNewMatrix (double a, double b, double c, double trans[3], double trans2[3][4], double cpara[3][4], double ret[3][4]) XXXBK. arGetInitRot (ARMarkerInfo *marker_info, double cpara[3][4], double rot[3][3]) XXXBK. arsInitCparam (ARSParam *sparam) arsGetImgFeature (int *num, int **area, int **clip, double **pos, int LorR) arsLabeling (ARUint8 *image, int thresh, int *label_num, int **area, double **pos, int **clip, int **label_ref, int LorR) arsGetLine (int x_coord[], int y_coord[], int coord_num, int vertex[], double line[4][3], double v[4][2], int LorR) arsGetMarkerInfo (ARUint8 *marker_num, int LorR) *image, ARMarkerInfo2 *marker_info2, int

int

int

int

int

int void ARInt16 * int ARMarkerInfo * int int double double

arsDetectMarker (ARUint8 *dataPtr, int thresh, ARMarkerInfo **marker_info, int *marker_num, int LorR) arsDetectMarkerLite (ARUint8 *dataPtr, int thresh, ARMarkerInfo **marker_info, int *marker_num, int LorR) arsGetTransMat (ARMarkerInfo *marker_infoL, ARMarkerInfo *marker_infoR, double center[2], double width, double transL[3][4], double transR[3][4]) arsGetTransMatCont (ARMarkerInfo *marker_infoL, ARMarkerInfo *marker_infoR, double prev_conv[3][4], double center[2], double width, double transL[3][4], double transR[3][4]) arsGetTransMat2 (double rot[3][3], double ppos2dL[][2], double ppos3dL[][3], int numL, double ppos2dR[][2], double ppos3dR[][3], int numR, double transL[3][4], double transR[3][4]) arsGetPosErr (double pos2dL[2], double pos2dR[2]) arsCheckPosition (double pos2dL[2], double pos2dR[2], double thresh) arsCheckMarkerPosition (ARMarkerInfo *marker_infoR, double thresh) *marker_infoL, ARMarkerInfo

double

double int int double

arsModifyMatrix (double rot[3][3], double trans[3], ARSParam *arsParam, double pos3dL[][3], double pos2dL[][2], int numL, double pos3dR[][3], double pos2dR[][2], int numR)

Variables
int arDebug activate artoolkit debug mode arImage internal image

ARUint8 *

int

arFittingMode fitting display mode use by ARToolkit. arImageProcMode define the image size mode for marker detection. arParam internal intrinsic camera parameter arImXsize internal image size in width. arImYsize internal image size in heigth arTemplateMatchingMode XXXBK. arMatchingPCAMode XXXBK. arImageL arImageR arsParam arsMatR2L [3][4]

int

ARParam

int

int

int

int

ARUint8 * ARUint8 * ARSParam double

Detailed Description
ARToolKit subroutines. Core of the ARToolKit Library. This file provides image analysis and marker detection routines. Differents routines give access to camera and marker configurations. Other routines manipulate marker info structures for deliver 3D transformation of markers (more precisely the position of the camera in function of the marker coordinate system). Remarks: History : Author: Hirokazu Kato kato@sys.im.hiroshima-cu.ac.jp Version: 3.1 Date: 01/12/07

Define Documentation
#define arMalloc( V, T, S )

Value:
{ if( ((V) = (T *)malloc( sizeof(T) * (S) )) == 0 ) \ {printf("malloc error!!\n"); exit(1);} }

allocation macro function allocate S elements of type T. Parameters:

V returned allocated area pointer T type of element S number of elements

Typedef Documentation
AR_PIXEL_FORMAT ARToolKit pixel-format specifiers. ARToolKit functions can accept pixel data in a variety of formats. This enumerations provides a set of constants you can use to request data in a particular pixel format from an ARToolKit function that returns data to you, or to specify that data you are providing to an ARToolKit function is in a particular pixel format. AR_PIXEL_FORMAT_RGB Each pixel is represented by 24 bits. Eight bits per each Red, Green, and Blue component. This is the native 24 bit format for the Mac platform. AR_PIXEL_FORMAT_BGR Each pixel is represented by 24 bits. Eight bits per each Blue, Red, and Green component. This is the native 24 bit format for the Win32 platform. AR_PIXEL_FORMAT_RGBA Each pixel is represented by 32 bits. Eight bits per each Red, Green, Blue, and Alpha component. AR_PIXEL_FORMAT_BGRA Each pixel is represented by 32 bits. Eight bits per each Blue, Green, Red, and Alpha component. This is the native 32 bit format for the Win32 platform. AR_PIXEL_FORMAT_ABGR Each pixel is represented by 32 bits. Eight bits per each Alpha, Blue, Green, and Red component. This is the native 32 bit format for the SGI platform. AR_PIXEL_FORMAT_ARGB Each pixel is represented by 32 bits. Eight bits per each Alpha, Red, Green, and Blue component. This is the native 32 bit format for the Mac platform. AR_PIXEL_FORMAT_MONO Each pixel is represented by 8 bits of luminance information. AR_PIXEL_FORMAT_2vuy 8-bit 4:2:2 Component Y'CbCr format. Each 16 bit pixel is represented by an unsigned eight bit luminance component and two unsigned eight bit chroma components. Each pair of pixels shares a common set of chroma values. The

components are ordered in memory; Cb, Y0, Cr, Y1. The luminance components have a range of [16, 235], while the chroma value has a range of [16, 240]. This is consistent with the CCIR601 spec. This format is fairly prevalent on both Mac and Win32 platforms. '2vuy' is the Apple QuickTime four-character code for this pixel format. The equivalent Microsoft fourCC is 'UYVY'. AR_PIXEL_FORMAT_yuvs 8-bit 4:2:2 Component Y'CbCr format. Identical to the AR_PIXEL_FORMAT_2vuy except each 16 bit word has been byte swapped. This results in a component ordering of; Y0, Cb, Y1, Cr. This is most prevalent yuv 4:2:2 format on both Mac and Win32 platforms. 'yuvs' is the Apple QuickTime four-character code for this pixel format. The equivalent Microsoft fourCC is 'YUY2'.

Function Documentation
int arActivatePatt( int pat_no ) activate a pattern on the recognition procedure. Activate a pattern to be check during the template matching operation. Parameters:

patt_no number of pattern to activate

Returns: return 1 in success, -1 if error int arDeactivatePatt( int pat_no ) desactivate a pattern on the recognition procedure. Desactivate a pattern for not be check during the template matching operation. Parameters:

patt_no number of pattern to desactivate

Returns: return 1 in success, -1 if error int ( ARUint8 * arDetectMark dataPtr, er int thresh, ARMarkerIn marker_info, fo ** int * marker_num ) main function to detect the square markers in the video input frame. This function proceeds to thresholding, labeling, contour extraction and line corner estimation

(and maintains an history). It's one of the main function of the detection routine with arGetTransMat. Parameters:

dataPtr

a pointer to the color image which is to be searched for square markers. The pixel format depend of your architecture. Generally ABGR, but the images are treated as a gray scale, so the order of BGR components does not matter. However the ordering of the alpha comp, A, is important. specifies the threshold value (between 0-255) to be used to convert the input thresh image into a binary image. marker_info a pointer to an array of ARMarkerInfo structures returned which contain all the information about the detected squares in the image

marker_num the number of detected markers in the image.

Returns: 0 when the function completes normally, -1 otherwise ARMarkerInfo2* arDetectMarker2 ( ARInt1 limage, 6* int label_num, int *

label_ref,

int * warea, doubl wpos, e* int * wclip, int

area_max,

int area_min, doubl factor, e int * marker_num ) XXXBK. XXXBK Parameters:

limage label_num label_ref warea wpos wclip area_max area_min factor

XXXBK XXXBK XXXBK XXXBK XXXBK XXXBK XXXBK XXXBK XXXBK

marker_num XXXBK
Returns: XXXBK XXXBK int ( ARUint8 * arDetectMarker Lite int ARMarkerI nfo ** int * )

dataPtr, thresh, marker_info, marker_num

main function to detect rapidly the square markers in the video input frame. this function is a simpler version of arDetectMarker that does not have the same error correction functions and so runs a little faster, but is more error prone Parameters:

dataPtr

a pointer to the color image which is to be searched for square markers. The pixel format depend of your architecture. Generally ABGR, but the images are treated as a gray scale, so the order of BGR components does not matter. However the ordering of the alpha component, A, is important. specifies the threshold value (between 0-255) to be used to convert the input thresh image into a binary image. marker_info a pointer to an array of ARMarkerInfo structures returned which contain all the information about the detected squares in the image

marker_num the number of detected markers in the image.

Returns: 0 when the function completes normally, -1 otherwise int arFreePatt( int patt_no ) remove a pattern from memory. desactivate a pattern and remove from memory. post-condition of this function is unavailability of the pattern. Parameters:

patt_no number of pattern to free

Returns: return 1 in success, -1 if error int ( double rot[3][3], arGetAngle double wa, * double wb, * double wc *

) extract euler angle from a rotation matrix. Based on a matrix rotation representation, furnish the cprresponding euler angles. Parameters:

rot the initial rotation matrix wa XXXBK:which element ? wb XXXBK:which element ? wc XXXBK:which element ?
Returns: XXXBK int ( ARUint image, arGetCode 8 * int * x_coord, int * int * int *

y_coord, vertex, code,

int * dir, double cf * ) XXXBK. XXXBK Parameters:

image

XXXBK

x_coord XXXBK y_coord XXXBK vertex code dir cf

Returns: XXXBK int ( ARInt16 * arGetConto ur int * XXXBK XXXBK XXXBK XXXBK

limage, label_ref,

int

label,

int clip[4], ARMarkerIn marker_info2 fo2 * ) XXXBK. XXXBK Parameters:

limage label_ref label clip

Returns: XXXBK void ( int * arGetImgFeatur e int **

XXXBK XXXBK XXXBK XXXBK

marker_info2 XXXBK

num, area,

int ** clip, double pos ** ) XXXBK. XXXBK Parameters:

num XXXBK area XXXBK clip XXXBK pos XXXBK

int ( ARMarkerI arGetInitR nfo * marker_info, ot double cpara[3][4], double )

rot[3][3]

XXXBK. XXXBK:initial of what ? Parameters:

marker_info XXXBK cpara rot

Returns: XXXBK int ( int arGetLine int int XXXBK XXXBK

x_coord[], y_coord[], coord_num,

int vertex[], doubl line[4][3], e doubl v[4][2] e ) estimate a line from a list of point. Compute a linear regression from a list of point. Parameters:

x_coord y_coord vertex line v

Returns: XXXBK ARMarkerInfo* arGetMarkerInfo

X coordinate of points Y coordinate of points

coord_num number of points

XXXBK XXXBK XXXBK

( ARUint8 *

image,

ARMarkerI marker_info2, nfo2 * int * marker_num ) information on

XXXBK Parameters:

image

XXXBK

marker_info2 XXXBK marker_num XXXBK

Returns: XXXBK int ( doubl arGetNewMatri e a, x doubl b, e doubl c, e doubl trans[3], e doubl trans2[3][4], e doubl cpara[3][4], e doubl ret[3][4] e ) XXXBK. XXXBK Parameters:

a b c

XXXBK XXXBK XXXBK

trans XXXBK trans2 XXXBK cpara XXXBK ret

Returns: XXXBK int ( ARUint image, arGetPatt 8 * int * x_coord, int * XXXBK

y_coord,

int * vertex, ARUint ext_pat[AR_PATT_SIZE_Y][AR_PATT_SIZE_X][3] 8

) Get a normalized pattern from a video image. This function returns a normalized pattern from a video image. The format is a table with AR_PATT_SIZE_X by AR_PATT_SIZE_Y Parameters:

image

video input image

x_coord XXXBK y_coord XXXBK vertex

Returns: XXXBK int ( doubl a, arGetRot e doubl b, e doubl c, e doubl rot[3][3] e ) create a rotation matrix with euler angle. Based on a euler description, furnish a rotation matrix. Parameters: XXXBK

ext_pat detected pattern.

a b c

XXXBK:which element ? XXXBK:which element ? XXXBK:which element ?

rot the resulted rotation matrix

Returns: XXXBK double ( ARMarkerI marker_info, arGetTransMat nfo * double center[2], double double )

width, conv[3][4]

compute camera position in function of detected markers. calculate the transformation between a detected marker and the real camera, i.e. the position and orientation of the camera relative to the tracking mark. Parameters:

marker_info the structure containing the parameters for the marker for which the camera center

position and orientation is to be found relative to. This structure is found using arDetectMarker. the physical center of the marker. arGetTransMat assumes that the marker is in x-y plane, and z axis is pointing downwards from marker plane. So vertex positions can be represented in 2D coordinates by ignoring the z axis information. The marker vertices are specified in order of clockwise. the size of the marker (in mm). the transformation matrix from the marker coordinates to camera coordinate frame, that is the relative position of real camera to the real marker

width conv
Returns: always 0.

double ( ARMarkerI arGetTransMatCo nfo * marker_info, nt double prev_conv[3][4], double double double ) compute camera position in function of detected marker with an history function. calculate the transformation between a detected marker and the real camera, i.e. the position and orientation of the camera relative to the tracking mark. Since this routine operate on previous values, the result are more stable (less jittering). Parameters:

center[2], width, conv[3][4]

marker_info the structure containing the parameters for the marker for which the camera
position and orientation is to be found relative to. This structure is found using arDetectMarker. the previous transformation matrix obtain. the physical center of the marker. arGetTransMat assumes that the marker is in x-y plane, and z axis is pointing downwards from marker plane. So vertex positions can be represented in 2D coordinates by ignoring the z axis information. The marker vertices are specified in order of clockwise. the size of the marker (in mm). the transformation matrix from the marker coordinates to camera coordinate frame, that is the relative position of real camera to the real marker

prev_conv center

width conv
Returns: always 0.

ARUint32 arGetVersion( char ** versionStringRef )

Get the ARToolKit version information in numberic and string format. As of version 2.72, ARToolKit now allows querying of the version number of the toolkit available at runtime. It is highly recommended that any calling program that depends on features in a certain ARToolKit version, check at runtime that it is linked to a version of ARToolKit that can supply those features. It is NOT sufficient to check the ARToolKit SDK header versions, since with ARToolKit implemented in dynamically-loaded libraries, there is no guarantee that the version of ARToolKit installed on the machine at run-time will as recent as the version of the ARToolKit SDK which the host program was compiled against. The version information is reported in binary-coded decimal format, and optionally in an ASCII string. See the config.h header for more discussion of the definition of major, minor, tiny and build version numbers. Parameters:

versionStringRef If non-NULL, the location pointed to will be filled with a pointer to a string

containing the version information. Fields in the version string are separated by spaces. As of version 2.72.0, there is only one field implemented, and this field contains the major, minor and tiny version numbers in dotted-decimal format. The string is guaranteed to contain at least this field in all future versions of the toolkit. Later versions of the toolkit may add other fields to this string to report other types of version information. The storage for the string is malloc'ed inside the function. The caller is responsible for free'ing the string.

Returns: Returns the full version number of the ARToolKit in binary coded decimal (BCD) format. BCD format allows simple tests of version number in the caller e.g. if ((arGetVersion(NULL) >> 16) > 0x0272) printf("This release is later than 2.72\n"); The major version number is encoded in the most-significant byte (bits 31-24), the minor version number in the secondmost-significant byte (bits 23-16), the tiny version number in the third-most-significant byte (bits 15-8), and the build version number in the least-significant byte (bits 7-0). int arInitCparam( ARParam * param ) initialize camera parameters. set the camera parameters specified in the camera parameters structure *param to static memory in the AR library. These camera parameters are typically read from a data file at program startup. In the video-see through AR applications, the default camera parameters are sufficient, no camera calibration is needed. Parameters:

param the camera parameter structure

Returns: always 0 ARInt16* arLabeling ( ARUint image, 8* int thresh, int *

label_num,

int ** area, double pos, **

int ** int ** )

clip, label_ref

extract connected components from image. Label the input image, i.e. extract connected components from the input video image. Parameters:

image thresh area pos clip label_ref

input image, as returned by arVideoGetImage() lighting threshold

label_num Ouput- number of detected components

On return, if label_num > 0, points to an array of ints, one for each detected component. On return, if label_num > 0, points to an array of doubles, one for each detected component. On return, if label_num > 0, points to an array of ints, one for each detected component. On return, if label_num > 0, points to an array of ints, one for each detected component.

Returns: returns a pointer to the labeled output image, ready for passing onto the next stage of processing. void arLabelingCleanup( void ) clean up static data allocated by arLabeling. In debug mode, arLabeling may allocate and use static storage. This function deallocates this storage. int arLoadPatt( const char * filename ) load markers description from a file load the bitmap pattern specified in the file filename into the pattern matching array for later use by the marker detection routines. Parameters:

filename name of the file containing the pattern bitmap to be loaded

Returns: the identity number of the pattern loaded or 1 if the pattern load failed. double ( doub rot[3][3], arModifyMatrix le doub trans[3], le doub cpara[3][4], le

doub vertex[][3], le doub pos2d[][2], le int num ) XXXBK. XXXBK Parameters:

rot

XXXBK

trans XXXBK cpara XXXBK vertex XXXBK pos2d XXXBK num

Returns: XXXBK int ( ARUint8 * arSavePat image, t ARMarkerIn marker_info, fo * char * filename ) save a marker. used in mk_patt to save a bitmap of the pattern of the currently detected marker. The saved image is a table of the normalized viewed pattern. Parameters: XXXBK

image filename

a pointer to the image containing the marker pattern to be trained.

marker_info a pointer to the ARMarkerInfo structure of the pattern to be trained.

The name of the file where the bitmap image is to be saved. Returns: 0 if the bitmap image is successfully saved, -1 otherwise. int ( doub arUtilMat2QuatP le m[3][4], os doub q[4], le doub p[3]

le ) extract a quaternion/position of matrix. Extract a rotation (quaternion format) and a position (vector format) from a transformation matrix. The precondition is an euclidian matrix. Parameters:

m source matrix q a rotation represented by a quaternion. p a translation represented by a vector.

Returns: 0 if the extraction success, -1 otherwise (quaternion not normalize) int ( doubl s[3][4], arUtilMatInv e doubl d[3][4] e ) Inverse a non-square matrix. Inverse a matrix in a non homogeneous format. The matrix need to be euclidian. Parameters:

s matrix input d resulted inverse matrix.

Returns: 0 if the inversion success, -1 otherwise Remarks: input matrix can be also output matrix int ( doubl arUtilMatMu e s1[3][4], l doubl s2[3][4], e doubl d[3][4] e ) Multiplication of two matrix. This procedure do a multiplication matrix between s1 and s2 and return the result in d : d=s1*s2. The precondition is the output matrix need to be different of input matrix. The precondition is euclidian matrix. Parameters:

s1 first matrix. s2 second matrix. d resulted multiplication matrix.

Returns: 0 if the multiplication success, -1 otherwise int ( doub arUtilQuatPos2 le q[4], Mat doub p[3], le doub m[3][4] le ) create a matrix with a quaternion/position. Create a transformation matrix from a quaternion rotation and a vector translation. Parameters:

q a rotation represented by a quaternion. p a translation represented by a vector. m destination matrix

Returns: always 0 void arUtilSleep( int msec ) sleep the actual thread. Sleep the actual thread. Parameters:

msec time to sleep (in millisecond)

double arUtilTimer( void ) get the time with the ARToolkit timer. Give the time elapsed since the reset of the timer. Returns: elapsed time (in milliseconds) void arUtilTimerReset( void ) reset the internal timer of ARToolkit. Reset the internal timer used by ARToolKit. timer measurement (with arUtilTimer()).

Variable Documentation
int arDebug activate artoolkit debug mode control debug informations in ARToolKit. the possible values are: 0: not in debug mode 1: in debug mode by default: 0

int arFittingMode fitting display mode use by ARToolkit. Correction mode for the distorsion of the camera. You can enable a correction with a texture mapping. the possible values are: AR_FITTING_TO_INPUT: input image AR_FITTING_TO_IDEAL: compensated image by default: DEFAULT_FITTING_MODE in config.h

ARUint8 * arImage internal image internal image used. (access only for debugging ARToolKit) by default: NULL int arImageProcMode define the image size mode for marker detection. Video image size for marker detection. This control if all the image is analyzed the possible values are : AR_IMAGE_PROC_IN_FULL: full image uses. AR_IMAGE_PROC_IN_HALF: half image uses. by default: DEFAULT_IMAGE_PROC_MODE in config.h

int arImXsize internal image size in width. internal image size in width (generally initialize in arInitCparam) int arImYsize internal image size in heigth internal image size in heigth (generally initialize in arInitCparam)

int arMatchingPCAMode XXXBK. XXXBK the possible values are : -AR_MATCHING_WITHOUT_PCA: without PCA AR_MATCHING_WITH_PCA: with PCA by default: DEFAULT_MATCHING_PCA_MODE in config.h ARParam arParam internal intrinsic camera parameter internal variable for camera intrinsic parameters int arTemplateMatchingMode XXXBK. XXXBK the possible values are : AR_TEMPLATE_MATCHING_COLOR: Color Template AR_TEMPLATE_MATCHING_BW: BW Template by default: DEFAULT_TEMPLATE_MATCHING_MODE in config.h

arMulti.h File Reference

ARToolkit multi pattern subroutines. More...

#include #include #include #include

<stdio.h> <AR/config.h> <AR/param.h> <AR/ar.h>

Data Structures
struct ARMultiEachMarkerInfoT multi-marker structure More... ARMultiMarkerInfoT global multi-marker structure More...

struct

Functions
ARMultiMarkerInfoT * arMultiReadConfigFile (const char *filename) loading multi-markers description from a file double arMultiGetTransMat (ARMarkerInfo ARMultiMarkerInfoT *config) *marker_info, int marker_num,

compute camera position in function of the multi-marker patterns (based on detected markers) int arMultiActivate (ARMultiMarkerInfoT *config) activate a multi-marker pattern on the recognition procedure. arMultiDeactivate (ARMultiMarkerInfoT *config) Desactivate a multi-marker pattern on the recognition procedure. arMultiFreeConfig (ARMultiMarkerInfoT *config) remove a multi-marker pattern from memory. arsMultiGetTransMat (ARMarkerInfo *marker_infoL, int marker_numL, ARMarkerInfo *marker_infoR, int marker_numR, ARMultiMarkerInfoT *config)

int

int

double

Detailed Description
ARToolkit multi pattern subroutines. This file extend ar core routines for multi-pattern tracking. You can obtain position of a local coordinate system based on the estimation of multiple markers tracking (each in relative position) Remarks: more efficient with uniform planar configuration History : Author: Hirokazu Kato kato@sys.im.hiroshima-cu.ac.jp Version: 1.0 Date: 01/09/05

Function Documentation
int arMultiActivate( ARMultiMarkerInfoT * config ) activate a multi-marker pattern on the recognition procedure. Activate a multi-marker for be checking during the template matching operation. Parameters:

config pointer to the multi-marker

Returns: 0 if success, -1 if error int arMultiDeactivate( ARMultiMarkerInfoT * config ) Desactivate a multi-marker pattern on the recognition procedure.

Desactivate a multi-marker for not be checking during the template matching operation. Parameters:

config pointer to the multi-marker

Returns: 0 if success, -1 if error int arMultiFreeConfig( ARMultiMarkerInfoT * config ) remove a multi-marker pattern from memory. desactivate a pattern and remove it from memory. Post-condition of this function is unavailability of the multi-marker pattern. Parameters:

config pointer to the multi-marker

Returns: 0 if success, -1 if error double ( ARMarkerInfo arMultiGetTrans * marker_info, Mat int marker_num, ARMultiMarke config rInfoT * ) compute camera position in function of the multi-marker patterns (based on detected markers) calculate the transformation between the multi-marker patterns and the real camera. Based on confident values of detected markers in the multi-markers patterns, a global position is return. Parameters:

marker_info list of detected markers (from arDetectMarker) marker_num number of detected markers config
Returns: ARMultiMarkerInfoT* arMultiReadConfigFile( const char * filename ) loading multi-markers description from a file Load a configuration file for multi-markers tracking. The configuration file furnishs pointer to each pattern description. Parameters:

filename name of the pattern file

Returns:

a pattern structure, NULL if error

gsub.h File Reference

ARToolkit OpenGL subroutines. More...

#include <AR/config.h> #include <AR/param.h> #include <AR/ar.h>

Functions
void argInit (ARParam *cparam, double zoom, int fullFlag, int xwin, int ywin, int hmd_flag) Initialise the gsub library. argLoadHMDparam (ARParam *lparam, ARParam *rparam) initialize camera for HMD. argCleanup (void) Close the gsub library. argSwapBuffers (void) swap the rendering buffer. argMainLoop (void(*mouseFunc)(int button, int state, int x, int y), void(*keyFunc)(unsigned char key, int x, int y), void(*mainFunc)(void)) start the program main loop with specified callback functions. argDrawMode2D (void) switch the rendering context for 2D rendering mode. void argDraw2dLeft (void) switch the rendering view to left eye (in 2D space) void argDraw2dRight (void) switch the rendering view to right eye (in 2D space) void argDrawMode3D (void) switch the rendering context for 3D rendering mode. argDraw3dLeft (void) switch the rendering view to left eye (in 3D space) argDraw3dRight (void) switch the rendering view to right eye (in 3D space) argDraw3dCamera (int xwin, int ywin) switch the rendering view for 3D rendering mode. argConvGlpara (double para[3][4], double gl_para[16]) transform ARToolKit matrix format to an openGL matrix format.

void

void

void

void

void

void

void

void

void

void

argConvGLcpara (ARParam *param, double gnear, double gfar, double m[16]) transform ARToolKit intrinsic camera parameters matrix format to an openGL matrix format. argDispImage (ARUint8 *image, int xwin, int ywin) display the video image. argDispHalfImage (ARUint8 *image, int xwin, int ywin) display half of the video image. argDrawSquare (double vertex[4][2], int xwin, int ywin) draw a 2D square. argLineSeg (double x1, double y1, double x2, double y2, int xwin, int ywin) Draw a line. argLineSegHMD (double x1, double y1, double x2, double y2) Draw a line with HMD mode. argInqSetting (int *hmdMode, int *gMiniXnum2, int *gMiniYnum2, void(**mouseFunc)(int button, int state, int x, int y), void(**keyFunc)(unsigned char key, int x, int y), void(**mainFunc)(void)) Get current configuration of gsub library. argsInit (ARSParam *scparam, double zoom, int twinFlag, int fullFlag, int xwin, int ywin) argsDraw3dCamera (int xwin, int ywin, int LorR, int stencil_flag) argsConvGLcpara (ARSParam *sparam, double gnear, double gfar, double mL[16], double mR[16]) argsDispImage (ARUint8 *image, int LorR, int xwin, int ywin) argsDispHalfImage (ARUint8 *image, int LorR, int xwin, int ywin) argsLineSeg (double x1, double y1, double x2, double y2, int xwin, int ywin, int LorR) argsDrawSquare (double vertex[4][2], int xwin, int ywin, int LorR)

void

void

void

void

void

void

void void void void void void void

Variables
int argDrawMode define the draw configuration mode. argTexmapMode define the texture map configuration mode.

int

Detailed Description
ARToolkit OpenGL subroutines. This file contains the main display functions used in ARToolkit Library. It contains wrapped functions of GLUT and openGL for doing real-time rendering. This choice give us a large multi-platform support for the display module of ARToolkit. Remarks: The supported stereo mode is interleaved stereo mode (only for i-glasses display). History : Author: Hirokazu Kato kato@sys.im.hiroshima-cu.ac.jp Version:

Date:

Function Documentation
void argCleanup( void ) Close the gsub library. This function clean the rendering context (GLUT and openGL). Call in the exit of your program. Remarks: BE CAREFUL, THIS FUNCTION DOESN'T RESET PERSPECTIVE MATRIX AND CURRENT GL STATE TO DEFAULT void ( ARPara argConvGLcpar m * param, a double gnear, double gfar, double m[16] ) transform ARToolKit intrinsic camera parameters matrix format to an openGL matrix format. XXXBK: not be sure of this function: this function must just convert 3x4 matrix to classical perspective openGL matrix. But in the code, you used arParamDecompMat that seem decomposed K and R,t, aren't it ? why do this decomposition since we want just intrinsic parameters ? and if not what is arDecomp ? Returned value is generally use with a Projection Matrix. Parameters:

param gnear near clipping plane value gfar m

far clipping plane value the resulted openGL matrix

void ( doubl para[3][4], argConvGlpara e doubl gl_para[16] e ) transform ARToolKit matrix format to an openGL matrix format. simple conversion for the openGL matrix (16 values and homogeneous matrix). Returned value

is generally use with a Model View Matrix. Parameters:

para

the ARToolKit matrix

gl_para the resulted openGL matrix

void ( ARUint argDispHalfImag 8 * image, e int xwin, int ) display half of the video image. Idem of argDispImage except than a quarter of the image is display (first left top quadrant, so size/2 in x and y). Parameters:

ywin

image image to display xwin ywin

XXXBK XXXBK

void ( ARUint image, argDispImage 8 * int xwin, int ) display the video image. Display in the back-buffer the video image in argument. For doing AR video background, this function must be called before any rendering of 3D object. Remarks: According to your argDrawMode, argTexmapMode and the internal image format the openGL function called is different and less or more efficient. with AR_DRAW_BY_GL_DRAW_PIXELS, unaffected by current camera parameters matrix but affected by glRasterPos3f. with AR_DRAW_BY_TEXTURE_MAPPING, affected by current current camera parameters matrix. You need generally call argDrawMode2D before this function. Parameters:

ywin

image image to display xwin

XXXBK

ywin

XXXBK

void argDraw2dLeft( void ) switch the rendering view to left eye (in 2D space) Combine with argDrawMode2D for rendering the left view. void argDraw2dRight( void ) switch the rendering view to right eye (in 2D space) Combine with argDrawMode2D for rendering the right view. void ( in argDraw3dCamer t xwin, a in ywin t ) switch the rendering view for 3D rendering mode. Update curent internal camera parameters for rendering in 3D space. this function complements argDrawMode3D. Parameters:

xwin length of rendering view (less than window length) ywin width of rendering view (less than window width)
void argDraw3dLeft( void ) switch the rendering view to left eye (in 3D space) Update curent internal camera parameters for rendering in 3D space for left eye. this function complements argDrawMode3D. void argDraw3dRight( void ) switch the rendering view to right eye (in 3D space) Update curent internal camera parameters for rendering in 3D space for left eye. this function complements argDrawMode3D. void argDrawMode2D( void ) switch the rendering context for 2D rendering mode. Update curent camera parameters (internal and external) for rendering 2D or 3D objects in the view plane (like text or 2D shape). This function define an orthographic projection in the image plane. It not define opengl state for rendering in image space (like for a bitmap copy).

void argDrawMode3D( void ) switch the rendering context for 3D rendering mode. Update curent camera parameters for rendering in 3D space. Generally call to reinializing model view matrix. void ( doubl vertex[4][2], argDrawSquare e int xwin, int ) draw a 2D square. Draw a square. The position of the square is affected by openGL model view matrix and call to argDrawMode2D argDrawMode3D. Generally call in a 2D mode (so after a argDrawMode2D). Parameters:

ywin

vertex corner of square. xwin ywin

XXXBK XXXBK

argIn ( ARParam cparam, it * double zoom, int int int int ) Initialise the gsub library. This function performs required initialisation of the gsub library. It must be called before any other argl*() functions are called. Parameters:

fullFlag, xwin, ywin, hmd_flag

cparam zoom fullFlag xwin ywin

the intrinsics parameters of the camera (used to defined openGL perspective matrix) defined a zoom parameter for the final result. full screen mode (1 enable, 0 disable). XXXBK. 0 if indifferent. XXXBK. 0 if indifferent.

hmd_flag enable stereo display mode (only interleaved configuration)

argInqS ( int * etting int *

hmdMode, gMiniXnum2,

int * gMiniYnum2, void(**)(int button, int mouseFunc, state, int x, int y) void(**)(unsigned char keyFunc, key, int x, int y) void(**)(void) mainFunc ) Get current configuration of gsub library. Retrieve current state of gsub library like the current callback functions. Parameters:

hmdMode

the current hmdMode

gMiniXnum2 XXXBK gMiniYnum2 XXXBK mouseFunc keyFunc mainFunc

the current mouse function callback the current key function callback the current main function callback

void ( doubl x1, argLineSeg e doubl y1, e doubl x2, e doubl y2, e int xwin, int ) Draw a line. Draw a segment.T The position of the line is affected by openGL model view matrix and call to argDrawMode2D argDrawMode3D. Generally call in a 2D mode (so after a argDrawMode2D). Parameters:

ywin

x1 y1 x2

x position of the first point. y position of the first point. x position of the second point.

y2

y position of the second point.

xwin XXXBK ywin XXXBK

void ( doub argLineSegHM le x1, D doub y1, le doub x2, le doub y2 le ) Draw a line with HMD mode. Draw a segment in HMD mode. Parameters:

x1 x position of the first point. y1 y position of the first point. x2 x position of the second point. y2 y position of the second point.
void ( ARPara argLoadHMDpar m * lparam, am ARPara rparam m* ) initialize camera for HMD. Load in the display module the intrinsic parameters of the two view, i.e camera (identify to the eyes). Parameters:

lparam parameter of left camera rparam parameter of right camera

void ( void(*)(int button, int argMainLo state, int x, int y) mouseFunc, op void(*)(unsigned char keyFunc, key, int x, int y) void(*)(void) mainFunc )

start the program main loop with specified callback functions. This function is called in the entry block of a program. User specify the main callback of his program. Users should not put routines calls after this function, generally never accessible. Parameters:

mouseFunc the user mouse function can be NULL. keyFunc mainFunc

the user keyboard function can be NULL. the user main update function can be NULL.

void argSwapBuffers( void ) swap the rendering buffer. Swap the back-buffer to the front-buffer. the pre-condition is that all the rendering functions have been called.

Variable Documentation
int argDrawMode define the draw configuration mode. Define the draw mode for display of the video background. The possible values are : AR_DRAW_BY_GL_DRAW_PIXELS: use the GL_DRAW_PIXELS function AR_DRAW_BY_TEXTURE_MAPPING: use a quad mesh with a texture mapping of the video. by default: DEFAULT_DRAW_MODE in config.h choice and performance depends on your hardware and your openGL driver.

int argTexmapMode define the texture map configuration mode. If the draw mode is AR_DRAW_BY_TEXTURE_MAPPING, you can configure the copy mode of the texture mapping. The possible values are : AR_DRAW_TEXTURE_FULL_IMAGE: texture mapping full resolution. AR_DRAW_TEXTURE_HALF_IMAGE: texture mapping half resolution. by default: DEFAULT_DRAW_TEXTURE_IMAGE in config.h

gsub_lite
A collection of useful OpenGL routines for ARToolKit. Updated: June 23, 2006 Includes: <AR/config.h>, <AR/ar.h>, <AR/param.h>

Sample code for example usage of gsub_lite is included with ARToolKit, in the directory <AR/examples/simpleLite>. gsub_lite is the preferred means for drawing camera video images acquired from ARToolKit's video libraries. It includes optimized texture handling, and a variety of flexible drawing options. gsub_lite also provides utility functions for setting the OpenGL viewing frustum and camera position based on ARToolKit- camera parameters and marker positions. gsub_lite does not depend on GLUT, or indeed, any particular window or event handling system. It is therefore well suited to use in applications which have their own window and event handling code. gsub_lite v2.7 is intended as a replacement for gsub from ARToolKit 2.65, by Mark Billinghurst (MB) & Hirokazu Kato (HK), with the following additional functionality: Support for true stereo and multiple displays through removal of most dependencies on global variables. Prepared library for thread-safety by removing global variables. Optimised texturing, particularly for Mac OS X platform. Added arglCameraFrustum to replace argDraw3dCamera() function. Renamed argConvGlpara() to arglCameraView() to more accurately represent its functionality. Correctly handle textures with non-RGBA handling. Library setup and cleanup functions. Version numbering.

It does however lack the following functionality from the original gsub library: GLUT event handling. Sub-window ("MINIWIN") and half-size drawing. HMD support for stereo via stencil.

This file is part of ARToolKit. ARToolKit is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. ARToolKit is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

arglCameraFrustum

Create an OpenGL perspective projection matrix.

void arglCameraFrustum( const ARParam *cparam, const double focalmin, const double focalmax, GLdouble m_projection[16]);
Parameter Descriptions

cparam
Pointer to a set of ARToolKit camera parameters for the current video source.

focalmax
The maximum distance at which geometry will be rendered. Any geometry further away from the camera than this distance will be clipped and will not be appear in a rendered frame. Thus, this value should be set high enough to avoid clipping of any geometry you care about. However, the precision of the depth buffer is correlated with the ratio of focalmin to focalmax, thus you should not set focalmax any higher than it needs to be. This value should be specified in the same units as your OpenGL drawing.

focalmin
The minimum distance at which geometry will be rendered. Any geometry closer to the camera than this distance will be clipped and will not be appear in a rendered frame. Thus, this value should be set low enough to avoid clipping of any geometry you care about. However, the precision of the depth buffer is correlated with the ratio of focalmin to focalmax, thus you should not set focalmin any lower than it needs to be. Additionally, geometry viewed in a stereo projections that is too close to camera is difficult and tiring to view, so if you are rendering stereo perspectives you should set this value no lower than the near-point of the eyes. The near point in humans varies, but usually lies between 0.1 m 0.3 m. This value should be specified in the same units as your OpenGL drawing.

m_projection
Pointer to a array of 16 GLdoubles, which will be filled out with a projection matrix suitable for passing to OpenGL. The matrix is specified in column major order. Discussion Use this function to create a matrix suitable for passing to OpenGL to set the viewing projection. Availability First appeared in ARToolKit 2.68.

arglCameraFrustumRH
(description)

void arglCameraFrustumRH( const ARParam *cparam, const double focalmin, const double focalmax, GLdouble m_projection[16]);

Parameter Descriptions

(name)
(description) function result (description) Discussion (description)

arglCameraView
Create an OpenGL viewing transformation matrix.

void arglCameraView( const double para[3][4], GLdouble m_modelview[16], const double scale);
Parameter Descriptions

para
Pointer to 3x4 matrix array of doubles which specify the position of an ARToolKit marker, as returned by arGetTransMat().

m_modelview
Pointer to a array of 16 GLdoubles, which will be filled out with a modelview matrix suitable for passing to OpenGL. The matrix is specified in column major order.

scale
Specifies a scaling between ARToolKit's units (usually millimeters) and OpenGL's coordinate system units. What you pass for the scalefactor parameter depends on what units you want to do your OpenGL drawing in. If you use a scalefactor of 1.0, then To use different OpenGL units, e.g. metres, then you would pass 0.001. Discussion Use this function to create a matrix suitable for passing to OpenGL to set the viewing transformation of the virtual camera. Availability First appeared in ARToolKit 2.68.

arglCameraViewRH
(description)

void arglCameraViewRH( const double para[3][4],

GLdouble m_modelview[16], const double scale);

Parameter Descriptions

(name)
(description) function result (description) Discussion (description)

arglCleanup
Free memory used by gsub_lite associated with the specified context.

void arglCleanup( ARGL_CONTEXT_SETTINGS_REF contextSettings);

Parameter Descriptions

contextSettings
A reference to ARGL's settings arglSetupForCurrentContext(). Discussion Should be called after no more argl* functions are needed, in order to prevent memory leaks etc. The library can be setup again for the context at a later time by calling arglSetupForCurrentContext() again. Availability First appeared in ARToolKit 2.68. for an OpenGL context, as returned by

arglDispImage
Display an ARVideo image, by drawing it using OpenGL.

void arglDispImage( ARUint8 *image, const ARParam *cparam, const double zoom, ARGL_CONTEXT_SETTINGS_REF contextSettings);

Parameter Descriptions

image
Pointer to the tightly-packed image data (as returned by arVideoGetImage()). The horizontal and vertical dimensions of the image data must exactly match the values specified in the fields cparam->xsize and cparam->ysize (see below). The first byte of image data corresponds to the first component of the top-left-most pixel in the image. The data continues with the remaining pixels of the first row, followed immediately by the pixels of the second row, and so on to the last byte of the image data, which corresponds to the last component of the bottom-right-most pixel.

cparam
Pointer to a set of ARToolKit camera parameters for the current video source. The size of the source image is taken from the fields xsize and ysize of the ARParam structure pointed to. Also, when the draw mode is AR_DRAW_BY_TEXTURE_MAPPING (see the documentation for the global variable arglDrawMode) the field dist_factor of the ARParam structure pointed to will be taken as the amount to un-warp the supplied image.

zoom
The amount to scale the video image up or down. To draw the video image double size, use a zoom value of 2.0. To draw the video image half size use a zoom value of 0.5.

contextSettings
A reference to ARGL's settings for the current OpenGL context, as returned by arglSetupForCurrentContext() for this context. It is the callers responsibility to make sure that the current context at the time arglDisplayImage() is called matches that under which contextSettings was created. Discussion This function draws an image from an ARVideo source to the current OpenGL context. This operation is most useful in video see-through augmented reality applications for drawing the camera view as a background image, but can also be used in other ways. An undistorted image is drawn with the lower-left corner of the bottom-left-most pixel at OpenGL screen coordinates (0,0), and the upper-right corner of the top-right-most pixel at OpenGL screen coodinates (x * zoom, y * zoom), where x and y are the values of the fields cparam->xsize and cparam->ysize (see below) and zoom is the value of the parameter zoom (also see below). If cparam>dist_factor indicates that an un-warping correction should be applied, the actual coordinates will differ from the values specified here. OpenGL state: Drawing is performed with depth testing and lighting disabled, and thus leaves the the depth buffer (if any) unmodified. If pixel transfer is by texturing (see documentation for arglDrawMode), the drawing is done in replacement texture environment mode. The depth test enable and lighting enable state and the texture environment mode are restored before the function returns. Availability First appeared in ARToolKit 2.68.

arglDispImageStateful
Display an ARVideo image, by drawing it using OpenGL, using and modifying current OpenGL state.

void arglDispImageStateful( ARUint8 *image,

const ARParam *cparam, const double zoom, ARGL_CONTEXT_SETTINGS_REF contextSettings);

Discussion This function is identical to arglDispImage except that whereas arglDispImage sets an orthographic 2D projection and the OpenGL state prior to drawing, this function does not. It also does not restore any changes made to OpenGL state. This allows you to do effects with your image, other than just drawing it 2D and with the lower-left corner of the bottom-left-most pixel attached to the bottom-left (0,0) of the window. For example, you might use a perspective projection instead of an orthographic projection with a glLoadIdentity() / glTranslate() on the modelview matrix to place the lower-left corner of the bottom-left-most pixel somewhere other than 0,0 and leave depth-testing enabled. See the documentation for arglDispImage() for more information. Availability First appeared in ARToolKit 2.68.2.

arglDistortionCompensationGet
Enquire as to the enable state of camera lens distortion compensation in arglDispImage.

int arglDistortionCompensationGet( ARGL_CONTEXT_SETTINGS_REF contextSettings, int *enable);

Parameter Descriptions

contextSettings
A reference to ARGL's settings for the current OpenGL context, as returned by arglSetupForCurrentContext() for this context.

enable
Pointer to an integer value which will be set to TRUE if distortion compensation is enabled in the specified context, or FALSE if it is disabled. function result TRUE if the distortion value was retreived, FALSE if an error occurred. Discussion By default, arglDispImage compensates for the distortion of the camera's acquired image caused by the lens when it draws. This function enquires as to whether arglDispImage is currently doing compensation or not. Availability First appeared in ARToolKit 2.71.

arglDistortionCompensationSet

Set compensation for camera lens distortion in arglDispImage to off or on.

int arglDistortionCompensationSet( ARGL_CONTEXT_SETTINGS_REF contextSettings, int enable);

Parameter Descriptions

contextSettings
A reference to ARGL's settings for the current OpenGL context, as returned by arglSetupForCurrentContext() for this context.

enable
TRUE to enabled distortion compensation, FALSE to disable it. The default state for new contexts is enable = TRUE. function result TRUE if the distortion value was set, FALSE if an error occurred. Discussion By default, arglDispImage compensates for the distortion of the camera's acquired image caused by the lens when it draws. By calling this function with enabled = FALSE, this compensation will be disabled in the specified drawing context. It may be re-enabled at any time. This function is useful if you need to draw an image, but do not know the extent of the camera's lens distortion (such as during distortion calibration). While distortion compensation is disabled, the dist_factor[] array in a the camera cparam structure passed to arglDispImage is ignored. Availability First appeared in ARToolKit 2.71.

arglDrawModeGet
Get method by which arglDispImage() is transfering pixels.

int arglDrawModeGet( ARGL_CONTEXT_SETTINGS_REF contextSettings);

Discussion Enquires as to the current method by which arglDispImage() is transferring pixels to OpenGL for display. See arglDrawModeSet() for more information. Availability First appeared in ARToolKit 2.72.

arglDrawModeSet
Set method by which arglDispImage() will transfer pixels.

void arglDrawModeSet(

ARGL_CONTEXT_SETTINGS_REF contextSettings, const int mode);

Discussion This setting determines the method by which arglDispImage transfers pixels of an image to OpenGL for display. Setting this variable to a value of AR_DRAW_BY_GL_DRAW_PIXELS specifies the use of the OpenGL DrawPixels functions to do the transfer. Setting this variable to a value of AR_DRAW_BY_TEXTURE_MAPPING specifies the use of OpenGL TexImage2D functions to do the transfer. The DrawPixels method is guaranteed to be available on all implementations, but arglDispImage does not correct the image for camera lens distortion under this method. In contrast, TexImage2D is only available on some implementations, but allows arglDispImage() to apply a correction for camera lens distortion, and additionally offers greater potential for accelerated drawing on some implementations. The initial value is AR_DRAW_BY_TEXTURE_MAPPING. Availability First appeared in ARToolKit 2.72.

arglPixelFormatGet
Get the format of pixel data in which arglDispImage*() is expecting data to be passed.

int arglPixelFormatGet( ARGL_CONTEXT_SETTINGS_REF contextSettings, AR_PIXEL_FORMAT *format, int *size);

Parameter Descriptions

contextSettings
A reference to ARGL's settings for the current OpenGL context, as returned by arglSetupForCurrentContext() for this context.

format
A symbolic constant for the pixel format in use. See AR_PIXEL_FORMAT in ar.h for a list of all possible formats.

size
The number of bytes of memory occupied per pixel, for the given format. function result TRUE if the pixel format and size values were retreived, FALSE if an error occurred. Discussion This function enquires as to the current format of pixel data being expected by the arglDispImage*() functions. The default format is determined by the value of AR_DEFAULT_PIXEL_FORMAT at the time the library was built. Availability First appeared in ARToolKit 2.71.

arglPixelFormatSet
Set the format of pixel data which will be passed to arglDispImage*()

int arglPixelFormatSet( ARGL_CONTEXT_SETTINGS_REF contextSettings, AR_PIXEL_FORMAT format);

Parameter Descriptions

contextSettings
A reference to ARGL's settings for the current OpenGL context, as returned by arglSetupForCurrentContext() for this context.

format
A symbolic constant for the pixel format being set. See AR_PIXEL_FORMAT in ar.h for a list of all possible formats. function result TRUE if the pixel format value was set, FALSE if an error occurred. Discussion (description) In gsub_lite, the format of the pixels (i.e. the arrangement of components within each pixel) can be changed at runtime. Use this function to inform gsub_lite the format the pixels being passed to arglDispImage*() functions are in. This setting applies only to the context passed in parameter contextSettings. The default format is determined by the value of AR_DEFAULT_PIXEL_FORMAT at the time the library was built. Usually, image data is passed in directly from images generated by ARVideo, and so you should ensure that ARVideo is generating pixels of the same format. Availability First appeared in ARToolKit 2.71.

arglSetupForCurrentContext
Initialise the gsub_lite library for the current OpenGL context.

ARGL_CONTEXT_SETTINGS_REF arglSetupForCurrentContext( void);

function result An ARGL_CONTEXT_SETTINGS_REF. See the documentation for this type for more info. Discussion This function performs required setup of the gsub_lite library for the current OpenGL context and must be called before any other argl*() functions are called for this context. An OpenGL context holds all of the state of the OpenGL machine, including textures and display lists etc. There will usually be one OpenGL context for each window displaying OpenGL content. Other argl*() functions whose operation depends on OpenGL state will require an

ARGL_CONTEXT_SETTINGS_REF. This is just so that they can keep track of per-context variables. You should call arglCleanup() passing in the ARGL_CONTEXT_SETTINGS_REF when you have finished with the library for this context. Availability First appeared in ARToolKit 2.68.

arglTexmapModeGet
Enquire whether full or half-resolution TexImage2D pixel-transfer is being used in arglDispImage().

int arglTexmapModeGet( ARGL_CONTEXT_SETTINGS_REF contextSettings);

Discussion Enquires as to the current value of the TexmapMode setting. See arglTexmapModeSet() for more info. Availability First appeared in ARToolKit 2.72.

arglTexmapModeSet
Determines use of full or half-resolution TexImage2D pixel-transfer in arglDispImage().

void arglTexmapModeSet( ARGL_CONTEXT_SETTINGS_REF contextSettings, const int mode);

Discussion When arglDrawModeSet(AR_DRAW_BY_TEXTURE_MAPPING) has been called, the value of this setting determines whether full or half-resolution data is transferred to the texture. Calling this function with a mode value of AR_DRAW_TEXTURE_FULL_IMAGE uses all available pixels in the source image data. A value of AR_DRAW_TEXTURE_HALF_IMAGE discards every second row in the source image data, defining a half-height texture which is then drawn stretched vertically to double its height. The latter method is well-suited to drawing interlaced images, as would be obtained from DV camera sources in interlaced mode or composite video sources. The initial value is AR_DRAW_TEXTURE_FULL_IMAGE. Availability First appeared in ARToolKit 2.72.

arglTexRectangleGet

Enquire as to use of rectangular TexImage2D pixel-transfer in arglDispImage().

int arglTexRectangleGet( ARGL_CONTEXT_SETTINGS_REF contextSettings);

Discussion Enquires as to the current value of the TexRectangle setting. See arglTexRectangleSet() for more info. Availability First appeared in ARToolKit 2.72.

arglTexRectangleSet
Determines use of rectangular TexImage2D pixel-transfer in arglDispImage().

void arglTexRectangleSet( ARGL_CONTEXT_SETTINGS_REF contextSettings, const int state);

Discussion On implementations which support the OpenGL extension for rectangular textures (of non power-oftwo size), and when arglDrawMode is set to AR_DRAW_BY_TEXTURE_MAPPING, the value of this variable determines whether rectangular textures or ordinary (power-of-two) textures are used by arglDispImage(). A value of TRUE specifies the use of rectangluar textures. A value of FALSE specifies the use of ordinary textures. If the OpenGL driver available at runtime does not support for rectangular textures, changing the value of this setting to TRUE will result calls to arglDispImage performing no drawing. Availability First appeared in ARToolKit 2.72.

gsubUtil.h File Reference

ARToolkit OpenGL utilities subroutines. More...

#include <AR/param.h>

Functions
void argUtilCalibHMD (int targetId, int thresh2, void(*postFunc)(ARParam *lpara, ARParam *rpara)) utility function for calibrate an HMD.

Detailed Description
ARToolkit OpenGL utilities subroutines. Complement routines of gsub module. Remarks: History : Author: Hirokazu Kato kato@sys.im.hiroshima-cu.ac.jp Version: Date:

Function Documentation
void ( int argUtilCalib targetId, HMD int thresh2, void(*)(ARParam *lpara, postFunc ARParam *rpara) ) utility function for calibrate an HMD. This function offers a full calibration run-time routines for an optical HMD (mono or stereo). It is useful for estimate transformation between user eye position and camera position. You will find more informations on the calibration routine on opticalcalibration.html .This function modify gsub state of left and right camera intrinsic parameters. Parameters:

targetId the target used for the calibration step. thresh2

lighting threshold value to use your application is mono display, only lpara contains a value. lpara and rpara are NULL if the calibration failed.

postFunc a callback function used to analysis computed internal camera parameters. if

matrix.h File Reference

ARToolkit algebric mathematics subroutines. More...

#include <math.h> #include <AR/config.h>

Data Structures
struct ARMat matrix structure. More... ARVec vector structure. More...

struct

Defines
#define ARELEM0(mat, r, c) ((mat)->m[(r)*((mat)->clm)+(c)]) macro function that give direct access to an element (0 origin) ARELEM1(mat, row, clm) ARELEM0(mat,row-1,clm-1) macro function that give direct access to an element (1 origin)

#define

Functions
ARMat * arMatrixAlloc (int row, int clm) creates a new matrix. arMatrixFree (ARMat *m) deletes a matrix. arMatrixDup (ARMat *dest, ARMat *source) copy a matrix arMatrixAllocDup (ARMat *source) dumps a new matrix arMatrixUnit (ARMat *unit) Creates a unit matrix. arMatrixAllocUnit (int dim) Creates a unit matrix. arMatrixMul (ARMat *dest, ARMat *a, ARMat *b) Multiply two matrix. arMatrixAllocMul (ARMat *a, ARMat *b) Multiply two matrix with memory allocation. arMatrixTrans (ARMat *dest, ARMat *source) transposes a matrix. arMatrixAllocTrans (ARMat *source) transposes a matrix with allocation. arMatrixInv (ARMat *dest, ARMat *source) inverse a matrix. arMatrixSelfInv (ARMat *m) inverses a matrix.

int

int

ARMat *

int

ARMat *

int

ARMat *

int

ARMat *

int

int

ARMat *

arMatrixAllocInv (ARMat *source) inverses a matrix. arMatrixDet (ARMat *m) compute determinant of a matrix. arMatrixPCA (ARMat *input, ARMat *evec, ARVec *ev, ARVec *mean) compute the PCA of a matrix. arMatrixPCA2 (ARMat *input, ARMat *evec, ARVec *ev) compute the PCA of a matrix. arMatrixDisp (ARMat *m) display content of a matrix. arVecAlloc (int clm) creates a new vector. arVecFree (ARVec *v) delete a vector. arVecDisp (ARVec *v) display a vector. arVecHousehold (ARVec *x) XXXBK. arVecInnerproduct (ARVec *x, ARVec *y) Computes the inner product of 2 vectors. arVecTridiagonalize (ARMat *a, ARVec *d, ARVec *e) XXXBK.

double

int

int

int

ARVec *

int

int

double

double

int

Detailed Description
ARToolkit algebric mathematics subroutines. This package include matrix, vector manipulation routine. In complement to must classical routines (inversion, innerproduct), it includes a PCA (Principal) Component Analysis) routine. For the structure of the matrix see ARMat. Remarks: History : Author: Hirokazu Kato kato@sys.im.hiroshima-cu.ac.jp Version: Date:

Function Documentation
ARMat arMatrixAlloc *( in row, t in clm t

) creates a new matrix. Allocate and initialize a new matrix structure. XXXBK initializing ?? to 0 m ?? Parameters:

row number of line clm number of column

Returns: the matrix structure, NULL if allocation is impossible ARMat * arMatrixAllocDup( ARMat * source ) dumps a new matrix Allocates and recopy the original source matrix. Parameters:

source the source matrix to copy

Returns: the matrix if success, NULL if error int arMatrixAllocInv( ARMat * source ) inverses a matrix. Inverses a matrix and copy the result in in a new allocated structure. Parameters:

source the matrix to inverse

Returns: the inversed matrix if success, NULL if error ARMat *( ARMa a, arMatrixAllocMul t * ARMa b t* ) Multiply two matrix with memory allocation. multiply two matrix and copy the result in a new allocate matrix (the source matrix is unmodified). the product is this one : dest = a * b Parameters:

a first matrix b second matrix

Returns: the allocated matrix if success, NULL if error ARMat * arMatrixAllocTrans( ARMat * source ) transposes a matrix with allocation. transposes a matrix and copy the result in a new allocate matrix (the source matrix is unmodified). Parameters:

source the matrix to transpose

Returns: the allocated matrix if success, NULL if error (creation or transposition impossible) int arMatrixAllocUnit( int dim ) Creates a unit matrix. Allocates and initializes a matrix to a an identity matrix. Parameters:

dim dimensions of the unit matrix (square)

Returns: the matrix allocated if success, NULL if error int arMatrixDet( ARMat * m ) compute determinant of a matrix. Compute the determinant of a matrix. Parameters:

m matrix source
Returns: the computed determinant int arMatrixDisp( ARMat * m ) display content of a matrix. Display in current console, the content of the matrix. The display is done line by line. Parameters:

m
Returns: 0 int ( ARMat dest, arMatrixDu *

p ARMat source * ) copy a matrix copy one matrix to another. The two ARMat must be allocated. Parameters:

dest

the destination matrix of the copy

source the original matrix source

Returns: 0 if success, -1 if error (matrix with different size) int arMatrixFree( ARMat * m ) deletes a matrix. Delete a matrix structure (deallocate used memory). Parameters:

m matrix to delete
Returns: 0 int ( ARMat dest, arMatrixInv * ARMat source * ) inverse a matrix. inverse a matrix and copy the result in a new one (the source matrix is unmodified). the destination matrix must be allocated. the source matrix need to be a square matrix. Parameters:

dest

result matrix of the inverse operation

source source matrix

Returns: 0 if success, -1 if error (not square matrix) int ( ARMat dest, arMatrixMul * ARMat a, * ARMat b * )

Multiply two matrix. Multiply two matrix and copy the result in another the product is this one : dest = a * b. The destination matrix must be allocated. Matrix a and b need to have the same size (the source matrix is unmodified). Parameters:

dest final matrix product a b

first matrix second matrix

Returns: 0 if success, -1 if error (multiplication impossible, or destination matrix have not comptabile size) int ( ARMat arMatrixPC * input, A ARMat evec, * ARVec ev, * ARVec mean * ) compute the PCA of a matrix. Compute the Principal Component Analysis (PCA) of a matrix. Parameters:

input source matrix evec eigen vector computed ev

eigen value computed

mean mean computed

Returns: 0 if success to compute, -1 otherwise int ( ARMat arMatrixPCA * input, 2 ARMat evec, * ARVec ev * ) compute the PCA of a matrix.

Compute the Principal Component Analysis (PCA) of a matrix. Parameters:

input source matrix evec result matrix ev

egein value computed Returns: 0 if success to compute, -1 otherwise int arMatrixSelfInv( ARMat * m ) inverses a matrix. Inverses a matrix and copy the result in the same structure. Parameters:

m the matrix to inverse

Returns: 0 if success, -1 if error int ( ARMat arMatrixTran * dest, s ARMat source * ) transposes a matrix. Transposes a matrix. The destination matrix must be allocated (the source matrix is unmodified). Parameters:

dest

the destination matrix of the copy

source the source matrix

Returns: 0 if success, -1 if error (source and destination matrix have different size) int arMatrixUnit( ARMat * unit ) Creates a unit matrix. Transforms the source parameter matrix to a unit matrix (all values are modified). the unit matrix needs to be allocated. Parameters:

unit the matrix to transform

Returns: 0 if success, -1 if error

ARVec * arVecAlloc( int clm ) creates a new vector. Allocates and initializes new vector structure. Parameters:

clm dimension of vector

Returns: the allocated vector, NULL if error (impossible allocation) int arVecDisp( ARVec * v ) display a vector. Display element of a vector. Parameters:

v the vector to display

Returns: 0 int arVecFree( ARVec * v ) delete a vector. Delete a vector structure (deallocate used memory). Parameters:

v the vector to delete

Returns: 0 double arVecHousehold ( ARVec * x ) XXXBK. XXXBK: for QR decomposition ?? (can't success to find french translation of this term) Parameters:

x XXXBK
Returns: XXXBK double ( ARVe x, arVecInnerproduct c * ARVe y c* )

Computes the inner product of 2 vectors. computes the inner product of the two argument vectors. the operation done is a=x.y (and a is return) Parameters:

x first vector source y second vector source

Returns: the computed innerproduct int ( ARMa arVecTridiagonal t * a, ize ARVe d, c* ARVe e c* ) XXXBK. XXXBK Parameters:

a XXXBK d XXXBK e XXXBK

Returns: XXXBK

param.h File Reference

ARToolkit global structure (parameters) subroutines. More...

#include <AR/config.h>

Data Structures
struct ARParam camera intrinsic parameters. More... struct ARSParam

Functions
int arParamGet (double global[][3], double screen[][2], int data_num, double mat[3][4])

XXXBK. int arParamDecomp (ARParam *source, ARParam *icpara, double trans[3][4]) XXXBK. int arParamDecompMat (double source[3][4], double cpara[3][4], double trans[3][4]) XXXBK. int arParamIdeal2Observ (const double dist_factor[4], const double ix, const double iy, double *ox, double *oy) Convert ideal screen coordinates of a vertex to observed ones. arParamObserv2Ideal (const double dist_factor[4], const double ox, const double oy, double *ix, double *iy) Convert observed screen coordinates of a vertex to ideal ones. arParamChangeSize (ARParam *source, int xsize, int ysize, ARParam *newparam) change the camera size parameters. arParamSave (char *filename, int num, ARParam *param,...) save a camera intrinsic parameters. arParamLoad (const char *filename, int num, ARParam *param,...) load the camera intrinsic parameters. arParamDisp (ARParam *param) display parameters. arsParamChangeSize (ARSParam *source, int xsize, int ysize, ARSParam *newparam) arsParamSave (char *filename, ARSParam *sparam) arsParamLoad (char *filename, ARSParam *sparam) arsParamDisp (ARSParam *sparam) arsParamGetMat (double matL[3][4], double matR[3][4], double cparaL[3][4], double cparaR[3][4], double matL2R[3][4])

int

int

int

int

int

int int int int int

Detailed Description
ARToolkit global structure (parameters) subroutines. This file contains principal routines for loading, saving, and modify camera parameters for of ARToolkit library. Different structures are used for modify in run-time this parameters in the library. A file structure is use for input/output. Remarks: History : Author: Takeshi Mita tmita@inolab.sys.es.osaka-u.ac.jp Shinsaku Hiura shinsaku@sys.es.osaka-u.ac.jp Hirokazu Kato kato@sys.im.hiroshima-cu.ac.jp Version: 4.1 Date:

01/12/07

Function Documentation
int ( ARPara arParamChange m * source, Size int xsize, int ysize, ARPara newparam m* ) change the camera size parameters. Change the size variable in camera intrinsic parameters. Parameters:

source xsize ysize

Returns: 0

name of the source parameters structure new length size new height size

newparam name of the destination parameters structure.

int ( ARPara arParamDeco m * source, mp ARPara icpara, m* double trans[3][4] ) XXXBK. XXXBK Parameters:

source XXXBK icpara XXXBK trans

Returns: XXXBK int ( doub source[3][4], arParamDecomp le XXXBK

Mat doub cpara[3][4], le doub trans[3][4] le ) XXXBK. XXXBK Parameters:

source input camera matrix cpara camera parameter to be set trans

Returns: XXXBK int arParamDisp( ARParam * param ) display parameters. Display the structure of the camera instrinsic parameters argument. Parameters: XXXBK

param structure to display

Returns: 0 int ( doubl global[][3], arParamGet e doubl screen[][2], e int data_num, doubl mat[3][4] e ) XXXBK. XXXBK Parameters:

global screen mat

Returns:

XXXBK XXXBK

data_num XXXBK
XXXBK

XXXBK int int( const arParamIdeal2Obs double erv const double const double double * ) Convert ideal screen coordinates of a vertex to observed ones. Ideal coordinates mean that the distortion of the camera is compensated (so a straight line looks straight). In observed coordinates the camera-distortion is not compensated and thus a straight line is not shown really straight. Parameters:

dist_factor[4], ix, iy, ox,

double * oy

dist_factor distorsion factors of used camera ix iy ox oy

x in ideal screen coordinates y in ideal screen coordinates resulted x in observed screen coordinates resulted y in observed screen coordinates

Returns: 0 if success, -1 otherwise int ( const arParamLoa char * filename, d int num, ARParam param, *

...
) load the camera intrinsic parameters. Load camera intrinsic parameters in the ARToolkit Library from a file (itself, an output of the calibration step). Parameters:

filename name of the parameters file. num param

Returns: number of variable arguments result of the loaded parameters

0 if success, -1 if Error (file not found, file structure problem) int ( const arParamObserv2I double deal const double const double double * ) Convert observed screen coordinates of a vertex to ideal ones. Ideal coordinates mean that the distortion of the camera is compensated (so a straight line looks straight). In observed coordinates the camera-distortion is not compensated and thus a straight line is not shown really straight. Parameters:

dist_factor[4], ox, oy, ix,

double * iy

dist_factor distorsion factors of used camera ox oy ix iy

x in observed screen coordinates y in observed screen coordinates resulted x in ideal screen coordinates resulted y in ideal screen coordinates

Returns: 0 if success, -1 otherwise int ( char * arParamSav filename, e int num, ARPara param, m*

...
) save a camera intrinsic parameters. Save manipulated camera intrinsic parameters in a file. Parameters:

filename name of the parameters file. num param

number of variable arguments parameters to save

Returns: 0 if success, -1 if Error (file not found, file structure problem)

video.h File Reference

ARToolkit video subroutines. More...

#include <AR/config.h> #include <AR/ar.h>

Functions
AR_DLL_API int arVideoDispOption (void) display the video option. arVideoOpen (char *config) open a video source. arVideoClose (void) close the video source. After your application has finished using a video stream, this function must be called to close the link to the input source, and free resources associated with the capture operation. arVideoCapStart (void) start the capture of video. arVideoCapStop (void) stop the capture of video. arVideoCapNext (void) call for the next grabbed video frame. arVideoGetImage (void) get the video image. arVideoInqSize (int *x, int *y) get the video image size, in pixels. ar2VideoDispOption (void) display the video option (multiple video inputs) ar2VideoOpen (char *config) open a video source (multiple video inputs) ar2VideoClose (AR2VideoParamT *vid) close a video source (multiple video inputs) ar2VideoCapStart (AR2VideoParamT *vid) start the capture of a video source (multiple video inputs) ar2VideoCapNext (AR2VideoParamT *vid) call for the next grabbed video frame of a video source (multiple video inputs) ar2VideoCapStop (AR2VideoParamT *vid)

AR_DLL_API int

AR_DLL_API int

AR_DLL_API int

AR_DLL_API int

AR_DLL_API int

AR_DLL_API ARUint8 *

AR_DLL_API int

AR_DLL_API int

AR_DLL_API AR2VideoParamT *

AR_DLL_API int

AR_DLL_API int

AR_DLL_API int

AR_DLL_API int

stop the capture of a video source (multiple video inputs) AR_DLL_API ARUint8 * ar2VideoGetImage (AR2VideoParamT *vid) get a video image from a video source (multiple video inputs) AR_DLL_API int ar2VideoInqSize (AR2VideoParamT *vid, int *x, int *y) get the video image size of a video source (multiple video inputs)

Detailed Description
ARToolkit video subroutines. This library provides multi-platform video input support for ARToolKit. It abstracts access to hardware video input available on different machines and operating systems. The actual supported platforms (and the driver/library used) are:

Windows: with Microsoft DirectShow (VFW obsolete). Linux: with Video4Linux library, GStreamer, IEEE1394 camera library and DV camera library. Macintosh: with QuickTime. SGI: with VL.

This library provides two sets of functions, depending on whether your program needs to use only one video stream, or more than one video stream. These two sets are functionally identical.

one camera: use the arVideo* functions. multiple cameras: use the ar2Video* functions.

More information on establishing video streams is available in the ARToolKit manual. Remarks: The arVideo* functions maintain the current video stream in a global variable and internally call the ar2Video* functions. History : modified by Thomas Pintaric (pintaric@ims.tuwien.ac.at) to add a fully transparent DirectShow Driver. modified by Hartmut Seichter (hartmut@technotecture.com) to add GStreamer video support Author: Hirokazu Kato kato@sys.im.hiroshima-cu.ac.jp Atsishi Nakazawa nakazawa@inolab.sys.es.osaka-u.ac.jp Thomas Pintaric pintaric@ims.tuwien.ac.at (Windows DirectShow video support). Philip Lamb phil@eden.net.nz (Macintosh Quicktime video support). Hartmut Seichter hartmut@technotecture.com (GStreamer Video support) Version:

4.3b Date: 03/02/02

Function Documentation
AR_DLL_API int ar2VideoCapNext ( AR2VideoParamT * vid ) call for the next grabbed video frame of a video source (multiple video inputs) Companion function to arVideoCapNext for multiple video sources. Parameters:

vid a video handle structure for multi-camera grabbing

AR_DLL_API int ar2VideoCapStart( AR2VideoParamT * vid ) start the capture of a video source (multiple video inputs) Companion function to arVideoCapStart for multiple video sources. Parameters:

vid a video handle structure for multi-camera grabbing

AR_DLL_API int ar2VideoCapStop( AR2VideoParamT * vid ) stop the capture of a video source (multiple video inputs) Companion function to arVideoCapStop for multiple video sources. Parameters:

vid a video handle structure for multi-camera grabbing

AR_DLL_API int ar2VideoClose( AR2VideoParamT * vid ) close a video source (multiple video inputs) Companion function to arVideoClose for multiple video sources. Parameters:

vid a video handle structure for multi-camera grabbing.

AR_DLL_API int ar2VideoDispOption( void ) display the video option (multiple video inputs) Companion function to arVideoDispOption, for multiple video sources.

AR_DLL_API ARUint8* ar2VideoGetImage( AR2VideoParamT * vid ) get a video image from a video source (multiple video inputs) Companion function to arVideoGetImage for multiple video sources. Parameters:

vid a video handle structure for multi-camera grabbing

AR_DLL_API ar2VideoInqSize int( AR2VideoPa vid, ramT * int * x, int * ) get the video image size of a video source (multiple video inputs) Companion function to arVideoInqSize for multiple video sources. Parameters:

vid a video handle structure for multi-camera grabbing

AR_DLL_API AR2VideoParamT* ar2VideoOpen ( char * config ) open a video source (multiple video inputs) Companion function to arVideoOpen for multiple video sources. This function can be called multiple times to open multiple video streams. The maximum number of streams is dependent on the operating system and the performance characteristics of the host CPU and video capture infrastructure. Parameters:

config string of the selected video configuration.

Returns: If the video path was successfully opened, this function returns a pointer to an AR2VideoParamT structure, an opaque structure which holds information and configuration for the video stream. This paramater should then be passed to other ar2Video* functions to specify which video stream is being operated upon. If the video path was not successfully opened, NULL will be returned. s AR_DLL_API int arVideoCapNext ( void ) call for the next grabbed video frame. This function should be called at least once per frame. It has several purposes, depending on the operating system. It allows the video driver to perform housekeeping tasks and also signals to the video grabber that your code has finished using the most recent video frame returned by arVideoGetImage(), and that the video driver may re-use the memory occupied by the frame. The effect of this call is operating-system dependent. The best place to call this

function is immediately after you have finished displaying the current video frame, i.e. after calling arglDispImage() or argDispImage(). Remarks: On some operating systems, this function is a no-op. Returns: 0 if successful, -1 if the video driver encountered an error. AR_DLL_API int arVideoCapStart( void ) start the capture of video. This function starts the video capture routine. Remarks: On some operating systems, capture operations may run in a separate execution thread. This call starts that thread. this function coupled with arVideoCapStop, can be call many times in your program (this may reduce the CPU load when video processing is stopped or for long and critical operations). Returns: 0 if successful, -1 if the capture couldn't be started. AR_DLL_API int arVideoCapStop( void ) stop the capture of video. This function stops the video capture routine. Remarks: On some operating systems, capture operations may run in a separate execution thread. This call stops that thread. this function coupled with arVideoCapStart, can be call many times in your program (this may reduce the CPU load when video processing is stopped or for long and critical operations). Returns: 0 if successful, -1 if the capture couldn't be stopped. AR_DLL_API int arVideoClose( void ) close the video source. After your application has finished using a video stream, this function must be called to close the link to the input source, and free resources associated with the capture operation. Returns: 0 if shut down successfully, otherwise -1. AR_DLL_API int arVideoDispOption ( void )

display the video option. The video configuration options vary by operating system and platform. This function outputs to the standard output the options available on the current OS and platform. Returns: 0 AR_DLL_API ARUint8* arVideoGetImage ( void ) get the video image. This function returns a buffer with a captured video image. The returned data consists of a tightly-packed array of pixels, beginning with the first component of the leftmost pixel of the topmost row, and continuing with the remaining components of that pixel, followed by the remaining pixels in the topmost row, followed by the leftmost pixel of the second row, and so on. The arrangement of components of the pixels in the buffer is determined by the configuration string passed in to the driver at the time the video stream was opened. If no pixel format was specified in the configuration string, then an operating- system dependent default, defined in <AR/config.h> is used. The memory occupied by the pixel data is owned by the video driver and should not be freed by your program. The pixels in the buffer remain valid until the next call to arVideoCapNext, or the next call to arVideoGetImage which returns a non-NULL pointer, or any call to arVideoCapStop or arVideoClose. Returns: A pointer to the pixel data of the captured video frame, or NULL if no new pixel data was available at the time of calling. AR_DLL_API arVideoInqSize int( int x, * int y * )

get the video image size, in pixels. This function returns the size of the captured video frame, in pixels. Parameters:

x a pointer to the length of the captured image y a pointer to the width of the captured image
Returns: 0 if the dimensions are found successfully, otherwise -1 AR_DLL_API int arVideoOpen( char * config ) open a video source. This function opens a video input path with the driver (and device) present on your platform. According to your operating system and the hardware the initialization will be different : a generic string structure is used for this issue. This function prepares the video stream for

capture, but capture will not actually begin until arVideoCapStart is called. Parameters: documentation for more information on this parameter. Returns: 0 if successful, -1 if a video path couldn't be opened

config string of the selected video configuration. See the video configuration