REAL TIME TRACKING SYSTEM USING DIGITAL IMAGE PROCESSING

by S.BALAJI Regd No: 24EC012 Sbalajichennai23@yahoo.co.in S.S.ARUN Regd No: 24EC010 arunchennai23@yahoo.co.in

Third Year

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING ST .JOSEPH S ENGINEERING COLLEGE CHENNAI 600 119. TAMILNADU

Contact Address: S.BALAJI S/o. Mr. B.SRINIVASAN 141 ALWARTHIRUNAGAR ANNEXE CHENNAI 87 TAMILNADU PHONE NO 044 23773638 9841590008

ABSTRACT
Tracking of moving objects, for measuring motion-parameters and obtaining visual records of the object in various stages of motion, is an important requirement in many fields spanning both military and industrial applications. This paper describes the design of a real-time raster-scan TV/FLIR camera based tracking system for fast moving objects using image processing techniques. A tracking-window is placed over the approximate location of the target, either automatically or by a human operator. Further, the tracing window also serves as a spatial band pass filter by restricting the target search region to the immediate vicinity of the target. The target image is segmented from the background using a new adaptive cooperative segmentation technique that utilizes background histogram in the immediate vicinity of the targetimage and edge strengths of the pixels within the tracking window. The target state is predicted over the next few image frames for generating orientation commands for the tracking-mount. The tracing-system successfully tracks targets even under lowcontrast and noisy imaging conditions.

INTRODUCTION Traditional methods of tracking, such as radar tracking, do not provide any photographic record of the target being tracked and are also known to interfere with the operating environment due to transmission of high-power electromagnetic waves. Increased availability of high speed image-processing hardware and efficient algorithms have now made it possible to attempt designing a real time video tracking system capable of tracking fast moving objects at close to medium ranges. In general there are two different approaches to object tracking a) Recognition based tracking In recognition-based tracking, the object pattern is recognized in successive image-fames and tracking is carried-out using its positional information. The advantage of this method of tracking is that it can be achieved in three dimensions, and the translation and rotation of the object can also be estimated. The obvious disadvantage is that only recognizable object can be tracked by this strategy. Object recognition is high level operations that can be computationally very costly to perform. Thus, the performance of the tracing system is limited by the efficiency of the recognition method, as well as the types of objects recognizable. b) Motion based tracking Motion based tracking systems are significantly different from recognition-based systems is that they rely entirely on motion detection to detect the moving object. They have the advantage of being able to track any moving object regardless of size of shape. Motion-based techniques can be further sub-divided into optic-flow tracking methods and motion-energy methods. Optic flow is the distribution of apparent velocities of movement of brightness patterns in an image. Since determining a complete optic-flow field quantitatively is both expensive and ill-posed. Another method of motion-tracking is the Motion energy detection. By calculating the temporal derivative of an image and thresholding at a suitable level to filter out noise, we can segment an image into regions of motion and inactivity. Although the temporal derivatives can be estimated by a more exact method, usually it is estimated by simple image subtraction, but this method of motion detection is subject to noise and yields imprecise values. One disadvantage of this method is that the pixel motion is detected but not quantified. Therefore, one cannot determine additional information, such as the exact location of the moving object. Another 3

disadvantage is that the techniques discussed are not suitable for application on active camera systems without modification. However, this still leaves us without any method of accurately locating the moving object which is very important for our problem. PROBLEM FORMULATION Here we consider design of a system for tracking of fast military targets (such as missiles/aircrafts) using an active raster-scan sensor (TV/Forward Looking Infra-Red-FLIR) with standard PAL video output, mounted on a pan/tilt platform. We have aimed at designing a tracking system capable of discriminating the target to be tracked with 50 image frames per second. The system should be able to locate the center of mass of the target projection on the image plane within about 1 percent of the sensor field-of-view (FOV) in a rapidly varying environment. It should generate a predicted observation angle for the next observation, and output the angular displacement of the target within the sensor FOV within 20msec after the observation was made. The system is also required to acquire targets entering the sensor FOV automatically without human intervention. In the proposed system, these requirements have been met, resulting in a real-time application of the imageprocessing technology in target tracking. OVERALL SYSTEM ARCHITECTURE The proposed real-time tracking system consists of the following subsystems, as in fig.1 1. Raster scan TV/Forward Looking Infra-Red (FLIR) sensor. 2. Tracking mount. 3. Mount-controller 4. Tracker processor. 5. Video-image processor 6. Input/output processor. The sensor is a high quality raster-scan Vidicon/CCD TV or FLIR camera used for obtaining accurate angular data of the position of an object in the sensor FOV. It is positioned by the tracking-mount which responds to azimuth and elevation drive commands from a mount controller. The camera output is digitized in the videoimage processor to 8 bit gray level pixels (corresponding to 256 different gray-levels) 4

Fig.1 Real Time Tracking System Block Diagram

and transferred on a parallel bus for distribution to input and output frame buffers in real-time. The digitized image frames are processed to adaptively segment each frame into background (or foreground) and target regions. The tracker processor next analyses the structure of the target region with respect to the previous frames, to establish the target position in the sensor FOV and also decides the level of confidence in the error data. It also computes the tracking window position and its size for the next frame, based on the tracking- errors and the segmented target size. The mount controller unit then filters the tracking errors to reduce the effects of the measurement noise and process noise and also predicts the target position over the next few frames. The predicted position of target is utilized to generate orientation commands for the tracking mount such that the target is brought to the center of sensor FOV. An I/O processor, having suitable interface with a human operator for test is changed interactively. It also provides tracking-data to and accepts operational-data from any remote central tracking system. The overall functional block diagram of the proposed tracing system, showing flow of data and its contents at various stages, is shown in fig 2.

START OF NEW FRAME

TRACKING ERRORS + PREV TW POSN

MODE ?
OPERATORS CONSOLE

DIFFERENCE-IMAGE PROCESSING Input Video TW IMAGE PROCESSING AND SEGMENTATION From Camera

SEGMENTED TARGET IMAGE TARGET-IMAGE PROCESSING TW: Tracking Window

TRACKING ERRORS ERROR FILTERING AND STATE PREDICTION TRACKING-MOUNT CONTROLLER PREDICTED ERRORS

Fig.2 Functional block diagram of Tracking System showing data flow and output after each stage. VIDEO-IMAGE PROCESSING The scene in the TV/FLIR camera FOV is digitized to form an image f(x.y), x=[1,2,3,4..n],y=[1,2,3,4,.m]. As the sensor scans the scene, the video signal is digitized at m equally spaced points across each horizontal scan-line. During each video field, there are n horizontal scan-lines which generate an n*m discrete matrix representation at 50 frames/sec. A resolution of m=512 pixels per standard PAL TV line results in a pixel generation at the rate of 8 million pixels per second. Thus every 125nsec, pixel intensity is digitized and quantized into 8-bits. IDENTIFICATION OF AREAS OF INTEREST The basic assumption of segmentation and motion detection algorithms is that the target image has some video intensity not present in the background in immediate vicinity of the target. Identification of areas of interest (i.e. positioning of a tracking window) within frames of sequence to isolate the moving target from the stationary background component is an important requirement since processing of the entire image-frames for segmenting the moving target may be computationally very costly. However, it is required to process entire frames when even approximate location of the moving target in the sensor FOV is not known. Placing a tracking window about the approximate location of the target facilitates sampling of background intensities relatively close to the target to accurately characterize the background intensity 8

distribution in the vicinity of the target. Further, the tracing window also serves as a spatial band pass filter by restricting the target search region to the immediate vicinity of the target. Therefore, differenceimage processing is performed in the acquisition phase when approximate location of the moving target in the sensor FOV is not known. Since the camera is initially stationary, there is no need to perform the camera-motion compensation before motion-energy detection. When the moving target has been acquired, the position of the tracking window for the next frame in the sequence is computed using a moving window algorithm which allows the tracking window to move with the target. This is to reduce the effects of the dynamic lags on the servo system while tracking an object exhibiting high angular motion. This algorithm utilizes the measured tracking errors in the previous frame and the confidence level computed for these errors. Since the displacement of the tracking window within the sensor FOV is known, the angular position of the object relative to the center of sensor FOV can be determined. An automatic adaptive window algorithm is used to adaptively adjust the tracking window size to encompass the object-detail based on the target size and spread measured by the tracker-processor facility for manually selecting the tracking window size, both initially and during tracking, is also provided in the system.

Segmentation of moving target from the background This is perhaps the most critical step in designing a reliable and accurate tracking system. Depending on the selection of tracking strategy tracking window is subdivided into two regions. 1. Target region 2. Background region The tracking window regions for different tracking strategies are shown in fig.3. If the target is expected to cover a significant portion of the sensor FOV, then one of the four partitioning strategies which allow the target image to extend beyond the tracking window boundaries is selected. However, when the target image is relatively small, then the more accurate Centroid mode portioning is selected. This adaptive

subdivision of tracking window also permits tracking of large targets which extend the tracking window boundaries by selecting the appropriate tracking strategy. During each field, the feature histogram is accumulated for those regions of tracking window marked as background into hB(x). Since hB(x) is non-negative and finite, it can be made to satisfy the requirements of a probability assignment function by normalization. hB <- hB/r
TOP LEFT TOP RIGHT

: Background
Region : Target Region CENTROI D

BOTTOM LEFT

BOTTOM RIGHT

Where r is the total number of background pixels. Depending upon this background histogram, target-cost Ct = [0,1] and background-cost Cb=[0,1] are assigned to each pixel (x,y) within the tracking window. A high target-cost Ct assigned to a pixel (xi, yi) belonging to the target image is low. Similarly, the background-cost Cb associated with a pixel indicates the likelihood i.e does not belong to the background.

10

Fig.3 Tracking window regions in different Tracking Strategies showing tracking window placement and Selection for different types of targets. A pixel (x,y) in the tracking window is either assigned the highest target-cost (=1) when its intensity level lies within the range il to ih (fig.4), or the lowest target-cost(=0) when its intensity level is less than (i1-w) or more than (ih+W). If the intensity level falls within the transition region, shown shaded in fig.4, it is assigned a target-cost proportional to its distance from the background histogram spread limits (i.e. il to ih). The spread limits of background histogram il and ih are derived automatically such that h(ih)=h(il)=h(imax)/K1 Where K 1 is a parameter selected based on the tracking environment. In case the background region is uniformly homogenous (like cloudless blue sky), the dispersion leveling the unimodal background histogram will be less. In such cases K1 will be assigned a higher value. In our simulations K1 has been chosen between 20 to 30 to yield the best results. The width W of the transition region is derived adaptively as W=(ih-il)/K2+1 Where K2 is a constant (typically K2=10). The choice of K2 is based on the expected target-background contrast condition. The transition region is broad when the expected contrast between the target and the background is poor and narrow when the contrast is good. The background-cost (Cb) assigned to each pixel (x,y) in the tracking window is equal to the complement of its associated target-cost Ct. Cb(x,y)=1.0-Ct(x,y) The target-cost and background-cost assigned to each pixel are optimized based on the previous image frame segmentation results during the automatic tracking phase. The optimization method assumes that the target remains almost stationary in the tracking window when the system is in auto-tracking mode. The target-cost/background-cost for each pixel is modified adaptively as If ( fprev(x,y)=target pixel) Cb(x,y) <-Cb(x,y)+C1 Ct(x,y) <-Ct(x,y)+ C1 Where C1=[0,1] is a constant derived adaptively depending on the measured trackingerrors for the previous frame. Thus, C1 will be large if tracking-errors are small and 11

vice-versa; and will be zero if the confidence-level for the previous frame lies below a certain pre-defined threshold, thereby avoiding enhancement of a false target.

h(i) h(i) Transition Region h(i) W i

Target Cost

Background Histogram

Transition Region

1.0

Background

Pixel Intensity

Fig 4: Tracking window regions for different tracking strategies CONCLUSION

12

An innovative design of a fast moving target tracking system is presented in this paper. A novel strategy of adaptive co-operative segmentation based on background histogram and pixel edge strength (within the tracking window) has been successfully developed. This computationally efficient scheme yields excellent results in segmenting the target in an image frame. The targets in each frame sequence can be successfully segmented, accurate tracking errors are estimated and repositioning and resizing of tracking window is satisfactorily performed even in noisy image frames. To conclude, the scheme presented here has been observed to be extremely effective in target tracking even under adverse imaging conditions. REFERENCES Y aloimonos & D Tsakiris, on the mathematics of visual tracing , image vision computing, vol 9 no.4 pp 235, 251, 1991 BKP Horn & Brian G Schunk, determining optical flow Artificial Intelligence, vol 17 pp 185-203,1981 IETE journal research-1997.

13