From Sports Tracking to Surveillance Tracking…

In the post Augmented TV Sports Coverage & Live TV Graphics, we saw how sports broadcasters increasingly make use of effects that highlight tracked elements in a sporting event, from the players in a football match to the ball they are playing with. So how else might we apply such tracking technologies?

According to Melvin Kranzberg’s first law of technology, “Technology is neither good nor bad; nor is it neutral”. In the sports context, we may be happy to thing that cameras can be used to track – and annotate – each player’s every move. But what if we take such technological capabilities and apply them elsewhere?

EXERCISE: As well as being used to support referees making decisions about boundary line events, such as whether a tennis ball landed “in” or “out”, or whether a football crossed the goal line, how might virtual boundaries be used as part of a video surveillance system? To what extent could image tracking systems also be used as part of a video surveillance system?

One way of using virtual boundaries as part of a video based surveillance system might be to use them as virtual trip wires, where breaches of a virtual boundary or fence can be used to flag a warning about a possible physical security breach and perhaps start a detailed recording of the scene.

ASIDE: The notion of virtual tripwires extends into other domains too. For example, for objects tracked using GPS, “geo-fences” can be defined that raise an alert when a tracked object enters, or leaves, a particular geographic area. The AIS ship identification system used to uniquely identify ships – and their locations – can be used as part of a geofenced application to raise an alert whenever a particular boat, such as a ferry, enters or leaves a port.

Video surveillance might also be used to track individuals through a videoed scene. For example, if a person of interest has been detected in a particular piece of footage, they might be automatically tracked through that scene. If multiple cameras cover the same area, persons of interest may be tracked across multiple video feeds, as described by Khan, Sohaib, Omar Javed, Zeeshan Rasheed, and Mubarak Shah. “Human tracking in multiple cameras.” In Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on, vol. 1, pp. 331-336. IEEE, 2001.

Where the environment is rather more constrained, such as an office block, tools such as the FXPAL DOTS Video Surveillance System allow for individuals to be tracked throughout the building. Optional filters also allow tracking or identification based on the colour of clothing, which may be meaningful in an environment where different colour uniforms or protective clothing are used to identify people by role – and perhaps by different access permission levels.

Once a hard computer science problem to solve, a wide variety of programming libraries and tools now support object identification and tracking. There are even Javascript libraries available, such as tracking.js, that are capable of tracking objects and faces streamed from a laptop camera using code that runs just in your browser.

Tracking is one thing – but identification of tracked entities is another. In some situations, however, tracked entities may carry clearly seen identifiers – such as car number plates. Automatic Number Plate Recognition (ANPR) is now a mature technology and is widely deployed against moving, as well as stationary, vehicles.

With technology firmly in place for tracking objects, and perhaps even identifying them, analysts are now turning their attention to systems that are capable of automatically identifying different events, or behaviours, within a visual scene, a step up from the simple “threshold crossing” behaviours used to implement virtual tripwires.

Once behaviours have been automatically identified, the visual scene may be overlaid with a statement of, or interpretation of, those behaviours.

Many technologies are developed for a particular purpose, but that does not prevent them being adopted for other purposes. When new technologies emerge, there are often many opportunities for businesses and entrepreneurs to find ways of using those technologies either on their own or in combination with other technologies. However, there are also risks, not least that the technology is used for a harmful purpose, or that that we do not approve of. More difficult is to try to predict what the consequences of using such technologies widely may be. As technologists, it’s our job to try to think critically about how emerging technologies may be used, whether for good, or evil, and contribute to debates about whether we want to approve the use of such technologies, or limit them in some way.