​

MANY DIGITAL video images are currently compressed using the MPEG-4 and MPEG-2 standards. And MPEG-7 may sound like just another compression term, a way to bundle video and send it back over the network. However, MPEG-7 is not used to compress video but rather to describe video data in a sophisticated way.

MPEG-7, also known as Multimedia Content Description Interface, is a standard that was established by the Moving Picture Experts Group. It uses XML (extensible markup language) to store what is called metadata (which is a description of the video data), which can then be attached to a time code and facilitate interoperable searching based on content.

“A computer can ‘look’ at an image and recognize certain features in it,” says Adam Lindsay, a researcher in the computing department at Lancaster University in the United Kingdom. Lindsay means that MPEG-7 can pull information from the video data, such as a person in a red shirt walking into the frame, and code it into a database along with the frame number or time code. Then, if users want to search that database for all the video where someone with a red shirt walks into view, they can.

Without MPEG-7 or one of the other metadata analysis tools that many analytics companies use, this type of specific content description has to be entered by hand (such as “tagging” in social networking applications). In that case, it is often limited to certain specific analytic “events” like a left-behind object or someone crossing a tripwire.

Lindsay and other experts see potential for MPEG-7’s use in the surveillance world. For one thing, “the amount of data generated can far outstrip any person's ability to watch it. Even if simple events and features can be marked in the video stream, [MPEG-7] makes seeking anomalies much easier,” says Lindsay.

Another advantage of MPEG-7 is that it is a standard. That is one of the reasons that Eptascape, Inc., uses it for its video analytics software. “It is always better to work with standards,” says Eptascape founder and CEO Marco Graziano. “It simply means that you make it easier for your product to coexist with other products, networks, and services in the market.”

Since it is a standard and not proprietary, MPEG-7 is easily interoperable and others can work with it. Additionally, since it has been around for years, various researchers have developed recommendations on computations, such as search and retrieval, according to James Annesley of Kingston University in London.

Graziano says the folks at Eptascape saw all the work that had already been done with MPEG-7, and that helped them decide to use it rather than develop their own metadata standard. “Why try to recreate in our own way what a large group of brilliant minds had already done?”

Eptascape uses MPEG-7 to provide tracking and identification of certain behaviors and events, including trespassing, crowd formation, and loitering. Additionally, more behaviors can be added if demand is there, says Luis Lajous, Eptascape’s vice president of marketing and sales.

Another use for MPEG-7 is content abstraction and scene re-creation. Eptascape calls this “adaptive transmission.” This is a technique of masking pictures so that privacy of the subject matter is intact. In masking, the scene is “re-created” using the XML data description of the scene, rather than a live video picture. For example, instead of seeing the details of a white truck driving by, a viewer might see a gray box driving down a static street background.

Lajous says Eptascape offers three levels of masking, each with an increasing degree of detail. “In this way, you remove any bias from security monitoring,” says Graziano, who adds that Eptascape was founded on the belief that increased surveillance camera deployment was compromising the public’s privacy.

“If you cannot see the color, sex, age, race, or general appearance of the person,” says Graziano, “the decisions or actions of the operator cannot be based on any discriminating factors.”

The XML re-creation also reduces the bandwidth needed for transmission, which is a helpful option when a site has many cameras going over the same network. And the computer can be set to switch to a live feed if an alert goes off.

Despite the considerable benefits of using MPEG-7, Eptascape appears to be one of the only major security companies currently touting its use for surveillance actions, although other analytics companies do employ metadata. “I think companies are not sure how to use [MPEG-7],” says Annesley. He notes that it does not have clearly defined procedures, but rather “it just defines various descriptors and recommended ways for encoding and decoding.”

Frank Nack, of the Informatics Institute of the University of Amsterdam, says, “the development [of this application] is still very specific…and thus currently expensive.”

According to Lindsay, “It’s a chicken-and-egg problem, really. MPEG-7 can offer a lot of advantages, but the whole machinery can be very demanding to implement.”

Indeed, it took Eptascape a year to develop a way to use MPEG-7 to create metadata in real time, says Lajous.

And MPEG-7 does have competition. Several vendors already have their own proprietary strategies for encoding metadata.

The National Retail Federation’s Association for Retail Technology Standards (ARTS) has launched a video analytics work team that is meeting to discuss standardization of metadata description, among other topics. The group’s mission is “to define XML standards to enable retailers to easily query and integrate information derived from video sources,” according to Ray Coulombe, Cisco Systems, Inc.’s representative on the work team.

The group hopes to start documenting metadata definitions this year. “It’s really a push to try to just get to the point where you’ve got some standards in the industry that people can rally around,” says Coulombe. “Because that’s good for everybody.”