Development of Video Retrieval Technology in Big Data Era

With the continuous development of the "harmonious society" and "safe city" construction, the country has entered a period of high tide for the construction of security facilities. Surveillance cameras have spread over every street in China's land, constantly monitoring and videotaping around the clock. However, having related videos does not mean finding the target information. Finding videos and analyzing videos often takes a lot of time and manpower. How to find relevant information more convenient and labor-saving in massive video? Now, with the growing demand for security intelligence, video retrieval technology has also been rapidly developed.

With the large-scale application of video surveillance systems in sensitive areas such as public security and transportation, there have been reports that electronic eye has helped solve the case. On the one hand, these examples reflect the social value of the video surveillance system from the front; on the other hand, it also leads to more in-depth requirements for video retrieval. In the actual application process, users often want to quickly find people or objects with obvious features from the massive tens of thousands of camera video recordings. The traditional video retrieval methods based on preset alarms and time are often ineffective for such deep-seated needs, and often have the regret of “scratching”. Therefore, how to quickly and accurately perform intelligent retrieval becomes an important issue for large-scale video surveillance systems. Question.

Video surveillance retrieval is aimed at the video surveillance platform to search the user for video surveillance information. The user enters the search information of the related monitoring event, and the system calculates and returns the search result.

Video monitoring focuses on the retrieval of video-based data. Its main purpose is to locate and find the cause of an event and the associated development process. The key information data of an event includes: time, location, person or object of the dominant event, and image and sound information. The more information the retrieval conditions convey, the more accurate the positioning, and the simpler the retrieval algorithm. On the contrary, the simpler the information transmitted by the retrieval conditions, the more vague the positioning and the more difficult the retrieval algorithm is when it is desired to accurately position. In general, the user expects the search conditions to be simple and accurate at the same time.

Common video retrieval technologies fall into two major categories: selected streams and selected times. For the selected code stream, it can be directly searched by the camera's name, number, etc., and select the stream of interest. Therefore, some specific applications can be derived: for example, knowing the geographical location, searching in the equipment resource hierarchy; knowing the equipment number, performing a quick search of the number; knowing the name, performing a fuzzy search on the name to determine the specific code stream; With the aid of an electronic map, the approximate location of the device is known and viewed on the map to select the stream. For the selected time, it is often performed on the basis of the selected stream, which may be the selection of one or more streams, accurate to the retrieval in seconds; it may also be all streams, paying attention to a certain The video at the beginning of the time point corresponds to multiple simultaneous simultaneous playback searches. The current mainstream slice retrieval, snapshot retrieval, and quick preview retrieval based on the time axis are all included in this category. First select the stream and select the time.

Slice search means that the user knows which position the object, person or car is specifically interested in, which stream it corresponds to, but it is uncertain when the change is found, appearance, disappearance or other state changes. The slicing search can be understood as a dichotomy, which divides the specified code stream in time and divides the snapshots to find the moment when the object of interest has changed. Snapshot search is based on the first selected moment, and then select a set of code streams or all streams to snapshot display, through the snapshot search. This application mainly solves the problem of knowing that some people, vehicles, objects, etc. have appeared at a certain time, but it is not clear at which position, and it can be quickly retrieved through a snapshot search. Based on the time axis drag search, it is the basis of the selected stream. By dragging the time axis to determine the search at the time point, the advantage is that the search rate can be freely controlled and the search for the selected stream can be completed quickly.

Because the security monitoring has inherent properties such as unpredictable objects and multi-system linkage, the retrieval based on image recognition technology and event-based retrieval become the right and left arms of security surveillance video retrieval.

Based on image recognition technology retrieval

As security video surveillance, accidents are often unpredictable. For the past video recordings, it is often necessary to set specific rules for the specific accidents and to extract the video data of interest according to certain specified conditions. These types of conditions are mainly divided into two categories, one is behavior-based, such as cross-line, retrograde, lost, leftovers, defects, regional invasion, population statistics, speed test, smoke and other functions of the analysis, the other is Based on recognition, such as face recognition, license plate recognition and other technologies. The above two categories are all based on image analysis, setting specific regions of interest for specific video data, setting rules, and performing image analysis retrieval techniques. This kind of technology has a higher requirement for the algorithm of image analysis.

Event-based retrieval

Security monitoring systems are often not a single video surveillance system. They will incorporate some third-party systems, such as access control, alarm, and fire protection systems, as well as some production-related systems, or some front-end or back-end intelligent analysis systems. . These systems combine to form a three-dimensional security system with a high level of security. And when these systems alarm video is often the user's most concern. For such searches, they can be summarized as event-based searches. The feature of this type of search is that the user pre-defines some conditions or rules. When the video meets these conditions or rules, it will be determined as a type of event, and the corresponding stream and time of this event will be stored one by one. For this kind of event-based retrieval, the relevant video can be retrieved only afterwards according to some conditions of the event. The feature of this type of search is that it can quickly retrieve specific video that meets a certain type of rule event and is highly targeted.

In order to more accurately find the determined event, the user can enter multiple search conditions at the same time to search, called multi-dimensional search. The multidimensional search can find the required video data more precisely on the premise that the user knows multiple conditions. Multidimensional search looks simple, but to achieve multidimensional search must have a powerful search engine. This is a feature that many software platform vendors ignore.

Fundamentally, video retrieval technology applied to security monitoring is based on intelligent video analysis technology. Intelligent video analysis technology refers to the use of computer vision visual analysis technology to analyze and track the targets that appear in the camera scene by separating the background and the target in the scene. In recent years, the term “big data” has been increasingly mentioned and used. It involves various industries and people use it to describe and define the massive data generated during the information explosion era. The introduction of intelligent video analysis technology can greatly enhance the retrieval efficiency and hit rate of the original massive surveillance video storage system. With the rapid development of the industry, the introduction of intelligent analysis technology will become a trend in the massive monitoring video storage system.

Intelligent video analysis technology is derived from computer vision technology. Computer vision technology is one of the branches of artificial intelligence research. It can establish a mapping relationship between image and image content description, so that the computer can understand video by digital image processing and analysis. The contents of the screen. In the era of big data, people pay more and more attention to intelligent video analysis technology. Intelligent video analysis relies on video algorithms to analyze the video content, extract key information in the video, mark or process related events, and form monitoring methods for corresponding events and alarms. People can quickly retrieve through various attribute descriptions. If the camera is regarded as the human eye, and the intelligent video surveillance system can be understood as the human brain, the intelligent video analysis technology uses the powerful computing capabilities of the processor to perform high-speed analysis on the massive data in the video screen to obtain the information that people need.

With the arrival of the era of big data, how to find or find out the video information that customers need most is a problem. Intelligent video analysis should be of great use in the future. Today, intelligent analysis products have been blooming in the security monitoring industry, and many products have been used in commercial and living environments. At present, intelligent video analysis technology mainly analyzes real-time video images and plays an early warning role. Application in this area has gradually matured. The following are the smart analysis features available for mainstream vendors:

Smart Search (Video Enrichment): This technology can be used for quick search afterwards. It can search for videos within a few days in a few seconds, or the same color vehicle or the same facial picture in the search time, and it can be hours or even days. Dozens of days of video recording are shortened to a few minutes, which greatly saves time for subsequent search.

ROI (Region of Interest) + ePTZ: Select ROIs for quick control of PTZ cameras, allowing users to navigate through specific areas of the megapixel camera without sacrificing resolution.

Transcoding: Provides smooth Internet video browsing, easy remote management, instant browsing, playback and PTZ control via iPad/iPhone or IE browser.

Custom user notification method: Support any map downloaded and stored in HTML format, such as Google Maps, can be re-edited to configure the camera; when the alarm occurs, the map will automatically pop up, users can directly click on the map The camera icon, to view the real-time screen; can also instantly view and play back the same camera video screen.

Intrusion detection technology (eg, cordon, alert zone, target presence, etc.): Achievable prevention, combined with video abstraction technology enables faster video content retrieval.

Behavior detection technology (such as: strenuous action, trailing, running, squat, reverse movement, etc.): It can realize the detection of abnormal behaviors and realize event prevention or alarm during events.

Due to the business needs of different industries and the differences in application scenarios, it is difficult to have an intelligent video analysis technology to meet the needs of different industries. On the contrary, intelligent video analysis technology will have more and more customized features and requirements in different industry applications. The specific performance is as follows:

l. Financial industry

In the financial industry applications, it is possible to combine ATM panel-based cameras with intelligent video analysis technology to automatically detect abnormal events such as the installation of card readers, sticking false advertising slips, left-over purses, or keys; using face analysis to automatically discover Faced with withdrawals, many people at the same time withdrawals and other unusual events; can also be combined with the ATM surveillance cameras in the self-help hall, the use of intelligent technology in a timely manner to track tailings, fights and other activities that threaten customers. In addition, it can also be combined with the bank's own business to launch an automatic authorization system based on face recognition.

2. Public security industry

In the public security industry application, the face recognition technology can be used to realize real-time capture of people's faces in the security checkpoints of stations, hotels, KTVs and other entertainment venues, and the ability of suspects to real-time control and escape; using face database retrieval technology can achieve The rapid identification of personnel status and the second-generation database check function, the discovery of duplicated personnel, to eliminate the situation of one-person multi-certificate; in the process of public security criminal investigation, the video-enrichment technology can be used to enrich the relevant crime video summary, using the search technology Achieve suspicious target matching search in different video clips, thus helping case handlers quickly find clues to solve cases, improve the efficiency of detection.

3, transportation industry

In the application of traffic monitoring system, intelligent video analysis technology can locate, identify and track targets in dynamic scenes through real-time analysis of surveillance video images, and analyze and judge the behavior of targets so that they can be done in time when an abnormal situation occurs. Respond to make early detection and active prevention, and help security personnel effectively avoid or effectively deal with ** or sudden incidents.

In addition to real-time prevention, intelligent analysis also plays an important role in improving the efficiency of video retrieval. Based on intelligent analysis, video surveillance video data can be automatically added to various analysis data, including various types of intelligent alarm events, face recognition information, character morphology information, etc. Based on these data, traffic security personnel can be based on various events and information. Search related videos efficiently and easily, freeing them from massive storage data and improving efficiency.

4, plant monitoring

With the popularization and deepening of the modern enterprise system in our country, and the informatization construction of the enterprise is deepening, it is a general trend to use intelligent video analysis technology to carry out security prevention work for enterprises. In the modern enterprise, the factory implements a video surveillance system, and the security and protection department can implement real-time weather monitoring at the entrance of the company's factory, factory buildings, office buildings, perimeter walls, and warehouses.

Intelligent video analysis technology can provide the function of virtual cordon for plant monitoring. On the tangible or intangible boundaries such as the factory's border lines and cordon lines, the technology can directly complete the perimeter warning alert task through the screen content identification of the surveillance camera. . The user is free to define a virtual cordon directly on the video surveillance screen. Once a target meets the alert rules, the device immediately sends a real-time alert to the monitoring personnel through the video surveillance system.

In addition, intelligent video analysis technology can also realize the property protection within the factory. Through the intelligent analysis module in the high-definition monitoring and management platform located in the management center, the system can intelligently analyze the image of warehouses, parking lots and other places to achieve suspicious personnel. Accurate judgment of property safety events such as abnormal movement of objects, and linkage alarm system to issue alarm signals in time.

Although the intelligent video analysis technology has been rapidly developed, it is undeniable that there are some problems such as false positives, missing reports, and lack of judgment in behavior. In order to improve the application of intelligent video analysis products, while improving the core algorithms, the current stage of intelligent video analysis technology will continue to work toward the following aspects:

First of all, it is necessary to improve the adaptability of the video analysis function module and take into account the needs of real-time, so as to adapt it to a more complex and changeable field environment.

Secondly, it combines with next-generation technologies such as intelligent video surveillance technology, cloud computing, and the Internet of Things to expand more intelligent analysis technologies.

Third, based on the needs of industry users, we will launch a subdivided market service to develop more targeted products to meet the needs of the on-site environment and improve the product's ability to discriminate.

Fourth, we must strengthen research and development efforts, continue to improve product performance indicators, improve product detection rates, and reduce false positives and false negatives.

Overall, intelligent video analytics can do many things. So video analysts and end users need to communicate effectively. Since intelligent video analysis is still a relatively new technology, the circle that knows this technology in the country currently only expands to the integrator level. Therefore, many application scenarios suitable for video analysis technology have yet to be developed in the market. But one thing is clear: companies must master core technologies and have independent research and development capabilities. The market for smart video analytics is composed of many subdivided and small markets, and new applications are constantly emerging. This will be an obvious feature of the market in the foreseeable future.