Highlights

 

  • Unified implementation of SWS, combining access control, multilevel threat detection, and multi-tiered alert systems.

  • Enhanced security through the utilization of optical imaging, PIR, and fingerprint sensors in the VSMD application.

  • Real-time threat awareness improved by an innovative alert module that generates alerts based on specific threats.

  • Redefinition of security frameworks through a holistic approach and the integration of an intelligent alert module.

  • Strengthening overall video surveillance security with the incorporation of biometric access control measures.

 

Abstract

This paper presents an advanced Surveillance Warning System (SWS) designed for next-generation video surveillance applications. While contemporary alert systems have improved analytics engines, their binary threat detection using only optical imaging sensors has limitations in low-light conditions, system security, and multilevel threat detection (i.e., low, moderate, and critical). Our approach integrates a novel alert module with an enhanced video analytics engine, contributing to a comprehensive framework integrating surveillance, access control, multilevel threat detection, and multi-tiered alert routing. We develop a Video Surveillance and Motion Detection (VSMD) application utilizing optical imaging, Passive Infrared (PIR), and fingerprint sensors for multilevel threat assessment, access control, and security. Unlike existing systems, we develop an intelligent alert module generating threat-specific alerts transmitted securely through cellular phones, the Internet, and encrypted radio frequency (RF) signals, significantly enhancing real-time threat awareness and response capabilities. Validation against existing solutions highlights the system's adaptability, rapid response, and overall security advancements in video surveillance.

Keywords

Video Surveillance

Multilevel threat

Multi-channel alert

Multi-sensor

PIR

1. Introduction

Surveillance Warning Systems (SWS) play a crucial role in safeguarding public, military, and critical infrastructures by continuously monitoring potential threats. These systems rely on diverse sensors to detect and classify events or activities that jeopardize the security and safety of the monitored area. Effectiveness is contingent upon seamlessly integrating key components: input sensors, analytics engine, alert module, and security features.

1.1. Evolution of Sensors & Analytics Engine

Over the last four decades, closed-circuit television (CCTV) systems have evolved from the first to the current third generation, incorporating intelligent features [1], [2], [3]. Progress in the system's analytics engines, classified into Computer Vision (CV)-based [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16] and Artificial Intelligence (AI)-based [17], [18], [19], [20], [21], [22], [23], [24] categories, primarily contributes to these advancements.

Classical CV-based surveillance systems continue to be prevalent despite the rise of AI. These systems utilize optical imaging sensors and spatial feature-based processing for anomaly detection. To illustrate, Tiwari and Verma applied optical sensors, color-based segmentation, and the Harris interest point detector for weapon detection [4]. Sanoob et al. [5] used optical imaging and passive infrared sensors to detect human and animal motion.

Besides, the literature commonly employs image histogram analysis for human detection [6], [7]. Filonenko et al. [8] proposed a method for smoke detection, anticipating potential explosions, using background subtraction and morphological analysis. Similarly, vehicle identification through histogram analysis is suggested in [9], while Rathor et al. [10] utilized histogram and local binary pattern analysis for animal detection.

The CV-based techniques also include human fall detection through fall motion vector analysis [11] and human gait classification via histogram analysis [12]. Video analytics engines predominantly rely on optical sensors for surveillance, although recent developments integrate infrared sensors [13], geo-location, and smoke sensors [14].

Furthermore, histogram analysis proves useful in diverse application scenarios, such as fire detection [15] and gun detection [16], with acceptable levels of accuracy.

1.2. Focus on Alert Module & Security Features

While there has been considerable attention given to the video analytics engine of a SWS, the alert module associated with it has been overlooked. Traditionally, SWSs have been equipped with non-customizable alert modules, thereby limiting their synergy with the analytics engine [1], [25]. The aforementioned SWSs [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16] are typically designed to detect objects and generate push notifications on computer screens, but they suffer from a limitation in multi-tier routing of alert signals required in a multi-level threat scenario.

There is a growing trend towards Internet-of-Things (IoT)-based alert routing. However, existing SWSs face security concerns, particularly with IoT-based alert modules being vulnerable to eavesdropping and hacking [26]. Additionally, the use of Frequency-Hopping Spread Spectrum (FHSS) communication technology in radio communication exposes SWSs to cyber threats like jamming [27].

To address these security challenges, a combination of FHSS with Advanced Encryption Standard-128 (AES-128) and Global System for Mobile Communications (GSM)-based alert provisions can enhance the security of the alert module [28]. Therefore, for a robust SWS, it is essential to incorporate multi-sensor input and implement multi-tiered routing of encrypted alert signals to ensure secure and reliable operation.

1.3. Research Questions

In the aforementioned context, our research aims to address the critical issue of unauthorized access in contemporary surveillance systems, posing a significant threat to the integrity and reliability of monitoring infrastructure. In light of this, we pose the following research question: 

How can unauthorized access to the surveillance system be prevented to ensure system integrity and reliability?

To tackle this challenge, our proposed system integrates a code-based (i.e., password) and robust biometric authentication option, enhancing access security and fortifying the system against potential breaches.

 

Additionally, we explore the optimal strategy for alert generation, prioritization, and secure transmission in response to potential threats, leading to our second research question: 

When a potential threat is detected, what is the optimal strategy for alert generation, prioritization, and secure transmission?

Our approach involves implementing a multi-tiered alert generation module that prioritizes threat-dependent alerts, ensuring secure and swift response capabilities.

 

Furthermore, we investigate how the surveillance system can ensure robustness against alert signal interference, forming our third research question: 

How can the surveillance system ensure robustness against alert signal interference?

To address this concern, our proposed system incorporates advanced encryption and security measures, safeguarding alert signals and their multi-tier routing, and thus, minimizing the risk of jamming or hacking attempts.

 

1.4. Contribution to Knowledge

To contribute to the existing body of knowledge, this paper introduces an intelligent alert module integrated into an enhanced video analytics engine for advanced SWSs. We propose a general framework (Section 2) capturing access control, multilevel threat detection, and multi-tiered alert routing, forming the basis for the proposed SWS (Section 3) using the Video Surveillance and Motion Detection (VSMD) application. VSMD, tailored for restricted areas, utilizes optical imaging, Passive Infrared (PIR) sensors for human motion tracking, and fingerprint sensors for secure access (Section 3.1). A new alert module is developed to generate multilevel alerts intelligently, routing them securely via mobile, internet-based alerts (GSM, GPRS), and encrypted Radio Frequency (RF) signals with FHSS (Section 3.2). The system seamlessly integrates this alert module with the multi-sensory analytics engine, enhancing versatility, efficacy, and security for rapid response capabilities (Section 4). The original contributions and distinctive technical advancements reported in this paper over existing solutions are as follows:

  • A comprehensive framework is introduced, unifying an SWS with access control, multilevel threat detection, and multi-tiered alert routing.

  • The innovative utilization of the VSMD application, equipped with optical imaging, PIR sensors, and fingerprint sensors, surpasses conventional binary threat detection by enabling multilevel threat assessment, access control, and enhanced security.

  • A novel alert module to intelligently generate threat-level-specific alerts, securely transmitted through mobile, Internet, and encrypted RF (FHSS) signals, thereby enhancing real-time threat awareness with swift response.

 

2. General Framework of the Proposed SWS

In this section, we conceptualize the key processes of a next-gen SWS for video surveillance, with its generalized framework discussed below and illustrated in Fig. 1.

Fig. 1
  1. Download : Download high-res image (71KB)
  2. Download : Download full-size image
Fig. 1

Video Acquisition and Storage  An SWS acquires raw video data of the monitored region using a set of optical imaging sensors (i.e., video-cameras). Sensors can be strategically placed to cover most areas and equipped with night vision or motion detection. A standalone video processing unit processes raw video data and converts the data format that the analytics engine can store and analyze.

Multilevel Threats Detection  An analytics engine is responsible for video data processing, analysis, and threat detection. Its efficacy in threat detection relies upon its capacity to identify multiple objects and accurately classify their respective threat levels. CV or AI methods use video processing to learn the object features and detect them, followed by threat classification considering the level(s) of safety and security risks posed by the detected objects.

Secure & Multi-tiered Alert Generation and Routing  An alert module conventionally triggers an alert, depending on the threat level and application context. Once triggered, the alert needs to be sent to the appropriate person or system for action. Sending an alert to security or law enforcement, launching an alarm, or other security actions are conceivable. In addition to alerting the staff, the system may lock doors, switch on flashing lights, etc.

Response and Action Plan  Once an alert has been routed, an appropriate response and action plan is developed and implemented. This can involve dispatching security personnel or law enforcement to the scene, initiating evacuation procedures, or implementing other measures to neutralize the threat.

Designing a next-gen video SWS requires the above considerations with an advanced analytics engine and intelligent alert module, to automate facility monitoring.

3. Design and Development of the Proposed SWS

In this section, we materialize the generalized framework of our proposed next-gen SWS. The basic functional flow of the SWS is illustrated in Fig. 2. This system broadly includes two major sides i.e. the video analytics engine VSMD and the multi-featured alert system. Major properties of the SWS encompass the following:

  • Video Sensors. The SWS captures raw video data in monitored areas using optical imaging sensors (i.e., video cameras).

  • PIR Sensing. PIR sensors are combined with optical ones to simultaneously augment the detection of human and animal presence, confirming trespassers.

  • User Authentication. The SWS prioritizes access security with a biometric authentication option.

  • Threat Detection. A CV/AI-based engine (e.g., VSMD) automates threat detection by processing and analysing video data.

  • Alert Generation & Routing. The SWS triggers threat-dependent, multi-tiered alerts for security or law enforcement, facilitated by the VSMD.

 

Fig. 2
  1. Download : Download high-res image (208KB)
  2. Download : Download full-size image
Fig. 2

The proposed SWS enhances peripheral security through a streamlined process. Users initiate the system by logging into the VSMD application using a combination of a password and fingerprint authentication. Within this application, users have control over sensitivity levels and various alert options, such as siren, call, Short Message Service (SMS), RF, and email.

Following the login process, the system synchronizes multiple sensors, including optical and PIR sensors, operating them simultaneously. These sensors collaborate to collect video data and motion information, which undergo preprocessing and are then fed into the VSMD application for object feature extraction and classification.

Upon detecting a potential threat, the VSMD prompts the alert module to initiate alert signals of appropriate level (see Sec. 3.2). Subsequently, the alert module issues multi-tiered secure alert messages to notify the relevant authorities. The entire process is visually depicted in Fig. 3.

Fig. 3
  1. Download : Download high-res image (134KB)
  2. Download : Download full-size image
Fig. 3

The following sections will provide a detailed exposition of the development and functional aspects of the proposed next-gen SWS.

3.1. VSMD with an Improved Analytics Engine

The VSMD application includes CV-based multilevel threat detection, encompassing multiple object recognition and tracking of their motion. The Histogram of Oriented Gradient (HOG)-based feature description technique is employed to detect humans, guns, vehicles, animals, and fires. HOG-based object detection is well-known for its effectiveness and extensively discussed in the literature. The extracted HOG features are input to Support Vector Machine (SVM) for object classification or detection. This CV-based technique is popular due to its simplicity and decent accuracy in object detection. Some of these detection approaches, along with their performance metrics, are presented in Table 1. Owing to its widespread acceptance and high accuracy, the proposed VSMD also relies on the HOG-assisted SVM object detection approach.

Table 1
Object Work Method Accuracy
Human Sangeetha & Deepa [6] HOG-SVM 95.76%
Vehicle Putra et al. [9] HOG-SVM 83%
Animal Rathor et al. [10] HOG-SVM 95.15%
Fire Wu et al. [15] HOG-SVM 80.5%
Gun Sumi & Dey [16] HOG-KNN 98.7%

As illustrated in Fig. 4, the object detection process involves gray-scale conversion, edge detection, and HOG feature extraction. To extract HOG features, pixel gradients are calculated for each 

8×8
non-overlapping pixel cell, producing a 9-bin gradient orientation histogram (each bin separated by 20 degrees). The resulting block-wise normalized gradient feature vectors, obtained from respective histograms, are then fed into an Error Correcting Output Code (ECOC) ensemble classifier for object classification. The ECOC employs multiple one-vs-one binary SVM classifiers, which are trained using a custom dataset containing images of firearms, explosions, animals, humans, and vehicles (sampled from [29], [30], [31], [32]).

 

Fig. 4