Conflict-Detection---GDP2

Airports are getting smarter – not just on the passenger side, but also on the maintenance and operations side. This is because operations are getting more complex, there are new threats from both humans and autonomous machines, and human operators are often working alongside robots in potentially safety-critical environments. 

The project designs, implements and tests a system for the detection of multiple-humans (crowds) for a surveillance system in a smart airport scenario.

Fig: Model Inference

1. Dataset

COCO download: COCO dataset used to train the segmentation model (HRNet)

Dataset download: Original and cleaned dataset (images cropped + black backgrounds)

Repartition	Original image	Image cropped

2. Segmentation Phase

This part consists in the first part of the model using HRNet used to get the predicted key points on each human body presents on an image, a video or a video launch.

Utilisation

Models required: Need to download the models folder and place it in the running folder according to the following architecture:

${POSE_ROOT}
├── data
├── experiments
├── lib
├── log
├── models
├── output
├── tools 
├── config
├── core
├── dataset 
├── demo
├── nms
├── utils
├── visualization
├── **model.h5**
├── **model.pickle**
├── **demo.py**

Running

The same python code 'demo.py' is used for this part. The model can be updated but the one used by default is in 'demo/inference-config.yaml'.

The parse can be updated according to the type of results needed:

python Segmentation_phase/demo/demo.py

• use --webcam when the input is a real-time camera.
• use --video [video-path] when the input is a video.
• use --image [image-path] when the input is an image.
• use --write to save the image, camera or video result.
• use --showFps to show the fps (this fps includes the detection part).

Fig: Example of segmentation

3. Classification Phase

Feature Approach

Data Pre-processing

• Plotting the key points from pre-cropped images on a blackbackground for the training

python Classification_phase/Features_Approach/preprocessing_vgg.py

Fig: Dataset sample

Training

• Training the pre-trained model VGG16 by transfer learning using the pre-processed data

python Classification_phase/Features_Approach/training_vgg.py

Model download: Model trained for using in the testing

Testing on pictures

• Testing the Neural Network to obtain predictions on images

python Classification_phase/Features_Approach/prediction_vgg.py

Video launch

• Testing the Neural Network to obtain real-time predictions on video launch

python Classification_phase/Features_Approach/demo_feature_approach.py

Pushing	Demo

Keypoint Approach

Data Pre-processing

• Storing the key points in CSV files after normalization on pre-cropped images

python Classification_phase/Keypoints_approach/pre_processing.py

Dataset sample	Data associated in the CSV file

Training

• Training the classifier (SVM or Multi-Layer Perceptron) using the pre-processed data

python Classification_phase/Keypoints_approach/training.py

Fig: Confusion Matrix for SVM

• Results

	Precision	Recall	F1-score
Normal	94%	98%	96%
Fight	95%	85%	90%
Accuracy			94%
Macro-average	94%	92%	93%
Weighted-Average	94%	94%	94%

• Training the Neural Network Classifier using the pre-processed data

python Classification_phase/Keypoints_approach/training_nn.ipynb

Fig: Confusion Matrix for Neural Network

• Results

	Precision	Recall	F1-score
Normal	97%	94%	95%
Fight	91%	95%	93%
Accuracy			94%
Macro-average	94%	94%	94%
Weighted-Average	94%	94%	94%

Testing on pictures

• Testing the SVM classifier to obtain predictions on images

python Classification_phase/Keypoints_approach/prediction.py

Fig: Image prediction example (Keypoints Approach - SVM)

• Testing the Neural Network classifier to obtain predictions on images

python Classification_phase/Keypoints_approach/testing_nn.py

Fig: Image prediction example (Keypoints Approach - Neural Network)

Video launch

• Testing the SVM classifier to obtain real-time predictions on video launch

python Classification_phase/Keypoints_approach/demo_keypoint_approach.py

Shooting	Punching

Fig: Multi-fight inference (SVM)

• Testing the Neural Network classifier to obtain real-time predictions on video launch

Trained model NN: Classification_phase/Keypoints_approach/NN
python Classification_phase/Keypoints_approach/testing_nn.py

Fig: Multi-fight inference (NN)

Airport scenarios

Scenario 1	Scenario 2	Scenario 3

Scenario 4	Scenario 5	Scenario 6