Skip to content

AdityaK-gits/Behaviour-First-Zero-Day-Detector-Using-Runtime-Telemetry-and-LSTM-Autoencoders

Repository files navigation

BFZDD — Behaviour-First Zero-Day Detector

AI-Driven Runtime Telemetry Analyzer Using LSTM, GRU, and Transformer Autoencoders

BFZDD (Behaviour-First Zero-Day Detector) is an AI-powered malware detection framework that identifies unknown and zero-day threats by analyzing runtime behavioral telemetry instead of relying on traditional signature-based detection methods.

The system learns patterns of legitimate software execution using sequence-model autoencoders and detects anomalous behaviors through reconstruction-based anomaly scoring. By focusing on runtime activity rather than known malware signatures, BFZDD can identify previously unseen malware families and sophisticated attack techniques.


Key Features

Behaviour-First Malware Detection

  • Converts runtime events into a structured token vocabulary.

  • Learns normal program behavior through sequence reconstruction.

  • Detects anomalies using:

    • Cross-Entropy Reconstruction Loss
    • Smoothed Probability Scoring
    • Optional KL-Regularization for stable learning

Multiple Deep Learning Architectures

BFZDD supports multiple neural architectures that can be selected dynamically through the user interface:

  • LSTM Autoencoder
  • GRU Autoencoder
  • Transformer-Lite Encoder

All architectures support incremental retraining and continual learning.

Interactive Streamlit Dashboard

A fully integrated web-based dashboard provides:

Dataset Analysis

  • Browse and review collected runtime traces.
  • Automatic benign/malicious trace labeling.
  • Dataset event frequency visualization.
  • Trace scoring using trained anomaly detection models.

Evaluation Metrics

  • Confusion Matrix
  • Precision, Recall, and F1-Score
  • ROC Curve and AUC Analysis
  • Model Performance Monitoring

Live Behaviour Analysis

  • Upload custom runtime traces.

  • Analyze precomputed telemetry data.

  • Interactive timeline visualizations.

  • Event-level anomaly heatmaps.

  • Automated verdict generation:

    • OK
    • QUARANTINE

Diagnostics & Utilities

  • One-click model snapshot generation.
  • Repository file browser (safe mode).
  • Visualization of anomalous event clusters.
  • Dataset frequency heatmaps.

No-Code Model Fine-Tuning

BFZDD enables complete model retraining directly through the dashboard without requiring terminal access.

Fine-Tuning Capabilities

  • Upload JSON runtime traces.
  • Automatic dataset integration.
  • Replay buffer training support.
  • Live training progress monitoring.
  • Loss visualization per epoch.
  • Automatic model checkpoint generation.
  • Threshold recalibration using benign datasets (P95/P99).
  • Exportable retraining package for offline GPU training.

Generated models are automatically saved and versioned for future deployment.


Model Architecture

Tokenization Pipeline

Runtime events are transformed into numerical tokens through:

  • Event normalization
  • Dynamic vocabulary expansion
  • Reserved padding tokens (PAD = 0)
  • Automatic vocabulary size estimation

Autoencoder Framework

The system supports:

build_model(
    arch="lstm" | "gru" | "transformer"
)

Architecture Flow:

Embedding Layer → Encoder → Vocabulary Projection → Sequence Reconstruction

Higher reconstruction error indicates abnormal behavior and potential malicious activity.

Anomaly Scoring

The scoring engine provides:

  • Average anomaly score (avg_loss)
  • Per-token reconstruction losses
  • Event-level anomaly localization

Supported scoring methods:

  • Cross-Entropy Scoring
  • Smoothed Probability Scoring

KL-Regularization

An optional KL-divergence regularizer improves model stability and reduces overfitting during continual learning.


Explainable AI Features

Understanding why a trace is classified as suspicious is critical in cybersecurity.

BFZDD provides:

  • Event-level anomaly identification
  • Top anomalous events ranking
  • Anomalous-event heatmaps
  • Dataset-wide event frequency heatmaps
  • Confusion Matrix analysis
  • ROC and AUC visualizations

These explainability features assist analysts in investigating suspicious program behavior efficiently.


Project Structure

project/
│
├── app.py                      # Streamlit Dashboard
├── model.py                    # Autoencoders & Scoring Engine
├── utils_viz.py                # Visualization Utilities
├── versioning.py               # Model Snapshot Manager
│
├── ae_model.pth                # Primary Trained Model
├── ae_model.pth.meta.json      # Model Metadata
│
├── dataset/
│   └── traces/                 # Runtime Trace Dataset
│
├── models/                     # Saved Model Versions
│
└── threshold.json              # Calibrated Detection Thresholds

Deployment

Local Execution

pip install -r requirements.txt
streamlit run app.py

Streamlit Cloud

The application can be deployed directly to Streamlit Cloud for browser-based access and demonstration.


Security Considerations

Since BFZDD analyzes malware-like behavioral patterns:

  • Untrusted scripts should never be executed on public cloud deployments.
  • Local execution should be performed within an isolated virtual machine.
  • A configurable DISABLE_EXECUTION flag prevents unsafe execution in public environments.

Retraining Package Export

BFZDD can generate a portable retraining package containing:

  • Uploaded benign traces
  • Replay buffer samples
  • Dataset manifests
  • Training metadata

This package can be transferred to GPU-enabled environments for large-scale offline retraining.


Technical Stack

  • Python
  • PyTorch
  • Streamlit
  • NumPy
  • Pandas
  • Scikit-learn
  • Matplotlib
  • Runtime Telemetry Analysis
  • Deep Learning Autoencoders
  • Explainable AI (XAI)

Impact & Applications

Why BFZDD?

  • Detects previously unseen malware families.
  • Eliminates dependence on signature databases.
  • Provides transparent, explainable detection results.
  • Supports continual learning and model evolution.
  • Integrates modern deep learning architectures.
  • Delivers an end-to-end cybersecurity AI platform through an intuitive web interface.

Suitable For

  • Cybersecurity Research
  • Zero-Day Malware Analysis
  • Behaviour-Based Threat Detection
  • AI/ML Security Projects
  • Internship and Placement Portfolios
  • Applied Deep Learning Demonstrations

Developer

Aditya Kolluru B.Tech Computer Science Student | AI & Cybersecurity Enthusiast

Email: adityakolluru2004@gmail.com Location: Bengaluru, India

Live Demo: https://iz222gve472hosjdwdeqvu.streamlit.app/

About

Behaviour-First Zero-Day Detector (BFZDD) An AI-powered malware detector that learns normal program behaviour using LSTM/GRU/Transformer autoencoders and flags anomalies in real time — enabling true zero-day detection beyond signatures. Includes live trace analysis, fine-tuning UI, model versioning, and event heatmaps.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages