readme update

Author: sshivanshg

README.md

@@ -1,189 +1,74 @@
 # Spatial Awareness through Ambient Wireless Signals
 
 WiFi Channel State Information (CSI) can double as a privacy-preserving motion sensor.  
-This repo contains everything we used to turn the WiAR dataset (Intel 5300 CSI captures) into a binary **presence detector** with visualizations, CLI utilities, and a reproducible notebook. The code sits in the `_archive/` directory, while the root keeps the student-facing assets (notebook, setup helpers, trained models).
+This repo contains everything we used to turn the WiAR dataset (Intel 5300 CSI captures) into a binary **presence detector** with visualizations, an interactive dashboard, and a reproducible notebook.
 
 ## Highlights
-- Parses raw 802.11n CSI traces from the WiAR dataset with `csiread`
-- Generates fixed-length CSI windows, extracts 14 statistical features, and fuses them into a binary activity dataset
-- Trains and tunes a Random Forest presence detector, saving joblib artifacts plus metrics
-- Provides live/recorded visualizations (heatmaps, probability curves) and HTML exports for presentations
-- Includes a single notebook that walks through data loading, synthetic empty-room generation, training, and evaluation
+- **Two Models**:
+    - **Random Forest**: Fast, feature-based (96% accuracy).
+    - **1D-CNN (Deep Learning)**: End-to-end learning on raw CSI (91%+ accuracy).
+- **Interactive Dashboard**: A Streamlit app for live demonstrations, featuring real-time simulation and model switching.
+- **End-to-End Notebook**: `Spatial_Awareness_Project.ipynb` walks through the entire pipeline (Data Loading -> Preprocessing -> Training -> Evaluation).
+- **Synthetic Data**: Generates "Empty Room" samples to handle class imbalance.
 
 ## Repository Tour
 ```
 .
-├── Spatial_Awareness_Project.ipynb      # End-to-end, commented walkthrough
-├── _archive/                            # Source modules and CLI utilities
-│   ├── scripts/                         # Data prep + pipeline CLIs
-│   ├── model_tools/                     # Training + visualization scripts
-│   └── src/                             # Library code (preprocess/models/train)
-├── models/                              # Saved Random Forest + scaler + pipeline
+├── Spatial_Awareness_Project.ipynb      # Main Project Notebook (Report)
+├── app.py                               # Interactive Streamlit Dashboard
+├── models/                              # Saved Models
+│   ├── presence_detector_rf.joblib      # Random Forest Model
+│   ├── presence_detector_scaler.joblib  # Scaler for RF
+│   └── presence_detector_cnn.pth        # CNN Model (PyTorch)
+├── model_tools/                         # Model Utilities
+│   ├── train_cnn.py                     # Script to train/retrain the CNN
+│   └── predict_from_raw.py              # CLI tool for single-file prediction
+├── data/                                # Dataset (Git-ignored)
 ├── requirements.txt                     # Python dependencies
-├── setup.sh                             # Student-friendly environment bootstrap
-├── Makefile                             # Convenience targets (setup/install/clean)
-└── pyproject.toml                       # Packaging metadata (setuptools)
+└── _archive/                            # Legacy/Helper scripts
 ```
 
-> The `data/` directory (raw WiAR captures and processed artifacts) is git-ignored. Create the structure described below before running the pipeline.
-
-### Key Components
-- `Spatial_Awareness_Project.ipynb`: Runs the full workflow in one place—loading CSI, preprocessing, generating synthetic no-activity samples, training the model, plotting metrics, and exporting artifacts.
-- `_archive/scripts/`: Small CLIs for dataset download (`fetch_wiar.sh`), window generation, feature extraction, binary fusion, validation, and a `run_pipeline.py` orchestrator.
-- `_archive/model_tools/`: Training + visualization entrypoints (`train_presence_detector.py`, `tune_presence_detector.py`, `visualize_activity_heatmap.py`, `visualize_samples.py`, `visualize_live_session.py`, `live_predict.py`, `predict_from_raw.py`, `view_data.py`). `model_tools/html/` stores executed notebook exports for quick demos.
-- `_archive/src/`: Reusable modules
-  - `preprocess/`: CSI loaders (`csi_loader.py`, `dat_loader.py`), windowing + normalization (`preprocess.py`), feature engineering (`features.py`), and WiAR inspection helpers.
-  - `models/motion_detector.py`: Runtime convenience wrapper that loads the scaler + model + metadata to score new CSI windows.
-  - `train/dataset.py`: PyTorch Dataset scaffold intended for future CNN/RNN work.
-- `models/`: `presence_detector_rf.joblib`, `presence_detector_scaler.joblib`, and `presence_detector_pipeline.joblib` generated by the training scripts.
-- `setup.sh` / `Makefile`: Lightweight automation to create a virtualenv and install requirements without digging into tooling details.
-
 ## Getting Started
-### Prerequisites
-- Python 3.10+ (3.11 works best)
-- `pip`, `venv`, and (for `.dat` parsing) `libpcap` headers if you plan to compile `csiread`
-- Optional: GNU Make, tmux, and JupyterLab
 
-### Option A — one-shot setup
+### 1. Setup Environment
 ```bash
-cd spatialaw
-chmod +x setup.sh
-./setup.sh
+# Create virtual environment
+python3 -m venv .venv
+source .venv/bin/activate
+
+# Install dependencies
+pip install -r requirements.txt
 ```
 
-### Option B — manual steps
+### 2. Run the Dashboard (Demo)
+The dashboard allows you to visualize the system in action.
 ```bash
-python3 -m venv venv
-source venv/bin/activate
-pip install --upgrade pip
-pip install -r requirements.txt
+streamlit run app.py
 ```
+**Features:**
+- **Model Selector**: Switch between Random Forest and CNN.
+- **Simulate Live Mode**: Replays a file as if it were a live stream.
+- **Stability Filter**: Smooths predictions over time.
 
-You can achieve the same with `make setup`, and rerun `make clean` to drop stray `__pycache__` or `.pyc` files.
+### 3. Run the Notebook
+Open `Spatial_Awareness_Project.ipynb` in Jupyter to see the full training and evaluation report.
 
-## Data Layout & Requirements
-All data lives under `data/` (ignored by git). Create the following folders before running the scripts:
+### 4. Train the CNN (Optional)
+If you want to retrain the Deep Learning model:
+```bash
+python model_tools/train_cnn.py
+```
+This will train the model on `data/processed/windows` and save it to `models/presence_detector_cnn.pth`.
+
+## Data Layout
+All data lives under `data/` (ignored by git).
 ```
 data/
-├── raw/
-│   └── WiAR/     # WiAR repository clone or downloaded archive
-└── processed/
-    ├── windows/
-    ├── features/
-    ├── binary/
-    └── synthetic_empty/
+├── raw/WiAR/           # Original Dataset
+└── processed/          # Generated Windows & Features
 ```
 
-**Dataset:** WiAR (16 motion classes captured with an Intel 5300 NIC). Use `_archive/scripts/fetch_wiar.sh` to download and unpack the official release into `data/raw/WiAR`.
-
-## Processing & Training Pipeline
-Each stage is a CLI that can be run independently or via `_archive/scripts/run_pipeline.py`.
-
-1. **Generate CSI windows**
-   ```bash
-   python _archive/scripts/generate_windows.py \
-     --input-dir data/raw/WiAR \
-     --out-dir data/processed/windows \
-     --T 256 \
-     --stride 64
-   ```
-   - Loads `.dat`, `.txt`, `.csv`, or `.npy` CSI files
-   - Applies denoising + z-score normalization (`preprocess.py`)
-   - Saves `window_*.npy`, `labels.csv`, and `window_generation_summary.json`
-
-2. **Extract 14 CSI features per window**
-   ```bash
-   python _archive/scripts/extract_features.py \
-     --windows-dir data/processed/windows \
-     --output-dir data/processed/features
-   ```
-   - Features include variance, entropy, Hilbert envelope stats, motion period, MAD, etc. (see `_archive/src/preprocess/features.py`)
-
-3. **Create the binary presence dataset**
-   ```bash
-   python _archive/scripts/process_binary_dataset.py \
-     --features-dir data/processed/features \
-     --output-dir data/processed/binary
-   ```
-   - Combines active WiAR activities into label `1`
-   - Selects low-motion segments as label `0` (with motion quantile filters)
-   - Writes `features.npy`, `labels.csv`, and `feature_names.json`
-
-4. **(Optional) Generate synthetic “empty room” samples**
-   ```bash
-   python _archive/scripts/generate_synthetic_empty.py \
-     --output-dir data/processed/synthetic_empty \
-     --n-samples 500
-   ```
-   - Matches the notebook’s synthetic-noise generator to balance classes when real idle captures are scarce.
-
-5. **Validate the binary dataset**
-   ```bash
-   python _archive/scripts/validate_binary_dataset.py \
-     --binary-dir data/processed/binary
-   ```
-   - Produces `validation_report.json` (label histograms, motion percentiles, leakage checks).
-
-6. **Run the whole pipeline (any subset of steps)**
-   ```bash
-   python _archive/scripts/run_pipeline.py --steps windows features binary validate
-   ```
-
-## Modeling & Visualization Tools
-Located in `_archive/model_tools/`:
-
-| Script | Purpose | Outputs |
-| --- | --- | --- |
-| `train_presence_detector.py` | Train + evaluate RandomForestClassifier on the binary dataset; optionally reloads existing scaler/model. | Saves `.joblib` models, `presence_detector_metrics.json`, confusion matrix, ROC curve, feature importance plots. |
-| `tune_presence_detector.py` | Grid search with GroupKFold cross-validation. | `models/tuning_results.json`, best model artifacts. |
-| `visualize_activity_heatmap.py` | Draws CSI heatmaps with predicted probabilities overlayed (great for demos). | PNG/interactive matplotlib windows. |
-| `visualize_samples.py` | Random window explorer for debugging feature quality. | Matplotlib figures. |
-| `visualize_live_session.py` | Streams pre-recorded sessions as if live, scoring each window. | Live matplotlib updates. |
-| `live_predict.py` | Connects to an incoming CSI stream (or directory) and prints rolling predictions. | Terminal output + optional plots. |
-| `predict_from_raw.py` | Convenience wrapper to score a single raw CSI file end-to-end. | Prints probability + label. |
-| `view_data.py` | Dumps `.npy` feature data, label counts, and metadata. | Terminal table summaries. |
-
-Executed HTML exports of the major scripts live in `_archive/model_tools/html/` so you can skim outputs without running the code.
-
-## Notebook Walkthrough (`Spatial_Awareness_Project.ipynb`)
-The notebook mirrors the CLI pipeline but keeps everything in one place for reports:
-- **Sections 1–2**: Data loading utilities for Intel 5300 `.dat` files, including a fallback parser and visual sanity checks (packet counts, amplitude heatmaps).
-- **Sections 3–4**: Preprocessing helpers (Butterworth filters, Hilbert envelope, z-score normalization) and window visualization.
-- **Section 5**: Feature extraction (the same 14 statistical descriptors used by the CLIs) with pandas summaries.
-- **Section 6**: Synthetic “empty room” generator + SMOTE oversampling to handle class imbalance when genuine idle captures are missing.
-- **Section 7**: Train/test split with `GroupShuffleSplit`, Random Forest training (balanced class weights), metrics (accuracy, precision, recall, F1, ROC-AUC), confusion matrix, ROC curve, and distribution plots.
-- **Section 8**: Model export via `joblib`, along with helper functions to reload the scaler/model for downstream scripts.
-- **Appendix**: Utility cells for plotting CSI heatmaps, inspecting feature importances, and sandboxing experimental architectures (there is an `add_cnn_to_notebook.py` helper in `_archive/scripts/create_notebook.py` for future deep learning work).
-
-Run it with JupyterLab (`jupyter lab Spatial_Awareness_Project.ipynb`) after setting up the environment.
-
-## Core Library Modules (`_archive/src/`)
-- `preprocess/csi_loader.py`: Recursively lists CSI recordings, infers labels from filenames, and converts `.dat`, `.txt`, `.csv`, or `.npy` files into numpy arrays.
-- `preprocess/dat_loader.py`: Thin wrapper over `csiread` + legacy parsing helpers for Intel 5300 `.dat` files, handling antenna permutations and amplitude extraction.
-- `preprocess/preprocess.py`: Windowing (`window_csi`), denoising (`denoise_window`), normalization (`normalize_window`), and serialization (`save_windows`).
-- `preprocess/features.py`: Defines the 14 handcrafted features used throughout the project.
-- `preprocess/inspect_wiar.py`: Quickly inspects WiAR metadata, packet counts, motion scores, and activity mappings.
-- `models/motion_detector.py`: Provides a simple `MotionDetector` class with `predict_proba` / `predict` methods that internally load the saved scaler + Random Forest.
-- `train/dataset.py`: PyTorch Dataset/Loader utilities for window tensors—handy if you want to extend the work with CNNs or transformers later.
-
-## Saved Artifacts & Reports
-- `models/presence_detector_rf.joblib`: Fitted RandomForestClassifier.
-- `models/presence_detector_scaler.joblib`: StandardScaler trained on the binary dataset.
-- `models/presence_detector_pipeline.joblib`: End-to-end pipeline object (scaler + model).
-- `_archive/model_tools/html/*.html`: Frozen notebook exports with visualizations for the detector, heatmaps, and random samples.
-- `data/processed/**`: (Not tracked) holds windows, features, synthetic data, binary dataset, plus validation reports.
-
-## Testing & Validation
-- Dataset sanity checks live inside `_archive/scripts/validate_binary_dataset.py`.
-- Group-aware train/test splits and cross-validation are enforced in the notebook and tuning script to avoid subject leakage.
-- There are no standalone pytest suites checked in; when extending the project consider wrapping the CLI scripts with regression tests.
-
-## References
-- **WiAR Dataset**: L. Guo et al., *“A Novel Benchmark on Human Activity Recognition Using WiFi Signals,”* IEEE Healthcom, 2017.
-- **Intel 5300 CSI Tool**: <http://dhalperi.github.io/linux-80211n-csitool/>
-- **csiread Library**: <https://github.com/citywu/csiread>
-
 ## Authors
 - Rishabh (230178)
 - Shivansh (230054)
-Newton School of Technology — Computer Networks + AI/ML capstone
+Newton School of Technology — Computer Networks + AI/ML Capstone