Skip to content

stacyshki/CVE-dashboard

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CVE Intelligence Dashboard

The story behind all

Dashboard

A fully automated vulnerability intelligence pipeline: daily data collection from three open APIs → PostgreSQL star schema on Supabase → interactive Streamlit dashboard.


What it does

The system collects every CVE (Common Vulnerabilities and Exposures) published by NIST and enriches each one with exploitation probability and real-world attack confirmation data. The result is a dashboard that goes beyond raw severity scores and lets you identify which vulnerabilities are actually dangerous right now.

A CVE is a registered, publicly disclosed software vulnerability. It is a fact about the existence of a flaw — not a log of attacks. Every CVE gets a unique ID (e.g. CVE-2024-12345), a CVSS severity score, and a structured description of the affected software. NVD publishes dozens to hundreds of new CVEs per day.


Data Sources

Source What it adds Key
NVD (NIST National Vulnerability Database) CVE descriptions, CVSS scores, attack attributes, affected products, reference links None
EPSS API (FIRST.org) Daily-updated probability that a CVE will be exploited in the next 30 days None
CISA KEV (Known Exploited Vulnerabilities) Confirmed in-the-wild exploitation — the highest-priority signal None

All three sources are completely free and open.


Database Schema (Star Schema)

The database follows a star schema: one central fact table surrounded by dimension tables.

                    dim_date
                       │
dim_weakness ──── fact_cves ──── dim_severity
                       │
              dim_attack_profile
                       │
                   dim_vendor

Dimension Tables

dim_severity — static lookup, seeded once by the schema

Column Description
severity_id Surrogate key
label CRITICAL / HIGH / MEDIUM / LOW / N/A
score_min/max CVSS range for this band

dim_date — one row per calendar day

Column Description
date_id Surrogate key
date The date itself
year, month, quarter, week, day_of_week Pre-computed for fast GROUP BY

dim_attack_profile — unique combinations of CVSS v3 attack attributes

Column Values Meaning
attack_vector NETWORK / ADJACENT / LOCAL / PHYSICAL Where the attacker must be to exploit
attack_complexity LOW / HIGH How repeatable the attack is
privileges_required NONE / LOW / HIGH What access the attacker needs beforehand
user_interaction NONE / REQUIRED Whether victim action is needed

dim_vendor — unique (vendor, product) pairs from CPE strings

Column Example Description
vendor microsoft Publisher extracted from CPE
product windows_10 Specific product name

Note: A high CVE count for a vendor reflects a large product portfolio and active disclosure process, not necessarily worse security practices.

dim_weakness — CWE root cause classifications

Column Example Description
cwe_id CWE-79 Common Weakness Enumeration ID
cwe_name Cross-site Scripting Human-readable name (seeded for top ~22 types)

CWE describes the type of flaw; CVE describes a specific instance of it.

Fact Table: fact_cves

Column Type Description
cve_id text Natural key, e.g. CVE-2024-12345
date_id FK References dim_date
severity_id FK References dim_severity
profile_id FK References dim_attack_profile (NULL if CVSS v3 missing)
vendor_id FK References dim_vendor (NULL if no CPE data)
weakness_id FK References dim_weakness (NULL if no CWE)
cvss_v3_score float 0.0 – 10.0
cvss_v3_vector text Full vector string, e.g. CVSS:3.1/AV:N/AC:L/...
cvss_v2_score float Fallback for CVEs published before ~2015
scope text UNCHANGED = exploit stays in-component; CHANGED = crosses security boundary
conf_impact text Effect on Confidentiality: NONE / LOW / HIGH
integ_impact text Effect on Integrity (data modification)
avail_impact text Effect on Availability (service disruption)
epss_score float 0.0 – 1.0: probability of exploitation in next 30 days
epss_percentile float Where this CVE sits among all CVEs by exploitation risk
ref_count int Total NVD reference links. Higher = more widely discussed
has_exploit_ref bool At least one reference tagged "Exploit" in NVD
has_patch_ref bool At least one reference tagged "Patch" or "Vendor Advisory"
is_kev bool CVE is in the CISA Known Exploited Vulnerabilities catalog
kev_added_date date When CISA confirmed active exploitation

View: v_cves_full

A SQL view that joins all five tables back into one flat surface.


Project Structure

cve-dashboard/
├── .github/
│   └── workflows/
│       └── daily_fetch.yml       # GitHub Actions cron at 07:00 UTC
├── .streamlit/
│   └── config.toml               # Theme for dashboard
├── app/
│   └── streamlit_app.py          # Dashboard (reads v_cves_full)
├── etl/
│   └── fetch_cves.py             # ETL: NVD + EPSS + KEV -> star schema
├── sql/
│   └── schema.sql                # Full schema
├── waker/
│   └── awake-streamlit.py        # Wakes Streamlit free tier app up
├── .env.example                  # Instructions for env setup
├── .gitignore
├── README.md
├── requirements.txt
└── index.md                      # analysis of dashboard on 6th April 2026

Local Setup

1. Clone the repo

git clone https://github.com/stacyshki/CVE-dashboard.git
cd CVE-dashboard
pip install -r requirements.txt

2. Create a Supabase project

  1. Sign up at supabase.com and create a new project

  2. Open SQL Editor, paste the contents of sql/schema.sql, and run it

  3. Go to Project Settings → Data API and copy:

    • Project ID → this is part of SUPABASE_URL
    • service_role secret key → this is SUPABASE_KEY

    (more info at .env.example)

Use service_role, not anon. The service role key bypasses Row Level Security and is required for the ETL to write data.

3. Set environment variables

Create a .env file or export directly:

export SUPABASE_URL=https://XXXXXXXXXXXXXXXXXXXX.supabase.co
export SUPABASE_KEY=your-service-role-key

Or use a .env file and load it:

# .env
SUPABASE_URL=https://XXXXXXXXXXXXXXXXXXXX.supabase.co
SUPABASE_KEY=your-service-role-key
export $(cat .env | xargs)

4. Initial data load (backfill)

The ETL defaults to fetching the last 1 day (FETCH_DAYS = 1). For an initial load edit etl/fetch_cves.py:

FETCH_DAYS = 60  # two month for instance

Then run:

python etl/fetch_cves.py

NVD rate-limits unauthenticated requests to 5 per 30 seconds. The script sleeps 6 seconds between paginated requests — this is intentional. A 60-day backfill takes some time.

After the backfill, reset FETCH_DAYS = 1 before committing.

5. Run the dashboard

streamlit run app/streamlit_app.py

Opens at http://localhost:8501.


Automating Daily Updates (GitHub Actions)

1. Push to GitHub

git init
git remote add origin https://github.com/username/CVE-dashboard.git
git add .
git commit -m "init"
git push -u origin main

2. Add repository secrets

Go to your repo on GitHub → Settings → Secrets and variables → Actions → New repository secret

Add two secrets:

  • SUPABASE_URL — your project URL
  • SUPABASE_KEY — your service_role key
  • STREAMLIT_APP_LINK — link-to-streamlit-cloud-app

3. Trigger manually (first time)

Go to Actions → Daily CVE Fetch → Run workflow

After that it runs automatically every day at 07:00 UTC (NVD updates around 06:00 UTC).

Go to Actions → Wake Streamlit App → Run workflow

After that it runs automatically every 2 hours to wake the free tier Streamlit App up.


Deploying the Dashboard (Streamlit Community Cloud)

  1. Push the project to GitHub (done above)
  2. Go to share.streamlit.io and sign in with GitHub
  3. Click New app → select your repo → set Main file path to app/streamlit_app.py
  4. Under Advanced settings → Secrets, add:
SUPABASE_URL = "https://xxxxxxxxxxxx.supabase.co"
SUPABASE_KEY = "your-service-role-key"
  1. Click Deploy. Your dashboard will be publicly accessible at a *.streamlit.app URL.

Forking

Another way (except local setup) could be a fork:

  • Replace etl/fetch_cves.py with a fetcher for your own API
  • Adjust sql/schema.sql to match your data shape
  • Update app/streamlit_app.py column names and chart logic accordingly
  • The GitHub Actions workflow and Streamlit Cloud deployment steps are identical

Environment variables (SUPABASE_URL, SUPABASE_KEY, STREAMLIT_APP_LINK) are the only secrets needed. No paid services required.

Contributors