A fully automated vulnerability intelligence pipeline: daily data collection from three open APIs → PostgreSQL star schema on Supabase → interactive Streamlit dashboard.
The system collects every CVE (Common Vulnerabilities and Exposures) published by NIST and enriches each one with exploitation probability and real-world attack confirmation data. The result is a dashboard that goes beyond raw severity scores and lets you identify which vulnerabilities are actually dangerous right now.
A CVE is a registered, publicly disclosed software vulnerability. It is a fact about the existence of a flaw — not a log of attacks. Every CVE gets a unique ID (e.g. CVE-2024-12345), a CVSS severity score, and a structured description of the affected software. NVD publishes dozens to hundreds of new CVEs per day.
| Source | What it adds | Key |
|---|---|---|
| NVD (NIST National Vulnerability Database) | CVE descriptions, CVSS scores, attack attributes, affected products, reference links | None |
| EPSS API (FIRST.org) | Daily-updated probability that a CVE will be exploited in the next 30 days | None |
| CISA KEV (Known Exploited Vulnerabilities) | Confirmed in-the-wild exploitation — the highest-priority signal | None |
All three sources are completely free and open.
The database follows a star schema: one central fact table surrounded by dimension tables.
dim_date
│
dim_weakness ──── fact_cves ──── dim_severity
│
dim_attack_profile
│
dim_vendor
dim_severity — static lookup, seeded once by the schema
| Column | Description |
|---|---|
severity_id |
Surrogate key |
label |
CRITICAL / HIGH / MEDIUM / LOW / N/A |
score_min/max |
CVSS range for this band |
dim_date — one row per calendar day
| Column | Description |
|---|---|
date_id |
Surrogate key |
date |
The date itself |
year, month, quarter, week, day_of_week |
Pre-computed for fast GROUP BY |
dim_attack_profile — unique combinations of CVSS v3 attack attributes
| Column | Values | Meaning |
|---|---|---|
attack_vector |
NETWORK / ADJACENT / LOCAL / PHYSICAL | Where the attacker must be to exploit |
attack_complexity |
LOW / HIGH | How repeatable the attack is |
privileges_required |
NONE / LOW / HIGH | What access the attacker needs beforehand |
user_interaction |
NONE / REQUIRED | Whether victim action is needed |
dim_vendor — unique (vendor, product) pairs from CPE strings
| Column | Example | Description |
|---|---|---|
vendor |
microsoft |
Publisher extracted from CPE |
product |
windows_10 |
Specific product name |
Note: A high CVE count for a vendor reflects a large product portfolio and active disclosure process, not necessarily worse security practices.
dim_weakness — CWE root cause classifications
| Column | Example | Description |
|---|---|---|
cwe_id |
CWE-79 |
Common Weakness Enumeration ID |
cwe_name |
Cross-site Scripting |
Human-readable name (seeded for top ~22 types) |
CWE describes the type of flaw; CVE describes a specific instance of it.
| Column | Type | Description |
|---|---|---|
cve_id |
text | Natural key, e.g. CVE-2024-12345 |
date_id |
FK | References dim_date |
severity_id |
FK | References dim_severity |
profile_id |
FK | References dim_attack_profile (NULL if CVSS v3 missing) |
vendor_id |
FK | References dim_vendor (NULL if no CPE data) |
weakness_id |
FK | References dim_weakness (NULL if no CWE) |
cvss_v3_score |
float | 0.0 – 10.0 |
cvss_v3_vector |
text | Full vector string, e.g. CVSS:3.1/AV:N/AC:L/... |
cvss_v2_score |
float | Fallback for CVEs published before ~2015 |
scope |
text | UNCHANGED = exploit stays in-component; CHANGED = crosses security boundary |
conf_impact |
text | Effect on Confidentiality: NONE / LOW / HIGH |
integ_impact |
text | Effect on Integrity (data modification) |
avail_impact |
text | Effect on Availability (service disruption) |
epss_score |
float | 0.0 – 1.0: probability of exploitation in next 30 days |
epss_percentile |
float | Where this CVE sits among all CVEs by exploitation risk |
ref_count |
int | Total NVD reference links. Higher = more widely discussed |
has_exploit_ref |
bool | At least one reference tagged "Exploit" in NVD |
has_patch_ref |
bool | At least one reference tagged "Patch" or "Vendor Advisory" |
is_kev |
bool | CVE is in the CISA Known Exploited Vulnerabilities catalog |
kev_added_date |
date | When CISA confirmed active exploitation |
A SQL view that joins all five tables back into one flat surface.
cve-dashboard/
├── .github/
│ └── workflows/
│ └── daily_fetch.yml # GitHub Actions cron at 07:00 UTC
├── .streamlit/
│ └── config.toml # Theme for dashboard
├── app/
│ └── streamlit_app.py # Dashboard (reads v_cves_full)
├── etl/
│ └── fetch_cves.py # ETL: NVD + EPSS + KEV -> star schema
├── sql/
│ └── schema.sql # Full schema
├── waker/
│ └── awake-streamlit.py # Wakes Streamlit free tier app up
├── .env.example # Instructions for env setup
├── .gitignore
├── README.md
├── requirements.txt
└── index.md # analysis of dashboard on 6th April 2026
git clone https://github.com/stacyshki/CVE-dashboard.git
cd CVE-dashboard
pip install -r requirements.txt-
Sign up at supabase.com and create a new project
-
Open SQL Editor, paste the contents of
sql/schema.sql, and run it -
Go to Project Settings → Data API and copy:
- Project ID → this is part of
SUPABASE_URL - service_role secret key → this is
SUPABASE_KEY
(more info at .env.example)
- Project ID → this is part of
Use service_role, not anon. The service role key bypasses Row Level Security and is required for the ETL to write data.
Create a .env file or export directly:
export SUPABASE_URL=https://XXXXXXXXXXXXXXXXXXXX.supabase.co
export SUPABASE_KEY=your-service-role-keyOr use a .env file and load it:
# .env
SUPABASE_URL=https://XXXXXXXXXXXXXXXXXXXX.supabase.co
SUPABASE_KEY=your-service-role-keyexport $(cat .env | xargs)The ETL defaults to fetching the last 1 day (FETCH_DAYS = 1). For an initial load edit etl/fetch_cves.py:
FETCH_DAYS = 60 # two month for instanceThen run:
python etl/fetch_cves.pyNVD rate-limits unauthenticated requests to 5 per 30 seconds. The script sleeps 6 seconds between paginated requests — this is intentional. A 60-day backfill takes some time.
After the backfill, reset FETCH_DAYS = 1 before committing.
streamlit run app/streamlit_app.pyOpens at http://localhost:8501.
git init
git remote add origin https://github.com/username/CVE-dashboard.git
git add .
git commit -m "init"
git push -u origin mainGo to your repo on GitHub → Settings → Secrets and variables → Actions → New repository secret
Add two secrets:
SUPABASE_URL— your project URLSUPABASE_KEY— your service_role keySTREAMLIT_APP_LINK— link-to-streamlit-cloud-app
Go to Actions → Daily CVE Fetch → Run workflow
After that it runs automatically every day at 07:00 UTC (NVD updates around 06:00 UTC).
Go to Actions → Wake Streamlit App → Run workflow
After that it runs automatically every 2 hours to wake the free tier Streamlit App up.
- Push the project to GitHub (done above)
- Go to share.streamlit.io and sign in with GitHub
- Click New app → select your repo → set Main file path to
app/streamlit_app.py - Under Advanced settings → Secrets, add:
SUPABASE_URL = "https://xxxxxxxxxxxx.supabase.co"
SUPABASE_KEY = "your-service-role-key"- Click Deploy. Your dashboard will be publicly accessible at a
*.streamlit.appURL.
Another way (except local setup) could be a fork:
- Replace
etl/fetch_cves.pywith a fetcher for your own API - Adjust
sql/schema.sqlto match your data shape - Update
app/streamlit_app.pycolumn names and chart logic accordingly - The GitHub Actions workflow and Streamlit Cloud deployment steps are identical
Environment variables (SUPABASE_URL, SUPABASE_KEY, STREAMLIT_APP_LINK) are the only secrets needed. No paid services required.