Skip to content

Commit 8ed4d9f

Browse files
WillIsbackWillIsback
committed
Initial commit: Token Security Analyzer v0.0.1
0 parents  commit 8ed4d9f

File tree

14 files changed

+2202
-0
lines changed

14 files changed

+2202
-0
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main, master]
6+
pull_request:
7+
branches: [main, master]
8+
9+
env:
10+
CARGO_TERM_COLOR: always
11+
12+
jobs:
13+
test:
14+
runs-on: ubuntu-latest
15+
steps:
16+
- uses: actions/checkout@v4
17+
18+
- name: Install Rust
19+
uses: dtolnay/rust-action@stable
20+
with:
21+
toolchain: nightly
22+
components: rustfmt, clippy
23+
24+
- name: Cache cargo
25+
uses: actions/cache@v4
26+
with:
27+
path: |
28+
~/.cargo/bin/
29+
~/.cargo/registry/index/
30+
~/.cargo/registry/cache/
31+
~/.cargo/git/db/
32+
target/
33+
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
34+
35+
- name: Check formatting
36+
run: cargo fmt --all -- --check
37+
38+
- name: Clippy
39+
run: cargo clippy --all-targets --all-features -- -D warnings
40+
41+
- name: Run tests
42+
run: cargo test --all-features
43+
44+
- name: Build release
45+
run: cargo build --release
46+
47+
# Run the analyzer on the demo project
48+
demo:
49+
runs-on: ubuntu-latest
50+
needs: test
51+
steps:
52+
- uses: actions/checkout@v4
53+
54+
- name: Install Rust
55+
uses: dtolnay/rust-action@stable
56+
with:
57+
toolchain: nightly
58+
59+
- name: Build
60+
run: cargo build --release
61+
62+
- name: Run demo analysis
63+
run: |
64+
./target/release/token-analyzer API_KEY examples/demo_project --thorough --verbose
65+
echo "Exit code: $?"
66+
continue-on-error: true # Expected to return 2 (security issues found)

‎.gitignore‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
/target
2+
Cargo.lock
3+
**/*.rs.bk
4+
*.pdb
5+
6+
# IDE
7+
.idea/
8+
.vscode/
9+
*.swp
10+
*.swo
11+
12+
# OS
13+
.DS_Store
14+
Thumbs.db

‎Cargo.toml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
[package]
2+
name = "token-analyzer"
3+
version = "0.0.1"
4+
edition = "2024"
5+
authors = ["WillIsback <[email protected]>"]
6+
description = "Fast, parallel token security analyzer - Detect exposed secrets, API keys, and sensitive tokens in your codebase"
7+
license = "MIT"
8+
repository = "https://github.com/WillIsback/token-analyzer"
9+
homepage = "https://github.com/WillIsback/token-analyzer"
10+
documentation = "https://docs.rs/token-analyzer"
11+
readme = "README.md"
12+
keywords = ["security", "secrets", "tokens", "api-key", "static-analysis"]
13+
categories = ["command-line-utilities", "development-tools", "authentication"]
14+
rust-version = "1.85"
15+
16+
[lib]
17+
name = "token_analyzer"
18+
path = "src/lib.rs"
19+
20+
[[bin]]
21+
name = "token-analyzer"
22+
path = "src/main.rs"
23+
24+
[dependencies]
25+
anyhow = "1.0"
26+
ignore = "0.4"
27+
parking_lot = "0.12"
28+
rayon = "1.10"
29+
regex = "1.11"
30+
serde = { version = "1.0", features = ["derive"] }
31+
serde_json = "1.0"
32+
33+
[dev-dependencies]
34+
tempfile = "3.14"
35+
36+
[profile.release]
37+
lto = true
38+
codegen-units = 1
39+
strip = true
40+
41+
[package.metadata.docs.rs]
42+
all-features = true
43+
rustdoc-args = ["--cfg", "docsrs"]

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2024 WillIsback
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md‎

Lines changed: 195 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
# Token Analyzer 🔐
2+
3+
[![Crates.io](https://img.shields.io/crates/v/token-analyzer.svg)](https://crates.io/crates/token-analyzer)
4+
[![Documentation](https://docs.rs/token-analyzer/badge.svg)](https://docs.rs/token-analyzer)
5+
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
6+
7+
**Fast, parallel token security analyzer** - Detect exposed secrets, API keys, and sensitive tokens in your codebase.
8+
9+
> Part of the [lazy-locker](https://github.com/WillIsback/lazy-locker) ecosystem for secure secret management.
10+
11+
## ✨ Features
12+
13+
- **🚀 Blazing fast** - Uses ripgrep's `ignore` crate for file walking (~170K files/sec)
14+
- **⚡ Parallel** - Leverages `rayon` for multi-threaded file scanning
15+
- **🧠 Smart** - Respects `.gitignore` and common ignore patterns
16+
- **🔐 Security-focused** - Detects dangerous patterns (print, log, echo)
17+
- **📁 Context-aware** - Prioritizes sensitive files (.env, configs)
18+
- **🎯 Entropy detection** - Identifies high-entropy strings (real secrets vs placeholders)
19+
- **🏷️ Known prefixes** - Detects 30+ known token formats (AWS, GitHub, Slack, OpenAI...)
20+
21+
## 📦 Installation
22+
23+
### From crates.io
24+
25+
```bash
26+
cargo install token-analyzer
27+
```
28+
29+
### From source
30+
31+
```bash
32+
git clone https://github.com/WillIsback/token-analyzer
33+
cd token-analyzer
34+
cargo install --path .
35+
```
36+
37+
## 🚀 Quick Start
38+
39+
### Command Line
40+
41+
```bash
42+
# Scan current directory for API_KEY usage
43+
token-analyzer API_KEY
44+
45+
# Scan specific directory
46+
token-analyzer API_KEY ./my-project
47+
48+
# Quick scan (1k files, 5s timeout)
49+
token-analyzer API_KEY ./my-project --fast
50+
51+
# Thorough scan (includes hidden files)
52+
token-analyzer API_KEY ./my-project --thorough
53+
54+
# JSON output for CI/CD integration
55+
token-analyzer API_KEY ./my-project --json
56+
```
57+
58+
### As a Library
59+
60+
```rust
61+
use token_analyzer::{TokenSecurityAnalyzer, AnalyzerConfig};
62+
use std::path::PathBuf;
63+
64+
let analyzer = TokenSecurityAnalyzer::new(AnalyzerConfig::default());
65+
let report = analyzer.analyze("API_KEY", &PathBuf::from("./my-project"))?;
66+
67+
println!("Found {} calls in {} files", report.total_calls, report.files.len());
68+
println!("Risk score: {} (critical files: {})", report.total_risk_score, report.critical_files);
69+
70+
for file in report.exposed_files() {
71+
println!("⚠️ {} - {:?}", file.path.display(), file.risk_level);
72+
for exposure in &file.exposures {
73+
println!(" Line {}: {}", exposure.line, exposure.exposure_type);
74+
}
75+
}
76+
```
77+
78+
## 📊 Example Output
79+
80+
```
81+
╭─────────────────────────────────────────────────────────────╮
82+
│ 🔐 Token Security Analysis Report │
83+
╰─────────────────────────────────────────────────────────────╯
84+
85+
Token: API_KEY
86+
Directory: ./my-project
87+
Duration: 7.63ms
88+
Files: 5 scanned
89+
90+
╭─────────────────────────────────────────────────────────────╮
91+
│ 📊 Summary │
92+
╰─────────────────────────────────────────────────────────────╯
93+
94+
Total calls: 10 in 5 files
95+
Risk score: 19 (critical files: 1)
96+
⚠️ EXPOSED: 3 files with potential plaintext exposure!
97+
98+
╭─────────────────────────────────────────────────────────────╮
99+
│ 📁 Files │
100+
╰─────────────────────────────────────────────────────────────╯
101+
102+
🔴 ⚠️ .env (1 calls, score: 4) [L5: Known prefix: OpenAI API Key]
103+
🟠 ⚠️ docker-compose.yml (2 calls, score: 6) [L10: High entropy]
104+
🟢 ⚠️ dangerous_code.js (3 calls, score: 3) [L2: Hardcoded, L5: Logged]
105+
🟢 safe_code.py (3 calls, score: 3)
106+
🟠 config.yml (1 calls, score: 3)
107+
```
108+
109+
## 🎯 Risk Levels
110+
111+
| Level | Icon | Description | Examples |
112+
|-------|------|-------------|----------|
113+
| Critical | 🔴 | Environment & secrets files | `.env`, `secrets.yml`, `*.pem`, `*.key` |
114+
| High | 🟠 | Infrastructure configs | `docker-compose.yml`, `terraform.tfvars`, `k8s/` |
115+
| Medium | 🟡 | Configuration files | `*.yml`, `*.toml`, `*.ini` |
116+
| Low | 🟢 | Regular source code | `*.py`, `*.js`, `*.rs` |
117+
118+
## 🏷️ Known Token Prefixes
119+
120+
Token Analyzer automatically detects secrets from popular services:
121+
122+
| Service | Prefix | Description |
123+
|---------|--------|-------------|
124+
| GitHub | `ghp_`, `gho_`, `ghs_` | Personal, OAuth, Server tokens |
125+
| AWS | `AKIA`, `ASIA` | Access Key IDs |
126+
| OpenAI | `sk-` | API Keys |
127+
| Slack | `xoxb-`, `xoxp-` | Bot & User tokens |
128+
| Stripe | `sk_live_`, `sk_test_` | Secret Keys |
129+
| Google | `AIza` | API Keys |
130+
| Hugging Face | `hf_` | Access Tokens |
131+
| And 20+ more... | | |
132+
133+
## 🔍 Detection Patterns
134+
135+
### Dangerous Patterns Detected
136+
137+
- **Hardcoded values**: `API_KEY = "sk-xxx..."`
138+
- **Print statements**: `print(API_KEY)`, `console.log(API_KEY)`
139+
- **Logging**: `logger.debug(f"Key: {API_KEY}")`
140+
- **Format strings**: `f"Using {API_KEY}"`, `format!("{}", API_KEY)`
141+
142+
### Safe Patterns (Not Flagged)
143+
144+
- Environment reads: `os.environ.get("API_KEY")`
145+
- Process env: `process.env.API_KEY`
146+
- Rust env: `std::env::var("API_KEY")`
147+
- Variable references: `${API_KEY}`
148+
149+
## 🔧 CLI Options
150+
151+
```
152+
USAGE:
153+
token-analyzer <TOKEN_NAME> [DIRECTORY] [OPTIONS]
154+
155+
OPTIONS:
156+
-f, --fast Quick scan (1k files, 5s timeout)
157+
-t, --thorough Complete scan (unlimited files, includes hidden)
158+
-j, --json Output results as JSON
159+
-v, --verbose Show progress and debug info
160+
--hidden Include hidden files
161+
--follow-links Follow symbolic links
162+
--timeout=MS Set timeout in milliseconds (default: 30000)
163+
--max-files=N Maximum files to scan (default: 10000, 0=unlimited)
164+
165+
EXIT CODES:
166+
0 No security issues found
167+
1 Error occurred
168+
2 Security issues detected
169+
```
170+
171+
## 🔗 Related Projects
172+
173+
- **[lazy-locker](https://github.com/WillIsback/lazy-locker)** - Secure TUI secret manager that uses token-analyzer for security audits. Replace your `.env` files with an encrypted vault!
174+
175+
## 📄 License
176+
177+
MIT License - see [LICENSE](LICENSE) for details.
178+
179+
## 🙏 Acknowledgments
180+
181+
- Developed with assistance from **Claude Opus 4.5** (Anthropic) - AI pair programming was used ethically to accelerate development while maintaining code quality and security best practices
182+
183+
## 🤝 Contributing
184+
185+
Contributions are welcome! Please feel free to submit a Pull Request.
186+
187+
1. Fork the repository
188+
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
189+
3. Commit your changes (`git commit -m 'Add amazing feature'`)
190+
4. Push to the branch (`git push origin feature/amazing-feature`)
191+
5. Open a Pull Request
192+
193+
---
194+
195+
Made with ❤️ and 🦀 by [WillIsback](https://github.com/WillIsback)

‎examples/demo_project/.example.env‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# ⚠️ DEMO FILE - Contains fake secrets for testing purposes
2+
# Never commit real .env files to version control!
3+
4+
# This will be detected as "Known prefix: OpenAI API Key"
5+
API_KEY="sk-1234567890abcdefghijklmnopqrstuvwxyz"
6+
7+
# This would also be detected
8+
DATABASE_URL="postgresql://admin:supersecretpassword@localhost:5432/mydb"
9+
10+
# Hugging Face token (known prefix)
11+
HF_TOKEN="hf_abcdefghijklmnopqrstuvwxyz123456"
12+
13+
# AWS credentials (known prefix AKIA)
14+
AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
15+
AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
16+
17+
# GitHub token (known prefix ghp_)
18+
GITHUB_TOKEN="ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

0 commit comments

Comments
 (0)