Skip to content

Commit d562427

Browse files
authored
merge python reqs and fix doc (#81)
Signed-off-by: Rashid Kaleem <230885705+arekay-nv@users.noreply.github.com>
1 parent 135f7a0 commit d562427

File tree

6 files changed

+57
-55
lines changed

6 files changed

+57
-55
lines changed

docs/CLI_QUICK_REFERENCE.md

Lines changed: 4 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -24,12 +24,11 @@ inference-endpoint benchmark offline \
2424
--endpoint URL \
2525
--model Qwen/Qwen3-8B \
2626
--dataset tests/datasets/dummy_1k.pkl \
27-
--report-path my_benchmark_report
27+
--report-dir my_benchmark_report
2828

2929
# YAML-based (YAML mode - no CLI overrides)
3030
inference-endpoint benchmark from-config \
31-
--config test.yaml \
32-
--output results.json
31+
--config test.yaml
3332
```
3433

3534
**Default Test Dataset:** Use `tests/datasets/dummy_1k.pkl` (1000 samples, ~133 KB) for local testing.
@@ -69,8 +68,7 @@ inference-endpoint info
6968
- `--model NAME` - Model name (required for CLI mode, e.g., Qwen/Qwen3-8B)
7069
- `--dataset, -d PATH` - Dataset file (required for CLI mode)
7170
- `--config, -c PATH` - YAML config file (required for from-config mode)
72-
- `--output, -o PATH` - Save results to JSON
73-
- `--report-path PATH` - Save detailed benchmark report with metrics
71+
- `--report-dir PATH` - Save detailed benchmark report with metrics
7472
- `--verbose, -v` - Increase verbosity (-vv for debug)
7573

7674
## Benchmark Options (CLI Mode Only)
@@ -96,9 +94,6 @@ inference-endpoint info
9694

9795
- `pkl` - Pickle format (default)
9896
- `hf` - HuggingFace datasets
99-
100-
**Coming Soon:**
101-
10297
- `jsonl` - JSON Lines format
10398

10499
## Test Modes
@@ -193,7 +188,7 @@ inference-endpoint init --template submission
193188
# 3. Run (YAML mode - no CLI overrides)
194189
inference-endpoint benchmark from-config \
195190
--config submission_template.yaml \
196-
--output official_results.json
191+
--report-dir official_results
197192
```
198193

199194
### Validate First

docs/DEVELOPMENT.md

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ source venv/bin/activate # On Windows: venv\Scripts\activate
2424

2525
# 3. Install development dependencies
2626
pip install -e .
27-
pip install -r requirements/dev.txt
27+
pip install -r requirements/base.txt
2828

2929
# 4. Install pre-commit hooks
3030
pre-commit install
@@ -287,15 +287,14 @@ python -m pdb -m pytest test_file.py
287287

288288
### Adding Dependencies
289289

290-
1. **Base Dependencies** (`requirements/base.txt`): Required for package to function
291-
2. **Development Dependencies** (`requirements/dev.txt`): Development tools, linters, and pre-commit hooks
292-
3. **Test Dependencies** (`requirements/test.txt`): Testing framework and utilities (pytest, pytest-asyncio, etc.)
290+
1. **Base Dependencies** (`requirements/base.txt`): Required for package to function, development tools, linters, and pre-commit hooks
291+
2. **Test Dependencies** (`requirements/test.txt`): Testing framework and utilities (pytest, pytest-asyncio, etc.)
293292

294293
### Updating Dependencies
295294

296295
```bash
297296
# Update all dependencies
298-
pip install --upgrade -r requirements/dev.txt
297+
pip install --upgrade -r requirements/base.txt
299298

300299
# Check for outdated packages
301300
pip list --outdated

docs/GITHUB_SETUP.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,6 @@ Runs pre-commit hooks (ruff, formatters) on every PR and push.
1414

1515
Automatically requests code reviews on new PRs.
1616

17-
### Branch Validator (`.github/workflows/branch-validator.yml`)
18-
19-
Enforces branch naming: `feature/*`, `bugfix/*`, `hotfix/*`, `docs/*`, `test/*`, `refactor/*`, `chore/*`, `release/*`.
20-
2117
### Test Workflow (`.github/workflows/test.yml`)
2218

2319
Runs pytest test suite and generates coverage reports.

examples/README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,14 @@ Benchmarking a real-world model served via open-source serving systems such as [
1616

1717
Compare `inference-endpoint` with [vLLM's](https://github.com/vllm-project/vllm) benchmarking tool.
1818

19+
### [04_GPTOSS120B_Example](04_GPTOSS120B_Example/)
20+
21+
Sample yaml configuration to benchmark `openai/gpt-oss-120b`.
22+
23+
### [05_Llama3.1-8B_Example](05_Llama3.1-8B_Example/)
24+
25+
Sample yaml configuration to benchmark `meta-llama/Llama-3.1-8B-Instruct`.
26+
1927
## Getting Help
2028

2129
- For general usage: See main [README](../README.md)

requirements/base.txt

Lines changed: 41 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
1-
# Base dependencies for the MLPerf Inference Endpoint Benchmarking System
2-
# These are the minimal dependencies required for the package to function
1+
# Dependencies for the MLPerf Inference Endpoint Benchmarking System
2+
# Install with: pip install -r requirements/base.txt
3+
4+
# ============================================================================
5+
# Core Runtime Dependencies
6+
# ============================================================================
37

48
# Core Python dependencies
59
typing-extensions>=4.0.0
@@ -8,18 +12,19 @@ typing-extensions>=4.0.0
812
psutil==6.1.1
913

1014
# Note: asyncio is part of Python 3.11+ standard library
11-
# Additional dependencies will be added as components are implemented
15+
# Async and networking
1216
aiohttp==3.12.15
13-
duckdb==1.4.0
14-
orjson==3.11.0
1517
pyzmq==27.0.2
1618
uvloop==0.21.0
17-
datasets==4.1.1
19+
20+
# Data handling
21+
duckdb==1.4.0
22+
orjson==3.11.0
23+
msgspec==0.19.0
1824
pydantic==2.12.0
1925
pydantic_core==2.41.1
20-
msgspec==0.19.0
2126

22-
# Needed to do tokenization for OSL reporting
27+
# Needed for tokenization and OSL reporting
2328
transformers==4.57.1
2429
numpy==2.3.4
2530
datasets==4.1.1
@@ -36,3 +41,31 @@ colorama==0.4.6
3641

3742
# Fix pytz-2024 import warning
3843
pytz==2025.2
44+
45+
# ============================================================================
46+
# Development Dependencies
47+
# ============================================================================
48+
49+
# Testing
50+
pytest>=7.0.0
51+
pytest-asyncio>=0.21.0
52+
pytest-cov>=4.0.0
53+
pytest-benchmark>=4.0.0
54+
pytest-timeout==2.4.0
55+
pytest-xdist==3.8.0
56+
57+
# Code quality
58+
ruff>=0.3.0
59+
60+
# Development tools
61+
pre-commit>=3.0.0
62+
63+
# Profiling tools
64+
line-profiler==5.0.0
65+
Pympler==1.1
66+
67+
# Documentation
68+
sphinx>=7.0.0
69+
sphinx-rtd-theme>=2.0.0
70+
sphinx-autodoc-typehints>=1.25.0
71+
myst-parser>=2.0.0

requirements/dev.txt

Lines changed: 0 additions & 29 deletions
This file was deleted.

0 commit comments

Comments
 (0)