Skip to content

Commit 53a6a28

Browse files
committed
feat: add Hugging Face Hub integration for dataset uploads in version 0.5.0
- Introduces push_to_hub() function for direct dataset uploads - Supports JSONL files, manifest, and dataset cards with metadata - Adds CLI flags for repository configuration and privacy options - Includes optional dependency on huggingface_hub>=0.20.0 - Provides example and test suite for HF Hub functionality
1 parent 1f55a33 commit 53a6a28

File tree

2 files changed

+19
-1
lines changed

2 files changed

+19
-1
lines changed

CHANGELOG.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,19 @@ The format is based on Keep a Changelog, and this project adheres to Semantic Ve
88

99
Nothing yet.
1010

11+
## [0.5.0] - 2025-01-11
12+
### Added
13+
- Hugging Face Hub integration for direct dataset uploads
14+
- `push_to_hub()` function in new `hf_hub` module to upload datasets to HF Hub
15+
- Uploads JSONL files (train.jsonl, val.jsonl), manifest.json, and auto-generated README.md
16+
- CLI flags: `--push-to-hub`, `--repo-id`, `--hf-token`, `--private`
17+
- Support for both public and private repositories
18+
- Auto-generated dataset cards with dataset statistics, model info, usage examples, and citation
19+
- Optional dependency: `huggingface_hub>=0.20.0` (install with `pip install toolsgen[hf]`)
20+
- Example in `examples/hf_hub_upload/` with dotenv configuration
21+
- Test suite for HF Hub functionality in `tests/test_hf_hub.py`
22+
- `push_to_hub` exported from main `toolsgen` package for easier imports
23+
1124
## [0.4.0] - 2025-01-10
1225
### Added
1326
- Quality tagging system for generated records

pyproject.toml

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ toolsgen = ["prompts/*.txt"]
1313

1414
[project]
1515
name = "toolsgen"
16-
version = "0.4.0"
16+
version = "0.5.0"
1717
description = "Generate tool-calling datasets from OpenAI-compatible tool specs"
1818
readme = "README.md"
1919
requires-python = ">=3.9"
@@ -40,6 +40,11 @@ dependencies = [
4040
"tqdm>=4.66.0",
4141
]
4242

43+
[project.optional-dependencies]
44+
hf = [
45+
"huggingface_hub>=0.20.0",
46+
]
47+
4348
[project.urls]
4449
Homepage = "https://github.com/atasoglu/toolsgen"
4550
Repository = "https://github.com/atasoglu/toolsgen"

0 commit comments

Comments
 (0)