2.0.0

New

• Enable content defined chunking by @lhoestq in #13
  • this enables fast deduped uploads

What's Changed

• Use uv project manager and add CI tests by @wengh in #12
• Update demo.ipynb by @CharlesCNorton in #10
• v2 by @lhoestq in #14

New Contributors

• @CharlesCNorton made their first contribution in #10

Full Changelog: 1.0.0...2.0.0

1.0.0

What's Changed

• Backport data source to pyspark 3 by @lhoestq in #11

Full Changelog: 0.1.1...1.0.0

0.1.1

What's Changed

• authentication to support reading gated/private datasets on HF
• support reading specific git revisions
• improved error messages
• Reduce requirements by deferring import until when it's actually needed by @wengh in #8
• Fix import for compatibility with older huggingface_hub by @wengh in #9

Full Changelog: 0.1.0...0.1.1

0.1.0

What's Changed

• Add basic HuggingFace Data Source Implementation by @allisonwang-db in #1
• initial pyproject.toml by @lhoestq in #2
• Add more features to huggingface reader by @allisonwang-db in #3
• Enable predicate pushdown by @lhoestq in #4
• Add HuggingFaceSink data source by @wengh in #5
• Support custom split name by renaming files by @wengh in #6
• Use same data source name for reader and writer by @wengh in #7

New Contributors

• @allisonwang-db made their first contribution in #1
• @lhoestq made their first contribution in #2

Full Changelog: https://github.com/huggingface/pyspark_huggingface/commits/0.1.0