Release 4.1.2

What's new

What's new

This patch release fixes an optional dependency installation issue and improves hashing performance.

🐛 Fixes

• Fixed chonkie optional install dependency name (semantic not semantics) [18a2f9d5]

⚡ Performance

• Switched from SHA256 to BLAKE3 for faster hashing [835e43dc]
  • Updated in setup.py, tests/test_parsing.py, and wdoc/utils/misc.py

Commits details since the last release

• [cb522e6] by @thiswillbeyourgithub, 10 seconds ago:
  bump version 4.1.1 -> 4.1.2

bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py

• [18a2f9d] by @thiswillbeyourgithub, 30 minutes ago:
  fix: chonkie optional install is called semantic not semantics
  Signed-off-by: thiswillbeyourgithub [email protected]

setup.py

• [835e43d] by @thiswillbeyourgithub, 31 minutes ago:
  perf: use blake3 instead of sha256
  Signed-off-by: thiswillbeyourgithub [email protected]

setup.py
tests/test_parsing.py
wdoc/utils/misc.py

Release 4.1.1

What's new

This release focuses on integrating Chonkie for semantic chunking, improving test reliability, and code quality enhancements through comprehensive linting.

Features

• Chonkie Semantic Chunking Integration
  • Implemented ChonkieSemanticSplitter using semantic chunking with memoization ([081e81a])
  • Added transform_documents method to ChonkieSemanticSplitter ([534cc90])
  • Replaced RecursiveCharacterTextSplitter with ChonkieSemanticSplitter in summarize.py ([77f1652])
  • Added chonkie to requirements ([7234f86])
  • Merged chonkie branch into dev ([f89390a])

Fixes

• Logging & Display

  • Fixed colors not appearing in loguru ([99502e7])
  • Fixed wrong logic for stdout color ([83e7fb9])

• Parsing & Type Hints

  • Allow LLM to mention "thinking" inside its thinking ([d2bca84])
  • Fixed error message when parsing thinking ([a50ec42])
  • Fixed typehint error for topk autoincrease ([615828a])

Refactor

• Split batch file loader into two files ([a0420fd])
• Comprehensive ruff linter run across codebase ([d9f7eac])
• Switched from black to ruff ([2d8a51b])
• Made ruff configuration less strict ([e04fc8d])

Tests

• DDG Test Improvements
  • Finally fixed DDG error not capturing output ([e1b2a87])
  • Capture DDG output properly ([d9c5ae9])
  • Set max DDG results to 10 to reduce failures ([ccdffd1])
  • Print output before error message ([66bf47c])
  • Better way to print output ([96c5186])
  • Don't use alias of grep ([a02ffbc])

Chore

• Updated pyfiglet font ([fd49cca])
• Cleaned up completed TODO items from README ([113c008], [ea4a99b], [8656327], [7206234], [4a4f4e8])
• Minor improvements ([b873469], [ca55c13])

Commits details since the last release

• [766c373] by @thiswillbeyourgithub, 4 seconds ago:
  bump version 4.1.0 -> 4.1.1

bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py

• [e1b2a87] by @thiswillbeyourgithub, 66 minutes ago:
  test: finally fixed the ddg error not capturing the output
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.sh

• [a02ffbc] by @thiswillbeyourgithub, 2 hours ago:
  test: dont use alias of grep
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.sh

• [66bf47c] by @thiswillbeyourgithub, 2 hours ago:
  test: print output before the error message
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.sh

• [4a4f4e8] by @thiswillbeyourgithub, 2 hours ago:
  todo: done the chonkie integration
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md

• [d9c5ae9] by @thiswillbeyourgithub, 2 hours ago:
  test: capture the ddg output
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.sh

• [96c5186] by @thiswillbeyourgithub, 2 hours ago:
  test: better way to print the output
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.sh

• [ccdffd1] by @thiswillbeyourgithub, 5 hours ago:
  test: set max ddg results to 10 because it fails too often
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.sh
tests/test_wdoc.py

• [a0420fd] by @thiswillbeyourgithub, 5 hours ago:
  new: split batch file loader into two files
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/batch_file_loader.py
wdoc/utils/load_recursive.py

• [7234f86] by @thiswillbeyourgithub, 6 hours ago:
  add chonkie to requirements
  Signed-off-by: thiswillbeyourgithub [email protected]

setup.py

• [f89390a] by @thiswillbeyourgithub, 25 hours ago:
  Merge branch 'chonkie' into dev

• [534cc90] by @thiswillbeyourgithub, 27 hours ago:
  feat: add transform_documents method to ChonkieSemanticSplitter
  Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4.5) [email protected]

wdoc/utils/misc.py

• [77f1652] by @thiswillbeyourgithub, 29 hours ago:
  refactor: replace RecursiveCharacterTextSplitter with ChonkieSemanticSplitter in summarize.py
  Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4.5) [email protected]

wdoc/utils/tasks/summarize.py

• [081e81a] by @thiswillbeyourgithub, 29 hours ago:
  feat: implement ChonkieSemanticSplitter using semantic chunking with memoization
  Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4.5) [email protected]

wdoc/utils/misc.py

• [615828a] by @thiswillbeyourgithub, 3 days ago:
  fix: typehint error for topk autoincrease
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/tasks/query.py

• [fd49cca] by @thiswillbeyourgithub, 3 days ago:
  better pyfiglet font
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/wdoc.py

• [99502e7] by @thiswillbeyourgithub, 3 days ago:
  fix: colors where not appearing in loguru
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/logger.py

• [83e7fb9] by @thiswillbeyourgithub, 3 days ago:
  fix wrong logic for stdout color
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/logger.py

• [b873469] by @thiswillbeyourgithub, 3 days ago:
  minor
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/logger.py

• [d2bca84] by @thiswillbeyourgithub, 3 days ago:
  fix: allow llm to mention thinking inside it's thinking
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [a50ec42] by @thiswillbeyourgithub, 3 days ago:
  fix: error message when parsing thinking
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [ca55c13] by @thiswillbeyourgithub, 4 days ago:
  minor
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md

• [113c008] by @thiswillbeyourgithub, 4 days ago:
  todo: no need to mention karakeep becauseit will be a loder
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md

• [ea4a99b] by @thiswillbeyourgithub, 4 days ago:
  todo: no more need to make an llm plugin because we support pipes now
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md

• [8656327] by @thiswillbeyourgithub, 4 days ago:
  todo: remove todo to move the task to their own file because it's done
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md

• [7206234] by @thiswillbeyourgithub, 4 days ago:
  todo: remove need for using dataclass to store tasks as its done
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md

• [d9f7eac] by @thiswillbeyourgithub, 4 days ago:
  run ruff linter everywhere

scripts/AnkiFiltered/AnkiFilteredDeckCreator.py
scripts/NtfySummarizer/NtfySummarizer.py
scripts/TheFiche/TheFiche.py
tests/test_parsing.py
tests/test_vectorstores.py
tests/test_wdoc.py
wdoc/main.py
wdoc/utils/batch_file_loader.py
wdoc/utils/customs/binary_faiss_vectorstore.py
wdoc/utils/customs/litellm_embeddings.py
wdoc/utils/embeddings.py
wdoc/utils/env.py
wdoc/utils/filters.py
wdoc/utils/interact.py
wdoc/utils/llm.py
wdoc/utils/loaders/init.py
wdoc/utils/loaders/anki.py
wdoc/utils/loaders/local_audio.py
wdoc/utils/loaders/local_html.py
wdoc/utils/loaders/local_video.py
wdoc/utils/loaders/logseq_markdown.py
wdoc/utils/loaders/online_media.py
wdoc/utils/loaders/pdf.py
wdoc/utils/loaders/shared_audio.py
wdoc/utils/loaders/youtube.py
wdoc/utils/misc.py
wdoc/utils/prompts.py
wdoc/utils/retrievers.py
wdoc/utils/tasks/parse.py
wdoc/utils/tasks/query.py
wdoc/utils/tasks/shared_query_search.py
wdoc/utils/tasks/types.py
wdoc/wdoc.py

• [e04fc8d] by @thiswillbeyourgithub, 4 days ago:
  less strict ruff
  Signed-off-by: thiswillbeyourgithub [email protected]

.pre-commit-config.yaml

• [2d8a51b] by @thiswillbeyourgithub, 4 days ago:
  switch from black to ruff
  Signed-off-by: thiswillbeyourgithub [email protected]

.pre-commit-config.yaml
setup.py

Release 4.1.0

What's new

What's new

This release focuses on robustness improvements, particularly around language detection, file loading, and error handling.

Features

• Task type system: Introduced dataclass-based task type storage for better type safety [7c95e3c]
• Source tag logging: Added failure count and success rate tracking to source tag logging [69dca45]

Fixes

• PowerPoint loader: Fixed TypeError when loading PowerPoint files [ebfc66c]
• Anki loader: Resolved forward reference error [73924e1]
• Language detection: Fixed potential edge case issue [2d928ab]
• Infinite loop detection:
  • Replaced simple loop counter with hash-based detection mechanism [bb147b3]
  • Adjusted loop counter threshold [fcf9ca5]

Enhancements

• Language detection improvements:
  • Better exception handling [0b9c6da]
  • Reduced debug log verbosity [d7589cc]
  • General improvements [c0e2ce7]
• Batch file loader: Reduced verbosity of progress logging [d207d98]
• Testing: Improved model detection logic [5257c5a]
• Post-install: Use logger.error instead of print during installation [c0795e9]

Refactoring

• wdoc class: Added dynamic interaction_settings property [f806b98]
• Type hints: Improved type annotations across multiple modules [a94a889, 920e5d3]

Documentation

• Help text: Fixed powerpoint filetype documentation incorrectly mentioning .doc/.docx instead of .ppt/.pptx [e9b29eb]

Dependencies

• Bumped litellm to enable latest OpenRouter pricing [577e6f6]

Maintenance

• Removed debug print statement [80f7f32]
• Better warning messages [faa5d3b]
• Fixed setup.py logger usage [4a672c1]

Commits details since the last release

• [5adc87e] by @thiswillbeyourgithub, 72 seconds ago:
  bump version 4.0.4 -> 4.1.0

bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py

• [0b9c6da] by @thiswillbeyourgithub, 2 hours ago:
  enh: better exception catcher in language detecction
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [80f7f32] by @thiswillbeyourgithub, 3 hours ago:
  remove a debug print
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders/init.py

• [d7589cc] by @thiswillbeyourgithub, 3 hours ago:
  enh: language detector reduce debug logs
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [c0e2ce7] by @thiswillbeyourgithub, 3 hours ago:
  enh: language detector
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [2d928ab] by @thiswillbeyourgithub, 3 hours ago:
  fix: potential issue in edge case when detecting language
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [69dca45] by @thiswillbeyourgithub, 4 hours ago:
  feat: add failure count and success rate to source tag logging
  Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4.5) [email protected]

wdoc/utils/batch_file_loader.py

• [f806b98] by @thiswillbeyourgithub, 4 hours ago:
  refactor: add dynamic interaction_settings property to wdoc class
  Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4.5) [email protected]

wdoc/wdoc.py

• [a94a889] by @thiswillbeyourgithub, 4 hours ago:
  minor: type hints
  Signed-off-by: thiswillbeyourgithub
  [email protected]

wdoc/wdoc.py

• [7c95e3c] by @thiswillbeyourgithub, 4 hours ago:
  new: use a dataclass to store the type of tasks
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/main.py
wdoc/utils/batch_file_loader.py
wdoc/utils/loaders/init.py
wdoc/utils/misc.py
wdoc/utils/retrievers.py
wdoc/utils/tasks/parse.py
wdoc/utils/tasks/summarize.py
wdoc/utils/tasks/types.py
wdoc/wdoc.py

• [5257c5a] by @thiswillbeyourgithub, 5 hours ago:
  better way to check for testing model
  Signed-off-by: thiswillbeyourgithub
  [email protected]

wdoc/utils/llm.py
wdoc/utils/misc.py
wdoc/wdoc.py

• [73924e1] by @thiswillbeyourgithub, 7 hours ago:
  fix: forward reference error in anki
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders/anki.py

• [ebfc66c] by @thiswillbeyourgithub, 7 hours ago:
  fix: typeerror when loading powerpoint files
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders/powerpoint.py

• [d207d98] by @thiswillbeyourgithub, 25 hours ago:
  enh: reduice verbosity of something that looked like an infinite loop but was not
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/batch_file_loader.py

• [920e5d3] by @thiswillbeyourgithub, 25 hours ago:
  fix: typehint in pdf
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders/pdf.py

• [bb147b3] by @thiswillbeyourgithub, 25 hours ago:
  refactor: replace loop counter with hash-based infinite loop detection
  Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4.5) [email protected]

wdoc/utils/batch_file_loader.py

• [4a672c1] by @thiswillbeyourgithub, 25 hours ago:
  fix: actually no we can't use loguru in setup.py
  Signed-off-by: thiswillbeyourgithub [email protected]

setup.py

• [faa5d3b] by @thiswillbeyourgithub, 26 hours ago:
  minor: better warning
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/batch_file_loader.py

• [fcf9ca5] by @thiswillbeyourgithub, 26 hours ago:
  fix: the loop counter has to be high enough to detect infinite loop
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/batch_file_loader.py

• [e9b29eb] by @thiswillbeyourgithub, 26 hours ago:
  doc: powerpoint filetype doc mentionned .doc and .docx instead of .ppt and .pptx
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/docs/help.md

• [577e6f6] by @thiswillbeyourgithub, 5 days ago:
  bump litellm, allows using the latest openrouter price by litellm
  Signed-off-by: thiswillbeyourgithub [email protected]

setup.py

• [c0795e9] by @thiswillbeyourgithub, 6 days ago:
  enh: use logger.error instead of print during the postinstall process
  Signed-off-by: thiswillbeyourgithub [email protected]

setup.py

Release 4.0.2

What's new

What's new

This release focuses on bug fixes, performance improvements, and code cleanup related to docstore filtering and retriever functionality.

🐛 Fixes

• Docstore filtering improvements

  • Fixed missing arguments when calling filter_docstore ([1a2442d])
  • Fixed type hints for filter_docstore ([d96c2f3]) and create_filter_metadata ([ee9cc6f])
  • Corrected docstore serialization behavior ([9c1d967])

• Retriever fixes

  • Fixed parent retriever when loading from embeddings ([cf9171d])
  • Fixed type hint for retrievers in edge cases ([39951a8])

⚡ Performance

• Do not store nor serialize the unfiltered docstore ([d29d3a5], [a9a0a35])
  • Renamed filter_docstore to filter_vectorstore for clarity

✨ Features

• Added timing measurements for docstore serialization and deletion ([375c1a1])

🧹 Chores

• Removed unused imports ([de18c65], [6525464])
• Version bump 4.0.1 → 4.0.2 ([83f23dd])

Commits details since the last release

• [83f23dd] by @thiswillbeyourgithub, 9 seconds ago:
  bump version 4.0.1 -> 4.0.2

bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py

• [a9a0a35] by @thiswillbeyourgithub, 17 minutes ago:
  rename filter_docstore to filter_vectorstore
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/filters.py
wdoc/wdoc.py

• [d29d3a5] by @thiswillbeyourgithub, 18 minutes ago:
  perf: do not store nor serialize the unfiltered docsstore
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/filters.py
wdoc/wdoc.py

• [cf9171d] by @thiswillbeyourgithub, 24 minutes ago:
  fix: parent retriever when loading from embeddings
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/retrievers.py

• [de18c65] by @thiswillbeyourgithub, 37 minutes ago:
  remove unused import
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/retrievers.py

• [39951a8] by @thiswillbeyourgithub, 37 minutes ago:
  fix: typehint of retrievers in an edge case
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/retrievers.py

• [375c1a1] by @thiswillbeyourgithub, 45 minutes ago:
  feat: add timing measurements for docstore serialization and deletion
  Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4.5) [email protected]

wdoc/utils/filters.py

• [6525464] by @thiswillbeyourgithub, 48 minutes ago:
  remove unused import
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/filters.py

• [9c1d967] by @thiswillbeyourgithub, 48 minutes ago:
  fix: actually the unfiltered docstore is serialized
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/filters.py
wdoc/wdoc.py

• [ee9cc6f] by @thiswillbeyourgithub, 67 minutes ago:
  fix: wrong type hint for create_filter_metadata
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/filters.py

• [d96c2f3] by @thiswillbeyourgithub, 69 minutes ago:
  fix: wrong type hint for filter_docstore
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/filters.py

• [1a2442d] by @thiswillbeyourgithub, 70 minutes ago:
  fix: forgot to pass arguments to filter_docstore
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/wdoc.py

Release 4.0.1

What's new

What's new

This release focuses on langfuse v3 compatibility and improved error handling.

🐛 Fixes

• Langfuse v3 compatibility

  • [89f5132] Update callback import for langfuse v3
  • [07257e0] Use langfuse opentelemetry for v3

• Document loading robustness

  • [3039bcf] Prevent crash when no documents remain after transform_documents
  • [101c7f7] Add assertion to verify documents were found

📝 Documentation

• [56866d1] Add warning for using youtube audio backend instead of whisper or deepgram

🔧 Maintenance

• [fb49e60] Bump version 4.0.0 → 4.0.1

Commits details since the last release

• [fb49e60] by @thiswillbeyourgithub, 13 seconds ago:
  bump version 4.0.0 -> 4.0.1

bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py

• [07257e0] by @thiswillbeyourgithub, 3 minutes ago:
  fix: use langfuse opentelemetry for v3
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [56866d1] by @thiswillbeyourgithub, 11 minutes ago:
  doc: add warning for using the youtube audio backend instead of whisper or deepgram
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders/youtube.py

• [89f5132] by @thiswillbeyourgithub, 14 minutes ago:
  fix: langfuse callback import changed for langfuse v3
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [3039bcf] by @thiswillbeyourgithub, 20 minutes ago:
  fix: do not crash if no documents after transform_documents is ran
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders/init.py

• [101c7f7] by @thiswillbeyourgithub, 29 minutes ago:
  add assert that docs were found
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders/init.py

Release 4.0.0

What's new

What's new

This release focuses on major performance improvements through lazy loading and deferred imports, extensive code refactoring for better maintainability, and improved testing infrastructure.

⚡ Performance

• Significantly faster startup time through deferred imports and lazy loading [52985d5, dce3c24, 3ffaec3]
  • Moved litellm imports to run only when needed [52985d5]
  • Deferred requests import [0b4c2fb]
  • Removed eager imports from __init__.py files [306d4ca]
  • Moved imports in loaders, embeddings, and core modules [de1cecc, 08b9206, fd2dcba, 1838e0f, 1bd4ced, f1740c4, 2b3d9e8, 6fbe51d, 6c74d8e, f306325]
  • Added lazy loading for document loaders with WDOC_LAZY_LOAD env var [7fc5fad, ce10c4b]

🔧 Fixes

• Fixed forward reference type hints across multiple modules [fd6a7e7, 22b44b4, 15a2746]
• Fixed signature wrapping for parse function [29dbf5d]
• Fixed API tests for DuckDuckGo and OpenRouter [8b9ebc2, 8f511dd, 32e036d, 048f99e]
• Fixed missing filetype handling in edge cases [0422dec]
• Fixed error for Word document loading [8cad00d]
• Fixed lazy loading logic (was reversed) [a35446f]
• Fixed query_task and search_task output handling [6f633e8, 8b95a81]
• Fixed error when summary doesn't output to file using pipe [2a85a6b]
• Fixed imports in loaders [ebd4558, af85343, 986abd2, 4e61a6f]
• Added missing audioop-lts requirement for Python 3.13+ [56bd634]

♻️ Refactoring

• Modularized loaders: Split monolithic loader file into separate modules [df1a0ad, d3ed873, f0a3fce, b249068, 984a8d3, def441f, fb421cc]
  • Created dedicated files for PDF, Anki, URL, audio, HTML, and other loaders
  • Enabled lazy loading of loader modules [7fc5fad]
• Extracted task-specific functions to separate modules:
  • Moved parse_doc to utils/tasks/parse.py [1c7c6e4]
  • Moved query/search retrieval logic to task modules [7982051, c2e6142]
  • Moved evaluate_doc_chain to shared_query_search.py [8965c48]
  • Extracted query splitting logic to shared utility [4bb54a5]
  • Moved source_replace to query.py [0ce5f4f]
  • Moved autoincrease_top_k to query.py [38e82b4]
• Split search and query task methods with better type hints [1d94644, 824f395, 319b8eb]
• Moved debug_exceptions to logger module [99cc99f]
• Moved VectorStore filtering code to filters.py [de4ce57]
• Added wdocSummary dataclass for type hinting [9fc51c0, 92f5c47]
• Added lazy caching for all_texts property [79b1661, 7b45948]
• Removed obsolete import_tricks.py [5116616]

🧪 Testing

• Improved test cleanup and temp folder removal [a768642, 35ef63e, c149f5d, 913378a]
• Better verbose output in cost tests [342ad3f]
• Use Mistral for OpenRouter API tests (zero data retention) [8f511dd]
• Added shell-based CLI test script for more reliable testing [cc74a84, 4170567]
• Added check for wdoc[full] installation [7cb9a3c]
• Updated Ollama embedding test to use embeddingsgemma [4d47631]
• Improved test assertions with more info [3d0f947]

📦 Dependencies

• Bumped langchain version [98fd2cb]
• Bumped litellm version [7aa2ce1]
• Bumped langfuse version (litellm bug fix) [fc16e5e]
• Updated general dependencies [616457c]
• Added unstructured to required dependencies [c98d0e9]
• Added bumpver to dev packages [54be0e2]

✨ Features

• Added wdoc[full] installation option for all optional dependencies [6321942]
• Added beartype runtime type validation for numpy arrays [691dbff]
• Prioritize throughput and Groq when using OpenRouter [f049846]
• Enable lazy loading of imports by default [7c2e397]

📝 Documentation

• Updated default models to latest Gemini in README and help [761ddd1, 0868086, 78e562f]
• Clarified that binary embeddings are not always better [fb611c4]
• Added link explaining fixed cache of LLM issue [fdc3c64]
• Improved docstrings for summarization functions [a06f570]
• Added docstring for VectorStore filtering [2bd8dcb]

🎨 Code Quality

• PEP8 formatting improvements [559fcc0, 13dd7d44, e10fb05]
• Removed unused imports [16c518e, cb9563e, b5a42ef]
• Improved type hints [78d399f, a801718, 0875b23]
• Import logger first to set log level [2dead91]
• Removed if True statement [263a45d]

🔖 Version

• Bumped version 3.3.1 → 4.0.0 [e1548c4]

Commits details since the last release

• [e1548c4] by @thiswillbeyourgithub, 12 seconds ago:
  bump version 3.3.1 -> 4.0.0

bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py

• [54be0e2] by @thiswillbeyourgithub, 47 seconds ago:
  add bumpver to dev packages
  Signed-off-by: thiswillbeyourgithub [email protected]

setup.py

• [37e80a7] by @thiswillbeyourgithub, 2 minutes ago:
  doc: todo
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md

• [a768642] by @thiswillbeyourgithub, 3 minutes ago:
  better trash removal
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/run_all_tests.sh

• [35ef63e] by @thiswillbeyourgithub, 16 minutes ago:
  less verbose test removal of temp folders
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/run_all_tests.sh

• [c149f5d] by @thiswillbeyourgithub, 30 minutes ago:
  enh: delete cache dir at start of test
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/run_all_tests.sh

• [ae6f28a] by @thiswillbeyourgithub, 32 minutes ago:
  minor: name of a test
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_wdoc.py

• [342ad3f] by @thiswillbeyourgithub, 39 minutes ago:
  better verbose output in cost test
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_wdoc.py

• [8f511dd] by @thiswillbeyourgithub, 63 minutes ago:
  fix: use mistral instead of openai when testing api from openrouter because it supports zero data retention
  Signed-off-by: thiswillbeyourgithub
  [email protected]

tests/test_wdoc.py

• [8b9ebc2] by @thiswillbeyourgithub, 65 minutes ago:
  fix: api test for ddg from the shell
  Signed-off-by: thiswillbeyourgithub
  [email protected]

tests/test_cli.sh

• [fd6a7e7] by @thiswillbeyourgithub, 72 minutes ago:
  fix forward reference for typehinting
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/wdoc.py

• [1428497] by @thiswillbeyourgithub, 72 minutes ago:
  fix forgot to test api using cli script
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/run_all_tests.sh

• [22b44b4] by @thiswillbeyourgithub, 85 minutes ago:
  fix forward reference type hints
  Signed-off-by: thiswillbeyourgithub
  [email protected]
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/tasks/query.py
wdoc/wdoc.py

• [15a2746] by @thiswillbeyourgithub, 89 minutes ago:
  fix forward reference type hints
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/llm.py
wdoc/utils/retrievers.py
wdoc/utils/tasks/query.py
wdoc/utils/tasks/summarize.py

• [29dbf5d] by @thiswillbeyourgithub, 2 hours ago:
  fix: signature wrapping for parse
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/main.py
wdoc/wdoc.py

• [7aa2ce1] by @thiswillbeyourgithub, 2 hours ago:
  bump litellm
  Signed-off-by: thiswillbeyourgithub [email protected]

setup.py

• [98fd2cb] by @thiswillbeyourgithub, 2 hours ago:
  bump langchain version
  Signed-off-by: thiswillbeyourgithub
  [email protected]

setup.py

• [616457c] by @thiswillbeyourgithub, 2 hours ago:
  bump dependencies
  Signed-off-by: thiswillbeyourgithub [email protected]

setup.py

• [dce3c24] by @thiswillbeyourgithub, 2 hours ago:
  actually use lazy import for litellm
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [52985d5] by @thiswillbeyourgithub, 3 hours ago:
  better startup time by defering litellm import
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/customs/litellm_embeddings.py
wdoc/utils/embeddings.py
wdoc/utils/llm.py
wdoc/utils/loaders/shared_audio.py
wdoc/utils/misc.py
wdoc/utils/retrievers.py
wdoc/utils/tasks/query.py
wdoc/utils/tasks/summarize.py
wdoc/wdoc.py

• [559fcc0] by @thiswillbeyourgithub, 3 hours ago:
  pep8
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/init.py

• [0b4c2fb] by @thiswillbeyourgithub, 3 hours ago:
  defer requests import
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders/shared_audio.py
wdoc/utils/misc.py

• [78d399f] by @thiswillbeyourgithub, 3 hours ago:
  type hint for multiquery retriever
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/retrievers.py

• [13dd7d1] by @thiswillbeyourgithub, 3 hours ago:
  pep8
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/main.py
wdoc/utils/customs/binary_faiss_vectorstore.py
wdoc/utils/embeddings.py
wdoc/utils/env.py
wdoc/utils/interact.py
wdoc/utils/llm.py
wdoc/utils/misc.py
wdoc/utils/prompts.py
wdoc/utils/tasks/parse.py
wdoc/utils/tasks/query.py
wdoc/utils/tasks/search.py

• [6ce23d4] by @thiswillbeyourgithub, 4 hours ago:
  fix import statements
  Signed-off-by: thiswillbeyourgithub <[email protected]...

Release 3.3.1

What's new

This release focuses on improving code quality through comprehensive type hint fixes and enhanced testing infrastructure.

🔧 Fixes

• Type Hints: Comprehensive type hint improvements across the codebase

  • Binary FAISS vectorstore type hints ([e46ed4a], [ac02a65], [da864cd], [95c3705], [81b36cb], [be3f352])
  • Loader function type hints ([e6fcad8], [e65abad], [b624373])
  • Semantic batching type hints ([f3a5289], [dd6ad29])
  • Prompt template type hints ([1b4ec86], [bc56beb])
  • General type hint fixes ([d4e99fd])

• Model Compatibility: Fixed issue where some models consider <answer> as implying </think> ([09684bb])

• Langchain Integration: Fixed callable_chain compatibility by creating runnables without decorators ([0c89cac])

✨ Enhancements

• Type Checking: Replaced manual type checking with import hook system ([56b353a], [6b3ddab])
• Logging: Reduced verbosity of litellm logging ([9a4a69c])
• Search: Added duplicate check for DuckDuckGo search results ([f68c8a4])

🧪 Tests

• Added comprehensive test for DuckDuckGo search functionality ([7dbd3c2])
• Fixed existing CLI tests ([781f6d6])

📦 Version

• Bumped version from 3.3.0 to 3.3.1 ([0690df9])

Commits details since the last release

• [0690df9] by @thiswillbeyourgithub, 41 seconds ago:
  bump version 3.3.0 -> 3.3.1

bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py

• [7dbd3c2] by @thiswillbeyourgithub, 8 hours ago:
  test: add test for DuckDuckGo search functionality
  Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) [email protected]

tests/test_wdoc.py

• [781f6d6] by @thiswillbeyourgithub, 9 hours ago:
  fix: test
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.py

• [e46ed4a] by @thiswillbeyourgithub, 14 hours ago:
  fix: typehint for marginal score search
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/customs/binary_faiss_vectorstore.py

• [ac02a65] by @thiswillbeyourgithub, 18 hours ago:
  fix: type hint of binary faiss
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/customs/binary_faiss_vectorstore.py

• [f68c8a4] by @thiswillbeyourgithub, 19 hours ago:
  add check for duplicate ddg result
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/batch_file_loader.py

• [da864cd] by @thiswillbeyourgithub, 19 hours ago:
  fix: binary faiss type hints
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/customs/binary_faiss_vectorstore.py

• [9a4a69c] by @thiswillbeyourgithub, 19 hours ago:
  enh: tune down the verbosity of litellm
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/wdoc.py

• [e6fcad8] by @thiswillbeyourgithub, 19 hours ago:
  fix: type hint of load_one_doc
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders.py

• [e65abad] by @thiswillbeyourgithub, 20 hours ago:
  Revert "fix: typehint of load_one_doc"
  This reverts commit f0037b54ac5ce317442e672f12e1da266b58c5c1.

wdoc/utils/loaders.py

• [b624373] by @thiswillbeyourgithub, 20 hours ago:
  fix: typehint of load_one_doc
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders.py

• [95c3705] by @thiswillbeyourgithub, 20 hours ago:
  fix: typehints for binary faiss
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/customs/binary_faiss_vectorstore.py

• [f3a5289] by @thiswillbeyourgithub, 20 hours ago:
  fix: type hints for semantic batching
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/tasks/query.py

• [d4e99fd] by @thiswillbeyourgithub, 20 hours ago:
  fix: type hints
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/wdoc.py

• [81b36cb] by @thiswillbeyourgithub, 21 hours ago:
  forgot some type hint for binary faiss
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/customs/binary_faiss_vectorstore.py

• [dd6ad29] by @thiswillbeyourgithub, 21 hours ago:
  fix: wrong typehint in semantic_batch
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/tasks/query.py

• [09684bb] by @thiswillbeyourgithub, 21 hours ago:
  fix: some models consider than implied
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [0c89cac] by @thiswillbeyourgithub, 22 hours ago:
  fix: actually callable_chain does not work for langchain so we have to make runnables without decorators
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/customs/callable_runnable.py
wdoc/utils/misc.py
wdoc/utils/tasks/query.py
wdoc/wdoc.py

• [6b3ddab] by @thiswillbeyourgithub, 22 hours ago:
  new: remove the ubiquitous optional_typecheck decorator
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/main.py
wdoc/utils/batch_file_loader.py
wdoc/utils/embeddings.py
wdoc/utils/interact.py
wdoc/utils/llm.py
wdoc/utils/loaders.py
wdoc/utils/logger.py
wdoc/utils/misc.py
wdoc/utils/prompts.py
wdoc/utils/retrievers.py
wdoc/utils/tasks/query.py
wdoc/utils/tasks/summarize.py
wdoc/utils/typechecker.py
wdoc/wdoc.py

• [56b353a] by @thiswillbeyourgithub, 22 hours ago:
  new: neutralize manual type checking and instead use the import hook
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/init.py
wdoc/utils/customs/callable_runnable.py
wdoc/utils/misc.py
wdoc/utils/tasks/query.py
wdoc/utils/typechecker.py
wdoc/wdoc.py

• [be3f352] by @thiswillbeyourgithub, 22 hours ago:
  fix: type hints in binary faiss
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/customs/binary_faiss_vectorstore.py

• [1b4ec86] by @thiswillbeyourgithub, 23 hours ago:
  fix: wrong type for Prompts class
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/prompts.py

• [bc56beb] by @thiswillbeyourgithub, 23 hours ago:
  add type checking to prompt template
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/prompts.py

Release 3.3.0

What's new

This release focuses on adding DuckDuckGo web search capabilities and introducing binary embeddings support for more efficient vector storage.

✨ New Features

DuckDuckGo Web Search Integration

• [372fe57] Add DuckDuckGo search support with URL extraction and metadata
• [273195e] Support wdoc wdb "your query" shorthand for web search
• [03bfe08] Add DuckDuckGo search tests and documentation

Binary Embeddings Support

• [c528bad] Add support for binary embeddings with 8x memory reduction
• [8f65197] Enable FAISS vectorstore compression by default
• [37ebd97] Create CompressedFAISS subclass with zlib compression

🐛 Bug Fixes

Core Functionality

• [0d72efd] Fix wrong decorator used for load_one_doc
• [edcf671] Fix ddg_region type (str not int)
• [66ab177] Fix type hints for ddg_safesearch and loading_failure
• [957936c] Use keyword arguments instead of fire when calling wdoc

Testing Environment

• [d3de58e] Fix piped input/output handling in pytest environment
• [42ff516] Prevent pipe usage in pytest environment
• [c78dc0b] Add pytest environment detection

🧪 Testing Improvements

• [1b09996] Fix the run_all_test script
• [8ed1d0c] Add comprehensive DuckDuckGo search functionality tests
• [b184177] Split CLI tests into separate test_cli.py file
• [9d7fe9c] Split parsing tests into separate test_parsing.py file
• [12b012d] Move vector store tests to dedicated test file

📚 Documentation

• [d7d6b04] Explain how to run tests in README
• [dc15001] Clarify how to disable parallel processing
• [df4b79f] Document debug mode's effect on loading_failure default
• [1832299] Add shell examples for DuckDuckGo usage

🔧 Enhancements

CLI/UX Improvements

• [7e994a6] Rename parse_file function to parse_doc
• [4aa247e] Re-ask for input when empty query provided in CLI
• [57d5d5f] Fix Fire's pager issue in CLI

Performance THISISANAMPERSAND Reliability

• [68d4c75] Bump LiteLLM to latest version for improved startup time
• [ab9c5e9] Add parallel processing option for Whisper audio splits
• [6b13044] Add loop counter and crash protection for recursive file processing

🔄 Version Update

• [6435133] Bump version from 3.2.5 → 3.3.0

Commits details since the last release

• [6435133] by @thiswillbeyourgithub, 36 minutes ago:
  bump version 3.2.5 -> 3.3.0

bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py

• [1b09996] by @thiswillbeyourgithub, 24 hours ago:
  test: fix the run_all_test script
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/run_all_tests.sh

• [d7d6b04] by @thiswillbeyourgithub, 24 hours ago:
  doc: explain how to run the tests
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md

• [62cc2ce] by @thiswillbeyourgithub, 24 hours ago:
  fix: ddg test
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.py

• [0d72efd] by @thiswillbeyourgithub, 24 hours ago:
  fix: wrong decorator used for load_one_doc
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders.py

• [dc15001] by @thiswillbeyourgithub, 24 hours ago:
  doc: clarify how to disable parallel processing
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/docs/help.md

• [e0453cb] by @thiswillbeyourgithub, 24 hours ago:
  minor: mention a type hint
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/batch_file_loader.py

• [edcf671] by @thiswillbeyourgithub, 24 hours ago:
  fix: ddg_region is actually a str not an int
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [df4b79f] by @thiswillbeyourgithub, 24 hours ago:
  doc: mention that debug changes the default value for loading_failure
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/docs/help.md

• [66ab177] by @thiswillbeyourgithub, 25 hours ago:
  fix: type of ddg_safesearch and loading_failure should be Literal
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [98b0867] by @thiswillbeyourgithub, 25 hours ago:
  doc: explain that loading_failure defaultto crash when parsing
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/docs/help.md

• [90eacb3] by @thiswillbeyourgithub, 25 hours ago:
  test: ddg should use us region by default
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.py

• [c8b1944] by @thiswillbeyourgithub, 25 hours ago:
  test: less severe check for pipes
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.py

• [6e12e5c] by @thiswillbeyourgithub, 25 hours ago:
  test: remove one -n auto arg
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/run_all_tests.sh

• [d3de58e] by @thiswillbeyourgithub, 2 days ago:
  fix: actually inside pytest we should not bypass piped input but only piped output
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/env.py
wdoc/utils/misc.py

• [5715bc4] by @thiswillbeyourgithub, 2 days ago:
  test: add env variable to detect if being called by pytest
  Signed-off-by: thiswillbeyourgithub
  [email protected]

tests/conftest.py

• [42ff516] by @thiswillbeyourgithub, 2 days ago:
  new: do not allow using pipe input or output in pytest environment
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/misc.py

• [c78dc0b] by @thiswillbeyourgithub, 2 days ago:
  new: detect when wdoc is called in pytest environment
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_wdoc.py
wdoc/utils/env.py
wdoc/utils/misc.py

• [fca39c0] by @thiswillbeyourgithub, 2 days ago:
  test: missing oneoff and failsafe when testing ddg
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.py

• [b2b4cf1] by @thiswillbeyourgithub, 2 days ago:
  test: fix missing quotation sign for args
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.py

• [13409b1] by @thiswillbeyourgithub, 2 days ago:
  test: fix a timeout not long enough
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.py

• [957936c] by @thiswillbeyourgithub, 2 days ago:
  fix: use keyword aguments instead of fire when calling wdoc
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/main.py

• [b034337] by @thiswillbeyourgithub, 2 days ago:
  minor
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/main.py

• [b44d730] by @thiswillbeyourgithub, 2 days ago:
  fix: replacing ddg_max_result
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/main.py

• [dfcaf3b] by @thiswillbeyourgithub, 2 days ago:
  fix: wrong way to replace ddg_max_result to ddg_max_results
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/main.py

• [adc991a] by @thiswillbeyourgithub, 2 days ago:
  actually no
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders.py

• [9ab4cbf] by @thiswillbeyourgithub, 2 days ago:
  fix: type hint of load_one_doc can be a list of string in case of error
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/loaders.py

• [5f7fcf4] by @thiswillbeyourgithub, 2 days ago:
  typo: Nvidia instead of NVidia
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md
tests/test_cli.py
wdoc/docs/examples.md

• [03bfe08] by @thiswillbeyourgithub, 2 days ago:
  test: add test for ddg search
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/run_all_tests.sh

• [48165fa] by @thiswillbeyourgithub, 2 days ago:
  test: clearer echo
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/run_all_tests.sh

• [08cac94] by @thiswillbeyourgithub, 2 days ago:
  remove unused import
  Signed-off-by: thiswillbeyourgithub [email protected]

tests/test_cli.py

• [3aabd2d] by @thiswillbeyourgithub, 2 days ago:
  style: format test_cli.py with linter
  Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) [email protected]

tests/test_cli.py

• [8ed1d0c] by @thiswillbeyourgithub, 2 days ago:
  feat: add test for DuckDuckGo search functionality with NVIDIA query
  Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) [email protected]

tests/test_cli.py

• [a8e3e04] by @thiswillbeyourgithub, 2 days ago:
  test: add test for DuckDuckGo search with NVIDIA query
  Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) [email protected]

tests/test_cli.py

• [1832299] by @thiswillbeyourgithub, 2 days ago:
  doc: add shell example for using duckduckgo
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/docs/examples.md

• [e6c4641] by @thiswillbeyourgithub, 2 days ago:
  typo
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/docs/examples.md

• [917ee51] by @...

Release 3.2.5

What's new

This release brings several improvements to command-line argument handling and filetype detection, along with key bug fixes and build process enhancements.

✨ Features

• CLI & Filetype Detection:
  • Enhanced handling of multiple implicit CLI arguments by utilizing a filetype detector to infer appropriate actions ([ab76610]).
  • Introduced a specific exception for clearer error reporting when filetypes cannot be inferred, including a new error for an undetectable filetype ([520f4ce], [39af223]).
• Build Process:
  • Integrated sphinx-apidoc into the ReadTheDocs build process via a pre-build job in .readthedocs.yaml ([cc86c7b]).

🐛 Fixes

• Corrected an issue with sys.argv handling that led to duplicated arguments ([e7cf185]).
• Updated litellm dependency to resolve crashes experienced on Windows environments ([cfff0ac]), see #20.

🛠️ Improvements & Refactoring

• Filetype Detection Internals:
  • Refactored filetype detection logic into a dedicated function for better modularity ([b453748]).
  • Added debug logging to the filetype detector to aid in troubleshooting ([05966c6]).
• Code Quality:
  • Improved documentation by adding docstrings to custom exception classes ([8e6ca1a]).

Chores

• Version bumped to 3.2.5 ([82b7f81]).

Commits details since the last release

• [82b7f81] by @thiswillbeyourgithub, 19 minutes ago:
  bump version 3.2.4 -> 3.2.5

bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py

• [e7cf185] by @thiswillbeyourgithub, 3 minutes ago:
  fix: badly handled sys.argv was duplicating args
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/main.py

• [cfff0ac] by @thiswillbeyourgithub, 19 minutes ago:
  fix: bump version of litellm because windows crashes
  Signed-off-by: thiswillbeyourgithub [email protected]

setup.py

• [cc86c7b] by @thiswillbeyourgithub (aider), 2 days ago:
  feat: add pre-build job to run sphinx-apidoc in .readthedocs.yaml

.readthedocs.yaml

• [ab76610] by @thiswillbeyourgithub, 2 days ago:
  new: use the filetype detector to infer what to do in case of multiple implicit arguments from the cli
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/main.py

• [05966c6] by @thiswillbeyourgithub, 2 days ago:
  enh: add debug prints to the filetype detector
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/batch_file_loader.py

• [520f4ce] by @thiswillbeyourgithub, 2 days ago:
  new: use a specific exception when we can't infer the filetype
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/batch_file_loader.py

• [8e6ca1a] by @thiswillbeyourgithub, 2 days ago:
  add docstring to some exceptions
  Signed-off-by: thiswillbeyourgithub
  [email protected]

wdoc/utils/errors.py

• [39af223] by @thiswillbeyourgithub, 2 days ago:
  add an error for undetectable filetype
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/errors.py

• [b453748] by @thiswillbeyourgithub, 2 days ago:
  new: put the filetype detection code in a separate function
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/batch_file_loader.py

Release 3.2.4

What's new

This release primarily focuses on significant documentation enhancements, crucial bug fixes for stability and build processes, and introduces updated dependencies and tokenization.

✨ New Features

• Upgraded default token estimation to use gpt-4o-mini tokenizer, replacing gpt-3.5-turbo ([6d41817]).
• Integrated the latest yt-dlp for YouTube downloads ([ab207b4]).
• Environment variable documentation is now automatically added to the EnvDataclass class __doc__ ([ed9dd38]).

🐛 Bug Fixes

• Resolved a crash on ReadTheDocs caused by missing yt-dlp dependency ([f5068a3]).
• Fixed an issue where accessing env.__class__ on ReadTheDocs could cause a crash ([4e180f0]).
• Corrected relative import paths in wdoc that were preventing Sphinx API documentation builds ([ade5930]).
• Fixed issues with the Sphinx API command in the FAQ section of the README ([38008aa], [ff093a2]).
• Ensured collapsible bars in documentation function correctly ([3cef833]).

📚 Documentation & Refinements

• Extensive updates and fixes to Sphinx documentation generation and content:
  • Addressed outdated Sphinx documentation files ([90bde99]).
  • Improved API autodoc parameters for clearer documentation ([243de66]).
  • Excluded private and special members from documentation ([7abedd4]).
  • Added Sphinx command to FAQ in README ([1e6602e]) and removed private members from it ([11ae11b]).
  • Updated copyright year to 2025 ([bd7e3c5]).
• Streamlined documentation structure and configuration:
  • Removed unused make files (Makefile, make.bat) for documentation ([07b0a7d]).
  • Removed unused argument for theme flyout display ([17bc5e6]).
  • Removed unused templates path ([6bffa20]) and CSS ([712df08]).
  • Removed duplicate README from the documentation source ([2b93162]).
  • Added a documentation table to the main index ([1dfe2b3]).

⚙️ Build & Chores

• Bumped version to 3.2.4 ([ed7a9c7]).

Commits details since the last release

• [ed7a9c7] by @thiswillbeyourgithub, 20 seconds ago:
  bump version 3.2.3 -> 3.2.4

bumpver.toml
docs/source/conf.py
setup.py
wdoc/wdoc.py

• [f5068a3] by @thiswillbeyourgithub, 13 minutes ago:
  fix: missing yt-dlp makes readthedock crash
  Signed-off-by: thiswillbeyourgithub [email protected]

setup.py

• [17bc5e6] by @thiswillbeyourgithub, 19 minutes ago:
  remove unused argument for theme flyout display
  Signed-off-by: thiswillbeyourgithub [email protected]

docs/source/conf.py

• [4e180f0] by @thiswillbeyourgithub, 22 minutes ago:
  fix: class attribute of env is accessed by readthedocks and should not crash
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/utils/env.py

• [243de66] by @thiswillbeyourgithub, 2 hours ago:
  saner api autodoc parameters
  Signed-off-by: thiswillbeyourgithub [email protected]

docs/source/conf.py

• [ed9dd38] by @thiswillbeyourgithub, 3 hours ago:
  new: add the environment variable documentation to the doc of the EnvDataclass class
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/docs/help.md
wdoc/utils/env.py

• [07b0a7d] by @thiswillbeyourgithub, 3 hours ago:
  doc: remove unused make files for doc
  Signed-off-by: thiswillbeyourgithub
  [email protected]

docs/Makefile
docs/make.bat

• [7abedd4] by @thiswillbeyourgithub, 4 hours ago:
  doc: dont include private nor special
  Signed-off-by: thiswillbeyourgithub
  [email protected]

docs/source/conf.py

• [38008aa] by @thiswillbeyourgithub, 2 hours ago:
  fix: sphinx api command of faq
  Signed-off-by: thiswillbeyourgithub
  [email protected]
  Signed-off-by: thiswillbeyourgithub
  [email protected]
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md

• [11ae11b] by @thiswillbeyourgithub, 4 hours ago:
  remove private from sphinx command
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md

• [90bde99] by @thiswillbeyourgithub, 4 hours ago:
  fix outdated sphinx doc
  Signed-off-by: thiswillbeyourgithub [email protected]

docs/source/wdoc.rst
docs/source/wdoc.utils.batch_file_loader.rst
docs/source/wdoc.utils.customs.compressed_embeddings_cache.rst
docs/source/wdoc.utils.customs.fix_llm_caching.rst
docs/source/wdoc.utils.customs.rst
docs/source/wdoc.utils.embeddings.rst
docs/source/wdoc.utils.env.rst
docs/source/wdoc.utils.errors.rst
docs/source/wdoc.utils.flags.rst
docs/source/wdoc.utils.import_tricks.rst
docs/source/wdoc.utils.interact.rst
docs/source/wdoc.utils.llm.rst
docs/source/wdoc.utils.loaders.rst
docs/source/wdoc.utils.logger.rst
docs/source/wdoc.utils.misc.rst
docs/source/wdoc.utils.prompts.rst
docs/source/wdoc.utils.retrievers.rst
docs/source/wdoc.utils.rst
docs/source/wdoc.utils.tasks.query.rst
docs/source/wdoc.utils.tasks.rst
docs/source/wdoc.utils.tasks.summarize.rst
docs/source/wdoc.utils.typechecker.rst
docs/source/wdoc.wdoc.rst

• [ff093a2] by @thiswillbeyourgithub, 4 hours ago:
  fix: sphinx api command of faq
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md

• [ade5930] by @thiswillbeyourgithub, 4 hours ago:
  fix: relative wdoc imports were stopping sphinx api build
  Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/init.py
wdoc/main.py
wdoc/utils/init.py
wdoc/utils/batch_file_loader.py
wdoc/utils/customs/init.py
wdoc/utils/embeddings.py
wdoc/utils/env.py
wdoc/utils/import_tricks.py
wdoc/utils/interact.py
wdoc/utils/llm.py
wdoc/utils/loaders.py
wdoc/utils/logger.py
wdoc/utils/misc.py
wdoc/utils/prompts.py
wdoc/utils/retrievers.py
wdoc/utils/tasks/init.py
wdoc/utils/tasks/query.py
wdoc/utils/tasks/summarize.py
wdoc/utils/typechecker.py
wdoc/wdoc.py

• [1e6602e] by @thiswillbeyourgithub, 5 hours ago:
  doc: add to faq the sphinx command
  Signed-off-by: thiswillbeyourgithub [email protected]

README.md

• [bd7e3c5] by @thiswillbeyourgithub, 5 hours ago:
  update copyright year to 2025
  Signed-off-by: thiswillbeyourgithub [email protected]

docs/source/conf.py

• [6bffa20] by @thiswillbeyourgithub, 6 hours ago:
  remove unused templates path in doc
  Signed-off-by: thiswillbeyourgithub
  [email protected]

docs/source/conf.py

• [2b93162] by @thiswillbeyourgithub, 6 hours ago:
  remove duplicate readme from the doc
  Signed-off-by: thiswillbeyourgithub [email protected]

docs/source/index.rst

• [3cef833] by @thiswillbeyourgithub, 6 hours ago:
  fix collapsible bar
  Signed-off-by: thiswillbeyourgithub [email protected]

docs/source/conf.py

• [712df08] by @thiswillbeyourgithub, 6 hours ago:
  remove unused css from the doc
  Signed-off-by: thiswillbeyourgithub [email protected]

docs/source/_static/custom.css
docs/source/conf.py

• [1dfe2b3] by @thiswillbeyourgithub, 6 hours ago:
  documentation table
  Signed-off-by: thiswillbeyourgithub [email protected]

docs/source/index.rst

• [6d41817] by @thiswillbeyourgithub, 25 hours ago:
  new: use gpt-4o-mini tokenizer by default to estimate tokens
  previously we used the ageing gpt-3.5-turbo

Signed-off-by: thiswillbeyourgithub [email protected]

wdoc/docs/help.md
wdoc/utils/misc.py

• [ab207b4] by @thiswillbeyourgithub, 25 hours ago:
  new: use the latest yt-dl install from yt-dlp
  Signed-off-by: thiswillbeyourgithub [email protected]

setup.py