-
Notifications
You must be signed in to change notification settings - Fork 199
feat: Add Isaacus integrations (Support for Kanon 2 Embedder) #2413
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add Isaacus integrations (Support for Kanon 2 Embedder) #2413
Conversation
|
Hi @abdurrahmanbutler, thanks a lot for your interest in integrating Isaacus with Haystack! It’s great to see your focus on retrieval for the legal domain, which is also very relevant to our users. For example, we’ve collaborated with MANZ on a similar use case: https://www.deepset.ai/case-studies/manz. For the integration, we’d suggest hosting it in your own repository. This way, you’ll fully own the code and can move at your own speed without depending on our review and release cycles. We’ll still be happy to provide feedback, and we’ll list your integration on our integrations page (https://haystack.deepset.ai/integrations) to ensure it gets visibility within the community. Let us know if you’d like any guidance or examples on how to set this up. |
Thanks for the feedback! I've migrated the integration to https://github.com/isaacus-dev/isaacus-haystack and published to PyPi. Do I need to do a PR to the Haystack integrations page to add a readme and metadata or is the repo sufficient? |
|
@abdurrahmanbutler that's great! I'd recommend opening a PR in https://haystack.deepset.ai/integrations following the contributions guide here https://github.com/deepset-ai/haystack-integrations?tab=readme-ov-file#how-to-contribute |
I've added a pr to the integrations repo! Thanks for the advice. Link below: |
|
@abdurrahmanbutler great! I'll go ahead and close this PR then and someone from the team should be able to review your other PR soon. |
Proposed Changes
Feature: Add Isaacus embedders to Haystack core integrations.
integrations/isaacus/:src/haystack_integrations/components/embedders/isaacus/__init__.pysrc/haystack_integrations/components/embedders/isaacus/text_embedder.pyIsaacusTextEmbedder(query embeddings; returns{"embedding": List[float]})src/haystack_integrations/components/embedders/isaacus/document_embedder.pyIsaacusDocumentEmbedder(document embeddings; writes todocument.embedding, returns{"documents": List[Document]})src/haystack_integrations/components/embedders/isaacus/utils.py(minimal HTTP client)tests/test_isaacus_embedder.pyREADME.md,CHANGELOG.md,pyproject.tomlHighlights
model(defaults to"kanon-2-embedder"),dimensions,overflow_strategy, andbatch_size.@component,run,@component.output_types).integrations/isaacus/**; no other files touched.How did you test it?
Unit tests (HTTP mocked)
integrations/isaacus/:hatch run test:all- all green.Manual verification (live API)
pip install -e integrations/isaacusDocuments withIsaacusDocumentEmbedder(model="kanon-2-embedder")IsaacusTextEmbedder+InMemoryEmbeddingRetrieverNotes for reviewer
Isaacus recently launched the Kanon 2 Embedder, an English language legal specialised embedding model that is the best performing model on the Massive Legal Embedding Benchmark.
We are submitting this pull request to extend support for the model to Haystack via an integration of our API and model.
Usage Example
Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:.