Skip to content

Conversation

@devin-ai-integration
Copy link
Contributor

feat(tantivy): implement full-text search plugin as Orama replacement

Summary

This PR implements the plugins/tantivy plugin with full-text search functionality using Tantivy 0.22, preparing it as a future replacement for the Orama-based search system. This is preparation work only - the plugin is not yet integrated into the desktop app.

Key additions:

  • Document schema with fields: id, doc_type, title, content, created_at
  • CRUD operations: add_document, update_document, delete_document, commit
  • Search operations: search (query parser) and search_fuzzy (fuzzy term matching)
  • Filtering support via created_at range queries (gte, lte, gt, lt, eq)
  • Thread-safe state management using tokio::sync::Mutex
  • Auto-generated TypeScript bindings for all commands

Review & Testing Checklist for Human

  • Verify schema alignment: Confirm the document schema (id, doc_type, title, content, created_at) matches what the current Orama search indexes. The doc_type field expects values like "session", "human", "organization"
  • Review thread safety: The state management clones schema before getting mutable writer access (lines 59-66 in ext.rs) - verify this pattern doesn't introduce race conditions
  • Check index persistence path: Index is stored at {app_data}/search_index - confirm this is the appropriate location and won't conflict with other data
  • Test search functionality manually: The search logic is untested - consider adding integration tests before actual Orama replacement

Recommended Test Plan

  1. Build the desktop app with this plugin enabled
  2. Call init() to create the index
  3. Add test documents via add_document() and commit()
  4. Verify search() and search_fuzzy() return expected results
  5. Test created_at filtering with various operators

Notes

  • The build_created_at_range_query function has an unused _field parameter with hardcoded field name - intentional simplification but worth noting
  • No migration strategy for schema changes - will need addressing before production use
  • Playwright test failures in CI are unrelated (browser environment issue)

Link to Devin run: https://app.devin.ai/sessions/5a14afbb59c9484b849f67a4a15d3652
Requested by: yujonglee (@yujonglee)

- Add comprehensive search functionality with tantivy 0.22
- Implement document operations: add, update, delete, commit
- Add search and fuzzy search with created_at filtering
- Support for session, human, and organization document types
- Generate TypeScript bindings for all commands
- Add state management with IndexState for thread-safe access
- Include error handling for all tantivy operations

Co-Authored-By: yujonglee <[email protected]>
@devin-ai-integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@netlify
Copy link

netlify bot commented Jan 3, 2026

Deploy Preview for hyprnote canceled.

Name Link
🔨 Latest commit 2ee7039
🔍 Latest deploy log https://app.netlify.com/projects/hyprnote/deploys/695917e8aeef370008054573

@netlify
Copy link

netlify bot commented Jan 3, 2026

Deploy Preview for hyprnote-storybook canceled.

Name Link
🔨 Latest commit 2ee7039
🔍 Latest deploy log https://app.netlify.com/projects/hyprnote-storybook/deploys/695917e831b4860008ecbba8

@netlify
Copy link

netlify bot commented Jan 3, 2026

Deploy Preview for howto-fix-macos-audio-selection canceled.

Name Link
🔨 Latest commit 2ee7039
🔍 Latest deploy log https://app.netlify.com/projects/howto-fix-macos-audio-selection/deploys/695917e8de96a70008623cc0

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
@yujonglee yujonglee merged commit e5b9c53 into main Jan 3, 2026
12 of 16 checks passed
@yujonglee yujonglee deleted the devin/1767446013-tantivy-search-plugin branch January 3, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants