Conversation
- 【big picture】 outline entry points, create_app, and server runtime - 【workflows】 explain running locally and dependency locations - 【conventions】 document app globals, rate limits, and security headers - 【integrations】 note optional services and external binaries config - 【where to look】 point to bootstrap, web logic, and dependency files
- 【feat】 add config_fulltext_search boolean column to settings - 【feat】 add checkbox to admin config page to toggle the feature - 【feat】 check config before using full-text DB in search rendering
- 【what】replace sqlite3 usage with sqlalchemy engine and connection - 【why】preserve fts extension loading and avoid sqlite thread issues - 【how】obtain raw DB connection to load extension and run param sql
- 【fix】 replace config.config_binariesdir with constants.BASE_DIR - 【fix】 build path bin/calibre/calibre-extensions/sqlite_extension - 【fix】 import constants and load extension if it exists - 【why】 use project base dir for reliable extension loading across envs
- 【feat】 include bm25(books_fts) as rank in FTS query to score matches - 【feat】 order by rank so search results are returned by relevance ♻️ refactor(search): use fixed extension path and stable dedupe - 【refactor】 load sqlite extension from container .so path for consistent env - 【refactor】 deduplicate fts ids while preserving order using dict.fromkeys - 【refactor】 simplify JOIN clause formatting for readability
|
For those interested, this functionality has now been added to Calibre-Web Automated (CWA) in addition to a full management console & automated indexing for the process that allows users to better control over the process as well as the ability to view the current indexing status of their library ect. Amazing work @webysther <3 |
- 【what】 change FTS select to use DISTINCT to remove duplicate ids - 【what】 remove bm25 rank, ordering and pagination logic - 【what】 simplify SQLAlchemy query to only filter by FTS ids - 【why】 fixes duplicate search results and simplifies retrieval logic
|
Hey @webysther - I had to pull the FTS changes from the upcoming release. The latest version still has a few blocking issues that would ship broken behaviour: The FTS SQL orders by rank but never selects a rank field, so the query errors and FTS silently falls back. That means relevance never actually works. What still needs fixing before it’s ready: Restore a valid FTS ranking query (or select a rank column) so relevance sorting actually works. Once those are fixed and tested, I’ll bring it back into CWA! |
All these are fixed already! |
- add config_hide_archived_search boolean setting to _Settings model (default True) - persist new config in admin _configuration_update_helper via checkbox - add checkbox to config_edit.html to control hiding archived books - update render_search_results to query user's ArchivedBook entries when enabled - exclude archived book ids from base and fts merged results; on exception fall back to empty set
- add config_merge_search boolean setting to database model with default False - add checkbox for config_merge_search to configuration UI (cps/templates/config_edit.html) - register config_merge_search checkbox in admin config update helper (cps/admin.py) - when enabled, include base simple-search results via calibre_db.get_search_results - when enabled, merge base_entries into merged results while skipping archived and deduping by id
OzzieIsaacs
changed the title
2 participants
Full Text Search is finally here, bringing a faster, more powerful way to discover books. Search now taps into Calibre’s full‑text index with an optional toggle in settings, while preserving the existing search as a safe fallback. Under the hood, the feature uses a dedicated FTS database, loads Calibre’s tokenizer extensions, and integrates cleanly with the existing /search flow. The result is an exciting, revitalizing upgrade to search—more accurate, more responsive, and ready for deeper discovery.
Recording.2026-01-29.182517.1.mp4
Current limitations
fix #2490
fix #2776
fix #3129
fix #3468
fix #2308
fix #1412
related #3476
related #2772
related #718