ENG-7618: Reflex Web Site Search

[LineIndent]

No description provided.

[linear]

ENG-7618 Reflex Web Search Algo

[greptile-apps]

Greptile Summary

This PR implements a comprehensive site search functionality for the Reflex documentation website (ENG-7618). The implementation consists of several key components working together:

Search Architecture: The PR replaces the previous Typesense cloud-based search with a local fuzzy search system. The new implementation uses pre-generated indexes stored as Python modules (docs_index.py and blog_index.py) containing structured metadata for all documentation and blog content.

Content Indexing: A new indexer script (scripts/indexer.py) processes markdown files from both docs and blog directories. It extracts metadata, cleans markdown content, generates summaries using the sumy library, and handles special URL patterns. The script creates two separate indexes - one for documentation with hierarchical organization and another for blog posts with frontmatter metadata.

Search Implementation: The typesense.py component has been completely rewritten to use local fuzzy matching with the textdistance library. It performs Levenshtein distance calculations with a 0.6 similarity threshold and shows suggestions by default, switching to search results when users type 3+ characters.

Dependencies: Two new libraries were added - sumy>=0.11 for extractive text summarization and textdistance>=4.6.3 for fuzzy string matching capabilities.

Documentation Cleanup: Several minimal AI Builder documentation files (overview.md, faq.md, intro.md) were emptied or removed as part of restructuring the documentation for better searchability.

The search system integrates into the existing navbar component structure and provides a self-contained solution that doesn't require external services or API keys, making deployment simpler and eliminating operational dependencies.

Confidence score: 3/5

• This PR requires careful review due to complete removal of documentation content and significant architectural changes
• Score reflects concerns about deleted documentation files and potential impact on user experience from removed search features
• Pay close attention to the emptied AI Builder documentation files and the simplified search UI functionality

_{8 files reviewed, 5 comments}

_{Edit Code Review Bot Settings | Greptile}