Skip to content

Restore boolean-search files from PR #83#86

Closed
adelpro wants to merge 173 commits intomainfrom
develop
Closed

Restore boolean-search files from PR #83#86
adelpro wants to merge 173 commits intomainfrom
develop

Conversation

@adelpro
Copy link
Copy Markdown
Owner

@adelpro adelpro commented Mar 11, 2026

Recovery PR - files were accidentally deleted during merge conflict resolution in commit c64b430.

Summary by CodeRabbit

Release Notes

  • New Features

    • Semantic search and filtering capabilities
    • Range-based verse queries (e.g., "2:255", "1:1-7")
    • Regex search with ReDoS protection and validation
    • Boolean query operators (+, -, |) for refined searches
    • Inverted index support for O(1) lookups
    • LRU caching for improved repeated search performance
  • Documentation

    • Comprehensive API reference guides
    • Getting Started and Quick Start tutorials
    • Architecture and performance optimization guides
  • Chores

    • Migrated to Yarn Berry package manager
    • Added CI/CD pipelines and bundle size monitoring

bessmasarri and others added 30 commits February 12, 2026 11:53
…with id, sura_name, sura_name_en, sura_name_romanization, total_verses, juz_id, page_id
feat: add utility function to detect Arabic characters (#3)
- Add Sura interface in types/index.ts
- Modify metadata.ts to derive data from quran.json
- Calculate total_verses, juz_ids, page_start/end dynamically
- Ensure single source of truth as requested in PR review
…and-ci

fix: strict filtering for verses and add CI workflow
adelpro and others added 28 commits March 7, 2026 18:04
Fixed first 5000 lines in word-map.json manually See #68
also exported clearBooleanOperators as a helper for UI and can be used to support the first one
Export hasBooleanOperators from boolean-search
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 11, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 83a25aa3-5399-4a37-8655-8ceeaf4b6e6b

📥 Commits

Reviewing files that changed from the base of the PR and between 87e4fe4 and 35d7e3f.

⛔ Files ignored due to path filters (17)
  • assets/itqan-logo.svg is excluded by !**/*.svg
  • examples/angular/.angular/cache/18.2.21/angular/angular-compiler.db is excluded by !**/*.db
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/@angular_common.js.map is excluded by !**/*.map
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/@angular_core.js.map is excluded by !**/*.map
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/@angular_forms.js.map is excluded by !**/*.map
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/@angular_platform-browser.js.map is excluded by !**/*.map
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/chunk-3HCJG636.js.map is excluded by !**/*.map
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/chunk-GYPLUZQO.js.map is excluded by !**/*.map
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/chunk-WDMUDEB6.js.map is excluded by !**/*.map
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/morphology-QCASLOZB-SKYVJWXW.js.map is excluded by !**/*.map
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/quran-IEIJ6Y6C-BEUGOST3.js.map is excluded by !**/*.map
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/quran-search-engine.js.map is excluded by !**/*.map
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/word-map-7GKBHIQA-W2V7ULD4.js.map is excluded by !**/*.map
  • package-lock.json is excluded by !**/package-lock.json
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
  • scripts/word-level-transliteration/raw_data/ayahs.csv is excluded by !**/*.csv
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (122)
  • .github/LABELS.md
  • .github/workflows/ci.yml
  • .github/workflows/detect-package-managers.yml
  • .github/workflows/gitleaks.yml
  • .github/workflows/size.yml
  • .gitignore
  • .husky/commit-msg
  • .husky/pre-commit
  • .yarnrc.yml
  • CHANGELOG.md
  • CONTRIBUTING.md
  • README.md
  • SUGGESTED_ISSUES.md
  • commitlint.config.js
  • docs/api/core.md
  • docs/api/inverted-index.md
  • docs/api/tokenizer.md
  • docs/architecture.md
  • docs/configuration.md
  • docs/english-arabic-search.md
  • docs/examples.md
  • docs/getting-started.md
  • docs/index.md
  • docs/installation.md
  • docs/performance.md
  • docs/quick-start.md
  • docs/search-syntax.md
  • examples/angular/.angular/cache/18.2.21/angular/angular-compiler.db-lock
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/@angular_common.js
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/@angular_core.js
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/@angular_forms.js
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/@angular_platform-browser.js
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/_metadata.json
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/chunk-3HCJG636.js
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/chunk-GYPLUZQO.js
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/chunk-WDMUDEB6.js
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/morphology-QCASLOZB-SKYVJWXW.js
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/package.json
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/quran-IEIJ6Y6C-BEUGOST3.js
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/quran-search-engine.js
  • examples/angular/.angular/cache/18.2.21/angular/vite/deps/word-map-7GKBHIQA-W2V7ULD4.js
  • examples/angular/src/app/app.component.ts
  • examples/angular/src/index.html
  • examples/angular/src/main.ts
  • examples/angular/src/styles.css
  • examples/angular/tsconfig.app.json
  • examples/angular/tsconfig.json
  • examples/nodejs/README.md
  • examples/nodejs/src/index.js
  • examples/vanilla-ts/README.md
  • examples/vanilla-ts/index.html
  • examples/vanilla-ts/src/main.ts
  • examples/vite-react/README.md
  • examples/vite-react/eslint.config.js
  • examples/vite-react/index.html
  • examples/vite-react/package.json
  • examples/vite-react/src/App.css
  • examples/vite-react/src/App.tsx
  • examples/vite-react/src/main.tsx
  • examples/vite-react/src/vite-env.d.ts
  • examples/vite-react/tsconfig.json
  • examples/vite-react/vite.config.ts
  • package.json
  • pnpm-workspace.yaml
  • scripts/word-level-transliteration/raw_data/quran_transliteration.txt
  • scripts/word-level-transliteration/word-level-transliteration.ts
  • src/core/layers/boolean-search.test.ts
  • src/core/layers/boolean-search.ts
  • src/core/layers/fuse-search.test.ts
  • src/core/layers/fuse-search.ts
  • src/core/layers/linguistic-search.ts
  • src/core/layers/regex-search.test.ts
  • src/core/layers/regex-search.ts
  • src/core/layers/semantic-search.test.ts
  • src/core/layers/semantic-search.ts
  • src/core/layers/simple-search.test.ts
  • src/core/layers/simple-search.ts
  • src/core/search.test.ts
  • src/core/search.ts
  • src/data/lemma-index.json
  • src/data/metadata.test.ts
  • src/data/metadata.ts
  • src/data/morphology.json
  • src/data/phonetic.json
  • src/data/root-index.json
  • src/data/semantic.json
  • src/data/word-index.json
  • src/data/word-map.json
  • src/errors/README.md
  • src/errors/base.error.test.ts
  • src/errors/base.error.ts
  • src/errors/data-load.error.test.ts
  • src/errors/data-load.error.ts
  • src/errors/error-codes.ts
  • src/errors/index.ts
  • src/errors/search.error.test.ts
  • src/errors/search.error.ts
  • src/errors/tokenization.error.test.ts
  • src/errors/tokenization.error.ts
  • src/errors/validation.error.test.ts
  • src/errors/validation.error.ts
  • src/index.ts
  • src/types/index.ts
  • src/utils/highlight.test.ts
  • src/utils/highlight.ts
  • src/utils/loader.test.ts
  • src/utils/loader.ts
  • src/utils/lru-cache.test.ts
  • src/utils/lru-cache.ts
  • src/utils/normalization.test.ts
  • src/utils/normalization.ts
  • src/utils/phonetic.test.ts
  • src/utils/phonetic.ts
  • src/utils/range-parser.test.ts
  • src/utils/range-parser.ts
  • src/utils/regex-validation.ts
  • src/utils/schema.test.ts
  • src/utils/schema.ts
  • src/utils/scoring.ts
  • src/utils/tokenization.test.ts
  • src/utils/tokenization.ts
  • tsc-errors.txt

📝 Walkthrough

Walkthrough

This PR implements v0.2.0 with comprehensive architecture reorganization, introducing layered search layers (semantic, regex, range, boolean), inverted index optimization, structured error handling framework, extensive documentation, and migration from pnpm to Yarn Berry package manager.

Changes

Cohort / File(s) Summary
GitHub Actions CI/CD Workflows
.github/workflows/ci.yml, detect-package-managers.yml, gitleaks.yml, size.yml
New workflows enforcing Yarn Berry usage, validating package-lock absence, running lint/build/test, and enforcing bundle size limits via size-limit.
Package Manager Migration
.husky/commit-msg, .husky/pre-commit, .yarnrc.yml, .gitignore, pnpm-workspace.yaml, package.json
Migrates from pnpm to Yarn Berry with workspaces; updates hooks to use yarn commands; removes pnpm config; adds Yarn-specific entries to .gitignore.
Core Search Architecture
src/core/search.ts, src/core/layers/boolean-search.ts, src/core/layers/fuse-search.ts, src/core/layers/linguistic-search.ts, src/core/layers/regex-search.ts, src/core/layers/semantic-search.ts, src/core/layers/simple-search.ts
Implements new layered search pipeline with range/regex shortcuts, semantic search, boolean operators, linguistic matching via inverted indexes, and improved scoring; refactors search entrypoint to orchestrate layers.
Error Handling System
src/errors/base.error.ts, src/errors/data-load.error.ts, src/errors/search.error.ts, src/errors/validation.error.ts, src/errors/tokenization.error.ts, src/errors/error-codes.ts, src/errors/index.ts
Introduces comprehensive error hierarchy with ErrorCode/ErrorType enums and specialized classes for data-loading, search, validation, and tokenization failures.
Type System Expansion
src/types/index.ts, src/core/search.ts, src/utils/loader.ts
Extends VerseInput with metadata fields, adds VerseWithFuseMatches, ParsedRange, InvertedIndex types, and expands MatchType/SearchCounts/AdvancedSearchOptions for new search modes.
Data Loaders & Validation
src/utils/loader.ts, src/utils/schema.ts, src/data/metadata.ts, src/data/semantic.json
Adds robust error-handled loaders with schema validation, buildInvertedIndex/loadInvertedIndex for optimization, SURAS metadata extraction, and semantic concept mappings for English-Arabic translation.
Utility Enhancements
src/utils/lru-cache.ts, src/utils/range-parser.ts, src/utils/regex-validation.ts, src/utils/normalization.ts, src/utils/highlight.ts, src/utils/scoring.ts, src/utils/tokenization.ts, src/utils/phonetic.ts
Introduces LRU caching, range parsing, regex safety validation, isArabic detection, enhanced highlighting with tokenTypes support, phonetic mapping, and improved tokenization with mode validation.
Comprehensive Documentation
docs/index.md, docs/getting-started.md, docs/installation.md, docs/quick-start.md, docs/search-syntax.md, docs/configuration.md, docs/architecture.md, docs/api/core.md, docs/api/tokenizer.md, docs/api/inverted-index.md, docs/performance.md, docs/english-arabic-search.md, docs/examples.md, CONTRIBUTING.md, README.md, CHANGELOG.md
Adds extensive user and contributor documentation covering architecture, API reference, search syntax, configuration, performance optimization, examples, and contribution guidelines.
Example Applications
examples/angular/src/app/app.component.ts, examples/nodejs/src/index.js, examples/vanilla-ts/src/main.ts, examples/vite-react/src/App.tsx, examples/vanilla-ts/index.html, examples/vite-react/index.html, examples/*/README.md, examples/*/package.json, examples/*/tsconfig.json, examples/*/vite.config.ts, examples/*/eslint.config.js
Updates all example apps to use new LRU caching, semantic/sura/juz filters, and demonstrates inverted index usage; replaces pnpm commands with yarn; adds configuration and styling updates.
Configuration & Metadata
commitlint.config.js, package.json, SUGGESTED_ISSUES.md
Adds JSDoc type annotation to commitlint config; updates package.json with new scripts, size-limit config, and workspace setup; removes SUGGESTED_ISSUES.md.
Cache File Cleanup
examples/angular/.angular/cache/18.2.21/angular/vite/deps/*
Removes entire Angular Vite dependency cache directory (build artifacts); does not affect source or configuration.
Test Suite
src/core/layers/*.test.ts, src/utils/*.test.ts, src/errors/*.test.ts, src/data/metadata.test.ts, tsc-errors.txt
Comprehensive test coverage for all new layers, utilities, error classes, and data handling; includes error reporting from TypeScript compilation.

Sequence Diagram

sequenceDiagram
    participant User
    participant Search Engine
    participant Range Layer
    participant Regex Layer
    participant Simple Layer
    participant Linguistic Layer
    participant Semantic Layer
    participant Fuzzy Layer
    participant Cache

    User->>Search Engine: search(query, data, maps, options)
    
    alt Cache Hit
        Search Engine->>Cache: lookup(query, options, pagination)
        Cache-->>Search Engine: cached results
        Search Engine-->>User: return cached results
    else Cache Miss
        Search Engine->>Range Layer: parseRangeQuery(query)
        alt Range Match
            Range Layer-->>Search Engine: range results
        else No Range Match
            Search Engine->>Regex Layer: validateRegex & search
            alt Regex Match
                Regex Layer-->>Search Engine: regex results
            else No Regex Match
                Search Engine->>Simple Layer: simpleSearch(query)
                Simple Layer-->>Search Engine: simple matches
                
                Search Engine->>Linguistic Layer: performAdvancedLinguisticSearch
                Linguistic Layer->>Linguistic Layer: check lemma/root via inverted index
                Linguistic Layer-->>Search Engine: linguistic matches
                
                alt Semantic Enabled
                    Search Engine->>Semantic Layer: performSemanticSearch
                    Semantic Layer-->>Search Engine: semantic matches
                end
                
                alt Fuzzy Enabled
                    Search Engine->>Fuzzy Layer: fuse.search(query)
                    Fuzzy Layer-->>Search Engine: fuzzy matches
                end
                
                Search Engine->>Search Engine: deduplicate & combine results
            end
        end
        
        Search Engine->>Cache: store(key, results)
        Search Engine-->>User: return results
    end
Loading

Estimated Code Review Effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly Related PRs

  • Feature/inverted index 8 #41: Extends inverted-index integration (buildInvertedIndex, loadInvertedIndex, threading optional invertedIndex into search layers).
  • Issue 14 filter #20: Modifies search filtering API with filterVerses implementation for suraId, juzId, suraName scoping.

Poem

🐰 Whiskers twitching with delight,
A new v0.2 shines so bright,
Layered searches, errors neat,
Yarn replaces old feet!
From range to semantic dreams,
Search flows through pristine streams!

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch develop

@adelpro adelpro closed this Mar 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.