v3: Multi-database driver, anchor support, crawl progress tracking - #51
Merged
Conversation
The action class, contract, config key, and tests were never wired up — anchor extraction is handled by DefaultIndexer::walkNodes(). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Headings without an id attribute now reset the current anchor to null, preventing content from being incorrectly linked to a previous section. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use strtok to strip any existing fragment before appending the anchor, preventing invalid URLs like url#existing#anchor. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SQLite and Array drivers now return one result per URL (keeping the best-ranked match), consistent with MeiliSearch's distinctAttribute behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Anchor IDs like 'installation' were affecting search ranking. Default searchableAttributes now excludes anchor while remaining overridable via user settings. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
freekmurze
marked this pull request as ready for review
February 28, 2026 23:53
freekmurze
changed the title
These are no longer supported in v3 (requires PHP 8.4+ and Laravel 12+).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
MySQL: Remove required (+) prefix from boolean query terms so stop words like "with" don't prevent matches. PostgreSQL: Create custom text search config (site_search) that uses English stemmer without stop word filtering. Build tsquery manually with :* prefix for prefix matching support. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The english_stem dictionary includes stop word filtering which removes common words like "here" and "with" from the index. Create a custom Snowball dictionary (site_search_stem) that only does stemming without stop word removal. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
freekmurze
changed the title
Instead of creating one row per text fragment extracted from a page, consolidate all entries for the same URL into a single row with concatenated entry text. This reduces document count from ~80x the number of pages to 1x. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
With consolidated entries per URL, the entry column can exceed MySQL's TEXT limit (64KB) on content-heavy pages like archives. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove URL deduplication so all drivers return every matching section with its anchor. This lets users see multiple relevant sections from the same page in search results.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is a new major version of laravel-site-search. The biggest change is that the package no longer requires Meilisearch by default. Instead, it ships with a unified DatabaseDriver that uses your application's database for full-text search, supporting SQLite, MySQL, and PostgreSQL out of the box with zero external dependencies.
New features
idattributes are extracted during indexing, andHit::urlWithAnchor()returns URLs likehttps://example.com/page#installationsite-search:listnow shows URLs found, URLs failed, finish reason, and latest crawl time. NewCrawlFinishedEventwithFinishReasonandCrawlProgressDefaultIndexernow extracts<meta name="description">content and includes it in search resultsBreaking changes
UriInterfaceparameters becomestring,ResponseInterfacebecomesCrawlResponsein all interfaces (SearchProfile,Indexer, events)Indexer::entries()now returnsarray<array{text: string, anchor: ?string}>instead ofstring[]Driverinterface has new required methods:finalizeIndex(),documentCount(): inturls_found,urls_failed,finish_reasononsite_search_configstable (migration required)create_site_search_documents_tablefor the unified DatabaseDriverMeiliSearchDrivertoDatabaseDriverSqliteDriverremoved: replaced byDatabaseDriverwhich uses Laravel's DB layer. If you usedSqliteDriverwith a customstorage_path, configure a dedicated SQLite database connection insteadBug fixes and code quality improvements
ArrayDriver::search()passing arguments in wrong order toSearchResultsconstructorListCommandhard-importingMeilisearch\Exceptions\ApiException(breaks when meilisearch package isn't installed)SiteSearchFacadealias from composer.json (class didn't exist)IndexNameNotSetexceptionIndexerinterface/or.void,self,mixed,array) across all commands, jobs, and interfaces$guarded/$castsfrom public properties to protected/casts()methodSiteSearchCrawlProfile::$baseUrltype frommixedtostringListCommandSee UPGRADING.md for detailed migration instructions.