Skip to content

Conversation

clydin
Copy link
Member

@clydin clydin commented Sep 8, 2025

This commit refactors the find_examples MCP tool and its associated database generator to leverage the structured YAML front matter present in the example markdown files.

Key changes:

  • The SQLite database schema is now relational, with dedicated columns for title, summary, and keywords.
  • An FTS5 virtual table indexes these structured fields, improving search relevance by allowing queries to target specific metadata.
  • The build-time database generator (tools/example_db_generator.js) now parses and validates the front matter of each example file using Zod. The build will fail if an example is missing a required field (title, summary), ensuring data integrity.
  • The runtime tool (packages/.../examples.ts) uses the same parsing logic but will warn and skip invalid files to be more resilient.

This change provides a more robust and accurate foundation for the example search feature, enabling more precise results and paving the way for future enhancements like semantic search.

This commit refactors the `find_examples` MCP tool and its associated database generator to leverage the structured YAML front matter present in the example markdown files.

Key changes:
- The SQLite database schema is now relational, with dedicated columns for `title`, `summary`, and `keywords`.
- An FTS5 virtual table indexes these structured fields, improving search relevance by allowing queries to target specific metadata.
- The build-time database generator (`tools/example_db_generator.js`) now parses and validates the front matter of each example file using Zod. The build will fail if an example is missing a required field (`title`, `summary`), ensuring data integrity.
- The runtime tool (`packages/.../examples.ts`) uses the same parsing logic but will warn and skip invalid files to be more resilient.

This change provides a more robust and accurate foundation for the example search feature, enabling more precise results and paving the way for future enhancements like semantic search.
This commit enhances the `find_examples` MCP tool by introducing advanced filtering capabilities, allowing for more precise and powerful queries.

The tool's input schema now accepts optional array-based filters for `keywords`, `required_packages`, and `related_concepts`. These filters are combined with the main full-text search query to narrow down results.

To support this, the underlying SQLite database schema has been extended with dedicated columns for this metadata. The build-time database generator and the runtime tool have both been updated to parse, validate, and store this structured data from the example file's front matter.

The query logic is now fully dynamic, constructing parameterized SQL queries to safely and efficiently filter the examples based on the provided criteria.
This commit improves the relevance of the `find_examples` MCP server tool by implementing weighted search ranking.

The FTS5 query now uses the `bm25()` ranking function to assign a higher weight to matches found in more important fields. Specifically, matches in the `title`, `summary`, and `keywords` are now weighted more heavily than matches in the main content.

This results in more accurate and intuitive search results, as examples where the query terms are central to the topic are ranked higher.
@clydin clydin added the target: major This PR is targeted for the next major release label Sep 8, 2025
@angular-robot angular-robot bot added detected: feature PR contains a feature commit area: @angular/cli labels Sep 8, 2025
@clydin clydin marked this pull request as ready for review September 8, 2025 17:40
@clydin clydin requested a review from alan-agius4 September 8, 2025 21:58
@clydin clydin added the action: review The PR is still awaiting reviews from at least one requested reviewer label Sep 8, 2025
@alan-agius4 alan-agius4 added action: merge The PR is ready for merge by the caretaker and removed action: review The PR is still awaiting reviews from at least one requested reviewer labels Sep 9, 2025
@alan-agius4 alan-agius4 merged commit 0d705c9 into angular:main Sep 9, 2025
37 checks passed
@clydin clydin deleted the mcp/find-examples-frontmatter branch September 9, 2025 12:15
@angular-automatic-lock-bot
Copy link

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

@angular-automatic-lock-bot angular-automatic-lock-bot bot locked and limited conversation to collaborators Oct 10, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

action: merge The PR is ready for merge by the caretaker area: @angular/cli detected: feature PR contains a feature commit target: major This PR is targeted for the next major release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants