[LFXV2-924] Add direct field filtering support for data object queries

[andrest50]

Ticket

https://linuxfoundation.atlassian.net/browse/LFXV2-924

Summary

Added a new filters query parameter that enables direct field filtering on any field within the resource's data object using OpenSearch term clauses. This provides a flexible way to filter resources by their data field values without requiring tags.

Changes

API Design

• Added filters parameter to query-resources and query-resources-count endpoints
• Format: field:value (e.g., status:active, priority:high)
• Fields are automatically prefixed with data. to scope filtering to the data object
• Multiple filters use AND logic (all must match)

Implementation

• Model: Added FieldFilter struct with Field and Value properties
• Parser: Created parseFilters() function to parse field:value format and auto-prefix with data.
• Template: Updated OpenSearch query template to render term clauses for each filter
• Converters: Integrated filter parsing into all payload converter functions

OpenSearch Query Example

Request:

GET /query/resources?filters=status:active&filters=priority:high&v=1

Generated OpenSearch query:

{
  "query": {
    "bool": {
      "filter": [
        { "term": { "latest": true } },
        { "term": { "data.status": "active" } },
        { "term": { "data.priority": "high" } }
      ]
    }
  }
}

Key Features

• ✅ Simple API - users specify field names without data. prefix
• ✅ Automatically scoped - all filters only apply to data object fields
• ✅ Resources without matching fields return no results (term clause behavior)
• ✅ Supports nested fields (e.g., project.id becomes data.project.id)
• ✅ Works with existing tags and tags_all filters

Testing

• Added 9 unit tests for parseFilters() function covering:
  • Single and multiple filters
  • Space trimming
  • Colons in values (URLs)
  • Invalid formats
  • Nested field names
• Added 3 integration tests for OpenSearch query rendering
• All existing tests pass (100% success rate)

Test Plan

• Unit tests for filter parsing pass
• Integration tests for query rendering pass
• All existing tests continue to pass
• Filter with single field works
• Filter with multiple fields (AND logic) works
• Filters work with existing tags/tags_all parameters
• Invalid filter format returns error
• Nested field names work correctly

🤖 Generated with Claude Code

[coderabbitai]

Walkthrough

This PR introduces filter functionality to the query service by adding a "filters" parameter (field:value format) to query endpoints. Filters are parsed into domain objects, propagated through the criteria conversion pipeline, and rendered in the OpenSearch query template as term queries, enabling field-based filtering in search operations.

Changes

Cohort / File(s)	Summary
Filter Parsing & Criteria Conversion cmd/service/converters.go	Adds private parseFilters helper to convert filter strings into FieldFilter objects. Updates payloadToCountPublicCriteria and payloadToCountAggregationCriteria signatures to return (SearchCriteria, error). All three converter functions (payloadToCriteria, payloadToCountPublicCriteria, payloadToCountAggregationCriteria) now populate Criteria.Filters. Returns wrapped errors on filter parse failures. Adds fmt and strings imports.
Service Layer Error Handling cmd/service/service.go	Updates QueryResourcesCount to handle error returns from count criteria converter functions, performing early returns with wrapped errors if filter parsing fails.
Domain Model internal/domain/model/search_criteria.go	Introduces new FieldFilter struct with Field and Value string fields. Adds Filters []FieldFilter field to SearchCriteria struct for representing AND-logic field-term clauses.
API Design design/query-svc.go	Adds filters payload attribute ([]string) and HTTP query parameter to query-resources and query-resources-count endpoints. Documents format as "field:value" with auto-prefixing of "data.".
OpenSearch Template Rendering internal/infrastructure/opensearch/template.go	Adds conditional block in template that iterates over filters (when present) and injects term queries with quoted field and value into the must clause.
Test Coverage cmd/service/converters_test.go	Adds comprehensive TestParseFilters suite covering valid single/multiple filters, space trimming, colons in values, invalid formats with error validation, empty/nil filters, and nested field names.
Search Infrastructure Tests internal/infrastructure/opensearch/searcher_test.go	Adds three test cases to TestOpenSearchSearcherRender validating filter rendering: single filter, multiple filters, and filters combined with existing tag filtering.

Sequence Diagram

sequenceDiagram
    actor Client
    participant Service as Query Service
    participant Converter as Criteria Converter
    participant Parser as Filter Parser
    participant Template as OpenSearch Template
    participant Search as OpenSearch Engine

    Client->>Service: GET /query/resources?filters=status:active
    Service->>Converter: payloadToCriteria(payload with filters)
    Converter->>Parser: parseFilters(["status:active"])
    Parser-->>Converter: FieldFilter{Field: "data.status", Value: "active"}
    Converter->>Converter: Set Criteria.Filters
    Converter-->>Service: SearchCriteria with Filters
    Service->>Template: Render query with Criteria.Filters
    Template->>Template: {{range .Filters}} → term query
    Template-->>Search: {"must": [..., {"term": {"data.status": "active"}}]}
    Search-->>Service: Results
    Service-->>Client: Response

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'LFXV2-924 Add direct field filtering support for data object queries' clearly and specifically describes the main change: adding a new field filtering feature to the query service.
Description check	✅ Passed	The description comprehensively details the changes, providing clear context about the new filters parameter, implementation approach, and testing coverage with an example query.
Linked Issues check	✅ Passed	The PR successfully implements the objective from LFXV2-924 by introducing term-based filters under bool.filter clause instead of must clause, directly addressing the performance optimization goal without document scoring.
Out of Scope Changes check	✅ Passed	All changes are directly related to implementing the field filtering feature and its integration with the OpenSearch query template, with no out-of-scope modifications detected.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch andrest50/query-direct-fields

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (2)

cmd/service/converters_test.go (1)

636-640: Consider validating against empty filter values.

The test case "invalid filter format (empty after colon)" expects success for input "status:", resulting in FieldFilter{Field: "data.status", Value: ""}.

Empty values in OpenSearch term queries typically return no results. Consider whether empty values should be rejected during parsing (similar to missing colons) to provide earlier feedback to API consumers.

cmd/service/converters.go (1)

44-49: Verify intentional difference in error handling strategies.

There's an inconsistency in how filter parsing errors are handled:

• payloadToCriteria (lines 44-49): Returns error and empty criteria on parse failure
• payloadToCountPublicCriteria (lines 125-135): Logs error but continues with empty filters
• payloadToCountAggregationCriteria (lines 162-172): Logs error but continues with empty filters

The count endpoints are more lenient, which may be intentional design (allowing partial results even with malformed filters). However, this could lead to confusing behavior where invalid filters are silently ignored in count queries but fail in resource queries.

Consider whether count endpoints should also return errors for invalid filter syntax to maintain consistency and provide clear feedback to API consumers.

Also applies to: 125-135, 162-172

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Jira integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 963b99e and 314cfff.

⛔ Files ignored due to path filters (10)

• gen/http/cli/lfx_v2_query_service/cli.go is excluded by !**/gen/**
• gen/http/openapi.json is excluded by !**/gen/**
• gen/http/openapi.yaml is excluded by !**/gen/**
• gen/http/openapi3.json is excluded by !**/gen/**
• gen/http/openapi3.yaml is excluded by !**/gen/**
• gen/http/query_svc/client/cli.go is excluded by !**/gen/**
• gen/http/query_svc/client/encode_decode.go is excluded by !**/gen/**
• gen/http/query_svc/server/encode_decode.go is excluded by !**/gen/**
• gen/http/query_svc/server/types.go is excluded by !**/gen/**
• gen/query_svc/service.go is excluded by !**/gen/**

📒 Files selected for processing (6)

• cmd/service/converters.go
• cmd/service/converters_test.go
• design/query-svc.go
• internal/domain/model/search_criteria.go
• internal/infrastructure/opensearch/searcher_test.go
• internal/infrastructure/opensearch/template.go

🧰 Additional context used

📓 Path-based instructions (4)

internal/domain/model/**

📄 CodeRabbit inference engine (CLAUDE.md)

Place core business entities (e.g., Resource, SearchCriteria, AccessCheck) under internal/domain/model/

Files:

• internal/domain/model/search_criteria.go

internal/infrastructure/opensearch/**

📄 CodeRabbit inference engine (CLAUDE.md)

Put OpenSearch implementations for resource search under internal/infrastructure/opensearch/

Files:

• internal/infrastructure/opensearch/template.go
• internal/infrastructure/opensearch/searcher_test.go

**/*_test.go

📄 CodeRabbit inference engine (CLAUDE.md)

Name Go test files with the *_test.go suffix and keep them alongside implementation files

Files:

• cmd/service/converters_test.go
• internal/infrastructure/opensearch/searcher_test.go

design/**

📄 CodeRabbit inference engine (CLAUDE.md)

Keep Goa API design specifications in the design/ directory

Files:

• design/query-svc.go

🧬 Code graph analysis (3)

cmd/service/converters_test.go (1)

internal/domain/model/search_criteria.go (1)

• FieldFilter (7-10)

internal/infrastructure/opensearch/searcher_test.go (1)

internal/domain/model/search_criteria.go (2)

• SearchCriteria (13-46)
• FieldFilter (7-10)

cmd/service/converters.go (2)

internal/domain/model/search_criteria.go (2)

• FieldFilter (7-10)
• SearchCriteria (13-46)

gen/query_svc/service.go (1)

• QueryResourcesPayload (145-168)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)

• GitHub Check: CodeQL analysis (go)
• GitHub Check: Agent

🔇 Additional comments (5)

internal/domain/model/search_criteria.go (1)

6-19: LGTM! Clean domain model additions.

The FieldFilter struct and Filters field are well-defined and integrate cleanly into the existing SearchCriteria structure. The comment clearly documents the AND logic behavior.

internal/infrastructure/opensearch/searcher_test.go (1)

314-345: Good test coverage for filter rendering.

The three new test cases effectively validate:

1. Single field filter rendering
2. Multiple field filters rendering
3. Combination of tags_all with filters

Note: Once the OpenSearch template is updated to place filters in the bool.filter clause (per the critical issue in template.go), verify that these tests still pass and update assertions if needed.

cmd/service/converters_test.go (1)

593-676: Comprehensive test coverage for filter parsing.

The test suite effectively covers all important parsing scenarios including edge cases. Well done!

cmd/service/converters.go (1)

19-40: Clean filter parsing implementation.

The parseFilters function correctly:

• Handles empty/nil input
• Splits on first colon to support colons in values
• Trims whitespace from field names and values
• Auto-prefixes fields with "data."
• Returns clear error messages for invalid formats

design/query-svc.go (1)

59-61: LGTM! Consistent API design for filters parameter.

The filters parameter is properly defined across both query-resources and query-resources-count endpoints with:

• Clear descriptions of the field:value format
• Documentation of automatic data. prefixing
• Appropriate examples
• Consistent HTTP parameter mappings

Also applies to: 84-84, 127-129, 154-154

[Copilot]

Pull request overview

This PR adds direct field filtering support for resource queries through a new filters query parameter. The implementation enables API consumers to filter resources by any field within the data object using OpenSearch term clauses, with fields automatically prefixed with data. to scope filtering appropriately.

Key Changes:

• Added filters parameter accepting field:value format with automatic data. prefixing
• Implemented filter parsing with AND logic for multiple filters
• Updated OpenSearch query template to render term clauses for filters

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
internal/domain/model/search_criteria.go	Added FieldFilter struct to represent field:value filter pairs
cmd/service/converters.go	Implemented parseFilters() function and integrated filter parsing into payload converter functions
cmd/service/converters_test.go	Added comprehensive unit tests for filter parsing covering various edge cases
internal/infrastructure/opensearch/template.go	Updated OpenSearch query template to render term clauses for each filter in the must array
internal/infrastructure/opensearch/searcher_test.go	Added integration tests verifying query rendering with filters
gen/query_svc/service.go	Added Filters field to query payload types with documentation
gen/http/query_svc/server/types.go	Updated payload builder functions to include filters parameter
gen/http/query_svc/server/encode_decode.go	Added filters query parameter decoding for both endpoints
gen/http/query_svc/client/encode_decode.go	Added filters query parameter encoding for client requests
gen/http/query_svc/client/cli.go	Updated CLI payload builders to parse filters from JSON input
gen/http/cli/lfx_v2_query_service/cli.go	Added CLI flags and usage examples for filters parameter
design/query-svc.go	Added filters attribute to API design specification for both endpoints
gen/http/openapi3.yaml	Added filters parameter documentation to OpenAPI 3.0 spec
gen/http/openapi3.json	Added filters parameter to OpenAPI 3.0 JSON spec
gen/http/openapi.yaml	Added filters parameter documentation to OpenAPI 2.0 spec
gen/http/openapi.json	Added filters parameter to OpenAPI 2.0 JSON spec

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.