Skip to content

Merge log level filtering with KQL filtering by extending query expressions #105

@coderabbitai

Description

@coderabbitai

Enhancement Request

Currently, the StructuredIrStreamReader::filter_log_events method handles log level filtering and KQL filtering separately:

  1. KQL filtering is applied first (if a KQL query is provided)
  2. Log level filtering is then applied on top of the KQL results

Proposed Enhancement

Merge the existing log level filtering with KQL filtering by extending user-supplied query expressions to include log level constraints. This would:

  • Simplify filtering logic: Use a single unified query approach instead of two separate filtering passes
  • Improve performance: Filter in one pass instead of applying filters sequentially
  • Enhance API consistency: Use KQL for all filtering needs rather than having separate parameters
  • Enable complex queries: Allow users to combine log levels with other criteria in sophisticated query expressions

Current Implementation

The current approach in src/clp_ffi_js/ir/StructuredIrStreamReader.cpp applies filters sequentially, which works but could be more efficient and user-friendly.

Expected Outcome

Users should be able to write KQL queries like:

  • level:INFO AND message:"error"
  • level:(ERROR OR WARN) AND timestamp > 2024-01-01

This would eliminate the need for separate log_level_filter parameter while providing more flexibility.


Requested by: @hoophalab
Context: PR #104
Reference: Comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions