Skip to content
Open
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
d264710
Define schemas for simple filter interface
nickpeihl Oct 29, 2025
5989e98
Add descriptions to schemas
nickpeihl Oct 29, 2025
f900770
Remove id and rename indexPattern to dataViewId
nickpeihl Oct 30, 2025
5037bbe
Merge remote-tracking branch 'upstream/main' into simple-filters-schema
nickpeihl Oct 31, 2025
fb0f138
Changes from node scripts/lint_ts_projects --fix
kibanamachine Oct 31, 2025
b49449a
Fix filter value schema arrays to be homogeneous
nickpeihl Nov 5, 2025
ab20892
Fix casing inconsistency
nickpeihl Nov 5, 2025
48689fc
Naming consistency
nickpeihl Nov 5, 2025
f13c024
Stricter discriminated filter conditions
nickpeihl Nov 5, 2025
539a159
Document controlledBy field
nickpeihl Nov 5, 2025
565848a
Remove duplicate stored filter schema
nickpeihl Nov 5, 2025
7f85963
Changes from node scripts/lint_ts_projects --fix
kibanamachine Nov 5, 2025
51c30b1
Rename path
nickpeihl Nov 5, 2025
a2979a9
Merge remote-tracking branch 'refs/remotes/origin/simple-filters-sche…
nickpeihl Nov 5, 2025
883b3b6
Rename stored filter schema to "storedFilterSchema" and "simpleFilter…
nickpeihl Nov 6, 2025
3cbf28b
Add constants package to avoid circular references
nickpeihl Nov 7, 2025
00bd945
Add isMultiIindex, filterType, and BWC legacy properties
nickpeihl Nov 7, 2025
8aa68b4
Changes from node scripts/generate codeowners
kibanamachine Nov 7, 2025
e72bb46
Merge branch 'main' into simple-filters-schema
nickpeihl Nov 7, 2025
6b5e025
Rename AsCodeFilter types
nickpeihl Nov 7, 2025
f94f926
Merge remote-tracking branch 'origin/simple-filters-schema' into simp…
nickpeihl Nov 7, 2025
d1c1ea2
As "asCode" prefix to exported schemas
nickpeihl Nov 7, 2025
27870ca
Don't export StoredFilter types
nickpeihl Nov 7, 2025
36da619
Use schema extend instead of spread
nickpeihl Nov 7, 2025
7702432
Remove unnecessary nested query object from DSL schema
nickpeihl Nov 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion src/platform/packages/shared/kbn-es-query-server/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@

export { timeRangeSchema } from './src/time_range';
export { querySchema } from './src/query';
export { filterSchema } from './src/filter';
export { appStateSchema, globalStateSchema, filterSchema } from './src/filter/stored_filter';
export { simpleFilterSchema } from './src/filter/simple_filter';

export type {
TimeRange,
Expand All @@ -19,4 +20,11 @@ export type {
FilterMeta,
AggregateQuery,
Query,
SimpleFilter,
SimpleFilterCondition,
SimpleFilterGroup,
SimpleDSLFilter,
SimpleFilterValue,
SimpleRangeValue,
StoredFilterState,
} from './src/types';
Original file line number Diff line number Diff line change
@@ -0,0 +1,261 @@
/*
Copy link
Contributor

@nreese nreese Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see type or isMultiIndex in the schema. Maps uses this to signal that the filter does not target any index. For example, install flights and web logs sample data. Create a map with layers from both. Draw a shape filter. Below is the created filter. isMultiIndex drives the UI so that an data view selector is not displayed in the DSL editor. type is used by editor for help message. See "Editing Elasticsearch Query DSL prevents filter geometry from displaying in map." in the screen shot. Maps also uses type to know which filters to display on the map extractFeaturesFromFilters

Screenshot 2025-11-05 at 6 43 21 PM
{
  "meta": {
   "type": "spatial_filter",
   "negate": false,
   "alias": "intersects shape",
   "disabled": false,
   "isMultiIndex": true
  },
  "query": {
   "bool": {
    "should": [
     {
      "bool": {
       "must": [
        {
         "exists": {
          "field": "DestLocation"
         }
        },
        {
         "geo_shape": {
          "ignore_unmapped": true,
          "DestLocation": {
           "relation": "intersects",
           "shape": {
            "coordinates": [
             [
              [
               -116.98646,
               49.95363
              ],
              [
               -69.4932,
               36.01698
              ],
              [
               -89.978,
               27.62399
              ],
              [
               -114.50929,
               36.07558
              ],
              [
               -116.98646,
               49.95363
              ]
             ]
            ],
            "type": "Polygon"
           }
          }
         }
        }
       ]
      }
     },
     {
      "bool": {
       "must": [
        {
         "exists": {
          "field": "geo.coordinates"
         }
        },
        {
         "geo_shape": {
          "ignore_unmapped": true,
          "geo.coordinates": {
           "relation": "intersects",
           "shape": {
            "coordinates": [
             [
              [
               -116.98646,
               49.95363
              ],
              [
               -69.4932,
               36.01698
              ],
              [
               -89.978,
               27.62399
              ],
              [
               -114.50929,
               36.07558
              ],
              [
               -116.98646,
               49.95363
              ]
             ]
            ],
            "type": "Polygon"
           }
          }
         }
        }
       ]
      }
     }
    ]
   }
  },
  "$state": {
   "store": "appState"
  }
 }

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the isMultiIndex and type properties to the schema to support spatial filters. But I wonder if we want to have a higher level of abstraction for API users to create spatial filters? Currently, this schema only supports spatial filters via a DSL query. I suppose that could always be a later enhancement.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, would be nice to have a better abstraction for those filters, especially if they're the reason for all the new properties in the schema. Doesn't have to be in this PR though.

* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the "Elastic License
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
* Public License v 1"; you may not use this file except in compliance with, at
* your election, the "Elastic License 2.0", the "GNU Affero General Public
* License v3.0 only", or the "Server Side Public License, v 1".
*/

/**
* Validation Schemas for Simplified Filter Interface
*
* These schemas are used for server validation of API requests and responses
* in * as Code APIs.
*/

import { schema } from '@kbn/config-schema';

// ====================================================================
// CORE FILTER OPERATOR AND VALUE SCHEMAS
// ====================================================================

/**
* Schema for range values used in numeric and date filters
*/
export const rangeValueSchema = schema.object({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about just rangeSchema?

gte: schema.maybe(
schema.oneOf([schema.number(), schema.string()], {
meta: { description: 'Greater than or equal to' },
})
),
lte: schema.maybe(
schema.oneOf([schema.number(), schema.string()], {
meta: { description: 'Less than or equal to' },
})
),
gt: schema.maybe(
schema.oneOf([schema.number(), schema.string()], {
meta: { description: 'Greater than' },
})
),
lt: schema.maybe(
schema.oneOf([schema.number(), schema.string()], {
meta: { description: 'Less than' },
})
),
});

/**
* Schema for all possible filter values
*/
export const filterValueSchema = schema.oneOf(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about just valueSchema

[
schema.string(),
schema.number(),
schema.boolean(),
schema.arrayOf(schema.string()),
schema.arrayOf(schema.number()),
schema.arrayOf(schema.boolean()),
rangeValueSchema,
],
{ meta: { description: 'Filter value - single value, array of homogeneous values, or range' } }
);

// ====================================================================
// BASE FILTER PROPERTIES (SHARED BY ALL SIMPLIFIED FILTERS)
// ====================================================================

/**
* Base properties shared by all simplified filters
*/
const baseFilterPropertiesSchema = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about just baseProperties. Its not a schema.object, its just a plain of javascript object. Its in the filters file so I don't think it needs the word filters in the name.

pinned: schema.maybe(
schema.boolean({
meta: { description: 'Whether the filter is pinned' },
})
),
disabled: schema.maybe(
schema.boolean({
meta: { description: 'Whether the filter is disabled' },
})
),
controlledBy: schema.maybe(
schema.string({
meta: {
description: 'Optional identifier for the component/plugin managing this filter',
},
})
),
dataViewId: schema.maybe(
schema.string({
meta: { description: 'Data view ID that this filter applies to' },
})
),
negate: schema.maybe(
schema.boolean({
meta: { description: 'Whether to negate the filter condition' },
})
),
label: schema.maybe(
schema.string({
meta: { description: 'Human-readable label for the filter' },
})
),
};

// ====================================================================
// SIMPLE FILTER CONDITION SCHEMAS
// ====================================================================

/**
* Common field property for all filter conditions
*/
const filterConditionFieldSchema = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about just conditionField?

field: schema.string({ meta: { description: 'Field the filter applies to' } }),
};

/**
* Schema for 'is' and 'is_not' operators with single value
*/
const filterConditionIsSingleSchema = schema.object({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about just singleConditionSchema?

...filterConditionFieldSchema,
operator: schema.oneOf([schema.literal('is'), schema.literal('is_not')], {
meta: { description: 'Single value comparison operators' },
}),
value: schema.oneOf([schema.string(), schema.number(), schema.boolean()], {
meta: { description: 'Single value for comparison' },
}),
});

/**
* Schema for 'is_one_of' and 'is_not_one_of' operators with array values
*/
const filterConditionIsOneOfSchema = schema.object({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about oneOfConditionSchema?

...filterConditionFieldSchema,
operator: schema.oneOf([schema.literal('is_one_of'), schema.literal('is_not_one_of')], {
meta: { description: 'Array value comparison operators' },
}),
value: schema.oneOf(
[
schema.arrayOf(schema.string()),
schema.arrayOf(schema.number()),
schema.arrayOf(schema.boolean()),
],
{ meta: { description: 'Homogeneous array of values' } }
),
});

/**
* Schema for 'range' operator with range value
*/
const filterConditionRangeSchema = schema.object({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about rangeConditionSchema?

...filterConditionFieldSchema,
operator: schema.literal('range'),
value: rangeValueSchema,
});

/**
* Schema for 'exists' and 'not_exists' operators without value
*/
const filterConditionExistsSchema = schema.object({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about existsConditionSchema?

...filterConditionFieldSchema,
operator: schema.oneOf([schema.literal('exists'), schema.literal('not_exists')], {
meta: { description: 'Field existence check operators' },
}),
// value is intentionally omitted for exists/not_exists operators
});

/**
* Discriminated union schema for simple filter conditions with proper operator/value type combinations
*/
export const simpleFilterConditionSchema = schema.oneOf(
[
filterConditionIsSingleSchema,
filterConditionIsOneOfSchema,
filterConditionRangeSchema,
filterConditionExistsSchema,
],
{ meta: { description: 'A filter condition with strict operator/value type matching' } }
);

// ====================================================================
// FILTER GROUP SCHEMA (RECURSIVE)
// ====================================================================

/**
* Schema for logical filter groups with recursive structure
* Uses lazy schema to handle recursive references
* Note: Groups only contain logical structure (type, conditions) - no metadata properties
*/
export const filterGroupSchema = schema.object(
{
type: schema.oneOf([schema.literal('and'), schema.literal('or')]),
conditions: schema.arrayOf(
schema.oneOf([
simpleFilterConditionSchema,
schema.lazy('filterGroup'), // Recursive reference
])
),
},
{ meta: { description: 'Grouped filters', id: 'filterGroup' } }
);

// ====================================================================
// RAW DSL FILTER SCHEMA
// ====================================================================

/**
* Schema for raw Elasticsearch Query DSL filters
*/
export const rawDSLFilterSchema = schema.object({
query: schema.recordOf(schema.string(), schema.any(), {
meta: { description: 'Elasticsearch Query DSL object' },
}),
});

// ====================================================================
// SIMPLE FILTER DISCRIMINATED UNION SCHEMA
// ====================================================================

/**
* Schema for simple condition filters (Tier 1)
*/
export const simpleConditionFilterSchema = schema.object(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simpleConditionFilterSchema and simpleFilterConditionSchema are very similar sounding names that do not distinguish between the 2.

{
...baseFilterPropertiesSchema,
condition: simpleFilterConditionSchema,
},
{ meta: { description: 'Simple condition filter' } }
);

/**
* Schema for grouped condition filters (Tier 2-3)
*/
export const simpleGroupFilterSchema = schema.object(
{
...baseFilterPropertiesSchema,
group: filterGroupSchema,
},
{ meta: { description: 'Grouped condition filter' } }
);

/**
* Schema for raw DSL filters (Tier 4)
*/
export const simpleDSLFilterSchema = schema.object(
{
...baseFilterPropertiesSchema,
dsl: rawDSLFilterSchema,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dsl: rawDSLFilterSchema,
query: rawDSLFilterSchema,

I think we only ever support the query DSL property, don't we? I see we now expect an object containing query for the dsl prop, but I'm not sure anything else is actually valid. Maybe not even necessary to change the prop name, but should we get rid of that layer of nesting and only accept a query object directly?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good question. I wonder though if keeping the dsl property name would help us discriminate for possible future filter types such as support for ES|QL filters? In that case, I might see us introducing another schema like

export const simplifiedESQLFilterSchema = schema.object({
  ...baseFilterPropertiesSchema,
  esql: esqlFilterSchema,
})

We could still get rid of the unnecessary nesting of the query property. WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping dsl and removing the nesting works for me! The prop name argument makes sense, it's mainly the extra layer of nesting I think could be confusing. And the prospect of ES|QL based filters is both intriguing and makes my head spin 😁

},
{ meta: { description: 'Raw DSL filter' } }
);

/**
* Main discriminated union schema for SimpleFilter
* Ensures exactly one of: condition, group, or dsl is present
*/
export const simpleFilterSchema = schema.oneOf(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I like the simple prefix. What is the plan with these schemas? If this is the preferred shape for filters moving forward, then the existing filter schemas should be prefixed with legacy or stored and then the schemas in this file should just be filter without a prefix.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current filters interface is very ubiquitous around Kibana as it is has historically only validated in runtime with a concrete TypeScript type. A few months ago I introduced a server-side validations schema that is nearly syntactically identical to the concrete Filter type to support server-side validation. Although these concrete and inferred filter types are nearly identical now, they might diverge as TypeScript runtime validation and server validation may have different requirements. Or maybe the inferred type from stored filter schema can replace the concrete TypeScript type without too much interference for developer experience. But that is likely outside the scope of the as-code efforts.

I see the schema introduced in this PR as a high level API interface over the stored filters specifically for as-code endpoints. It provides stricter validation on the API endpoints and is intended to be easier for API consumers. To that end, I think it makes sense to rename the filterSchema to storedFilterSchema and drop the simple prefix in the API schemas. WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To that end, I think it makes sense to rename the filterSchema to storedFilterSchema and drop the simple prefix in the API schemas. WDYT?

That sounds like a good plan.

[simpleConditionFilterSchema, simpleGroupFilterSchema, simpleDSLFilterSchema],
{ meta: { description: 'A filter which can be a condition, group, or raw DSL' } }
);
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,16 @@
*/

import { schema } from '@kbn/config-schema';
import { FilterStateStore } from '@kbn/es-query';

const filterStateStoreSchema = schema.oneOf(
[schema.literal(FilterStateStore.APP_STATE), schema.literal(FilterStateStore.GLOBAL_STATE)],
{
meta: {
description:
"Denote whether a filter is specific to an application's context (e.g. 'appState') or whether it should be applied globally (e.g. 'globalState').",
},
}
);
export const appStateSchema = schema.literal('appState');
export const globalStateSchema = schema.literal('globalState');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, I might have approved prematurely. Didn't notice the type errors until now. Is it related to this change, maybe we need to stick with FilterStateStore here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the FilterStateStore enum causes a circular reference error since that would mean @kbn/es-query and @kbn/es-query-server depend on each other. Maybe we can export that enum (with future room for any others) from a new package (ex. @kbn/es-query-constants). Or is there an existing constants package we should use?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not aware of another package that would be good for this, so the new constants package sounds good 👍


const filterStateStoreSchema = schema.oneOf([appStateSchema, globalStateSchema], {
meta: {
description:
"Denote whether a filter is specific to an application's context (e.g. 'appState') or whether it should be applied globally (e.g. 'globalState').",
},
});

export const filterMetaSchema = schema.object(
{
Expand All @@ -31,6 +30,7 @@ export const filterMetaSchema = schema.object(
group: schema.maybe(
schema.string({ meta: { description: 'The group to which this filter belongs.' } })
),
relation: schema.maybe(schema.string()),
// field is missing from the Filter type, but is stored in SerializedSearchSourceFields
// see the todo in src/platform/packages/shared/kbn-es-query/src/filters/helpers/update_filter.ts
field: schema.maybe(schema.string()),
Expand Down
28 changes: 27 additions & 1 deletion src/platform/packages/shared/kbn-es-query-server/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,20 @@ import type {
relativeTimeRangeSchema,
} from './time_range';
import type { aggregateQuerySchema, querySchema } from './query';
import type { filterSchema, filterMetaSchema } from './filter';
import type {
filterSchema,
filterMetaSchema,
globalStateSchema,
appStateSchema,
} from './filter/stored_filter';
import type {
simpleFilterSchema,
simpleFilterConditionSchema,
filterGroupSchema,
rawDSLFilterSchema,
filterValueSchema,
rangeValueSchema,
} from './filter/simple_filter';

export type TimeRange = Writable<TypeOf<typeof timeRangeSchema>>;
export type AbsoluteTimeRange = TypeOf<typeof absoluteTimeRangeSchema>;
Expand All @@ -26,3 +39,16 @@ export type AggregateQuery = Writable<TypeOf<typeof aggregateQuerySchema>>;

export type Filter = Writable<TypeOf<typeof filterSchema>>;
export type FilterMeta = Writable<TypeOf<typeof filterMetaSchema>>;

/**
* Schema-inferred types for Simple Filter API
*
* These types are inferred from validation schemas and provide runtime validation compatibility.
*/
export type SimpleFilter = Writable<TypeOf<typeof simpleFilterSchema>>;
export type SimpleFilterCondition = Writable<TypeOf<typeof simpleFilterConditionSchema>>;
export type SimpleFilterGroup = Writable<TypeOf<typeof filterGroupSchema>>;
export type SimpleDSLFilter = Writable<TypeOf<typeof rawDSLFilterSchema>>;
export type SimpleFilterValue = Writable<TypeOf<typeof filterValueSchema>>;
export type SimpleRangeValue = Writable<TypeOf<typeof rangeValueSchema>>;
export type StoredFilterState = TypeOf<typeof appStateSchema> | TypeOf<typeof globalStateSchema>;
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,5 @@
"kbn_references": [
"@kbn/config-schema",
"@kbn/utility-types",
"@kbn/es-query",
],
}