-
Notifications
You must be signed in to change notification settings - Fork 8.5k
[Dashboards as code] Define schemas for As Code API filter interface #241198
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 14 commits
d264710
5989e98
f900770
5037bbe
fb0f138
b49449a
ab20892
48689fc
f13c024
539a159
565848a
7f85963
51c30b1
a2979a9
883b3b6
3cbf28b
00bd945
8aa68b4
e72bb46
6b5e025
f94f926
d1c1ea2
27870ca
36da619
7702432
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,261 @@ | ||||||
| /* | ||||||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||||||
| * or more contributor license agreements. Licensed under the "Elastic License | ||||||
| * 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side | ||||||
| * Public License v 1"; you may not use this file except in compliance with, at | ||||||
| * your election, the "Elastic License 2.0", the "GNU Affero General Public | ||||||
| * License v3.0 only", or the "Server Side Public License, v 1". | ||||||
| */ | ||||||
|
|
||||||
| /** | ||||||
| * Validation Schemas for Simplified Filter Interface | ||||||
| * | ||||||
| * These schemas are used for server validation of API requests and responses | ||||||
| * in * as Code APIs. | ||||||
| */ | ||||||
|
|
||||||
| import { schema } from '@kbn/config-schema'; | ||||||
|
|
||||||
| // ==================================================================== | ||||||
| // CORE FILTER OPERATOR AND VALUE SCHEMAS | ||||||
| // ==================================================================== | ||||||
|
|
||||||
| /** | ||||||
| * Schema for range values used in numeric and date filters | ||||||
| */ | ||||||
| export const rangeValueSchema = schema.object({ | ||||||
|
||||||
| gte: schema.maybe( | ||||||
| schema.oneOf([schema.number(), schema.string()], { | ||||||
| meta: { description: 'Greater than or equal to' }, | ||||||
| }) | ||||||
| ), | ||||||
| lte: schema.maybe( | ||||||
| schema.oneOf([schema.number(), schema.string()], { | ||||||
| meta: { description: 'Less than or equal to' }, | ||||||
| }) | ||||||
| ), | ||||||
| gt: schema.maybe( | ||||||
| schema.oneOf([schema.number(), schema.string()], { | ||||||
| meta: { description: 'Greater than' }, | ||||||
| }) | ||||||
| ), | ||||||
| lt: schema.maybe( | ||||||
| schema.oneOf([schema.number(), schema.string()], { | ||||||
| meta: { description: 'Less than' }, | ||||||
| }) | ||||||
| ), | ||||||
| }); | ||||||
|
|
||||||
| /** | ||||||
| * Schema for all possible filter values | ||||||
| */ | ||||||
| export const filterValueSchema = schema.oneOf( | ||||||
|
||||||
| [ | ||||||
| schema.string(), | ||||||
| schema.number(), | ||||||
| schema.boolean(), | ||||||
| schema.arrayOf(schema.string()), | ||||||
| schema.arrayOf(schema.number()), | ||||||
| schema.arrayOf(schema.boolean()), | ||||||
| rangeValueSchema, | ||||||
| ], | ||||||
| { meta: { description: 'Filter value - single value, array of homogeneous values, or range' } } | ||||||
| ); | ||||||
|
|
||||||
| // ==================================================================== | ||||||
| // BASE FILTER PROPERTIES (SHARED BY ALL SIMPLIFIED FILTERS) | ||||||
| // ==================================================================== | ||||||
|
|
||||||
| /** | ||||||
| * Base properties shared by all simplified filters | ||||||
| */ | ||||||
| const baseFilterPropertiesSchema = { | ||||||
|
||||||
| pinned: schema.maybe( | ||||||
| schema.boolean({ | ||||||
| meta: { description: 'Whether the filter is pinned' }, | ||||||
| }) | ||||||
| ), | ||||||
| disabled: schema.maybe( | ||||||
| schema.boolean({ | ||||||
| meta: { description: 'Whether the filter is disabled' }, | ||||||
| }) | ||||||
| ), | ||||||
| controlledBy: schema.maybe( | ||||||
| schema.string({ | ||||||
| meta: { | ||||||
| description: 'Optional identifier for the component/plugin managing this filter', | ||||||
| }, | ||||||
| }) | ||||||
| ), | ||||||
| dataViewId: schema.maybe( | ||||||
| schema.string({ | ||||||
| meta: { description: 'Data view ID that this filter applies to' }, | ||||||
| }) | ||||||
| ), | ||||||
| negate: schema.maybe( | ||||||
| schema.boolean({ | ||||||
| meta: { description: 'Whether to negate the filter condition' }, | ||||||
| }) | ||||||
| ), | ||||||
| label: schema.maybe( | ||||||
| schema.string({ | ||||||
| meta: { description: 'Human-readable label for the filter' }, | ||||||
| }) | ||||||
| ), | ||||||
| }; | ||||||
|
|
||||||
| // ==================================================================== | ||||||
| // SIMPLE FILTER CONDITION SCHEMAS | ||||||
| // ==================================================================== | ||||||
|
|
||||||
| /** | ||||||
| * Common field property for all filter conditions | ||||||
| */ | ||||||
| const filterConditionFieldSchema = { | ||||||
|
||||||
| field: schema.string({ meta: { description: 'Field the filter applies to' } }), | ||||||
| }; | ||||||
|
|
||||||
| /** | ||||||
| * Schema for 'is' and 'is_not' operators with single value | ||||||
| */ | ||||||
| const filterConditionIsSingleSchema = schema.object({ | ||||||
|
||||||
| ...filterConditionFieldSchema, | ||||||
| operator: schema.oneOf([schema.literal('is'), schema.literal('is_not')], { | ||||||
| meta: { description: 'Single value comparison operators' }, | ||||||
| }), | ||||||
| value: schema.oneOf([schema.string(), schema.number(), schema.boolean()], { | ||||||
| meta: { description: 'Single value for comparison' }, | ||||||
| }), | ||||||
| }); | ||||||
|
|
||||||
| /** | ||||||
| * Schema for 'is_one_of' and 'is_not_one_of' operators with array values | ||||||
| */ | ||||||
| const filterConditionIsOneOfSchema = schema.object({ | ||||||
|
||||||
| ...filterConditionFieldSchema, | ||||||
| operator: schema.oneOf([schema.literal('is_one_of'), schema.literal('is_not_one_of')], { | ||||||
| meta: { description: 'Array value comparison operators' }, | ||||||
| }), | ||||||
| value: schema.oneOf( | ||||||
| [ | ||||||
| schema.arrayOf(schema.string()), | ||||||
| schema.arrayOf(schema.number()), | ||||||
| schema.arrayOf(schema.boolean()), | ||||||
| ], | ||||||
| { meta: { description: 'Homogeneous array of values' } } | ||||||
| ), | ||||||
| }); | ||||||
|
|
||||||
| /** | ||||||
| * Schema for 'range' operator with range value | ||||||
| */ | ||||||
| const filterConditionRangeSchema = schema.object({ | ||||||
|
||||||
| ...filterConditionFieldSchema, | ||||||
| operator: schema.literal('range'), | ||||||
| value: rangeValueSchema, | ||||||
| }); | ||||||
|
|
||||||
| /** | ||||||
| * Schema for 'exists' and 'not_exists' operators without value | ||||||
| */ | ||||||
| const filterConditionExistsSchema = schema.object({ | ||||||
|
||||||
| ...filterConditionFieldSchema, | ||||||
| operator: schema.oneOf([schema.literal('exists'), schema.literal('not_exists')], { | ||||||
| meta: { description: 'Field existence check operators' }, | ||||||
| }), | ||||||
| // value is intentionally omitted for exists/not_exists operators | ||||||
| }); | ||||||
|
|
||||||
| /** | ||||||
| * Discriminated union schema for simple filter conditions with proper operator/value type combinations | ||||||
| */ | ||||||
| export const simpleFilterConditionSchema = schema.oneOf( | ||||||
| [ | ||||||
| filterConditionIsSingleSchema, | ||||||
| filterConditionIsOneOfSchema, | ||||||
| filterConditionRangeSchema, | ||||||
| filterConditionExistsSchema, | ||||||
| ], | ||||||
| { meta: { description: 'A filter condition with strict operator/value type matching' } } | ||||||
| ); | ||||||
|
|
||||||
| // ==================================================================== | ||||||
| // FILTER GROUP SCHEMA (RECURSIVE) | ||||||
| // ==================================================================== | ||||||
|
|
||||||
| /** | ||||||
| * Schema for logical filter groups with recursive structure | ||||||
| * Uses lazy schema to handle recursive references | ||||||
| * Note: Groups only contain logical structure (type, conditions) - no metadata properties | ||||||
| */ | ||||||
| export const filterGroupSchema = schema.object( | ||||||
| { | ||||||
| type: schema.oneOf([schema.literal('and'), schema.literal('or')]), | ||||||
| conditions: schema.arrayOf( | ||||||
| schema.oneOf([ | ||||||
| simpleFilterConditionSchema, | ||||||
| schema.lazy('filterGroup'), // Recursive reference | ||||||
| ]) | ||||||
| ), | ||||||
| }, | ||||||
| { meta: { description: 'Grouped filters', id: 'filterGroup' } } | ||||||
| ); | ||||||
|
|
||||||
| // ==================================================================== | ||||||
| // RAW DSL FILTER SCHEMA | ||||||
| // ==================================================================== | ||||||
|
|
||||||
| /** | ||||||
| * Schema for raw Elasticsearch Query DSL filters | ||||||
| */ | ||||||
| export const rawDSLFilterSchema = schema.object({ | ||||||
| query: schema.recordOf(schema.string(), schema.any(), { | ||||||
| meta: { description: 'Elasticsearch Query DSL object' }, | ||||||
| }), | ||||||
| }); | ||||||
|
|
||||||
| // ==================================================================== | ||||||
| // SIMPLE FILTER DISCRIMINATED UNION SCHEMA | ||||||
| // ==================================================================== | ||||||
|
|
||||||
| /** | ||||||
| * Schema for simple condition filters (Tier 1) | ||||||
| */ | ||||||
| export const simpleConditionFilterSchema = schema.object( | ||||||
|
||||||
| { | ||||||
| ...baseFilterPropertiesSchema, | ||||||
| condition: simpleFilterConditionSchema, | ||||||
| }, | ||||||
| { meta: { description: 'Simple condition filter' } } | ||||||
| ); | ||||||
|
|
||||||
| /** | ||||||
| * Schema for grouped condition filters (Tier 2-3) | ||||||
| */ | ||||||
| export const simpleGroupFilterSchema = schema.object( | ||||||
| { | ||||||
| ...baseFilterPropertiesSchema, | ||||||
| group: filterGroupSchema, | ||||||
| }, | ||||||
| { meta: { description: 'Grouped condition filter' } } | ||||||
| ); | ||||||
|
|
||||||
| /** | ||||||
| * Schema for raw DSL filters (Tier 4) | ||||||
| */ | ||||||
| export const simpleDSLFilterSchema = schema.object( | ||||||
| { | ||||||
| ...baseFilterPropertiesSchema, | ||||||
| dsl: rawDSLFilterSchema, | ||||||
|
||||||
| dsl: rawDSLFilterSchema, | |
| query: rawDSLFilterSchema, |
I think we only ever support the query DSL property, don't we? I see we now expect an object containing query for the dsl prop, but I'm not sure anything else is actually valid. Maybe not even necessary to change the prop name, but should we get rid of that layer of nesting and only accept a query object directly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a good question. I wonder though if keeping the dsl property name would help us discriminate for possible future filter types such as support for ES|QL filters? In that case, I might see us introducing another schema like
export const simplifiedESQLFilterSchema = schema.object({
...baseFilterPropertiesSchema,
esql: esqlFilterSchema,
})We could still get rid of the unnecessary nesting of the query property. WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Keeping dsl and removing the nesting works for me! The prop name argument makes sense, it's mainly the extra layer of nesting I think could be confusing. And the prospect of ES|QL based filters is both intriguing and makes my head spin 😁
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure I like the simple prefix. What is the plan with these schemas? If this is the preferred shape for filters moving forward, then the existing filter schemas should be prefixed with legacy or stored and then the schemas in this file should just be filter without a prefix.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The current filters interface is very ubiquitous around Kibana as it is has historically only validated in runtime with a concrete TypeScript type. A few months ago I introduced a server-side validations schema that is nearly syntactically identical to the concrete Filter type to support server-side validation. Although these concrete and inferred filter types are nearly identical now, they might diverge as TypeScript runtime validation and server validation may have different requirements. Or maybe the inferred type from stored filter schema can replace the concrete TypeScript type without too much interference for developer experience. But that is likely outside the scope of the as-code efforts.
I see the schema introduced in this PR as a high level API interface over the stored filters specifically for as-code endpoints. It provides stricter validation on the API endpoints and is intended to be easier for API consumers. To that end, I think it makes sense to rename the filterSchema to storedFilterSchema and drop the simple prefix in the API schemas. WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To that end, I think it makes sense to rename the filterSchema to storedFilterSchema and drop the simple prefix in the API schemas. WDYT?
That sounds like a good plan.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,17 +8,16 @@ | |
| */ | ||
|
|
||
| import { schema } from '@kbn/config-schema'; | ||
| import { FilterStateStore } from '@kbn/es-query'; | ||
|
|
||
| const filterStateStoreSchema = schema.oneOf( | ||
| [schema.literal(FilterStateStore.APP_STATE), schema.literal(FilterStateStore.GLOBAL_STATE)], | ||
| { | ||
| meta: { | ||
| description: | ||
| "Denote whether a filter is specific to an application's context (e.g. 'appState') or whether it should be applied globally (e.g. 'globalState').", | ||
| }, | ||
| } | ||
| ); | ||
| export const appStateSchema = schema.literal('appState'); | ||
| export const globalStateSchema = schema.literal('globalState'); | ||
|
||
|
|
||
| const filterStateStoreSchema = schema.oneOf([appStateSchema, globalStateSchema], { | ||
| meta: { | ||
| description: | ||
| "Denote whether a filter is specific to an application's context (e.g. 'appState') or whether it should be applied globally (e.g. 'globalState').", | ||
| }, | ||
| }); | ||
|
|
||
| export const filterMetaSchema = schema.object( | ||
| { | ||
|
|
@@ -31,6 +30,7 @@ export const filterMetaSchema = schema.object( | |
| group: schema.maybe( | ||
| schema.string({ meta: { description: 'The group to which this filter belongs.' } }) | ||
| ), | ||
| relation: schema.maybe(schema.string()), | ||
| // field is missing from the Filter type, but is stored in SerializedSearchSourceFields | ||
| // see the todo in src/platform/packages/shared/kbn-es-query/src/filters/helpers/update_filter.ts | ||
| field: schema.maybe(schema.string()), | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,6 +9,5 @@ | |
| "kbn_references": [ | ||
| "@kbn/config-schema", | ||
| "@kbn/utility-types", | ||
| "@kbn/es-query", | ||
| ], | ||
| } | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see
typeorisMultiIndexin the schema. Maps uses this to signal that the filter does not target any index. For example, install flights and web logs sample data. Create a map with layers from both. Draw a shape filter. Below is the created filter.isMultiIndexdrives the UI so that an data view selector is not displayed in the DSL editor.typeis used by editor for help message. See "Editing Elasticsearch Query DSL prevents filter geometry from displaying in map." in the screen shot. Maps also usestypeto know which filters to display on the map extractFeaturesFromFiltersThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've added the
isMultiIndexandtypeproperties to the schema to support spatial filters. But I wonder if we want to have a higher level of abstraction for API users to create spatial filters? Currently, this schema only supports spatial filters via a DSL query. I suppose that could always be a later enhancement.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, would be nice to have a better abstraction for those filters, especially if they're the reason for all the new properties in the schema. Doesn't have to be in this PR though.