IllegalStateException thrown in SemanticQueryBuilder

[bpintea]

Description

For a _field_caps request with params: {types=, ignore_unavailable=true, expand_wildcards=open, allow_no_indices=true, index=*,-.*, serverlessRequest=true, include_empty_fields=false}, resulting in a status: 500, the following suppressed exception is logged:

java.lang.IllegalStateException: No inference results set for [semantic_text] field [description_embedding]
	at org.elasticsearch.inference@9.0.0/org.elasticsearch.xpack.inference.queries.SemanticQueryBuilder.doRewriteBuildSemanticQuery(SemanticQueryBuilder.java:169)
	at org.elasticsearch.inference@9.0.0/org.elasticsearch.xpack.inference.queries.SemanticQueryBuilder.doRewrite(SemanticQueryBuilder.java:155)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:281)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.BoolQueryBuilder.rewriteClauses(BoolQueryBuilder.java:399)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.BoolQueryBuilder.doRewrite(BoolQueryBuilder.java:353)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:281)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.BoolQueryBuilder.rewriteClauses(BoolQueryBuilder.java:399)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.BoolQueryBuilder.doRewrite(BoolQueryBuilder.java:353)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:281)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.search.builder.SubSearchSourceBuilder.rewrite(SubSearchSourceBuilder.java:90)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.search.builder.SubSearchSourceBuilder.rewrite(SubSearchSourceBuilder.java:38)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:57)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:40)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:125)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.search.builder.SearchSourceBuilder.rewrite(SearchSourceBuilder.java:1178)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.search.builder.SearchSourceBuilder.rewrite(SearchSourceBuilder.java:94)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:57)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:40)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.search.internal.ShardSearchRequest$RequestRewritable.rewrite(ShardSearchRequest.java:588)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.search.internal.ShardSearchRequest$RequestRewritable.rewrite(ShardSearchRequest.java:577)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:57)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.search.SearchService.queryStillMatchesAfterRewrite(SearchService.java:1740)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.fieldcaps.FieldCapabilitiesFetcher.canMatchShard(FieldCapabilitiesFetcher.java:245)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.fieldcaps.FieldCapabilitiesFetcher.doFetch(FieldCapabilitiesFetcher.java:113)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.fieldcaps.FieldCapabilitiesFetcher.fetch(FieldCapabilitiesFetcher.java:77)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.fieldcaps.TransportFieldCapabilitiesAction$NodeTransportHandler.lambda$messageReceived$0(TransportFieldCapabilitiesAction.java:591)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.ActionListener.completeWith(ActionListener.java:356)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.fieldcaps.TransportFieldCapabilitiesAction$NodeTransportHandler.messageReceived(TransportFieldCapabilitiesAction.java:570)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.fieldcaps.TransportFieldCapabilitiesAction$NodeTransportHandler.messageReceived(TransportFieldCapabilitiesAction.java:565)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:579)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$3.onResponse(SecurityServerTransportInterceptor.java:632)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$3.onResponse(SecurityServerTransportInterceptor.java:621)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:640)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.ResizeRequestInterceptor.intercept(ResizeRequestInterceptor.java:105)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.ValidateRequestInterceptor.intercept(ValidateRequestInterceptor.java:20)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.ShardSearchRequestInterceptor.intercept(ShardSearchRequestInterceptor.java:23)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.UpdateRequestInterceptor.intercept(UpdateRequestInterceptor.java:27)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.SearchRequestInterceptor.intercept(SearchRequestInterceptor.java:20)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.IndicesAliasesRequestInterceptor.intercept(IndicesAliasesRequestInterceptor.java:127)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.SearchRequestCacheDisablingInterceptor.intercept(SearchRequestCacheDisablingInterceptor.java:53)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.BulkShardRequestInterceptor.intercept(BulkShardRequestInterceptor.java:85)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:638)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:634)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.interceptor.DlsFlsLicenseRequestInterceptor.intercept(DlsFlsLicenseRequestInterceptor.java:106)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService.runRequestInterceptors(AuthorizationService.java:634)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService.handleIndexActionAuthorizationResult(AuthorizationService.java:619)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$13(AuthorizationService.java:517)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:1033)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:999)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:33)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.RBACEngine.authorizeIndexAction(RBACEngine.java:384)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeAction(AuthorizationService.java:510)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService.maybeAuthorizeRunAs(AuthorizationService.java:442)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorize$3(AuthorizationService.java:329)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:257)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:33)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.RBACEngine.lambda$resolveAuthorizationInfo$0(RBACEngine.java:156)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRoles$4(CompositeRolesStore.java:202)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRole$5(CompositeRolesStore.java:220)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247)
	at org.elasticsearch.xcore@9.0.0/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$0(RoleReferenceIntersection.java:49)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.support.GroupedActionListener.onResponse(GroupedActionListener.java:57)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.buildRoleFromRoleReference(CompositeRolesStore.java:317)
	at org.elasticsearch.xcore@9.0.0/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$1(RoleReferenceIntersection.java:53)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at org.elasticsearch.xcore@9.0.0/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.buildRole(RoleReferenceIntersection.java:53)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRole(CompositeRolesStore.java:218)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRoles(CompositeRolesStore.java:195)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.RBACEngine.resolveAuthorizationInfo(RBACEngine.java:152)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:345)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.transport.ServerTransportFilter.lambda$inbound$1(ServerTransportFilter.java:114)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:247)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.action.ActionListenerImplementations$MappedActionListener.onResponse(ActionListenerImplementations.java:97)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticate(AuthenticatorChain.java:93)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:264)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:201)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.transport.ServerTransportFilter.authenticate(ServerTransportFilter.java:127)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.transport.ServerTransportFilter.inbound(ServerTransportFilter.java:105)
	at org.elasticsearch.security@9.0.0/org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:643)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:76)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:1098)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1023)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:34)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1023)
	at org.elasticsearch.server@9.0.0/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1570)\n

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[benwtrent]

It is strange that field caps is making a semantic query at all :/ I wonder what is going on there.

[benwtrent]

LOL, looking at the comments on that line of code this is throwing on:

                // This should never happen, but throw on it in case it ever does
                throw new IllegalStateException(
                    "No inference results set for [" + semanticTextFieldType.typeName() + "] field [" + fieldName + "]"
                );

The curse of "This should never happen" has struck again!

[benwtrent]

@Mikep86 @carlosdelest ^ I wonder if this is because FieldCaps isn't rewriting queries like we expect for can_match?

Basically, field_caps allows index_filter which does a can_match phase to determine if all the shards for a given index will rewrite to a match_none.

This makes me wonder if we should return a MatchAllQuery if inference results is null?

[Mikep86]

++, smells like a missing rewrite to me. It's plausible that we missed this during testing since the can_match phase is skipped most of the time.