xds: Implementation of Unified Matcher and CEL Integration #12640
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| /* | ||
| * Copyright 2026 The gRPC Authors | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package io.grpc.xds.internal.matcher; | ||
|
|
||
| import dev.cel.common.CelAbstractSyntaxTree; | ||
| import dev.cel.common.CelOptions; | ||
| import dev.cel.runtime.CelRuntime; | ||
| import dev.cel.runtime.CelRuntimeFactory; | ||
|
|
||
| /** | ||
| * Shared utilities for CEL-based matchers and extractors. | ||
| */ | ||
| final class CelCommon { | ||
| private static final CelOptions CEL_OPTIONS = CelOptions.newBuilder() | ||
| .enableComprehension(false) | ||
| .maxRegexProgramSize(100) | ||
| .build(); | ||
|
|
||
|
|
||
| private static final dev.cel.runtime.CelStandardFunctions FUNCTIONS = | ||
| dev.cel.runtime.CelStandardFunctions.newBuilder() | ||
| .filterFunctions((func, over) -> { | ||
| if (func == dev.cel.runtime.CelStandardFunctions.StandardFunction.STRING) { | ||
| return false; | ||
| } | ||
| if (func == dev.cel.runtime.CelStandardFunctions.StandardFunction.ADD) { | ||
| return !over.equals( | ||
| (Object) dev.cel.runtime.standard.AddOperator.AddOverload.ADD_STRING) | ||
|
||
| && !over.equals( | ||
| (Object) dev.cel.runtime.standard.AddOperator.AddOverload.ADD_LIST); | ||
| } | ||
| return true; | ||
| }) | ||
| .build(); | ||
|
|
||
| static final CelRuntime RUNTIME = CelRuntimeFactory.standardCelRuntimeBuilder() | ||
| .setStandardEnvironmentEnabled(false) | ||
| .setStandardFunctions(FUNCTIONS) | ||
| .setOptions(CEL_OPTIONS) | ||
| .build(); | ||
|
|
||
| private CelCommon() {} | ||
|
|
||
| static void checkAllowedVariables(CelAbstractSyntaxTree ast) { | ||
| for (java.util.Map.Entry<Long, dev.cel.common.ast.CelReference> entry : | ||
| ast.getReferenceMap().entrySet()) { | ||
| dev.cel.common.ast.CelReference ref = entry.getValue(); | ||
| // If overload_id is empty, it's a variable reference or type name. | ||
| // We only support "request". | ||
| if (!ref.value().isPresent() && ref.overloadIds().isEmpty()) { | ||
| if (!"request".equals(ref.name())) { | ||
| throw new IllegalArgumentException( | ||
| "CEL expression references unknown variable: " + ref.name()); | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| /* | ||
| * Copyright 2026 The gRPC Authors | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package io.grpc.xds.internal.matcher; | ||
|
|
||
| import dev.cel.common.CelAbstractSyntaxTree; | ||
| import dev.cel.common.types.SimpleType; | ||
| import dev.cel.runtime.CelEvaluationException; | ||
| import dev.cel.runtime.CelRuntime; | ||
|
|
||
| /** | ||
| * Executes compiled CEL expressions. | ||
| */ | ||
| public final class CelMatcher { | ||
| private final CelRuntime.Program program; | ||
|
|
||
| private CelMatcher(CelRuntime.Program program) { | ||
| this.program = program; | ||
| } | ||
|
|
||
| /** | ||
| * Compiles the AST into a CelMatcher. | ||
| * Throws an Exception if validation or evaluation fails during compilation setup. | ||
| */ | ||
| public static CelMatcher compile(CelAbstractSyntaxTree ast) | ||
| throws Exception { | ||
| // CelEvaluationException -> inside cel-runtime -> Allowed in production signatures | ||
| // CelValidationException -> inside cel-compiler -> Forbidden in production signatures | ||
| if (ast.getResultType() != SimpleType.BOOL) { | ||
| throw new IllegalArgumentException( | ||
| "CEL expression must evaluate to boolean, got: " + ast.getResultType()); | ||
| } | ||
| CelCommon.checkAllowedVariables(ast); | ||
| CelRuntime.Program program = CelCommon.RUNTIME.createProgram(ast); | ||
| return new CelMatcher(program); | ||
| } | ||
|
|
||
| /** | ||
| * Evaluates the CEL expression against the input activation. | ||
| */ | ||
| public boolean match(Object input) throws CelEvaluationException { | ||
| Object result; | ||
| if (input instanceof dev.cel.runtime.CelVariableResolver) { | ||
| result = program.eval((dev.cel.runtime.CelVariableResolver) input); | ||
| } else if (input instanceof java.util.Map) { | ||
| @SuppressWarnings("unchecked") | ||
| java.util.Map<String, ?> mapInput = (java.util.Map<String, ?>) input; | ||
| result = program.eval(mapInput); | ||
| } else { | ||
| throw new CelEvaluationException( | ||
| "Unsupported input type for CEL evaluation: " + input.getClass().getName()); | ||
| } | ||
|
|
||
| if (result instanceof Boolean) { | ||
| return (Boolean) result; | ||
| } | ||
| throw new CelEvaluationException( | ||
| "CEL expression must evaluate to boolean, got: " + result.getClass().getName()); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| /* | ||
| * Copyright 2026 The gRPC Authors | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package io.grpc.xds.internal.matcher; | ||
|
|
||
| import dev.cel.common.CelAbstractSyntaxTree; | ||
| import dev.cel.common.types.SimpleType; | ||
| import dev.cel.runtime.CelEvaluationException; | ||
| import dev.cel.runtime.CelRuntime; | ||
|
|
||
| /** | ||
| * Executes compiled CEL expressions that extract a string. | ||
| */ | ||
| public final class CelStringExtractor { | ||
| private final CelRuntime.Program program; | ||
|
|
||
| private CelStringExtractor(CelRuntime.Program program) { | ||
| this.program = program; | ||
| } | ||
|
|
||
| /** | ||
| * Compiles the AST into a CelStringExtractor. | ||
| * Throws an Exception if validation or evaluation fails during compilation setup. | ||
| */ | ||
| public static CelStringExtractor compile(CelAbstractSyntaxTree ast) | ||
| throws Exception { | ||
| if (ast.getResultType() != SimpleType.STRING && ast.getResultType() != SimpleType.DYN) { | ||
| throw new IllegalArgumentException( | ||
| "CEL expression must evaluate to string, got: " + ast.getResultType()); | ||
| } | ||
| CelCommon.checkAllowedVariables(ast); | ||
| CelRuntime.Program program = CelCommon.RUNTIME.createProgram(ast); | ||
| return new CelStringExtractor(program); | ||
| } | ||
|
|
||
| /** | ||
| * Evaluates the CEL expression against the input activation and returns the string result. | ||
| * Returns null if the result is not a string. | ||
| */ | ||
| public String extract(Object input) throws CelEvaluationException { | ||
| Object result; | ||
| if (input instanceof dev.cel.runtime.CelVariableResolver) { | ||
| result = program.eval((dev.cel.runtime.CelVariableResolver) input); | ||
| } else if (input instanceof java.util.Map) { | ||
| @SuppressWarnings("unchecked") | ||
| java.util.Map<String, ?> mapInput = (java.util.Map<String, ?>) input; | ||
| result = program.eval(mapInput); | ||
| } else { | ||
| throw new CelEvaluationException( | ||
| "Unsupported input type for CEL evaluation: " + input.getClass().getName()); | ||
| } | ||
|
|
||
| if (result instanceof String) { | ||
| return (String) result; | ||
| } | ||
| return null; | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you plan on using both the compilation and runtime packages, you could consider just taking a dependency on
dev.cel:celinstead, which includes both. It will likely make your dependency management much easier.On a separate note -- let me know if you'd like us to cut a new release.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We purposefully don't want a dependency on the cel parser. Also, it seems dev.cel:cel is busted, in that it re-defines dev.cel.runtime instead of depending on the other artifact. That will cause hard-to-debug duplicate classes in the class path.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For artifact issue -- it's tracked, we can look into prioritizing it: (b/328302799).
The PR here is referencing CelCompiler, so I think you'll need the compiler dependency (unless if it shouldn't have been included here).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem is pretty serious, as it prevents dependency management from working properly and it is not something regular people are able to debug. It's the sort of thing where everything looks fine, but then you make an unrelated change and somehow suddenly breaks CEL. So the concern is in support costs.
Really, I'm wary about merging this with the broken cel jars; it's best not to have them in our dependency tree. Even with the code disabled they can still harm users. Maybe it is best to make them
compileOnlyuntil things are fixed, which also means parts of testing and feature stabilization would be delayed until it is fixed.The PR is only using CelCompiler in tests. It will need to get reorganized to avoid the dependency for production.
@shivaspeaks, you may want to glance at go/grpc-cel-integration. It was supposed to have been superseded by the gRFC, but I'm seeing the gRFC doesn't make a point of calling out some of the goals (like not depending on the CEL compiler). It's longer than you probably want to read in full, but you may just skim it to find the "interesting" parts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll look into this next week. I was actually under the impression this wasn't a hard blocker. Thanks for surfacing it up.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is an observation @ejona86! I was trying what the doc said:
"
Production code: no need to depend on the full dev.cel:cel package.
Testing code: it's acceptable to use unstable CEL methods and depend on dev.cel:cel / dev.cel:compiler in testing code.
"
When
ChannelAndServerBuilderTestruns, it uses Guava's ClassPath to scan allio.grpc.*classes. When the JVM Verifier loadsCelMatcherorPredicateEvaluator, it traverses the method signatures and fields, eventually hittingCodePointStream. BecauseCodePointStreamimplementsCharStream, the JVM tries to loadCharStream. Since dev.cel:runtime didn't bring ANTLR transitively, and I removed dev.cel:compiler (which does bring ANTLR) from implementation, the JVM throwsNoClassDefFoundError. I am being forced to addcompileOnly(libraries.cel.compiler)or else we can addimplementation 'org.antlr:antlr4-runtime'There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've merged in the fix for this: google/cel-java#954. Runtime pulling in the antlr dependency is unintended, which is also fixed with this PR. I'll cut a new release sometime next week.
In the interim, I've pushed a snapshot version if you'd like to locally verify the fix (and let me know if you encounter any issues): https://central.sonatype.com/repository/maven-snapshots/dev/cel/runtime/0.12.0-SNAPSHOT/runtime-0.12.0-SNAPSHOT.pom
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes its working with the snapshot version.
I assume there is no periodic release schedule for cel-java, like every 6 weeks for grpc (https://github.com/grpc/grpc-java/milestones or like go/grpc-oss-release-dates)? Is it done as and when required a new release for cel?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the most part, we cut a release whenever as needed. At a minimum, we'll cut one once per quarter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
0.12.0 has been cut. Release notes here: https://github.com/google/cel-java/releases/tag/v0.12.0