Skip to content

Commit 90393bf

Browse files
authored
Fix memory leak: ExecutionEngine recreated per query appending to global function registry (#5222)
* Fix memory leak: ExecutionEngine recreated per query appending to global function registry The OpenSearchExecutionEngine was not scoped as @singleton in the Guice module, causing a new instance to be created for every SQL/PPL request via injector.getInstance(SQLService.class). Each construction called registerOpenSearchFunctions(), which appended entries to the static PPLFuncImpTable.INSTANCE.externalFunctionRegistry via compute(). After N queries, the GEOIP function list contained N entries (each holding GeoIpFunction, SqlIdentifier, CalciteFuncSignature, PPLTypeChecker, etc.), leading to ~2GB heap retention after ~730K queries. Fix: 1. Add @singleton to the ExecutionEngine provider in OpenSearchPluginModule so only one instance is created per Guice injector. 2. Change registerExternalOperator() from compute()/append to put()/replace so repeated registrations overwrite instead of accumulating. Signed-off-by: Kai Huang <kaihuang@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> * Apply spotless formatting Signed-off-by: Kai Huang <kaihuang@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <kaihuang@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com>
1 parent df8f259 commit 90393bf

File tree

2 files changed

+4
-7
lines changed

2 files changed

+4
-7
lines changed

core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -423,14 +423,10 @@ public void registerExternalOperator(BuiltinFunctionName functionName, SqlOperat
423423
functionName.name(),
424424
operator instanceof SqlUserDefinedFunction);
425425
CalciteFuncSignature signature = new CalciteFuncSignature(functionName.getName(), typeChecker);
426-
externalFunctionRegistry.compute(
426+
externalFunctionRegistry.put(
427427
functionName,
428-
(name, existingList) -> {
429-
List<Pair<CalciteFuncSignature, FunctionImp>> list =
430-
existingList == null ? new ArrayList<>() : new ArrayList<>(existingList);
431-
list.add(Pair.of(signature, (builder, args) -> builder.makeCall(operator, args)));
432-
return list;
433-
});
428+
List.of(
429+
Pair.of(signature, (FunctionImp) (builder, args) -> builder.makeCall(operator, args))));
434430
}
435431

436432
/**

plugin/src/main/java/org/opensearch/sql/plugin/config/OpenSearchPluginModule.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ public StorageEngine storageEngine(OpenSearchClient client, Settings settings) {
5858
}
5959

6060
@Provides
61+
@Singleton
6162
public ExecutionEngine executionEngine(
6263
OpenSearchClient client, ExecutionProtector protector, PlanSerializer planSerializer) {
6364
return new OpenSearchExecutionEngine(client, protector, planSerializer);

0 commit comments

Comments
 (0)