Address Netty failure at Agent Execute runtime

[brianf-aws]

Description

Upon making a CVE fix #4298 . which involved bumping netty, there was a netty exception.

? ERROR][o.o.m.e.a.a.MLAgentExecutor] [integTest-0] Failed to run conversational agent
?  org.opensearch.OpenSearchStatusException: Error communicating with remote model: java.lang.IllegalStateException: unexpected message type: LastHttpContent$1, state: 0
?  	at org.opensearch.ml.engine.algorithms.remote.MLSdkAsyncHttpResponseHandler.onError(MLSdkAsyncHttpResponseHandler.java:108) [opensearch-ml-algorithms-2.19.4.0-SNAPSHOT.jar:?]
?  	at software.amazon.awssd

There exists a cherry pick which bumped netty on mainline but required code changes
#4175 . The issue here is that the version catalog in that mainline commit was not synced from core 2.19.4 . Making the change non-trivial

Reviewer objectives

• To ensure at least 1 CI passes that no errors are related to netty
• To review commits from use mainline versions.aws via hardcode and above
• To note that the lack of version catalog is because core's 2.19.4 version catalog is 11 months old!

Related Issues

Resolves the snapshot PR #4143

Next steps

• backport the Core version catalog to 2.19.4 [OS core]
• Update 2.19.4 to have proper version catalog [ML-Commons]

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[brianf-aws]

I don't think this new change is a result of the failures

 ERROR][o.o.m.e.a.a.MLAgentExecutor] [integTest-0] Failed to run flow agent
»  java.lang.IllegalArgumentException: SearchIndexTool's two parameter: index and query are required!
»  	at org.opensearch.ml.engine.tools.SearchIndexTool.run(SearchIndexTool.java:108) [opensearch-ml-algorithms-2.19.4.0-SNAPSHOT.jar:?]
»  	at org.opensearch.ml.engine.algorithms.agent.MLFlowAgentRunner.run(MLFlowAgentRunner.java:160) [opensearch-ml-algorithms-2.19.4.0-SNAPSHOT.jar:?]
»  	at org.opensearch.ml.engine.algorithms.agent.MLAgentExecutor.executeAgent(MLAgentExecutor.java:326) [opensearch-ml-algorithms-2.19.4.0-SNAPSHOT.jar:?]
»  	at org.opensearch.ml.engine.algorithms.agent.MLAgentExecutor.lambda$execute$4(MLAgentExecutor.java:248) [opensearch-ml-algorithms-2.19.4.0-SNAPSHOT.jar:?]
»  	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) [?:?]
»  	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) [?:?]
»  	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) [?:?]

  org.opensearch.OpenSearchStatusException: Error validating input schema, if you think this is expected, please update your 'input' field in the 'interface' field for this model: Validation failed: [$.parameters.prompt: string found, integer expected] for instance: {"algorithm":"REMOTE","parameters":{"prompt":"Say this is a test"},"action_type":"PREDICT"} with schema: {"properties":{"parameters":{"properties":{"prompt":{"description":"This is a test description field","type":"integer"}}}}}
»  	at org.opensearch.ml.action.prediction.TransportPredictionTaskAction.validateInputSchema(TransportPredictionTaskAction.java:270) ~[?:?]
»  	at org.opensearch.ml.action.prediction.TransportPredictionTaskAction$1.lambda$onResponse$0(TransportPredictionTaskAction.java:181) ~[?:?]
»  	at org.opensearch.core.action.ActionListener$1.onResponse(ActionListener.java:82) [opensearch-core-2.19.4-SNAPSHOT.jar:2.19.4-SNAPSHOT]
»  	at org.opensearch.ml.helper.ModelAccessControlHelper.validateModelGroupAccess(ModelAccessControlHelper.java:140) [opensearch-ml-2.19.4.0-SNAPSHOT.jar:2.19.4.0-SNAPSHOT]
»  	at org.opensearch.ml.action.prediction.TransportPredictionTaskAction$1.onResponse(TransportPredictionTaskAction.java:134) [opensearch-ml-2.19.4.0-SNAPSHOT.jar:2.19.4.0-SNAPSHOT]
»  	at org.opensearch.ml.action.prediction.TransportPredictionTaskAction$1.onResponse(TransportPredictionTaskAction.java:123) [opensearch-ml-2.19.4.0-SNAPSHOT.jar:2.19.4.0-SNAPSHOT]
»  	at org.opensearch.ml.action.prediction.TransportPredictionTaskAction.doExecute(TransportPredictionTaskAction.java:221) [opensearch-ml-2.19.4.0-SNAPSHOT.jar:2.19.4.0-SNAPSHOT]