[Resiliency] Unhandled RuntimeException leak internal JVM details in API response

[sjs004]

Is your feature request related to a problem? Please describe

While working on 20669 I noticed that internal exception stack trace was exposed in API Response

Below is the exact response I got

{
  "error": {
    "root_cause": [
      {
        "type": "class_cast_exception",
        "reason": "class org.opensearch.search.fetch.subphase.highlight.UnifiedHighlighter$1 cannot be cast to class org.opensearch.index.mapper.annotatedtext.AnnotatedTextFieldMapper$AnnotatedHighlighterAnalyzer (org.opensearch.search.fetch.subphase.highlight.UnifiedHighlighter$1 is in unnamed module of loader 'app'; org.opensearch.index.mapper.annotatedtext.AnnotatedTextFieldMapper$AnnotatedHighlighterAnalyzer is in unnamed module of loader java.net.FactoryURLClassLoader @762637be)"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "highlight-issue",
        "node": "V2MAV9B5RhmTAH-LS2HClw",
        "reason": {
          "type": "class_cast_exception",
          "reason": "class org.opensearch.search.fetch.subphase.highlight.UnifiedHighlighter$1 cannot be cast to class org.opensearch.index.mapper.annotatedtext.AnnotatedTextFieldMapper$AnnotatedHighlighterAnalyzer (org.opensearch.search.fetch.subphase.highlight.UnifiedHighlighter$1 is in unnamed module of loader 'app'; org.opensearch.index.mapper.annotatedtext.AnnotatedTextFieldMapper$AnnotatedHighlighterAnalyzer is in unnamed module of loader java.net.FactoryURLClassLoader @762637be)"
        }
      }
    ],
    "caused_by": {
      "type": "class_cast_exception",
      "reason": "class org.opensearch.search.fetch.subphase.highlight.UnifiedHighlighter$1 cannot be cast to class org.opensearch.index.mapper.annotatedtext.AnnotatedTextFieldMapper$AnnotatedHighlighterAnalyzer (org.opensearch.search.fetch.subphase.highlight.UnifiedHighlighter$1 is in unnamed module of loader 'app'; org.opensearch.index.mapper.annotatedtext.AnnotatedTextFieldMapper$AnnotatedHighlighterAnalyzer is in unnamed module of loader java.net.FactoryURLClassLoader @762637be)",
      "caused_by": {
        "type": "class_cast_exception",
        "reason": "class org.opensearch.search.fetch.subphase.highlight.UnifiedHighlighter$1 cannot be cast to class org.opensearch.index.mapper.annotatedtext.AnnotatedTextFieldMapper$AnnotatedHighlighterAnalyzer (org.opensearch.search.fetch.subphase.highlight.UnifiedHighlighter$1 is in unnamed module of loader 'app'; org.opensearch.index.mapper.annotatedtext.AnnotatedTextFieldMapper$AnnotatedHighlighterAnalyzer is in unnamed module of loader java.net.FactoryURLClassLoader @762637be)"
      }
    }
  },
  "status": 500
}

This results in CWE-200: Exposure of Sensitive Information, as internal implementation details are leaked to the client, including:

• Full Java class hierarchies
• Anonymous inner class identifiers (e.g., UnifiedHighlighter$1)
• Classloader hashes and module states

Describe the solution you'd like

I have started looking into the SearchService (specifically executeQueryPhase and executeFetchPhase). Currently, these methods catch generic Exception types, log them at a trace level, and re-throw them. I haven’t yet spent too much time to dig deeper into potential ways to fix this

Question for Maintainers

Before I deep dive into a specific implementation, I would like to ask:

• Do you agree that sanitizing these "unexpected" 500 errors is a priority for project resiliency and security?
• Where do you feel the "Sanitization Gate" belongs architecturally? Should it be handled at the source (e.g., SearchService) to prevent raw exceptions from crossing the transport layer or somewhere else if it is already part of current design?

I am happy to contribute a fix, but I want to ensure the approach aligns with the project's long-term vision for exception handling

Related component

Search:Resiliency

Describe alternatives you've considered

No response

Additional context

No response

[sjs004]

@msfroh Suggested that it maybe due to some flag that is set when running the code as developer. But I checked in my another laptop where I had public docker image for OpenSearch (3.1.0) & still got the same response

@msfroh do you think that same setting is enabled for published docker image? Ideally it shouldn't be

[andrross]

I believe http.detailed_errors.enabled is the setting that controls this. The default is true but if you set it to false I believe these details will be omitted.