Bring Trajectory Analysis into a key innovation part - Failure Mode Analysis

[DhavalRepo18]

Currently, the trajectory analysis feature is independent and utilises the past structures of the AgentHive and MetaAgent.

What we ideally like to do.

A separate core module for FM discovery
3. Build an independent component (with some default)

https://github.com/IBM/AssetOpsBench/tree/main/src/TrajFM

We will remove the src folder and bring into a separate module like

core/trajectroy_analysis/failure_mode?

[DhavalRepo18]

@ShuxinLin I tagged as you can suggest while we restructure where to put the code.

[bradleyjeck]

It would be great to integrate the trajectory analysis with the scenario-server based evals and mlflow. Maybe even using trajectories from mlflow as an input format.

[DhavalRepo18]

We want to decouple a core module first and then it will get the data for analysis from multiple place. MLflow now has an MCP Server to retrieve the trace, so it can also be leveraged more actively.

Ideally, this PR will rework the above https://github.com/IBM/AssetOpsBench/tree/main/src/TrajFM and then produce an executable codebase for trajectory analysis. The trajectory can be stored either locally or dynamically extracted from the MLflow tracking server.

[DhavalRepo18]

@ShuxinLin, where do we expect to put the refactored code? What shall be the directory structure?

Any suggestions?
src/trajectory_analysis
src/trajectroy_analysis/failure_mode

Also, @nianjunz, we will not use ReactXen for LLM dependency; we have LLM here in src/llm.