allow flush training data to rmove stale data

[kaushikmitr]

This pull request adds a comprehensive flush API to the training server, enabling manual clearing of training data, test data, and metrics, along with corresponding status and error handling endpoints. The test suite is expanded to cover these new endpoints and behaviors.

Training Server: Flush API and Data Status

• Implemented a /flush endpoint with a FlushRequest/FlushResponse model, allowing manual clearing of training data, test data, and metrics, with detailed reporting and error handling.
• Added a /data/status endpoint to report current counts of training/test samples, metrics, and bucket distribution. (latencypredictor-v1/training_server.py)

Testing Enhancements

• Expanded the test suite to cover the flush API, its error handling, and the data status endpoint, ensuring correct behavior for all flush scenarios. (latencypredictor-v1/test_dual_server_client.py)
• Updated prediction and training server URLs to use placeholder IPs for easier configuration. (latencypredictor-v1/test_dual_server_client.py)

SLO Scorer: Bug Fix

• Updated PodPredictionResult to include a PrefixCacheScore field for use to correctly use prefix cache score logic.

[kaushikmitr]

@kfswain @BenjaminBraunDev @ahg-g

[kaushikmitr]

/retest

[ahg-g]

/approve
/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, kaushikmitr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ahg-g]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment