Skip to content

Commit 3ea40e7

Browse files
committed
improvements to metrics migration plan
1 parent c093afd commit 3ea40e7

File tree

3 files changed

+201
-335
lines changed

3 files changed

+201
-335
lines changed

tests/e2e/metrics_migration/metric_score_diff.ipynb

Lines changed: 197 additions & 174 deletions
Large diffs are not rendered by default.

tests/e2e/metrics_migration/plan-for-metrics-migration.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -268,7 +268,7 @@ __all__ = [
268268

269269
### 2.2 Generate Metric Configuration
270270

271-
The only thing you need to create is the `METRIC_CONFIG` dictionary for Cell 2 of the notebook. Use the template below based on your metric type:
271+
Generate the `METRIC_CONFIG` dictionary for Cell 2 of the notebook. Print it to console for easy copy-pasting. Use the template below based on your metric type:
272272

273273
#### Configuration Template
274274

@@ -426,14 +426,15 @@ The `dataset_fields` list tells the notebook which fields to extract from the te
426426
- Adjust your metric implementation
427427
- Re-run the notebook to verify improvements
428428

429-
6. **Document findings**: Note the following for your E2E tests:
429+
6. **Document findings**: Print a migration summary with the following information:
430430
- Mean absolute difference
431431
- Percentage of samples within tolerance
432432
- Recommended tolerance level
433433
- Any patterns or anomalies observed
434434
- Edge cases that need special handling
435+
- Key implementation details and algorithm differences
435436

436-
**No files are saved** - all results are displayed inline for quick validation!
437+
**Output approach**: Print the METRIC_CONFIG and migration summary directly to console/output instead of creating files. This allows for easy copy-pasting without cluttering the repository.
437438

438439
---
439440

tests/e2e/metrics_migration/test_context_recall_migration.py

Lines changed: 0 additions & 158 deletions
This file was deleted.

0 commit comments

Comments
 (0)