-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Description
I think some of the failing tests might actually be wrong.
For example, consider:
rbstar/rbstar/tests/test_rb_metrics.py
Line 36 in 19d3552
| assert result.lower_bound == pytest.approx(0.744, rel=1e-3), f"Lower bound {result.lower_bound} does not match expected value 0.744" |
This test is computing RBP with phi = 0.8 on the ranking [ [1], [2], [3], ..., [25] ]with positive docs [1] and [2] and negative document [3].
The lower-bound RBP score should thus be:
((1 - 0.8 ) / 0.8 ) * (0.8 ^ 1 + 0.8 ^ 2) = 0.360
since docs [1] and [2] are both relevant at ranks 1 and 2.
Could someone please verify? We'll need to check the other tests carefully too.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels