Performance Measurement

[JMMackenzie]

Once the first prototype is running, explore large-scale testing to see where time is being lost and optimize if/where possible.

[JMMackenzie]

MSMARCO dev (~7k topics) with depth = 1000 runs takes 20 seconds to compute RBP scores.

The read of the file takes 10 seconds itself; the computation takes another 10 seconds.

We need to optimize!

[JMMackenzie]

A bit of quick profiling shows that our read and to_rbranking_dict are taking a lot of cycles.

The command here was RBA on 1000 element runs, with about 7000 queries.

[mpetri]

is this still true

[JMMackenzie]

I believe so, yeah -- do you want me to run more profiling?