[Bug] max_round in bootstrampfewshot is not working as described in document

[HarryWhy]

What happened?

the DSPY document says:
in Bootstrapfewshot:
Max_round:
Number of iterations to attempt generating the required bootstrap examples. If unsuccessful after max_rounds, the program ends. Defaults to 1.

so it would be like the whole train dataset will run up to max_rounds number until it collects the enough number of bootstrapped samples. (if still not enough to max round , it will stop).

but the testing shows:
in sequence, from sample1, if the metric return score is < thread hold, it will re-run this sample immediately and check the metric return score, if still < thread hold, then it will run again, up to max_rounds number; if in certain round the metric return score is > thread hold, then it will collected it as 'good' one and move to next smaple2.

so the problem is:
if max_rounds is working like this, when set it > 1, it is making the samples that are placed further back in the order have less chance to be trained.

why?
for example, if we only want 1 bootstrapped sample from sample1 and sample2:
Based on document say:
If it is working as document says, sample1 failed to pass the thread hold, then sample2 will be the next one, if sample2 also not pass thread hold. and when round 1 is over, it will start round 2, and sample1 will be trained one more time, if still not pass, then sample1 is trained one more time, it is go on and on, until up to the max rounds.
so in the case, sample2 has chance to be trained.

based on testing:
Now it is working like , try up to max_ruonds times on sample1, MAYBE the 2nd try on sample1 it passes the thread hold, then program stops, sample2 is never be trained.

Steps to reproduce

set max_rounds > 1

DSPy version

dspy==3.1.2

[vacmar01]

What you described is the intended behavior. max_rounds does mean how often the whole dataset is tried, but how often each sample is rolled out.