coverage: 97.792% (+0.01%) from 97.78%
when pulling 92ba787 on agologan:main
into cf6ab33 on cucumber:main.

From looking at the code, I coudl not tell how the pickles are sorted. If this is a random sort, based on how they are read from the file system, this will lead to errors. You must sort all pickles in a unique way. As a side note, since you use modulo, sorting by file size and then alphabetically will give better performance compared to alphabetically only.

@Tallyb can you elaborate on what errors you are foreseeing? Are you able to reproduce them in a sample project?

This PR doesn't deal with any sorting - that is handled in a later phase, after filtering.

let's take a simple example:
Assume we have tests A,B,C,D, E, F, G and we run them on shard = 3.
Each shard is running in a different machine, so it is completely unaware of what is happening on the other machines.

Machine 1: tests are read in the following order - A, B, C, D, E, F, G. tests taken - A, D, G. (positions: 1, 4, 7)
Machine 2: tests are read in the following order - B, A, C, D, E, F, G. tests taken: A, E, (2, 5)
Machine 2: tests are read in the following order - B, C, A, D, E, F, G. tests taken: A, F, (3, 6)
As you can see, A is tested 3 times, while B and C are not tested at all. to make sure everything is tested, you must guarantee the same order in which tests are being read.

The code in all instances does 3 things in this order:

• discovers paths
• filters the paths
• applies ordering (if not default: defined)

Can see this is load_sources.ts or run_cucumber.ts

Sharding is applied after discovery but before further ordering.

So discovery is responsible for the defined order and that can be seen in paths.ts

Getting back to your example, all 3 machines run the exact same code to discover
the paths which as the docs put it "roughly means alphabetical order of file path
followed by sequential order within each file".

So all 3 machines will discover the tests in an order which roughly resembles
A, B, C, D, E, F, G order, and is the same across them.

Then all 3 machines will apply the same filtering logic, which would leave them
with the same sequence, to which sharding is applied.

Picking nth test on each shard will provide uniform distribution of all tests afterwards.

As a last step if a random order is used, those left on each shard are reshuffled.

Thanks for laying that out @agologan.

As you say, the initial order of sources and pickles is deterministic given the same configuration, so I don't believe we have a problem here.

Ok, as long as it is deterministic, there is no issue.
You can use my tip for improving performance by sorting by size (assuming size ~= execution time).