How are upstream servers selected?

[timkgh]

I left the default Load-balancing setting enabled and I'm seeing something that doesn't add up. I use DoH 9.9.9.9 and 1.1.1.2 as upstreams. From ping and dig tests, 1.1.1.2 should be a lot faster, <10ms query response time vs 20ms for 9.9.9.9
But if I grep querylog.json, 9.9.9.9 is preferred, 40K queries went to it vs only 4.5K queries to 1.1.1.2

Is there a way to set strict order? i.e. use 1.1.1.2 for all queries, fall back to 9.9.9.9 only if 1.1.1.2 is not available and then switch back again when available.

[ameshkov]

There's no simple way to do that right now. However, using "parallel queries" would probably suit you. If 1.1.1.2 responds faster, its response will be used most of the time. If it fails to respond, 9.9.9.9 will be used instead (unless it also fails).

[timkgh]

I tried parallel queries and indeed 1.1.1.2 is used a lot more than 9.9.9.9 now, 3:1 ratio.

It appears there is a bug in the "load balancing" selection, or maybe the metric that determines what fast means needs to be adjusted?

[ameshkov]

The only explanation that comes to my mind is that 9.9.9.9 responds to a small share of queries MUCH faster than 1.1.1.2 and this brings the overall average RTT for it to a smaller value.

[timkgh]

@ameshkov This is strange. I selected a different pair of DNS servers and used the "load balancing" setting. Again, the slower of the two seems to be preferred by a large margin. Are you sure the logic that is supposed to select the fastest server is not inverted? lol

[timkgh]

I think the problem here might be with DoT and DoH. If the RTT of a server is checked only occasionally the connection might close and then it's expensive to redo the TLS handshake, hence the server might appear slow when it would otherwise be fast if the connection were kept open as long as possible.