Bogus DNS server returns response larger than edns-outgoing-bufsize but does not indicate truncation - how to handle that

[maltris]

Hello dear community,

I found a special case where PowerDNS recursor is querying an authoritative DNS server that:

• will enable edns
• but send a response much larger than the edns-outgoing-bufsize setting indicates
• and not set any TC=1 flag

This leads to a problem because PowerDNS will SERVFAIL on that:

Error parsing packet of 1285 bytes (rd=0), out of bounds: basic_string_view::at: __pos (which is 1232) >= this->size() (which is 1232)

Seems that the UDP response will not fit the buffer. Which is okay and logical. But thats where the topic already ends for PowerDNS, because he will not retry via TCP at this point. I found no config setting to make him retry after receiving such bogus response. Which also is logical, because he never received any TC flag and therefore will not try.

I researched within RFC 7766 and RFC 8906 and found no reference where it is stated that the client MUST NOT retry querying TCP after getting a UDP response WITHOUT the TC flag. I have also read somewhere that another DNS recursor, Unbound, is actually doing exactly that. If he receives a bogus message, he will retry TCP in hopes that it works. (Or at least has this configurable.)

Needless to say, that the DNS operator at the other end should fix their DNS. But would there be a way to make PowerDNS more universal by adding an option like that? (Of course it will lead to delays in the DNS request-response cycle, but caching etc. will mitigate that mostly).

[omoerbeek]

Unbound is known to go through great efforts to get an answer, even if it is clear auths encountered are broken. PowerDNS Recursor is not as forgiving. It will try to work around some common misconfigured or malfunctioning auths cases, but to keep performance high and the code not too complex we are not handling al possible cases of broken auths.

[maltris]

Thanks for the swift answer. So as I see, this will stay like that. I am as well going to contact the other DNS operator hoping to get a fix on his behalf. I actually now found an issue dating back to January 2023 with the same questions: #12449 #10402 #9632

I investigated a little further about possible remedies to work around this as a general solution and found that because I employ dnsdist in front of the setup, since version 1.8.0, dnsdist is able to redispatch DNS queries in specific conditions to alternative upstreams. Taken from the dnsdist manual with slight adjustments:

newServer({address="127.0.0.1:5057", checkType="SOA", pool={"unboundfallback"}, checkName="google.de", mustResolve=false, maxCheckFailures=5})

function makeQueryRestartable(dq)
  -- make it possible to restart that query later
  -- by keeping a copy of the initial DNS payload around
  dq:setRestartable()
  return DNSAction.None
end
function restartOnServFail(dr)
  -- if the query was SERVFAIL and not already tried on the restarted pool
  if dr.rcode == DNSRCode.SERVFAIL and dr.pool ~= 'unboundfallback' then
    -- assign this query to a new pool
    dr.pool = 'unboundfallback'
    -- discard the received response and
    -- restart the processing of the query
    dr:restart()
  end
  return DNSResponseAction.None
end
addAction(AllRule(), LuaAction(makeQueryRestartable))
addResponseAction(RCodeRule(DNSRCode.SERVFAIL), LuaResponseAction(restartOnServFail))

This will redirect the on the first failed response to the DNS resolver running at 127.0.0.1:5057 and then (if configured) will fill its cache with the valid response encountered from the fallback. For the duration of the cache, there will be 100% of the time a valid answer and when the cache expired, the cycle will start again.

This is likely not the best solution, but it shows what dnsdist is capable of and what one could do to build a very error tolerant DNS when it is not possible to quickly get another DNS operator to fix his things.