Optimization suggestion

[Emjayen]

For the regular overlapped I/O path, even at low data-rates (<= 40G), you can achieve considerable performance improvements by using Direct I/O, which for sockets is accomplished by way of disabling the AFD-level buffering (SO_SNDBUF=0), eliding a copy.

This would mostly be only appropriate for the server-side due to the bulk-data transfer, which amortizes the overhead of direct I/O. Chatty traffic is better off served by the AFD buffering.

One caveat is that this will, due to the I/O system's architecture, always result in the IRP being pended (ie., STATUS_PENDING reported), however, outside of it being truly pended due QoS/tx ring space/ect, it'll complete by the time you inspect it's status in the ISB effective immediately.

With that in mind, to achieve the same benefit as FILE_SKIP_COMPLETION_PORT_ON_SUCCESS one should suppress enqueuing a port packet for transmits and resort to polling instead, firstly immediately after the I/O submission (ie., WSASend) then at some other opportune time later (probably opportunistically during a wakeup)

[guhetier]

Hi @Emjayen, to understand, are you facing a scenario were Direct I/O produces a measurable improvement?