Concurrent atomic data writes on Windows can fail with `ERROR_ACCESS_DENIED`

[jmschonfeld]

Describe the bug
With multiple concurrent atomic writes to the same file on Windows, some of the renames (from the temporary file to the final destination) via SetFileInformationByHandle will fail with ERROR_ACCESS_DENIED. Currently, it's unclear exactly why this is failing even with the more robust usage of SetFileInformationByHandle which is supposed to be an atomic operation like rename on POSIX systems. We had originally believed it may be because the system marks the destination as pending deletion before performing the rename, however when hitting ERROR_ACCESS_DENIED, RtlGetLastNTstatus does not show the ERROR_DELETE_PENDING code. We had also investigated whether a simple retry mechanism (retry upon ERROR_ACCESS_DENIED up to some max N retries) would work, however that did not seem to resolve the issue either.

We have found one change which does resolve the symptoms, but it does not seem correct for all circumstances and likely merits more investigation into the root cause to confirm it is the right approach.

To Reproduce

import Dispatch
import Foundation

let group = DispatchGroup()
let queue =
    DispatchQueue(label: "org.swift.sr83606.queue", attributes: .concurrent)

let path = URL(fileURLWithPath: "file.txt")
let contents = "text".data(using: .utf8)!

let count = Int(CommandLine.arguments.last ?? "1") ?? 1
for id in 0..<count {
  group.enter()
  queue.async {
    defer { group.leave() }
    do {
      try contents.write(to: path, options: [.atomic])
    } catch {
      print("Attempt \(id) failed: \(error)")
    }
  }
}
group.wait()

Note: we found that converting the above to swift concurrency in a swift-testing test did not reproduce the issue.

Expected behavior
Multiple concurrent atomic writes to the same file should not result in failures, instead each should succeed with the result being an atomic write.

Configuration (please complete the following information):

• Swift Version: main and 6.2
• OS: Windows

Regression information:
Likely since at least 6.0 from the Foundation re-core

Additional context
This is the root cause of the issue described in swiftlang/swift#83606 in which atomic writes of response files by the swift driver can fail on 6.2 resulting in failed compilation in SwiftPM when using response files on Windows.