aws s3 sync from S3 -> Windows issue when directory casing does not match

[mike406]

Describe the bug

Hello, I am looking to report an issue I am seeing with running aws s3 sync. I am trying to sync an S3 bucket to an FSx SMB share mounted as a drive on a Windows EC2 instance. I am noticing that some folders in the destination FSx share are missing files that are present in S3 and they are not being downloaded. Nothing about these files are strange, they are PDFs and .docx files. Any thoughts on why files present in S3 are being ignored and not downloaded? If I sync the exact subfolder specifically, then they do get downloaded.

The full command I am using is aws s3 sync s3://{my-bucket} \\share\path --delete

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

I expect all files present in S3 to be downloaded to the FSx share if they are not present there.

Current Behavior

Certain files are not being downloaded so some folders have less files than they should.

Reproduction Steps

See my comment below - I did end up figuring out how to reproduce the issue after some testing.

Possible Solution

No response

Additional Information/Context

No response

CLI version used

2.27.48.0

Environment details (OS name and version, etc.)

Windows Server 2019

[mike406]

Quick followup, is it possible it is a casing issue with Windows?

For example, the path in the source S3 bucket currently is /folder1/FOLDER2/folder3/
And the path in the destination Windows FSx share is /folder1/folder2/folder3/

It seems that at some point the casing for one of the folders was changed in the S3 bucket and this does not get updated at the destination in future syncs. Is this possible this is the cause and is there any way to work around this to ensure that all files get downloaded?

Edit: Yes this appears to be the cause I am able to replicate it now - if there is a casing mismatch with directories on Windows and S3 and you are also using the --delete parameter it deletes all files at the destination even if those files are supposed to be there and downloads files at the same time which looks like it is resulting in some race condition where some files are missing at when it finishes.

[RyanFitzSimmonsAK]

Hi @mike406, thank you for reaching out. I was able to reproduce the behavior you observed. You're correct that it's related to a casing mismatch between the S3 prefix and the local directory name. We were also able to reproduce this on macOS. I've brought this issue up to the team to address.

Regarding the race condition, I was not able to reproduce that. In my testing, deletes always took place before downloads. Could you provide debug logs of downloads taking place before deletes? You can get debug logs by adding --debug to your command, and redacting any sensitive information. Thanks!