DirectoryNotEmptyException on mill __.clean

[Sailsman63]

Not sure if this is an issue with mill, os-lib, or a quirk of my particular system.

When I run mill __.clean I get:

1 targets failed
clean java.nio.file.DirectoryNotEmptyException: /mnt/develop/Shared/Source/SPManager/out/clean
    java.base/sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:247)
    java.base/sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:105)
    java.base/java.nio.file.Files.delete(Files.java:1152)
    os.remove$all$.apply(FileOps.scala:296)
    os.remove$all$.apply(FileOps.scala:287)
    scala.collection.immutable.List.foreach(List.scala:333)
    mill.main.MainModule.$anonfun$clean$1(MainModule.scala:276)
    mill.define.Applicative$Applyer.$anonfun$zipMap$1(Applicative.scala:35)
    mill.define.Task$MappedDest.evaluate(Task.scala:396)

Notes:

• mill version 0.9.2 0.9.12
• Duplicated with several JDKs
• The listed directory is, indeed, empty when I check using ls -la (shows hidden files)
• The module cache folders have been removed correctly

[lefou]

Which platform?
Mill 0.9.2 isn't even a proper release. Use at least 0.9.3, better 0.9.12
One wild guess. You're on Windows and there was a file present, which was still open, but which got deleted later, probably on JVM shutdown.

[lefou]

Can you tell which NTFS driver do you use? My suspicion is, that it's some race condition. I can't tell if NTFS needs to lock files on Linux too or if that is a Windows-only issue.

Is this issue reproducible? Does a second clean task also fail? Do you have similar issues when working with other tools on this partiion?

[Sailsman63]

The default for Arch >= 5.15, which is the open-source version of paragon's ntfs3 driver

It is reproducible in the sense that it is consistently repeatable, even after system reboots and swapping out JDK versions. I have not tried a second mill project yet. Yes, two calls to __.clean in a row will both fail. I have not noticed other, similar, issues; but I have also only used a few tools widely and consistently, so I might have missed something like that.

This might well be a timing issue on my end - the drive is old, and running on a hardware adapter.

[lefou]

I think this probably is an OS-Lib issue or even reproducible with pure Java. Unfortunately, without any way to reproduce it, it's not a very actionable issue (for others). So you might try to isolate it into a small repro example. The essential logic is in MainModule.clean: we collect all dirs to remove and then simply run a paths.foreach(os.remove.all) on it. You might want to try it with Mill 0.10.x first, as we changed the out dir layout and with it also the clean logic.
One other thought is, in case it is always failing on the out/clean dir, that we probably have an issue with the clean target logging into a file inside out/clean (on Mill 0.9.x). But this should be an issue then under Windows too. I need to check whether we have an test case for that, which runs on CI in Windows.

[Sailsman63]

Okay, finally was able to upgrade my system to mill 0.10.0, and that seems to have fixed it.

Side notes:

• This was specific to the ntfs partition. When I copied the project folder to an ext4 partition, the 0.9.x mill worked correctly.
• I started playing around with my command line tools, and found that the lock also blocked rm -rf. The locked files were not actually in out/clean, instead there were a series of files out/mill-worker-P5UGrv/9mNp4fHPROIxE5GHJMRU\=-1/.fuse_hidden<HASHCODE> that were owned by a live java/mill process. Killing the process allowed me to rm -rf

[lefou]

Thanks for the update. Please note, that mill clean does not try to remove these out/mill-worker-* folders, as these contain socket to communicate with Mill server processes, so you seeing a different issue of locked files, but in the end, both issues result in a file system that locks resources and probably does not respond in a timely manner. We may move the location for temporary mill-worker directories in a future release or make it configurable.