Add support for permissions and symlinks in os.zip/unzip with vendored zip source code from Apache Ant

Readme.adoc

@@ -1233,7 +1233,8 @@ def apply(dest: os.Path,
           includePatterns: Seq[Regex] = List(),
           preserveMtimes: Boolean = false,
           deletePatterns: Seq[Regex] = List(),
-          compressionLevel: Int = -1 /* 0-9 */): os.Path
+          compressionLevel: Int = -1, /* 0-9 */
+          followLinks: Boolean = true): os.Path
 ----
 
 The zip object provides functionality to create or modify zip archives. It supports:
@@ -1243,14 +1244,19 @@ The zip object provides functionality to create or modify zip archives. It suppo
 - Exclude Patterns (-x): You can specify files or patterns to exclude while zipping.
 - Include Patterns (-i): You can include specific files or patterns while zipping.
 - Delete Patterns (-d): You can delete specific files from an existing zip archive.
-- Configuring whether or not to preserve filesyste mtimes and permissions
+- Symbolic Links (-y): You can configure to zip symbolic links as symbolic links on Linux/Unix by setting `followLinks = false`. Symbolic links are zipped as the referenced files by default on Linux/Unix, and always on Windows.
+- Configuring whether or not to preserve filesyste mtimes.
+- Preserving Unix file permissions.
 
 This will create a new zip archive at `dest` containing `file1.txt` and everything
 inside `sources`. If `dest` already exists as a zip, the files will be appended to the
 existing zip, and any existing zip entries matching `deletePatterns` will be removed.
 
-Note that `os.zip` doesn't support creating/unpacking symlinks or filesystem permissions
-in Zip files, because the underlying `java.util.zip.Zip*Stream` doesn't support them.
+When modifying an existing zip file,
+- Unix file permissions will be preserved if Java Runtime Version >= 14.
+- If using Java Runtime Version < 14, Unix file permissions are not preserved, even for existing zip entries.
+- Symbolics links will always be stored as the referenced files.
+- Existing symbolic links stored in the zip might lose their symbolic link file type field and become broken.
 
 ===== Zipping Files and Folders
 
@@ -1375,6 +1381,8 @@ assert(paths == Seq(unzippedFolder / "File.txt"))
 This can be useful for streaming the zipped data to places which are not files:
 over the network, over a pipe, etc.
 
+File permissions will be preserved. Symbolic links will be zipped as the referenced files by default on Linux/Unix, and always on Windows. To zip them as symbolic links on Linux/Unix, set `followLinks = false`.
+
 ==== `os.unzip`
 
 ===== Unzipping Files
@@ -1384,7 +1392,7 @@ over the network, over a pipe, etc.
 os.unzip(os.Path("/path/to/archive.zip"), Some(os.Path("/path/to/destination")))
 ----
 
-This extracts the contents of `archive.zip` to the specified destination.
+This extracts the contents of `archive.zip` to the specified destination. It supports preserving file permissions and symbolic links.
 
 
 ===== Excluding Files While Unzipping
@@ -1407,7 +1415,7 @@ You can list the contents of the zip file without extracting them:
 os.unzip.list(os.Path("/path/to/archive.zip"))
 ----
 
-This will print all the file paths contained in the zip archive.
+This will print all the file paths contained in the zip archive. File permissions and symbolic links will not be preserved.
 
 ==== `os.unzip.stream`
 
@@ -1427,6 +1435,9 @@ os.unzip.stream(
 This can be useful if the zip file does not exist on disk, e.g. if it is received over the network
 or produced in-memory by application logic.
 
+File permissions and symbolic links are not supported since permissions and symlink mode are stored as external attributes which might reside in the central directory located at the end of the zip archive.
+For more a more detailed explanation see the `ZipArchiveInputStream` vs `ZipFile` section at https://commons.apache.org/proper/commons-compress/zip.html.
+
 OS-Lib also provides the `os.unzip.streamRaw` API, which is a lower level API used internally
 within `os.unzip.stream` but can also be used directly if lower-level control is necessary.
 
@@ -1464,6 +1475,8 @@ finally zipFile3.close()
 of the zip file rather than a bare path on the filesystem. Note that you need to call `ZipRoot#close()`
 when you are done with it to avoid leaking filesystem resources.
 
+File permissions are only supported for Java Runtime Version >= 14. Symbolic links are not supported. Using `os.zip.open` on a zip archive that contains symbolic links might break the links.
+
 === Filesystem Metadata
 
 ==== `os.stat`
@@ -1793,7 +1806,7 @@ is run:
 
 * `cwd`: the working directory of the subprocess
 * `env`: any additional environment variables you wish to set in the subprocess
-  in addition to those passed via `propagateEnv`. You can also set their values 
+  in addition to those passed via `propagateEnv`. You can also set their values
   to `null` to remove specific variables.
 * `stdin`: any data you wish to pass to the subprocess's standard input
 * `stdout`/`stderr`: these are ``os.Redirect``s that let you configure how the

build.mill

@@ -23,6 +23,7 @@ object Deps {
   val acyclic = ivy"com.lihaoyi:::acyclic:0.3.18"
   val jna = ivy"net.java.dev.jna:jna:5.15.0"
   val geny = ivy"com.lihaoyi::geny::1.1.1"
+  val ant = ivy"org.apache.ant:ant:1.10.15"
   val sourcecode = ivy"com.lihaoyi::sourcecode::0.4.2"
   val utest = ivy"com.lihaoyi::utest::0.8.4"
   val expecty = ivy"com.eed3si9n.expecty::expecty::0.16.0"
@@ -78,13 +79,7 @@ trait MiMaChecks extends Mima {
   )
 }
 
-trait OsLibModule
-    extends CrossScalaModule
-    with PublishModule
-    with AcyclicModule
-    with SafeDeps
-    with PlatformScalaModule { outer =>
-
+trait OsLibPublishModule extends PublishModule {
   def publishVersion = VcsVersion.vcsState().format()
   def pomSettings = PomSettings(
     description = artifactName(),
@@ -99,6 +94,14 @@ trait OsLibModule
       Developer("lihaoyi", "Li Haoyi", "https://github.com/lihaoyi")
     )
   )
+}
+
+trait OsLibModule
+    extends OsLibPublishModule
+    with CrossScalaModule
+    with AcyclicModule
+    with SafeDeps
+    with PlatformScalaModule { outer =>
 
   trait OsLibTestModule extends ScalaModule with TestModule.Utest with SafeDeps {
     def ivyDeps = Agg(Deps.utest, Deps.sourcecode)
@@ -169,6 +172,8 @@ object os extends Module {
 
   object jvm extends Cross[OsJvmModule](scalaVersions)
   trait OsJvmModule extends OsModule with MiMaChecks {
+    def moduleDeps = super.moduleDeps ++ Seq(os.zip)
+
     object test extends ScalaTests with OsLibTestModule {
       override def ivyDeps = T { super.ivyDeps() ++ Agg(Deps.expecty) }
 
@@ -194,6 +199,55 @@ object os extends Module {
     object nohometest extends ScalaTests with OsLibTestModule
   }
 
+  object zip extends JavaModule with OsLibPublishModule {
+    def apacheAntZipOriginalSource: T[PathRef] = Task(persistent = true) {
+      if (!_root_.os.exists(Task.dest / "unzipped")) {
+        val antVersion = Deps.ant.version
+        _root_.os.unzip.stream(
+          requests.get.stream(
+            s"https://repo1.maven.org/maven2/org/apache/ant/ant/$antVersion/ant-$antVersion-sources.jar"
+          ),
+          Task.dest / "unzipped"
+        )
+      }
+
+      PathRef(Task.dest / "unzipped" / "org/apache/tools/zip")
+    }
+
+    /**
+     * Shades Apache Ant
+     * [[`org.apache.tools.zip` https://ant.apache.org/manual/api/org/apache/tools/zip/package-summary.html package]] to
+     * provide Unix file permission and symbolic link support for `os.zip` and `os.unzip`
+     *
+     * A third party dependency is needed since JDK's own
+     * [[`jdk.zipfs` https://docs.oracle.com/en/java/javase/14/docs/api/jdk.zipfs/module-summary.html]] does not support
+     * symbolic links and only supports file permissions since JDK 14.
+     *
+     * Apache Ant `org.apache.tools.zip` was chosen over Apache Commons Compress due to the former not having any
+     * third party dependency, only depending on Java core libraries while the later also depends on Apache Commons IO.
+     *
+     * To avoid classpath conflicts, the dependency is shaded and compiled from source. Only the `org.apache.tools.zip`
+     * package, not the entire Ant codebase, is needed. This only adds < 100kb to Os-Lib jar size.
+     */
+    def generatedSources = T {
+      val pkg = "os.shaded_org_apache_tools_zip"
+      val zipSrc = T.dest / "os/shaded_org_apache_tools_zip"
+      _root_.os.makeDir.all(zipSrc)
+
+      // Move from "package org.apache.tools.zip" to "package os.shaded_org_apache_tools_zip"
+      // Make all classes package private (private [os]) by removing any `public` access modifier
+      _root_.os.walk.stream(apacheAntZipOriginalSource().path)
+        .filter(_.ext == "java")
+        .foreach { p =>
+          val content = _root_.os.read(p)
+            .replaceAll("org.apache.tools.zip", pkg)
+          _root_.os.write(zipSrc / p.last, content)
+        }
+
+      Seq(PathRef(T.dest))
+    }
+  }
+
   /*object native extends Cross[OsNativeModule](scalaVersions)
   trait OsNativeModule extends OsModule with ScalaNativeModule {
     def scalaNativeVersion = "0.5.2"