Skip to content

Conversation

@sake92
Copy link
Collaborator

@sake92 sake92 commented Mar 6, 2025

Seems like we can open the ZIP after it is created, and then add the permissions to the ZIP entries.
See the docs, it needs to be done through java's zip filesystem https://docs.oracle.com/en/java/javase/17/docs/api/jdk.zipfs/module-summary.html

Same with unzip, we extract the files and then dig into the ZIP via zip filesystem to fetch the perms.

Seems like it works. ¯\(ツ)

@sake92 sake92 marked this pull request as ready for review March 6, 2025 00:57
@sake92
Copy link
Collaborator Author

sake92 commented Mar 6, 2025

I've tested it locally with the cosmocc.zip from the linked issue in mill, seems to work fine (executables):

.......
-rwxrwxr-x 1 sake sake    2461 Mar  6 02:15 cosmoar*
-rwxrwxrwx 1 sake sake       7 Mar  6 02:15 cosmoc++*
-rwxrwxr-x 1 sake sake   18647 Mar  6 02:15 cosmocc*
-rwxrwxr-x 1 sake sake    8667 Mar  6 02:15 cosmocross*
-rwxrwxr-x 1 sake sake    2418 Mar  6 02:15 cosmoinstall*
-rw-rw-r-- 1 sake sake      46 Mar  6 02:15 cosmoranlib

@sake92 sake92 requested a review from lihaoyi March 6, 2025 01:17
val relPath = path.subRelativeTo(dest).toString
if (os.zip.shouldInclude(relPath, excludePatterns, includePatterns)) {
val entry = zipFS.getPath(relPath)
val permissions = Files.getPosixFilePermissions(entry)
Copy link
Contributor

@kiendang kiendang Mar 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should use the more low-level Files.getAttribute(entry, "zip:permissions") here.
In case the zip entry does not have permission information, Files.getAttribute(entry, "zip:permissions") would be empty and we could skip os.perms.set.

If using Files.getPosixFilePermissions, according to the doc,

The "permissions" attribute is not optional in the "posix" view so a default set of permissions are used for entries that do not have access permissions stored in the Zip file. The default set of permissions are
OWNER_READ
OWNER_WRITE
GROUP_READ

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the unzipped file would get some default permissions even if we dont set them, right?
With this approach we at least can set it with the "defaultPermissions" property.
Currently I set it to empty

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When there's no permission information stored with the zip entry, we want it to have the default permissions set by the user's os/file system/umask when unzipped right, not the zipfs "defaultPermissions"? Also the zipfs "defaultPermissions" is never empty. When it's null/unset it's set to [OWNER_READ, OWNER_WRITE, GROUP_READ]. So we should check with Files.getAttribute(entry, "zip:permissions") instead, and skip the os.perms.set part if it's empty.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you are right, I'm not super familiar with *nix permissions.
Fixed now, looks good?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I didn't have a correct understanding of umask either. There're no default permissions in Unix. Applications set permissions to files they create. Umask will then mask some of the permissions. What you did before was totally fine. https://askubuntu.com/questions/44542/what-is-umask-and-how-does-it-work

@sake92
Copy link
Collaborator Author

sake92 commented Mar 7, 2025

Closing based on discussion in #356 (comment)
Seems like it's best to have a separate artifact for zipping.

@sake92 sake92 closed this Mar 7, 2025
lihaoyi pushed a commit that referenced this pull request Apr 18, 2025
…d zip source code from Apache Ant (#374)

### Vendored code

Follow the discussion in #356, this adds support for permissions and
symlinks in `os.zip`/`unzip` with vendored zip source code from Apache
Ant.

The vendored source code is generated by the `os.zip.apacheAntZipSource`
task and put in `os/zip`. It's shaded with the package renamed from
`org.apache.tools.zip` to `os.shaded_org_apache_tools_zip`.

`scala-steward.conf` was added and configured to run
`os.zip.apacheAntZipSource` on `org.apache.ant:ant` updates.

### Features

This brings support for permissions and symlinks to `zip` (for creating
new zips, not modifying existing ones), `zip.stream` and `unzip`. As for
modifying existing zips, we would still have to rely on `jdk.zipfs`
which does not support symlinks.

| | file permissions | symlinks |
| --- | --- | --- |
| `os.zip.open` | if Java Runtime Version >= 14 | |
| `os.zip` (create new) | ✅ | ✅ |
| `os.zip` (modify existing) | if Java Runtime Version >= 14 | |
| `os.zip.stream` | ✅ | ✅ |
| `os.unzip` | ✅ | ✅ |
| `os.unzip.stream` | | |

### TODO
- [ ] **(Advice needed)** make sure we comply with Apache Ant's license
to include the code here. Would appreciate opinions on this as I'm not
an expert.
- [ ] **(Advice needed)** make `ZipOps` JVM only
- [x] tests
- [x] make sure things don't break on Windows
- [x] add permission support to modifying existing zips with `jdk.zipfs`
like what @sake92 did in #371
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants