8364281: Reduce JNI usage in Linux attach provider

[marschall]

Reduce the usage of JNI in the Linux attach provider by making more use of the Path API.

• use the Path API to read file permissions
• use the Path API to write to and read from Unix domain sockets
• use JDK internal APIs to access the euid and eguid

After this the sole usage of JNI in the Linux attach provider will be to send a signal using kill.

I ran the tier1 and serviceability test suites and they both pass. I verified I can attach to and attach using a locally built JDK. I didn't do any Docker related tests.

I split the changes into two commits:

• The first commit ports the permission check code more or less directly from C to Java using integer based checks.
• The second commit changes the permission check code to use higher level Java abstractions (UserPrincipal, Set, ...)

I hope this makes it easier to review and verify the changes preserve the semantics.

To the best of my knowledge uid_t is an unsigned 32 bit integer. VM#geteuid() returns it as long, unsigned. However UnixUserPrincipals#fromUid expects an int, I believe casting is the correct way to convert in this case.

I tried to keep unrelated changes to VirtualMachineImpl to a minimum. I did however replace all usages of java.io.File with java.nio.file.Path and made two methods static. We could get rid of the socket_path instance variable if we instead use socket_address to flag a disconnect.

Ultimately the same changes can be applied to the macOS an AIX implementations.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8364281: Reduce JNI usage in Linux attach provider (Enhancement - P4)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/26712/head:pull/26712
$ git checkout pull/26712

Update a local copy of the PR:
$ git checkout pull/26712
$ git pull https://git.openjdk.org/jdk.git pull/26712/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 26712

View PR using the GUI difftool:
$ git pr show -t 26712

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/26712.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back marschall! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

@marschall The following label will be automatically applied to this pull request:

• serviceability

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 00: Full (80b045ac)

[slovdahl]

Thanks for the work so far, nice to get rid of JNI. 🙂

There is a problemlisted test related to this that could make sense to run manually (https://bugs.openjdk.org/browse/JDK-8341518, it's failing in Oracle's CI but I have not been able to reproduce the failure myself, discussed a bit in #21331 too).

I'll see if I can find time to check out this change locally and run some tests, otherwise, feel free to run it yourself.