Late Christmas Present - GlassFish 7.1.0

[dmatej]

• Soon we release a version 7.0.20
• We will need to make a lot of maintenance in future months, it would help to reduce differences between versions.

So my idea to be discussed here is to make these changes to 7.1.0, which would be released early in 2025. I will add list of issues which would be a part of the release:

• Abandon support for Java11

  • That was required to validate TCK tests for Jakarta EE 10. That is now just in a maintenance mode and I believe we can release 7.0.x version if there would be some important change, however I don't think it would be required.
  • Jakarta EE 10 TCK can be run on JDK17 too and GlassFish passes them.

• Rename all com.sun.packages to org.glassfish.main.*

  • Mandatory to be able to continue with JPMS
  • Already existing issue Refactor com.sun.* classes to org.glassfish.main #23627

• Introduce JPMS at least to GF bootstrap, refactor dependencies to make it possible

  • Already done in Redesign of GlassFish bootstrap #25183
  • The Redesign of GlassFish bootstrap #25183 should be a safe point of these changes.
  • This phase was the hardest part, took me nearly two months.
  • I already started another part to link all related constants together and make those dependencies consistent.
  • This PR makes all other changes doable at least.

• Introduce Microprofile Health

  • Nearly done in Add Microprofile Health #25186
  • Requires JDK17+

• Logging will start with the domain, see JDK loggers can initialize the JDK LogManager before GlassFishLogManager is set #25133 (the original issue which led me to start with Redesign of GlassFish bootstrap #25183)

And I would like to know - what do you think about this?

[anbusampath]

• Abandon support for Java11

Welcome move.

[dmatej]

So, lets quickly document differences between 7.0 -> 7.1, and 8.0.0-M12 -> 8.0.0 (after #25640):

JPMS

• If you used path ${glassfish.home}/glassfish/modules/admin-cli.jar to execute asadmin commands, it will not work any more. At least not so simply, because you have to specify the classpath and modulepath on command line. Originally manifest contained classpath, however JPMS hate when something is not perfect, ie. when you have multiple jars with classpaths which contain same classes on different paths. JPMS requires perfection, so you have to ensure it.
• asadmin and asenv scripts take care of it, and also always should have been used for the communication with the server.
• The admin-cli.jar also moved from the modules directory to the glassfish directory; server doesn't use it, just asadmin.
• Unfortunately even TCK used the jar file, so I had to fix that - actually it makes things easier even for old versions; TCK classpaths contained nonexisting jars, and also added jars just to get jars from their classpaths. As you know, we do a lot of cleanup in the TCK.
• If you used gf-client.jar, I had to spend some time with refactoring, cleanup, to comply with JPMS. Java Agents don't support JPMS yet and everything gets complicated with custom classloaders and hacks GF used originally, however again, with some perfect isolation of dependencies, miracles happen and it just works - and even makes sense.

Lessons:

• JPMS x Agents don't play well together.
• JPMS will refuse to work when your packages overlap between jar files, and you cannot hack it even using reflection.
• JPMS ensures security, visibility of packages.
• You can open JDK package just to your named package, you don't have to use UNNAMED_ALL.

PKCS12

• We switched from JKS to PKCS12 as Java21 started to print warnings.
• When I started simply renaming the file in sources, I have notices quite a lot of techdebt related.
  • Default files with expired certificates
  • Codebase had several implementations of the same thing.
• Self-Signed certificates still cannot be generated in latest JDK, except using internal classes or KeyTool. So I created some adapter.
• Another TCK problem - no support for p12 files, I had to add it.

Logging

• Changes are not complete yet, however now it is already possible to define log manager by the system property as usual. However you will then lose some features depending on GJULE.

Integration with other tools

• Changes affect tools supporting GlassFish; we will slowly visit known projects and create PRs to address them.
• TCK - TBD, right now I am adopting TCK11 for GF8.
• Cargo maven plugin if I remember well?
• Arquillian Containers - see 2.x.x versions on https://github.com/OmniFish-EE/arquillian-container-glassfish - 2.1.0 is behind the door.
• ?

Asadmin arguments

• For many years asadmin prints warnings when you put ie. --terse or --passwordfile after the command while it should be between asadmin and the command. Instead of warnings it is not supported any more.

What Else?

• Watch https://github.com/eclipse-ee4j/glassfish/milestone/62 for 7.1
• Watch https://github.com/eclipse-ee4j/glassfish/milestone/54 for 8.0
• Your help is appreciated, including reporting bugs with reproducers.

[kbachl]

is there a chance that standard log4j / slf4j logging just works out of the box?
the change to JUL and the logging incompatibilities with the nearly default used logging frameworks is a nightmare so far :(

[dmatej]

Actually you can use both without problems, but you have to carefully decide which direction is the one you want.
SLF4J is very flexible, what I would use in your case these options:

1. Completely standalone logging of the application - that would not contain anything from the server, it can be based on SLF4J API which would then direct all log records to some implementation, ie. LOG4J2 or LOG4J1 ... I see below you use both which I don't think is a good solution. Also this way you should not use server.log file, but own file managed by the implementation.

2. Use server.log and JUL for the output, SLF4J API in your application. Then your application would use just this dependency, while formatting and log levels etc would be a responsibility of the server.

        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-jdk14</artifactId>
        </dependency>

If some of your dependencies need LOG4J or LOG4J2 directly, then you need to collect their logs, SLF4J will direct them to the right place.

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>log4j-over-slf4j</artifactId>
    <version>2.0.17</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-to-slf4j</artifactId>
    <version>2.25.2</version>
</dependency>

3. Switching whole server to SLF4J should be possible since 7.1.0, however I did not have time yet to try that, and it will mean some limitations like reconfiguration will not be possible without restart, asadmin commands related to logging might not work properly etc. It should be possible, but I would not recommend it.

See also:

• https://www.slf4j.org/legacy.html#log4j-over-slf4j
• https://logging.apache.org/log4j/2.3.x/log4j-slf4j-impl/

[OndroMih]

@kbachl We fixed issues with SLF4J earlier already, maybe we didn’t promote it enough. Do you still have issues with log4j/slf4j in GlassFish? Which version of GlassFish?

What’s your use case - do you want to use log4j/slf4j in an app, or you want to configure GlassFish to log via log4j instead of slf4j? The first case should be definitely possible now. The second one is a bit more complicated but should work too, although there’s no guide for it to follow.

[kbachl]

It did some cleanup and using only

<dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-jdk14</artifactId>
        </dependency>

it works. I still miss some log messages, but that maybe to lower than info level. Need to have a look here. Still wonder why it just worked out of the box in payara 5 however.

[dmatej]

I wonder what dependencies really were on Payara, the "reload" projects is not much old, originally it was old log4j before its security catastrophe. Maybe former version did not enable some part; another possibility is that Payara still uses the old logging which was in GlassFish6 and before, which was configured AFTER server initialization, so under some circumstances it could override initialization of your application.

In GF 7.0.x part of initialization is done immediately as process starts, and later when it loads its configuration it finishes the second part.
With GF 7.1.x we simplify it even more, the whole configuration is set by the JVM option just like in any other Java application, and can be reloaded automatically.

I would say that even if it would work with Payara, it is incorrect and it worked just because of Payara's issues with logging (re)configuration, which are same as in older GlassFish versions.

[kbachl]

Thanks for your help so far. I now reworked the app (Java EE 8) to use current Jakarta EE 10 and it runs with an unbelievable performance uplift of 50-125% (depending on app area). Never imagined this.
However, one thing that still keeps me wondering is why System.out.* and System.err.* (e.g: System.out.println("Test?"); ) won't show up in the log file anymore?

I tried using -Dcom.sun.enterprise.server.logging.GFRedirect=true and -Dcom.sun.enterprise.server.logging.GFSystemOut=true but to no luck?

[OndroMih]

Logging of stdout and stderr works in GlassFish out of the box. I just tried with GlassFish 7.0.25 - I added System.out.println("Hello"); into a REST resource, and I see the INFO log, both in GlassFish's standard output and in server.log:

[2025-09-29T18:40:13.388824+02:00] [GF 7.0.25] [INFO] [] [jakarta.enterprise.logging.stdout] [tid: _ThreadID=178 _ThreadName=http-listener-1(3)] [levelValue: 800] [[
  Hello]]

If I change it to System.err.println("Hello");, I get an ERROR message

[2025-09-29T18:42:41.779990+02:00] [GF 7.0.25] [SEVERE] [] [jakarta.enterprise.logging.stderr] [tid: _ThreadID=180 _ThreadName=http-listener-1(5)] [levelValue: 1000] [[
  Hello]]

There must be something wrong in your configuration.

[dmatej]

Check logging.properties, there is the option

org.glassfish.main.jul.handler.GlassFishLogHandler.redirectStandardStreams=true

If it stopped working, I guess somewhere in your code something called System.setOut and System.setErr - it could be some logging library or anything else. There is no protection of these methods (it was possible with the SecurityManager, but that will be removed from Java in future versions and also was disabled by default), so we cannot prevent it.

Maybe it could be visible who/when called it if you would set -verbose:jni on server-config.

[dmatej]

So finally we are there: https://github.com/eclipse-ee4j/glassfish/releases/tag/7.1.0
However I did not have much time to play with SLF4J, I have to catch my breath a bit and flow through these things once more to check and document what is or is not possible now.

[anbusampath]

So finally we are there: https://github.com/eclipse-ee4j/glassfish/releases/tag/7.1.0

Thank you for the magnificent work on GlassFish 7.1.0.