Java 11 Compatibility

Date	2018 09 11	Contacts	María Arias de Reyna
Status	Research	Release	GeoNetwork 4.0
Resources	Initial research undertaken by GeoCat, expect additional activities to be required across the GeoNetwork community	Ticket #	OSGeo Java codesprint 2018
Source code	n/a
Funding	GeoCat providing initial assessment on behalf of GeoNetwork Enterprise product

Overview

The Java ecosystem is now lead by OpenJDK and has changed release cadence (two numbered release each year, with long-term-support releases every three years).

With this in mind there are two long-term-support releases:

Java 8 (LTS): 2014-2026 AdoptOpenJDK
Java 11 (LTS): 2018-2024 AdoptOpenJDK
Java 18 (LTS): 2021-TBA

Aside: We were initially concerned that Java 8 would no longer be supported, but an industry response lead by Red Hat, Eclipse Foundation and others is continuing to make OpenJDK 8 available to their customers and for Windows and macOS at http://adoptopenjdk.net.

Technical Details

2018 Initial Research

María Arias de Reyna participated in OSGeo Java 2018 Code Sprint:

OSGeo-Java-codesprint-2018

Results:

Naive Attempt: Run with maven/jetty
- As expected Spring fails to load
- Warnings on use of reflection "All illegal access operations will be denied in a future release"
Build with maven
- Missing javax classes (and probably even more missing/moved core JDK classes
- Warnings on use of reflection "All illegal access operations will be denied in a future release"

Requirements:

Replace Spring with latest version: from 3.2.0 to 5
Locate and replace all imports referencing missing JDK API
Replace other dependencies that may be struggling with JDK 11
Check all reflection

Assessment:

A new release of GeoTools is needed: Java 9 introduced a restriction on Service Registry (used by the GeoTools library to coordinate plugins) a new release of GeoTools will be required before GeoNetwork can run on Java 11
With some care releases of GeoNetwork should be compatible with both Java 8 and Java 11.

2020 Follow-up Research

Progress:

As part of https://github.com/geonetwork/core-geonetwork/wiki/Bolsena-2020:

Use of jdeps to short-list dependencies to replace (dreaded split-package problem)

Dependencies

For each dependency we need to look at how it is failing in Java 11, and if a new release is available to address the problem. As an example we know that an upgrade to Spring 5 is required for Java 11 compatibility.

Dependencies have some common failings:

Use of some internal com.sun classes or API.

This can occur when a dependency is using reflection to access a codec or other api directly for greater control, rather than making use of a more limited public api.

When running often only the first warning is reported, use --illegal-access=warn to report more.

Mitigation: Check if an official API is now available, this is often the case for reflection on the Unsafe class for improved memory management control. As an alternative look for an open source project that performs a similar function.
Unable to access a javax package that is expected to be included in the Java Runtime.

This can occur when a java runtime module is not turned on during startup (we can adjust this with JVM startup parameters for jetty or tomcat).

More seriously this can occur when the API is no longer part of Java, either because Java Enterprise Edition content has now moved to the Eclipse Foundation Jakarta project, or because functionality has been dropped from the Java ecosystem completely.

Example: java.xml.bind is no longer included in Java 11, and must be added as a dependency:
```
      <dependency>
          <groupId>javax.xml.bind</groupId>
          <artifactId>jaxb-api</artifactId>
          <version>${jaxb.api.version}</version>
      </dependency>
```
Reference: https://www.dariawan.com/tutorials/java/using-jaxb-java-11/
Split package error preventing the jar from being loaded

This occurs when a jar tries to publish a class into a package that is already used by another jar. As an example many of our xml libraries try to create additional classes in javax.xml package and fail to load as a result. Mitigation: Ideally upgrade to a newer official version, or if we have to recompile our own fork (refactoring the code to use a unique package).

Key dependency upgrades:

Migrate to GeoTools 23.0:addresses the GeoTools blocker found in 2018 activity
Upgrade to Spring 5
Upgrade for JPA 2

Dependency audit:

ant-1.6.5.jar -> JDK removed internal API

org.apache.tools.ant.taskdefs.email.UUMailer       -> sun.misc.UUEncoder                                 JDK internal API (JDK removed internal API)

batik-ext-1.10.jar

split package: org.w3c.dom.events [jrt:/java.xml, batik-ext-1.10.jar]

camel-core-2.14.4.jar -> jdk.unsupported

org.apache.camel.com.googlecode.concurrentlinkedhashmap.ConcurrentHashMapV8 -> sun.misc.Unsafe                                    JDK internal API (jdk.unsupported)
org.apache.camel.com.googlecode.concurrentlinkedhashmap.ConcurrentHashMapV8$1 -> sun.misc.Unsafe                                    JDK internal API (jdk.unsupported)
org.apache.camel.com.googlecode.concurrentlinkedhashmap.ConcurrentHashMapV8$TreeBin -> sun.misc.Unsafe                                    JDK internal API (jdk.unsupported)

core-3.11.0-SNAPSHOT.jar -> java.base

org.fao.geonet.kernel.url.UrlChecker               -> sun.net.ftp.FtpLoginException                      JDK internal API (java.base)

dom4j-1.6.1.jar -> JDK removed internal API
   org.dom4j.datatype.DatatypeAttribute               -> org.relaxng.datatype.DatatypeException             JDK internal API (JDK removed internal API)
   org.dom4j.datatype.DatatypeAttribute               -> org.relaxng.datatype.ValidationContext             JDK internal API (JDK removed internal API)
   org.dom4j.datatype.DatatypeElement                 -> org.relaxng.datatype.DatatypeException             JDK internal API (JDK removed internal API)
   org.dom4j.datatype.DatatypeElement                 -> org.relaxng.datatype.ValidationContext             JDK internal API (JDK removed internal API)
   org.dom4j.datatype.SchemaParser                    -> org.relaxng.datatype.DatatypeException             JDK internal API (JDK removed internal API)
   org.dom4j.datatype.SchemaParser                    -> org.relaxng.datatype.ValidationContext             JDK internal API (JDK removed internal API)

ehcache-core-2.6.2.jar -> jdk.unsupported
   net.sf.ehcache.pool.sizeof.UnsafeSizeOf            -> sun.misc.Unsafe                                    JDK internal API (jdk.unsupported)

split package: javax.transaction [jrt:/java.transaction, geronimo-jta_1.0.1B_spec-1.0.1.jar]

split package: javax.transaction.xa [jrt:/java.sql, geronimo-jta_1.0.1B_spec-1.0.1.jar]

groovy-all-2.3.11.jar -> java.desktop
groovy-all-2.3.11.jar -> jdk.unsupported
   groovy.json.internal.FastStringUtils               -> sun.misc.Unsafe                                    JDK internal API (jdk.unsupported)
   groovy.json.internal.FastStringUtils$StringImplementation$1 -> sun.misc.Unsafe                                    JDK internal API (jdk.unsupported)
   groovy.json.internal.FastStringUtils$StringImplementation$2 -> sun.misc.Unsafe                                    JDK internal API (jdk.unsupported)
   groovy.ui.ConsoleApplet                            -> java.awt.peer.ComponentPeer                        JDK internal API (java.desktop)
   groovy.ui.ConsoleApplet                            -> sun.awt.CausedFocusEvent$Cause                     JDK internal API (java.desktop)
   groovy.ui.ConsoleApplet                            -> sun.java2d.pipe.Region                             JDK internal API (java.desktop)
guava-18.0.jar -> jdk.unsupported
   com.google.common.cache.Striped64                  -> sun.misc.Unsafe                                    JDK internal API (jdk.unsupported)
   com.google.common.cache.Striped64$1                -> sun.misc.Unsafe                                    JDK internal API (jdk.unsupported)
   com.google.common.cache.Striped64$Cell             -> sun.misc.Unsafe                                    JDK internal API (jdk.unsupported)
   com.google.common.primitives.UnsignedBytes$LexicographicalComparatorHolder$UnsafeComparator -> sun.misc.Unsafe                                    JDK internal API (jdk.unsupported)
   com.google.common.primitives.UnsignedBytes$LexicographicalComparatorHolder$UnsafeComparator$1 -> sun.misc.Unsafe                                    JDK internal API (jdk.unsupported)

hadoop-core-1.0.0.jar -> java.base
hadoop-core-1.0.0.jar -> java.security.jgss
   org.apache.hadoop.net.NetUtils$QualifiedHostResolver -> sun.net.dns.ResolverConfiguration                  JDK internal API (java.base)
   org.apache.hadoop.net.NetUtils$QualifiedHostResolver -> sun.net.util.IPAddressUtil                         JDK internal API (java.base)
   org.apache.hadoop.security.KerberosName            -> sun.security.krb5.Config                           JDK internal API (java.security.jgss)
   org.apache.hadoop.security.KerberosName            -> sun.security.krb5.KrbException                     JDK internal API (java.security.jgss)
   org.apache.hadoop.security.SecurityUtil            -> sun.security.jgss.krb5.Krb5Util                    JDK internal API (java.security.jgss)
   org.apache.hadoop.security.SecurityUtil            -> sun.security.krb5.Credentials                      JDK internal API (java.security.jgss)
   org.apache.hadoop.security.SecurityUtil            -> sun.security.krb5.PrincipalName                    JDK internal API (java.security.jgss)
   org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1 -> sun.security.jgss.GSSUtil                          JDK internal API (java.security.jgss)

Dependencies to review from jdeps audit:

Java 8: fails due to a bug in jdeps

jdeps -apionly -R . *.jar
Exception in thread "main" java.lang.InternalError: Missing message: warn.skipped.entry

Java 9: OpenJDK 9 and Oracle JDK 9.0.4

jdeps --jdk-internals -R *.jar
Exception in thread "main" java.lang.module.FindException: Module com.sun.istack.runtime not found, required by com.sun.xml.bind

Go through jars in batches so we can figure out what is wrong: