Migrate to newer Java version (LTS JDK/JRE/JVM)

[abyrd]

I'm intentionally not stating a specific version in the title as the version numbers may change over time each time we search for or think about this. Anywhere in this ticket that I say JDK I actually mean JDK/JRE/JVM.

There is a pre-existing draft PR #760 where I tested out migration to the most recent LTS release 17. The initial sticking point here was the Java module system, which is a bit of a mess to deal with, requires replicating most of the dependency information from the build, and doesn't bring us any immediate benefits. There seem to be ways to work around this.

Like many other things, moving to a new JDK version will not play well with backward compatibility of worker versions. Despite the "write once run anywhere" concept of Java, older workers will likely need to run on the JDK they've run on in the past - we've always run into problems (e.g. Java 9 module enforcement) when we try to run them on newer JDKs. We might have to stop allowing old worker versions, stick to only one worker version, or conditionally install three different JDKs on the worker machines based on which worker version is selected. Or perhaps with any luck we can get the old worker JARs running on new JDKs using switches to turn off the module enforcement.

Here are the relevant Java releases:

• JDK 17 September 2021 (current LTS)
• JDK 19 September 2022 (current short-term)
• JDK 20 March 2023 (next short-term)
• JDK 21 September 2023 (next LTS, could bring major new features)

Many features that are particularly important for data-crunching software like R5 seem to have been moved from JDK 20 to JDK 21. Here are the pages listing features in JDK 21:
https://openjdk.org/projects/jdk/21/spec/
https://bugs.openjdk.org/secure/Dashboard.jspa?selectPageId=21418

Some of the really interesting stuff is:
Value Objects (all final fields and no references): https://openjdk.org/jeps/8277163
Primitive Classes: https://bugs.openjdk.org/browse/JDK-8251554
Generics over Primitive Types: https://bugs.openjdk.org/browse/JDK-8046267

For example, this would allow us to represent street graph elements as a List of compound objects in-place within the List, rather than a List of references to heap-allocated objects, or (the current situation) a flyweight cursor object seeking over parallel arrays of primitives. The reduction in pointer indirection and improvements in memory locality should be significant. Significant enough that we may want to grab a pre-release of JDK 21 as soon as it's available and start learning to use it / evaluating benefits in a branch.

[abyrd]

Java LTS 21 is scheduled for release 19 September 2023, though unfortunately it still doesn’t include primitive classes.