Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to Java 17 and Kryo 5 #3994

Merged
merged 9 commits into from
Mar 17, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/cibuild.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@ jobs:
with:
fetch-depth: 0
# Java setup step completes very fast, no need to run in a preconfigured docker container
- name: Set up JDK 11
- name: Set up JDK 17
uses: actions/setup-java@v1
with:
java-version: 11
java-version: 17
- name: Cache local Maven repository
uses: actions/cache@v2
with:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/performance-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ jobs:
with:
fetch-depth: 0

- name: Set up JDK 11
- name: Set up JDK 17
uses: actions/setup-java@v1
with:
java-version: 11
java-version: 17

- name: Set up Maven
uses: stCarolas/[email protected]
Expand Down
9 changes: 1 addition & 8 deletions docs/Basic-Tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This page should allow you to set up and test your own OTP2 server. If all goes

## Get Java

As a Java program, OTP must be run within a Java virtual machine (JVM), which is provided as part of the Java runtime (JRE) or Java development kit (JDK). OTP2 is compatible with Java 11 or later. We recommend running on Java 11 rather than a later version, as it is a long-term support release. Run `java -version` to check that you have version 11 or newer of the JVM installed. If you do not, you will need to install a recent OpenJDK or Oracle Java package for your operating system.
As a Java program, OTP must be run within a Java virtual machine (JVM), which is provided as part of the Java runtime (JRE) or Java development kit (JDK). OTP2 is compatible with Java 17 or later. We recommend running on Java 17 rather than a later version, as it is a long-term support release. Run `java -version` to check that you have version 17 or newer of the JVM installed. If you do not, you will need to install a recent OpenJDK or Oracle Java package for your operating system.

## Get OTP

Expand Down Expand Up @@ -44,13 +44,6 @@ If you have extracted a smaller PBF file from a larger region, be sure to put on
A typical command to start OTP looks like `java -Xmx2G -jar otp.shaded.jar <options>`. The
`-Xmx` parameter sets the limit on how much memory OTP is allowed to consume. GTFS and OSM data sets are often very large, and OTP is relatively memory-hungry. You will need at least 1GB of memory when working with the Portland TriMet data set, and several gigabytes for larger inputs. If you have
sufficient memory in your computer, set this to a couple of gigabytes (e.g. `-Xmx2G`). Java uses a [garbage collection](https://en.wikipedia.org/wiki/Garbage_collection_(computer_science)) approach to memory management, which requires some "breathing room" to efficiently operate. Without sufficient free memory OTP can grind to a halt. [VisualVM](https://visualvm.github.io) is a good way to inspect Java memory usage, especially with the [VisualGC plugin](https://visualvm.github.io/plugins.html).

Java 11 has tighter security restrictions than previous versions, so when running OTP under Java 11 you will see warnings like this:
```
WARNING: An illegal reflective access operation has occurred
WARNING: Please consider reporting this to the maintainers of com.esotericsoftware.kryo.util.UnsafeUtil
```
These warnings are expected to remain for a while, until all libraries OTP2 depends on have fully migrated to Java 11 and we have upgraded them all.

## Building Graphs

Expand Down
53 changes: 33 additions & 20 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@
</scm>

<properties>
<otp.serialization.version.id>19</otp.serialization.version.id>
<otp.serialization.version.id>20</otp.serialization.version.id>
<!-- Lib versions - keep list sorted on property name -->
<geotools.version>26.2</geotools.version>
<jackson.version>2.13.1</jackson.version>
<jackson.version>2.13.2</jackson.version>
<jersey.version>2.34</jersey.version>
<junit.version>5.7.2</junit.version>
<micrometer.version>1.7.4</micrometer.version>
<junit.version>5.8.2</junit.version>
<micrometer.version>1.8.3</micrometer.version>
<netcdf4.version>5.5.2</netcdf4.version>
<netty.version>4.1.74.Final</netty.version>
<!-- Other properties -->
Expand All @@ -37,7 +37,7 @@
<!-- Set argLine to an empty string so that running the tests without the coverage agent works.
When running `mvn jacoco:prepare-agent test` argLine is replaced with the one activating the agent.
-->
<argLine></argLine>
<argLine/>
</properties>

<distributionManagement>
Expand Down Expand Up @@ -92,7 +92,7 @@
<version>3.8.1</version>
<configuration>
<!-- Target Java versions -->
<release>11</release>
<release>17</release>
</configuration>
</plugin>
<plugin>
Expand Down Expand Up @@ -194,16 +194,29 @@
@{argLine}
-Xmx2G
-Dfile.encoding=UTF-8
--illegal-access=permit
--add-opens java.base/java.io=ALL-UNNAMED
--add-opens java.base/java.lang=ALL-UNNAMED
--add-opens java.base/java.lang.module=ALL-UNNAMED
--add-opens java.base/java.math=ALL-UNNAMED
--add-opens java.base/java.net=ALL-UNNAMED
--add-opens java.base/java.text=ALL-UNNAMED
--add-opens java.base/java.time=ALL-UNNAMED
--add-opens java.base/java.time.zone=ALL-UNNAMED
--add-opens java.base/java.util=ALL-UNNAMED
--add-opens java.base/java.util.concurrent.locks=ALL-UNNAMED
--add-opens java.base/java.util.regex=ALL-UNNAMED
--add-opens java.base/jdk.internal.reflect=ALL-UNNAMED
--add-opens java.base/jdk.internal.misc=ALL-UNNAMED
--add-opens java.base/jdk.internal.loader=ALL-UNNAMED
--add-opens java.base/jdk.internal.ref=ALL-UNNAMED
--add-opens java.base/jdk.internal.util=ALL-UNNAMED
--add-opens java.base/jdk.internal.util.jar=ALL-UNNAMED
--add-opens java.base/jdk.internal.module=ALL-UNNAMED
--add-opens java.base/java.lang=ALL-UNNAMED
--add-opens java.base/sun.net.www.protocol.http=ALL-UNNAMED
--add-opens java.base/sun.net.www.protocol.jar=ALL-UNNAMED
--add-opens java.base/sun.util.calendar=ALL-UNNAMED
--add-opens java.base/sun.util.locale=ALL-UNNAMED
--add-opens java.xml/org.xml.sax.helpers=ALL-UNNAMED
</argLine>
<!-- Jenkins needs XML test reports to determine whether the build is stable. -->
<disableXmlReport>false</disableXmlReport>
Expand Down Expand Up @@ -366,7 +379,7 @@
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.2.10</version>
<version>1.2.11</version>
</dependency>
<!-- Jersey uses java.util logging, redirect it to slf4j API (we use the Logback implementation) -->
<dependency>
Expand Down Expand Up @@ -461,7 +474,7 @@
<dependency>
<groupId>org.entur</groupId>
<artifactId>netex-java-model</artifactId>
<version>1.0.11</version>
<version>1.0.13</version>
</dependency>

<!-- SIRI -->
Expand Down Expand Up @@ -511,8 +524,8 @@
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.10.19</version>
<artifactId>mockito-core</artifactId>
<version>4.4.0</version>
<scope>test</scope>
</dependency>
<dependency>
Expand All @@ -526,13 +539,13 @@
<dependency>
<groupId>com.conveyal</groupId>
<artifactId>kryo-tools</artifactId>
<version>1.3.0</version>
<version>1.4.0</version>
</dependency>
<!-- Direct dependency with -jre version, in order to not resolve to the limited -android version -->
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>31.0.1-jre</version>
<version>31.1-jre</version>
</dependency>
<!-- Jersey annontation-driven REST web services (JAX-RS implementation) -->
<dependency>
Expand Down Expand Up @@ -652,19 +665,19 @@
<dependency>
<groupId>net.java.dev.jets3t</groupId>
<artifactId>jets3t</artifactId>
<version>0.8.1</version>
<version>0.9.4</version>
</dependency>
<!-- OpenStreetMap protobuf (PBF) definitions and parser -->
<dependency>
<groupId>org.openstreetmap.osmosis</groupId>
<artifactId>osmosis-osm-binary</artifactId>
<version>0.47.3</version>
<version>0.48.3</version>
</dependency>
<!-- Command line parameter parsing -->
<dependency>
<groupId>com.beust</groupId>
<artifactId>jcommander</artifactId>
<version>1.78</version>
<version>1.82</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
Expand All @@ -674,7 +687,7 @@
<dependency>
<groupId>com.graphql-java</groupId>
<artifactId>graphql-java</artifactId>
<version>17.2</version>
<version>17.3</version>
</dependency>
<dependency>
<groupId>com.graphql-java</groupId>
Expand All @@ -684,12 +697,12 @@
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.5</version>
<version>4.5.13</version>
</dependency>
<dependency>
<groupId>commons-cli</groupId>
<artifactId>commons-cli</artifactId>
<version>1.4</version>
<version>1.5.0</version>
<scope>test</scope>
</dependency>
<dependency>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -261,13 +261,6 @@ private void interline(Collection<TripPattern> tripPatterns, Graph graph) {
}
}

// Copy all interline relationships into the field holding them in the graph.
// TODO: verify whether we need to be keeping track of patterns at all here, or could just accumulate trip-trip relationships.
for (P2<TripPattern> patterns : interlines.keySet()) {
for (P2<Trip> trips : interlines.get(patterns)) {
graph.interlinedTrips.put(trips.first, trips.second);
}
}
t2gran marked this conversation as resolved.
Show resolved Hide resolved
LOG.info("Done finding interlining trips.");
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ public void write(Kryo kryo, Output output, BuildConfig object) {
}

@Override
public BuildConfig read(Kryo kryo, Input input, Class<BuildConfig> type) {
public BuildConfig read(Kryo kryo, Input input, Class<? extends BuildConfig> type) {
return new BuildConfig(
ConfigLoader.nodeFromString(input.readString(), SOURCE),
SOURCE,
Expand Down
19 changes: 0 additions & 19 deletions src/main/java/org/opentripplanner/kryo/HashBiMapSerializer.java

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ public void write(Kryo kryo, Output output, RouterConfig object) {
output.writeString(object.toJson());
}
@Override
public RouterConfig read(Kryo kryo, Input input, Class<RouterConfig> type) {
public RouterConfig read(Kryo kryo, Input input, Class<? extends RouterConfig> type) {
return new RouterConfig(
ConfigLoader.nodeFromString(input.readString(), SOURCE),
SOURCE,
Expand Down
Loading