Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Building with OpenJDK 11 #212

Closed
df7cb opened this issue Apr 1, 2019 · 25 comments
Closed

Building with OpenJDK 11 #212

df7cb opened this issue Apr 1, 2019 · 25 comments

Comments

@df7cb
Copy link

df7cb commented Apr 1, 2019

OpenJDK 8 was removed from Debian/unstable, so the trick "build with Java 8, run with Java 11" no longer works.

Trying to build with openjdk-11-headless results in:

[INFO] ---------------------< org.postgresql:pljava-api >----------------------
[INFO] Building PL/Java API 1.5.3-SNAPSHOT                                [2/8]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ pljava-api ---
[INFO] 
[INFO] --- maven-resources-plugin:3.0.1:resources (default-resources) @ pljava-api ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/cbe/projects/postgresql/pljava/pljava.git/pljava-api/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ pljava-api ---
[INFO] Compiling 24 source files to /home/cbe/projects/postgresql/pljava/pljava.git/pljava-api/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] Failure executing javac, but could not parse the error:
Fatal Error: Unable to find package java.lang in classpath or bootclasspath

[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for PostgreSQL PL/Java 1.5.3-SNAPSHOT:
[INFO] 
[INFO] PostgreSQL PL/Java ................................. SUCCESS [  4.198 s]
[INFO] PL/Java API ........................................ FAILURE [  1.567 s]
[INFO] PL/Java backend Java code .......................... SKIPPED
[INFO] PL/Java backend native code ........................ SKIPPED
[INFO] PL/Java Deploy ..................................... SKIPPED
[INFO] PL/Java Ant tasks .................................. SKIPPED
[INFO] PL/Java examples ................................... SKIPPED
[INFO] PL/Java packaging .................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  6.443 s
[INFO] Finished at: 2019-04-01T09:24:29+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project pljava-api: Compilation failure
[ERROR] Failure executing javac, but could not parse the error:
[ERROR] Fatal Error: Unable to find package java.lang in classpath or bootclasspath

We'd need a new release that supports a newer OpenJDK for apt.postgresql.org. Thanks!
@jcflack
Copy link
Contributor

jcflack commented Apr 2, 2019 •
edited
Loading

Hi Christoph,

Thanks for the heads up. Is there a bit of lead time available on this? How long before sid becomes a version people are clamoring for?

Java 9 brought big changes that might require substantial edits in the build system, and I am not sure it will be practical to make those edits in the PL/Java 1.5.x branch while also keeping it buildable on older versions. So far, my (somewhat inchoate) plan has been to keep 1.5.x as the branch that everybody can still build on ancient systems, and make the-major-version-that-master-is-becoming be the modern one, that will probably build only on Java 9 and later. But I still have a lot of work planned for that branch.

Meanwhile, a little playing around this evening suggests that not-very-extensive changes to the 1.5.x pom will allow it to go further:

--- a/pom.xml
+++ b/pom.xml
@@ -90,15 +90,12 @@
                                        <source>1.6</source>
                                        <target>1.6</target>
                                        <encoding>${project.build.sourceEncoding}</encoding>
-                                       <compilerArguments>
-                                               <bootclasspath>${env.JAVA_HOME}/jre/lib/rt.jar</bootclasspath>
-                                       </compilerArguments>
                                </configuration>
                        </plugin>
                        <plugin>
                                <groupId>org.apache.maven.plugins</groupId>
                                <artifactId>maven-jar-plugin</artifactId>
-                               <version>2.6</version>
+                               <version>3.0.2</version>
                        </plugin>
                        <plugin>
                                <groupId>org.apache.maven.plugins</groupId>

Honestly, I don't know what that bootclasspath entry was ever doing there to begin with. Thomas added it back in 2013, and the commit message doesn't say why. Taking it out does not break the current build for me with Java 8 at all. If building with Java 12 instead of 11, the source and target should both be changed (to a minimum of 7, as 12 dropped support for building for targets 6 and earlier). If it will definitely never be run on any earlier JVM, 11 might be a good choice, because a new feature in that class file version makes nested class access checks more efficient. (I have not measured the size of the effect.)

Perhaps as an interim solution, you could build with a patch to the pom?

However, the above isn't enough yet. With those changes, it gets as far as the embedded JavaScript and then fails. That seems to be related to this Maven issue, which looks like it has a complicated history, maybe it was merged in Maven 3.5.1, or 3.5.2, but has been reopened, and the last comment (as of now) says it was reverted. It is marked critical, so I assume there will be progress....

So far, I have not tried to simply download a more recent Maven to see how far that gets. What version are you building with?

@df7cb
Copy link
Author

df7cb commented Apr 2, 2019 via email

@jcflack
Copy link
Contributor

jcflack commented Apr 3, 2019

No joy ... tried with Maven 3.5.2 and 3.6.0, still falls over right at the first javascript bit.

I've added a comment on the MNG-6275 issue asking if that's where the trouble is, and the outlook for a resolution if so.

@jcflack
Copy link
Contributor

jcflack commented May 10, 2019

Another thing worth knowing:

Today saw the first production release of GraalVM, 19.0.

This release of GraalVM is (among other things) a Java SE 8 JVM, and PL/Java 1.5.2 needs no special attention to build and/or run with it (other than to add -Dpolyglot.js.nashorn-compat=true on the mvn command line).

It would be great to see GraalVM packaged for the distro, as it can do very cool things (as can PL/Java, when running on it).

@jcflack
Copy link
Contributor

jcflack commented Jun 14, 2019

I've pushed a branch trackjdk/REL1_5_STABLE/java9 that will build for me in the post-Java-8 JDKs (but will also still build with the older ones), if you have time to give it a try.

Java 12 eliminated the ability to build with runtime compatibility back to 6. If built with 12, the classfiles will only be recognized by Java 7 and later JREs.

This branch is based on the latest merges to REL1_5_STABLE, so it should also work with PG 12. If it works and I merge it, a release supporting PG 12 and building with later JREs could shortly follow.

@df7cb
Copy link
Author

df7cb commented Jun 24, 2019

Hi,
I finally got to test this. It's complaining it can't find jni.h:

export JAVA_HOME = /usr/lib/jvm/default-java
mvn --batch-mode -Pwnosign -Psaxon-examples -Dso.debug=true -Dpgsql.pgconfig=/usr/lib/postgresql/11/bin/pg_config -Dpljava.libjvmdefault=/usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so clean install
...
[ERROR] /home/cbe/projects/postgresql/pljava/pljava.git/pljava-so/src/main/include/pljava/pljava.h:16:10: fatal error: jni.h: Datei oder Verzeichnis nicht gefunden
...

The file is there, though:

openjdk-11-jdk-headless:amd64: /usr/lib/jvm/java-11-openjdk-amd64/include/jni.h

@jcflack
Copy link
Contributor

jcflack commented Jun 24, 2019 via email

@jcflack
Copy link
Contributor

jcflack commented Jun 25, 2019

In my environment it is finding a copy of jni.h that was bundled in the gcc package, of all places.

I had never noticed that. That is not very reassuring.

@jcflack
Copy link
Contributor

jcflack commented Jun 25, 2019

I think the current tip of the trackjdk/REL1_5_STABLE/java9 branch should work now.

@df7cb
Copy link
Author

df7cb commented Jun 26, 2019

Confirmed with Java 11 and PG11 and 12 on Debian unstable.
On the other side of the spectrum, there's a problem though. Xenial, Java 8, PG9.5:

2019-06-26 14:32:09.362 CEST [3580] cbe@postgres FEHLER:  java.sql.SQLException: SPI function SPI_prepare failed with error SPI_ERROR_UNCONNECTED
2019-06-26 14:32:09.362 CEST [3580] cbe@postgres ANWEISUNG:  CREATE EXTENSION pljava

@jcflack
Copy link
Contributor

jcflack commented Jun 27, 2019

This patch isn't what broke that, so I've opened issue #221 for that (SPI_ERROR_UNCONNECTED in PG 9.6 and earlier). I'll go ahead and pull-request this patch, and then deal with #221 shortly.

@jcflack jcflack mentioned this issue Jun 27, 2019
Merged
@df7cb
Copy link
Author

df7cb commented Jun 27, 2019

Let me know when there's a branch that you'd like having tested!

jcflack added a commit that referenced this issue Jun 28, 2019
@jcflack
Merge pull request #222 from trackjdk/REL1_5_STABLE/java9
3d7abe7 
Permits building the 1.5 series with Java 9 or newer JDKs, otherwise
without radical change or loss of back compatibility.

The Java 12 compiler can no longer generate code for targets older than
Java 7, so if a PL/Java must be built to run in Java 6, the build must
be done with a JDK no newer than 11.

Addresses issue #212.
@jcflack
Copy link
Contributor

jcflack commented Jun 29, 2019

A branch worth testing now would be REL1_5_STABLE. :)

@df7cb
Copy link
Author

df7cb commented Aug 13, 2019

I finally tested 1_5 and it looks good. Are you planning to do a release soonish?

@jcflack
Copy link
Contributor

jcflack commented Aug 13, 2019

Thanks for checking. Was it a recent REL1_5_STABLE? There were a couple pull requests merged just last night.

At the moment, I am holding "soonish" to be "some time between this month and general availability of PostgreSQL 12". I still have WIP on #225 that will probably go in. I continue to be of two minds about #200 for this release ... most of the identified bugs are fixed (in bug/REL1_5_STABLE/issue200) but some of the fixes would change old behavior, and the more I learn about the old behavior, the longer the #200 bug report gets.

Also there's been a lot of progress completing the Saxon example to fill in the ISO SQL XQuery-based functionality that isn't in PG core. That's on a WIP branch here that hasn't been pushed to github yet but probably will after another rebase or two, and be part of the release.

@df7cb
Copy link
Author

df7cb commented Aug 13, 2019

It was from today, c9e48ed.

Maybe don't wait for everything to be perfect? You could still do another release once 12 is out.

@jcflack
Copy link
Contributor

jcflack commented Sep 22, 2019

Ok, I'll say you've talked me into saving the #200 fixes for a later release. #225 is in, and the fleshed-out XQuery example. REL1_5_STABLE right now should be pretty much it. I have just committed release notes.

@jcflack
Copy link
Contributor

jcflack commented Oct 4, 2019

Believed resolved in 1.5.3.

@jcflack jcflack closed this as completed Oct 4, 2019
@df7cb
Copy link
Author

df7cb commented Oct 4, 2019

Hi,
sorry for not having tested that yet. I've been busy with the PG12 release and other bits. Will give it a try next week.
Thanks!

@df7cb
Copy link
Author

df7cb commented Oct 10, 2019

I'm afraid it doesn't work with java 11:

+ mvn --batch-mode -Pwnosign -Psaxon-examples -Dso.debug=true -Dpgsql.pgconfig=/usr/lib/postgresql/11/bin/pg_config -Dpljava.libjvmdefault=/usr/lib/jvm/java-11-openjdk-amd64/lib/server/libjvm.so clean install
[...]
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 74 source files to /home/cbe/projects/postgresql/pljava/postgresql-pljava/pljava/target/classes
[INFO] /home/cbe/projects/postgresql/pljava/postgresql-pljava/pljava/src/main/java/org/postgresql/pljava/internal/Session.java: /home/cbe/projects/postgresql/pljava/postgresql-pljava/pljava/src/main/java/org/postgresql/pljava/internal/Session.java uses or overrides a deprecated API.
[INFO] /home/cbe/projects/postgresql/pljava/postgresql-pljava/pljava/src/main/java/org/postgresql/pljava/internal/Session.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/cbe/projects/postgresql/pljava/postgresql-pljava/pljava/src/main/java/org/postgresql/pljava/jdbc/AbstractResultSet.java: Some input files use unchecked or unsafe operations.
[INFO] /home/cbe/projects/postgresql/pljava/postgresql-pljava/pljava/src/main/java/org/postgresql/pljava/jdbc/AbstractResultSet.java: Recompile with -Xlint:unchecked for details.
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /home/cbe/projects/postgresql/pljava/postgresql-pljava/pljava/src/main/java/org/postgresql/pljava/jdbc/SQLXMLImpl.java:[597,54] cannot find symbol
  symbol:   method newFactory()
  location: class javax.xml.stream.XMLInputFactory
[ERROR] /home/cbe/projects/postgresql/pljava/postgresql-pljava/pljava/src/main/java/org/postgresql/pljava/jdbc/SQLXMLImpl.java:[784,78] cannot find symbol
  symbol:   method newFactory()
  location: class javax.xml.stream.XMLInputFactory
[ERROR] /home/cbe/projects/postgresql/pljava/postgresql-pljava/pljava/src/main/java/org/postgresql/pljava/jdbc/SQLXMLImpl.java:[1121,80] cannot find symbol
  symbol:   method newFactory()
  location: class javax.xml.stream.XMLOutputFactory
[ERROR] /home/cbe/projects/postgresql/pljava/postgresql-pljava/pljava/src/main/java/org/postgresql/pljava/jdbc/SQLXMLImpl.java:[2899,71] cannot find symbol
  symbol:   method newFactory()
  location: class javax.xml.stream.XMLInputFactory
[ERROR] /home/cbe/projects/postgresql/pljava/postgresql-pljava/pljava/src/main/java/org/postgresql/pljava/jdbc/SQLXMLImpl.java:[2901,72] cannot find symbol
  symbol:   method newFactory()
  location: class javax.xml.stream.XMLOutputFactory
[INFO] 5 errors 
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for PostgreSQL PL/Java 1.5.3:
[INFO] 
[INFO] PostgreSQL PL/Java ................................. SUCCESS [  6.721 s]
[INFO] PL/Java API ........................................ SUCCESS [  5.011 s]
[INFO] PL/Java backend Java code .......................... FAILURE [  1.324 s]
[INFO] PL/Java backend native code ........................ SKIPPED
[INFO] PL/Java Deploy ..................................... SKIPPED
[INFO] PL/Java Ant tasks .................................. SKIPPED
[INFO] PL/Java examples ................................... SKIPPED
[INFO] PL/Java packaging .................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  13.610 s
[INFO] Finished at: 2019-10-10T11:44:50+02:00

$ java -version
openjdk version "11.0.5-ea" 2019-10-15
OpenJDK Runtime Environment (build 11.0.5-ea+9-post-Debian-1)
OpenJDK 64-Bit Server VM (build 11.0.5-ea+9-post-Debian-1, mixed mode, sharing)

@jcflack
Copy link
Contributor

jcflack commented Oct 10, 2019

Oh bother. I didn't actually test in Java 6 this time, or in 11, I tested in 7, 8, and 12.

Those methods were introduced in 7. The pre-java9 compilers did not catch the problem because the -source 1.6/-target 1.6 options only affect syntax and code generation, but don't verify that the API is limited to the named release. (The generated code would have thrown a runtime LinkageError at those points if run on a 1.6 VM.)

The smarter --release 6 option used with the newer compilers actually checks that all the API existed in the chosen release. My testing with Java 12 wasn't sufficient to cover 11, because 12 drops support for release 6.

Looks like I'll achieve a shorter interval between 1.5.3 and 1.5.4 than there was from .1 to .2.

@jcflack jcflack mentioned this issue Oct 15, 2019
Closed
@jcflack
Copy link
Contributor

jcflack commented Oct 15, 2019 •
edited
Loading

I was mistaken above: the methods in question were not introduced in 7; they are documented and present in 6. Where they aren't present is in the ct.sym representation of release 6 API that the javac compiler uses to verify the use of compatible API. That seems really to be a bug in the ct.sym file shipped in the JDK.

Continued in #235.

@jcflack
Copy link
Contributor

jcflack commented Oct 22, 2019

Hi Christoph,

If there's any chance you could test REL1_5_STABLE before I make it another release, it would be great to know if you experience any problem again.

Regards,
-Chap

@df7cb
Copy link
Author

df7cb commented Oct 29, 2019

I just tested on sid (Java 11) and on xenial (Java 8), for PG 9.4 .. 12. Everything fine.

Let me know if there's any interesting distributions with Java versions in between that I should cover as well.

@jcflack
Copy link
Contributor

jcflack commented Oct 29, 2019

Thanks! I guess zesty and artful had Java 9 packages, but I've built on AdoptOpenJDK 9 and 10 here, so I think I'll take the chance. I've released 1.5.4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@df7cb @jcflack