[CI] Test suite failures

[andrewazores]

Describe the feature

Since #188 the CI test suites are failing, but working (for me) locally.

https://github.com/cryostatio/cryostat3/commits/main/

Anything other information?

No response

[andrewazores]

#188 might be a red herring actually. Here are logs from a failing run:

 2023-12-19 15:33:07,321 INFO  [tc.qua.io/cryostat/cryostat-db:latest] (build-23) Container quay.io/cryostat/cryostat-db:latest is starting: be315bb910ddc21e98f7159eb7ca9c5f9459af92be587b17f9ca1ea5d04a7c9c
2023-12-19 15:33:08,977 ERROR [tc.qua.io/cryostat/cryostat-db:latest] (build-23) Could not start container: com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"cause":"readObjectStart: expect { or n, but found \u0000, error found in #0 byte of ...||..., bigger context ...||...","message":"container create failed (no logs from conmon): conmon bytes \"\": readObjectStart: expect { or n, but found \u0000, error found in #0 byte of ...||..., bigger context ...||...","response":500}

	at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:247)
	at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.post(DefaultInvocationBuilder.java:102)
	at org.testcontainers.shaded.com.github.dockerjava.core.exec.StartContainerCmdExec.execute(StartContainerCmdExec.java:31)
	at org.testcontainers.shaded.com.github.dockerjava.core.exec.StartContainerCmdExec.execute(StartContainerCmdExec.java:13)
	at org.testcontainers.shaded.com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21)
	at org.testcontainers.shaded.com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35)
	at org.testcontainers.shaded.com.github.dockerjava.core.command.StartContainerCmdImpl.exec(StartContainerCmdImpl.java:43)
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:441)
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:344)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:334)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:322)
	at io.quarkus.devservices.postgresql.deployment.PostgresqlDevServicesProcessor$1.startDatabase(PostgresqlDevServicesProcessor.java:72)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.startDevDb(DevServicesDatasourceProcessor.java:291)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.launchDatabases(DevServicesDatasourceProcessor.java:140)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:864)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:282)
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
	at java.base/java.lang.Thread.run(Thread.java:840)
	at org.jboss.threads.JBossThread.run(JBossThread.java:501)

2023-12-19 15:33:08,987 ERROR [tc.qua.io/cryostat/cryostat-db:latest] (build-23) There are no stdout/stderr logs available for the failed container

Could it just be the timing of a cryostat-db change coinciding with #188?

[andrewazores]

In #217 I tried reverting #188 and it didn't help. I also tried bisecting database images to several weeks back and nothing worked there either.

[andrewazores]

Also tried spinning up a resource-constrained F38 VM and could not reproduce the test failures with current main. I'll have to try with an Ubuntu VM like what the CI runner does.

[andrewazores]

containers/podman#21024

[andrewazores]

Looks like this has been resolved in the CI infra:

https://github.com/cryostatio/cryostat3/actions/runs/7386934828