"failed-to-read" tests make the pipeline UNSTABLE

[emanuelelevo]

Hello, we use sbt test for executing tests and generating test-results.

Randomly, junit (1.34) fails to read some of the .xml

Stacktrace
Failed to read test report file /var/jenkins/workspace/xxxxxx/target/test-reports/TEST-xxxxxxx.TaskUtilsTest.xml
org.dom4j.DocumentException: Error on line 18 of document  : Content is not allowed in trailing section.
	at org.dom4j.io.SAXReader.read(SAXReader.java:511)
	at org.dom4j.io.SAXReader.read(SAXReader.java:392)
	at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:178)
	at hudson.tasks.junit.TestResult.parse(TestResult.java:369)
	at hudson.tasks.junit.TestResult.parsePossiblyEmpty(TestResult.java:299)
	at hudson.tasks.junit.TestResult.parse(TestResult.java:224)
	at hudson.tasks.junit.TestResult.parse(TestResult.java:196)
	at hudson.tasks.junit.TestResult.<init>(TestResult.java:151)
	at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:143)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3122)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:375)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.xml.sax.SAXParseException; lineNumber: 18; columnNumber: 32; Content is not allowed in trailing section.
	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:400)
	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1473)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$TrailingMiscDriver.next(XMLDocumentScannerImpl.java:1431)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
	at org.dom4j.io.SAXReader.read(SAXReader.java:494)
	... 17 more

This makes the step UNSTABLE and leads the entire pipeline state result to UNSTABLE, stopping dev from being able to merge on GitHub (continuous-integration check).

Is there a way to prevent junit from going UNSTABLE? I tried the following and it didn't work:

    post {
        always {
            catchError(buildResult: 'SUCCESS', stageResult: 'SUCCESS') {
                junit 'target/test-reports/*.xml'
                archiveArtifacts artifacts: "target/scala-${DEFAULT_SCALA_VERSION}/scoverage-report/**"
                archiveArtifacts artifacts: 'target/test-reports/**'
                step([$class: 'ScoveragePublisher', reportDir: "target/scala-${DEFAULT_SCALA_VERSION}/scoverage-report", reportFile: 'scoverage.xml'])
            }
        }

also

    post {
        always {
            junit allowEmptyResults: true, healthScaleFactor: 0.0, testResults:'target/test-reports/*.xml'
            archiveArtifacts artifacts: "target/scala-${DEFAULT_SCALA_VERSION}/scoverage-report/**"
            archiveArtifacts artifacts: 'target/test-reports/**'
            step([$class: 'ScoveragePublisher', reportDir: "target/scala-${DEFAULT_SCALA_VERSION}/scoverage-report", reportFile: 'scoverage.xml'])
            
        }

unfortunately, the above didn't help.

[timja]

There's a PR for it here that needs some minor fixing up: #98