OverlappingFileLockException when using ID sync file

[yawkat]

Hi,

I have a working jazzer target and would like to run testing in parallel. For this purpose I tried the -fork option. However, using it yielded an error:

ERROR: Coverage IDs are out of sync
ERROR: Coverage IDs are out of sync
ERROR: Coverage IDs are out of sync
ERROR: Coverage IDs are out of sync
com.code_intelligence.jazzer.agent.CoverageIdException: Failed to synchronize coverage IDs
ERROR: Coverage IDs are out of sync
        at com.code_intelligence.jazzer.agent.SynchronizedCoverageIdStrategy.obtainFirstId(CoverageIdStrategy.kt:169)
        at com.code_intelligence.jazzer.agent.RuntimeInstrumentor.instrument(RuntimeInstrumentor.kt:168)
        at com.code_intelligence.jazzer.agent.RuntimeInstrumentor.transformInternal(RuntimeInstrumentor.kt:138)
        at com.code_intelligence.jazzer.agent.RuntimeInstrumentor.transform(RuntimeInstrumentor.kt:79)
        at com.code_intelligence.jazzer.agent.RuntimeInstrumentor.transform(RuntimeInstrumentor.kt:125)
        at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
        at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541)
        at java.base/java.lang.ClassLoader.defineClass1(Native Method)
        at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1012)
        at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
        at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:862)
        at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:467)
        at io.micronaut.core.io.service.SoftServiceLoader.lambda$collectDynamicServices$4(SoftServiceLoader.java:191)
        at io.micronaut.core.io.service.SoftServiceLoader$ServiceInstanceLoader.compute(SoftServiceLoader.java:549)
        at java.base/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:194)
        at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
        at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
        at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
        at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
        at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: java.nio.channels.OverlappingFileLockException
        at java.base/sun.nio.ch.FileLockTable.checkList(FileLockTable.java:229)
        at java.base/sun.nio.ch.FileLockTable.add(FileLockTable.java:123)
        at java.base/sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1252)
ERROR: Coverage IDs are out of sync
        at java.base/java.nio.channels.FileChannel.lock(FileChannel.java:1089)
        at com.code_intelligence.jazzer.agent.SynchronizedCoverageIdStrategy.obtainFirstId(CoverageIdStrategy.kt:121)
        ... 25 more
com.code_intelligence.jazzer.agent.CoverageIdException: Failed to synchronize coverage IDs
ERROR: Coverage IDs are out of sync
        at com.code_intelligence.jazzer.agent.SynchronizedCoverageIdStrategy.obtainFirstId(CoverageIdStrategy.kt:169)
        at com.code_intelligence.jazzer.agent.RuntimeInstrumentor.instrument(RuntimeInstrumentor.kt:168)
ERROR: Coverage IDs are out of sync
        at com.code_intelligence.jazzer.agent.RuntimeInstrumentor.transformInternal(RuntimeInstrumentor.kt:138)
        at com.code_intelligence.jazzer.agent.RuntimeInstrumentor.transform(RuntimeInstrumentor.kt:79)
        at com.code_intelligence.jazzer.agent.RuntimeInstrumentor.transform(RuntimeInstrumentor.kt:125)
        at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)

To narrow this down further, I've also tried just running the fuzzer with an --id_sync_file=, giving the same result without a fork. Am I missing some setup step that is required for the id sync file?

I'm using jazzer 0.10.0 from github releases.

[bertschneider]

Thx for the report. The stack trace indicates that classes were loaded concurrently by Micronaut service loaders from a ForkJoinPool. This probably caused the file lock exception you encountered. So it's more of a concurrent class loading as a fork issue.

I created #310 to resolve this. Could you give it a try?

BTW --id_sync_file is only used internally and you don't need to set it for your runs.

[yawkat]

Yes that patch seems to fix it! Thanks.