Parsing multiple DOTA2 replays eventually leads to crash

[AlbPpp]

Hi,

I'm currently trying to parse multiple replays in a row. However, after parsing some, the next replay crashes the program.

The following exceptions may be thrown in the runner thread the first time that runner.tick() is called:

java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
	at [email protected]/skadistats.clarity.wire.Packet.parse(Packet.java:36)
	at [email protected]/skadistats.clarity.model.engine.AbstractDotaEngineType$1.parse(AbstractDotaEngineType.java:89)
	at [email protected]/skadistats.clarity.processor.reader.InputSourceProcessor.processSource(InputSourceProcessor.java:203)
	at [email protected]/skadistats.clarity.event.AbstractInvocationPoint.invoke(AbstractInvocationPoint.java:86)
	at [email protected]/skadistats.clarity.event.Event.raise(Event.java:41)
	at [email protected]/skadistats.clarity.processor.runner.AbstractFileRunner.initAndRunWith(AbstractFileRunner.java:39)
	at [email protected]/skadistats.clarity.processor.runner.ControllableRunner.lambda$runWith$0(ControllableRunner.java:243)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at [email protected]/skadistats.clarity.wire.Packet.parse(Packet.java:34)
	... 7 more
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had invalid wire type.
	at [email protected]/com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:99)
	at [email protected]/com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:515)
	at [email protected]/com.google.protobuf.GeneratedMessage.parseUnknownField(GeneratedMessage.java:205)
	at [email protected]/skadistats.clarity.wire.common.proto.Demo$CDemoSyncTick.<init>(Demo.java:8733)
	at [email protected]/skadistats.clarity.wire.common.proto.Demo$CDemoSyncTick.<init>(Demo.java:8691)
	at [email protected]/skadistats.clarity.wire.common.proto.Demo$CDemoSyncTick$1.parsePartialFrom(Demo.java:8769)
	at [email protected]/skadistats.clarity.wire.common.proto.Demo$CDemoSyncTick$1.parsePartialFrom(Demo.java:8764)
	at [email protected]/com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:104)
	at [email protected]/com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:124)
	at [email protected]/com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:129)
	at [email protected]/com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
	at [email protected]/skadistats.clarity.wire.common.proto.Demo$CDemoSyncTick.parseFrom(Demo.java:8817)
	... 12 more

or

java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
	at [email protected]/skadistats.clarity.wire.Packet.parse(Packet.java:36)
	at [email protected]/skadistats.clarity.model.engine.AbstractDotaEngineType$1.parse(AbstractDotaEngineType.java:89)
	at [email protected]/skadistats.clarity.processor.reader.InputSourceProcessor.processSource(InputSourceProcessor.java:203)
	at [email protected]/skadistats.clarity.event.AbstractInvocationPoint.invoke(AbstractInvocationPoint.java:86)
	at [email protected]/skadistats.clarity.event.Event.raise(Event.java:41)
	at [email protected]/skadistats.clarity.processor.runner.AbstractFileRunner.initAndRunWith(AbstractFileRunner.java:39)
	at [email protected]/skadistats.clarity.processor.runner.ControllableRunner.lambda$runWith$0(ControllableRunner.java:243)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at [email protected]/skadistats.clarity.wire.Packet.parse(Packet.java:34)
	... 7 more
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
	at [email protected]/com.google.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:89)
	at [email protected]/com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:158)
	at [email protected]/skadistats.clarity.wire.common.proto.Demo$CDemoSyncTick.<init>(Demo.java:8727)
	at [email protected]/skadistats.clarity.wire.common.proto.Demo$CDemoSyncTick.<init>(Demo.java:8691)
	at [email protected]/skadistats.clarity.wire.common.proto.Demo$CDemoSyncTick$1.parsePartialFrom(Demo.java:8769)
	at [email protected]/skadistats.clarity.wire.common.proto.Demo$CDemoSyncTick$1.parsePartialFrom(Demo.java:8764)
	at [email protected]/com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:104)
	at [email protected]/com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:124)
	at [email protected]/com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:129)
	at [email protected]/com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
	at [email protected]/skadistats.clarity.wire.common.proto.Demo$CDemoSyncTick.parseFrom(Demo.java:8817)
	... 12 more

or

java.io.IOException: FAILED_TO_UNCOMPRESS(5)
	at org.xerial.snappy.SnappyNative.throw_error(SnappyNative.java:98)
	at org.xerial.snappy.SnappyNative.rawUncompress(Native Method)
	at org.xerial.snappy.Snappy.rawUncompress(Snappy.java:474)
	at org.xerial.snappy.Snappy.uncompress(Snappy.java:513)
	at org.xerial.snappy.Snappy.uncompress(Snappy.java:488)
	at [email protected]/skadistats.clarity.processor.packet.PacketReader.readFromSource(PacketReader.java:17)
	at [email protected]/skadistats.clarity.model.engine.AbstractDotaEngineType$1.parse(AbstractDotaEngineType.java:91)
	at [email protected]/skadistats.clarity.processor.reader.InputSourceProcessor.processSource(InputSourceProcessor.java:173)
	at [email protected]/skadistats.clarity.event.AbstractInvocationPoint.invoke(AbstractInvocationPoint.java:86)
	at [email protected]/skadistats.clarity.event.Event.raise(Event.java:41)
	at [email protected]/skadistats.clarity.processor.runner.AbstractFileRunner.initAndRunWith(AbstractFileRunner.java:39)
	at [email protected]/skadistats.clarity.processor.runner.ControllableRunner.lambda$runWith$0(ControllableRunner.java:243)
	at java.base/java.lang.Thread.run(Thread.java:833)

Generally, it seems like that after some time, reading from the file does not work properly anymore.

I'm using a ControllableRunner with a MappedFileSource. At the time of the exception, the file source also reports an incorrect (much too low) last tick of the replay as well.

Parsing the replays that cause the exception on their own works totally fine however, so I do no think that they are corrupted.

The replays I use are fairly recent, from November 1st.

Do you have any idea what could cause this and/or how to fix it?

Thanks!

[AlbPpp]

Addition:
With the exceptions I also encountered the "unkown top level message of kind SOURCE2/XXXX", although I don't think that is the actual cause, since the replay on its own can be parsed normally.

[spheenik]

Hmm weird. I have a lot of people that parse replays after one another in the same VM and it works fine.
Clarity-Analyzer does the same, if you open a second file after the first one, no problems.

Are you able to come up with a small example of how I can reproduce this?