SEVERE error encountered on Jet instance

[wildnez]

Ran the demo as documented with one exception - 2 IMDG members instead of 3, and ran into the below error after 7 mins run:

SEVERE: [localhost]:5710 [JetInABox] [3.1] Execution of job 'AdjustMerchantTransactionAverage', execution 245a-d151-63fc-9ae2 failed after 408,673 ms
com.hazelcast.jet.JetException: Exception in ProcessorTasklet{AdjustMerchantTransactionAverage/Draw Transactions from preAuth map#0}: com.hazelcast.jet.JetException: Null key from key extractor, edge: between("Draw Transactions from preAuth map", "Aggregate average transaction amount by merchant-prepare").partitioned(?)
	at com.hazelcast.jet.impl.execution.TaskletExecutionService$CooperativeWorker.runTasklet(TaskletExecutionService.java:315)
	at java.base/java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:803)
	at com.hazelcast.jet.impl.execution.TaskletExecutionService$CooperativeWorker.run(TaskletExecutionService.java:289)
	at java.base/java.lang.Thread.run(Thread.java:834)
	at ------ submitted from ------.(Unknown Source)
	at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolve(InvocationFuture.java:126)
	at com.hazelcast.spi.impl.AbstractInvocationFuture$1.run(AbstractInvocationFuture.java:251)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
	at com.hazelcast.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:64)
	at com.hazelcast.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:80)
Caused by: com.hazelcast.jet.JetException: Null key from key extractor, edge: between("Draw Transactions from preAuth map", "Aggregate average transaction amount by merchant-prepare").partitioned(?)
	at com.hazelcast.jet.core.Edge$KeyPartitioner.getPartition(Edge.java:588)
	at com.hazelcast.jet.impl.execution.OutboundCollector$Partitioned.offer(OutboundCollector.java:193)
	at com.hazelcast.jet.impl.execution.OutboxImpl.doOffer(OutboxImpl.java:241)
	at com.hazelcast.jet.impl.execution.OutboxImpl.offerInternal(OutboxImpl.java:141)
	at com.hazelcast.jet.impl.execution.OutboxImpl.offer(OutboxImpl.java:93)
	at com.hazelcast.jet.core.AbstractProcessor.tryEmit(AbstractProcessor.java:310)
	at com.hazelcast.jet.core.AbstractProcessor.emitFromTraverser(AbstractProcessor.java:410)
	at com.hazelcast.jet.core.AbstractProcessor.emitFromTraverser(AbstractProcessor.java:423)
	at com.hazelcast.jet.impl.connector.StreamEventJournalP.emitResultSet(StreamEventJournalP.java:209)
	at com.hazelcast.jet.impl.connector.StreamEventJournalP.complete(StreamEventJournalP.java:194)
	at com.hazelcast.jet.impl.execution.ProcessorTasklet.stateMachineStep(ProcessorTasklet.java:367)
	at com.hazelcast.jet.impl.execution.ProcessorTasklet.call(ProcessorTasklet.java:232)
	at com.hazelcast.jet.impl.execution.TaskletExecutionService$CooperativeWorker.runTasklet(TaskletExecutionService.java:308)
	at java.base/java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:803)
	at com.hazelcast.jet.impl.execution.TaskletExecutionService$CooperativeWorker.run(TaskletExecutionService.java:289)
	at java.base/java.lang.Thread.run(Thread.java:834)

[myawnhc]

Yeah, I’ve tried to filter the data to exclude the item with the null key but the filter didn’t work, so I need to try again.

[wildnez]

Is a null key deliberate decision? I wonder what purpose it would otherwise solve.

[myawnhc]

I think this has been resolved but leaving open as the currently code is not starting up this pipeline at all (moving some of the work from executors to jet pipelines in an effort to better illustrate Jet + IMDG working together). So chances are the specific pipeline that was failing doesn't get reactivated; but if it does, I think the issue occurred when we ran out of data and wrapped around the input file to the beginning - an off-by-one error had us looking for a non-existent transaction ID.