OutOfMemoryError: Requested array size exceeds VM limit

[StrongestNumber9]

I was investigating a race condition with file truncation and as I was constantly writing to a file and truncating it from another thread, this happened:

Exception in thread "pool-1-thread-1" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
        at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
        at java.base/java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:120)
        at java.base/java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:95)
        at java.base/java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:137)
        at com.teragrep.rlo_13.MonitoredFileConsumer.readFile(MonitoredFileConsumer.java:88)
        at com.teragrep.rlo_13.MonitoredFileConsumer.accept(MonitoredFileConsumer.java:129)
        at com.teragrep.rlo_13.MonitoredFileConsumer.accept(MonitoredFileConsumer.java:32)
        at com.teragrep.rlo_12.FileTask.run(FileTask.java:63)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

Not a big file by any means:

# wc -l /tmp/rlo_13_race/log/input.txt 
122297 /tmp/rlo_13_race/log/input.txt

# du -sh /tmp/rlo_13_race/log/input.txt 
118M    /tmp/rlo_13_race/log/input.txt

[StrongestNumber9]

Most likely related to #6

[StrongestNumber9]

Reproduced it with if(i%10000000==0) { and Thread.sleep(5000); in the ManualRaceConditionTest

[kortemik]

please check with du -h --apparent-size

       --apparent-size
              print apparent sizes rather than device usage; although the apparent size is usually smaller, it may be
              larger due to holes in ('sparse') files, internal fragmentation, indirect blocks, and the like

[kortemik]

it is very likely that the row size exceeds the jvm memory size. please create another issue for it.