Gencon occurs OOME when creating new String but other GC policies does not.

[BurryPotter]

Java -version output

java -version
openjdk version "1.8.0_362"
IBM Semeru Runtime Open Edition (build 1.8.0_362-b09)
Eclipse OpenJ9 VM (build openj9-0.36.0, JRE 1.8.0 Windows 11 amd64-64-Bit Compressed References 20230207_599 (JIT enabled, AOT enabled)
OpenJ9 - e68fb24
OMR - f491bbf6f
JCL - eebde685ec based on jdk8u362-b09)

Summary of problem

The Gencon collector appears to encounter difficulties in creating new String when there is limited remaining memory available.

During program execution, I continue to allocate memory until an OutOfMemoryError (OOME) is triggered. And then I release 100MB of space and create new simple strings. But it throws OOME again when I create new strings. Interestingly, if I don't use the Gencon collector and instead use a different collector, I am able to create strings successfully.

Furthermore, I had tested it on JDK8, JDK11 and JDK17 (But the information provided later is from JDK8), and found that the Gencon collector throws an OOME (OutOfMemoryError) when creating strings, while other collectors can successfully create strings. Maybe, the GenCon collector handles string allocation differently, which could explain the OutOfMemoryError (OOME) during string creation.

Reproduce step

1. Download the source code Case.java

import java.util.ArrayDeque;

public class Case {
    private static final int MB = 1024 * 1024;

    public static void main(String[] var0) throws Exception {
        ArrayDeque<byte[]> arrayDeque = new ArrayDeque<>();
        boolean breakFlag = false;
        while (!breakFlag) {
            try {
                arrayDeque.add(new byte[MB]);
            } catch (OutOfMemoryError outOfMemoryError) {
                int mb = arrayDeque.size();
                for (int i = 0; i < 100 && !arrayDeque.isEmpty(); ++i) {
                    arrayDeque.removeLast();
                }
                System.gc();
                breakFlag = true;
                System.out.println("Allocate " + mb + "MB memory.");
            }
        }
    }
}

2. compile and run

>java  -Xms4096m -Xmx4096m  -XX:-HeapDumpOnOutOfMemoryError  -Xgcpolicy:optavgpause Case
Allocate 4095MB memory.

>java  -Xms4096m -Xmx4096m  -XX:-HeapDumpOnOutOfMemoryError  -Xgcpolicy:optthruput Case 
Allocate 4095MB memory.

>java  -Xms4096m -Xmx4096m  -XX:-HeapDumpOnOutOfMemoryError  -Xgcpolicy:balanced Case  
Allocate 3072MB memory.

>java  -Xms4096m -Xmx4096m  -XX:-HeapDumpOnOutOfMemoryError  -Xgcpolicy:gencon Case  
Exception in thread "main" java.lang.OutOfMemoryError: Java 堆空间
        at Case.main(Case.java:19)

3. expecte result

After releasing 100MB of memory, the program should have enough space to create this short string "Allocate " + mb + "MB memory." without throwing an OOME.

Diagnostic files

I run the following command, and then upload the dump files to Google Driven. dump files

> java  -Xms4096m -Xmx4096m   -Xgcpolicy:gencon Case
JVMDUMP055I
JVMDUMP032I JVM 使用“D:\projects\Cases\target\classes\core.20230523.154108.32924.0001.dmp”来请求 System 转储以响应事件
JVMDUMP010I System 转储已写入 D:\projects\Cases\target\classes\core.20230523.154108.32924.0001.dmp
JVMDUMP032I JVM 使用“D:\projects\Cases\target\classes\heapdump.20230523.154108.32924.0002.phd”来请求 Heap 转储以响应事件
JVMDUMP010I Heap 转储已写入 D:\projects\Cases\target\classes\heapdump.20230523.154108.32924.0002.phd
JVMDUMP032I JVM 使用“D:\projects\Cases\target\classes\javacore.20230523.154108.32924.0003.txt”来请求 Java 转储以响应事件
JVMDUMP010I Java 转储已写入 D:\projects\Cases\target\classes\javacore.20230523.154108.32924.0003.txt
JVMDUMP032I JVM 使用“D:\projects\Cases\target\classes\Snap.20230523.154108.32924.0004.trc”来请求 Snap 转储以响应事件
JVMDUMP010I Snap 转储已写入 D:\projects\Cases\target\classes\Snap.20230523.154108.32924.0004.trc
JVMDUMP013I 已处理转储事件“systhrow”，详细信息：“java/lang/OutOfMemoryError”。
JVMDUMP055I
JVMDUMP032I JVM 使用“D:\projects\Cases\target\classes\heapdump.20230523.154959.32924.0005.phd”来请求 Heap 转储以响应事件
JVMDUMP010I Heap 转储已写入 D:\projects\Cases\target\classes\heapdump.20230523.154959.32924.0005.phd
JVMDUMP032I JVM 使用“D:\projects\Cases\target\classes\javacore.20230523.154959.32924.0006.txt”来请求 Java 转储以响应事件
JVMDUMP010I Java 转储已写入 D:\projects\Cases\target\classes\javacore.20230523.154959.32924.0006.txt
JVMDUMP032I JVM 使用“D:\projects\Cases\target\classes\Snap.20230523.154959.32924.0007.trc”来请求 Snap 转储以响应事件
JVMDUMP010I Snap 转储已写入 D:\projects\Cases\target\classes\Snap.20230523.154959.32924.0007.trc
JVMDUMP013I 已处理转储事件“systhrow”，详细信息：“java/lang/OutOfMemoryError”。
Exception in thread "main" java.lang.OutOfMemoryError: Java 堆空间
        at Case.main(Case.java:19)

OutOfMemoryError: Java Heap Space

I run the following command to get details informations, witch is writed into gencongc.log file gencongc.log

java  -Xms4096m -Xmx4096m -verbose:gc -Xverbosegclog:gencongc.log -XX:-HeapDumpOnOutOfMemoryError -Xgcpolicy:gencon Case

[dmitripivkine]

@amicic FYI

[amicic]

@BurryPotter, thanks for reporting this problem.

However, for the most part this is expected behaviour of gencon. It’s mostly a side effect of a couple of things:


• literals (generally, interned-strings) are pre-tenured (allocated directly from Tenure space)
• relatively rigid boundary between Nursery and Tenure subspaces in gencon
• (Possibly some other things like Allocate-Survivor separation in Nursery and Small-Large (SOA-LOA) object separation in Tenure, but to a smaller degree)

Gencon’s Nursery default size is up to 1/4 of total heap. Tenure default size is up to total heap size. But once heap is fully expanded, we don’t not allow changing that boundary between Tenure and Nursery.


In this case the heap is fully expanded (Xms==Xmx), so for the duration of the run the fixed ratio of 3-to-1 is maintained (3GB Tenure, 1G Nursery).

In a case of heap gradually expanding the ratio could be different, so that Tenure space is more then 3/4 of the total heap (for example 3.5G Tenure and 0.5G Nurseru), but again, if heap gets fully expanded, and while it stays fully expanded the ratio does not change.

The test case, after filling up the heap, will remove 100MB of objects, but it will be young objects (by virtue of doing removeLast), hence freeing up all memory in Nursery and none in Tenure. Hence, the pre-tenure allocation fails, without an attempt to expand Tenure (at the cost of Nursery).

Some workaround, to at least validate the statements and better understand how gencon’s sizing work:

• Reduce Nursery size so that some of objects that are removed end up being in Tenure. For example, the test will pass even with gencon if -Xmn128M is provided (due to Nursery having reserved Survivor space that is empty, the amount of objects in Nursery will be actually less than full Nursery size, in this case even less the 100M)
• Start without fully expanding heap (without Xms option altogether or with a smaller value), allowing (possibly by chance) the heap expansion gradually converge to some other Heap separation between Tenure and Nursery (and SOA-LOA, Allocate-Survivor separation), so that Tenure space ends up having a bit of free memory at the end of the final pre-tenure allocation
• Modify the test case to do removeFirst (instead of removeLast), that will remove objects from Tenure.

In theory, we could perhaps fix this behaviour, by for example allowing Tenure to expand at the cost of Nursery contraction to satisfy a pre-tenure allocation (if Nursery has some free space, which is indeed the case here). And it’s been a consideration to allow Tenure expansion (at the cost of Nursery) for many year, but it’s not as trivial as it may sound, and we will probably not allow it any time soon. Allowing interned-string allocation from Nursery would also be a major change in behaviour, even less likely to happen.

Overall, for the time being, we should just accept this behaviour as per design (and if acceptable for your real-live scenarios, work around it, with more or less suggestions I provided).

[BurryPotter]

Thank you for your response. It appears to be an issue caused by the design and may be challenging to resolve within a short period of time. I will make modifications to my code to avoid similar problems in the future.

[amicic]

Closing it as working as designed, and no plans to change the behaviour any time soon.

[BurryPotter]

I'd like to reopen this issue. In this test program, it clearly has free memory available, yet it fails to allocate strings.
Combining the above responses with my own understanding, I think this is a design problem of the garbage collector. And since Gencon is the default garbage collector for J9, I believe it deserves to be made even more perfect. I can understand that this issue might not be easy to fix, but I hope it can be incorporated into the future plans.

[hzongaro]

@dmitripivkine, I assume you intended to reopen this issue when you moved it to the Deep Backlog milestone. If not, please close it.