Thomas Preißler

software developer,
addicted to KISS

I’m responsible for a Java application running in OpenShift. This application has to process a huge amount of data occassionally, but most of the time the application idles and waits for new input.

The application takes a huge amount of memory during processing the data. But when the processing job has been completed, the memory can be released and returned to the operating system. Unfortunately, this doesn’t happen. The JVM seems to keep the memory forever.

OpenShift Metrics

What’s going on?

Currently the G1 garbage collector may not return committed Java heap memory to the operating system in a timely manner. G1 only returns memory from the Java heap at either a full GC or during a concurrent cycle. Since G1 tries hard to completely avoid full GCs, and only triggers a concurrent cycle based on Java heap occupancy and allocation activity, it will not return Java heap memory in many cases unless forced to do so externally. – JEP 346

The motivation of JEP 346 describes perfectly what I observed from my application. When it should release memory, there’s no need anymore to run the garbage collector at all. Therefore, it will keep the memory until the next huge processing phase starts and requires the memory again.

How can this be fixed?

JEP 346 has exactly this issue in mind: Promptly Return Unused Committed Memory from G1. JEP 346 has been implemented in Java 12. Unfortunately, I have to use Java 11 and cannot benefit from this improvment.

But there are other garbage collectors than the default G1. Ruslan Synytsky has a well-written blog post about the memory consumption of different garbage collectors.

Based on his observations there’s a huge difference in the behaviour of the memory consumption. It seems that the Shenandoah GC might be a really good option for my application. Next to ZGC, Shenandoah is one of the newest garbage collectors and available as production ready in Java 15. But Shenandoah has been backported to OpenJDK 11 and is available since version 11.0.9. Therefore I’m able to use it. Let’s give it a try:

java -XX:+UseShenandoahGC -jar app.jar

OpenShift Metrics

It works! As you can see, some time after finishing the hard processing work, Shenandoah returns most of the memory back to the operating system.

Since reducing the heap is an expensive operation, Shenandoah takes a delay of 5 minutes (300,000 ms) by default to release any memory back to the operating system. You can set Shenandoah to be more aggressive, but the command line options for tuning Shenandoah are still marked as experimental. However, I’m fine with the default 5 minute delay.

java \
  -XX:+UseShenandoahGC \
  -XX:+UnlockExperimentalVMOptions \
  -XX:ShenandoahUncommitDelay=1000 \
  -XX:ShenandoahGuaranteedGCInterval=10000 \
  -jar app.jar

Comments are welcome on Twitter or LinkedIn.