MentaBlog

Simplicity is everything

Real-time Java programming without GC

 |  3 Comments

If you are writing latency-sensitive applications in Java it is paramount that you gain control over the garbage collector. Although you cannot turn off the GC, you can and should adopt some coding techniques that will delay the garbage collector indefinitely. But before we examine these techniques, we need a reliable way to profile our programs for allocated memory and collected memory, so we can know accurately the amount of garbage (i.e. de-referenced objects) we are leaving behind. Meet GCUtils, a simple tool I wrote to (really) force the GC and measure these values. Check the examples below: (If you are curious how I was able to force the GC you can take a look in the source code here)

Ad: CoralBits: the bits and pieces of real-time programming in Java without GC overhead: high performance data-structures, memory allocation and gc instrumentation classes, ByteBuffer and byte array utilities and much more.


// Nothing allocated or collected

GCUtils.init();
String s = null;
GCUtils.measure();

System.out.println("Memory allocated: " + GCUtils.getMemoryAllocated());
System.out.println("Memoty collected: " + GCUtils.getMemoryCollected());

// OUTPUT:
//Memory allocated: 0
//Memoty collected: 0

// One string allocated but nothing collected

GCUtils.init();
String s = new String(SOMETHING);
GCUtils.measure();

System.out.println("Memory allocated: " + GCUtils.getMemoryAllocated());
System.out.println("Memoty collected: " + GCUtils.getMemoryCollected());

// OUTPUT:
// Memory allocated: 32
// Memoty collected: 0

// Nothing allocated and one string collected

String s = new String(SOMETHING);
GCUtils.init();
s = null;
GCUtils.measure();

System.out.println("Memory allocated: " + GCUtils.getMemoryAllocated());
System.out.println("Memoty collected: " + GCUtils.getMemoryCollected());

// OUTPUT:
// Memory allocated: 0
// Memoty collected: 32

// One string allocated and one string collected

String s = new String(SOMETHING);
GCUtils.init();
String x = new String(SOMETHING);
s = null;
GCUtils.measure();

System.out.println("Memory allocated: " + GCUtils.getMemoryAllocated());
System.out.println("Memoty collected: " + GCUtils.getMemoryCollected());

// OUTPUT:
// Memory allocated: 32
// Memoty collected: 32

Object Pooling

Now that we have a profiling tool in hand we can start to investigate some common Java operations. Let’s start with a simple HashMap and see what happens when we add and remove an object:

GCUtils.init();
map.put(KEY, OBJ);
map.remove(KEY);
GCUtils.measure();

System.out.println("Memory allocated: " + GCUtils.getMemoryAllocated());
System.out.println("Memoty collected: " + GCUtils.getMemoryCollected());

// OUTPUT:
//Memory allocated: 32
//Memoty collected: 32

What the code above demonstrates is that the java.util.HashMap produces garbage every time you add and remove an object. That happens because it internally creates an Entry object to hold the key/value pair and discards it the moment the entry is removed. So the first mandatory technique to avoid the garbage collector is to use object pooling: always re-use, never discard.

It is obvious from the test above that the Java collections are not suitable for real-time systems. They not just produce garbage but they are also not optimized for speed. Your best bet is to code your own optimized data structures or look for a real-time open source implementation. Interesting enough I found one called Javolution but unfortunately their FastMap produces a lot of garbage as you can see below:

FastMap<String, String> map = new FastMap<String, String>();
GCUtils.init();
for(int i = 0; i < 32; i++) {
	set.add(VALUE);
	set.remove(VALUE);
}
GCUtils.measure();

System.out.println("Memory allocated: " + GCUtils.getMemoryAllocated());
System.out.println("Memoty collected: " + GCUtils.getMemoryCollected());

// OUTPUT:
//Memory allocated: 1049840
//Memoty collected: 1031344

Bootstrapping

The writers of the JDK code were not low-latency traders so it does not come as a surprise that the internals of the Java libraries are not attentive to garbage creation. Fortunately the JVM provides the bootstrap command-line option that allows you to override the JDK classes with your own implementations. The goal is not to rewrite everything but to fix those important spots that will be present in the critical path of your application. One such example is the EPoll selector implementation from the NIO libraries. Check its code below: (from the source here)

private int updateSelectedKeys() {
    int entries = pollWrapper.updated;
    int numKeysUpdated = 0;
    for (int i = 0; i < entries ; i++) {
        int nextFD = pollWrapper.getDescriptor(i);
        SelectionKeyImpl ski = (SelectionKeyImpl) fdToKey.get(new Integer(nextFD));
        // ski is null in the case of an interrupt
        if (ski != null) {
            int rOps = pollWrapper.getEventOps(i);
            if (selectedKeys.contains(ski)) {
                if (ski.channel.translateAndSetReadyOps(rOps, ski)) {
                    numKeysUpdated++;
                }
            } else {
                ski.channel.translateAndSetReadyOps(rOps, ski);
                if ((ski.nioReadyOps() & ski.nioInterestOps()) != 0) {
                    selectedKeys.add(ski);
                    numKeysUpdated++;
                }
            }
        }
    }
    return numKeysUpdated;
}

The code above is executed every time a select operation is called on a selector and as you can see it creates garbage with the new Integer(nextFD). It also uses java.util.HashMap and java.util.HashSet to store the file descriptors and the selected keys. As we have already seen those guys also create garbage. It is not hard to fix these JDK problems until your critical selector loop is leaving no garbage behind.

Conclusion

If you are developing applications where low-latency is essential you absolutely cannot afford to have the Garbage Collector kick in with its stop-the-world pauses. Since it is not possible to turn off the GC, producing ZERO garbage is the only option. By using a LIFO object pool to recycle objects you not only take care of the garbage problem but you may even end up with these objects in the L1 cache. You should use object pooling in your code as well as in the JDK libraries through bootstrapping. By taking control over your trash, you can warmup your critical applications, force a garbage collector early in the game and then watch them go through production without being disturbed by the GC.

UPDATE: As Peter mentioned here you should also disable TLAB with -XX:-UseTLAB as it allocates memory in chunks and can influence the number reported by freeMemory().

3 Comments

  1. Pingback: Inter-socket communication with less than 2 microseconds latency | MentaBlog

  2. Pingback: Real-time Java programming without GC | G3nt00r's Blog

  3. Have you tried Trove for a HashMap implementation that doesnt create garbage?

    Map map2 = new THashMap();

    GCUtils.init();
    map2.put(KEY, OBJ);
    map2.remove(KEY);
    GCUtils.measure();

    System.out.println(“Memory allocated: ” + GCUtils.getMemoryAllocated());
    System.out.println(“Memoty collected: ” + GCUtils.getMemoryCollected());

    // OUTPUT:
    //Memory allocated: 0
    //Memoty collected: 0

Leave a Reply    (moderated so note that SPAM will not be approved!)