What Go Taught Us About Java Garbage Collection

As Java developers, we take a lot of inspiration from HotSpot. It is the gold standard for many of us, and for good reason. But measuring ParparVM against Go recently sent me down a different path.

My first thought was stack allocation. If Go was keeping more objects off the heap, that would explain a lot: fewer allocations, fewer pointers to follow, less work for the collector. It is the same attraction that makes Valhalla’s value objects so interesting to Java developers. Why pay for a separate object when what you need is a value?

Looking deeper, allocation wasn’t the whole explanation. Go’s runtime makes different choices about when to collect and how to share the work. Could we benefit from some of those choices too?

ParparVM is a good place to try. We translate Java to C and compile ahead of time. Our closed-world build sees the application before it runs, and we control the object layout, reference handling, and collector together. In that respect, our constraints are closer to Go’s native runtime than to HotSpot’s dynamic execution model.

That investigation became this week’s GC work. It started with the memory we allowed garbage to occupy and ended up reaching the way an image cache decides what to keep.

The memory we were giving away

A collector needs room to let the application allocate between collections. More room can mean fewer collections, which saves CPU time. But a larger allowance only makes sense if the workload benefits from it.

Our minimum allocation threshold was 24 MB. In the test, the process reached 98 MB of resident memory while holding very little live data. We lowered the threshold and watched memory, throughput, and latency together.

Throughput and p99 barely moved beyond the noise between runs. Resident memory did. After tuning the lower-floor configuration, we went from 98 MB to 38 MB. Those are process RSS measurements, including memory outside the Java heap.

Resident memory at each GC trigger setting in the experiment. The final tuned configuration reached 38 MB.

The smarter formula had the wrong input

The obvious next step was to make the threshold follow the live set. A small live set would get a small allowance; a larger one would get more room. We tried three versions of that idea and ran into the same underlying problem: our counters weren’t counting what the policy needed. The investigation is recorded beside the collector implementation.

An ordinary sweep samples retired pages. It does not count all the live objects on partial pages, and mutator-owned current pages are not swept. Large objects use another heap path. Even the major sweep’s policy counters omit part of that population. The number available to the policy therefore cannot safely stand in for the whole live set.

One attempt held on to an old live-set estimate after the objects were gone. The resulting high threshold suppressed the very sweeps needed to return pages to the operating system.

We made the floor configurable in the runtime instead:

#ifndef CN1_BIBOP_GC_MIN_TRIGGER_BYTES
#define CN1_BIBOP_GC_MIN_TRIGGER_BYTES CN1_BIBOP_GC_TRIGGER_BYTES
#endif

A runtime build can now choose the smaller floor. The default remains 24 MB while we work through more workloads. The experiment had already shown that a smaller allowance could release memory without giving up speed.

Put waiting threads to work

Reducing the garbage allowance also brought us back to the marking work itself. Our GcPause loop allocates 20 million short-lived objects while keeping a 4,096-node live set. Median and p99 were comparable to Go, but the long stalls were where we had the most work to do:

Four markers cut those stalls substantially. Go still finished its worst pause in about 20 ms, giving us a clear target for the next round.

We also added mutator assistance. When an allocating thread reaches the limit on how far it can run ahead of collection, it can mark a batch of objects instead of waiting. The thread registers as active before releasing the worklist lock, so the collector cannot finish while that batch is still in flight.

An old arm64 corruption report had kept us on the serial default. Since then, we have fixed barriers, object validation, and root handling. A dedicated workflow now exercises one and four markers on arm64 and four on x64. Parallel marking and assistance remain on that experimental path as we work through those runs.

The image you will need again in a second

A decoded image costs memory to keep and CPU time to recreate. That makes it a different problem from the short-lived garbage in the allocation loop.

Imagine scrolling through a gallery and then back up to the image you just passed. If the cache kept it, the next frame can reuse it. If a memory warning emptied the cache, that same gesture sends the application back through decoding.

PR #5732 gives the collector real weak and soft references, including a policy that favors recently accessed data. The collector can now make that choice instead of treating the entire cache as permanent storage.

The old low-memory workaround

ParparVM’s WeakReference used to hold its referent in an ordinary object field. The translator emitted the same mark operation as for any strong reference. As long as the wrapper remained reachable, its supposedly weak referent remained reachable too. There was no SoftReference implementation to provide a separate policy.

The iOS port worked around missing soft-reference behavior with a strong-reference table. A memory warning called flushSoftRefMap(), which replaced the table. That low-memory signal approximated “the VM may discard this cache,” but only as a bulk operation.

Image decoding, scaled images, RGB copies, rounded borders, and rasterized gradients all have reasons to cache disposable results. Their desired contract is simple: keep the result if useful, and return null when it must be reconstructed. A port-wide table held until a warning cannot express that contract precisely.

A weak field must stop being an ordinary edge

The referent now lives in Reference. The translator recognizes that field and emits reference discovery instead of tracing it as an unconditional strong edge. The collector decides whether to retain a soft referent and later clears references whose objects are eligible for reclamation.

A small image cache can now use the Java API directly:

import java.lang.ref.SoftReference;
import com.codename1.ui.Image;
import java.io.IOException;

final class PreviewCache {
    private SoftReference<Image> cached;

    Image get() throws IOException {
        Image result = cached == null ? null : cached.get();
        if (result == null) {
            result = Image.createImage("/preview.png");
            cached = new SoftReference<Image>(result);
        }
        return result;
    }
}

The local variable keeps a successful result strongly reachable while the caller uses it. A cleared reference sends us back through image creation. This example assumes one calling thread; a shared cache needs synchronization around its own state.

Reading during collection is the dangerous case

Suppose the collector scans a thread and moves on. That thread then calls get() and keeps the referent in a local. If the collector clears the reference and frees the object without noticing the read, Java code receives a dangling native pointer.

The fix couples reference reads to the snapshot-at-the-beginning barrier machinery. A relevant read is logged while marking is active. Reference processing remains inside the termination loop, with the barrier armed, so new work can force another pass before sweeping.

Logging every reference read made the collector repeatedly reopen marking. The filter excludes objects already marked in the current epoch or still fresh, matching the conditions that would prevent clearing them. After filtering those reads, termination passes fell from 32 to one.

Recency helps keep the cache useful

Each successful get() refreshes the reference’s rank. The collector uses the age of that access when choosing what to retain, so the image you keep returning to tends to outlast the one you passed once.

The RefPolicy benchmark ran five interleaved repetitions with simulated process-memory ceilings and matching checksums:

RefPolicy results with a simulated 160 MB memory ceiling.

The weak-reference check cleared 255 of 256 probe referents, where the old implementation cleared none. Ranked retention also beat the bulk flush: the cache kept more of the useful results while occupying less memory. Comparing recency with other eviction policies is a good next experiment.

Back to the screen

The next step is moving framework caches onto these references. The iOS softReferenceMap override and existing cache call sites are still in place; we now have the collector support to replace that workaround. We can measure the change in image decoding and scrolling as well as in memory.

I started this investigation wondering whether Go’s allocation choices explained its performance. We found work on both sides of an allocation: how to avoid creating the object, and how to handle it once it exists. Tomorrow’s map and boxing story returns to the first question and our “poor man’s Valhalla” work.

An AOT runtime gives us room to pursue both ideas together. We can change an object’s representation and how the collector handles it, then expose the same Java API. The app still gets a safe reference from get() while collection runs. Keeping that promise inside ParparVM lets application developers concentrate on what they want to keep on screen.

Discussion

Which costs more in your image-heavy screen: retaining decoded images or reconstructing them after a cache flush?