How Garbage Collection Works: Freeing Memory in Java, Go, and Python

1. Architectural Foundations & Mechanics

To understand automatic memory management across languages, one must look directly at the underlying hardware boundaries. Processors interact with physical DRAM through hierarchical cache lines (L1, L2, and L3) across NUMA nodes. When an application creates an object, the runtime must locate available virtual address space, assign ownership, initialize metadata headers, and eventually recycle that memory block once references cease to exist. How a runtime executes these steps dictates its allocation latency, cache locality, CPU utilization, and tail pause times.

Java HotSpot: The Generational Hypothesis & Region-Based Evacuation

Java’s memory management rests on an empirical observation known as the Weak Generational Hypothesis: the vast majority of allocated objects survive for only a brief window of time (such as local variables inside short-lived methods, transient JSON parsers, or temporary string builders). Objects that survive this initial window tend to live for an extended duration (such as long-lived connection pools, cached domain models, and application singletons).

HotSpot’s memory architecture structures the heap into separate generational spaces to exploit this distribution:

• Eden Space & TLABs: When a Java thread allocates memory, it rarely contends for a global heap lock. Instead, the runtime assigns each thread a Thread-Local Allocation Buffer (TLAB) inside the young generation's Eden space. Allocation within a TLAB is a fast operation requiring only a pointer bump: the runtime reads the current allocation pointer, adds the requested object size in bytes, compares it against the TLAB limit, and updates the pointer. This takes just a few CPU instructions without atomic locks.
• Evacuation Collection (Minor GC): When Eden fills up, the JVM initiates a minor collection. Because most young objects are already dead, the collector does not spend time identifying dead memory. Instead, it begins at the root set (thread stack frames, static variables, JNI handles) and traces only the live objects, copying (evacuating) them into a designated Survivor space (S0 or S1). The remaining contents of Eden are instantly treated as free space by resetting the TLAB pointers to the beginning of the memory block. Reclamation cost is strictly proportional to the volume of surviving objects, not the size of the heap.
• Card Tables & Post-Write Barriers: Tracing the young generation without scanning the entire old generation presents an architectural hurdle: what if an old-generation object holds a reference to a young-generation object? To solve this without a full heap scan, HotSpot maintains a byte array called the Card Table. Each byte represents a 512-byte region of old-generation memory. Whenever an application thread writes a reference into an object field, the JIT-compiled code executes a post-write barrier:

  void post_write_barrier(oop* field_address, oop new_val) {
      size_t card_index = ((size_t)field_address) >> 9; // Divide by 512
      byte_map_base[card_index] = DIRTY_BYTE;          // Mark card as dirty (0x0)
  }

  During a minor collection, the collector scans only the memory blocks corresponding to dirty cards in the Card Table, drastically cutting root scanning overhead.
• Modern Region-Based Collectors (G1 & ZGC): The Garbage-First (G1) collector abstracts the monolithic generational heap into up to 2,048 contiguous memory regions ranging from 1 MB to 32 MB. G1 models which regions hold the highest ratio of dead objects and reclaims those "garbage-first." The Z Garbage Collector (ZGC) pushes latency boundaries further down into the sub-millisecond range by performing nearly all collection phases concurrently, including object relocation. ZGC achieves this using Colored Pointers (which embed reference metadata directly into the upper bits of 64-bit pointers) combined with a JIT-injected Load Barrier. If an application thread reads a reference pointing to an object currently being relocated, the load barrier intercepts the read, updates the pointer to its new address immediately (a self-healing mechanism), and returns the valid object without stalling the thread.

Go Runtime: Non-Generational, Low-Latency Concurrent Tri-Color Mark-Sweep

The Go runtime takes a fundamentally different path. Go deliberately omits generational collection and object compaction. This engineering decision is anchored in two foundational properties of the Go programming language:

1. Compile-Time Escape Analysis: The Go compiler analyzes the lexical scope and lifetime of every variable. If a variable’s reference does not escape the call frame of its originating function, it is allocated directly on the goroutine’s stack. When the function returns, the stack pointer increments, reclaiming that memory at zero runtime cost. In idiomatic Go, the vast majority of short-lived objects never reach the heap.
2. Value Semantics & Contiguous Memory Layout: Go allows developers to declare arrays of structs, embedded structs, and flat slice buffers. Unlike Java, where an array of objects is an array of pointers scattered across the heap, a Go slice of structs stores all data fields sequentially in a single contiguous memory block. This reduces total heap object counts and pointer indirection by orders of magnitude, weakening the operational advantage that generational collectors derive from high infant mortality.

Go manages heap allocations using a segregated size-class architecture derived from Google’s TCMalloc:

• Allocation Hierarchy (mcache, mcentral, mheap): Each logical processor (P in Go's scheduler) owns a local memory cache (mcache) containing 68 size-class spans (mspan). When a goroutine allocates an object smaller than 32 KB, the runtime assigns it to the exact size-class bucket in the local mcache without acquiring a mutex lock. If the local span runs out of blocks, it requests a refill from the shared mcentral, which locks only that specific size-class. Objects larger than 32 KB bypass the size classes and are allocated directly from the global page allocator (mheap).
• The Tri-Color Marking Abstraction: Go discovers live objects during its concurrent phase by categorizing every heap object into one of three virtual colors:
  • White: Unvisited candidate objects. When marking begins, all heap objects are White. At the end of the mark phase, any remaining White objects are unreachable and will be swept.
  • Grey: Visited objects that the collector has discovered, but whose child pointers have not yet been evaluated. Grey objects reside in a work queue.
  • Black: Verified live objects whose outgoing references have been scanned. Black objects contain no unscanned outgoing references and cannot point directly to White objects unless mediated by a write barrier.
• The Hybrid Write Barrier: Because the collector runs concurrently with active application goroutines (mutators), a mutator could hide an unvisited White object behind a Black object while destroying the path from any Grey object. If this happened, the collector would incorrectly sweep the live White object. Go prevents this using a hybrid write barrier combining Dijkstra-style insertion and Yuasa-style deletion rules. When a pointer in the heap is modified during concurrent marking, the barrier shades the old pointer Grey (if not already shaded) and shades the newly installed pointer Grey:

  // Conceptual representation of Go's hybrid write barrier
  func gcWriteBarrier(slot *uintptr, ptr uintptr) {
      shade(*slot) // Preserve object if it was about to be overwritten (Yuasa)
      shade(ptr)   // Mark new object grey to prevent hiding (Dijkstra)
      *slot = ptr  // Perform the actual memory write
  }

  This barrier ensures that goroutine stacks remain Black without requiring expensive stack rescans, reducing Stop-the-World pauses to brief setup and teardown phases that routinely complete in under 50 microseconds.
• GC Pacer & Mark Assist: The Go runtime controls GC scheduling using a feedback loop known as the Pacer. By default, Go sets GOGC=100, meaning the collector triggers when the heap grows by 100% over the live heap size determined at the end of the previous cycle. The collector dedicates 25% of available CPU cores (GOMAXPROCS) to background marking. However, if mutator goroutines allocate memory faster than the background marking workers can scan, the Pacer engages Mark Assist: it temporarily forces the rapidly allocating goroutine to stop application work and scan a quota of Grey objects proportional to the volume of memory it is trying to allocate. This prevents runaway memory inflation at the cost of transient latency spikes on aggressive allocator threads.

Python CPython: Deterministic Reference Counting & Auxiliary Cycle Collection

CPython’s memory architecture represents the simplest and most deterministic of the three runtimes, though it carries distinct trade-offs for CPU throughput and multi-threaded scaling.

• The PyObject Header & Reference Counting: Every object in Python is represented internally by a C structure containing the PyObject_HEAD macro. This header includes two foundational fields:

  typedef struct _object {
      _PyObject_HEAD_EXTRA // Doubly-linked list pointers when debugging
      Py_ssize_t ob_refcnt; // 64-bit reference counter
      struct _typeobject *ob_type; // Pointer to Python type descriptor
  } PyObject;

  Every assignment, parameter pass, or container append increments ob_refcnt using the Py_INCREF() macro. When a variable goes out of scope, is deleted with del, or is overwritten, the interpreter executes Py_DECREF(). The moment ob_refcnt drops to zero, the object is deallocated on the spot:

  #define Py_DECREF(op) do {                \
      if (--(op)->ob_refcnt == 0)           \
          _Py_Dealloc((PyObject *)(op));    \
  } while (0)

  This delivers deterministic reclamation: file descriptors, sockets, and memory buffers are freed the exact moment their last reference disappears, without waiting for an asynchronous collection phase.
• The Circular Reference Problem: Reference counting breaks down when objects reference each other in a closed loop. For example, if object A references object B, and B references A, both will have an ob_refcnt of at least 1, even if all external variables pointing to A and B are removed from the global and local scopes. Pure reference counting would leak both objects permanently.
• The Auxiliary Generational Cycle Detector: To eliminate circular reference leaks, CPython runs an auxiliary cyclic collector. This collector ignores non-container types (like integers, floats, and strings, which cannot store outgoing references) and monitors container objects (like dictionaries, lists, tuples, and user-defined class instances). It organizes these containers into three linked-list generations:
  • Generation 0: Contains all newly allocated container objects. When allocations minus deallocations exceed a configurable threshold (default: 700), a Gen 0 collection triggers.
  • Generation 1: Containers that survive a Gen 0 collection are promoted here. Gen 1 is collected after a certain number of Gen 0 cycles have executed (default: 10).
  • Generation 2: Long-lived containers that have survived multiple sweeps reside here. Gen 2 is scanned least frequently (default: every 10 Gen 1 cycles).
• Trial Deletion Algorithm: When scanning a generation, CPython breaks cycles by temporarily copying each container's ob_refcnt into an internal field called gc_refs. It then iterates through all containers in the candidate set and decrements the gc_refs of every child object referenced by that container. After this pass, any object whose gc_refs drops to zero is referenced only by objects within that isolated candidate pool; no external references point to it. The collector isolates these unreachable groups and frees them, resolving the cycle.
• Small-Object Memory Allocation (pymalloc): Standard operating system allocators (like glibc's malloc) introduce CPU overhead and severe virtual memory fragmentation when handling millions of tiny object allocations. CPython tackles this with pymalloc, a specialized allocator active for requests up to 512 bytes:
  • Arenas: 256 KB memory allocations obtained directly from the system allocator via mmap or malloc, aligned to 256 KB boundaries.
  • Pools: Each Arena is divided into 4 KB Pools. A Pool is dedicated to a single fixed size class (ranging from 8 bytes to 512 bytes in multiples of 8).
  • Blocks: Each Pool is carved into uniform memory blocks. Allocation within a pool uses an internal free-list pointer. Memory returns to the pool via simple pointer reassignment, avoiding system-call overhead during intensive object creation.

2. Mathematical Modeling & Resource Bounds

Memory management systems operate within strict mechanical limits governed by CPU cycles, bus bandwidth, and addressable DRAM. We can express the efficiency and trade-offs of these systems through formal resource models.

1. Generational Collection Efficiency (Java HotSpot)

The throughput efficiency of a generational copying collector ($E_{\text{gen}}$), measured as the bytes of memory reclaimed per unit of pause time, can be formulated as: