Your technical library
Language Runtimes & Concurrency Models
12 modules · 75 topics
About this book
Language Runtimes & Concurrency Models
Why does a Python service not get faster with more threads? Why does a Go service's tail latency move when its heap grows, a Java service run slowly for its first minute, or a Node process stall while one handler parses a large JSON body? Each answer is a design decision inside the language runtime, with a cost that some other runtime chose not to pay. This course teaches those decisions on three axes — how code is executed, how memory is reclaimed, and how concurrent work is expressed and scheduled — with CPython, HotSpot, V8, Go, Rust and the BEAM (Erlang's virtual machine) as reference points, so you can place any runtime, even one you have never used, on the same map. It builds on Operating Systems and Data Structures, and starts where the kernel's mechanisms stop.
Execution comes first: interpreters and virtual machines, object models and inline caching, and just-in-time and ahead-of-time compilation. Then memory: garbage collection: foundations, generational and concurrent collection, and memory management without a tracing collector, which decides object lifetimes before the program runs — in the compiler, in the type system, or by grouping objects that die together. Then concurrency: runtime threads and schedulers, async/await and event loops, message passing, supervision and structured concurrency, and global interpreter locks. Language memory models ties the three axes together, and runtimes as points in the design space closes the course by placing each runtime on the map and choosing one for a workload.
75 of 75 topics are authored. Read in dependency order. Planned lessons are labelled and excluded from progress; references cover authored material.
Read at your own pace. Each topic is one continuous page; mark it done as you finish.
Start with 01.1 · Bytecode: flattening the tree into an instruction streamReview what you have finished↷ 0 skipped topics- 03.1Tiered execution: when a method is worth compiling
- 03.2Baseline JITs: removing dispatch without optimising
- 03.3Speculative optimisation: compiling for the types you have seen
- 03.4Deoptimisation: rebuilding an interpreter frame from optimised code
- 03.5Tracing JITs: compiling the path the loop actually takes
- 03.6Meta-compilation: a JIT derived from an interpreter
- 03.7WebAssembly: a VM designed to be validated and compiled, not speculated
- 03.8JIT or AOT: peak performance, warm-up and startup
- 04.1Reachability, roots and safepoints: what a collector must find
- 04.2Reference counting: immediate reclamation, paid on every pointer write
- 04.3Cycle collection: finding garbage that counts cannot see
- 04.4Mark-sweep: trace from the roots, free the rest
- 04.5Copying collection: cost proportional to what survives
- 04.6Mark-compact and mark-region: defragmenting in place
- 04.7Allocation fast paths: bump pointers and thread-local buffers
- 04.8Finalizers and weak references: when death must do something
- 04.9Tracing and counting are duals: a unified view of collection
- 05.1Generational collection: most objects die young
- 05.2Write barriers: card tables and remembered sets
- 05.3Concurrent marking: the tri-colour invariant and its barriers
- 05.4Moving objects under a running program: read barriers and coloured pointers
- 05.5Region-based collection: choosing what to evacuate within a pause target
- 05.6Pacing a concurrent collector: finishing before the heap runs out
- 05.7Measuring collection: pauses, mutator utilisation and hidden costs
- 05.8Choosing a collector: throughput, pause and footprint
- 06.1Escape analysis: allocations that never reach the heap
- 06.2Ownership and moves: memory management as a compile-time rule
- 06.3Borrowing: shared xor mutable, checked by the compiler
- 06.4Shared ownership: counting and run-time checks return
- 06.5Arenas and regions: freeing a whole lifetime at once
- 06.6Counting, tracing, ownership or regions: choosing a memory strategy
- 07.1Stackful coroutines: where a million stacks live
- 07.2Work-stealing task schedulers: local deques and a provable bound
- 07.3The Go scheduler: why G, M and P are three things
- 07.4Preempting user-level tasks: cooperative checks, signals and reductions
- 07.5Virtual threads: continuations under an unchanged blocking API
- 08.1async/await as a compiler transform: functions into state machines
- 08.2Poll or callback: who drives a future
- 08.3Self-referential futures: why a suspended state must not move
- 08.4The JavaScript event loop: tasks, microtasks and run-to-completion
- 08.5Stackful or stackless: function colour and the concurrency-model trade-off
- 09.1Channels: communicating sequential processes in a runtime
- 09.2Actors: addresses, mailboxes and no shared state
- 09.3The BEAM: per-process heaps and preemption by reductions
- 09.4Supervision: let it crash, and restart to a known state
- 09.5Structured concurrency: task lifetimes nested like blocks
- 09.6Cancellation: stopping a task without corrupting state
- 11.1Compiler reordering: why threads cannot be a library
- 11.2Happens-before and data races inside one program
- 11.3Data-race-free programs are sequentially consistent
- 11.4The Java memory model: bounding what a racy program may do
- 11.5C++ and Rust atomics: orderings as a language contract
- 11.6The Go memory model: DRF-SC with bounded races and channel edges
- 11.7Detecting data races: vector clocks over one execution
- 11.8Memory models compared: catch-fire, bounded races and race-free by construction
- 12.1Case study: CPython — counting, a lock and an ecosystem that fixes both
- 12.2Case study: the JVM (HotSpot) — speculation, a menu of collectors and virtual threads
- 12.3Case study: V8 and Node — four tiers on one thread
- 12.4Case study: Go — AOT, a non-moving concurrent collector and a scheduler in the runtime
- 12.5Case study: Rust and Tokio — no runtime in the language, one in a library
- 12.6Case study: the BEAM — isolation first, speed second
- 12.7Choosing a runtime for a workload
01.2Interpreter dispatch: switch, threaded code and the branch predictor
01.3Stack versus register virtual machines: fewer instructions or smaller code
01.4Representing dynamic values: boxing, tagged pointers and NaN-boxing
02.1Attribute lookup: why obj.x is a walk through hash tables
02.2Hidden classes: giving dynamic objects a static layout
02.3Inline caches: remembering the last answer at each call site
02.4Quickening: an interpreter that rewrites its own bytecode
02.5Why dynamic languages are slow: an overhead budget
03.1Tiered execution: when a method is worth compiling
03.2Baseline JITs: removing dispatch without optimising
03.3Speculative optimisation: compiling for the types you have seen
03.4Deoptimisation: rebuilding an interpreter frame from optimised code
03.5Tracing JITs: compiling the path the loop actually takes
03.6Meta-compilation: a JIT derived from an interpreter
03.7WebAssembly: a VM designed to be validated and compiled, not speculated
03.8JIT or AOT: peak performance, warm-up and startup
04.1Reachability, roots and safepoints: what a collector must find
04.2Reference counting: immediate reclamation, paid on every pointer write
04.3Cycle collection: finding garbage that counts cannot see
04.4Mark-sweep: trace from the roots, free the rest
04.5Copying collection: cost proportional to what survives
04.6Mark-compact and mark-region: defragmenting in place
04.7Allocation fast paths: bump pointers and thread-local buffers
04.8Finalizers and weak references: when death must do something
04.9Tracing and counting are duals: a unified view of collection
05.1Generational collection: most objects die young
05.2Write barriers: card tables and remembered sets
05.3Concurrent marking: the tri-colour invariant and its barriers
05.4Moving objects under a running program: read barriers and coloured pointers
05.5Region-based collection: choosing what to evacuate within a pause target
05.6Pacing a concurrent collector: finishing before the heap runs out
05.7Measuring collection: pauses, mutator utilisation and hidden costs
05.8Choosing a collector: throughput, pause and footprint
06.1Escape analysis: allocations that never reach the heap
06.2Ownership and moves: memory management as a compile-time rule
06.3Borrowing: shared xor mutable, checked by the compiler
06.4Shared ownership: counting and run-time checks return
06.5Arenas and regions: freeing a whole lifetime at once
06.6Counting, tracing, ownership or regions: choosing a memory strategy
07.1Stackful coroutines: where a million stacks live
07.2Work-stealing task schedulers: local deques and a provable bound
07.3The Go scheduler: why G, M and P are three things
07.4Preempting user-level tasks: cooperative checks, signals and reductions
07.5Virtual threads: continuations under an unchanged blocking API
08.1async/await as a compiler transform: functions into state machines
08.2Poll or callback: who drives a future
08.3Self-referential futures: why a suspended state must not move
08.4The JavaScript event loop: tasks, microtasks and run-to-completion
08.5Stackful or stackless: function colour and the concurrency-model trade-off
09.1Channels: communicating sequential processes in a runtime
09.2Actors: addresses, mailboxes and no shared state
09.3The BEAM: per-process heaps and preemption by reductions
09.4Supervision: let it crash, and restart to a known state
09.5Structured concurrency: task lifetimes nested like blocks
09.6Cancellation: stopping a task without corrupting state
10.1The native extension API: a runtime's hardest contract
10.2The GIL: what it protects, and what it never did
10.3Isolated interpreters: a lock per interpreter, not per process
10.4Free-threaded Python: removing the GIL without slowing one thread
11.1Compiler reordering: why threads cannot be a library
11.2Happens-before and data races inside one program
11.3Data-race-free programs are sequentially consistent
11.4The Java memory model: bounding what a racy program may do
11.5C++ and Rust atomics: orderings as a language contract
11.6The Go memory model: DRF-SC with bounded races and channel edges
11.7Detecting data races: vector clocks over one execution
11.8Memory models compared: catch-fire, bounded races and race-free by construction
12.1Case study: CPython — counting, a lock and an ecosystem that fixes both
12.2Case study: the JVM (HotSpot) — speculation, a menu of collectors and virtual threads
12.3Case study: V8 and Node — four tiers on one thread
12.4Case study: Go — AOT, a non-moving concurrent collector and a scheduler in the runtime
12.5Case study: Rust and Tokio — no runtime in the language, one in a library
12.6Case study: the BEAM — isolation first, speed second
12.7Choosing a runtime for a workload
Skipped topics
Moved past without finishing. A skipped topic still closes its module, and Continue goes on without it — open one to read it or mark it understood.
Glossary
Terms as the lessons define them, each linked to the lesson that defines it.
Tree-walk interpreter
A tree-walk interpreter executes the syntax tree directly.
Go to Bytecode: flattening the tree into an instruction stream →Environment (variable binding)
Variables live in an environment: a hash map from names to values, chained to the enclosing scope's map.
Go to Bytecode: flattening the tree into an instruction stream →resolution pass
Compute it in a pre-execution resolution pass: one traversal visiting every node once — in the node count — with no control flow and no side effects, seeing both sides of a branch and a loop body once.
Go to Bytecode: flattening the tree into an instruction stream →bytecode
The deeper fix removes the tree from the execution path entirely — compile each subtree once, in the same post-order the walker evaluated, and emit the result as bytecode: a dense, linear array of instructions.
Go to Bytecode: flattening the tree into an instruction stream →opcode
An instruction starts with a one-byte opcode naming the operation — in many formats the instruction is only a single byte long, hence "bytecode" — followed by the operand bytes that operation needs.
Go to Bytecode: flattening the tree into an instruction stream →constant pool
Constants too big for operand bytes, like a string literal, live beside the code in the chunk's constant pool, an array of values the load instruction indexes with one byte; the JVM keeps a constant pool per class in the same spirit.
Go to Bytecode: flattening the tree into an instruction stream →operand stack
An operand stack carries every intermediate value with no pointers between them.
Go to Bytecode: flattening the tree into an instruction stream →virtual machine
What executes the stream is a virtual machine: a software emulation of an idealized processor whose instruction set is the bytecode.
Go to Bytecode: flattening the tree into an instruction stream →Instruction pointer (VM)
Its core is the same three-phase loop the hardware runs — fetch the next instruction, decode which operation it names, execute it — and the state it needs mirrors hardware's: an instruction pointer always aimed at the next byte to read, plus the operand stack for temporaries.
Go to Bytecode: flattening the tree into an instruction stream →Dispatch
Dispatch is everything the interpreter does between instructions: advance the instruction pointer, decode the next opcode, and transfer control to the code that performs it.
Go to Interpreter dispatch: switch, threaded code and the branch predictor →switch dispatch
The portable way to write the loop is a switch dispatch: a giant switch inside for (;;), one case per opcode.
Go to Interpreter dispatch: switch, threaded code and the branch predictor →jump table
The compiler turns it into a jump table — an array of code addresses indexed by opcode:
Go to Interpreter dispatch: switch, threaded code and the branch predictor →Direct threaded code
Direct threaded code removes the table lookup by storing code addresses directly in the instruction stream: a VM instruction is the address of its handler.
Go to Interpreter dispatch: switch, threaded code and the branch predictor →Indirect threaded code
Indirect threaded code stores not the handler address but a pointer to a per-instruction record whose first field is that address — one extra load per dispatch, a small price for keeping the instruction stream relocatable.
Go to Interpreter dispatch: switch, threaded code and the branch predictor →token threading
For interpreters written in plain C, where the bytecode must stay a stream of opcode bytes, the same effect is reached by token threading: keep the jump table, but give every handler its own jump through it — written with the GNU C labels-as-values extension as a computed goto:
Go to Interpreter dispatch: switch, threaded code and the branch predictor →computed goto
For interpreters written in plain C, where the bytecode must stay a stream of opcode bytes, the same effect is reached by token threading: keep the jump table, but give every handler its own jump through it — written with the GNU C labels-as-values extension as a computed goto:
Go to Interpreter dispatch: switch, threaded code and the branch predictor →Superinstructions
Superinstructions attack the same tax from the other side: pre-fuse sequences of bytecodes that always appear together (compare-then-branch) into one VM instruction, so the loop runs fewer times per unit of real work and the fused body's branches become ordinary, predictable static branches.
Go to Interpreter dispatch: switch, threaded code and the branch predictor →Replication
Replication goes the other direction — duplicate handler code so that each replica's dispatch site sees a smaller, more predictable successor set.
Go to Interpreter dispatch: switch, threaded code and the branch predictor →stack virtual machine
A stack virtual machine answers by convention: operands are always the top few entries of an operand stack, so no instruction needs to say where they are.
Go to Stack versus register virtual machines: fewer instructions or smaller code →register virtual machine
A register virtual machine removes the second array.
Go to Stack versus register virtual machines: fewer instructions or smaller code →Value representation
Value representation is the encoding that decides where the type tag lives, what it costs per operation, and how much of the word the payload keeps.
Go to Representing dynamic values: boxing, tagged pointers and NaN-boxing →tagged pointer
A tagged pointer spends those bits as the type tag: if the low bit is set the word is a pointer (mask the tag away and dereference); if it is clear the word is not a pointer at all but an immediate value stored in the remaining bits.
Go to Representing dynamic values: boxing, tagged pointers and NaN-boxing →Smi
V8 calls its tagged small integer a Smi.
Go to Representing dynamic values: boxing, tagged pointers and NaN-boxing →quiet NaN
The all-ones exponent does not denote a number but a class: any pattern with all exponent bits set and a nonzero fraction is a NaN, and one whose top fraction bit is also set is a quiet NaN — a NaN guaranteed to propagate quietly through arithmetic rather than trap.
Go to Representing dynamic values: boxing, tagged pointers and NaN-boxing →NaN-boxing
NaN-boxing moves into that space.
Go to Representing dynamic values: boxing, tagged pointers and NaN-boxing →Pointer compression
Pointer compression confines all of an isolate's heap objects to one contiguous 4 GiB region, the cage, whose base is 4-GiB-aligned; a stored pointer is then just the 32-bit offset within the cage.
Go to Representing dynamic values: boxing, tagged pointers and NaN-boxing →method resolution order
First, the class hierarchy is itself a row of dictionaries: each class's tp_dict holds its methods and class attributes, and the search order across them is the method resolution order (MRO) — the linearization that lists the class, then its bases, before object.
Go to Attribute lookup: why obj.x is a walk through hash tables →descriptor
Second, a class attribute is not necessarily a passive value: an object whose type defines __get__ (and optionally __set__ or __delete__) is a descriptor — its __get__ is invoked with the instance instead of returning the object itself.
Go to Attribute lookup: why obj.x is a walk through hash tables →data descriptor
A descriptor that also defines __set__ or __delete__ — a property, say — is a data descriptor, and it takes priority over everything the instance itself might hold.
Go to Attribute lookup: why obj.x is a walk through hash tables →method cache
The method cache is a 4096-entry direct-mapped table indexed by the type's version tag combined with the interned name's address: any resolved or conclusively-absent name is replayed in one compare, and a class mutation bumps the type's version tag and invalidates its entries.
Go to Attribute lookup: why obj.x is a walk through hash tables →compact ordered dictionary
Raymond Hettinger's observation (implemented as the compact ordered dictionary in CPython 3.6) splits the array in two: a sparse index of small integers in the range , and a dense entries array holding exactly the live (key, value) pairs in insertion order.
Go to Attribute lookup: why obj.x is a walk through hash tables →Key-sharing dictionaries
Key-sharing dictionaries (PEP 412, CPython 3.3) split the table at exactly that seam: the keys object — index array plus the dense (key, hash-source) entries — lives once on the type object (ht_cached_keys), and each instance's dictionary stores only a dense values array, a row of pointers in the same order the shared keys prescribe.
Go to Attribute lookup: why obj.x is a walk through hash tables →values array
Key-sharing dictionaries (PEP 412, CPython 3.3) split the table at exactly that seam: the keys object — index array plus the dense (key, hash-source) entries — lives once on the type object (ht_cached_keys), and each instance's dictionary stores only a dense values array, a row of pointers in the same order the shared keys prescribe.
Go to Attribute lookup: why obj.x is a walk through hash tables →hidden class
The shared, immutable descriptor that records it is a hidden class — SELF's map, V8's Map, JavaScriptCore's Structure, SpiderMonkey's Shape — and it is what turns a dictionary read into an indexed load.
Go to Hidden classes: giving dynamic objects a static layout →transition tree
Because an edge is found-or-created, never duplicated, every object that replays the same additions from the same root lands on the same leaf, and the leaves form a transition tree.
Go to Hidden classes: giving dynamic objects a static layout →elements kind
The shape still records one fact about the elements: their elements kind.
Go to Hidden classes: giving dynamic objects a static layout →inline cache
An inline cache stores the last answer inside the instruction itself, so a repeated shape costs one compare instead of a lookup.
Go to Inline caches: remembering the last answer at each call site →monomorphic inline cache
This is a monomorphic cache — one remembered answer per site.
Go to Inline caches: remembering the last answer at each call site →polymorphic inline cache
The polymorphic inline cache (PIC, Hölzle–Chambers–Ungar 1991) generalises the site from one remembered answer to all of them: on a miss, the handler generates a small stub routine that tests the receiver's shape against each seen so far and jumps to the matching method (or, for a property load, reads the matching slot).
Go to Inline caches: remembering the last answer at each call site →megamorphic
Informally, call sites show a trimodal distribution: most are monomorphic, a meaningful minority are polymorphic with a few receiver types — typically under ten — and a rare few are megamorphic, touching very many types.
Go to Inline caches: remembering the last answer at each call site →Type feedback
Type feedback is the name for this doubled life: the cache that makes the interpreter fast is simultaneously the evidence the later tiers compile against.
Go to Inline caches: remembering the last answer at each call site →Quickening
Quickening closes that gap by rewriting the instruction itself: a generic LOAD_ATTR becomes, in place, a guarded specialist like LOAD_ATTR_INSTANCE_VALUE that already knows which case it is.
Go to Quickening: an interpreter that rewrites its own bytecode →dynamic locality
The bet underneath is Deutsch and Schiffman's observation from the Smalltalk-80 work — dynamic locality of type usage: at any single instruction occurrence, operand types almost never change across executions, and their inline caches stayed effective about 95% of the time.
Go to Quickening: an interpreter that rewrites its own bytecode →inline cache entries
An instruction that can specialize reserves extra code units immediately after itself: the inline cache entries, zeroed by the compiler and owned by the site.
Go to Quickening: an interpreter that rewrites its own bytecode →invocation counter
Attach an increment-and-compare at each: an invocation counter that ticks on entry, and a back-edge counter that ticks on each branch to an earlier point.
Go to Tiered execution: when a method is worth compiling →back-edge counter
Attach an increment-and-compare at each: an invocation counter that ticks on entry, and a back-edge counter that ticks on each branch to an earlier point.
Go to Tiered execution: when a method is worth compiling →ladder of tiers
That band is the whole reason real VMs run a ladder of tiers — interpreter, then a baseline tier, then an optimising tier — each step buying code one notch faster for methods one notch hotter, each with its own counter and its own θ.
Go to Tiered execution: when a method is worth compiling →on-stack replacement
The fix is on-stack replacement: compile the loop itself, then on the next back-edge translate the live frame — locals, operand-stack slots, program counter — into the compiled frame's layout and resume mid-iteration in machine code.
Go to Tiered execution: when a method is worth compiling →baseline JIT
That is a baseline JIT — the tier that removes dispatch without optimising.
Go to Baseline JITs: removing dispatch without optimising →template JIT
Because each emitted fragment is effectively a template of the handler, the design is also called a template JIT.
Go to Baseline JITs: removing dispatch without optimising →Copy-and-patch compilation
Copy-and-patch compilation (Xu & Kjolstad, OOPSLA 2021) answers: reuse the compiler's own output.
Go to Baseline JITs: removing dispatch without optimising →binary stencil
Each handler is compiled once at library-build time into a binary stencil — object code with holes where run-time values must go: literal constants, stack-frame offsets, jump targets to neighbouring stencils.
Go to Baseline JITs: removing dispatch without optimising →Supernode
Stencils come in variants (operand in register versus stack, operand versus constant) and as supernodes, fused fragments for common instruction shapes, all generated from shared C++ templates — the paper's WebAssembly library carries 1,666 stencils in 35 kB; its expression-language library, built with heavy supernode coverage, carries 98,831 in 17.5 MB.
Go to Baseline JITs: removing dispatch without optimising →continuation-passing style
Stencils pass control to the next stencil as a tail call — continuation-passing style at the machine level — which Clang lowers to a jump, and a stencil laid down right after its caller makes even that jump removable.
Go to Baseline JITs: removing dispatch without optimising →Speculative optimisation
Speculative optimisation is the tier that trusts the record: compile a hot method as if the types it has observed were facts, fence each assumption with one cheap guard, and let the fast path run as if the checks had passed at compile time.
Go to Speculative optimisation: compiling for the types you have seen →guard
Speculative optimisation is the tier that trusts the record: compile a hot method as if the types it has observed were facts, fence each assumption with one cheap guard, and let the fast path run as if the checks had passed at compile time.
Go to Speculative optimisation: compiling for the types you have seen →feedback vector
In V8 the record sits in a feedback vector hung off the closure — one slot per interesting site, so the Add bytecode knows its operands have only ever been Smis, and each property-load IC knows which shape it saw and which slot paid out.
Go to Speculative optimisation: compiling for the types you have seen →uncommon branch elimination
So the cold case is emitted as a separate copy that never merges back: Hölzle's uncommon branch elimination.
Go to Speculative optimisation: compiling for the types you have seen →Class-hierarchy analysis
Class-hierarchy analysis answers it once, at compile time: if m has exactly one implementation across the loaded hierarchy, a send to a C-typed receiver compiles to that body — a direct call or an inline — with no test that could ever fire.
Go to Speculative optimisation: compiling for the types you have seen →dependency link
SELF kept dependency links between compiled code and the source objects it relied on — change an inlined method and the compiled copy is discarded, or deoptimised if live; a JVM's class-hierarchy optimisation records the identical edge, and class loading walks it.
Go to Speculative optimisation: compiling for the types you have seen →deoptimisation
The machinery that performs that translation — machine registers and stack slots back into source-level activations — is deoptimisation.
Go to Deoptimisation: rebuilding an interpreter frame from optimised code →virtual activation
Frames: inlining merged several calls into one activation, so a single physical frame can hold the work of many virtual activations — the source-level frames the program believes are live — and a deopt must give each one a real frame of its own.
Go to Deoptimisation: rebuilding an interpreter frame from optimised code →scope descriptor
In Hölzle's SELF design (Hölzle, Chambers and Ungar, PLDI 1992) — the ancestor of every modern scheme — each scope contained in a compiled method (the outer method plus everything inlined into it) gets a scope descriptor: which method the virtual activation runs, which scope it was inlined into and at which send, plus one descriptor per argument, per local, and per evaluated-but-unconsumed expression on the operan…
Go to Deoptimisation: rebuilding an interpreter frame from optimised code →interrupt point
SELF made the set explicit: optimised code may be interrupted only at interrupt points — method prologues, loop back-edges, and every site that can raise an error — with pending events deferred to the next one, a latency bounded by the longest straight-line stretch, typically a few dozen instructions.
Go to Deoptimisation: rebuilding an interpreter frame from optimised code →lazy deoptimisation
Lazy deoptimisation: the frame that must go sits underneath live activations, so the runtime patches the return address of the frame it called and lets normal returns do the waiting — when control would re-enter the doomed frame, it lands in the transform routine instead, with the frame now topmost and expandable.
Go to Deoptimisation: rebuilding an interpreter frame from optimised code →tracing JIT compiler
Both are empirical claims about real dynamic-language workloads, and both are the working assumptions a tracing JIT compiler is built on — first explored by HP's Dynamo project for machine code, then applied to bytecode interpreters.
Go to Tracing JITs: compiling the path the loop actually takes →trace
Recording then begins: the interpreter executes one iteration as usual, but every operation it performs is also written to a linear instruction log, the trace.
Go to Tracing JITs: compiling the path the loop actually takes →type map
The entry condition is the type map — the types the live values had when recording began — and a stored trace is callable only when the current program point and type map match it.
Go to Tracing JITs: compiling the path the loop actually takes →side exit
The VM takes a side exit — a branch out of the compiled fragment to a small stub that reconstructs the interpreter's state at exactly that program point.
Go to Tracing JITs: compiling the path the loop actually takes →snapshots
This is deoptimisation by another name, and the machinery is the same: each guard carries metadata — LuaJIT calls these records snapshots, emitted sparsely and compressed — saying how to rebuild every interpreter local and stack slot from the values the trace holds.
Go to Tracing JITs: compiling the path the loop actually takes →branch trace
Each side exit keeps its own counter, and once it crosses the hotness threshold the VM records a branch trace starting at that exit and running until it rejoins the flow — usually at the same loop header.
Go to Tracing JITs: compiling the path the loop actually takes →trace stitching
The original exit is then patched to jump straight to the branch trace instead of falling out to the interpreter, a splice called trace stitching.
Go to Tracing JITs: compiling the path the loop actually takes →trace tree
One recorded path thereby grows into a trace tree: single-entry at the loop header, multiple-exit at its guards, covering every path through the loop that is hot enough to matter.
Go to Tracing JITs: compiling the path the loop actually takes →blacklisting
Some operations could not be recorded at all (eval, exceptions, some native calls), aborting every trace that touched them; the engine answered with blacklisting — back off after a failed recording (32 crossings), give up after two failures — and even patched the loop-header bytecode to a no-op so a blacklisted loop never paid the monitor call again.
Go to Tracing JITs: compiling the path the loop actually takes →meta-tracing
PyPy's move — meta-tracing — is to run that machinery on the language interpreter itself.
Go to Meta-compilation: a JIT derived from an interpreter →green variable
The JIT cannot guess which interpreter variables form the user program's program counter, so the author marks them — the green variables (pc, bytecode) are declared part of the trace's position key, the identity that tells the JIT which recorded loop a trace belongs to; everything else (regs, the accumulator) is red — ordinary traced state, free to differ each lap.
Go to Meta-compilation: a JIT derived from an interpreter →position key
The JIT cannot guess which interpreter variables form the user program's program counter, so the author marks them — the green variables (pc, bytecode) are declared part of the trace's position key, the identity that tells the JIT which recorded loop a trace belongs to; everything else (regs, the accumulator) is red — ordinary traced state, free to differ each lap.
Go to Meta-compilation: a JIT derived from an interpreter →fallback interpreter
PyPy's answer is a fallback interpreter: a second variant of the language interpreter, compiled without tracing, which runs until a safe resumption point — always reachable after at most one bytecode operation.
Go to Meta-compilation: a JIT derived from an interpreter →partial evaluator
A partial evaluator — classically named mix — takes a program and a known subset of its inputs and produces the residual program that computes the remaining inputs.
Go to Meta-compilation: a JIT derived from an interpreter →mix
A partial evaluator — classically named mix — takes a program and a known subset of its inputs and produces the residual program that computes the remaining inputs.
Go to Meta-compilation: a JIT derived from an interpreter →first Futamura projection
Futamura stated this in 1971 as the first Futamura projection:
Go to Meta-compilation: a JIT derived from an interpreter →structured control flow
Instead there are structured control flow constructs — block, loop, if/else, each closed by end and properly nested — and branch instructions br, br_if, br_table that name a target by nesting depth: label 0 is the innermost enclosing construct, label 1 the next one out.
Go to WebAssembly: a VM designed to be validated and compiled, not speculated →linear memory
The program's store is a linear memory: one large byte array, addressed in the 1.0 format by unsigned 32-bit integers (WebAssembly 3.0 adds an opt-in 64-bit memory, where the trick below no longer applies), grown in pages of 64 KiB — a size chosen as the least common multiple of minimum page sizes on modern hardware so the scheme below can lean on real page protection.
Go to WebAssembly: a VM designed to be validated and compiled, not speculated →effective address
The effective address is computed in 33 bits — deliberately without wraparound — and the access traps unless .
Go to WebAssembly: a VM designed to be validated and compiled, not speculated →warm-up
Compile at run time and the program pays warm-up — interpretation, profiling and compile time spent before peak is reached — but keeps the right to speculate: a guard plus a deopt routine makes an assumption recoverable, so the compiler may use what it merely believes.
Go to JIT or AOT: peak performance, warm-up and startup →startup snapshot
A startup snapshot does not freeze code; it freezes an initialised world — the heap, class metadata, the parsed framework — so a run begins mid-way through its own initialisation.
Go to JIT or AOT: peak performance, warm-up and startup →image heap
GraalVM pushes furthest: run initialisers at image build time and ship the resulting objects as an image heap, copy-on-write mapped so every process and isolate shares the read-only pages — the same page-sharing a prefork server gets for free when workers inherit a parent's already-warm memory via copy-on-write.
Go to JIT or AOT: peak performance, warm-up and startup →Profile-guided optimisation
Profile-guided optimisation instruments a special build, runs representative workloads, and feeds the recorded profile into the optimised build — speculation captured offline and frozen in.
Go to JIT or AOT: peak performance, warm-up and startup →reachable
Something must decide which objects are dead — and the honest question, "will this object be accessed again?", is undecidable, so the collector answers a different one it can decide: is this object reachable from a root the program can dereference next?
Go to Reachability, roots and safepoints: what a collector must find →mutator
The mutator is the application — everything that allocates and stores pointers.
Go to Reachability, roots and safepoints: what a collector must find →collector
The collector is the runtime machinery that reclaims storage.
Go to Reachability, roots and safepoints: what a collector must find →root
A root is any pointer into the graph that the mutator holds outside it — in a register, a stack slot, a global — and can dereference next.
Go to Reachability, roots and safepoints: what a collector must find →root set
The root set is not a place but an inventory taken per thread at collection time.
Go to Reachability, roots and safepoints: what a collector must find →false retention
An integer that happens to equal an object start is treated as a pointer — false retention, memory kept alive by an accident of bit patterns.
Go to Reachability, roots and safepoints: what a collector must find →stack map
At every program point where a collection may begin, the compiler emits a stack map — HotSpot's name is an OopMap — a per-frame table giving, for that PC, each register's and stack slot's disposition: live object reference, dead, callee-saved (with the caller register it restores), or a derived pointer paired with its base.
Go to Reachability, roots and safepoints: what a collector must find →GC safepoint
The PCs the compiler described are the safepoints: allocation sites, calls and returns, and loop back-edges.
Go to Reachability, roots and safepoints: what a collector must find →safepoint poll
At each safepoint the compiler inserts a cheap safepoint poll: the interpreter checks on branching and returning bytecodes, while compiled code reads a per-thread polling word — or a polling page the VM arms to fault.
Go to Reachability, roots and safepoints: what a collector must find →time to safepoint
To start a collection the VM arms the poll, then waits for the slowest thread to reach one; that wait is the time to safepoint.
Go to Reachability, roots and safepoints: what a collector must find →Reference counting
Reference counting answers the same question locally — each object tallies the pointers that name it, and the decrement that takes the tally to zero frees the object at once, inside the pointer store that killed it.
Go to Reference counting: immediate reclamation, paid on every pointer write →write barrier
Every pointer write then runs a small write barrier: increment the new target first, then decrement the old one.
Go to Reference counting: immediate reclamation, paid on every pointer write →deferred reference counting
So deferred counting simply ignores stack and register references between collections and makes the count mean what it can mean: the number of references from other heap objects.
Go to Reference counting: immediate reclamation, paid on every pointer write →allocate-as-dead
Shahriyar et al.'s deferred, coalesced counter adds two young-object plays: lazy mod-buffer insertion (a "new" bit in the header replaces logging at allocation) and allocate-as-dead — objects are born with count zero and only become live if a first collection finds a reference to them, so the common case of dying young costs no count work at all.
Go to Reference counting: immediate reclamation, paid on every pointer write →RC Immix
RC Immix therefore counts in a tracing collector's heap: the line-and-block layout of the Immix mark-region design, which the mark-compact topic derives later with Immix's original 128-byte lines.
Go to Reference counting: immediate reclamation, paid on every pointer write →stuck count
A stuck count: a count field that saturates can never be decremented again (Deutsch and Bobrow's own design had few-bit counts with the same property), so a hot object quietly leaks until a backup trace recomputes the field.
Go to Reference counting: immediate reclamation, paid on every pointer write →mark bit
The oldest reply is still the baseline: leave every object where it lies, set a mark bit on each one the walk reaches, and afterwards make one linear pass over the heap — everything without a bit is garbage and goes back to the allocator.
Go to Mark-sweep: trace from the roots, free the rest →Mark-sweep collection
Mark-sweep collection is McCarthy's 1960 design for Lisp, and its two costs point in opposite directions: marking pays for what is live, sweeping pays for all of it.
Go to Mark-sweep: trace from the roots, free the rest →mark phase
The mark phase is the traversal the previous topic derived, with one addition: each object it first reaches gets a mark so the traversal terminates and so the sweep can ask later "did the walk reach you?".
Go to Mark-sweep: trace from the roots, free the rest →worklist
The mechanism worth deriving is how the walk is run — as an explicit worklist (usually called the mark stack), not as recursion.
Go to Mark-sweep: trace from the roots, free the rest →mark stack
The mechanism worth deriving is how the walk is run — as an explicit worklist (usually called the mark stack), not as recursion.
Go to Mark-sweep: trace from the roots, free the rest →mark bitmap
The alternative is a mark bitmap: a side bit vector, one bit per allocation granule — the heap's smallest allocatable unit, typically a word or two — held in a contiguous array per heap block.
Go to Mark-sweep: trace from the roots, free the rest →granule
The alternative is a mark bitmap: a side bit vector, one bit per allocation granule — the heap's smallest allocatable unit, typically a word or two — held in a contiguous array per heap block.
Go to Mark-sweep: trace from the roots, free the rest →sweep phase
The sweep phase has the less glamorous job, and the reason it exists is the definition of the problem: marking finds live objects positively, by following edges, but a dead object is characterised only by the absence — nothing points at it, so nothing leads to it.
Go to Mark-sweep: trace from the roots, free the rest →size-segregated free list
The pass is deliberately dull: walk the heap start to end, clear the bit on anything marked, and thread anything unmarked into a free list — usually several size-segregated free lists, one per size class, so allocation searches only its own class.
Go to Mark-sweep: trace from the roots, free the rest →Lazy sweeping
Lazy sweeping (Hughes's form) leaves the unswept blocks on a per-class reclamation list and sweeps only what allocation asks for: a request of size class sweeps blocks from 's list until 's free list is nonempty, then allocates.
Go to Mark-sweep: trace from the roots, free the rest →Copying collection
Copying collection asks a stranger question: what if the collector never looked at the dead at all?
Go to Copying collection: cost proportional to what survives →semispace
Split the managed heap into two equal semispaces.
Go to Copying collection: cost proportional to what survives →from-space
Only one is ever live: the from-space, in which the mutator allocates; the other, the to-space, sits empty and reserved.
Go to Copying collection: cost proportional to what survives →to-space
Only one is ever live: the from-space, in which the mutator allocates; the other, the to-space, sits empty and reserved.
Go to Copying collection: cost proportional to what survives →forwarding pointer
One operation carries the correctness: the forwarding pointer.
Go to Copying collection: cost proportional to what survives →Mark-compact collection
Mark-compact collection asks whether both escapes are avoidable at once — can the survivors be moved inside the space they already occupy, leaving one contiguous free region and no reserved half?
Go to Mark-compact and mark-region: defragmenting in place →Compaction
Compaction is the family of answers, and its simplest member that handles variable-sized objects is sliding compaction: preserve the allocation order of the survivors and slide them toward one end of the heap, so the live prefix grows from the bottom and the free region is whatever is left at the top.
Go to Mark-compact and mark-region: defragmenting in place →sliding compaction
Compaction is the family of answers, and its simplest member that handles variable-sized objects is sliding compaction: preserve the allocation order of the survivors and slide them toward one end of the heap, so the live prefix grows from the bottom and the free region is whatever is left at the top.
Go to Mark-compact and mark-region: defragmenting in place →mark-region collector
A mark-region collector divides the heap into fixed-size regions, marks any region containing a live object as unavailable, and hands the rest back whole — reclamation at region grain, with a bump allocator inside each free region.
Go to Mark-compact and mark-region: defragmenting in place →Immix block
The heap is a sequence of blocks of 32 KB, each divided into lines of 128 B — 256 lines per block.
Go to Mark-compact and mark-region: defragmenting in place →Immix line
The heap is a sequence of blocks of 32 KB, each divided into lines of 128 B — 256 lines per block.
Go to Mark-compact and mark-region: defragmenting in place →line map
The collector records liveness in a line map: one byte per line plus four per block, 260 bytes per block — under 1% of the heap.
Go to Mark-compact and mark-region: defragmenting in place →hole (Immix)
Reclamation is still two-grained: entirely free blocks return to a global pool, but a block holding even a few live lines is not lost — its runs of free lines, its holes, are recycled for allocation.
Go to Mark-compact and mark-region: defragmenting in place →opportunistic evacuation
Immix's answer is opportunistic evacuation — compaction folded into the mark pass, applied only to the blocks that need it and only while the space budget holds.
Go to Mark-compact and mark-region: defragmenting in place →pinned object
Two escape valves keep the "opportunistic" honest: a pinned object (or one reached after the evacuation budget ran out) is simply marked in place, no correctness lost; and a small reserve — 2.5% of the heap by default, insensitive between 1 and 3% — is held back from allocation as compaction headroom so the estimates almost always fit.
Go to Mark-compact and mark-region: defragmenting in place →compaction headroom
Two escape valves keep the "opportunistic" honest: a pinned object (or one reached after the evacuation budget ran out) is simply marked in place, no correctness lost; and a small reserve — 2.5% of the heap by default, insensitive between 1 and 3% — is held back from allocation as compaction headroom so the estimates almost always fit.
Go to Mark-compact and mark-region: defragmenting in place →Sequential allocation
Sequential allocation — the bump pointer — is the only allocation scheme that fits the budget, and the Garbage Collection Handbook's first design for exactly this reason.
Go to Allocation fast paths: bump pointers and thread-local buffers →thread-local allocation buffer
Thread-caching allocators put a per-thread cache in front of the shared heap so the common case takes no lock; the managed version is a thread-local allocation buffer (TLAB) — a private slice of the free region that only its owning thread bumps.
Go to Allocation fast paths: bump pointers and thread-local buffers →segregated-fits allocation
The managed-heap version of the fix is segregated-fits allocation: a family of free lists, one per size class — small classes at fine granularity (a word or two apart), larger classes spaced wider, commonly toward powers of two — so a request is rounded up to a class and answered by popping that class's list head.
Go to Allocation fast paths: bump pointers and thread-local buffers →finalization
Two mechanisms grow out of that: finalization, which lets the program act at an object's death, and weak references, which let it hold an object without keeping it.
Go to Finalizers and weak references: when death must do something →weak reference
Two mechanisms grow out of that: finalization, which lets the program act at an object's death, and weak references, which let it hold an object without keeping it.
Go to Finalizers and weak references: when death must do something →resurrection
That is resurrection, and it is legal in every runtime that offers object-resident finalization.
Go to Finalizers and weak references: when death must do something →reference queue
Any weak or phantom reference can be registered with a reference queue: when the collector clears the reference it enqueues the reference object — the container, not the referent.
Go to Finalizers and weak references: when death must do something →ephemeron
Hayes's ephemeron is that kind: a pair whose key edge is weak and whose value edge is conditional — the value is traced only if the key is already known reachable.
Go to Finalizers and weak references: when death must do something →least fixed point
That is the least fixed point, and the objects with positive count are the live set — the same set reachability defined by closure under the roots.
Go to Tracing and counting are duals: a unified view of collection →greatest fixed point
When it drains, the counts satisfy the equation maximally: the greatest fixed point.
Go to Tracing and counting are duals: a unified view of collection →Generational collection
Generational collection organises the heap around that observation — the weak generational hypothesis that most objects die young — so the frequent, cheap collections go where the dying is and the heap-proportional ones become rare.
Go to Generational collection: most objects die young →weak generational hypothesis
Generational collection organises the heap around that observation — the weak generational hypothesis that most objects die young — so the frequent, cheap collections go where the dying is and the heap-proportional ones become rare.
Go to Generational collection: most objects die young →nursery
The nursery is a small space where every object is born.
Go to Generational collection: most objects die young →minor collection
When it fills, a minor collection — a scavenge, in Ungar's word — runs exactly Cheney's algorithm on that space alone: live young objects are copied into a survivor space, and the exhausted space, garbage and all, is abandoned unvisited.
Go to Generational collection: most objects die young →old generation
The old generation is the rest of the heap, collected only by a rare major collection — the whole-heap mark-sweep or mark-compact the previous module priced — which is where the long pause now lives, amortised over the objects that earned it.
Go to Generational collection: most objects die young →major collection
The old generation is the rest of the heap, collected only by a rare major collection — the whole-heap mark-sweep or mark-compact the previous module priced — which is where the long pause now lives, amortised over the objects that earned it.
Go to Generational collection: most objects die young →promotion
Each young object carries an age, the count of scavenges it has survived; when the age reaches a tenuring threshold , its next copy lands in the old generation — promotion (Ungar's term is tenuring).
Go to Generational collection: most objects die young →tenuring
Each young object carries an age, the count of scavenges it has survived; when the age reaches a tenuring threshold , its next copy lands in the old generation — promotion (Ungar's term is tenuring).
Go to Generational collection: most objects die young →remembered set
The collector maintains a remembered set — the old objects that currently hold a pointer into the nursery — and scans it as extra roots at each scavenge.
Go to Generational collection: most objects die young →Premature promotion
Premature promotion drags them into the old generation, where they sit uncollectable until the next major collection — dead bytes inflating the live heap and hastening the rare expensive pause.
Go to Generational collection: most objects die young →card (garbage collection)
Partition the heap into fixed-size cards — conventionally 512 bytes — and keep a contiguous side array of one byte per card: the card table.
Go to Write barriers: card tables and remembered sets →card table
Partition the heap into fixed-size cards — conventionally 512 bytes — and keep a contiguous side array of one byte per card: the card table.
Go to Write barriers: card tables and remembered sets →crossing map
The second is a lookup called a crossing map (or card-crossing table): objects do not align to card boundaries, so the first bytes of a dirty card may be the tail of an object that began in the previous card; the table records, per card, where the first object header at or before the card's start lies, which turns "scan the dirty card" into a walkable sequence of slots.
Go to Write barriers: card tables and remembered sets →sequential store buffer
Set insertion is far too dear to run on every store — insert-plus-dedup is a call, not a couple of instructions — so precise barriers record into a staging structure first: the sequential store buffer, a bump-pointer log of interesting slots.
Go to Write barriers: card tables and remembered sets →Tri-colour marking
Tri-colour marking partitions every object into three sets: white objects have not been reached — they are provisionally condemned; grey objects are reached but have unscanned children — they are exactly the worklist; black objects are reached and fully scanned — the collector is done with them and never looks again.
Go to Concurrent marking: the tri-colour invariant and its barriers →Strong tri-colour invariant
The maintained property is the strong tri-colour invariant: no black object ever points to a white object.
Go to Concurrent marking: the tri-colour invariant and its barriers →snapshot-at-the-beginning
Marking then covers everything reachable when the mark began, unchanged by mid-mark writes: a snapshot-at-the-beginning (SATB) semantics — Dijkstra's mutator operated under its opposite, an incremental update of the collector's view.
Go to Concurrent marking: the tri-colour invariant and its barriers →Weak tri-colour invariant
What it maintains is the weak tri-colour invariant: a black object may point to a white object — but only while that object is reachable from some grey object through an all-white chain; Pirinen names the white object grey-protected.
Go to Concurrent marking: the tri-colour invariant and its barriers →floating garbage
The relaxed invariant permits black→white edges mid-mark, and that relaxation is the cost model: objects that die during the cycle are still marked and survive it — floating garbage, memory held one cycle too long, the price of never consulting the live shape mid-trace.
Go to Concurrent marking: the tri-colour invariant and its barriers →read barrier
The operation that must be made safe is the pointer load — and that requirement, the read barrier, is the entire design space of concurrent compaction.
Go to Moving objects under a running program: read barriers and coloured pointers →Brooks forwarding pointer
Give each heap object one extra word — a Brooks forwarding pointer or indirection word — that always points at the object's current body; when the object is created it points just past itself.
Go to Moving objects under a running program: read barriers and coloured pointers →coloured pointer
A ZGC reference is a 64-bit coloured pointer: address bits that locate the object plus a few meta bits — the colour — recording what the collector currently knows about it.
Go to Moving objects under a running program: read barriers and coloured pointers →multi-maps
The metadata bits sit at bit positions 42–45 — inside the virtual address — and ZGC multi-maps the heap: the same physical pages are mapped at three virtual ranges, one per colour that selects a view (the mmap lesson's "one physical range at several addresses", put to work).
Go to Moving objects under a running program: read barriers and coloured pointers →load barrier
ZGC calls its read barrier a load barrier: injected wherever compiled code loads a reference field, it checks whether the loaded pointer's colour is currently good.
Go to Moving objects under a running program: read barriers and coloured pointers →soft real-time goal
The premise to be precise about first: G1 takes a soft real-time goal — in the 2004 paper's formulation, no more than ms of stop-the-world collection in any ms time slice; in JDK 21, a maximum pause-time hint of -XX:MaxGCPauseMillis (200 ms by default) per GCPauseIntervalMillis. "Soft" is the operative word: the goal is met with high probability over the run, never with certainty for a given …
Go to Region-based collection: choosing what to evacuate within a pause target →heap region
So divide the heap into a set of equal-sized heap regions — each a contiguous range of virtual memory, sized ergonomically in JDK 21 to about 2,048 regions and clamped to 1–512 MB (a -XX:G1HeapRegionSize option pins it).
Go to Region-based collection: choosing what to evacuate within a pause target →humongous object
One object larger than the region cannot be allowed to break the scheme: a humongous object — one at least half a region's size in JDK 21 (the 2004 paper used three-quarters) — gets its own dedicated sequence of contiguous regions in the old generation, and the tail space of the last region is simply lost to allocation until the object dies.
Go to Region-based collection: choosing what to evacuate within a pause target →GC efficiency
Rank the candidates by GC efficiency — the estimated garbage a region yields per predicted millisecond it costs:
Go to Region-based collection: choosing what to evacuate within a pause target →mixed collection
After a mark finishes, a pause may also take discretionary old regions — a partially-young pause in the paper's terms, a mixed collection in JDK 21 — while the budget allows, harvesting whatever the mark identified as cheap to collect.
Go to Region-based collection: choosing what to evacuate within a pause target →collection set candidate
Remark and Cleanup complete it; Cleanup also chooses the collection set candidates — old regions likely to repay collection — preferring little live data and low connectivity, and dropping any region whose reclaimable space is under G1HeapWastePercent (5% of heap) or that is more than G1MixedGCLiveThresholdPercent (85%) live.
Go to Region-based collection: choosing what to evacuate within a pause target →pacer
Between cycles the collector does nothing at all; the algorithm that picks where on the ramp it wakes is called the pacer, and pacing is the whole subject of this lesson.
Go to Pacing a concurrent collector: finishing before the heap runs out →heap goal
The heap goal is the size the heap is allowed to reach before a finished cycle resets it — live heap plus of overhead.
Go to Pacing a concurrent collector: finishing before the heap runs out →assist ratio
The runtime's answer is to make allocation itself pay: while a mark is active, every byte a goroutine allocates accrues a debt of bytes of scan work, where is the assist ratio.
Go to Pacing a concurrent collector: finishing before the heap runs out →mark assist
The goroutine settles it by doing that much marking inside its next allocation — a mark assist — or by spending credit that the dedicated mark workers bank in a global pool as they work.
Go to Pacing a concurrent collector: finishing before the heap runs out →soft memory limit
A workload whose live heap spikes to fill a container gets a goal proportioned to the spike, so the ceiling must be handed to the runtime directly: GOMEMLIMIT, or debug.SetMemoryLimit since Go 1.19, a soft limit on total runtime memory.
Go to Pacing a concurrent collector: finishing before the heap runs out →GC death spiral
As live data approaches the usable limit, runway shrinks toward zero, cycles run back to back, and GC utilization climbs until the mutator starves — the death spiral.
Go to Pacing a concurrent collector: finishing before the heap runs out →utilisation
Give the mutator a utilisation inside any window — the fraction of the window it is running — then take the worst case: the minimum mutator utilisation is the minimum utilisation over all windows of size .
Go to Measuring collection: pauses, mutator utilisation and hidden costs →Minimum mutator utilisation
Give the mutator a utilisation inside any window — the fraction of the window it is running — then take the worst case: the minimum mutator utilisation is the minimum utilisation over all windows of size .
Go to Measuring collection: pauses, mutator utilisation and hidden costs →intrinsic cost
Define the intrinsic cost of a workload as the ideal cost of running it — including the best benefit any GC could bring (better locality, say) but none of the GC's costs.
Go to Measuring collection: pauses, mutator utilisation and hidden costs →distilled cost
It is not measurable directly, but it can be overestimated: subtract from a run's total cost the GC costs that are explicitly identifiable — the cycles burned inside STW pauses, where no mutator runs — and the remainder, the distilled cost, is still at least the intrinsic cost.
Go to Measuring collection: pauses, mutator utilisation and hidden costs →Minimum distilled cost
Run the same workload under several collectors and take the smallest distilled cost: the minimum distilled cost (MDC) is the best overestimate of intrinsic available.
Go to Measuring collection: pauses, mutator utilisation and hidden costs →LBO
Then each collector's LBO — lower bound on its absolute GC overhead — is its total cost minus the MDC, and the NLBO normalises by the MDC.
Go to Measuring collection: pauses, mutator utilisation and hidden costs →NLBO
Then each collector's LBO — lower bound on its absolute GC overhead — is its total cost minus the MDC, and the NLBO normalises by the MDC.
Go to Measuring collection: pauses, mutator utilisation and hidden costs →Escape analysis
Escape analysis turns this into a small lattice of states: NoEscape — the object dies inside its allocating method; ArgEscape — it escapes the method, but only by becoming reachable through a parameter (the caller's object, or a new object stored into a parameter's field), never by another thread; GlobalEscape — it may outlive everything, anywhere.
Go to Escape analysis: allocations that never reach the heap →connection graph
To answer the question mechanically, Choi's analysis builds a connection graph: one object node per allocation site, a reference node per variable and field, and edges recording where references can travel — a points-to edge when a variable names an object, a deferred edge when an assignment merely copies a reference between variables, a field edge linking an object to its field nodes.
Go to Escape analysis: allocations that never reach the heap →parameter tag
The compiler runs the flow analysis bottom-up over a package's functions and records, for each pointer parameter, where it can leak — to the heap, through a write, as a call's target, or into a result — as parameter tags saved into the compiled package's export data, so every static call site knows what its callee might do with an argument.
Go to Escape analysis: allocations that never reach the heap →scalar replacement
The production JVM answer is not a stack-allocated object at all: scalar replacement dissolves the object — each field becomes a local variable that register allocation or the frame absorbs as ordinary data, so nothing object-shaped remains: no header, no bump, nothing for the collector to find.
Go to Escape analysis: allocations that never reach the heap →lock elision
The same verdict buys a third optimisation for free: a NoEscape object can never be observed by another thread, so lock elision removes its synchronization — in that trace the synchronized(key) block disappears along with the object.
Go to Escape analysis: allocations that never reach the heap →Partial escape analysis
Partial escape analysis — Stadler, Würthinger and Mössenböck's 2014 algorithm in Graal — iterates the control-flow graph keeping a per-allocation state instead of a per-method verdict.
Go to Escape analysis: allocations that never reach the heap →materialisation
When an instruction needs the object to exist — here, the store cacheKey = key on the miss branch — the allocation and its field stores are emitted at that point, a step the algorithm calls materialisation.
Go to Escape analysis: allocations that never reach the heap →ownership
Rust's ownership system is that idea enforced as a rule.
Go to Ownership and moves: memory management as a compile-time rule →drop
Each value in Rust has exactly one owner — the variable binding that holds it; there is only ever one owner at a time; and when the owner goes out of scope, the value is dropped, its destructor run and its storage reclaimed.
Go to Ownership and moves: memory management as a compile-time rule →move
After let s2 = s1, memory looks like a shallow copy — the three-word triple lands in s2's slot and the heap bytes never move — but the compiler has marked s1 as no longer owning.
Go to Ownership and moves: memory management as a compile-time rule →deterministic
Because the free site is static, destruction is deterministic — a guarantee neither run-time mechanism gives.
Go to Ownership and moves: memory management as a compile-time rule →RAII
Under ownership the destructor is an ordinary call emitted at a brace you can point to, and C++ programmers already know the pattern as RAII — resource acquisition is initialisation: the value's destructor runs where the name dies.
Go to Ownership and moves: memory management as a compile-time rule →drop flag
Move a value on only one branch of an if and whether the binding still owns at its scope end is a run-time fact — so Rust tracks it with a hidden drop flag, a boolean per binding toggled as the value is initialised and uninitialised, consulted wherever a drop is conditional.
Go to Ownership and moves: memory management as a compile-time rule →Copy
An i32 is four bytes held entirely in the binding — copying its bits duplicates the whole value — so it implements Copy: all integer and floating-point types, bool, char, and tuples built entirely of Copy members.
Go to Ownership and moves: memory management as a compile-time rule →affine type
A linear type must be used exactly once; an affine type may be used at most once — weakening is allowed (a value may die unused, dropped at its scope end) but contraction is not (it may never be duplicated).
Go to Ownership and moves: memory management as a compile-time rule →moved-from state
C++11 introduced the same operation and chose differently at the same junction. std::string s2 = std::move(s1) pilfers the heap buffer just as a Rust move does — but s1 remains a live object in a valid but unspecified state: its class invariants hold, its destructor still runs at scope end, and every member function with no preconditions on the value — size(), empty(), clear(), assignment — stays legal.
Go to Ownership and moves: memory management as a compile-time rule →reference
The loan's instrument is the reference: an address you can follow to a value owned by someone else.
Go to Borrowing: shared xor mutable, checked by the compiler →borrowing
Taking a reference is called borrowing; like real borrowing it ends, and the thing borrowed was never yours to destroy.
Go to Borrowing: shared xor mutable, checked by the compiler →shared reference
A shared reference &T permits reads, and it is Copy — it can be duplicated freely and handed out again, which is exactly why it cannot permit writes: a pointer you can copy can never be the only route to the data.
Go to Borrowing: shared xor mutable, checked by the compiler →mutable reference
A mutable reference &mut T permits writes — and is deliberately not Copy — so it can promise what a copyable pointer never could: exclusivity.
Go to Borrowing: shared xor mutable, checked by the compiler →reborrow
The holder of a reference can lend it onwards — a reborrow like &mut *x creates a borrow of a borrow that must end before its parent's lifetime does; while the child loan is live the parent is unusable, like a lock taken on the parent's access.
Go to Borrowing: shared xor mutable, checked by the compiler →aliasing rule
The first half is the aliasing rule; the second half is the lifetime check, and both are decided by the same machinery.
Go to Borrowing: shared xor mutable, checked by the compiler →lifetime
Every borrow carries a lifetime — the set of program points where the loan is enforced, where writing the referent or moving its owner is forbidden.
Go to Borrowing: shared xor mutable, checked by the compiler →Non-lexical lifetimes
Non-lexical lifetimes (RFC 2094; shipped for edition 2018 in Rust 1.31, all editions in 1.36, and the only checker since 1.63) redefine a lifetime as a region of the control-flow graph — the minimal set of points satisfying every constraint the program's uses impose.
Go to Borrowing: shared xor mutable, checked by the compiler →interior pointer
No iterator invalidation: a pointer into a container's storage — an interior pointer, in the proof model's vocabulary — is a borrow of that container; while it is live the invariant forbids the mutation that would move the buffer.
Go to Borrowing: shared xor mutable, checked by the compiler →two-phase borrow
The accepted fix is the two-phase borrow: the mutable borrow is only reserved until its first write, so it may overlap reads harmlessly — starting from [1, 2, 3], v becomes [1, 2, 3, 3].
Go to Borrowing: shared xor mutable, checked by the compiler →Stacked Borrows
Stacked Borrows (2020) tracked the live pointers to each location as a stack: a legal run uses them in well-nested order, and the optimizer's uniqueness assumptions follow.
Go to Borrowing: shared xor mutable, checked by the compiler →Tree Borrows
Tree Borrows (PLDI 2025) replaces the stack with the shape borrows actually make: a tree of tags per allocation — every new reference is a child of the reference it was derived from — with a small permission machine at each location, run once per tag:
Go to Borrowing: shared xor mutable, checked by the compiler →protector
Function-argument references are protected for the call's duration — a protected tag takes UB instead of merely losing permissions, which is what makes fn(x: &mut i32) genuinely mean "nobody else may touch it while I run".
Go to Borrowing: shared xor mutable, checked by the compiler →Rc<T>
In Rust that is Rc<T>: a smart pointer holding a strong count beside the value.
Go to Shared ownership: counting and run-time checks return →Arc<T>
The cross-thread form is Arc<T> — the same counting, with every update made atomic, so two threads incrementing at once cannot merge into one increment.
Go to Shared ownership: counting and run-time checks return →Weak<T>
Weak<T> is a non-owning edge into the same object: Rc::downgrade(&a) produces one, bumping a separate weak count that plays no part in liveness.
Go to Shared ownership: counting and run-time checks return →Interior mutability
The rule they relaxed was "one owner"; the rule still standing is "shared xor mutable." Interior mutability relaxes the second rule by moving the same check — many readers or one writer — from compile time to run time, behind a type that enforces it dynamically.
Go to Shared ownership: counting and run-time checks return →RefCell<T>
RefCell<T> is the single-threaded form. borrow() and borrow_mut() return checked handles — Ref/RefMut — and the cell counts active borrows at run time: a second borrow_mut while one is live, or a borrow while a borrow_mut is live, is not a compiler error but a panic.
Go to Shared ownership: counting and run-time checks return →marker trait
Rust's answer is not a rule in the borrow checker but two marker traits in the type system: Send — "this value may move to another thread" — and Sync — "a shared &T may be used from another thread," or equivalently T is Sync iff &T is Send.
Go to Shared ownership: counting and run-time checks return →Send
Rust's answer is not a rule in the borrow checker but two marker traits in the type system: Send — "this value may move to another thread" — and Sync — "a shared &T may be used from another thread," or equivalently T is Sync iff &T is Send.
Go to Shared ownership: counting and run-time checks return →Sync
Rust's answer is not a rule in the borrow checker but two marker traits in the type system: Send — "this value may move to another thread" — and Sync — "a shared &T may be used from another thread," or equivalently T is Sync iff &T is Send.
Go to Shared ownership: counting and run-time checks return →arena
This topic derives the allocator that frees the lifetime instead: the arena, or region.
Go to Arenas and regions: freeing a whole lifetime at once →lifetime group
Assign each object, at creation, to the lifetime group it will die with — and free the group.
Go to Arenas and regions: freeing a whole lifetime at once →region
Every storable value lives in a region, and the store is literally a stack of them: a program runs letregion ρ in e end — push a fresh region , evaluate , then pop and free wholesale.
Go to Arenas and regions: freeing a whole lifetime at once →region inference
Region inference is the typed version of choosing a lifetime group, and the compiler chooses it, not the programmer: unification puts two values in the same region whenever their types must agree, each expression carries an effect recording the regions it reads () and writes (), and a region that leaves no observable footprint gets wrapped in its own letregion — the mas…
Go to Arenas and regions: freeing a whole lifetime at once →region polymorphism
A function callable at many sites gets a region-polymorphic signature — the paper's example types a successor function as , reading an int wherever you keep it and writing the answer into a fresh region.
Go to Arenas and regions: freeing a whole lifetime at once →guard page
At the far end sits a guard page — a protected page the overflow check trips on, so a stack that runs off its reservation faults as a clean SIGSEGV or a catchable StackOverflowError instead of silently writing into whatever memory follows.
Go to Stackful coroutines: where a million stacks live →stack segments
A goroutine's stack was a linked list of small fixed-size stack segments, allocated from a pool — a few KiB apiece (the design document's experiments used the 4 KiB default).
Go to Stackful coroutines: where a million stacks live →hot split
Keith Randall's 2013 design document names the failure: the hot split.
Go to Stackful coroutines: where a million stacks live →stack chunk objects
A running virtual thread's frames live on its carrier's stack like any other; when the task unmounts — blocked on a socket, a queue take — the frames are copied into stack chunk objects, a linked chain of heap blocks that grow and shrink with depth.
Go to Stackful coroutines: where a million stacks live →ready deque
Each worker keeps a ready deque of suspended threads and follows four rules:
Go to Work-stealing task schedulers: local deques and a provable bound →thief
To steal, an idle worker — the thief — picks a uniformly random victim and takes its topmost thread.
Go to Work-stealing task schedulers: local deques and a provable bound →victim
To steal, an idle worker — the thief — picks a uniformly random victim and takes its topmost thread.
Go to Work-stealing task schedulers: local deques and a provable bound →parallel slackness
Its parallel slackness on workers is , the parallelism left over per worker.
Go to Work-stealing task schedulers: local deques and a provable bound →critical
An instruction on the computation's leftmost unfinished path — a critical instruction — sits at or near the top of some deque, by the ancestor-chain lemma.
Go to Work-stealing task schedulers: local deques and a provable bound →P (scheduler processor)
Everything a thread needs in order to run Go code — but does not need to be — moves onto a third object, P, which the source describes as "a resource that is required to execute Go code." The whole contract is one line: an M must hold a P to run Go code, and an M that is idle — or stuck in a system call for long — gives its P up.
Go to The Go scheduler: why G, M and P are three things →runnext
Each P owns a 256-slot local run queue — the per-worker queue from the previous topic — plus a single runnext slot, a mcache (the allocation cache that used to be welded to the thread), a timer heap, and scheduler statistics like the schedtick counter.
Go to The Go scheduler: why G, M and P are three things →network poller
The network poller wraps epoll (or its platform equivalent) exactly once — a single shared epoll file descriptor for the whole process.
Go to The Go scheduler: why G, M and P are three things →sysmon
Every guarantee so far that isn't self-enforcing routes through one watcher. sysmon is a thread with no P — it cannot run goroutines, which is exactly why it can watch the scheduler: a monitor that needed a P would starve alongside everything it monitors.
Go to The Go scheduler: why G, M and P are three things →Cooperative preemption
Cooperative preemption puts the stop where the compiler already works: the yield check is folded into a check the code was going to run anyway.
Go to Preempting user-level tasks: cooperative checks, signals and reductions →asynchronous safe-point
Go's shipped answer, per the conservative inner-frame sub-proposal: an asynchronous safe-point is any user-code PC where a conservative scan of the innermost frame and the saved registers can find the roots — the outer frames already carry precise maps.
Go to Preempting user-level tasks: cooperative checks, signals and reductions →reduction
A reduction is at least one per Erlang function call — the name descends from Prolog's goal-reduction — and the count is the slot.
Go to Preempting user-level tasks: cooperative checks, signals and reductions →continuation
A continuation is a suspended sequential computation reified as a Java object — the instruction pointer plus the live frames that resume it.
Go to Virtual threads: continuations under an unchanged blocking API →virtual thread
A virtual thread is a java.lang.Thread whose task body runs inside one; the JDK's own scheduler assigns it to a carrier — an ordinary platform thread — by mounting it: the continuation's code runs on the carrier's stack — and after the first unmount, each re-mount copies the saved frames back on, lazily: HotSpot thaws a few frames at a time and plants a return barrier that thaws more only when execution returns in…
Go to Virtual threads: continuations under an unchanged blocking API →carrier thread
A virtual thread is a java.lang.Thread whose task body runs inside one; the JDK's own scheduler assigns it to a carrier — an ordinary platform thread — by mounting it: the continuation's code runs on the carrier's stack — and after the first unmount, each re-mount copies the saved frames back on, lazily: HotSpot thaws a few frames at a time and plants a return barrier that thaws more only when execution returns in…
Go to Virtual threads: continuations under an unchanged blocking API →pinning
The runtime calls this pinning: the virtual thread is mounted but cannot unmount, so the wait holds the carrier and its OS thread for the duration.
Go to Virtual threads: continuations under an unchanged blocking API →suspension point
An await marks a suspension point: a call the function must leave if the operation is not yet complete, and re-enter later at the statement after.
Go to async/await as a compiler transform: functions into state machines →async state machine
A local needed after the suspension point becomes a field of a generated object — the async state machine.
Go to async/await as a compiler transform: functions into state machines →coroutine frame
C++20 allocates a coroutine frame — dynamically allocated storage, via operator new, when the coroutine begins — holding the promise object, a copy of every parameter, the suspension-point descriptor, and the locals and temporaries whose lifetimes span a suspend; the frame is freed when the coroutine finishes or is destroyed.
Go to async/await as a compiler transform: functions into state machines →callback model
The callback model — JavaScript's promises, C#'s tasks — has the suspended side register a continuation: a record saying "when the value exists, run this handler on it".
Go to Poll or callback: who drives a future →poll model
The poll model — Rust's futures — registers no continuation at all.
Go to Poll or callback: who drives a future →executor
The suspended machine exposes one method, poll; an executor calls it, and a machine that cannot finish returns a marker meaning "not ready — ask again", but only after arming a handle that will tell the executor when asking is worthwhile.
Go to Poll or callback: who drives a future →waker
The poll model needs none of that: the machine that resumes is the same object that suspended, so the only new information is when to poll again, and a single waker handle answers it.
Go to Poll or callback: who drives a future →reaction record
A promise holds a state — pending, fulfilled or rejected — a result slot, and two lists of reaction records, one per direction of settlement.
Go to Poll or callback: who drives a future →eager promise
That is the eager convention; C# makes the same choice — an async method executes synchronously until its first await, so the returned task is already running before anyone could subscribe to it.
Go to Poll or callback: who drives a future →derived promise
Subscribing is separate from starting. p.then(f) does not touch the operation p represents: it mints a fresh derived promise plus its capability record, wraps f and the absent rejection handler into two reaction records, and appends them to p's lists — or, if p has already settled, enqueues a job immediately.
Go to Poll or callback: who drives a future →self-referential
The generated future is self-referential: one of its fields references another field of the same value.
Go to Self-referential futures: why a suspended state must not move →Unpin
A type that is Unpin is not address-sensitive — moving it is harmless — and Unpin is an auto trait, so almost every type is Unpin automatically; for them Pin<&mut T> dereferences mutably and behaves exactly like &mut T.
Go to Self-referential futures: why a suspended state must not move →address-sensitive
A type that is Unpin is not address-sensitive — moving it is harmless — and Unpin is an auto trait, so almost every type is Unpin automatically; for them Pin<&mut T> dereferences mutably and behaves exactly like &mut T.
Go to Self-referential futures: why a suspended state must not move →structural pinning
The friction shows up the moment code touches a pinned object directly: With an ordinary &mut TWith Pin<&mut T> Pass the reference twice; the compiler reborrows implicitlyNo reborrow — call .as_mut() to get a fresh Pin each time Assign through it: *p = vAssign with Pin::set, which drops the old pointee first Project to a field freely: &mut s.fieldNeeds unsafe map_unchecked, or the pin-project/pin…
Go to Self-referential futures: why a suspended state must not move →job
The unit ECMAScript schedules is a job: an abstract closure with no parameters that initiates an ECMAScript computation "when no other ECMAScript computation is currently in progress." The host decides when queued jobs run — that part is deliberately unspecified — but the spec fixes three guarantees around every job, whatever the host:
Go to The JavaScript event loop: tasks, microtasks and run-to-completion →run to completion
The third clause is the run to completion guarantee, and it is why JavaScript programmers never write locks.
Go to The JavaScript event loop: tasks, microtasks and run-to-completion →task queue
The coarse queue is the task queue, and there can be several per event loop.
Go to The JavaScript event loop: tasks, microtasks and run-to-completion →task
The coarse queue is the task queue, and there can be several per event loop.
Go to The JavaScript event loop: tasks, microtasks and run-to-completion →task source
A task is a struct — steps, a task source, an associated document — and the ordering guarantee is narrower than it first appears: order is FIFO within a task source (every timer fires in schedule order), but the loop itself chooses which task queue to draw from on each turn, in an implementation-defined manner.
Go to The JavaScript event loop: tasks, microtasks and run-to-completion →microtask
The fine queue is the microtask queue, and the spec is blunt about the difference: "the microtask queue is not a task queue." There is exactly one per event loop, it drains in strict FIFO, and it fills from three places: queueMicrotask, MutationObserver delivery, and every promise reaction.
Go to The JavaScript event loop: tasks, microtasks and run-to-completion →microtask checkpoint
Each turn of the loop: pick a task queue with a runnable task, run that task's steps to completion, then perform a microtask checkpoint — dequeue and run microtasks until the queue is empty, including microtasks enqueued during the drain.
Go to The JavaScript event loop: tasks, microtasks and run-to-completion →rendering opportunity
A window event loop runs a second sequence in parallel with the one above: wait until some navigable has a rendering opportunity — decided by the user agent from hardware refresh rate, visibility and throttling, typically at most once per refresh (~16.7 ms at 60 Hz, and a page that cannot sustain 60 may be dropped to 30 or fewer) — then queue a global task on the rendering task source.
Go to The JavaScript event loop: tasks, microtasks and run-to-completion →phase (Node)
Node runs the identical job contract but a heavier loop, implemented by libuv as a ring of phases, each with its own FIFO callback queue: timers (setTimeout/setInterval callbacks whose thresholds elapsed) → pending callbacks (deferred system I/O callbacks) → idle, prepare (internal) → poll → check → close callbacks → back to timers.
Go to The JavaScript event loop: tasks, microtasks and run-to-completion →fiber
A function calls perform (In_line ic) with no marking on itself or any caller; the handler upstack captures the continuation — a fiber, a small heap-allocated stack — and resumes it when the I/O completes.
Go to Stackful or stackless: function colour and the concurrency-model trade-off →communicating sequential process
A communicating sequential process is an ordinary sequential program; a parallel command runs several of them at once, under one hard rule — the processes are disjoint: none may mention a variable another assigns.
Go to Channels: communicating sequential processes in a runtime →rendezvous
And there is no buffered form at all: every communication is a rendezvous, two parties meeting or neither moving.
Go to Channels: communicating sequential processes in a runtime →channel
A channel is that object: a value any number of goroutines may send to and receive from without further synchronization, carrying values of one element type. make(chan T) builds an unbuffered channel — the pure rendezvous; make(chan T, C) builds a buffered channel, a bounded queue of capacity bolted under the same interface.
Go to Channels: communicating sequential processes in a runtime →buffered channel
A channel is that object: a value any number of goroutines may send to and receive from without further synchronization, carrying values of one element type. make(chan T) builds an unbuffered channel — the pure rendezvous; make(chan T, C) builds a buffered channel, a bounded queue of capacity bolted under the same interface.
Go to Channels: communicating sequential processes in a runtime →sudog
An entry on a wait queue is a sudog: a pseudo-g that exists because the g-to-wait-list relation is many-to-many — one goroutine can be parked on several wait lists (a select), and one channel can hold many parked goroutines — so each waiting site gets its own record holding the goroutine and, crucially, a pointer to the slot the value should land in or come from.
Go to Channels: communicating sequential processes in a runtime →actor
In 1973 Hewitt, Bishop and Steiger proposed collapsing every computational object — functions, data structures, semaphores, ports, processes — into a single kind of thing: the actor.
Go to Actors: addresses, mailboxes and no shared state →Task (actor model)
Communication travels inside a task, a triple of a tag that distinguishes it from every other task, a target — a mail address — and the communication itself.
Go to Actors: addresses, mailboxes and no shared state →mail address
Communication travels inside a task, a triple of a tag that distinguishes it from every other task, a target — a mail address — and the communication itself.
Go to Actors: addresses, mailboxes and no shared state →replacement behaviour
An actor is a mail address plus a behaviour: when it accepts a communication it may do at most three things — send a finite set of new communications, create a finite set of new actors, and compute a replacement behaviour that answers the next communication.
Go to Actors: addresses, mailboxes and no shared state →mail queue
It buffers communications and serialises arrivals per target: the order in which communications arrive at a given actor is a linear order, recorded on that target's mail queue.
Go to Actors: addresses, mailboxes and no shared state →activation ordering
There is no global clock either: each agent's local time orders its own events, and events at different agents are related only through causal links Agha calls the activation ordering, making the global order of events a partial order.
Go to Actors: addresses, mailboxes and no shared state →selective receive
The model's queue is FIFO in arrival order, but Erlang's receive need not take the front: it accepts selective receive, returning the first message from the start of the queue that matches one of its patterns, and leaves every earlier non-matching message queued.
Go to Actors: addresses, mailboxes and no shared state →Process control block
A fresh process owns a process control block (PCB — registers, stack pointer, program counter, bookkeeping), a combined stack and heap region, an internal mailbox of messages it has scanned, an external in-queue of messages not yet delivered, and a pool of m-bufs — heap fragments holding incoming message data.
Go to The BEAM: per-process heaps and preemption by reductions →m-buf
A fresh process owns a process control block (PCB — registers, stack pointer, program counter, bookkeeping), a combined stack and heap region, an internal mailbox of messages it has scanned, an external in-queue of messages not yet delivered, and a pool of m-bufs — heap fragments holding incoming message data.
Go to The BEAM: per-process heaps and preemption by reductions →flat size
The sender computes the receiver-side footprint of Msg — its flat size, the space a deep copy needs — then chooses a destination: if the receiver is suspended and its main lock is free and its heap has room, the term is copied straight onto P2's heap (the on_heap strategy); otherwise it lands in an m-buf allocated by the sender.
Go to The BEAM: per-process heaps and preemption by reductions →refc binary
Two payloads escape the per-byte term: a refc binary (payload > 64 B lives once in the shared binary allocator; the send copies only its ProcBin handle and bumps a refcount) and a literal on the same node (shared until its module unloads).
Go to The BEAM: per-process heaps and preemption by reductions →ProcBin
Two payloads escape the per-byte term: a refc binary (payload > 64 B lives once in the shared binary allocator; the send copies only its ProcBin handle and bumps a refcount) and a literal on the same node (shared until its module unloads).
Go to The BEAM: per-process heaps and preemption by reductions →heap binary
Binaries of 64 B or less are heap binaries — ordinary heap data, copied like anything else.
Go to The BEAM: per-process heaps and preemption by reductions →link set
Every process in its link set receives an exit signal {'EXIT', Pid, Why}, where Why is the exception's reason.
Go to Supervision: let it crash, and restart to a known state →Link (Erlang)
Every process in its link set receives an exit signal {'EXIT', Pid, Why}, where Why is the exception's reason.
Go to Supervision: let it crash, and restart to a known state →system process
A process that evaluates process_flag(trap_exit, true) opts out of dying: it becomes a system process, and its exit signals arrive as ordinary mailbox messages it can act on.
Go to Supervision: let it crash, and restart to a known state →monitor
The monitor supplies the one-directional watch: erlang:monitor(process, B) makes A receive {'DOWN', Ref, process, B, Why} when B dies, with no link, no trapping and no shared fate.
Go to Supervision: let it crash, and restart to a known state →exception
He splits the cases precisely — an exception is raised when the run-time system does not know what to do, an error is when the programmer does not know what to do.
Go to Supervision: let it crash, and restart to a known state →error
If some other process fixes the error, what should the code that hit it do?
Go to Supervision: let it crash, and restart to a known state →worker
A worker does the task and may fail; a supervisor is a system process that traps exits, watches its workers and restarts them.
Go to Supervision: let it crash, and restart to a known state →supervisor
A worker does the task and may fail; a supervisor is a system process that traps exits, watches its workers and restarts them.
Go to Supervision: let it crash, and restart to a known state →child specification
The child specification is the whole contract: how to start the process — a start_link, so every supervised child is born linked to its supervisor — when to restart it, and how to stop it.
Go to Supervision: let it crash, and restart to a known state →restart strategy
Which other children a death takes with it is the restart strategy, and Armstrong's AND/OR tree is the exact ancestor: an OR supervisor treats its children as independent — a death restarts the dead child and nothing else; an AND supervisor's children are coordinated, so one death takes down and restarts the whole set.
Go to Supervision: let it crash, and restart to a known state →restart intensity
OTP's bound is the restart intensity: a supervisor records a timestamp for each restart it performs, and if the restarts inside the last period seconds would exceed intensity, it does not perform that restart — it terminates all its children and exits itself, with reason shutdown.
Go to Supervision: let it crash, and restart to a known state →error kernel
Armstrong's design rule is to identify that part of the system deliberately — the error kernel, often a small real-time memory-resident database holding hardware state — and to declare which functions are expected never to raise.
Go to Supervision: let it crash, and restart to a known state →Well-behaved function
A well-behaved function is the name he gives it: an exception inside a WBF is defined to be an error — the deviation between observed and specified behaviour — and is written to the permanent error log.
Go to Supervision: let it crash, and restart to a known state →structured concurrency
The repair is structured concurrency — a term coined by Martin Sústrik and popularised by Smith's essay, with Erlang's supervision trees credited as the ancestor of its error handling.
Go to Structured concurrency: task lifetimes nested like blocks →task group
The construct that enforces it is a scope for children: Trio calls it a nursery (async with trio.open_nursery() as nursery), Python a task group, Java a StructuredTaskScope, Kotlin a coroutine scope, Swift a task group.
Go to Structured concurrency: task lifetimes nested like blocks →cooperative cancellation
For everything with shared state, cancellation must instead be cooperative cancellation: the task stops itself, at program points where its invariants are already intact.
Go to Cancellation: stopping a task without corrupting state →cancellation point
POSIX's deferred default contains the whole idea in one sentence: if a thread is cancelable, its type is deferred, and a request is pending, the thread is cancelled when it calls a function that is a cancellation point.
Go to Cancellation: stopping a task without corrupting state →cancel token
That object is the cancel token, which Smith traces to Joe Duffy's cancellation work in C#. .NET ships it as a split pair: the owner of the policy holds a CancellationTokenSource; each operation receives the lightweight CancellationToken it vends and may pass copies onward.
Go to Cancellation: stopping a task without corrupting state →level-triggered
That is the level-triggered property: the cancelled state stays true for every later check, including checks inside cleanup code.
Go to Cancellation: stopping a task without corrupting state →edge-triggered
The edge-triggered alternative — the interrupt — is an event consumed once, and Smith's contrast makes the difference concrete.
Go to Cancellation: stopping a task without corrupting state →cancel scope
A cancel scope is a with block — with trio.move_on_after(10): — that pushes a token onto an invisible per-task stack; every blocking primitive Trio provides consults the stack before suspending and raises Cancelled if any visible scope has fired.
Go to Cancellation: stopping a task without corrupting state →cancel safety
Tokio names the property a branch must have to survive that loop: a future is cancellation safe if, whenever it has not completed, dropping it and creating it again is a no-op.
Go to Cancellation: stopping a task without corrupting state →global interpreter lock
The lock that protects every interpreter structure is called the global interpreter lock — the GIL.
Go to The GIL: what it protects, and what it never did →switch interval
At v3.14.7 the GIL is a small structure per interpreter (Python/ceval_gil.c): a boolean locked, guarded by gil->mutex and signalled on the condition variable gil->cond — the condition-variable pair from Operating Systems, doing lock hand-off duty; a holder's identity last_holder and a switch_number incremented on every change of ownership; and a timeout, the switch interval, default 5000 µs, readable an…
Go to The GIL: what it protects, and what it never did →drop request
Only after a full switch interval without a release does the waiter set the drop request — a bit in the holder's per-thread eval breaker (eval_breaker) word, which the eval loop checks at frame entry (RESUME_CHECK) and on every backward jump (JUMP_BACKWARD) — at least once per loop iteration.
Go to The GIL: what it protects, and what it never did →eval breaker
Only after a full switch interval without a release does the waiter set the drop request — a bit in the holder's per-thread eval breaker (eval_breaker) word, which the eval loop checks at frame entry (RESUME_CHECK) and on every backward jump (JUMP_BACKWARD) — at least once per loop iteration.
Go to The GIL: what it protects, and what it never did →Free-threaded Python
Free-threaded Python (PEP 703; experimental in 3.13, officially supported in 3.14 under PEP 779) replaces the lock with a chain of narrower mechanisms, each chosen against a specific cost the naive removal creates.
Go to Free-threaded Python: removing the GIL without slowing one thread →Biased reference counting
Biased reference counting (BRC) encodes that ownership in the header.
Go to Free-threaded Python: removing the GIL without slowing one thread →merge
The missing operation is a merge: fold the local count into the shared count with a compare-and-swap, clear ob_tid, and let the shared count alone decide from then on.
Go to Free-threaded Python: removing the GIL without slowing one thread →immortal
The cheapest count update is the one never performed, so PEP 703 makes the permanently-hot set immortal: ob_ref_local parked at UINT32_MAX, with incref and decref compiled to no-ops against it.
Go to Free-threaded Python: removing the GIL without slowing one thread →Python critical sections
PEP 703's answer is Python critical sections: hold locks only for the span of one operation, and suspend — release, then reacquire — outer locks at exactly the points where the thread would block, which are exactly the points where the GIL would have been released.
Go to Free-threaded Python: removing the GIL without slowing one thread →as-if rule
An optimiser's license is the as-if rule: it may perform any transformation that preserves the program's observable behaviour when run alone.
Go to Compiler reordering: why threads cannot be a library →loop-invariant code motion
To the compiler, done inside the loop is loop-invariant: nothing in the loop's own text changes it, so the load is a candidate for the oldest optimisation in the book — compute it once and keep it in a register.
Go to Compiler reordering: why threads cannot be a library →register promotion
The profitable transform is register promotion: keep x in a register r through the loop, spilling and reloading only around the rare lock call.
Go to Compiler reordering: why threads cannot be a library →memory model
What neither side can supply is the thing in the middle: a definition, owned by the language, of which writes a read may observe — a memory model as a contract between the programmer and the compiler, in Cox's framing.
Go to Compiler reordering: why threads cannot be a library →memory actions
The objects being ordered are memory actions, split in two.
Go to Happens-before and data races inside one program →data operations
Ordinary loads and stores are data operations.
Go to Happens-before and data races inside one program →synchronization operations
Locks, unlocks and atomic read–modify–write operations are synchronization operations — actions whose declared purpose is communicating between threads.
Go to Happens-before and data races inside one program →memory locations
Actions address memory locations: each scalar object counts as its own location, so x and y are different locations even when they share a cache line — the one twist is that contiguous bit-fields inside a single struct count as one location.
Go to Happens-before and data races inside one program →program order
The first edge set needs no mechanism: program order, written , orders each thread's actions the way that thread's sequential semantics defines them.
Go to Happens-before and data races inside one program →synchronizes-with
The second set is the language's whole account of cross-thread order: enumerated synchronizes-with edges, written .
Go to Happens-before and data races inside one program →conflict
Two accesses conflict when they touch the same memory location and at least one is a write — Adve and Hill's phrasing: "not both reads".
Go to Happens-before and data races inside one program →data race
A data race is then two conflicting data operations, performed by different threads, unordered by happens-before:
Go to Happens-before and data races inside one program →sequentially consistent
A multiprocessor is sequentially consistent if "the result of any execution is the same as if the operations of all the processors were executed in some sequential order, and the operations of each individual processor appear in this sequence in the order specified by its program." One interleaving, each thread's operations in its own order — the semantics a programmer reaches for without being asked.
Go to Data-race-free programs are sequentially consistent →weak ordering
Hardware is weakly ordered with respect to a synchronization model if and only if it appears sequentially consistent to every program that obeys the model — where a synchronization model is just a set of constraints on memory accesses saying how and when synchronization must be done.
Go to Data-race-free programs are sequentially consistent →synchronization model
Hardware is weakly ordered with respect to a synchronization model if and only if it appears sequentially consistent to every program that obeys the model — where a synchronization model is just a set of constraints on memory accesses saying how and when synchronization must be done.
Go to Data-race-free programs are sequentially consistent →data-race-free
The synchronization model they instantiate it with — Data-Race-Free-0 — is the one every language memory model now inherits.
Go to Data-race-free programs are sequentially consistent →out-of-thin-air value
That shapes the design brief into three promises — sequential consistency for data-race-free programs, no out-of-thin-air values even for racy ones, and ordinary compiler optimisations left legal.
Go to The Java memory model: bounding what a racy program may do →synchronization order
For the programmer who needs order back, the JMM defines synchronisation actions — volatile reads and writes, monitor enter and exit, thread start, join and interrupt, and the default writes — and puts them all in one total order, the synchronization order, consistent with each thread's program order.
Go to The Java memory model: bounding what a racy program may do →freeze action
The JLS defines a freeze action on each final field, performed at the end of its constructor, and a guarantee that follows from it: a thread that sees only a properly published reference to an object sees that object's final fields exactly as the constructor left them — and sees everything those fields transitively reference at least as up-to-date as at the freeze.
Go to The Java memory model: bounding what a racy program may do →memory ordering
C++11's answer, inherited nearly verbatim by C11 and Rust, is the contract this lesson derives: a data race on a non-atomic variable is undefined behaviour — the program may catch fire — while every atomic operation carries an explicit memory ordering saying which reorderings and which visibility the machine must respect.
Go to C++ and Rust atomics: orderings as a language contract →wild branch
Now the race lands: x changes to 5 while foo runs, the reload picks up 5, the table is indexed out of bounds, and control jumps to an arbitrary address — a wild branch to random code.
Go to C++ and Rust atomics: orderings as a language contract →modification order
An std::atomic<T> makes accesses indivisible and gives each atomic object its own total modification order — a single history of writes per location, agreed upon by all threads.
Go to C++ and Rust atomics: orderings as a language contract →counting semaphore
The document's claim for the capacity edge is that it lets a buffered channel model a counting semaphore: items in the channel count active uses, capacity is the maximum, a send acquires and a receive releases.
Go to The Go memory model: DRF-SC with bounded races and channel edges →dynamic race detector
A dynamic race detector neither searches program text nor controls scheduling.
Go to Detecting data races: vector clocks over one execution →epoch
The replacement is the epoch, written : clock value paired with the thread it belongs to — two machine words where a vector needed (the implementation packs it into 32 bits: 8 for the thread id, 24 for the clock).
Go to Detecting data races: vector clocks over one execution →tiered compilation
HotSpot's arrangement — tiered compilation, introduced in Java SE 7 and the server VM's default — executes that contract as a ladder (jit-tiering priced the steps):
Go to Case study: the JVM (HotSpot) — speculation, a menu of collectors and virtual threads →code cache
Tiering's byproduct, the code cache, grew fivefold and split into three segments: non-method code (a fixed 3 MB), profiled (C1) and non-profiled (C2).
Go to Case study: the JVM (HotSpot) — speculation, a menu of collectors and virtual threads →compressed oops
Heap references on 64-bit builds are compressed oops — 32-bit object offsets scaled by the 8-byte alignment — good for ~4 billion objects or ~32 GB of heap, decoded as a shift and add; under 4 GB even the scale is free.
Go to Case study: the JVM (HotSpot) — speculation, a menu of collectors and virtual threads →AOT cache
The AOT cache (JEP 483, JDK 24) changes the deal: a training run records which classes were read, parsed, loaded and linked, and a production run loads them already linked.
Go to Case study: the JVM (HotSpot) — speculation, a menu of collectors and virtual threads →platform thread
A platform thread is the kernel's: one java.lang.Thread per OS thread, priced like one — a megabyte-scale stack reservation plus scheduler bookkeeping, an OS comfortable with a few thousand active threads where a server may hold a million sockets.
Go to Case study: the JVM (HotSpot) — speculation, a menu of collectors and virtual threads →feedback vector (V8)
Alongside the bytecode sits the feedback vector: per-function records of what each site actually met — which maps at each property access, which call target at each call.
Go to Case study: V8 and Node — four tiers on one thread →pointer-compression cage
With pointer compression on, Smis carry 31 bits and span , and heap objects are addressed as 32-bit offsets from the base of one 4 GiB cage.
Go to Case study: V8 and Node — four tiers on one thread →Which runtime mechanism for this job?
A problem, the preconditions the mechanism needs, the guarantee on offer, the cost model it lives in, the failure case, and the topic that derives the trade-off.
Interpreter first, or pay for compilation up front?
AOT wins the first seconds; a tiered JIT wins steady state by compiling what the profile proves hot.
More detail
Problem: the binary must be fast now and faster later, and both wishes spend the same CPU. Preconditions: a measurable warm-up budget and a hot-code fraction small enough that profiling pays. Guarantee: AOT gives flat startup; tiering gives peak throughput that beats AOT on speculative wins it can withdraw. Cost: warm-up time and the metadata a profiler keeps. Failure case: a short-lived process whose lifetime ends inside the interpreter's discount window — the JIT never recovers its scouting fee.
How many compilation tiers does this VM need?
Each tier is an online bet: spend compile time only on code that has already spent interpretation time.
More detail
Problem: optimising everything is slow to start; interpreting everything is slow forever. Preconditions: a dispatch fast enough to observe hotness, and code that actually repeats. Guarantee: each tier raises the threshold, so compile cost tracks measured heat. Cost: per-tier transition machinery and the profiling data that feeds it. Failure case: the plateau where tier two compiles code the tier-one thresholds misjudged — hot while observed, cold forever after.
Is tracing the hot loop better than compiling whole methods?
Tracing compiles the path actually taken — branches and all — but pays when the path splits.
More detail
Problem: method boundaries hide the loop that matters behind calls that do not. Preconditions: hot paths are stable across iterations; type feedback agrees. Guarantee: the trace inlines the hot path across call edges, giving one straight-line region to optimise. Cost: every side exit is a deopt, and path explosion compiles the same basic blocks many ways. Failure case: a polymorphic loop whose "hot path" changes per iteration — the tracer spends everything guarding against yesterday's path.
Speculate, or compile conservatively?
Speculation is renting peak speed against a deopt you hope never fires — the bet needs a cheap exit.
More detail
Problem: safe code must handle cases the profile has never shown, and handling them costs the common path. Preconditions: a deopt path that reconstructs interpreter state correctly. Guarantee: the optimised code is correct in every reachable state because the guard either holds or exits to a frame that does. Cost: guard instructions everywhere, and frame-reconstruction machinery in reserve. Failure case: a rare-but-real input shape that deopts in a loop — the runtime pays interpretation plus bailout plus recompilation each time.
How big can the inline cache get before it stops helping?
Monomorphic and polymorphic sites are free wins; megamorphic sites must bail to a generic path instead of chaining entries.
More detail
Problem: a property access or call site sees a small set of shapes, but "small" is per-site and data-dependent. Preconditions: receiver shapes cluster below the cache's polymorphism cap. Guarantee: a hit is a compare and a jump — around an order of magnitude cheaper than a generic lookup on the course's cost model. Cost: megamorphic sites thrash and must fall back, which is why they exist as a first-class mode. Failure case: treating a megamorphic site as cacheable — the chain grows, the compares dominate, and a hash lookup would have won.
Reference counting, tracing, or ownership?
Counting frees predictably but cycles leak; tracing finds cycles but pauses; ownership is static but makes shared ownership an explicit, counted exception.
More detail
Problem: reclamation must be timely, complete, and cheap — pick two. Preconditions: the heap shape (cycles? sharing? mutation rate?) is stated. Guarantee: RC reclaims at the last pointer; tracing reclaims exactly the unreachable; ownership makes reclamation a compile-time fact. Cost: RC pays per-pointer write; tracing pays throughput and latency in collection; ownership pays in program structure, and in counting wherever sharing is opted back in. Failure case: cyclic structures under pure RC, or a latency budget under a stop-the-world mark — each strategy has a heap it cannot serve.
When does a tracing GC beat counting?
When the graph cycles or mutates faster than counts are worth — tracing pays once per collection instead of per pointer write.
More detail
Problem: reference counts cannot see a disconnected cycle, and decrement-every-write is a tax paid on every store. Preconditions: enough allocation churn that RC's per-write cost exceeds periodic tracing, or real cycles that leak otherwise. Guarantee: reachability, not counting — cycles die with their last root edge. Cost: pause or barrier overhead, and floating garbage until the next collection. Failure case: a heap that is mostly long-lived and acyclic — tracing buys completeness it never uses.
How many generations earn their bookkeeping?
Two generations catch the young-die-young hump; a third only pays if middle-aged objects die faster than full-collection cost.
More detail
Problem: most objects die young, so scanning the whole heap wastes work on the old. Preconditions: the workload actually follows the generational hypothesis — measure survival, do not assume it. Guarantee: a nursery-sized scavenge bounds pause time while a rare full pass reclaims the rest. Cost: remembered-set bookkeeping on every old-to-young write, plus promotion policy. Failure case: middle-aged objects dying past the nursery — they promote, then die in the expensive tier, paying for both ends of the ride.
Should the collector move objects?
Moving kills fragmentation and enables bump allocation; it costs forwarding barriers and pins every native reference.
More detail
Problem: a non-moving heap fragments until a large allocation fails with free space everywhere. Preconditions: the runtime controls all references — handles, read barriers, or a pause where fix-up is safe. Guarantee: live objects pack into one region; free space is a pointer increment. Cost: every read may pay a forwarding check, and pinned objects carve holes in compaction. Failure case: a C API that stores raw pointers — the contract forbids movement, and compaction is simply unavailable, as JNI-critical sections show.
Which collector for this heap and latency budget?
Throughput budget says parallel; pause budget says concurrent; huge heaps say regional — the axes are stated before the switch.
More detail
Problem: the collector question arrives as a flag, but it is a workload question. Preconditions: live-set size, allocation rate, and a pause-time percentile stated. Guarantee: each design point dominates only on the axis it spends on — Parallel on throughput, G1 on bounded-stall balance, ZGC/Shenandoah on pauses at heap scale. Cost: concurrent marks spend CPU while the app runs; regional sets spend metadata per region. Failure case: tuning a concurrent collector on a throughput workload — GC thread time is billed as mutator slowdown.
Shared memory with locks, or an actor per piece of state?
Shared threads maximise coupling speed; one loop maximises predictability; actors buy isolation at the cost of copies.
More detail
Problem: concurrent work must coordinate, and the model chosen decides where races can even exist. Preconditions: the sharing pattern — mutable state touched by all, or owned state passed as messages. Guarantee: shared memory gives the cheapest communication the memory model still permits; the event loop serialises races out of existence; actors make every boundary a copy. Cost: threads spend correctness discipline; the loop spends latency on any blocking call; actors spend bandwidth on data movement. Failure case: a blocking call on the event loop, or an actor forced to serve state every other actor reads — its mailbox becomes the bottleneck the shared memory would have avoided.
Spawn threads, or multiplex a few?
M:N scheduling trades one big scheduling problem for many small ones — a local queue per worker, stealing when idle.
More detail
Problem: an OS thread per task costs memory per idle task and a kernel transition per switch. Preconditions: tasks outnumber cores and block rarely, or the runtime detects blocking. Guarantee: work-stealing keeps every worker busy while runnable tasks exist — idle workers pull from busy queues. Cost: queue operations on the hot path, plus the preemption needed to stop a runaway task from starving its queue. Failure case: a task that blocks without telling the runtime — it holds a worker hostage, and the whole queue behind it stalls.
Async/await or threads-with-locks?
Await splits the program at yield points; locks split it at critical sections — the colour tax buys stackless concurrency.
More detail
Problem: concurrent I/O wants cheap tasks, but a shared-colour language makes every callee a viral commitment. Preconditions: an ecosystem that already speaks the colour — an async library stack, an executor to drive it. Guarantee: thousands of in-flight operations fit in memory because each costs a state machine, not a stack. Cost: colouring propagates through callers, cancellation is cooperative, and blocking code poisons the executor. Failure case: the sync call inside async — one blocking function holds its worker thread for the call's duration, invisibly to the type system; on a single-threaded loop that is the whole executor, and no cooperative budget can interrupt it.
Channel, mailbox, or shared queue?
A channel couples sender and receiver in one operation; a mailbox decouples them and queues until the reader is ready.
More detail
Problem: two concurrent pieces must exchange data; the question is whether they synchronise or buffer. Preconditions: whether backpressure must reach the sender (channel) or absorb it (mailbox/queue). Guarantee: a channel's rendezvous is itself a synchronisation edge — Go's memory model makes the send happens-before the receive. Cost: an unbuffered channel stalls the sender; a mailbox can grow without bound. Failure case: an unbounded mailbox under a slow consumer — memory, not deadlock, is the failure the BEAM teaches you to watch.
Restart on failure, or defend against it?
Let it crash when faults are transient and state can restart clean; defend the parts whose errors are not recoverable.
More detail
Problem: defensive code for every failure is impossible, and some failures are transient enough that restart beats recovery. Preconditions: failures that a clean state fixes — a wedged connection, corrupted scratch state — and an error kernel small enough to hold the parts that must not crash. Guarantee: a supervisor restarts a child to a known state with bounded restart intensity; escalation stops runaway restarts. Cost: state lost on restart, and restart storms absorbed by intensity caps. Failure case: a deterministic bug or poisoned input — restart recurs the crash, and intensity just delays it.
When does a GIL-style lock pay for itself?
When the extension contract forbids parallel internals anyway, one lock is the cheapest correct design — and the ceiling it imposes is the argument for removing it.
More detail
Problem: non-atomic reference counts plus C extensions assuming single-threaded access make fine-grained locking a rewrite of the ecosystem. Preconditions: an extension ABI that exposes object layout and counts, as `native-extension-contract` derives. Guarantee: interpreter internals and counts stay correct under threads without touching a single extension. Cost: CPU-bound threads cap at one core, and I/O threads convoy behind the bound one. Failure case: a CPU-bound workload on threads — the folklore says it parallelises; the lock says it serialises.
Escape the GIL by isolation or by free-threading?
Isolate when per-interpreter state and sharing-via-copies fit; free-thread when shared mutable parallelism is the point.
More detail
Problem: one lock per process wastes the cores under it; the exit is either more interpreters or no lock. Preconditions: isolation needs shareable data or copy costs bounded; free-threading needs the PEP-703 machinery (biased counts, immortal objects, per-object locks) and extension compatibility. Guarantee: subinterpreters give parallelism with process-like isolation; free-threading gives true shared-memory parallelism inside one interpreter. Cost: subinterpreters pay memory per interpreter and copying across; free-threading pays a single-thread slowdown and per-object-lock complexity. Failure case: extensions that assumed the GIL — under free-threading they must lock per object or race, and under isolation they must reinitialise per interpreter.
What may my program observe under races?
If the program is data-race-free under sequential consistency, every execution is sequentially consistent; what a racy program may do is a separate, language-specific question.
More detail
Problem: reasoning about interleavings is only possible if the language promises something; reorderings make raw interleaving intuition wrong. Preconditions: the program synchronises every shared access — locks, acquire/release, channels — so no data race exists on SC executions. Guarantee: compilers may reorder freely between synchronisation, but no thread can observe it — the model reduces to interleavings. Cost: the language must say which reorderings are legal, and the programmer must use the synchronisation vocabulary. Failure case: assuming DRF-SC holds on a racy program — now the answer depends on the model: Java bounds the read, C++ calls it undefined, Go admits bounded race values.
Which atomics ordering is strong enough?
seq_cst is the default that matches intuition; acquire/release is the payment for cheap synchronisation; relaxed is for counters that order nothing.
More detail
Problem: sequential consistency costs fences the hardware must execute; weaker orderings ask the programmer to carry the order. Preconditions: a data race on non-atomics is UB, so shared accesses must be atomic; then pick the ordering the pattern needs — release/acquire for message passing, relaxed for statistics, seq_cst for the store-buffer patterns that need it. Guarantee: each ordering is a contract with stated allowed outcomes, and standard mappings to x86 and AArch64 instructions exist for each. Cost: weaker orderings move reasoning from the model to the programmer. Failure case: a relaxed flag used for publication — the write to the payload is invisible to the reader that "saw" the flag, and no fence was paid for.