Your technical library
Operating Systems in Depth
11 modules · 70 topics
About this book
Operating Systems in Depth
This course continues Operating Systems for a reader who finished it and wants the internals it deliberately passed over: what the scheduler does beyond a fair share, how the kernel allocates its own memory, the lock-free toolbox, how devices and file systems are really built, how the kernel is traced, extended and bypassed, how programs are confined and how they are still attacked, how virtual machines run on it, and the case studies — AFS and `docker run` — that need those pieces. Each topic opens with the problem it inherits and closes with what it leaves open.
The prerequisite is the introductory course, read in order or at least to the topic each lesson names: this course assumes its mechanisms are known, recalls one in a sentence where a lesson leans on it, and links the lesson that taught it. Many university "advanced operating systems" courses spend half their time on distributed systems — logical clocks, distributed mutual exclusion, agreement; here that material belongs to System Design (logical and vector clocks, consensus), and this course keeps to the single machine. The same plain-language policy holds — short paragraphs, every term defined at first use, one assumptions sentence per derivation, and real tool output over folklore.
70 of 70 topics are authored. Read Operating Systems first, then this course in dependency order. Planned lessons are labelled and excluded from progress; references cover authored material.
Read at your own pace. Each topic is one continuous page; mark it done as you finish.
Start with 01.1 · Deferred interrupt work: top halves, softirqs and workqueuesReview what you have finished↷ 0 skipped topics- 01.1Deferred interrupt work: top halves, softirqs and workqueues
- 01.2Kernel clocks: clock sources, monotonic time and the vDSO
- 01.3Kernel timers: timer wheels, high-resolution timers and tickless kernels
- 01.4Scheduling classes: real-time, deadline and pluggable schedulers above EEVDF
- 01.5Kernel preemption: from voluntary preemption points to PREEMPT_RT
- 01.6Per-CPU run queues and cache affinity
- 01.7Load balancing and work stealing across cores
- 01.8Energy-aware scheduling on heterogeneous cores
- 02.1Inverted page tables: one entry per frame
- 02.2TLB shootdown: invalidating translations on every core
- 02.3The kernel's address space: shared mappings and split tables
- 02.4The slab allocator: caches of kernel objects
- 02.5Huge pages: fewer TLB entries for large memory
- 02.6NUMA: memory at different distances
- 04.1Async-signal safety: what a signal handler may do
- 04.2Lock-free stacks and the ABA problem
- 04.3Safe memory reclamation: hazard pointers and epochs
- 04.4Sequence locks: readers retry instead of locking
- 04.5Read-copy-update: readers that never block or write
- 04.6Thread-caching allocators: jemalloc, tcmalloc and mimalloc
- 04.7Choosing a synchronisation primitive
- 05.1The IOMMU: page tables for devices
- 05.2Accelerators and the OS: pinned memory, device page tables and unified memory
- 05.3The kernel packet path: from NIC ring to socket buffer
- 05.4SSD garbage collection, wear levelling and write amplification
- 05.5NVMe: queue pairs sized for parallel media
- 05.6I/O schedulers: mq-deadline, BFQ, Kyber and none
- 05.7RAID 4, 5 and 6: parity and the small-write problem
- 05.8RAID rebuild windows and mean time to data loss
- 06.1File locking: flock, POSIX record locks, OFD locks and lock files
- 06.2Linux page reclaim: file versus anonymous pages, and MGLRU
- 06.3Zero-copy I/O: sendfile, splice and MSG_ZEROCOPY
- 06.4End-to-end data integrity: checksums and scrubbing
- 06.5Copy-on-write file systems: never overwrite in place
- 06.6Log-structured file systems: every write appended
- 06.7Segment cleaning: the cost of a log
- 06.8Journaling, copy-on-write or a log: choosing a crash-consistency strategy
- 07.1Tracing mechanisms: tracepoints, kprobes, uprobes and ftrace
- 07.2Loadable kernel modules: extending the kernel with full trust
- 07.3Microkernels: moving services into user space
- 07.4eBPF programs and hooks: sandboxed code inside the kernel
- 07.5The eBPF verifier: static proof of safety
- 07.6eBPF maps: shared state between kernel programs and user space
- 07.7Kernel bypass: DPDK, SPDK and user-space drivers
- 08.1Mandatory access control: LSM, SELinux and AppArmor
- 08.2seccomp: filtering system calls
- 08.3Privilege separation and broker sandboxes
- 08.4Network namespaces: private network stacks and veth pairs
- 08.5User namespaces: root inside, unprivileged outside
- 08.6Sandboxed container runtimes: gVisor, Kata and microVMs
- 09.1Memory-corruption exploits: overwriting control data
- 09.2Stack canaries: detecting a smashed frame
- 09.3Non-executable memory, and return-oriented programming
- 09.4Address-space layout randomisation and its entropy
- 09.5Control-flow integrity: shadow stacks and indirect-branch checks
- 09.6Pipelines, branch prediction and speculative execution
- 09.7Cache-timing side channels: Flush+Reload and Prime+Probe
- 09.8Meltdown: reading kernel memory during a transient window
- 09.9Spectre: mistraining the predictor
01.2Kernel clocks: clock sources, monotonic time and the vDSO
01.3Kernel timers: timer wheels, high-resolution timers and tickless kernels
01.4Scheduling classes: real-time, deadline and pluggable schedulers above EEVDF
01.5Kernel preemption: from voluntary preemption points to PREEMPT_RT
01.6Per-CPU run queues and cache affinity
01.7Load balancing and work stealing across cores
01.8Energy-aware scheduling on heterogeneous cores
02.1Inverted page tables: one entry per frame
02.2TLB shootdown: invalidating translations on every core
02.3The kernel's address space: shared mappings and split tables
02.4The slab allocator: caches of kernel objects
02.5Huge pages: fewer TLB entries for large memory
02.6NUMA: memory at different distances
03.1Position-independent code: running at any address
03.2Readiness versus completion: why Linux AIO fell short
03.3io_uring: asynchronous system calls through shared rings
03.4User-space threads: M:N scheduling over asynchronous I/O
04.1Async-signal safety: what a signal handler may do
04.2Lock-free stacks and the ABA problem
04.3Safe memory reclamation: hazard pointers and epochs
04.4Sequence locks: readers retry instead of locking
04.5Read-copy-update: readers that never block or write
04.6Thread-caching allocators: jemalloc, tcmalloc and mimalloc
04.7Choosing a synchronisation primitive
05.1The IOMMU: page tables for devices
05.2Accelerators and the OS: pinned memory, device page tables and unified memory
05.3The kernel packet path: from NIC ring to socket buffer
05.4SSD garbage collection, wear levelling and write amplification
05.5NVMe: queue pairs sized for parallel media
05.6I/O schedulers: mq-deadline, BFQ, Kyber and none
05.7RAID 4, 5 and 6: parity and the small-write problem
05.8RAID rebuild windows and mean time to data loss
06.1File locking: flock, POSIX record locks, OFD locks and lock files
06.2Linux page reclaim: file versus anonymous pages, and MGLRU
06.3Zero-copy I/O: sendfile, splice and MSG_ZEROCOPY
06.4End-to-end data integrity: checksums and scrubbing
06.5Copy-on-write file systems: never overwrite in place
06.6Log-structured file systems: every write appended
06.7Segment cleaning: the cost of a log
06.8Journaling, copy-on-write or a log: choosing a crash-consistency strategy
07.1Tracing mechanisms: tracepoints, kprobes, uprobes and ftrace
07.2Loadable kernel modules: extending the kernel with full trust
07.3Microkernels: moving services into user space
07.4eBPF programs and hooks: sandboxed code inside the kernel
07.5The eBPF verifier: static proof of safety
07.6eBPF maps: shared state between kernel programs and user space
07.7Kernel bypass: DPDK, SPDK and user-space drivers
08.1Mandatory access control: LSM, SELinux and AppArmor
08.2seccomp: filtering system calls
08.3Privilege separation and broker sandboxes
08.4Network namespaces: private network stacks and veth pairs
08.5User namespaces: root inside, unprivileged outside
08.6Sandboxed container runtimes: gVisor, Kata and microVMs
09.1Memory-corruption exploits: overwriting control data
09.2Stack canaries: detecting a smashed frame
09.3Non-executable memory, and return-oriented programming
09.4Address-space layout randomisation and its entropy
09.5Control-flow integrity: shadow stacks and indirect-branch checks
09.6Pipelines, branch prediction and speculative execution
09.7Cache-timing side channels: Flush+Reload and Prime+Probe
09.8Meltdown: reading kernel memory during a transient window
09.9Spectre: mistraining the predictor
10.1Binary translation and paravirtualisation
10.2Nested paging: two-dimensional address translation
10.3Live migration: moving a running virtual machine with pre-copy and post-copy
10.4Trusted execution environments: enclaves and confidential VMs
10.5Unikernels and library operating systems
11.1AFS: whole-file caching with callbacks
11.2Case study: what starting a container actually does
Skipped topics
Moved past without finishing. A skipped topic still closes its module, and Continue goes on without it — open one to read it or mark it understood.
Glossary
Terms explained by the authored lessons.
Docker run anatomy
A container start is ~64 syscalls: unpack layers, overlay mount, clone(CLONE_NEW*), cgroup writes, capability drop + no_new_privs + seccomp, pivot_root, execve — nothing new under the kernel.
More detail
The exercise is the point: every mechanism the container uses was taught as its own topic, and the shared kernel is the one thing left on the table at the end.
AFS whole-file caching + callbacks
Fetch the whole file at open, serve reads locally, and let the server promise a callback when the file changes — polling becomes notification.
More detail
Howard’s TOCS’88 numbers are the argument: ~90% of RPCs vanished and a server went from ~38% CPU at scale to supporting 50–70 users. Last-closer-wins and callback recovery after a server crash are the semantics you give up for the scale.
Microkernel IPC cost
Shrink the kernel to IPC + address spaces + scheduling; drivers and file systems become user-space servers — one crash stops owning the machine.
More detail
The price is crossings: a file read that cost one syscall in a monolith pays several context switches through servers. Liedtke’s L4 answer was fast IPC (~10× Mach); seL4 pushed the argument to its limit with a fully verified 8,700-line kernel.
Unikernel and exokernel
The exokernel exports raw devices and lets each app pick its abstractions; the unikernel links app + library OS into one hypervisor image — no user/kernel boundary at all.
More detail
Mirage showed the payoff: an OCaml-only image ~10× smaller and seconds-to-boot on Xen. The bill arrives at operations — no shell, no ps, and every libc quirk becomes the app’s problem.
veth pair and bridge
A container’s eth0 is the inner half of a virtual Ethernet pair — the outer end plugs into a bridge that links to the host stack.
More detail
Traffic crosses the boundary at link layer, so NAT gives containers outbound reachability without new interfaces. The path container→veth→bridge→host is why container networking debugs like ordinary networking until the last hop.
User namespace and uid mapping
Root inside the box, nobody outside: uid 0 in a container maps to an unprivileged host uid via /etc/subuid ranges.
More detail
Capabilities evaluate against the user namespace that owns the object, so a container’s CAP_SYS_ADMIN reaches its own netns but not the host’s mount table. Rootless containers are the endgame — the unshared kernel surface visible to a real attacker shrinks to uid-range checks.
Sandboxed runtimes
Containers share the host kernel; gVisor reimplements the syscall surface in a userspace kernel, Kata/Firecracker put each container behind a hypervisor boundary.
More detail
Firecracker’s numbers are the argument: ~125ms boot, <5MiB overhead per microVM, because it emulates 5 devices not QEMU’s full machine. systrap vs ptrace is gVisor’s internal trade — interception speed against filtering granularity.
Binary translation and paravirtualisation
Two escapes from an unvirtualisable ISA: rewrite sensitive instructions at load (VMware), or edit the guest to call hypercalls (Xen).
More detail
Xen moved the guest to ring 1 and batched MMU updates (2048 per hypercall) — the guest source was the price. Both pay in complexity what hardware assist later paid in silicon.
Nested paging
Guest virtual → guest physical → host physical: shadow tables were software trickery; EPT/NPT make the hardware walk both levels.
More detail
A 4-level × 4-level walk costs (4+1)(4+1)−1 = 24 references worst-case per TLB miss; huge pages cut the depth on both dimensions. Shadow paging’s sync bugs were the alternative’s real price.
TEE and confidential VMs
Remove the host OS and hypervisor from the TCB: SGX enclaves for a process slice, SEV-SNP/TDX encrypting whole guest memory, attestation proving what booted.
More detail
The attestation chain is signatures over measurements rooted in the chip. Side channels survive — the enclave boundary stops software reads, not the cache’s timing leak.
Speculative execution
Prediction turns a pipeline stall into a gamble — wrong guesses retire nothing architecturally but leave microarchitectural residue.
More detail
CPI becomes 1 + βρp: branch frequency × miss rate × penalty. BTB, history tables and the RSB all guess before resolution; the guesses write into caches and predictors that the commit stage never cleans.
Cache timing side channel
Hit vs miss latency is a measurable bit — Prime+Probe and Flush+Reload turn it into a read primitive across a protection boundary.
More detail
A timing threshold splits the two cases; AES T-table nibble recovery shows how per-key structure leaks. Repetition with majority vote beats noise — coverage and rounds are the levers.
Meltdown transient window
A faulting load briefly yields its value to dependent instructions before the exception lands — a window to launder kernel memory through the cache.
More detail
The transient instruction reads kernel data, touches a probe-array line, and the flush+reload readout survives the rollback. KPTI’s fix is architectural: unmap the kernel from user page tables entirely.
Spectre and BTB poisoning
The attacker trains the victim’s own predictor — mistrain the BTB or history, then let speculation execute a disclosure gadget on data the victim can reach.
More detail
No fault is needed — the mis-speculated path runs in the victim’s domain. Retpolines, IBRS and fences each close a variant; the class is unkillable in general because prediction is what makes pipelines fast.
Stack buffer overflow
Writes past a buffer reach the saved return address — the frame layout is the exploit map.
More detail
Offset arithmetic to the return slot decides the payload position; the same overwrite pattern reaches heap metadata and dangling pointers (use-after-free). Injected shellcode needs writable-executable memory — which is why the defences target exactly that.
Stack canary
A random value between buffer and return address, checked before return — an overflow must cross it to win.
More detail
Brute force costs 2^n guesses per canary, but a forking server that reuses the canary shrinks it to 8×256 byte-at-a-time leaks. Anything the overwrite skips (non-linear writes, heap targets) bypasses it entirely.
NX bit and ROP
W^X — a page is writable or executable, never both — which pushes the attacker from injected code to borrowed code.
More detail
ret2libc reuses libc; ROP stitches gadgets ending in ret. The gadget census is the attack surface: ~1 ret-compatible byte sequence per 178 bytes of text. RELRO and BIND_NOW remove the GOT as a writable target.
ASLR entropy budget
Randomise each region’s base — mmap 28 bits, stack 22, brk 18 on Linux — and an attacker’s guess costs 2^n trials.
More detail
k probes against a 2^n space succeed with probability 1−(1−2⁻ⁿ)^k; the Apache derandomisation attack exploited a fork-and-retry server to break it in ~216 s. Re-randomisation helps only if the crash doesn’t disclose the layout.
Control-flow integrity
Verify that every indirect call targets a member of a statically computed equivalence class — the precision-vs-cost trade in one number.
More detail
ID-based CFI groups targets into classes (finer classes cost more checks); a shadow stack makes return addresses tamper-proof — software versions cost ~21%, CET’s shstk + ENDBR bring it into hardware. Arm PAC signs pointers in unused VA bits.
LSM and MAC
Mandatory policy the subject cannot override: SELinux labels everything and rules allow transitions; AppArmor confines by path.
More detail
The LSM hooks sit inside the kernel’s decision path after the DAC check — MAC is a second gate, not a replacement. SELinux’s type enforcement is expressible as matrix cells; AppArmor trades precision for readable policy.
seccomp filter
A classic-BPF program run at every syscall entry that returns kill/errno/trap/allow — hundreds of syscalls narrowed to a handful.
More detail
Strict mode is the extreme (read/write/exit only); filter mode matches on number + arguments but cannot dereference pointers — open()’s path argument is out of reach. Cost is per-syscall filter evaluation, cheap until the filter grows.
Broker–worker split
Privilege separation: an unprivileged worker parses untrusted input; a small privileged broker answers a narrow IPC protocol.
More detail
OpenSSH split the network-facing parser from the key-holding monitor; Chromium put renderers in sandboxes behind a broker. The accounting is attack surface: reachable syscalls × privileges before vs after — seccomp caps the worker’s reachable set.
Loadable kernel module
Code linked into the running kernel with full privileges — extension without a rebuild, and a bug or hostile module owns the machine.
More detail
init_module resolves symbols against EXPORT_SYMBOL and runs in ring 0; signing and lockdown are the only gates. The trust is total, which is exactly the problem eBPF was built to remove.
eBPF program + hook points
Sandboxed bytecode JITed and attached at named kernel events — tracepoints, kprobes, socket filters, XDP, cgroup, LSM.
More detail
Each program type names where it runs and what context it sees; the bpf() syscall loads and attaches it. Helper functions are the kernel’s API surface, not arbitrary calls.
The eBPF verifier
Static proof before JIT: abstract interpretation over register types and value ranges, with branch-split refinement and a complexity cap.
More detail
Bounded loops (5.3+), pointer arithmetic checks and termination are enforced per-path; the verifier is itself an attack surface — CVE-2020-8835 was a verifier bypass. Refusal is the common case, not a bug.
eBPF maps
Typed shared state (hash, array, per-CPU, ring buffer, LRU) that survives between program invocations and talks to user space.
More detail
Per-CPU maps cost entries × value × CPUs but remove contention; ring buffer uses reserve/commit so producers never block. bpffs pinning makes a map survive the program that made it.
Tracepoints, kprobes, uprobes
Three attach mechanisms: compiled-in static points, dynamic instruction patching on kernel code, and the same trick on user code.
More detail
kprobe patches the target instruction (optprobe makes it a jump); uprobes do it in a user binary. Each hit costs f = r·c_hit — on a hot path the probe itself is the load.
Kernel bypass
DPDK/SPDK/XDP-class paths that pull the device queue into user space — trading kernel services for the syscalls and copies they cost.
More detail
VFIO + hugepages + a PMD give the app the NIC ring; the price is reimplementing what the kernel provided (TCP, safety, isolation). Bypass wins when the kernel path’s per-packet cost exceeds the app’s own stack.
Readiness vs completion
epoll tells you a descriptor can proceed; a completion model tells you an operation finished — regular files are always “ready” so the former cannot async-ify disk I/O.
More detail
Linux AIO only honoured O_DIRECT, fell back to blocking for metadata and buffer slots, and paid a 104-byte copy plus two syscalls per op. Completion is the semantics you actually want; io_uring is where Linux got it.
Submission/completion rings
io_uring places two shared rings in memory: the app writes SQEs, the kernel writes CQEs — syscall-free steady state.
More detail
Registered buffers and files skip per-op copies and table walks; linked SQEs build chains; polled mode removes interrupts. The surface is so fast it became the top kernel exploit target — the kCTF finding that motivated it is in the sources.
M:N scheduling
Thousands of user-space threads multiplexed onto a few kernel threads — a user scheduler owns the fast path and hands blocking calls to the kernel.
More detail
Go’s G-M-P model and Java’s virtual threads (JEP 444/491) both run this way; the hard part is a blocking syscall hijacking the kernel thread, which is what work-stealing and handoff solve.
Copy-on-write tree
Update by rewriting the path to the root — the new root’s generation and checksum decide which tree survived the crash.
More detail
ZFS/btrfs never overwrite in place; a transaction-group batch publishes one uberblock. Snapshots become reference-counted tree roots — nearly free to take, but dead data pins space until every reference dies.
Segment cleaning and write cost
LFS makes every write sequential by paying a cleaner that copies live data out of segments before reuse.
More detail
Effective write rate R_eff = R/(write cost), where write cost ≈ 2/(1−u) at utilisation u — the log is free only while segments stay empty. Greedy cleaning of cold segments beats uniform cleaning under hot/cold locality.
End-to-end checksums
A checksum stored beside data cannot detect a misplaced write — the check must bind data to its address, and the media must be distrusted at every layer.
More detail
ZFS keeps block checksums in the parent pointer so a phantom write or bit rot below the fs is caught. Sector-level CRC has real miss rates — empirical 0.38% on one study vs the ideal 2⁻⁸ — the corruption surface is wider than the math.
Journaling vs CoW vs LFS
Three designs answering one question — how to make a multi-block update atomic — priced in write amplification, fragmentation and recovery time.
More detail
A journal writes twice and returns to in-place update; CoW rewrites a path per update and pays in snapshots and fragmentation; LFS writes once sequentially and pays a cleaner forever. Pick by whether you fear recovery time, write amplification or cleaning debt.
Position-independent code
Code that runs at any load address because distances inside one object never move — only the inter-object table is relocated.
More detail
A load-time patch would have to rewrite text for every process; PIC moves the patchable surface to one GOT per process (R_text = u instructions vs R_GOT = s + d slots). The cost is an indirection per external reference.
Watermarks and kswapd
Free memory drained past LOW wakes background reclaim; reaching MIN blocks allocation directly.
More detail
kswapd sweeps inactive pages back toward HIGH so the allocator rarely meets MIN. The watermark gap is the only slack between “the cache has all memory” and “allocation stalls”.
Refault and generations
A fault on a recently evicted page is evidence the eviction was wrong — MGLRU turns that signal into generational age.
More detail
Active/inactive lists per kind get a second-chance scan; swappiness prices anonymous pages against file pages when choosing what to scan. Multi-generational LRUs replace the two lists with generations.
Zero-copy paths
sendfile keeps data inside the kernel; splice moves page references through a pipe; MSG_ZEROCOPY lends the network stack pinned pages.
More detail
All three trade fixed overhead for deleted per-byte copies — break-even sits where n·γ ≈ F, roughly tens of KiB. Below it, copying is the cheaper path.
Write amplification
Flash writes divided by host writes — reclamation copies live pages the host never asked to move.
More detail
Greedy cleaning drives WA toward 1/(1−u) for utilisation u: nearly-full devices amplify the most. Over-provisioning and TRIM both buy cheaper victims; wear levelling spreads the remaining erases.
NVMe queue pair
A submission ring and a completion ring in host memory — one pair per core, rung by doorbell writes.
More detail
SATA’s single 32-deep queue serialized a parallel medium. Per-core pairs make submission lock-free; phase tags mark completed entries and polled completion beats interrupts when service is microseconds.
mq-deadline, BFQ and Kyber
Bound the wait, budget the bandwidth, or throttle toward a latency target — three answers to request ordering.
More detail
Reads and writes are not symmetric, so deadline expires writes eventually; BFQ assigns budgets not just order; Kyber holds tokens and adjusts from measured latency. none exists for devices fast enough that ordering is overhead.
Parity and the small-write problem
XOR parity gives single-disk redundancy at one disk of capacity — but every small write costs four I/Os.
More detail
P_new = (C_old ⊕ C_new) ⊕ P_old needs read-old-data, read-old-parity, write-both. RAID 4 serializes that on one parity disk; RAID 5 rotates parity to spread it. The write hole is the crash window between data and parity updates.
Rebuild window and URE
Degraded mode lasts until the spare is rebuilt — days on modern capacities — and every byte read during it can hit an unrecoverable error.
More detail
MTTDL ≈ MTTF²/(n(n−1)MTTR) makes the window the denominator. Field data (Schroeder) shows failures are neither independent nor memoryless, and at multi-TB reads a URE during rebuild is likely, not rare.
IOMMU
Page tables for devices: the kernel maps bus addresses, the device touches only what was mapped.
More detail
Translation converts an unrestricted DMA engine into a bounded one — isolation for drivers and virtualisation alike. The price is a walk per I/O plus map/unmap calls, amortised over buffer size b.
sk_buff
The one buffer object that climbs the whole receive stack — ring, driver, protocol layers, socket queue.
More detail
Allocating it once and passing it by pointer is what keeps per-packet work inside the arrival budget; copying per layer would not fit tens of nanoseconds.
RSS and multi-queue NICs
Receive-side scaling hashes flows to hardware queues so each core drains its own ring.
More detail
Queues multiply until the cores are fed: one ring serialises packet arrival on one CPU. The hash keeps a flow’s packets ordered on one queue.
Treiber stack
A lock-free LIFO where push and pop are single CAS on the head pointer.
More detail
The CAS on the head is the linearisation point — the stack is correct because one atomic update commits each operation. It is not safe to free popped nodes while readers might still dereference them.
Linearisation point
The instant inside an operation’s interval where it appears to take effect.
More detail
For the Treiber stack it is the successful CAS. Finding it is the correctness argument; a failed CAS is a retry, not a half-committed operation.
ABA problem
A CAS succeeds because the value returned to A even though the structure changed underneath.
More detail
Pointer reuse after free is the classic trigger: head reads A, A is popped and freed, a new A lands at the same address, and CAS commits a stale list. Tagged pointers or reclamation schemes break the alias.
Hazard pointer
A published per-thread pointer announcing “I may still read this node”.
More detail
Reclaimers scan all hazard pointers before freeing: at most H = N·K announcements for N threads holding K each, bounding unreclaimed garbage at NR retired nodes.
Epoch-based reclamation
Freeing is deferred until every thread has passed a quiescent point in a new epoch.
More detail
Cheaper per-read than hazard pointers but a slow or preempted thread stalls reclamation for everyone; the bound on retained garbage is looser and workload-dependent.
Sequence lock
A writer bumps a sequence counter; readers retry if it changed or is odd mid-read.
More detail
Readers never block the writer and write nothing shared. It cannot protect pointers readers dereference — a torn read of data is caught by the counter, a dereferenced freed pointer is not.
Read-copy-update
Publish a new version by pointer swap; reclaim the old after all readers finish.
More detail
Read-side critical sections have no writes, no atomic RMW and no ordering against the updater. The cost is the grace-period wait and deferred reclamation memory, paid by updaters.
Grace period and quiescent state
An interval after which no pre-swap reader can still hold the old pointer.
More detail
On a non-preemptible kernel every context switch is quiescent; user-space RCU needs explicit quiescent calls. A stalled reader extends the grace period and pins the old copy.
Thread-caching allocator
A malloc that serves common sizes from a per-thread cache before touching shared state.
More detail
Lock acquisitions drop roughly by the cache batch factor B, but idle caches hold memory proportional to the number of live caches. Remote frees need an ownership protocol (mimalloc uses CAS on the owning heap’s list).
Async-signal-safe function
A function callable from a signal handler that may interrupt arbitrary state.
More detail
The POSIX list exists because a handler may run while its thread holds an internal lock (e.g. malloc’s). Calling printf from a handler is the canonical corruption.
Self-pipe and signalfd
Convert signal arrival into a byte or record readable from a pollable descriptor.
More detail
Both move handling to the main loop where only the safe list constrains the handler itself. signalfd is Linux-only and cannot report SIGKILL or SIGSTOP.
Clock source and clock event
Read a hardware-based timeline versus request an interrupt at a future point.
More detail
The roles are distinct even when one device provides both. Suppressing periodic ticks need not stop counter-based time reads.
Monotonic and boottime clocks
Linux elapsed-time domains differ in their handling of suspend.
More detail
MONOTONIC excludes suspended time; BOOTTIME includes it. Neither follows discontinuous wall-clock steps. Monotonic observations may be equal and can receive frequency correction.
Raw clock and frequency adjustment
Uncorrected hardware-based elapsed time differs from a rate-corrected timeline.
More detail
Linux MONOTONIC_RAW excludes suspend and frequency corrections. Raw is not a promise of better accuracy relative to an external reference.
Counter wrap horizon
The maximum interval within which a modular difference identifies elapsed increments.
More detail
The simple conversion requires fewer than one full counter period between base and observation. A mask cannot reconstruct an unknown number of missed wraps.
vDSO
Kernel-provided user-space code can answer supported clock reads without a syscall.
More detail
The library uses deliberately published data and a coherent snapshot protocol; unsupported modes can fall back. This does not expose arbitrary protected kernel memory.
Timer wheel
Bucket deadlines by tick position to reduce ordering work.
More detail
A bounded wheel supports cheap insertion and cancellation with retained node handles. Processing k expirations still costs at least k operations.
Hashed and hierarchical timer wheels
Track repeated turns or progressively coarser time levels.
More detail
Hashed buckets can require repeated distant-timer inspections. Classic hierarchies cascade into finer levels; Linux v6.12 timeout wheels instead use non-cascading coarse batching.
High-resolution timer
Represent deadlines independently of a coarse periodic tick.
More detail
Linux v6.12 uses cached balanced-tree timer queues. Fine representation does not guarantee immediate callback execution or nanosecond hardware accuracy.
Dynamic ticks and NO_HZ
Suppress unnecessary periodic scheduler ticks in eligible conditions.
More detail
NO_HZ_IDLE covers idle CPUs; NO_HZ_FULL extends suppression to eligible single-runnable-task execution. Other interrupts and housekeeping obligations remain.
Timer slack
Permission to coalesce eligible expirations through bounded intentional delay.
More detail
A deadline d with slack s permits scheduling within [d, d+s], not early expiration. PR_SET_TIMERSLACK zero restores the default; eventual thread execution can suffer additional delay.
Slab and object cache
A cache carves backing extents into reusable aligned object slots.
More detail
Constructed-state reuse is distinct from resetting each logical payload. One live object can keep a mostly empty slab from returning its backing.
SLUB per-CPU supply
Local freelists reduce repeated centralized allocation work.
More detail
Linux v6.12 still coordinates remote frees and slow paths; architecture, debugging and PREEMPT_RT affect its locking behavior.
HugeTLB
Linux explicit huge-page mappings and pool accounting.
More detail
Persistent pool pages are reserved for huge-page use and cannot be swapped under the documented v6.12 contract. Contiguous capacity and alignment still constrain allocation.
Transparent huge pages
Kernel-managed larger backing or mappings for eligible regions.
More detail
Traditional PMD-sized THP differs from PTE-mapped multi-size THP. Compaction, sparse waste and region lifetime can outweigh fewer translation walks.
Memory compaction
Move eligible live pages to create larger contiguous free extents.
More detail
Total free bytes alone are insufficient. Moving contents and repairing mappings cost work, and immovable pages can prevent success.
NUMA and first touch
Memory cost depends on CPU-to-memory placement; an allocating touch can choose backing.
More detail
A node need not equal a socket. Reserving addresses or reading shared zero backing need not place private pages near the eventual worker.
NUMA memory policy
Control preferred or allowed nodes for eligible future page allocations.
More detail
Local and preferred policies allow fallback, binding constrains eligibility, and interleave distributes pages. Policy changes do not automatically migrate existing backing.
Automatic NUMA balancing
Fault-sampled placement adjustments that can migrate pages toward users.
More detail
Sampling and migration impose overhead and need stable future reuse to repay it. A sampling fault does not imply storage I/O.
Inverted page table
A base table indexed by physical frame rather than virtual page.
More detail
A hash index can locate a complete context-and-VPN key. Aliases and nonresident mappings require additional bookkeeping.
TLB shootdown
Coordinated retirement of cached mappings on relevant processors.
More detail
Notification is not completion. Reusing retired frames requires the protocol’s local and remote ordering guarantees.
Lazy invalidation
Deferring a flush while preventing stale use before a mandatory repair check.
More detail
Generation checks can support safe re-entry. Linux v6.12 x86 treats freeing page-table pages more strictly because speculative walkers matter too.
Direct map
A predictable kernel virtual alias of mapped physical memory.
More detail
Identity mapping uses equal numeric addresses; other layouts use an offset. The conversion is valid only within its intended mapped domain.
Trampoline
Protected entry/exit code reachable across a translation-context change.
More detail
xv6 maps the same physical code page at the same VA in both views. Its presence in a user-side table does not grant user-mode execution.
Page Table Isolation
Restricted user-side and fuller kernel-side translation views.
More detail
Linux PTI retains transition mappings and switches views on entry/exit. Lower user-table structures can be shared; a table switch need not flush every TLB entry.
Run queue
Structure tracking runnable competitors.
More detail
It need not be FIFO. Per-CPU queues decentralise selection but still coordinate remote wake-ups and migration.
CPU affinity
A locality preference or explicit allowed-CPU set.
More detail
Pinning restricts eligibility; it does not reserve exclusive execution or guarantee cache residency. Linux masks are per thread.
Work stealing
Underloaded workers pull eligible work from other queues.
More detail
Ownership transfers exactly once. Saved delay must justify coordination and lost locality; runtime jobs and kernel threads are different layers.
Scheduling domain
Topology-related CPU set for balancing.
More detail
Nested domains consider progressively wider groups. Shared hardware and allowed masks constrain useful destinations.
Energy-aware scheduling
Placement considering energy and available capacity.
More detail
Compare affected-system energy over one horizon under a performance constraint. Peak watts alone do not identify the best placement.
Deferred interrupt work
Processing handed off from the urgent handler to a later execution context.
More detail
The context determines whether sleeping is allowed: ordinary worker threads and WQ_BH callbacks have different contracts.
top half
The top half is the immediate interrupt handler: it runs as soon as the interrupt is delivered and does only the urgent part.
Go to Deferred interrupt work: top halves, softirqs and workqueues →bottom half
The bottom half is the deferred processing the top half arranges to run later.
Go to Deferred interrupt work: top halves, softirqs and workqueues →softirq
A softirq is one of a small, fixed set of kernel-defined kinds of deferred work, each with a handler and a "pending" flag per CPU.
Go to Deferred interrupt work: top halves, softirqs and workqueues →tasklet
A tasklet is a single postponed function built on top of softirqs, with one extra promise: the same tasklet never runs on two CPUs at once. (Different tasklets can.)
Go to Deferred interrupt work: top halves, softirqs and workqueues →threaded IRQ
A threaded IRQ moves a device's interrupt handling into its own kernel thread.
Go to Deferred interrupt work: top halves, softirqs and workqueues →workqueue
A workqueue takes work items — records naming a function to call and the data it needs — and runs them later on worker threads.
Go to Deferred interrupt work: top halves, softirqs and workqueues →wall clock
A wall clock relates an instant to a calendar epoch.
Go to Kernel clocks: clock sources, monotonic time and the vDSO →CLOCK_REALTIME
Linux's CLOCK_REALTIME provides this kind of time and can change discontinuously when the system time is set.
Go to Kernel clocks: clock sources, monotonic time and the vDSO →monotonic clock
A monotonic clock is a clock whose readings never decrease, whatever happens to the wall clock, so that the difference of two readings is a valid elapsed time.
Go to Kernel clocks: clock sources, monotonic time and the vDSO →CLOCK_MONOTONIC
Linux's CLOCK_MONOTONIC is one.
Go to Kernel clocks: clock sources, monotonic time and the vDSO →CLOCK_BOOTTIME
Linux CLOCK_MONOTONIC excludes suspended time; CLOCK_BOOTTIME includes it.
Go to Kernel clocks: clock sources, monotonic time and the vDSO →Slewing
A time-synchronisation daemon such as an NTP client corrects the kernel's clock in two ways: by stepping it, which makes wall time jump, or by frequency adjustment, often called slewing, which changes the rate of the software clock gradually.
Go to Kernel clocks: clock sources, monotonic time and the vDSO →clock source
A clock source is a hardware counter that ticks at a steady rate, plus the driver code that reads it.
Go to Kernel clocks: clock sources, monotonic time and the vDSO →clock event device
A clock event device instead accepts a future target and raises an interrupt when that target is reached.
Go to Kernel clocks: clock sources, monotonic time and the vDSO →Timestamp counter (TSC)
On x86, the Intel and AMD processor family in most laptops and servers, the timestamp counter (TSC) is not automatically a count of instructions or a counter advancing at the CPU's changing execution frequency.
Go to Kernel clocks: clock sources, monotonic time and the vDSO →Resolution
Resolution is the clock interface's time granularity; accuracy concerns agreement with the intended reference.
Go to Kernel clocks: clock sources, monotonic time and the vDSO →accuracy
Resolution is the clock interface's time granularity; accuracy concerns agreement with the intended reference.
Go to Kernel clocks: clock sources, monotonic time and the vDSO →timer
A timer records a future expiration and the work to make eligible when that expiration arrives.
Go to Kernel timers: timer wheels, high-resolution timers and tickless kernels →timeout
A timeout uses a timer to bound waiting for some other event.
Go to Kernel timers: timer wheels, high-resolution timers and tickless kernels →indexed priority queue
An indexed priority queue also remembers each timer's position, so cancellation can find that timer without scanning the whole queue.
Go to Kernel timers: timer wheels, high-resolution timers and tickless kernels →hashed timer wheel
A hashed timer wheel also stores timers beyond one revolution.
Go to Kernel timers: timer wheels, high-resolution timers and tickless kernels →hierarchical timer wheel
A hierarchical timer wheel uses several arrays with progressively coarser time units.
Go to Kernel timers: timer wheels, high-resolution timers and tickless kernels →Cascading
Cascading is this migration toward finer resolution as expiration approaches.
Go to Kernel timers: timer wheels, high-resolution timers and tickless kernels →Dynamic ticks
Dynamic ticks, also called a tickless kernel, suppress periodic scheduler-tick interrupts when they are unnecessary and arrange the next required event instead.
Go to Kernel timers: timer wheels, high-resolution timers and tickless kernels →scheduling class
A scheduling class is one of the kernel's self-contained scheduling policies, with its own per-CPU run queue and its own rule for picking a task.
Go to Scheduling classes: real-time, deadline and pluggable schedulers above EEVDF →scheduling policy
A thread's scheduling policy attribute, set with sched_setattr or sched_setscheduler, chooses which class schedules it.
Go to Scheduling classes: real-time, deadline and pluggable schedulers above EEVDF →static priority
A real-time thread has a static priority from 1 (lowest) to 99 (highest) that the scheduler never adjusts.
Go to Scheduling classes: real-time, deadline and pluggable schedulers above EEVDF →RT throttling
RT throttling caps how much time real-time and deadline tasks together may use in every accounting period.
Go to Scheduling classes: real-time, deadline and pluggable schedulers above EEVDF →Earliest Deadline First
Earliest Deadline First (EDF) is the preemptive policy that always runs the ready job with the earliest absolute deadline.
Go to Scheduling classes: real-time, deadline and pluggable schedulers above EEVDF →Constant Bandwidth Server
Linux enforces it with the Constant Bandwidth Server (CBS), from Luca Abeni and Giorgio Buttazzo (1998).
Go to Scheduling classes: real-time, deadline and pluggable schedulers above EEVDF →Throttled (deadline task)
When it reaches zero before the task finishes, the task is throttled: it may not run again until its current deadline, when its runtime is refilled and its deadline moves one period later.
Go to Scheduling classes: real-time, deadline and pluggable schedulers above EEVDF →admission control
So the kernel runs admission control: when a thread asks to become SCHED_DEADLINE, or to change its parameters, the kernel adds up the declared bandwidths of the deadline tasks sharing that group of CPUs and refuses the request, with EBUSY, if the total would exceed a cap.
Go to Scheduling classes: real-time, deadline and pluggable schedulers above EEVDF →Bandwidth (deadline task)
So the kernel runs admission control: when a thread asks to become SCHED_DEADLINE, or to change its parameters, the kernel adds up the declared bandwidths of the deadline tasks sharing that group of CPUs and refuses the request, with EBUSY, if the total would exceed a cap.
Go to Scheduling classes: real-time, deadline and pluggable schedulers above EEVDF →sched_ext
Linux 6.12 added sched_ext, a class whose picking rule is supplied at run time as a BPF program: code that the kernel checks with a verifier before running it, so that it cannot crash the kernel or loop forever (eBPF programs and hooks explains the mechanism).
Go to Scheduling classes: real-time, deadline and pluggable schedulers above EEVDF →Kernel preemption
Kernel preemption is the ability of the scheduler to switch a CPU away from a task while that task is executing kernel code, rather than only when the task blocks, yields or returns to user space.
Go to Kernel preemption: from voluntary preemption points to PREEMPT_RT →scheduling latency
The quantity at stake is scheduling latency: the time from the event that makes a task runnable — a timer expiring, a packet arriving — to the moment that task actually starts running.
Go to Kernel preemption: from voluntary preemption points to PREEMPT_RT →preemption point
A preemption point is a place in kernel code where the author has checked that no spinlock is held and no per-CPU update is half done, so switching is safe.
Go to Kernel preemption: from voluntary preemption points to PREEMPT_RT →need-resched flag
When a wake-up or the timer tick decides the running task should give way, the scheduler sets that task's need-resched flag (TIF_NEED_RESCHED in Linux).
Go to Kernel preemption: from voluntary preemption points to PREEMPT_RT →Voluntary preemption
Voluntary preemption is the kernel configuration that adds such points throughout the kernel.
Go to Kernel preemption: from voluntary preemption points to PREEMPT_RT →preempt count
The preempt count (preempt_count) is incremented on entry to every region in which switching is unsafe and decremented on exit; preemption is allowed only when it is zero.
Go to Kernel preemption: from voluntary preemption points to PREEMPT_RT →Full preemption
Full preemption (the PREEMPT configuration) is this arrangement: becomes the longest region with a nonzero preempt count — in practice the longest spinlock hold or explicitly non-preemptible section.
Go to Kernel preemption: from voluntary preemption points to PREEMPT_RT →PREEMPT_RT
PREEMPT_RT is Linux's real-time configuration, and it attacks directly by making most of those regions preemptible.
Go to Kernel preemption: from voluntary preemption points to PREEMPT_RT →Lazy preemption
Lazy preemption, merged in Linux 6.13 (January 2025), separates the two cases with a second flag: a wake on behalf of an ordinary, fair-class task waits for the next timer tick — at most about 1 ms at a 1000 Hz tick (kernel timers covers the tick rate) — while a wake for a real-time or deadline task preempts exactly as under full preemption.
Go to Kernel preemption: from voluntary preemption points to PREEMPT_RT →PREEMPT_DYNAMIC
Because no one model suits every machine, most distributions build kernels with PREEMPT_DYNAMIC, which selects one model at boot with the preempt= parameter (LWN, "The long road to lazy preemption", 2024).
Go to Kernel preemption: from voluntary preemption points to PREEMPT_RT →Placement
Placement chooses a CPU for a task; selection chooses which of that CPU's competitors runs next.
Go to Per-CPU run queues and cache affinity →selection
Placement chooses a CPU for a task; selection chooses which of that CPU's competitors runs next.
Go to Per-CPU run queues and cache affinity →per-CPU run-queue
In a per-CPU run-queue design, each CPU normally updates and selects from its own structure.
Go to Per-CPU run queues and cache affinity →serial path
A serial path is one resource that competing CPUs must use one at a time.
Go to Per-CPU run queues and cache affinity →Cache affinity
Cache affinity is the preference to run a task where useful cached state may survive.
Go to Per-CPU run queues and cache affinity →Pinning (CPU)
A singleton mask is often called pinning: the thread may execute only on that CPU.
Go to Per-CPU run queues and cache affinity →Load imbalance
Load imbalance is unequal demand relative to available capacity.
Go to Load balancing and work stealing across cores →makespan
With no arrivals, blocking or overhead, the batch's makespan—time until all jobs finish—is .
Go to Load balancing and work stealing across cores →Push migration
Push migration starts from a CPU or balancing controller that identifies excess work and sends an eligible task elsewhere.
Go to Load balancing and work stealing across cores →Pull migration
Pull migration starts from an underloaded CPU that looks for work to receive.
Go to Load balancing and work stealing across cores →SMT sibling
SMT siblings are hardware threads sharing a physical core's resources.
Go to Load balancing and work stealing across cores →socket
A socket is a processor package that can contain multiple cores and shared caches.
Go to Load balancing and work stealing across cores →Hysteresis
Hysteresis means requiring an improvement margin before switching decisions, so tiny estimate changes do not cause back-and-forth migration.
Go to Load balancing and work stealing across cores →Power
Power is the rate of energy consumption, measured in watts, or joules per second.
Go to Energy-aware scheduling on heterogeneous cores →Energy
Energy is the accumulated consumption over an interval.
Go to Energy-aware scheduling on heterogeneous cores →Heterogeneous CPU system
A heterogeneous CPU system contains processors with different performance characteristics. “Big” and “little” are informal labels for higher-performance and lower-power core types, not guarantees that one always wins on energy.
Go to Energy-aware scheduling on heterogeneous cores →capacity
A capacity estimate expresses how much work a CPU can deliver relative to a reference.
Go to Energy-aware scheduling on heterogeneous cores →energy model
An energy model associates performance states with estimated costs so placement can compare alternatives.
Go to Energy-aware scheduling on heterogeneous cores →performance domain
A performance domain groups CPUs whose performance-state choice is coupled; an operating performance point (OPP) identifies one such state.
Go to Energy-aware scheduling on heterogeneous cores →operating performance point
A performance domain groups CPUs whose performance-state choice is coupled; an operating performance point (OPP) identifies one such state.
Go to Energy-aware scheduling on heterogeneous cores →Utilisation tracking
Utilisation tracking estimates recent task demand.
Go to Energy-aware scheduling on heterogeneous cores →frequency governor
A frequency governor chooses requested operating frequency.
Go to Energy-aware scheduling on heterogeneous cores →Alias (mapping)
An alias is another virtual mapping of the same physical storage.
Go to Inverted page tables: one entry per frame →Generation (TLB)
A generation is a version number for changes to a mapping.
Go to TLB shootdown: invalidating translations on every core →Batching (TLB invalidation)
Batching combines several mapping changes before coordinating one invalidation round.
Go to TLB shootdown: invalidating translations on every core →kernel address space
The kernel address space is the set of virtual mappings through which privileged code reaches its instructions, stacks, data, physical-memory management structures and device interfaces.
Go to The kernel's address space: shared mappings and split tables →supervisor-only mapping
A supervisor-only mapping is present for privileged use but denies access from user code. “Present in the user's page table” and “accessible to user mode” are different statements.
Go to The kernel's address space: shared mappings and split tables →identity mapping
An identity mapping is the special case where the virtual and physical addresses are numerically equal.
Go to The kernel's address space: shared mappings and split tables →guard page
Kernel stacks illustrate why a direct map is not the only useful view. xv6 also maps each kernel stack in a high virtual region with an inaccessible guard page below it.
Go to The kernel's address space: shared mappings and split tables →Trampoline (kernel entry)
A trampoline is a small entry/exit code region mapped at the same virtual address in both translation contexts.
Go to The kernel's address space: shared mappings and split tables →Page Table Isolation (PTI)
Page Table Isolation (PTI) is Linux's split-view design.
Go to The kernel's address space: shared mappings and split tables →text relocation
The direct fix is a text relocation: leave every absolute address blank in the file and let the loader write them in once the base is chosen.
Go to Position-independent code: running at any address →PC-relative addressing
Same-object code references (an internal helper, a branch inside a function) become PC-relative: the instruction encodes its displacement from itself, that is from the program counter, to the target, and that displacement is a link-time constant requiring no relocation of any kind.
Go to Position-independent code: running at any address →Global offset table (GOT)
The global offset table (GOT) is a table in a position-independent object's writable data holding one address per symbol whose location is unknown at link time.
Go to Position-independent code: running at any address →interposition
One consequence surprises most readers: the first definition the loader finds wins, so a name this object exports may still bind to another object's version — an effect called interposition, whose lookup rules dynamic linking lays out.
Go to Position-independent code: running at any address →Position-independent executable (PIE)
The same mechanism turned inward, a position-independent executable (PIE), lets the kernel place even the main program at a random address, a defence that address-space layout randomisation develops.
Go to Position-independent code: running at any address →completion model
In the completion model, a program submits a whole operation — descriptor, offset, buffer, length — and is later notified that the operation has finished, with its result; in the readiness model, it is notified only that an operation could now proceed, and must then perform it.
Go to Readiness versus completion: why Linux AIO fell short →readiness model
In the completion model, a program submits a whole operation — descriptor, offset, buffer, length — and is later notified that the operation has finished, with its result; in the readiness model, it is notified only that an operation could now proceed, and must then perform it.
Go to Readiness versus completion: why Linux AIO fell short →Proactor pattern
The proactor pattern is an event loop driven by completions rather than readiness.
Go to Readiness versus completion: why Linux AIO fell short →Submission queue (io_uring)
io_uring, which Axboe merged in Linux 5.1 in 2019, makes the communication channel itself the interface. io_uring_setup(entries, params) returns a file descriptor; three regions hang off it by mmap — the call that maps a file or device into the process's address space: the submission queue ring (SQ), the completion queue ring (CQ), and a flat array of submission queue entries (SQEs).
Go to io_uring: asynchronous system calls through shared rings →Completion queue (io_uring)
io_uring, which Axboe merged in Linux 5.1 in 2019, makes the communication channel itself the interface. io_uring_setup(entries, params) returns a file descriptor; three regions hang off it by mmap — the call that maps a file or device into the process's address space: the submission queue ring (SQ), the completion queue ring (CQ), and a flat array of submission queue entries (SQEs).
Go to io_uring: asynchronous system calls through shared rings →io_uring
io_uring is Linux's asynchronous system-call interface: a process and the kernel share two ring buffers in memory, the process describes operations as submission queue entries on one ring, and the kernel posts completion queue entries carrying each result on the other, so that a single system call — or none — can cover many operations.
Go to io_uring: asynchronous system calls through shared rings →SQE (submission queue entry)
io_uring is Linux's asynchronous system-call interface: a process and the kernel share two ring buffers in memory, the process describes operations as submission queue entries on one ring, and the kernel posts completion queue entries carrying each result on the other, so that a single system call — or none — can cover many operations.
Go to io_uring: asynchronous system calls through shared rings →CQE (completion queue entry)
io_uring is Linux's asynchronous system-call interface: a process and the kernel share two ring buffers in memory, the process describes operations as submission queue entries on one ring, and the kernel posts completion queue entries carrying each result on the other, so that a single system call — or none — can cover many operations.
Go to io_uring: asynchronous system calls through shared rings →SQPOLL
IORING_SETUP_SQPOLL (SQPOLL mode) starts a kernel thread that busy-polls the SQ tail: the application submits by storing into shared memory and never calls io_uring_enter for submission at all.
Go to io_uring: asynchronous system calls through shared rings →User-space thread
User-space threads — also called green threads — are threads the language runtime creates, switches and schedules itself, without the kernel knowing they exist.
Go to User-space threads: M:N scheduling over asynchronous I/O →Carrier thread
M:N scheduling runs M such threads on N kernel threads, the carrier threads, with M usually far larger than N; when a user-space thread would block, the runtime parks it and runs another on the same carrier.
Go to User-space threads: M:N scheduling over asynchronous I/O →linearization point
Each operation's meaning collapses into its successful CAS, which is therefore its linearization point, the single instant at which the operation appears to take effect: before it, nothing changed; after it, the new chain is the stack.
Go to Lock-free stacks and the ABA problem →obstruction-free
obstruction-free code guarantees that a thread completes if it runs alone, so progress is promised only once contention is removed;
Go to Lock-free stacks and the ABA problem →lock-free
An implementation is lock-free if, whenever threads are taking steps, some operation completes within a finite number of steps, whatever the scheduler does.
Go to Lock-free stacks and the ABA problem →wait-free
It is wait-free if every operation completes within a bounded number of its own thread's steps, however the other threads are scheduled.
Go to Lock-free stacks and the ABA problem →tagged pointer
A tagged pointer widens the CAS word to carry a counter beside the address, so A→B→A reads as distinct versions and the stale CAS fails — practical where double-width compare-and-swap exists.
Go to Lock-free stacks and the ABA problem →retired
A removed node passes through four states: reachable from the shared structure, then unlinked by the remover's CAS, then retired (the remover is done and wants the memory back), then freed.
Go to Safe memory reclamation: hazard pointers and epochs →limbo list
A deleted object goes onto the current epoch's limbo list, a list of retired objects waiting for the epoch to move on — allowed only once no shared references to it remain and none can still be created (the deleting operation owns that obligation; Fraser's skip list, where a node can be unlinked while its insertion completes at higher levels, needs a deferral flag so the last-finishing operation files the limbo en…
Go to Safe memory reclamation: hazard pointers and epochs →read-side critical section
Five calls carry the contract in the Linux kernel: rcu_read_lock() and rcu_read_unlock() delimit a read-side critical section in which a thread may hold a protected reference; rcu_dereference() reads the published pointer; rcu_assign_pointer() performs the publishing swap with release ordering; and synchronize_rcu() — or its callback-queuing sibling call_rcu() — waits out the reclamation window before the removed …
Go to Read-copy-update: readers that never block or write →quiescent state
Such events are quiescent states.
Go to Read-copy-update: readers that never block or write →grace period
The interval from the removal until every CPU has passed a quiescent state is a grace period; at its end, every reader that could hold the removed node has exited its section and the node may be freed.
Go to Read-copy-update: readers that never block or write →transfer cache
TCMalloc names these explicitly: the front-end is a per-thread or per-CPU object cache, the middle-end is a per-size-class transfer cache, a shared store of object batches that one cache hands back and another picks up, plus a central free list, and the back-end is a pageheap. jemalloc's equivalent layers are the per-thread tcache, the shared arenas — an arena is an independent heap with its own lock and bins, so …
Go to Thread-caching allocators: jemalloc, tcmalloc and mimalloc →arena
TCMalloc names these explicitly: the front-end is a per-thread or per-CPU object cache, the middle-end is a per-size-class transfer cache, a shared store of object batches that one cache hands back and another picks up, plus a central free list, and the back-end is a pageheap. jemalloc's equivalent layers are the per-thread tcache, the shared arenas — an arena is an independent heap with its own lock and bins, so …
Go to Thread-caching allocators: jemalloc, tcmalloc and mimalloc →remote free
Such a remote free — freeing an object that another thread's cache or heap handed out — raises the question of where the object should go.
Go to Thread-caching allocators: jemalloc, tcmalloc and mimalloc →RSS
Cached and span-level free memory keeps RSS — the resident set size, the physical memory the process actually occupies — high even after the application frees everything — the same lesson the baseline allocator taught about free not lowering RSS, now multiplied by T or P caches. jemalloc's answer is time-based: unused dirty pages are returned to the kernel along a sigmoidal decay curve, ten seconds by default from…
Go to Thread-caching allocators: jemalloc, tcmalloc and mimalloc →restartable sequence
TCMalloc's restartable sequences deserve a word: a restartable sequence (rseq, a Linux system call merged in Linux 4.18 in 2018) is a short block of user-space instructions that the kernel restarts from the beginning if the thread was preempted or migrated mid-way, so push and pop need neither locks nor contended atomics; that is a Linux-specific mechanism underneath the abstraction.
Go to Thread-caching allocators: jemalloc, tcmalloc and mimalloc →IOVA
An IOVA — an I/O virtual address, also written "DMA address" — is what a device sees where software would see a physical address.
Go to The IOMMU: page tables for devices →IOMMU domain
The IOMMU indexes a root table with that identifier to find the device's context, and the context names a translation domain — a private set of second-level page tables walked just as multi-level page tables resolve a process's virtual address, including the same sparse materialisation: IOVA space the device never uses costs no physical pages, only unmapped top-level entries.
Go to The IOMMU: page tables for devices →IOTLB
Every map/unmap pair writes page-table entries and then invalidates the IOMMU's translation cache — the IOTLB — so a stale mapping cannot survive revocation; where devices cache translations themselves, invalidation must reach them too.
Go to The IOMMU: page tables for devices →Pinned memory
Pinned (page-locked) memory is host memory whose virtual-to-physical mapping the kernel has promised not to change while the pin is held: the frames are not swapped, migrated or compacted, so a device may be given their addresses and DMA to them at any time.
Go to Accelerators and the OS: pinned memory, device page tables and unified memory →Heterogeneous memory management
Heterogeneous memory management (HMM) is the Linux infrastructure that lets a device share a process's address space: it keeps a device's page table a coherent mirror of the CPU page table, faults pages in for the device through the ordinary page-fault path, and represents device memory as pages the kernel's migration machinery can move data to and from.
Go to Accelerators and the OS: pinned memory, device page tables and unified memory →NAPI
NAPI is the Linux network stack's event-handling mechanism: a device's interrupt schedules a NAPI instance, which then polls the device from softirq context, up to a budget per call, and re-enables the interrupt only once the device has no more work.
Go to The kernel packet path: from NIC ring to socket buffer →Generic Receive Offload
Generic Receive Offload (GRO) may first merge several consecutive segments of the same TCP flow into one larger packet — the receive-side half of the offload family segmentation offloads covers — so the layers above do their per-packet work once for the batch.
Go to The kernel packet path: from NIC ring to socket buffer →Receive-side scaling
Receive-side scaling (RSS) decides which queue a packet joins.
Go to The kernel packet path: from NIC ring to socket buffer →Garbage collection (SSD)
Garbage collection picks a victim block, copies its still-valid pages elsewhere (they go through the same write frontier as host writes), then erases it into the free pool.
Go to SSD garbage collection, wear levelling and write amplification →Spare factor
The spare factor is the fraction of flash the host can never fill, and the matching over-provisioning factor.
Go to SSD garbage collection, wear levelling and write amplification →over-provisioning
The spare factor is the fraction of flash the host can never fill, and the matching over-provisioning factor.
Go to SSD garbage collection, wear levelling and write amplification →TRIM
The TRIM command closes that gap (TRIM is its name in the ATA command set used by SATA — Serial ATA, the standard interface for PC disks and SSDs; SCSI calls it UNMAP and NVMe calls it Deallocate, and Linux sends it through fstrim or the discard mount option): the host tells the device "these logical blocks are gone," letting the FTL drop their mappings and mark the physical pages dead immediately.
Go to SSD garbage collection, wear levelling and write amplification →Wear levelling
Wear levelling works the opposite trade.
Go to SSD garbage collection, wear levelling and write amplification →NVMe
NVM Express (NVMe — NVM for non-volatile memory) is the device interface designed for flash from the start: a register and command protocol for storage attached directly to PCIe, with no disk-era controller in between.
Go to NVMe: queue pairs sized for parallel media →Queue pair (NVMe)
An NVMe queue pair is a submission queue and a completion queue, both rings in host memory: the host places commands in the submission queue and announces them by writing a doorbell register; the controller fetches them by DMA and posts a completion entry for each into the completion queue.
Go to NVMe: queue pairs sized for parallel media →mq-deadline
mq-deadline keeps, per I/O priority class and direction, a sector-sorted list for efficient batches plus a FIFO list carrying an expiry time.
Go to I/O schedulers: mq-deadline, BFQ, Kyber and none →BFQ
BFQ (Budget Fair Queueing) is a proportional-share scheduler: every process doing I/O gets a queue with a weight, and queues are served one at a time for a budget measured in sectors.
Go to I/O schedulers: mq-deadline, BFQ, Kyber and none →Kyber
Kyber takes the opposite approach: it barely reorders at all.
Go to I/O schedulers: mq-deadline, BFQ, Kyber and none →Erasure
Under the fail-stop model a dead disk is a located loss — an erasure: the controller knows exactly which positions are missing, so a single checksum per stripe suffices.
Go to RAID 4, 5 and 6: parity and the small-write problem →parity
The checksum is parity: for blocks , store a parity block
Go to RAID 4, 5 and 6: parity and the small-write problem →small-write problem
This is the small-write problem.
Go to RAID 4, 5 and 6: parity and the small-write problem →write hole
This is the write hole, the parity form of the consistent-update problem from mirroring.
Go to RAID 4, 5 and 6: parity and the small-write problem →Degraded mode
When a disk dies in a redundant array, the array enters degraded mode: it keeps serving requests, but reads touching the dead disk are reconstructed on the fly — a one-block read becomes an disk fan-out — and redundancy is gone.
Go to RAID rebuild windows and mean time to data loss →Rebuild (RAID)
The array stays degraded until rebuild: a spare (hot or newly swapped) is written block-by-block with the reconstructed contents of the dead disk, which for RAID 5 means reading every block of every surviving disk and XORing each stripe.
Go to RAID rebuild windows and mean time to data loss →Mean time to data loss
The mean time to data loss (MTTDL) of a redundant array is the expected time until failures overlap in a way its redundancy cannot repair — for a single-parity array, until a second disk fails inside the repair window (mean time to repair) that follows a first.
Go to RAID rebuild windows and mean time to data loss →Unrecoverable read error
Rebuild adds a quieter failure source: it is a full-surface read of every survivor, and every read carries a chance of an unrecoverable read error (URE) — a sector whose internal error correction cannot fix it.
Go to RAID rebuild windows and mean time to data loss →lock file
So a lock file is a file whose existence means "someone holds the lock":
Go to File locking: flock, POSIX record locks, OFD locks and lock files →advisory lock
An advisory lock is a lock the kernel records and arbitrates between processes that request it, but does not enforce on I/O: a process that never asks for the lock can still read and write the file, subject only to its permissions.
Go to File locking: flock, POSIX record locks, OFD locks and lock files →record lock
A record lock (also called a byte-range lock) is requested through fcntl with a struct flock describing the range:
Go to File locking: flock, POSIX record locks, OFD locks and lock files →Open file description lock (OFD lock)
Open file description locks, or OFD locks, combine the two families: byte ranges like POSIX record locks, owned by the open file description like flock.
Go to File locking: flock, POSIX record locks, OFD locks and lock files →mandatory lock
The obvious alternative is a mandatory lock, one the kernel enforces on every read and write: an access that overlaps another process's incompatible lock blocks, or fails with EAGAIN on a non-blocking descriptor.
Go to File locking: flock, POSIX record locks, OFD locks and lock files →Lease (NFSv4)
NFS version 4 puts locking into the protocol and ties it to a lease: the client must renew its state with the server within a lease period, 90 seconds by default on a Linux server.
Go to File locking: flock, POSIX record locks, OFD locks and lock files →Page reclaim
The consequence is that reclaim, freeing frames that are in use, is how the kernel finds memory for most allocations on a busy machine.
Go to Linux page reclaim: file versus anonymous pages, and MGLRU →refault
A clean file page already lives on disk: drop it and the only price is a possible future refault, reading it back when it is next needed.
Go to Linux page reclaim: file versus anonymous pages, and MGLRU →Watermark (zone)
Linux divides each NUMA node's memory into zones by what hardware can address (on x86-64, DMA, DMA32 and Normal), and each zone keeps three thresholds measured in free pages, its watermarks.
Go to Linux page reclaim: file versus anonymous pages, and MGLRU →kswapd
Dropping below LOW fails the fast-path check; the allocator then wakes kswapd, the kernel's background reclaim thread (one per NUMA node), and retries the allocation against MIN, which usually succeeds because MIN reserves a cushion for exactly this moment. kswapd keeps scanning in the background until some zone reaches HIGH; the gap between LOW and HIGH is the hysteresis that stops it flickering on and off.
Go to Linux page reclaim: file versus anonymous pages, and MGLRU →direct reclaim
If free pages fall under MIN before kswapd catches up, the allocating thread performs direct reclaim itself: it scans, isolates and frees pages inside its own timeslice, paying reclaim latency synchronously.
Go to Linux page reclaim: file versus anonymous pages, and MGLRU →shadow entry
A shadow entry stays behind in the page-cache XArray slot the page occupied, recording the eviction time in units of a per-lruvec counter, nonresident_age, that advances on every eviction.
Go to Linux page reclaim: file versus anonymous pages, and MGLRU →swappiness
The kernel tracks a running reclaim cost for each type — essentially how recently pages of that type refaulted — and divides scan pressure in inverse proportion to cost, weighted by swappiness, the vm.swappiness setting (0 to 200, default 60) that swap in practice introduced.
Go to Linux page reclaim: file versus anonymous pages, and MGLRU →Multi-generational LRU (MGLRU)
The multi-generational LRU (MGLRU), developed by Yu Zhao at Google and merged in Linux 6.1 (2022), keeps the same goals but replaces the active/inactive pair with a sliding window of generations per lruvec.
Go to Linux page reclaim: file versus anonymous pages, and MGLRU →Zero-copy I/O
Zero-copy I/O is the family of interfaces that move data between kernel objects without the CPU copying it through a user buffer.
Go to Zero-copy I/O: sendfile, splice and MSG_ZEROCOPY →Socket error queue
The kernel therefore owes the caller a notification, delivered on the socket's error queue, a side queue of messages about sent data that is read separately from ordinary received data and is also used for transmit timestamps.
Go to Zero-copy I/O: sendfile, splice and MSG_ZEROCOPY →latent sector error
The loud failure is the latent sector error (LSE): a sector that can no longer be read at all, detected by the drive's own error-correcting code and reported as a read failure.
Go to End-to-end data integrity: checksums and scrubbing →Silent block corruption
Silent block corruption returns wrong data with no error.
Go to End-to-end data integrity: checksums and scrubbing →misdirected write
A misdirected write writes the right data to the wrong address — corrupting the victim block and leaving the intended one stale.
Go to End-to-end data integrity: checksums and scrubbing →lost write
A lost write reports success while never persisting at all.
Go to End-to-end data integrity: checksums and scrubbing →checksum
A checksum is a small fixed-size value computed from a block's contents at write time and stored apart from the block, so that recomputing it at read time and comparing the two detects most changes to the contents.
Go to End-to-end data integrity: checksums and scrubbing →CRC
Fletcher in 1982, keeps two running sums and detects all single-bit errors — and double-bit ones under its block-length bound — at a fraction of a CRC's cost.
Go to End-to-end data integrity: checksums and scrubbing →Scrubbing
Scrubbing closes the window: the system periodically walks all data and checksums in the background.
Go to End-to-end data integrity: checksums and scrubbing →shadow paging
The same argument climbs the tree one level at a time — this design is shadow paging, the database technique of writing new pages to fresh locations and flipping one root pointer, recast as a file system.
Go to Copy-on-write file systems: never overwrite in place →copy-on-write file system
A copy-on-write file system never overwrites a live block: every change is written to free space, each ancestor that points at a changed block is itself rewritten to a new location, and an update becomes visible only when a single root pointer is switched to the new tree.
Go to Copy-on-write file systems: never overwrite in place →uberblock
In ZFS that point is the uberblock, the root of one big B-tree-shaped pointer structure whose leaves are data and whose interior nodes carry each child's address and checksum — a checksum being a small digest recomputed on read to catch corruption — the idea end-to-end data integrity takes up later in this module.
Go to Copy-on-write file systems: never overwrite in place →Transaction group (ZFS)
ZFS batches updates into transaction groups — one group open and accepting writes, one quiescing, one syncing to disk at any time, with a group closing every few seconds — so one uberblock write commits an entire batch of leaf updates at once, amortising the root switch over arbitrarily much work.
Go to Copy-on-write file systems: never overwrite in place →Segment (LFS)
LFS buffers every update — data blocks and metadata alike — into an in-memory segment.
Go to Log-structured file systems: every write appended →inode map
LFS answers with one level of indirection: the inode map (imap), an array from inode number to the disk address of the inode's most recent version.
Go to Log-structured file systems: every write appended →checkpoint region
To bootstrap, LFS keeps one fixed structure — the checkpoint region (CR) — holding pointers to the latest imap pieces.
Go to Log-structured file systems: every write appended →Roll-forward recovery
LFS rolls forward: from the last consistent checkpoint it reads the segments written afterward; each segment carries a summary block listing what the segment contains, so recovery sees every new inode and data block, updates the in-memory imap accordingly, and adjusts the segment usage table — LFS's per-segment count of how many bytes are still live — for the deletions and overwrites the new entries imply.
Go to Log-structured file systems: every write appended →Segment cleaning
Segment cleaning is the maintenance loop that recycles them: pick victim segments, read each one in, copy its still-live blocks to fresh segments at the frontier, and mark the victims clean.
Go to Segment cleaning: the cost of a log →segment summary block
Instead each segment carries a segment summary block recording, for every block, which file and offset it belongs to plus a version identifier.
Go to Segment cleaning: the cost of a log →Write cost (LFS)
Define write cost as the total disk-busy time per unit of new data written, normalised so that streaming to a clean disk at full bandwidth is 1.0.
Go to Segment cleaning: the cost of a log →Greedy cleaning
The obvious victim policy is greedy: always clean the segment with the smallest live fraction.
Go to Segment cleaning: the cost of a log →Hot and cold data
Real workloads are not uniform; they have hot data, overwritten often, and cold data, rarely touched.
Go to Segment cleaning: the cost of a log →Cost-benefit cleaning
The fix is cost-benefit victim selection.
Go to Segment cleaning: the cost of a log →tracepoint
A tracepoint is a call site a kernel developer placed deliberately — trace_sched_switch(), trace_block_rq_issue() — marking a semantic event with a fixed set of fields.
Go to Tracing mechanisms: tracepoints, kprobes, uprobes and ftrace →kprobe
A kprobe (kernel probe) instruments an arbitrary instruction address.
Go to Tracing mechanisms: tracepoints, kprobes, uprobes and ftrace →kretprobe
A kretprobe instruments the return: at function entry its handler swaps the stacked return address for a trampoline address and saves the original in a pooled kretprobe_instance.
Go to Tracing mechanisms: tracepoints, kprobes, uprobes and ftrace →optimized kprobe
Where the instruction window allows it, an optimized kprobe replaces five bytes with a direct jmp to a detour buffer containing the copied instructions and a jump back — the trap disappears entirely.
Go to Tracing mechanisms: tracepoints, kprobes, uprobes and ftrace →uprobe
A uprobe (user-space probe) is placed by file offset: the user computes where the function starts (the disassembler objdump -T /bin/zsh shows zfree at offset 0x46420) and writes p /bin/zsh:0x46420 to uprobe_events; the kernel patches a breakpoint into the inode's pages, so every process mapping that file traps there.
Go to Tracing mechanisms: tracepoints, kprobes, uprobes and ftrace →ftrace
Dynamic patching still needs a per-site registration. ftrace, the kernel's built-in function tracer (controlled through files under /sys/kernel/tracing), instead compiles the kernel with the compiler's profiling flag -pg, which emits a call to the profiling hook (__fentry__ on x86) at the top of nearly every function, and a build-time pass (recordmcount, the job objtool performs on modern builds) collects every su…
Go to Tracing mechanisms: tracepoints, kprobes, uprobes and ftrace →vermagic
Two compatibility checks ride along, and the loader's other gates are worth knowing by name: mechanismwhat it does vermagic stringrecords the kernel version and build configuration the module was compiled against; a mismatch fails the load with ENOEXEC rather than corrupting the running kernel CONFIG_MODVERSIONSevery exported symbol also carries a CRC of its argument and return types, so a module built against a d…
Go to Loadable kernel modules: extending the kernel with full trust →rootkit
A malicious module is total: it can hide files and processes, intercept credentials, and patch anything writable — the classic chassis for a rootkit, malware that hides its own presence by rewriting what the kernel reports.
Go to Loadable kernel modules: extending the kernel with full trust →Tainted kernel
Everything else is marking: an invalid signature or a proprietary or out-of-tree build leaves the kernel tainted — a record upstream bug reports may be screened on, not a restriction on anything the module itself can do.
Go to Loadable kernel modules: extending the kernel with full trust →Microkernel
A µ-kernel (microkernel) is a kernel reduced to the few mechanisms that must be privileged, with everything else running as ordinary processes.
Go to Microkernels: moving services into user space →trusted computing base
Containment shrinks the trusted computing base (TCB) — the code whose correctness every security guarantee depends on — by moving code out. seL4, an L4-family kernel built at NICTA, Australia's national ICT research centre, completes the argument by shrinking what remains until it can be verified.
Go to Microkernels: moving services into user space →JIT compiler
Two things then happen in the kernel before the program exists: the verifier statically proves it safe, and a just-in-time (JIT) compiler — one that translates code to the host's machine instructions when it is loaded rather than ahead of time — translates it to native machine code — JITed eBPF is, per the kernel's own design documentation, "indistinguishable from native kernel C code".
Go to eBPF programs and hooks: sandboxed code inside the kernel →eBPF verifier
The eBPF verifier is the part of the kernel's bpf() load path that analyses a program without running it and admits it only if it can show that every execution terminates, reads no uninitialised register or stack slot, and accesses only memory the program's type entitles it to; a program it cannot prove safe is rejected.
Go to The eBPF verifier: static proof of safety →Tnum
Scalars carry more than a type: the verifier tracks unsigned bounds (umin/umax), signed bounds (smin/smax), and a tnum ("tracked number") — a (value; mask) pair in which the mask marks which bits are unknown and the value gives the known ones.
Go to The eBPF verifier: static proof of safety →abstract interpretation
This is abstract interpretation — running a program over an abstract domain (here types, bounds and tnums) that over-approximates every concrete value it could hold, so that a property proved for the abstraction holds for every real run.
Go to The eBPF verifier: static proof of safety →eBPF map
An eBPF map is a kernel-owned data structure with a fixed type, key size, value size and capacity, created through bpf() and reachable both from verified programs, through helper calls, and from user space, through a file descriptor; it outlives any single program invocation.
Go to eBPF maps: shared state between kernel programs and user space →poll-mode driver
Its poll-mode driver (PMD) is a user-space driver built on VFIO: its documentation describes the PMD as user-space APIs that access RX and TX descriptors "directly without any interrupts", the single sanctioned exception being a link-status-change interrupt.
Go to Kernel bypass: DPDK, SPDK and user-space drivers →Mandatory access control
Mandatory access control (MAC) is access control in which a system-wide policy, set by the administrator and enforced by the kernel, decides whether a subject may perform an operation on an object; no user, including the object's owner, can change the policy's verdict by changing the object's permissions.
Go to Mandatory access control: LSM, SELinux and AppArmor →Linux Security Modules (LSM)
The Linux Security Modules (LSM) framework is the result.
Go to Mandatory access control: LSM, SELinux and AppArmor →type enforcement
Its core enforcement is type enforcement: types name both subject domains (conventionally *_t, like httpd_t) and object classes' labels (like httpd_sys_content_t), and access requires an explicit allow rule binding them:
Go to Mandatory access control: LSM, SELinux and AppArmor →seccomp
seccomp (“secure computing”) is the Linux mechanism by which a thread restricts the system calls it, and every process it later becomes, may make.
Go to seccomp: filtering system calls →Privilege separation
Privilege separation splits one program into a small privileged part and a larger unprivileged part that handles untrusted input; the unprivileged part obtains the few privileged operations it needs by sending requests over an interprocess channel, and the privileged part checks each request against the program's current state before performing it.
Go to Privilege separation and broker sandboxes →network namespace
A network namespace is a private copy of the whole network stack that the packet path traced: interface devices, the IPv4 and IPv6 stacks, routing tables, firewall rules, /proc/net (a symlink to the process's own /proc/<pid>/net), /sys/class/net, the per-namespace sysctl tree under /proc/sys/net, port allocation, and the UNIX abstract socket namespace.
Go to Network namespaces: private network stacks and veth pairs →veth pair
The connective tissue is the veth pair (virtual Ethernet): two linked network devices always created together, like a virtual crossover cable — a packet transmitted on one end is immediately received on the other.
Go to Network namespaces: private network stacks and veth pairs →Linux bridge
To fan out to N containers — and to let them reach each other — the host ends are enslaved to a kernel bridge device, a virtual layer-2 (Ethernet) switch living in the initial namespace.
Go to Network namespaces: private network stacks and veth pairs →user namespace
A user namespace gives its members their own numbering of user and group IDs, translated to real IDs outside by a mapping table, and its own set of capabilities, which count only for resources that belong to namespaces it owns.
Go to User namespaces: root inside, unprivileged outside →Rootless container
A container that uses one to run as root inside while its owner is unprivileged outside is called a rootless container.
Go to User namespaces: root inside, unprivileged outside →kuid
The kernel keeps one true identity — the kernel uid (kuid) — and the map governs two directions: syscalls that return IDs (getuid, the credential fields of stat) translate kuids down into the caller's namespace, while permission checks translate the process's IDs up into the initial namespace before judging.
Go to User namespaces: root inside, unprivileged outside →microVM
Kata Containers, formed in 2017 from Intel's Clear Containers and Hyper's runV projects, and Firecracker, the virtual machine monitor AWS open-sourced in December 2018, take the other road from hardware-assisted virtualisation: keep a real kernel, but give each container its own inside a VM small enough to run like a process — a microVM.
Go to Sandboxed container runtimes: gVisor, Kata and microVMs →buffer overflow
A buffer overflow is a write that continues past the end of the memory allocated for a buffer, overwriting whatever the program stores next to it.
Go to Memory-corruption exploits: overwriting control data →control data
When the overwritten bytes are control data — a return address, a function pointer, an allocator's link — the overflow can redirect the program's execution.
Go to Memory-corruption exploits: overwriting control data →shellcode
The standard trick places the payload in the buffer itself: fill buf with machine code — the shellcode, historically a hand-assembled execve("/bin/sh") — and set the planted address back to buf.
Go to Memory-corruption exploits: overwriting control data →NOP sled
Second, a NOP sled: prepend a run of no-op instructions (0x90 on x86) before the shellcode, so the planted address only needs to land anywhere in the sled — execution slides down it into the code.
Go to Memory-corruption exploits: overwriting control data →Use-after-free
Use-after-free is the same aliasing with the order reversed: a program keeps using a pointer to memory it has already freed.
Go to Memory-corruption exploits: overwriting control data →W^X
The policy layered on the bit is W^X ("write xor execute"): no page is ever simultaneously writable and executable.
Go to Non-executable memory, and return-oriented programming →return-to-libc
Solar Designer, a security researcher, published the answer on the Bugtraq mailing list in August 1997: return-to-libc writes the address of system() over the return slot and lays the argument — a pointer to the string /bin/sh, planted or found — where a callee would look.
Go to Non-executable memory, and return-oriented programming →Return-oriented programming
Hovav Shacham's paper “The Geometry of Innocent Flesh on the Bone” (ACM CCS, 2007) completes it: one borrowed call generalizes to a program, a technique he named return-oriented programming (ROP).
Go to Non-executable memory, and return-oriented programming →Gadget
Chop library code at every ret byte and you get gadgets: instruction sequences ending in ret that each perform one small step.
Go to Non-executable memory, and return-oriented programming →Address-space layout randomisation (ASLR)
Address-space layout randomisation (ASLR) places each region of a process's address space — stack, heap, memory-mapped libraries, and the program's own code when it is built position-independent — at a base address drawn at random when the program is loaded, so that an attacker who can overwrite a pointer does not know what value to write.
Go to Address-space layout randomisation and its entropy →Kernel address-space layout randomisation (KASLR)
The same idea works at kernel scope: kernel address-space layout randomisation (KASLR, enabled by CONFIG_RANDOMIZE_BASE) shifts the kernel image itself at boot, and the nokaslr boot option disables it.
Go to Address-space layout randomisation and its entropy →information leak
Worse, ASLR protects locations, not code — an information leak — any bug that lets the attacker read memory they should not — needs no write primitive at all: one disclosed pointer re-bases every gadget in the library.
Go to Address-space layout randomisation and its entropy →control-flow graph
The starting point is the program's control-flow graph (CFG): a graph whose nodes are blocks of straight-line code and whose edges are the transfers of control the program's own structure permits — this call may reach that function, this return may land after that call site.
Go to Control-flow integrity: shadow stacks and indirect-branch checks →Control-flow integrity (CFI)
Control-flow integrity (CFI) is the property that every computed transfer of control at run time follows an edge of the program's control-flow graph, fixed before the program runs; an implementation enforces it by checking each computed transfer's destination just before the transfer happens.
Go to Control-flow integrity: shadow stacks and indirect-branch checks →Equivalence class (CFI)
A hijacker who redirects the transfer to anything outside the source's equivalence class — the set of destinations sharing its expected label — trips the check, no matter how much of the address space they can name.
Go to Control-flow integrity: shadow stacks and indirect-branch checks →shadow stack
A shadow stack is the structural alternative: CALL pushes the return address twice — once on the normal stack and once on an isolated stack the application cannot write — and RET compares the two copies, faulting if they differ.
Go to Control-flow integrity: shadow stacks and indirect-branch checks →Memory-safe language
A memory-safe language is one whose rules — checked by the compiler, the runtime, or both — guarantee that a program only reads and writes memory it currently owns and within the bounds of the object it names: no write past a buffer, no access through a freed or dangling pointer.
Go to Control-flow integrity: shadow stacks and indirect-branch checks →branch predictor
A branch predictor answers two questions before the branch executes — which way will it go, and where does it go — and the front end fetches and executes down the predicted path.
Go to Pipelines, branch prediction and speculative execution →branch target buffer
A branch target buffer (BTB) caches the destination of recently taken jumps, so the fetch stage can redirect itself before the branch instruction is even decoded.
Go to Pipelines, branch prediction and speculative execution →return stack buffer
For conditional branches the live question is direction, predicted from recent outcomes in history tables; indirect branches get their own predictors, and ret instructions a dedicated return stack buffer (RSB) mirroring recent call addresses.
Go to Pipelines, branch prediction and speculative execution →out-of-order execution
The first is out-of-order execution: instructions are decoded into micro-operations — the simpler internal steps an instruction becomes — issued as soon as their operands are ready rather than strictly in program order, and collected in a reorder buffer (ROB) that holds finished-but-uncommitted work.
Go to Pipelines, branch prediction and speculative execution →reorder buffer
The first is out-of-order execution: instructions are decoded into micro-operations — the simpler internal steps an instruction becomes — issued as soon as their operands are ready rather than strictly in program order, and collected in a reorder buffer (ROB) that holds finished-but-uncommitted work.
Go to Pipelines, branch prediction and speculative execution →Retirement
The second is retirement in order: results leave the ROB and become architectural state — real registers, program counter, visible memory — only at the head of the buffer, in program order.
Go to Pipelines, branch prediction and speculative execution →Transient instruction
Instructions executed erroneously under a misprediction are called transient: gone from the program's point of view.
Go to Pipelines, branch prediction and speculative execution →side channel
A side channel is any physical effect of a computation — time, power, cache state — from which an observer can infer data the computation never meant to reveal.
Go to Cache-timing side channels: Flush+Reload and Prime+Probe →Prime+Probe
Prime+Probe is contention used as a sensor:
Go to Cache-timing side channels: Flush+Reload and Prime+Probe →Flush+Reload
Flush+Reload then loops through three phases per slot:
Go to Cache-timing side channels: Flush+Reload and Prime+Probe →Spectre gadget
Where the victim speculates to is a Spectre gadget: a fragment in victim-executable memory — and with megabytes of shared libraries mapped into a typical process, gadget space is ample — whose speculative run copies a secret into a covert channel.
Go to Spectre: mistraining the predictor →retpoline
A retpoline replaces an indirect jump with a return that speculation cannot mistrain.
Go to Spectre: mistraining the predictor →dynamic binary translation
Between interpretation and trap-and-emulate lies the mechanism VMware built its product line on: dynamic binary translation.
Go to Binary translation and paravirtualisation →translation cache
Each TU is compiled into a fragment of host code and stored in a translation cache inside the monitor's address range.
Go to Binary translation and paravirtualisation →paravirtualisation
This is paravirtualisation: the guest is ported to a virtual machine interface that resembles, but is not, the real hardware, and a hypercall is the guest's explicit call into the hypervisor, as a system call is a process's call into its kernel.
Go to Binary translation and paravirtualisation →hypercall
Xen, built by Paul Barham and colleagues at the University of Cambridge and published in 2003, gives the guest a modified interface — not the x86 it was written for, but a close cousin where every sensitive operation is replaced by an explicit hypercall into Xen, and where events back from the monitor arrive asynchronously rather than as interrupts.
Go to Binary translation and paravirtualisation →guest virtual address
The guest OS owns its per-process page tables, translating a guest virtual address (GVA) to a guest physical address (GPA) — what the guest believes is a machine frame.
Go to Nested paging: two-dimensional address translation →guest physical address
The guest OS owns its per-process page tables, translating a guest virtual address (GVA) to a guest physical address (GPA) — what the guest believes is a machine frame.
Go to Nested paging: two-dimensional address translation →Machine address
The hypervisor owns a second, per-VM table translating each GPA to a real machine address (also called a system physical address or machine frame number).
Go to Nested paging: two-dimensional address translation →shadow page table
The software answer is shadow paging: the hypervisor computes the composition itself and writes a private shadow page table that maps each guest virtual address straight to a machine address.
Go to Nested paging: two-dimensional address translation →EPT violation
And faults split by which table actually failed: a missing guest mapping injects an ordinary page fault into the guest, while an unmapped GPA or an EPT-permission violation is an EPT violation, a VM exit naming the faulting guest-physical address — the hypervisor's counterpart of a page fault.
Go to Nested paging: two-dimensional address translation →page-walk cache
A page-walk cache stores intermediate walk results; Ravi Bhargava and colleagues showed in 2008 that caching only the guest dimension's reads helps little — most references still reach the memory hierarchy — while a cache that also holds the nested dimension's entries is the one that pays, because the grid is deep but highly redundant: sibling entries share every upper-level read, which is exactly what a walk cach…
Go to Nested paging: two-dimensional address translation →Downtime
Downtime is how long no copy of the guest is running, which clients see as an outage.
Go to Live migration: moving a running virtual machine with pre-copy and post-copy →Total migration time
Total migration time is how long from the start of migration until the source can be switched off.
Go to Live migration: moving a running virtual machine with pre-copy and post-copy →pre-copy
Clark and colleagues' pre-copy migration, built on the Xen hypervisor, does it in rounds:
Go to Live migration: moving a running virtual machine with pre-copy and post-copy →Dirty logging
Dirty logging works like this:
Go to Live migration: moving a running virtual machine with pre-copy and post-copy →writable working set
They called it the writable working set, by analogy with the working set of pages a program reads and writes.
Go to Live migration: moving a running virtual machine with pre-copy and post-copy →Live migration
Live migration moves a running virtual machine — its memory, CPU state and device state — from one physical host to another while the guest keeps executing for all but a short final pause, and without the source host needing to remain available afterwards.
Go to Live migration: moving a running virtual machine with pre-copy and post-copy →auto-converge
QEMU, the user-space half of the KVM stack, has a migration option called auto-converge: when migration stops making progress it throttles the guest's virtual CPUs, starting at 20% of their time by default and adding 10% each time progress stalls again.
Go to Live migration: moving a running virtual machine with pre-copy and post-copy →Post-copy
Post-copy migration, evaluated for virtual machines by Michael Hines and Kartik Gopalan in 2009, sends the CPU and device state first and resumes the guest on the destination immediately.
Go to Live migration: moving a running virtual machine with pre-copy and post-copy →trusted execution environment
A trusted execution environment (TEE) inverts the direction of isolation: it runs code where the platform owner can schedule it but cannot read or silently rewrite it.
Go to Trusted execution environments: enclaves and confidential VMs →enclave
Intel SGX (Software Guard Extensions) takes the smallest unit: an enclave is a protected region inside an ordinary process.
Go to Trusted execution environments: enclaves and confidential VMs →enclave page cache
Its pages live in the enclave page cache (EPC), a reserved range of DRAM that the processor encrypts as data leaves the chip, so someone snooping the memory bus sees only ciphertext.
Go to Trusted execution environments: enclaves and confidential VMs →confidential VM
A confidential VM moves the boundary out to the entire guest.
Go to Trusted execution environments: enclaves and confidential VMs →measurement
Every TEE therefore builds a launch measurement, a hash accumulated over everything loaded before the code starts: SGX folds each page's contents, offset and flags into MRENCLAVE as EADD runs (plus the signer identity MRSIGNER at EINIT); SNP's launch digest folds contents, GPA layout and page type.
Go to Trusted execution environments: enclaves and confidential VMs →Remote attestation
Remote attestation is a protocol in which hardware signs a report containing a measurement of the software it launched, bound to a verifier's fresh nonce, with a key that chains to the hardware vendor; a verifier that checks the signature, the measurement and the nonce learns which code is running inside the protected environment on genuine hardware.
Go to Trusted execution environments: enclaves and confidential VMs →exokernel
Dawson Engler, Frans Kaashoek and James O'Toole's exokernel, built at MIT and published in 1995, ends the kernel's job at secure multiplexing: the small kernel exports hardware resources through a low-level interface, and an untrusted library operating system (libOS) linked into each application implements the conventional abstractions — VM, IPC, filesystems — itself.
Go to Unikernels and library operating systems →library operating system
Dawson Engler, Frans Kaashoek and James O'Toole's exokernel, built at MIT and published in 1995, ends the kernel's job at secure multiplexing: the small kernel exports hardware resources through a low-level interface, and an untrusted library operating system (libOS) linked into each application implements the conventional abstractions — VM, IPC, filesystems — itself.
Go to Unikernels and library operating systems →Secure binding
Secure bindings perform the protection check once, when a resource is bound to an application, and then let the application use it without crossing the kernel — a validated TLB entry or a downloaded packet filter is the canonical instance.
Go to Unikernels and library operating systems →unikernel
A unikernel is a specialised, sealed, single-purpose virtual machine image, built by compiling an application together with only the library-OS components it uses, that runs directly on a hypervisor with no general-purpose guest kernel.
Go to Unikernels and library operating systems →Callback (AFS)
In AFS, a callback is the server's promise to a client that caches a file that it will notify that client before allowing any other workstation's modification of the file to take effect.
Go to AFS: whole-file caching with callbacks →Last closer wins
Concurrent writers on different machines do not interleave: the last Store overwrites the file (last closer wins), losing the other session's edits entirely rather than mixing them.
Go to AFS: whole-file caching with callbacks →Mechanism versus workload
Decisions supported by the authored lessons.
Which scheduler, if any?
Reorder for fairness only when the device is slow enough that ordering hides it.
More detail
mq-deadline bounds write waits cheaply; BFQ buys per-workload bandwidth at budget cost; Kyber throttles to a latency target. On NVMe-class devices none can win outright.
Mirror or parity?
Mirroring buys simple fast redundancy at half capacity; parity buys capacity at a small-write tax.
More detail
RAID 1/10 need no parity computation and rebuild by copying; RAID 5/6 add the four-I/O small write and the write hole. Capacity per dollar is what parity is for.
Map each I/O or keep static mappings?
IOMMU map/unmap calls amortise over buffer lifetime — the walk is the steady-state price.
More detail
Long-lived bounce-buffer-free mappings make isolation nearly free; short-lived mappings pay T_map + T_unmap per I/O. That asymmetry, not the walk, decides where the IOMMU earns its place.
Lock-free or lock-based?
Lock-free removes blocking, not coordination — the CAS still serialises the structure.
More detail
Choose lock-free for progress guarantees under preemption/failure, not raw throughput. It brings reclamation and ABA obligations a mutex does not.
Hazard pointers or epochs?
Per-object protection with bounded garbage versus cheap reads with fragile reclamation.
More detail
Hazard pointers cost a publish+scan per use but bound garbage tightly; epochs need threads to reach quiescent points, so a stalled thread pins everything.
Seqlock, RCU or rwlock?
Match the reader’s cost and the data’s shape.
More detail
Seqlock readers retry on concurrent writes and cannot follow pointers; RCU readers never retry but updates defer reclamation; rwlocks charge readers a shared write. Pick by read:write ratio and whether contents are self-contained.
Handle in the handler or defer to the loop?
Keep handlers on the async-signal-safe list and do real work in the main loop.
More detail
Self-pipe and signalfd turn a signal into pollable input. Only the short safe list (no stdio, no malloc) is permitted inside the handler itself.
Civil time, elapsed time or suspend-aware age?
Choose clock progress to match the question.
More detail
Use REALTIME for civil timestamps, MONOTONIC for awake elapsed time and BOOTTIME when suspend must count. Avoid cross-domain subtraction; retain the library API so supported vDSO paths remain available.
Ordered timer queue or wheel?
Trade arbitrary deadline precision against cancellation-heavy coarse timeouts.
More detail
An indexed heap is often the simpler baseline. A wheel needs explicit granularity, horizon and collision costs; neither data structure alone solves callback lifetime or concurrent cancellation.
Base pages or larger mappings?
Measure translation savings against setup, sparse waste and lifetime.
More detail
Dense long-lived working sets can benefit; short-lived sparse regions may not. Multi-size physical allocation need not imply a larger hardware leaf.
Local placement, binding or interleave?
Choose locality, capacity constraints or bandwidth distribution deliberately.
More detail
Align first allocation with stable worker ownership; binding may fail with free memory elsewhere. Interleave distributes pressure but need not minimize each worker’s remote fraction.
Flat, hierarchical or inverted mappings?
Compare actual density, occupied prefixes, resident frames and hardware support.
More detail
A small dense array can be simplest. Hierarchy omits empty prefixes; inversion bounds base records by frames but needs keyed lookup and alias metadata.
Immediate or batched shootdown?
Amortize notification only within the revocation and reclamation contract.
More detail
Batching reduces fixed per-target costs but delays safe frame reuse. Never count sent interrupts as completed invalidations.
Shared kernel view or split entry tables?
Establish required isolation before minimizing transition overhead.
More detail
Shared supervisor mappings simplify entry. Split views preserve a minimal transition path while excluding other kernel mappings during user execution; required mitigations are not optional performance knobs.
Preserve affinity or migrate?
Compare saved delay with movement and cold-state costs.
More detail
Respect masks and topology. A smaller queue is not necessarily less work; a hard pin can strand idle capacity.
Low power or faster completion?
Minimise energy among feasible placements, not peak watts.
More detail
Use one horizon and include idle costs and shared performance states. Measured rates and costs decide which CPU wins.
Immediate handling or deferred work?
Keep urgent work bounded; defer when latency or execution context requires it.
More detail
Deferral adds queueing, lifetime and synchronisation obligations. Verify both the immediate latency budget and consumer capacity.