Your technical library
Algorithms
15 modules · 67 topics
About this book
Algorithms
This course teaches how to turn a real problem into a procedure you can trust: say precisely what a correct answer is, derive the algorithm one decision at a time, prove why it works, count what it costs in a named model, and decide what to change when the requirements change. It is written for any engineer who calls sort, search and graph libraries and wants to know what they do and when they fail. It builds on Data Structures: that course owns the representations — arrays, hash tables, heaps, trees — and this one owns the procedures that use them.
The core path runs through seven modules. Problem modeling and reasoning turns a product request into a computational problem, defines what a correct answer is and counts the work a recursive procedure does. Scanning and searching covers pointer scans, sliding windows and binary search over any yes-or-no question that flips once. Sorting and selection, graph exploration and structure and shortest paths follow, then two ways of building a best answer: greedy algorithms, justified by exchange arguments, and dynamic programming, derived from the states a problem really has.
The advanced track widens the toolkit: exact search under constraints (backtracking, branch and bound, SAT solvers), flow and matching, text-processing algorithms, randomized and streaming algorithms, hardness, approximation, and online decisions — what to do when no fast exact algorithm exists — geometry and numerical computation, and parallel and data-movement-aware algorithms, where moving data costs more than computing on it. Worked decision studies closes both tracks by choosing among procedures as requirements change. Along the way each topic says what it inherits, what it leaves open, and the workload where a simpler procedure wins.
67 topics across 15 modules, in dependency order.
Code conventions: where a lesson has code, it states the algorithm in pseudocode first, then in Python using the names from the worked trace. Pseudocode comments use #, costs are counted in a named unit such as comparisons or relaxations, and every bound states whether it is worst-case, expected, or amortised.
67 of 67 topics are authored; the rest are labelled planned and excluded from progress. Each topic opens with the problem earlier topics left open and closes with what it still leaves open. Glossary and decision-guide entries cover the authored material and grow with the course.
Read at your own pace. Each topic is one continuous page; mark it done as you finish.
Start with 01.1 · Problem contracts: turn a request into a problemReview what you have finished↷ 0 skipped topics- 01.1Problem contracts: turn a request into a problem
- 01.2Correctness and termination: why the answer is right and the loop ends
- 01.3Recursive decomposition: solve the smaller instance first
- 01.4Recurrences: count the work of a recursive procedure
- 01.5Euclid and repeated squaring: integer arithmetic algorithms
- 03.1Insertion sort and Timsort: cheap when the input is nearly ordered
- 03.2Merge sort: divide, then merge
- 03.3Quicksort: in place, if the pivot cooperates
- 03.4Heapsort: a structure gives you a sort for free
- 03.5Quickselect and top-k: the kth item without a full sort
- 03.6Counting and radix sort: escape the comparison lower bound
- 04.1Breadth-first search: explore by distance
- 04.2Depth-first search: explore by going deeper
- 04.3Topological ordering: every dependency comes first
- 04.4Strongly connected components: collapse the cycles
- 04.5Bridges and articulation points: which single failure disconnects the graph
- 04.6LCA, binary lifting and heavy-light decomposition: path queries on trees
- 05.1Edge relaxation and DAG shortest paths: improve estimates one edge at a time
- 05.2Dijkstra's algorithm: settle the closest vertex first
- 05.3Bellman-Ford: negative weights and the cycle that has no answer
- 05.4Floyd-Warshall and Johnson's algorithm: every route at once
- 05.5A* search: aim the search with a heuristic
- 07.1Dynamic programming: find the state that is sufficient
- 07.2Knapsack: indivisible choices under a budget
- 07.3Edit distance and sequence alignment: how different are two sequences
- 07.4Interval dynamic programming: optimise over a contiguous range
- 07.5Dynamic programming on trees and DAGs: aggregate up a structure
- 07.6Recurrence optimisation: make the transition cheaper
- 09.1Ford-Fulkerson and residual networks: undo a flow you already sent
- 09.2Max-flow min-cut, Edmonds-Karp and Dinic: the cut certifies the flow
- 09.3Minimum-cost flow: cheapest way to send the required amount
- 09.4Bipartite matching and assignment (Hopcroft-Karp, Hungarian): pair workers with jobs
- 09.5Edmonds' blossom algorithm: matching when the graph is not bipartite
- 13.1Robust geometric predicates: which side of the line
- 13.2Convex hulls and point-in-polygon: the smallest enclosing polygon
- 13.3Sweep line and closest pair: process events in order
- 13.4Fast Fourier transform: multiply polynomials by changing representation
- 13.5Gaussian elimination and conditioning: solve for many unknowns at once
- 13.6Bisection and Newton's method: find a root and know when to stop
01.2Correctness and termination: why the answer is right and the loop ends
01.3Recursive decomposition: solve the smaller instance first
01.4Recurrences: count the work of a recursive procedure
01.5Euclid and repeated squaring: integer arithmetic algorithms
02.1Two-pointer scans: two indices, one pass
02.2Sliding windows: maintain a summary of the last k items
02.3Binary search: find the boundary, not the value
02.4Binary search on the answer: monotone feasibility
03.1Insertion sort and Timsort: cheap when the input is nearly ordered
03.2Merge sort: divide, then merge
03.3Quicksort: in place, if the pivot cooperates
03.4Heapsort: a structure gives you a sort for free
03.5Quickselect and top-k: the kth item without a full sort
03.6Counting and radix sort: escape the comparison lower bound
04.1Breadth-first search: explore by distance
04.2Depth-first search: explore by going deeper
04.3Topological ordering: every dependency comes first
04.4Strongly connected components: collapse the cycles
04.5Bridges and articulation points: which single failure disconnects the graph
04.6LCA, binary lifting and heavy-light decomposition: path queries on trees
05.1Edge relaxation and DAG shortest paths: improve estimates one edge at a time
05.2Dijkstra's algorithm: settle the closest vertex first
05.3Bellman-Ford: negative weights and the cycle that has no answer
05.4Floyd-Warshall and Johnson's algorithm: every route at once
05.5A* search: aim the search with a heuristic
06.1Exchange arguments: when a greedy choice is safe
06.2Interval scheduling: choose the most non-overlapping reservations
06.3Minimum spanning trees (Kruskal and Prim): connect everything as cheaply as possible
06.4Huffman coding: short codes for frequent symbols
07.1Dynamic programming: find the state that is sufficient
07.2Knapsack: indivisible choices under a budget
07.3Edit distance and sequence alignment: how different are two sequences
07.4Interval dynamic programming: optimise over a contiguous range
07.5Dynamic programming on trees and DAGs: aggregate up a structure
07.6Recurrence optimisation: make the transition cheaper
08.1Backtracking: extend a partial answer or abandon it
08.2Branch and bound: prune with a bound, not just a rule
08.3SAT and constraint modelling: say the problem in clauses
09.1Ford-Fulkerson and residual networks: undo a flow you already sent
09.2Max-flow min-cut, Edmonds-Karp and Dinic: the cut certifies the flow
09.3Minimum-cost flow: cheapest way to send the required amount
09.4Bipartite matching and assignment (Hopcroft-Karp, Hungarian): pair workers with jobs
09.5Edmonds' blossom algorithm: matching when the graph is not bipartite
10.1Knuth-Morris-Pratt: do not restart after a mismatch
10.2Rabin-Karp rolling hash: compare fingerprints, then verify
10.3Aho-Corasick: many patterns in one pass
10.4Regular expressions as automata: match a language, not a string
10.5LZ77 dictionary compression: replace repeats with references
11.1Reservoir sampling: a fair sample from a stream of unknown length
11.2Heavy hitters (Misra-Gries): find the keys that dominate
11.3MinHash and locality-sensitive hashing: find near-duplicates without every pair
11.4Probability amplification and evaluation: repeat trials, measure the right thing
12.1Reductions and NP-hardness: show a problem is at least as hard as another
12.2Approximation algorithms: guarantee a ratio instead of the optimum
12.3Online algorithms and competitive analysis: decide before you see the future
13.1Robust geometric predicates: which side of the line
13.2Convex hulls and point-in-polygon: the smallest enclosing polygon
13.3Sweep line and closest pair: process events in order
13.4Fast Fourier transform: multiply polynomials by changing representation
13.5Gaussian elimination and conditioning: solve for many unknowns at once
13.6Bisection and Newton's method: find a root and know when to stop
14.1Parallel reduce and scan: combine in a tree, not a line
14.2The I/O model and cache-oblivious algorithms: count block transfers
15.1Case study: a dependency-aware scheduler
15.2Case study: a search and similarity pipeline
15.3Case study: a bounded-memory telemetry pipeline
Skipped topics
Moved past without finishing. A skipped topic still closes its module, and Continue goes on without it — open one to read it or mark it understood.
Glossary
The vocabulary of the authored lessons, grouped by subject and linked to the topic that derives each idea.
Problem contract
The fixed statement of input shape, output requirement, and workload parameters that makes a request computable.
More detail
An algorithm exists only relative to a contract: what counts as the input, what a valid answer is, which answers count as optimal, and which parameters a cost is stated in. Two contracts differing only in a tie rule can legitimately select different procedures.
Loop invariant
A predicate that holds on entry to a loop and is preserved by every iteration.
More detail
Initialisation, preservation, and exit are the three proof obligations: the invariant is established before the loop, kept by the body, and strong enough at exit to imply the postcondition. Partial correctness still needs a separate termination argument.
Termination measure
A non-negative integer quantity that strictly decreases on every iteration, forcing the loop to end.
More detail
A loop that preserves its invariant can still run forever. Naming a measure bounded below — typically a count of remaining candidates or unprocessed elements — turns “it finishes” into a theorem rather than a hope.
Divide and conquer
Split a problem into independent smaller instances, solve each, and combine the answers.
More detail
The decomposition must name the split point, the base case, and the combine step. The method pays when the subproblems are genuinely independent; overlapping subproblems are the signal for dynamic programming instead.
Recursion tree
The tree of recursive calls whose per-level work sums give a recurrence its closed form.
More detail
Each node carries the non-recursive work of one call. Summing level by level exposes whether the total is root-heavy, balanced, or leaf-heavy, which is the derivation the master theorem abbreviates.
Master theorem
A summary table for divide-and-conquer recurrences once the level sums have been computed.
More detail
Comparing the leaf count n^(log_b a) against the per-call work f(n) selects the case: leaves dominate, all levels equal, or root dominates. It is a bookkeeping device for the recursion-tree argument, not a substitute for it.
Modular inverse
The multiplier that undoes a modulo multiplication, produced by Euclid’s extended algorithm.
More detail
A number a has an inverse modulo m exactly when gcd(a, m) = 1. The extended Euclidean algorithm returns coefficients with as + mt = gcd, so s mod m is the inverse; hashing and RSA-style arithmetic both consume it.
Exponentiation by squaring
Computing a^e in O(log e) multiplications by halving the exponent.
More detail
Square on even exponents, square-and-multiply on odd ones: each bit of e costs at most two multiplications. The method is the same in any monoid — the modulus, the matrix, or the semiring is a parameter.
Two-pointer scan
Two indices advance through a sorted structure, each moving only forward, deciding a pair property in one pass.
More detail
The argument is exhaustion: each step discards a candidate that can never pair again, so O(n) moves settle it. The precondition is monotonicity of the predicate in the pointer positions — without it the scan drops real answers.
Sliding window
A contiguous range with a maintained aggregate, grown and shrunk as each end advances once.
More detail
Both ends move monotonically, so the whole scan is O(n) while recomputing the aggregate from scratch would be O(n²). The window’s aggregate must support its two updates — extend and retract — which is where the monotone-frontier structures pay.
Half-open interval invariant
The discipline [lo, hi): the answer always lies in the current range, which shrinks on every probe.
More detail
Half-open bounds make empty ranges exact (lo == hi) and keep midpoint selection termination-safe — the overflow-safe midpoint and the never-revisit convention are what make the trace terminate. Boundary answers (first true, last false, insertion point) are all read off the same invariant.
Feasibility monotone predicate
A yes/no test on a parameter that stays answered once it flips, so the answer is a boundary found by search.
More detail
“Can we finish within budget B?” is monotone when extra budget never breaks a feasible plan. Searching the predicate over the parameter space turns an optimisation question into O(log range) feasibility checks — the parameter must be ordered and the test honest about all constraints.
Inversion
A pair of elements out of relative order; the count of them is a sorting instance’s distance from sorted.
More detail
Insertion sort does exactly one swap per inversion, so its running time is Θ(n + inv) — linear on nearly-sorted input. That adaptivity is why it survives inside hybrid sorters as the small-run finisher.
Comparison lower bound
Any comparison sort needs Ω(n log n) comparisons in the worst case because a tree of height h distinguishes at most 2^h orders.
More detail
There are n! permutations to separate, so h ≥ lg(n!) = Ω(n log n). The bound is about the model — comparisons only — which is exactly what radix and counting sorts step around.
Partition scheme
Rearranging an array around a pivot so everything left of it precedes it and everything right follows it.
More detail
The scheme’s loop invariant decides both correctness and worst cases: Lomuto is simpler but degenerates on duplicates, Hoare handles them. Pivot choice — median-of-three, random, or median-of-medians — is the only defence against adversarial input.
Heap
A complete binary tree in an array whose parent-before-child order supports O(log n) sift operations.
More detail
Heapsort builds the heap in place — O(n) by bottom-up sift-down — then extracts each maximum. The structure’s maintenance internals are the Data Structures course’s binary-heap lesson; here it is a sorting machine.
Median-of-medians
Choosing the pivot as the median of group medians so each side holds a guaranteed fraction of the input.
More detail
Groups of five give a pivot discarding at least 3n/10 elements, making the selection recurrence T(n) ≤ T(n/5) + T(7n/10) + O(n) = O(n). Quickselect with a random pivot is simpler and faster in practice; MoM is the deterministic guarantee.
Stable sort
A sort that preserves the input order of equal keys — the precondition radix sort relies on per digit.
More detail
Counting sort emits elements in arrival order within each key class, so sorting digit by digit least-significant-first composes correctly. An unstable digit pass silently scrambles the higher digits’ work.
Discovery order
BFS visits vertices in non-decreasing distance from the source — the property every BFS application spends.
More detail
The queue holds at most two consecutive layers, and a vertex is dequeued before any farther vertex. That is what makes the tree it produces a shortest-path tree in unweighted graphs — the claim is about unweighted edges only.
DFS timestamps
Discovery and finish times whose nesting encodes the ancestor relation: u is an ancestor of v iff [d[u], f[u]] ⊇ [d[v], f[v]].
More detail
Edge classification (tree, back, forward, cross) is read off the intervals. White-path theorem and the parenthesis structure are the two lemmas nearly every DFS application consumes.
Topological order
An ordering of a DAG in which every edge points forward — a schedule respecting all dependencies.
More detail
It exists iff the graph is acyclic; Kahn’s algorithm produces it by repeatedly removing zero-indegree vertices, and a leftover vertex set is the certificate of a cycle. The order is not unique — the choice among ready vertices is free.
Strongly connected component
A maximal vertex set where every pair is mutually reachable; the condensation of the graph into a DAG of components.
More detail
Two DFS passes on G and G^T produce the components — the finish-order of the first pass is what makes the second pass peel SCCs in topological order. Every cyclic dependency question reduces to “is it one SCC?”
Low-link value
The earliest discovery time reachable from a subtree via tree edges plus at most one back edge — the measure that detects bridges and cut vertices.
More detail
A tree edge (u, v) is a bridge exactly when low[v] > disc[u]: nothing in v’s subtree escapes back above u. The same low[] computation flags articulation points at the root and internal cases separately.
Euler tour indexing
Assigning each tree node its entry and exit times so ancestry and subtree membership become interval tests.
More detail
u is an ancestor of v iff tin[u] ≤ tin[v] and tout[v] ≤ tout[u] — an O(1) answer after one DFS. Lifting tables and sparse-table RMQ build on the same indexing for lowest common ancestor.
Edge relaxation
The operation d[v] = min(d[v], d[u] + w(u, v)) from which every shortest-path algorithm is assembled.
More detail
Relaxation is safe: it never makes a true distance smaller. The whole theory is ordering — on a DAG, relaxing edges in topological order settles each vertex once; that single pass is the fastest shortest-path algorithm in the course.
Dijkstra’s settled set
Extracting the minimum tentative distance makes it final — the vertex can never be improved later.
More detail
The argument needs non-negative weights: the extracted vertex’s distance is a shortest path because every alternative route exits the settled set through a vertex no closer. Negative edges break the proof, not the code.
Negative cycle detection
After |V|−1 relaxation rounds every distance is final, so any still-improvable edge certifies a reachable negative cycle.
More detail
A shortest path uses at most |V|−1 edges; round k settles all k-edge shortest paths. The (|V|)th pass is pure detection — the algorithm trades Dijkstra’s speed for negative-weight correctness and a cycle witness.
Distance-product closure
Shortest paths between all pairs computed as repeated squaring or DP over intermediate vertices — Floyd–Warshall and Johnson both reduce to it.
More detail
Floyd–Warshall permits intermediate vertices one at a time, Θ(n³) in space-conscious form. Johnson reweights with one Bellman–Ford pass so sparse graphs run n Dijkstras — the choice between them is density.
Admissible heuristic
A goal-distance estimate that never overestimates — the precondition making A*’s goal test on extraction correct.
More detail
With h ≤ true remaining cost, first extraction of the goal is optimal; consistency (h(u) ≤ w(u,v) + h(v)) additionally makes reopening impossible. An inadmissible heuristic is fast and wrong — the classic trade.
Exchange argument
Showing any optimal solution can be transformed into the greedy choice without loss — the core greedy proof.
More detail
Take an optimum, swap in the greedy element, and show the result is no worse: then an optimum containing the greedy choice exists, and induction finishes. The argument fails when the swap can degrade the solution — that is where DP takes over.
Interval scheduling
Selecting maximum non-overlapping intervals by greedily taking the one that ends soonest.
More detail
The exchange is direct: the first interval of any optimal schedule finishes no earlier than the greedy pick, so swapping never shrinks the schedule. Weighted and resource-constrained variants break the argument — the lesson’s counterexamples mark the boundary.
Cut property
The lightest edge crossing any cut of the graph belongs to some minimum spanning tree — the single theorem behind both Kruskal and Prim.
More detail
Kruskal applies it to the cut between components; Prim to the cut around the grown tree. Distinct weights make the MST unique; ties need a tie-break argument, and the property is exactly what justifies taking the cheapest crossing edge.
Prefix-free code
A binary code where no codeword prefixes another, so the coded stream self-delimits — Huffman builds the optimal one.
More detail
Huffman’s exchange: the two rarest symbols can sit as deepest siblings in an optimal tree, so merging them is safe and recursion finishes. Average length is within a bit of the entropy floor per symbol.
DP state
The parameterisation of a subproblem whose answer can be computed from other states — the design act of dynamic programming.
More detail
A state is right when the optimal-substructure and overlapping-subproblem properties both hold. Writing the recurrence is easy once the state is honest; most failed DPs are states too small to be self-contained.
Knapsack update order
Iterating capacity backwards for 0-1 items so each item is used at most once — the detail that makes the table correct.
More detail
Forward iteration lets item i feed the row it is already writing, silently allowing reuse — the 0/1 and unbounded knapsacks differ only in loop direction. The lesson’s counterexample trace makes the failure visible.
Edit distance
Minimum cost to transform one string into another by insertions, deletions, and substitutions — the alignment DP.
More detail
The table cell (i, j) is the distance between prefixes; traceback recovers the actual alignment, not just its cost. Space can drop to two rows for the value, but the alignment itself needs the table or a divide-and-conquer reconstruction.
Interval DP
DP on contiguous intervals, filled by increasing length, for problems whose subproblems are substring-shaped.
More detail
Optimal-matrix-chain and burst-balloons style recurrences try a split point k inside [i, j]. Filling by length guarantees both halves are known before the interval is — evaluation order is the whole correctness argument.
Rooted-tree DP
Dynamic programming whose states are subtrees, computed children-first up the root.
More detail
Tree problems decompose along edges without cycles, so the recurrence’s subproblems are the children’s subtrees — one DFS fills the table. On DAGs the same trick needs memoisation since states are shared, not nested.
Memoisation vs tabulation
Top-down recursion with a cache versus bottom-up table filling — same recurrence, different constant factors and reachable states.
More detail
Memoisation visits only reachable states but pays call overhead; tabulation visits all states predictably and enables space squeezing. The lesson derives when the table can drop dimensions — when the recurrence reads only recent rows.
Pruning bound
A test that discards a partial solution when no completion can beat the incumbent — what turns exhaustive search into practical search.
More detail
The bound must be safe: a prune that discards a true optimum is wrongness, not speed. State domination and symmetry breaking are the two sound prunes the lesson derives.
Branch and bound
Systematic search that explores partial assignments in best-bound order, keeping a provable incumbent gap.
More detail
Unlike plain backtracking, the bounding function orders the frontier — the search can stop when the best unexplored bound cannot beat the incumbent. The guarantee is optimality; the cost is whatever the bound quality buys.
SAT encoding
Translating a problem’s constraints into clauses so a general solver decides satisfiability.
More detail
The encoding is the algorithm: clause growth, variable count, and propagation cost per decision decide whether the solver finishes. “Heuristic failed” versus “infeasible” is decided by the solver’s completeness, not by the encoding.
Residual network
The graph of remaining capacities plus reverse edges that let later augmentations retract earlier flow.
More detail
Backward edges are what makes greedy augmentation optimal: a path can push flow “back” around a corner. Without them, the first augmenting path could block the maximum — the residual view is the entire correctness story.
Max-flow min-cut theorem
The maximum flow value equals the minimum cut capacity — certificate and stopping condition in one.
More detail
When no augmenting path exists, the reachable set from s is a cut of value exactly |f|, proving optimality. Capacity assumptions matter: integral capacities give integral flows and the Ford–Fulkerson iteration bound.
Successive shortest paths
Augmenting along minimum-cost residual paths until the required flow is met, for flow that carries a per-unit price.
More detail
Each augmentation preserves optimality when paths are shortest in the residual graph; negative reduced costs are handled by potentials. The lesson trades the plain max-flow loop for a cost-aware one.
Alternating and augmenting path (matching)
A path alternating unmatched and matched edges whose symmetric difference with the matching grows it by one.
More detail
Berge’s theorem: a matching is maximum iff no augmenting path exists — search becomes the algorithm. In bipartite graphs the search is directed and fast; König’s theorem equates matching size and vertex cover there.
Blossom contraction
Shrinking an odd-length alternating cycle to a single vertex so augmenting-path search continues in general graphs.
More detail
Odd cycles break the bipartite alternating search — inside a blossom the path can turn either way. Edmonds’ algorithm contracts the cycle, recurses, and expands it back: the guarantee stays a maximum matching.
Failure link
For each pattern position, the longest proper prefix that is also a suffix — where matching resumes after a mismatch.
More detail
The table is built on the pattern alone in O(m); the text pointer never backs up, so matching is O(n + m). The failure function is itself the border structure the prefix-function definition names.
Rolling hash
A hash of a window updatable in O(1) per slide — the Rabin–Karp fingerprint.
More detail
Subtract the outgoing character’s contribution, shift the base, add the incoming one. Matches are verified exactly because fingerprint equality is probable, not certain — the verifier is what makes it a matcher rather than a guesser.
Trie automaton
KMP generalised to a dictionary: a trie of all patterns plus failure links between trie nodes.
More detail
One pass of the text reports every occurrence of every pattern in O(n + total matches). The failure edges are computed breadth-first so a node’s link is always shallower — the same border idea, lifted to a tree.
Regex automaton
An NFA built from the regex whose state set is simulated character by character — exact matching without backtracking.
More detail
Simulating all live states makes each character O(m) states — total O(nm) with no exponential blowup. The backtracking alternative in some engines can explode on adversarial patterns; the automaton’s guarantee is the point.
LZ77 factorisation
Splitting a text into phrases that are each a copy of an earlier phrase — the grammar DEFLATE encodes.
More detail
DEFLATE combines LZ77 matches with Huffman-coded literals and lengths; the compressed size is a bound on the parse quality. The lesson derives the factorisation cost and where the entropy bound sits.
Reservoir sample
A k-item sample of a stream of unknown length in which every position has equal inclusion probability.
More detail
The i-th arrival replaces a uniform resident with probability k/i, keeping the invariant at every prefix. The ledger is exact — the fairness contract holds at whatever n the stream ends at.
Misra–Gries counters
c counters tracking candidate heavy hitters, decremented all at once when a full table meets a new key.
More detail
Deterministic two-sided bound: f̂(x) ≤ f(x) ≤ f̂(x) + n/(c+1). Every key above n/(c+1) survives as a candidate — candidacy, not certification; a verification pass makes the list honest.
MinHash banding
Splitting a signature into b bands of r rows so documents sharing a whole band become candidates.
More detail
Candidacy probability is S(J) = 1 − (1 − J^r)^b, a tunable S-curve: b and r place the knee at the similarity threshold. Independent tables amplify it — the probabilistic candidate contract.
Amplification
Running a randomized procedure repeatedly to push its success probability toward 1, priced in independent runs.
More detail
k independent runs of a one-sided estimator turn failure p into p^k; the evaluation protocol reports the distribution of measured recall, not a single run. The discipline separates “the bound” from “the measurement”.
Polynomial-time reduction
Transforming every instance of problem A into an instance of B in polynomial time, so B is at least as hard as A.
More detail
A ≤_p B plus A NP-hard makes B NP-hard; the direction is the common error — you reduce the known-hard problem to the new one. One full worked reduction anchors the lesson.
Approximation ratio
A provable factor ρ between the algorithm’s answer and the optimum, for problems where exact is out of reach.
More detail
The ratio is proved against a bound on OPT — often a lower bound like a relaxation — never measured against OPT directly. Different problems have different achievable ρ; the guarantee is the product, not the heuristic.
Competitive ratio
The worst-case ratio between an online algorithm’s cost and the offline optimum that sees the whole request sequence.
More detail
The adversary chooses requests knowing the algorithm but not its coins — randomized algorithms get expected ratios against oblivious adversaries. Ski rental’s 2 − 1/B is the canonical derivation.
Exact geometric predicate
Answering orientation and in-circle questions by exact sign, never by comparing rounded coordinates.
More detail
Floating-point evaluation of the determinant can give the wrong sign on near-degenerate inputs, silently inverting the algorithm’s decisions. Expansion arithmetic and filtered evaluation are the fix hierarchy.
Convex hull
The smallest convex polygon containing the point set — output-sensitive algorithms derive it from a sorted order.
More detail
Andrew’s monotone chain builds the hull in O(n log n) via sorted input and two passes; containment queries then binary-search the boundary. The lesson separates the point-set hull from half-plane intersection.
Sweep line
An imaginary vertical line advancing left to right, maintaining only the active objects in a balanced structure.
More detail
Closest pair needs only points within δ of the line — a constant-size candidate strip — so the sweep is O(n log n). Event points and the status invariant are the whole algorithm.
FFT convolution
Multiplying polynomials or convolving sequences via the discrete Fourier transform in O(n log n).
More detail
Pointwise multiplication in the transform domain is the convolution in the time domain; the butterfly recursion with roots of unity gives the O(n log n) evaluation and its inverse. Integer convolution needs rounding-care or NTT variants.
Condition number
The factor κ(A) by which input perturbations to a linear system can amplify into the answer.
More detail
Gaussian elimination on an ill-conditioned system loses digits regardless of algorithm quality — κ(A)·ε_mach is the floor on accuracy. Pivoting fixes instability, not conditioning; the lesson separates the two failure modes.
Stopping criterion
Deciding a numerical iteration is close enough — residual, step size, or bracket width each answer a different question.
More detail
A small step can hide a stagnant iterate; a small residual can hide a flat function near the root. The lesson prices bracketing methods’ guarantees against Newton’s speed and states which criterion certifies which claim.
Work and span
The total operation count T₁ and the critical-path depth T∞ — the two quantities a parallel bound is made of.
More detail
Brent’s bound T_p ≤ T₁/p + T∞ prices any schedule on p processors; the scan’s two-sweep structure is the example where span and work are both derived. Associativity is the operator contract the whole argument rests on.
Block transfer model
Counting transfers of B-word blocks between fast memory of M words and slow storage — the cost model where scans cost n/B.
More detail
Binary search costs Θ(log(N/B)) probes but a B-tree costs Θ(log_B N) — the difference is what a fetched block buys. Cache-oblivious algorithms reach the same bounds without naming B or M.
Pipeline contract
The per-stage promise about an output set — exact filter, probabilistic candidate generator, exact reranker — that makes a composed system checkable.
More detail
Correctness is recovered only at exact stages; recall is decided at the probabilistic stage and never improves downstream. The case studies price each stage in its own counters rather than claiming a global sublinear bound.
Graham bound
Any list schedule on w workers finishes within (2 − 1/w)·C* — the bound that makes a heuristic schedule a contract.
More detail
The envelope C ≤ W/w + (1 − 1/w)·L combines total work and critical path; the case study spends it on a stated DAG, then re-derives it when a job duration is misestimated. Dependency cycles block the schedule entirely before the bound applies.
Error shape
The stated form of an approximate component’s wrongness — one-sided bound, deterministic deficit, or fairness invariant — priced from its byte share.
More detail
A bounded-memory pipeline is a budget split: ε, covered share, and sample size are each bought with bytes. Components sharing a stream keep private contracts; merging adds the sketch exactly, doubles the counter bound, and preserves the reservoir’s marginal.
Which procedure do I reach for?
A problem, the preconditions its procedure needs, the guarantee on offer, the cost model it lives in, the failure case, and the topic that derives the trade-off.
Is my procedure correct — and does it finish?
Prove it: an invariant plus a decreasing measure beats any number of tests.
More detail
Problem: a passed test suite does not exclude a wrong case nobody tried. Preconditions: the loop and its exit can be stated as predicates. Guarantee: initialisation + preservation + exit give partial correctness; a strictly decreasing non-negative measure gives termination. Cost: the analysis is free — it runs at proof time, not run time. Failure case: an invariant too weak to imply the postcondition, or none covering the exit path.
What does my recursive procedure cost?
Sum the recursion tree level by level; the master theorem is the summary, not the method.
More detail
Problem: a recursive call’s total work is not visible in its own body. Preconditions: the recurrence is known, including how the problem shrinks. Guarantee: the level sum gives the closed form; root-heavy, balanced, and leaf-heavy each collapse to a different dominant term. Cost: counted in the unit the lesson names (comparisons, moves, multiplications). Failure case: unequal splits or data-dependent depth, where a guessed balanced bound is simply wrong.
How do I turn a request into a solvable problem?
Fix identity, input shape, and the output requirement before picking a procedure.
More detail
Problem: “clean up these records” has no algorithm until the contract exists. Preconditions: none — that is the work. Guarantee: a contract makes correctness and cost checkable statements. Cost: contract choices change which procedures are even legal. Failure case: an underspecified identity rule discovered after the algorithm ships.
How do I compute under a modulus?
Euclid for gcd and inverses, squaring for powers — both logarithmic.
More detail
Problem: hashing and cryptographic arithmetic need gcd, inverses, and huge powers mod m. Preconditions: integer (or fixed-width) inputs; an inverse exists only when gcd(a, m) = 1. Guarantee: Euclid takes O(log min(a, b)) steps; exponentiation by squaring takes O(log e) multiplications. Cost: counted in multiplications of bounded integers. Failure case: fixed-width overflow inside the multiply — Python’s unbounded integers hide it, u64 does not.
Should I recurse, memoise, or fill a table?
Recurse when subproblems are independent; when they overlap, the decomposition becomes a DP state.
More detail
Problem: a natural recursion re-solves shared subproblems exponentially often. Preconditions: subproblems are parameterisable by a small state. Guarantee: independent splits give divide-and-conquer bounds; overlapping ones give a DP whose size is the state count. Cost: the memo or table is the state space, not the recursion depth. Failure case: a “subproblem” that isn’t self-contained — it needs context the state doesn’t carry.
Can one pass answer this instead of a nested loop?
On a sorted or monotone structure, two forward-moving indices settle a pair property in O(n).
More detail
Problem: pairwise questions default to Θ(n²) enumeration. Preconditions: the predicate is monotone in the pointer positions — moving a pointer can only shrink the candidate set one way. Guarantee: each step discards a candidate that can never matter again, so the scan terminates with the answer or its absence. Cost: O(n) moves. Failure case: a predicate that isn’t monotone — the pointers skip real answers silently.
A window property over all contiguous ranges?
Keep a window aggregate updated as both ends move once — O(n) for a whole family of range questions.
More detail
Problem: “longest/shortest subarray with property P” costs O(n²) evaluated per range. Preconditions: P is such that extending the window can be undone, and the aggregate supports extend and retract in O(1) amortised. Guarantee: both ends move monotonically, so the scan is linear in the sequence. Cost: one pass plus the structure’s update cost. Failure case: aggregates that can’t retract (no inverse operation) — they need a different structure.
Which boundary do I actually want?
First-true, last-true, insertion point — all are the same half-open search with a different predicate.
More detail
Problem: off-by-one at range edges is where search code dies. Preconditions: a monotone predicate over a finite ordered range. Guarantee: the [lo, hi) invariant shrinks to exactly the boundary asked, with termination from strict shrinkage. Cost: O(log n) probes. Failure case: the two-boundary convention mixed with closed intervals — pick one convention and the invariant decides for you.
The answer is “the smallest parameter that works”?
Turn an optimisation question into O(log range) feasibility checks when feasibility is monotone.
More detail
Problem: minimising a parameter by trial is expensive when trials are exact. Preconditions: “works at x ⇒ works at all larger x” (or the reverse) over an ordered parameter domain. Guarantee: the search returns the exact boundary value. Cost: log(range) feasibility evaluations; each evaluation must itself be correct. Failure case: a feasibility test that lies — the search then optimises the wrong problem perfectly.
Which sort does this input deserve?
Nearly-sorted input wants an adaptive sort; the inversion count is the measure.
More detail
Problem: a quadratic sort can beat n log n on the right input. Preconditions: input is nearly sorted (inv small) or very short. Guarantee: insertion sort runs Θ(n + inv) — linear where the comparison bound allows it. Cost: Θ(n²) worst case when inv is large. Failure case: using it blind on adversarial input; hybrid sorters keep it only for small runs.
Merge sort or quicksort?
Merge sort when you need the guarantee or stability; quicksort when constant factors and in-place matter.
More detail
Problem: the Θ(n log n) worst case is not the whole story. Preconditions: merge sort needs O(n) auxiliary space; quicksort needs a pivot scheme that survives hostile input. Guarantee: merge sort is Θ(n log n) always and stable; quicksort is expected Θ(n log n) with better locality. Cost: measured in comparisons and moves. Failure case: a fixed-pivot quicksort on adversarial input degrades to Θ(n²).
Need sorted order online, or just extreme values fast?
A heap gives both ends of the sorted order on demand; heapsort spends it all at once in place.
More detail
Problem: repeated max-extraction without full sorting cost. Preconditions: only the extremes matter per step, or the whole sort is wanted and in-place is required. Guarantee: Θ(n log n) worst case with O(1) extra space. Cost: n extractions at O(log n). Failure case: needing stability or good constants — heapsort is neither stable nor cache-friendly.
Just the k-th element, or the top k?
Quickselect answers order statistics in expected Θ(n); a heap of size k answers streaming top-k.
More detail
Problem: full sorting prices a question that only needs one rank. Preconditions: selection needs random access; streaming top-k needs only a heap of k. Guarantee: quickselect expected Θ(n), MoM deterministic Θ(n); the heap holds the top k exactly. Cost: much less than a sort. Failure case: believing a partial sort is free — a heap of k costs Θ(n + k log n) total.
Small integer keys — skip the comparison bound?
When keys live in a small universe or few digits, non-comparison sorts beat Θ(n log n) — legally.
More detail
Problem: the lower bound only governs comparison sorts. Preconditions: keys bounded (u small) or structured into O(1) digit passes. Guarantee: counting sort Θ(n + u), radix Θ(n·d/w) word passes. Cost: extra arrays and stability requirements. Failure case: u or digit count larger than assumed — the “linear” sort becomes the expensive one.
Unweighted shortest paths, or just reachability?
BFS gives shortest hop counts and layers in O(n + m) — nothing fancier needed on unweighted graphs.
More detail
Problem: shortest-path machinery is overkill for hop counts. Preconditions: edges unweighted; source given. Guarantee: discovery order is non-decreasing distance, so first visit is shortest. Cost: O(n + m), O(n) extra. Failure case: weighted edges — BFS answers the wrong question, silently and confidently.
Need the structure of the graph, not just the answer?
DFS timestamps classify edges and encode ancestry — the substrate for cycles, SCCs, and bridges.
More detail
Problem: “is it reachable” hides the structure questions underneath. Preconditions: a graph you can afford to recurse through. Guarantee: discovery/finish intervals make ancestor and edge-class queries O(1). Cost: O(n + m) plus recursion depth risk on deep graphs. Failure case: iterative conversions that mislabel the intervals — the invariant is the encoding.
A dependency order, or proof none exists?
Kahn’s algorithm produces a valid schedule or returns the leftover subgraph — the cycle witness.
More detail
Problem: scheduling under precedence constraints. Preconditions: the constraint graph is explicit. Guarantee: a produced order respects every edge; a stuck algorithm with remaining vertices certifies a cycle. Cost: O(n + m). Failure case: treating “no schedule yet” as “no schedule exists” — the leftover set decides.
Which sets are mutually dependent?
Two DFS passes collapse a directed graph into its condensation DAG of SCCs.
More detail
Problem: cyclic dependencies defeat schedule and optimisation passes. Preconditions: directed graph; reachability in both directions is the question. Guarantee: the condensation is a DAG — the coarsest faithful acyclic view. Cost: O(n + m). Failure case: treating one big SCC as separately solvable — the whole component is the unit.
Where are the single points of failure in this graph?
One DFS computes low[] values; bridge and articulation tests are comparisons on it.
More detail
Problem: connectivity that survives removing one edge or vertex. Preconditions: undirected connected graph (or run per component). Guarantee: low[v] > disc[u] certifies bridge (u, v) exactly. Cost: O(n + m). Failure case: the root’s separate articulation rule — it needs two children, not the low test.
Repeated ancestor or path questions on one tree?
Euler indexing answers ancestry in O(1); lifting tables answer LCA in O(log n) after linear prep.
More detail
Problem: per-query walks up the tree are linear each. Preconditions: a rooted tree, static across queries. Guarantee: interval tests and binary lifting are exact on any tree. Cost: O(n) or O(n log n) preprocessing, then per-query constants. Failure case: dynamic trees — the whole encoding goes stale.
Shortest paths — which algorithm matches the weights?
DAG → one topological pass; non-negative → Dijkstra; negative edges → Bellman–Ford.
More detail
Problem: the cheapest correct algorithm depends entirely on the weight contract. Preconditions: check weights and acyclicity first — both are part of the input contract. Guarantee: DAG relaxation is Θ(n + m) and exact; the others add machinery only when the weights demand it. Cost: relaxation count decides. Failure case: running Dijkstra where a negative edge lurks.
Non-negative weights and a single source?
Extract-min settles vertices in order of true distance — the workhorse when no edge is negative.
More detail
Problem: single-source shortest paths at scale. Preconditions: every weight non-negative — that is the settled-set argument’s entire premise. Guarantee: extraction order is distance order; stopping at the target is exact. Cost: Θ((n + m) log n) with a binary heap. Failure case: a negative edge makes the “final” distances wrong without error.
Negative edges, or negative-cycle detection?
The |V|−1-round relaxation that tolerates negative weights and certifies negative cycles.
More detail
Problem: the weight contract forbids Dijkstra. Preconditions: directed or undirected (undirected negative edges are already negative cycles). Guarantee: after |V|−1 rounds distances are δ-exact; a further improvable edge certifies a reachable negative cycle. Cost: Θ(nm). Failure case: reporting distances when a reachable negative cycle makes them −∞.
All pairs, sparse or dense?
Floyd–Warshall for dense simplicity; Johnson (reweight + n Dijkstras) when the graph is sparse.
More detail
Problem: n × n distance answers. Preconditions: Johnson needs one Bellman–Ford pass for potentials; Floyd–Warshall needs none but is Θ(n³) always. Guarantee: both exact; Johnson wins precisely when m ≪ n². Cost: Θ(n³) vs Θ(nm log n). Failure case: Floyd–Warshall’s memory at scale, or paying it on a sparse graph.
A goal to aim the search at?
A* = Dijkstra plus an admissible heuristic; the estimate only pays if it never overestimates.
More detail
Problem: uniform-cost search wastes work away from the goal. Preconditions: a heuristic h with h ≤ true remaining cost (admissible); consistency additionally prevents reopens. Guarantee: first extraction of the goal is optimal. Cost: potentially far less than Dijkstra; a zero heuristic degenerates to it. Failure case: an inadmissible h — fast, confident, wrong.
Does the greedy swap survive an exchange test?
Greedy is right when any optimum can absorb the greedy choice without loss — the proof to run before shipping it.
More detail
Problem: a plausible greedy rule may be exactly wrong. Preconditions: the problem’s constraint can be stated so the exchange is checkable. Guarantee: if the swap never worsens the optimum, greedy is optimal by induction. Cost: the argument is proof-time. Failure case: the swap degrades some optimum — that counterexample is where DP starts.
Select non-overlapping intervals?
Earliest-finish-first is optimal for the unweighted problem — and wrong the moment weights or resources enter.
More detail
Problem: maximum compatible interval set. Preconditions: equal weights, single resource, intervals comparable by finish time. Guarantee: the exchange shows any optimum admits the greedy first pick. Cost: Θ(n log n) for the sort. Failure case: weighted intervals, multiple machines, or release times — each break the exchange differently.
Which MST algorithm fits this graph?
Kruskal on edge lists, Prim on adjacency with a priority queue — both spend the cut property.
More detail
Problem: connect everything minimally. Preconditions: undirected, connected, weighted; distinct weights give a unique MST. Guarantee: the lightest edge across any cut is safe — both algorithms are that theorem iterated. Cost: Θ(m log n) either way; Prim’s constants win on dense graphs. Failure case: applying it to a directed or disconnected graph.
Need a code as short as the frequencies allow?
Huffman’s merge of the two rarest symbols produces the optimal prefix-free code.
More detail
Problem: fixed-width codes waste space on skewed frequencies. Preconditions: a symbol distribution; the code must be prefix-free for streaming decode. Guarantee: the greedy merge is optimal among prefix codes by the deepest-sibling exchange. Cost: Θ(n log n) over the alphabet. Failure case: frequencies that change between encoding and decoding, or needing better-than-symbol-level entropy.
A recursive cost that recomputes the same subproblems?
Define the state so each subproblem is self-contained, then the recurrence writes itself.
More detail
Problem: overlapping recursion explodes exponentially. Preconditions: optimal substructure plus a small state space. Guarantee: each state is solved once — correctness rides on the state being complete. Cost: states × per-state work. Failure case: a state that misses a parameter the answer depends on — every table value is then off.
Bounded quantities to allocate?
Capacity-indexed DP; the 0/1 version iterates capacity backwards — the direction is the correctness.
More detail
Problem: pick items under a resource limit. Preconditions: capacities integral (or discretised), item set static. Guarantee: the DP is exact for the stated variant; update order separates 0/1 from unbounded. Cost: Θ(n·C) — pseudo-polynomial, so huge C is the trap. Failure case: forward iteration on 0/1 items silently permitting reuse.
How far apart are these two sequences?
The prefix DP returns distance and the traceback the alignment — both, not one.
More detail
Problem: diff, fuzzy match, alignment. Preconditions: operation costs defined (unit costs or weighted). Guarantee: the table is the true minimum; the path through it is a witness alignment. Cost: Θ(nm) time and space — space can be squeezed to rows, reconstruction can’t. Failure case: needing the alignment but keeping only the last row.
Subproblems are contiguous intervals?
Fill intervals by increasing length so both halves of every split are ready.
More detail
Problem: matrix-chain multiplication and burst-style problems whose subproblems are [i, j] ranges. Preconditions: the combine depends only on the two sides. Guarantee: length-ordered filling makes the O(n³) split search exact. Cost: Θ(n³) typical. Failure case: an interval whose split needs outside context — the state is then wrong, not the order.
Tree or DAG structure to exploit?
Post-order DFS fills tree DP children-first; on DAGs the same recurrence needs memoisation.
More detail
Problem: structure-constrained optimisation (independent set on trees, longest path on DAGs). Preconditions: rooted tree or acyclic order available. Guarantee: each subtree/dag state computed once from its sub-states. Cost: Θ(n) states on trees. Failure case: shared substructure treated as nested — an unmemoised DAG recursion explodes.
Is the recurrence worth optimising before coding?
Space squeezing, quadrangle/Knuth speedups, and memo-versus-table are fill-order decisions made before the table exists.
More detail
Problem: a correct DP is too slow or too big. Preconditions: the recurrence’s dependency pattern is fully understood. Guarantee: same answers, smaller cost — the optimisations preserve semantics. Cost: analysis time. Failure case: dropping a dimension the recurrence still reads, or assuming a quadrangle inequality the costs don’t satisfy.
Feasible candidates countable but vast?
DFS over partial solutions with a safe prune — sound prunes only discard proven-dead branches.
More detail
Problem: enumeration space is exponential but constraints kill most of it early. Preconditions: partial solutions extend incrementally; prunes provably preserve an optimum. Guarantee: complete search — infeasibility is a real answer. Cost: bound quality decides. Failure case: a prune that can drop a true optimum — then “infeasible” lies.
Need the incumbent gap provable mid-search?
Best-bound frontier ordering turns backtracking into a procedure that knows how far from optimal it currently sits.
More detail
Problem: exact optimum wanted, runtime unknown. Preconditions: a bounding function estimating completions’ best possible value. Guarantee: when the best unexplored bound can’t beat the incumbent, the incumbent is optimal — early exit is sound. Cost: bound tightness is the runtime. Failure case: a loose bound explores nearly everything; a bound that overestimates silently loses the optimum.
Expressible as constraints a solver understands?
Encode the problem as clauses once; a complete solver returns a witness or a certificate of none.
More detail
Problem: hand-rolled search re-derives propagation every time. Preconditions: the problem admits a small clausal/constraint encoding. Guarantee: a complete solver’s UNSAT is a proof, not a heuristic shrug. Cost: encoding size × propagation per decision. Failure case: an encoding whose clause count itself explodes — the solver is fine, the model is the bug.
Maximum throughput through a capacitated network?
Augment along residual paths; when none remains, the reachable set is a cut proving optimality.
More detail
Problem: routing, assignment, or capacity-allocation phrased as flow. Preconditions: capacities on edges, a source and a sink; integral capacities give an integral optimum. Guarantee: no augmenting path ⟺ flow is maximum — the cut is the certificate. Cost: implementation-dependent; Edmonds–Karp is Θ(nm²). Failure case: irrational/real capacities where naive augmentation may not terminate.
Same flow, but every unit has a price?
Successive shortest augmenting paths keep each augmentation cheapest — cost-aware flow.
More detail
Problem: throughput with per-unit cost (shipping with prices, assignment with costs). Preconditions: costs per edge, required flow value; potentials keep reduced costs non-negative. Guarantee: augmenting along shortest residual paths preserves optimality. Cost: shortest-path computation per augmentation. Failure case: negative cycles in the residual — they must be cancelled first.
Pair two disjoint sets perfectly?
Alternating augmenting paths grow the matching until Berge’s theorem says stop — assignment problems included.
More detail
Problem: bipartite matching or assignment. Preconditions: the bipartition is real — edges only across. Guarantee: no augmenting path ⟺ maximum matching; König gives a vertex cover of equal size as the certificate. Cost: Hopcroft–Karp Θ(√n·m); Hungarian for weighted. Failure case: general graphs — odd cycles break the directed search.
Matching, but the graph isn’t bipartite?
Edmonds’ blossom contraction handles odd cycles — the extra machinery the bipartite shortcut skips.
More detail
Problem: pairing in a general undirected graph. Preconditions: undirected only; weights or unweighted both have versions. Guarantee: blossom contraction preserves maximum matchings — the algorithm stays exact. Cost: Θ(n³) classical implementations. Failure case: assuming the bipartite alternating search works — an odd cycle is a silent counterexample.
One pattern, one text, no backtracking?
The failure table makes the text pointer monotone — Θ(n + m) exact matching.
More detail
Problem: matching with worst-case guarantees. Preconditions: pattern and text over the same alphabet; pattern preprocessable. Guarantee: every occurrence reported, text never rescanned. Cost: Θ(m) table, Θ(n) scan. Failure case: pattern updates invalidate the table — per-query patterns may prefer a rolling hash.
Many windows, one fingerprint each?
An O(1) sliding hash turns pattern search into fingerprint comparison plus exact verification.
More detail
Problem: many-length-equal windows or multiple probes. Preconditions: a hash family; verification available (fingerprints are probable, not certain). Guarantee: no occurrence is missed; false positives are caught by the verifier — expected Θ(n + m). Cost: worst case degrades under adversarial hash inputs. Failure case: skipping verification — collisions then report false matches.
A whole dictionary in one pass?
Trie plus failure links reports every occurrence of every pattern in one text scan.
More detail
Problem: thousands of patterns per document (AV signatures, keyword filters). Preconditions: pattern set static enough to amortise the build. Guarantee: all matches reported in Θ(n + |matches|). Cost: build proportional to total pattern length. Failure case: patterns changing per query — the build must be paid again.
A regex with a worst-case bound?
Thompson’s NFA simulation is Θ(nm) with no backtracking blowup — the engine’s guarantee, not the pattern’s.
More detail
Problem: regex on untrusted or huge text where backtracking engines can explode. Preconditions: pattern compiles to an automaton; features like backreferences leave the NFA world. Guarantee: per-character simulation is exact — match iff some path accepts. Cost: Θ(m) automaton, Θ(nm) worst scan. Failure case: needing captures/backrefs — that is a different (backtracking) machine.
Store it once, transmit it smaller?
LZ77 parses the text into self-references; DEFLATE prices the parse and Huffman-codes the tokens.
More detail
Problem: lossless compression of text-like data. Preconditions: repeated substrings exist — the factorisation finds them. Guarantee: decompressible exactly; the compressed size is a bound the parse earns. Cost: sliding-window match search is the work. Failure case: already-compressed or entropy-dense input — expansion instead of compression.
A fair sample of a stream of unknown length?
k slots, one draw per item: after every prefix each position is held with probability k/i.
More detail
Problem: bounded-memory uniform sampling without knowing n. Preconditions: stream arrives once; fairness defined per position. Guarantee: the invariant is exact at every prefix — not asymptotically, exactly. Cost: Θ(n) time, Θ(k) state. Failure case: needing weighted or stratified fairness — the plain reservoir answers position-fairness only.
Which keys dominate the stream?
Misra–Gries counters give a deterministic candidate set; a verification pass certifies it.
More detail
Problem: find keys above φn in bounded memory. Preconditions: insertions only (deletions break the decrement logic); a replay or recount for certification. Guarantee: every key above n/(c+1) survives candidacy — the protocol emits exactly the certified set. Cost: Θ(c) state. Failure case: treating candidacy as certification — false candidates are the design, not a bug.
Find similar items without comparing all pairs?
MinHash signatures plus banding make candidacy an S-curve — the knee placed at the threshold wanted.
More detail
Problem: all-pairs similarity is quadratic. Preconditions: Jaccard (or another LSH-able) similarity; the candidate contract accepts probabilistic misses. Guarantee: candidacy probability 1 − (1 − J^r)^b per table, amplified by independent tables. Cost: signature length × bands per item. Failure case: needing guaranteed recall — LSH’s misses are silent by contract.
How sure is “probably correct”?
Repeat the estimator, price the failure as p^k, and report the measured distribution — never a single run.
More detail
Problem: a probabilistic answer needs a confidence, not a vibe. Preconditions: independent repetitions possible; an exact baseline exists for evaluation. Guarantee: k independent runs bound failure by p^k; evaluation reports recall distributions against ground truth. Cost: multiplicative in runs. Failure case: correlated runs — repetition of the same seed amplifies nothing.
Is this problem even solvable fast?
A polynomial reduction from a known-hard problem settles the status — direction matters.
More detail
Problem: effort is wasted optimising an NP-hard problem. Preconditions: a known-hard source problem and a valid instance map. Guarantee: A ≤_p B with A NP-hard makes B NP-hard — the reduction is the proof. Cost: the mapping must be polynomial. Failure case: reducing the wrong way (B ≤_p A proves nothing about B).
Exact is out of reach — what’s the best provable gap?
Prove a factor ρ against a bound on OPT — the guarantee is the product, measured against the optimum never computed.
More detail
Problem: NP-hard optimisation where a fast exact answer doesn’t exist. Preconditions: a bound on OPT (relaxation, dual, structure) the argument can compare against. Guarantee: cost ≤ ρ·C* always — on every instance. Cost: the algorithm is polynomial; ρ is the price. Failure case: a “ratio” demonstrated empirically — without the bound it’s a heuristic, not an approximation.
Decide now without knowing the future requests?
The competitive ratio prices decisions made under an adversary who sees the algorithm but not its coins.
More detail
Problem: caching, renting, scheduling where the input streams past. Preconditions: the adversary model named — oblivious for randomized algorithms. Guarantee: cost ≤ ρ·OPT + O(1) against the offline optimum. Cost: the ratio is the price of ignorance. Failure case: measuring against a non-adversarial average — the competitive claim is worst-case.
Does my geometry code survive near-degenerate input?
Compute signs exactly — a rounded determinant can flip the algorithm’s decisions.
More detail
Problem: collinear or cocircular points break floating-point orientation tests. Preconditions: coordinates bounded; predicates reducible to determinant signs. Guarantee: exact or filtered evaluation gives the right sign every time. Cost: fast float first, exact arithmetic only when the filter is uncertain. Failure case: comparing rounded coordinates — wrong answers with no error raised.
Boundary of a point set, or “is this point inside”?
The hull is sorted-order machinery; containment queries then binary-search the boundary.
More detail
Problem: hull construction or repeated inside/outside questions. Preconditions: points in the plane (higher dimensions are a different problem class). Guarantee: monotone-chain hull is exact in O(n log n); containment is O(log n) per query after it. Cost: the sort dominates. Failure case: assuming containment without the convexity the test leans on.
Geometric pairs — closest, crossing, overlapping?
A vertical sweep keeps only the active strip — closest pair and intersection in O(n log n).
More detail
Problem: pairwise geometric questions over n objects. Preconditions: events sortable by x; a status structure ordered by y. Guarantee: only objects within the strip matter — the invariant discards the rest provably. Cost: O(n log n). Failure case: a status structure that can’t maintain the strip order — the sweep then scans everything.
Multiply polynomials or convolve sequences?
Evaluate at roots of unity, multiply pointwise, transform back — Θ(n log n) convolution.
More detail
Problem: polynomial products, signal convolutions, string-with-wildcards tricks. Preconditions: sizes padded to a power of two; rounding care for integer outputs. Guarantee: exact in the transform domain; integer convolution needs NTT or error analysis. Cost: Θ(n log n). Failure case: naive floating-point on huge integers — rounding errors corrupt the result.
Solve Ax = b — how wrong can the answer get?
κ(A) prices the problem; pivoting prices the method — conditioning is the floor no algorithm beats.
More detail
Problem: linear solves where inputs carry measurement error. Preconditions: estimate κ(A) or at least detect it; pivoting for stability. Guarantee: error ≈ κ·ε_mach is the floor for any method. Cost: Θ(n³) dense elimination. Failure case: trusting a pivoted solve on an ill-conditioned A — stability ≠ conditioning.
When do I stop an iterative numerical method?
Bracket width certifies position; residual certifies the function value — pick the test that proves the claim you make.
More detail
Problem: iterative root-finding needs an exit that means something. Preconditions: a bracket (guaranteed) or a convergent scheme like Newton (fast, unbracketed). Guarantee: bracketing halves the interval — the bound is the interval width; Newton doubles digits when it converges. Cost: iterations × evaluation cost. Failure case: a small step on a stagnant iterate, or a flat function where small residual ≠ near the root.
Parallel combine — is the operator associative?
Work T₁ and span T∞ both derived; Brent’s bound prices the schedule — all resting on the operator’s contract.
More detail
Problem: reducing or scanning a large sequence on p processors. Preconditions: the combine operator is associative (and the scan additionally tracks order). Guarantee: T_p ≤ T₁/p + T∞; the two-sweep scan is exact. Cost: work overhead vs the sequential Θ(n). Failure case: floating-point fold is not associative — the parallel answer differs from sequential, legitimately but silently.
Memory-bound: does the algorithm know about blocks?
Count transfers, not comparisons: scans cost N/B, B-trees Θ(log_B N), and cache-oblivious code never names B.
More detail
Problem: performance dominated by memory traffic, not compute. Preconditions: the two-level model fits (fast M, block B); tall-cache M = Ω(B²) where transpose bounds need it. Guarantee: bounds in transfers — merge sort Θ((N/B)lg(N/M)), optimal sort Θ((N/B)log_{M/B}(N/B)). Cost: redesign of data layout. Failure case: optimising comparisons while the real cost is I/O.
Schedule under dependencies on w workers?
Any list schedule lands inside Graham’s (2 − 1/w)·C* envelope — stated as a contract, tested by a cycle report.
More detail
Problem: a DAG of jobs on a fixed fleet. Preconditions: durations known (or flagged as estimates), precedence graph acyclic — the SCC check runs first. Guarantee: C ≤ W/w + (1 − 1/w)L for any list schedule. Cost: Θ(n log n) priority list scheduling. Failure case: cycles block everything, and misestimated durations quietly move L — the study re-derives the bound after perturbation.
Filter + similarity in one query?
Compose an exact filter, a probabilistic index, and an exact reranker — recall is decided at the probabilistic stage.
More detail
Problem: conjunctive queries no single structure answers. Preconditions: each stage’s contract stated; an exact scan exists to measure recall against. Guarantee: exactness is terminal — downstream stages cannot recover a candidate the index never emitted. Cost: per-stage counters; breakeven on the index build is computed in queries. Failure case: under-filling the post-filter — a shallow K returns fewer than k answers silently.
Three deliverables, one bounded state?
Split the byte budget across sketch, counters, and reservoir — each buys a different error shape.
More detail
Problem: counts, heavy hitters, and a fair sample under a fixed memory cap. Preconditions: the question’s φ and error tolerances stated before the split is priced. Guarantee: each component’s contract is private — merges add the sketch exactly, double the counter bound, keep the reservoir marginal. Cost: the split is the design; degradation stays inside stated bounds. Failure case: an exact map past its silent-eviction cliff — wrongness with no flag.