The inventions, papers, launches and failures our courses teach, oldest first. Open an event for what came before it, who did it and why it mattered, and follow a link to the lesson that teaches it.
Before 1800
Algorithms before machines: procedures written for people to carry out.
PaperAlgorithms
The Euclidean Algorithm
Repeatedly replacing the larger number by its remainder finds the greatest common divisor, and the procedure provably stops.
The story
Before
Greek mathematicians had geometric and number-theoretic results about divisibility, but a method for finding the largest common divisor of two numbers, stated as a general procedure with a proof that it ends, was not written down as such.
What happened, and who
Around 300 BCE Euclid of Alexandria set out the procedure in Book VII of the Elements, his treatise on geometry and number theory. Given two numbers, subtract the smaller from the larger (in modern form, replace the larger by its remainder) until the numbers are equal; that value is their greatest common divisor.
Why it mattered
It is often cited as the oldest algorithm still in everyday use. Its extended form gives modular inverses, which is how RSA key generation finds a private exponent, and the same remainder loop appears wherever code reduces a fraction or computes with modular arithmetic.
A Sanskrit treatise on metre computes large powers with a few squarings and doublings instead of one multiplication per step.
The story
Before
Counting the patterns of a poetic metre needs powers of two, and the plain method is to double once for every unit of the exponent, so the work grows in step with the exponent itself.
What happened, and who
The Chandaḥśāstra, the earliest known Sanskrit treatise on prosody, is attributed to the Indian scholar Piṅgala and is conventionally dated to the last centuries BCE, roughly the third to second century BCE; the dating is approximate. It shortens the calculation by splitting the exponent into halves, using squarings and doublings where repeated doubling would take many more steps.
Why it mattered
This is an early instance of the halve-the-exponent trick, whose cost grows with the number of digits of the exponent rather than its size. That trick is exactly what makes modular exponentiation practical in RSA and Diffie–Hellman, where exponents run to hundreds of digits.
The mathematics computing would need: logic, probability and switching.
PaperAlgorithms
The Poisson Distribution
The count distribution for rare, independent events, which later became the standard model of random arrivals.
The story
Before
The binomial distribution gave the probability of a number of successes, but computing it was unwieldy when there were very many trials and each was very unlikely to succeed.
What happened, and who
In 1837 the French mathematician Siméon Denis Poisson published the distribution in his treatise Recherches sur la probabilité des jugements en matière criminelle et en matière civile, a study of the probability of verdicts in criminal and civil trials. It gives the chance of seeing exactly k events when they occur independently at a constant average rate.
Why it mattered
The distribution stayed a curiosity until it was applied to counts of real events. It is now the standard model of random arrivals in queueing analysis, so requests to a server or packets at a router are first modelled as Poisson streams, and the exponential gaps between them follow from it.
The division steps never exceed five times the digits of the smaller number, and the worst case is consecutive Fibonacci numbers.
The story
Before
Euclid's algorithm had been in use for about two thousand years, and it was known to stop, but nobody had given a bound on how many steps it could take.
What happened, and who
In 1844 the French mathematician Gabriel Lamé proved that the number of division steps is at most five times the number of decimal digits of the smaller number. The worst case arises when the inputs are consecutive Fibonacci numbers, each remainder shrinking as slowly as it possibly can.
Why it mattered
It is widely regarded as the first analysis of an algorithm's running time, and it ties the cost of a gcd to the size of the input's digits rather than its value. That logarithmic bound is what lets modular inverses and RSA key generation run quickly on numbers hundreds of digits long.
AND, OR and NOT become operations on true and false values, obeying algebraic laws like numbers do.
The story
Before
Logic was argued in prose and in Aristotle's syllogisms. It had no calculus in which an argument could be rewritten by rule rather than judged by reading.
What happened, and who
In 1854 the English mathematician George Boole published An Investigation of the Laws of Thought, treating logical statements as symbols that can be combined by operations resembling multiplication and addition, taking only the values true and false.
Why it mattered
Boole's algebra had no engineering use in his lifetime. Claude Shannon showed in 1937 that switching circuits obey it exactly, so every logic gate, ALU and bitwise operator in a program is a Boolean expression, and circuits can be simplified by algebra.
Random telephone calls turn out to follow the Poisson distribution, which lets an exchange be sized by calculation.
The story
Before
Telephone exchanges were sized by guesswork and experience. A company had no way to say how many lines would keep the chance of a caller finding no free line below a chosen level.
What happened, and who
Agner Krarup Erlang, a Danish mathematician who had joined the Copenhagen Telephone Company in 1908, published a paper in 1909 showing that calls made at random follow the Poisson distribution. In 1917 he published the formulae for the probability that a call is lost and that a caller must wait.
Why it mattered
This founded queueing theory. The same mathematics now sizes any pool of shared resources, such as telephone trunks, threads, connections or servers, against random demand, and Erlang's formulae remain the working tools of telecommunications capacity planning.
Two cross-coupled vacuum tubes hold one of two stable states after a pulse, a circuit that remembers a bit.
The story
Before
Vacuum-tube circuits could amplify signals and switch them, but they had no way to keep a value once the input went away.
What happened, and who
The British physicists William Eccles and F. W. Jordan filed a patent in June 1918 for a circuit of two triode vacuum tubes, each feeding the other's input. It settles into one of two states and stays there until a pulse pushes it into the other, and the patent called it a trigger relay.
Why it mattered
This bistable loop is the ancestor of every latch, flip-flop and SRAM cell, so it is the way hardware stores a single bit. It appeared in the 1943 Colossus codebreaking machine, and registers and caches are still built from the same idea in transistors.
Every component adds its cheapest outgoing edge at once, and the rounds repeat until one tree remains.
The story
Before
The Moravian electric power company needed a cheap way to link many towns into one grid, but no procedure existed for choosing the cheapest set of links connecting them all.
What happened, and who
In 1926 the Czech mathematician Otakar Borůvka, asked to design an efficient electricity network for Moravia, devised the first minimum spanning tree algorithm. Each round, every connected component picks its cheapest edge to another component and all those edges are added together.
Why it mattered
This is the origin of the minimum spanning tree problem. Because the rounds are independent, it is the ancestor of parallel and faster MST algorithms, and the problem itself still appears in network design and clustering. Jarník (1930) and Prim (1957) found other approaches later.
What can be computed at all, and circuits that do logic.
PaperAI
Thompson Sampling
Pick each option with the probability that it is the best one given the evidence so far.
The story
Before
In a medical trial, patients were assigned to treatments in fixed proportions decided in advance, even after early results suggested one treatment was better.
What happened, and who
In 1933 the American pathologist William R. Thompson published "On the likelihood that one unknown probability exceeds another in view of the evidence of two samples" in Biometrika. It computes the probability that one treatment is better than another and proposes giving each new patient a treatment with that probability.
Why it mattered
Thompson's idea was largely ignored for decades, then rediscovered as one of the best strategies for the multi-armed bandit problem. It is now used in online experiments and recommendation, where a system must balance trying options against using the best one found so far.
Keeping only a matrix's largest singular values gives the closest possible matrix of lower rank.
The story
Before
Reducing a large table of data to a few dimensions was done by ad hoc methods, with no guarantee that the result was the best approximation of a given size.
What happened, and who
In September 1936 Carl Eckart and Gale Young published "The approximation of one matrix by another of lower rank" in the journal Psychometrika. They proved that truncating the singular value decomposition gives the nearest matrix of rank k in the Frobenius norm; Leon Mirsky extended it in 1960 to a wider family of norms.
Why it mattered
The theorem is the guarantee behind principal component analysis, image compression, latent semantic analysis and matrix-factorisation recommenders. When an engineer truncates an SVD, this result says no other rank-k matrix would have been closer.
Defines an abstract machine and proves that some questions, such as the halting problem, cannot be decided by any algorithm.
The story
Before
The mathematician David Hilbert had asked for a mechanical procedure to decide whether any mathematical statement is provable, but 'mechanical procedure' had no precise definition. Alonzo Church had answered the question a few months earlier in his own formalism.
What happened, and who
Alan Turing's paper "On Computable Numbers, with an Application to the Entscheidungsproblem" was read to the London Mathematical Society on 12 November 1936 and appeared in volume 42 of its Proceedings (1936–37). It describes a simple machine with a tape and shows that it can compute anything that can be computed by a definite procedure, and that no such machine can decide, for every program, whether it halts.
Why it mattered
It gave computation a formal definition and the first proof that some problems are uncomputable. Every later argument that a problem is undecidable, or that one problem is at least as hard as another, rests on this foundation, and a modern computer is a practical realisation of Turing's universal machine.
A master's thesis shows that networks of switches obey Boole's algebra, so circuits can be designed by calculation.
The story
Before
Engineers designed telephone relay circuits by trial and error, with no theory to say whether a layout was correct or minimal.
What happened, and who
In 1937 Claude Shannon, a master's student at MIT, wrote a thesis showing that relay and switching circuits can be described and simplified with Boole's algebra of logic. It was published in 1938 as "A Symbolic Analysis of Relay and Switching Circuits" in the Transactions of the American Institute of Electrical Engineers.
Why it mattered
It turned circuit design into algebra: a logic function can be written as an expression, simplified on paper, and built. Every digital computer's gates and adders are designed this way, and the same algebra is why bitwise operators in code behave as they do.
The first electronic computers, and the program stored in memory.
InventionHardware
Harvard Mark I
A room-sized electromechanical computer that read its instructions from a punched paper tape, kept apart from its data.
The story
Before
Calculations were done by people with desk calculators or by single-purpose punched-card machines, and none could run a long sequence of different operations automatically.
What happened, and who
Howard Aiken of Harvard designed the machine and IBM built it, shipping it to Harvard in February 1944 and presenting it there on 7 August 1944. It used about 3,500 relays and counters, and it read its instructions from a 24-channel punched paper tape while the numbers lived in separate mechanical registers.
Why it mattered
Its separate instruction tape and data store, later called the Harvard architecture, is a foil for the stored-program design in which code and data share one memory. Modern processors still use the Harvard split in their separate instruction and data caches.
Describes a computer whose instructions and data share one memory, the design that carries von Neumann's name.
The story
Before
ENIAC held its program in wiring and switches, so changing the program meant days of re-cabling the machine.
What happened, and who
On 30 June 1945 the mathematician John von Neumann distributed a draft report at the University of Pennsylvania describing the planned EDVAC, a successor to ENIAC, drawing on ideas developed with J. Presper Eckert and John Mauchly. It set out a control unit, an arithmetic unit and a single memory holding both program and data. Only von Neumann's name was on it.
Why it mattered
The report defined the stored-program architecture that almost every computer still uses, and gave the machine's program a form that can be edited like data. It also gave us the von Neumann bottleneck: instructions and data compete for one path to memory.
Telephone traffic load gets a unit: one line kept busy for the whole hour is one erlang.
The story
Before
Telephone companies measured how heavily their lines were used in different local units, so traffic figures from different exchanges or countries could not be compared directly.
What happened, and who
In 1946 an international telephone standards committee named the unit of traffic the erlang, after the Danish mathematician Agner Krarup Erlang. One erlang is the load of a single line in use continuously; equivalently, an average of one call in progress at any moment.
Why it mattered
Load expressed as a number of busy servers is the ancestor of the utilisation used in every queueing calculation. Capacity planning for trunks, call centres and server pools still starts from the offered load in erlangs.
The first general-purpose electronic computer, programmed by setting switches and plugging cables.
The story
Before
Electromechanical machines such as the Harvard Mark I did arithmetic with relays and gears, orders of magnitude slower than electronics could allow.
What happened, and who
The University of Pennsylvania unveiled ENIAC on 14 February 1946. J. Presper Eckert and John Mauchly had built it, with thousands of vacuum tubes, for the US Army to compute artillery firing tables. To change its program, operators rewired cables and set switches, which could take days.
Why it mattered
ENIAC showed that electronic computing at scale was possible, and its rewiring problem is the problem that the stored-program design was created to remove. The lesson that programs should be data, not wiring, shaped every computer that followed.
An algorithm that walks from corner to corner of the feasible region to optimise a linear objective.
The story
Before
The US Air Force planned large logistics problems with ad hoc rules and desk calculators. There was no systematic method for finding the best plan subject to many linear constraints.
What happened, and who
George Dantzig, a mathematical adviser to the US Air Force, formulated planning problems as linear programs in 1946–47 and, adding an objective function in mid-1947, developed the simplex method over about a year. It moves along the edges of the feasible polytope, at each step to a neighbouring corner with a better objective value.
Why it mattered
This launched linear programming and modern operations research. Airline crew scheduling, supply chains and refinery blending are solved by simplex-based solvers today, and the same ideas underlie network simplex and the solvers for integer programs.
Two gold contacts pressed onto a germanium crystal amplify a signal: the first semiconductor amplifier.
The story
Before
Electronics ran on vacuum tubes. A tube needed a heated cathode, so it drew power, ran hot, took time to warm up and burned out; a machine built from thousands of them spent much of its life being repaired. Relays, the other switch, were mechanical and slow.
What happened, and who
At Bell Telephone Laboratories in Murray Hill, New Jersey, John Bardeen and Walter Brattain, in William Shockley's solid-state group, got their point-contact device to amplify on 16 December 1947. They showed it to the laboratory's officials on 23 December, and Bell Labs announced it to the press on 30 June 1948.
Why it mattered
The transistor is a switch with no moving parts and no heater, and it can be made very small. Every processor since is built from transistors used as switches, wired into logic gates, and shrinking them is what made computers cheap enough to be everywhere. Bardeen, Brattain and Shockley shared the 1956 Nobel Prize in Physics for it.
A tiny computer ran the first program held in electronic memory, a search for the highest factor of a number.
The story
Before
The stored-program design of the EDVAC report existed only on paper, and no machine had yet run a program kept in the same electronic memory as its data.
What happened, and who
On 21 June 1948 the Small-Scale Experimental Machine, nicknamed the Baby, ran its first program at the University of Manchester. Frederic Williams and Tom Kilburn designed it with Geoff Tootill, using Williams's cathode-ray-tube memory, and Kilburn wrote the program, which found the highest proper factor of a number.
Why it mattered
It was the first working demonstration that code can live in the same memory as data and be executed from it. The Baby was a test bed for the memory technology, and it led directly to the Manchester Mark 1 and the first commercial stored-program computers.
Defines the bit, entropy and channel capacity, and proves limits on compression and on reliable transmission.
The story
Before
Communication engineers built telegraph, telephone and radio systems without a measure of information, so nobody could say how far a message could be compressed or how reliably it could be sent through noise.
What happened, and who
Claude Shannon of Bell Labs published "A Mathematical Theory of Communication" in the Bell System Technical Journal, in two parts beginning in July 1948. It defines information in bits, entropy as the average information of a source, and the capacity of a noisy channel, and proves the source coding theorem and the noisy-channel coding theorem.
Why it mattered
It founded information theory. Entropy is the floor under lossless compression such as Huffman coding and DEFLATE, channel capacity is the target of error-correcting codes, and cross-entropy is the loss function used to train most machine learning classifiers.
The first practical stored-program computer in regular service, with the first assembler and the closed subroutine.
The story
Before
Programmers on the earliest machines wrote raw binary numbers or plugged cables, and each program was written from scratch with no library of reusable routines.
What happened, and who
At the University of Cambridge, Maurice Wilkes's team ran EDSAC's first program on 6 May 1949, and the machine then served researchers regularly. Its Initial Orders, a small loader, read programs written in single-letter mnemonics, and David Wheeler devised the closed subroutine, a routine that can be called from anywhere and returns to its caller.
Why it mattered
EDSAC started the software lineage of assemblers, subroutine libraries and linked calls. Every function call in a modern compiled program, and the calling conventions that make libraries usable, descend from Wheeler's subroutine idea.
Computers become products, with the first compilers, operating systems and lasting algorithms.
PaperHardware
Booth's Multiplication Recoding
A run of 1-bits in a multiplier is replaced by one subtraction and one addition, and signs are handled uniformly.
The story
Before
Shift-and-add multipliers spent one addition for every 1-bit of the multiplier, including long runs of ones, and needed separate handling for negative operands.
What happened, and who
In 1951 the British engineer Andrew Booth of Birkbeck College published a signed binary multiplication technique in the Quarterly Journal of Mechanics and Applied Mathematics. It recodes the multiplier so that a run of ones costs a subtraction at its start and an addition just past its end, and it works directly on two's-complement numbers.
Why it mattered
Booth recoding and its later radix-4 form became the basis of multipliers in real arithmetic logic units, halving the number of partial products in hardware. It shows how a change of number representation can save work in a circuit.
The control unit becomes a small stored program, in which each machine instruction is a sequence of micro-steps.
The story
Before
A computer's control logic was a tangle of gates wired for each instruction, which was hard to design correctly and hard to change.
What happened, and who
In July 1951 Maurice Wilkes of Cambridge proposed microprogramming at the inaugural conference of the Manchester University computer. He stored the control signals for each instruction in a regular array, with a second array choosing the next step, and coined the term.
Why it mattered
It made complex instruction sets practical and let one design be implemented on machines of different speeds, as in IBM's System/360 family. Later processors could patch bugs by updating microcode, and modern x86 chips still translate instructions into micro-operations internally.
The first-order conditions that a solution of a constrained optimisation problem must satisfy, now named for Karush, Kuhn and Tucker.
The story
Before
Lagrange multipliers handled constraints written as equalities, but many real problems have inequality constraints such as 'no more than this budget', for which no general condition was in wide use.
What happened, and who
In 1951 Harold Kuhn and Albert Tucker of Princeton published "Nonlinear Programming" in the proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, stating necessary conditions for optimisation with inequality constraints. William Karush had derived the same conditions in his 1939 University of Chicago master's thesis, which went unnoticed until later.
Why it mattered
The conditions are the basis of duality theory and of the support vector machine's formulation, and they underlie nearly every constrained optimiser. Checking them is how a solver decides that it has reached an optimum.
A number that measures how surprised you are, on average, by data from one distribution when you expected another.
The story
Before
Shannon's entropy measured the information in a single distribution. There was no standard way to measure how far one distribution is from another.
What happened, and who
In March 1951 the American mathematicians Solomon Kullback and Richard Leibler published "On Information and Sufficiency" in the Annals of Mathematical Statistics. They defined the mean information per observation for discriminating one probability law from another, now called the Kullback–Leibler divergence.
Why it mattered
KL divergence is the quantity behind cross-entropy loss, so minimising it is what training a classifier does. It also drives variational inference, and it is used to compare models and to measure drift between distributions.
A way to find where a function crosses a level using only noisy measurements, by shrinking the step size over time.
The story
Before
Root-finding methods such as Newton's method assumed the function could be evaluated exactly. When each measurement was noisy, there was no procedure guaranteed to converge.
What happened, and who
In September 1951 Herbert Robbins and Sutton Monro published "A Stochastic Approximation Method" in the Annals of Mathematical Statistics. Their scheme moves the estimate against each noisy reading by a step that shrinks according to a schedule, and they gave conditions under which it converges.
Why it mattered
It is the ancestor of stochastic gradient descent, the algorithm that trains nearly every neural network, which does exactly this with noisy gradients from small batches of data. The shrinking step-size schedule is still the theory behind learning-rate decay.
Each item has one hidden strength, and the chance that one beats another depends on the difference of their strengths.
The story
Before
Experiments that compare items two at a time, such as tasting two products and choosing one, produced tables of who beat whom but had no single-parameter model to turn them into a ranking.
What happened, and who
In 1952 the statisticians Ralph Bradley and Milton Terry published "Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons" in Biometrika. It gives each item a positive strength and makes the probability that i beats j equal to its strength divided by the sum of the two.
Why it mattered
The model is the standard way to turn pairwise choices into scores. It underlies the reward models trained from human preference comparisons in language-model alignment and the ratings of chatbot arenas.
Repeatedly merging the two rarest symbols builds the shortest possible prefix-free code.
The story
Before
Shannon–Fano coding, built from the top down by splitting the symbols into groups, was known not to always give the shortest code. Nobody had found a method proved optimal.
What happened, and who
David Huffman, a graduate student at MIT, found the bottom-up method while choosing between a term paper and a final exam in Robert Fano's information theory class. It was published in September 1952 as "A Method for the Construction of Minimum-Redundancy Codes" in the Proceedings of the IRE.
Why it mattered
Huffman coding is used inside DEFLATE (zip and gzip), JPEG and MP3. It is the textbook example of a greedy algorithm whose correctness is proved by an exchange argument.
A three-part shorthand, A/S/c, names a queue by its arrivals, its service times and its number of servers.
The story
Before
Queueing models were described in words each time, so it was awkward to say which arrival pattern, service pattern and number of servers a result applied to.
What happened, and who
In 1953 the British statistician David G. Kendall published "Stochastic Processes Occurring in the Theory of Queues and their Analysis by the Method of the Imbedded Markov Chain" in the Annals of Mathematical Statistics. It introduced the A/S/c notation, where A describes the time between arrivals, S the service-time distribution and c the number of servers, as in M/M/1.
Why it mattered
Kendall notation is the shared vocabulary of queueing theory. Engineers use M/M/1 and M/M/c to name the models behind server, thread-pool and call-centre sizing.
Records are dropped into buckets chosen by a function of their key, with colliding records kept in a chain.
The story
Before
To find a record by its contents, a program searched through a list or a sorted table, taking time that grew with the number of records. Addressing by content directly was not possible.
What happened, and who
In January 1953 Hans Peter Luhn, a researcher at IBM, wrote an internal memorandum proposing to put items into buckets by a function of the key, with items that fall in the same bucket chained together. The same year, at IBM, Gene Amdahl, Elaine McGraw, Nathaniel Rochester and Arthur Samuel implemented hashing with open addressing for the IBM 701 assembler.
Why it mattered
This is the origin of the hash table, which gives lookups in constant expected time. Hash tables now back dictionaries and sets in most programming languages, database indexes and caches.
Many small crossbar switches, wired in stages, can behave as one big switch that never blocks a connection.
The story
Before
A telephone exchange that could connect any line to any other used a single crossbar with a crosspoint for every pair of lines, which became impossibly large as the number of lines grew.
What happened, and who
In March 1953 Charles Clos of Bell Labs published "A Study of Non-Blocking Switching Networks" in the Bell System Technical Journal (volume 32, issue 2). He showed how to build a nonblocking switch from three stages of small crossbars, and how many middle-stage switches are needed.
Why it mattered
The Clos design is the basis of modern datacentre networks. The fat-tree fabrics used from about 2008 onwards are folded Clos networks, letting operators build very large networks from identical commodity switches.
A random walk that accepts or rejects each step samples from a distribution too complicated to draw from directly.
The story
Before
Physicists wanted average properties of a system of many interacting molecules, which needs samples from a distribution over an astronomically large set of configurations. Direct sampling was impossible.
What happened, and who
In June 1953 Nicholas Metropolis, Arianna and Marshall Rosenbluth, and Augusta and Edward Teller published "Equation of State Calculations by Fast Computing Machines" in the Journal of Chemical Physics, from Los Alamos, where the calculations ran on the MANIAC computer. Their method proposes a random move and accepts or rejects it with a probability that depends on how the energy changes.
Why it mattered
This is the origin of Markov chain Monte Carlo. W. K. Hastings generalised it in 1970, and it is now central to Bayesian statistics, where it lets a program sample from posterior distributions with no closed form.
The most that can flow through a network equals the capacity of its narrowest cut.
The story
Before
The capacity of a network of links, such as a railway system, was studied case by case. It was not known that the largest flow and the cheapest set of links to cut are two views of one number.
What happened, and who
On 19 November 1954 L. R. Ford Jr. and D. R. Fulkerson of the RAND Corporation issued the research memorandum "Maximal Flow through a Network" (RM-1400), which proves that the maximum flow equals the minimum cut capacity. The problem was formulated by T. E. Harris for a model of Soviet railway traffic; the journal version appeared in the Canadian Journal of Mathematics in 1956.
Why it mattered
The theorem is the foundation of network optimisation. It underlies algorithms for bipartite matching, project selection and image segmentation, and it explains why the bottleneck in a network is a cut.
Assigning n workers to n jobs at least cost in polynomial time, by adjusting prices until a perfect matching appears.
The story
Before
Finding the cheapest assignment of workers to jobs by trying every pairing takes factorial time, and solving it as a general linear program ignored its structure.
What happened, and who
In 1955 Harold Kuhn published "The Hungarian method for the assignment problem" in Naval Research Logistics Quarterly. He named the method after the Hungarian mathematicians Dénes Kőnig and Jenő Egerváry, whose earlier work it builds on, and James Munkres refined its analysis in 1957.
Why it mattered
It was one of the first combinatorial optimisation algorithms shown to run in polynomial time, and it introduced the primal-dual approach used in many later algorithms. Assignment problems in scheduling and tracking objects across video frames still use it.
Designed in twenty minutes without pencil and paper, to show off a new computer.
The story
Before
Finding the shortest route between two places meant examining all the possible paths, which becomes hopeless on a map of any size.
What happened, and who
In 1956 Edsger Dijkstra, at the Mathematisch Centrum in Amsterdam, needed a problem to demonstrate the ARMAC computer to non-specialists and chose the shortest route between two Dutch cities. By his own account he worked out the algorithm in about twenty minutes while shopping with his fiancée. He published it in 1959 as "A note on two problems in connexion with graphs" in Numerische Mathematik.
Why it mattered
It became the core of network routing, including the OSPF protocol, of map navigation and of countless graph libraries. It is the standard example of a greedy algorithm, settling one vertex at a time in order of distance.
The first operating system for the IBM 704, which started the next job by itself when one finished.
The story
Before
Between jobs, an operator had to load the next program by hand while the expensive machine sat idle, and every program carried its own code to drive the input and output devices.
What happened, and who
In 1956 Robert L. Patrick of General Motors Research and Owen Mock of North American Aviation built GM-NAA I/O for the IBM 704. It automatically started the next program when the previous one finished, and it supplied shared input and output routines. It built on a monitor that General Motors programmers had written for the IBM 701 in 1955.
Why it mattered
This batch monitor was an early operating system, and it set the pattern for automatic job sequencing and shared device routines. Those two jobs, running programs in turn and hiding device details, are still the core of what an operating system does.
Scan the edges from cheapest to dearest and keep each one that does not close a cycle.
The story
Before
Otakar Borůvka had solved the minimum spanning tree problem in 1926, but his work was little known outside Czechoslovakia, and other mathematicians searched for a simple method.
What happened, and who
In 1956 the American mathematician Joseph Kruskal published "On the shortest spanning subtree of a graph and the traveling salesman problem" in the Proceedings of the American Mathematical Society. It sorts the edges by weight and adds each edge that connects two different components, stopping when one tree remains.
Why it mattered
With a union-find structure to test whether an edge closes a cycle, Kruskal's greedy scan is the standard minimum spanning tree algorithm in practice. It is used in network design and as a step in clustering.
Read-only memory that a customer, not the factory, could fill in, first built for a missile's guidance computer.
The story
Before
The constants in a read-only memory were fixed when it was manufactured, so changing a value meant making a new one. That was inflexible for targeting data that might change.
What happened, and who
In 1956 Wen Tsing Chow, at the Arma division of American Bosch Arma in Garden City, New York, invented the programmable read-only memory at the request of the US Air Force. It stored the targeting constants in the airborne computer of the Atlas E/F intercontinental ballistic missile, and the patent was kept secret for several years.
Why it mattered
The idea of a chip that can be written once after manufacture started the line of EPROM, EEPROM and flash memory. Flash now holds firmware and stores data in phones, laptops and servers.
A word for a group of bits handled together, respelled from 'bite' so it could not be mistaken for 'bit'.
The story
Before
Computers handled data in groups of bits of various sizes, and there was no standard name for an addressable group smaller than a machine word.
What happened, and who
In June 1956 Werner Buchholz, working on IBM's Stretch computer, coined the word byte. In Stretch the size of a byte was variable, encoded in the instruction, and early documents show sizes of one to six bits.
Why it mattered
The 8-bit byte became universal after IBM's System/360 adopted it in the 1960s. Memory is now addressed, sized and priced in bytes, and every file and network protocol counts in them.
Dynamic Programming and the Curse of Dimensionality
A book turns multistage decisions into tables of solved subproblems, and names the exponential blow-up of the state space.
The story
Before
Problems where a decision now changes the choices later were solved by ad hoc recursion or by listing every sequence of decisions, whose number grows exponentially with the number of stages. Each problem needed its own trick.
What happened, and who
In 1957 Richard Bellman, then at the RAND Corporation, published the book Dynamic Programming (Princeton University Press). It set out a general method: define the value of a state, relate it to the values of the states that follow, and tabulate them once. The same book uses the phrase "curse of dimensionality" for how the number of states explodes as variables are added.
Why it mattered
Dynamic programming underlies sequence alignment, shortest paths, Viterbi decoding of hidden Markov models and reinforcement learning. The curse of dimensionality is why tabulating every state stops working for large problems, and so why approximate methods exist.
An assign-and-recentre loop for choosing signal levels that sat in a Bell Labs memo for 25 years before becoming k-means.
The story
Before
Pulse-code modulation turns a continuous signal into a small set of numbered levels, but there was no systematic way to choose where those levels should sit. Engineers placed them by hand or on a uniform grid, which wastes levels where the signal is rare.
What happened, and who
In 1957 Stuart Lloyd at Bell Labs devised an iteration for it: assign each sample to its nearest level, then move each level to the mean of its samples, and repeat. He circulated it as an internal Bell Labs memo, and it was not published until 1982, in IEEE Transactions on Information Theory.
Why it mattered
The same loop, applied to points in space instead of signal samples, is the standard k-means algorithm and is still called Lloyd's algorithm. James MacQueen gave it the name "k-means" in 1967, after Lloyd had written it but before anyone could cite it.
Grow one tree from a seed vertex by repeatedly adding the cheapest edge that leaves it, and it becomes a minimum spanning tree.
The story
Before
Vojtěch Jarník had published this algorithm in Czech in 1930, in a paper on a minimal-connection problem in the journal of the Moravian Natural Science Society. It reached almost no readers outside Czechoslovakia, so the method was found again by others before it became known.
What happened, and who
In 1957 Robert Prim, working at Bell Labs, published the same procedure in the Bell System Technical Journal, in a paper on shortest connection networks. Edsger Dijkstra described it again in 1959 in his three-page note on two graph problems.
Why it mattered
Prim's version, which keeps the vertices outside the tree in a priority queue keyed by their cheapest connecting edge, is the one taught and implemented today. The algorithm is credited to all three names in different books: Jarník's, Prim's, or DJP.
Relax every edge repeatedly and shortest paths emerge, even when some edge weights are negative.
The story
Before
Dijkstra's algorithm, published a year later, settles each vertex once and so fails when an edge weight is negative: a later, cheaper route through a negative edge can undercut a vertex already settled. A method that tolerated negative weights was needed for problems such as currency or cost-adjustment graphs.
What happened, and who
Richard Bellman published "On a Routing Problem" in the Quarterly of Applied Mathematics in 1958, applying dynamic programming to shortest paths: repeat, for as many rounds as there are vertices, a sweep that relaxes every edge. Lester Ford had described the same relaxation idea in a 1956 RAND report on network flow, and Edward Moore published a variant in 1957, so the name pairs Bellman with Ford.
Why it mattered
The algorithm can also report a negative cycle, where no shortest path exists. Run in a distributed way, with each node relaxing routes using only its neighbours' tables, it becomes distance-vector routing, the design behind early ARPANET routing and later protocols such as RIP.
A room-sized IBM 704 taught itself to tell cards marked on the left from cards marked on the right, after about 50 trials.
The story
Before
The neuron models of the time, such as the 1943 McCulloch–Pitts unit, had weights set by hand and no rule for adjusting them from examples. A machine that improved itself from data had not been shown.
What happened, and who
In July 1958 the US Office of Naval Research showed the press a perceptron built by Frank Rosenblatt of the Cornell Aeronautical Laboratory in Buffalo, simulated on an IBM 704. Fed punched cards, it learned to separate left-marked from right-marked cards after about 50 trials. The New York Times reported it on 8 July 1958 under the headline "New Navy Device Learns by Doing".
Why it mattered
The perceptron was the first trained neural network: a weighted sum whose weights are corrected after each mistake, the ancestor of the learning rules behind today's networks. The Navy's claims of walking and talking machines set expectations that Minsky and Papert's 1969 analysis of its limits helped deflate, which cooled neural-network research for years.
Transistors, resistors and capacitors made from one piece of germanium instead of being wired together as separate parts.
The story
Before
Every circuit was assembled from separate transistors, diodes, resistors and capacitors soldered together. As circuits grew to thousands of parts, the hand wiring became the limit: it cost money, took space and was where faults appeared.
What happened, and who
On 12 September 1958 Jack Kilby, an engineer newly hired at Texas Instruments, showed his managers a circuit made on a single sliver of germanium, its parts formed in the material and joined by fine wires. When connected to an oscilloscope it produced a steady sine wave, showing it worked. He filed a patent in February 1959.
Why it mattered
Kilby showed that the parts of a circuit could be made from the same material at the same time, the idea every chip depends on. His wired version could not be mass-produced; Robert Noyce's planar design, months later, solved that. Kilby shared the 2000 Nobel Prize in Physics for it.
Three pages give the shortest-path algorithm every engineer knows, along with a method for minimum spanning trees.
The story
Before
Shortest-path methods existed, such as Lester Ford's relaxation scheme of 1956, but they could revisit a vertex many times and gave no simple guarantee of when a distance was final. Dijkstra had worked out a faster idea in 1956 at the Mathematisch Centrum in Amsterdam, but it was unpublished.
What happened, and who
In 1959 Edsger Dijkstra published "A Note on Two Problems in Connexion with Graphs" in the journal Numerische Mathematik (volume 1, pages 269–271). It gave a method to find the shortest route between two nodes when all edge lengths are non-negative, and one to build a minimum spanning tree.
Why it mattered
With non-negative weights, the nearest unsettled vertex is already final, so each vertex can be settled exactly once. Dijkstra's algorithm, usually run with a priority queue, is the basis of link-state routing protocols such as OSPF and of route finding in maps and games; A* is a guided version of it.
A checkers program that improved by playing against itself, until its author said it played better than he did.
The story
Before
Game-playing programs scored positions with an evaluation function fixed by their programmers, so a program played only as well as its author's judgement. Nobody had shown a program improving that judgement from experience.
What happened, and who
Arthur Samuel of IBM worked on a checkers program from the early 1950s. His 1959 paper "Some Studies in Machine Learning Using the Game of Checkers", in the IBM Journal of Research and Development, described two kinds of learning: remembering positions it had already searched, and adjusting the weights of its evaluation function by playing thousands of games against copies of itself. The paper states that a computer can be programmed to learn to play better than the person who wrote the program.
Why it mattered
It was the first working learner to improve through self-play, and the paper's use of the phrase "machine learning" helped give the field its name. Learning an evaluation function from the outcome of games is the idea behind TD-Gammon and, at far greater scale, AlphaGo and AlphaZero.
A transistor switched by the electric field across a thin glass layer on silicon, and easy to make small.
The story
Before
The field-effect transistor had been proposed decades earlier, but attempts to make one failed. Charges trapped at the surface of the semiconductor, called surface states, screened out the electric field before it could control the current.
What happened, and who
In 1959 Mohamed Atalla and Dawon Kahng at Bell Labs made the first working silicon transistor with an insulated gate. Growing a layer of silicon dioxide on silicon by heating it greatly reduced the surface states, letting the gate's field reach the silicon. They demonstrated the device in 1960.
Why it mattered
The metal-oxide-semiconductor field-effect transistor, or MOSFET, needs little current to hold its state and shrinks well, so it can be packed by the billion on a chip. It is the switch inside processors, memory and phones, and CMOS logic is built from pairs of them.
A tree that spends one level per character of the key, so lookup cost depends on the key's length and prefixes come free.
The story
Before
Dictionaries were stored as sorted lists, searched by comparing whole keys, or in hash tables, which find an exact key quickly but cannot list every key that starts with a given prefix. Neither shares the storage of keys that begin alike.
What happened, and who
In 1959 René de la Briandais described a tree indexed by the successive characters of a key, in a paper on file searching with variable-length keys. In 1960 Edward Fredkin gave it the name in his paper "Trie Memory", from retrieval, and meant it to be said like "tree".
Why it mattered
Lookup takes time proportional to the key's length, whatever the number of keys, and every key sharing a prefix sits under one node. Tries and their compressed variants sit behind autocomplete, spell checkers and the longest-prefix matching that IP routers use to choose a route.
A whole circuit built on one silicon chip, joined by printed aluminium lines instead of hand-attached wires.
The story
Before
Jack Kilby's 1958 integrated circuit was cut from germanium and had its parts joined by external fine wires, which made it slow to build by hand and unsuited to mass production.
What happened, and who
In 1959 Robert Noyce of Fairchild Semiconductor built on Jean Hoerni's planar process, in which a layer of silicon oxide protects the chip's surface, to design a circuit in which all components lie on one silicon chip and are joined by aluminium lines laid over the oxide. His patent application, US 2,981,877, was filed in July 1959.
Why it mattered
Making the connections as part of the same photographic process as the components meant thousands could be produced together at low cost. This planar, monolithic form is how every integrated circuit has been made since, and it is what made Moore's doubling of components per chip possible.
Time-sharing, compatible families of machines, and the first packet-switched network.
DiscoveryAlgorithms
Karatsuba Multiplication
Splitting each number in half needs three half-size multiplications instead of four, beating the schoolbook method.
The story
Before
Everyone multiplied n-digit numbers by the schoolbook method, which takes about n² single-digit steps. The mathematician Andrey Kolmogorov conjectured that no method could do fundamentally better.
What happened, and who
In 1960 Anatoly Karatsuba, a 23-year-old student at Moscow State University, attended a seminar where Kolmogorov posed that conjecture, and within a week found an algorithm that disproved it, using about n^1.585 steps. It was published in 1962 in a paper credited to Kolmogorov and to Karatsuba and Yuri Ofman.
Why it mattered
It was the first multiplication faster than quadratic, and it showed that the schoolbook method was not the limit. Big-number libraries still use it for numbers of moderate size, and it opened the way to Toom–Cook and to multiplication by fast Fourier transform.
Alternate between evaluating a policy and greedily improving it; in a finite problem it stops at the best policy.
The story
Before
Bellman's dynamic programming found the best decisions for a Markov decision process by value iteration, repeatedly updating value estimates. It only reaches the answer in the limit, and never says when the best policy has been found.
What happened, and who
In 1960 Ronald Howard, then at MIT, published the book Dynamic Programming and Markov Processes (MIT Press). It gave policy iteration: compute the exact value of the current policy, change each state's action to the best one under those values, and repeat until nothing changes.
Why it mattered
A finite problem has finitely many deterministic policies and each round strictly improves the policy until it stops, so the method terminates at an optimal policy, often in few rounds. Its evaluate-then-improve pattern is the core of the generalised policy iteration that describes most reinforcement-learning algorithms.
A recursive estimator blends a model's prediction with each new noisy measurement, weighting each by how uncertain it is.
The story
Before
Norbert Wiener's filters from the 1940s were designed for signals whose statistics do not change over time, and were worked out in the frequency domain. They handled neither a system whose state changes, nor a stream of measurements arriving one at a time.
What happened, and who
In March 1960 Rudolf Kálmán, a Hungarian-born engineer, published "A New Approach to Linear Filtering and Prediction Problems" in the Journal of Basic Engineering. It described a filter that keeps an estimate of a system's state and its uncertainty, and updates both with each new measurement without storing past data.
Why it mattered
Only the latest estimate and its uncertainty need be kept, so it runs in real time on small machines. It was used in navigation for the Apollo programme and is inside GPS receivers, radar trackers and the sensor fusion of phones and robots.
A language paper describes a collector that finds every cell still reachable from the program, and reclaims the rest.
The story
Before
Programmers of the time allocated and freed storage by hand. In programs that build linked lists of symbols, where cells are shared and it is hard to tell when the last use has ended, this led to memory leaks and to cells freed while still in use.
What happened, and who
In April 1960 John McCarthy of MIT published "Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part I" in Communications of the ACM. It defined the language Lisp and described a collector that marks every cell reachable from the program's roots and then sweeps the rest back to a free list.
Why it mattered
It is the first garbage collector, and the reachability idea, that memory is garbage when no path from a root leads to it, underlies tracing collectors in Java, Go, JavaScript and .NET. Mark-and-sweep's pauses and fragmentation are the costs that later collectors were designed to reduce.
A search for a satisfying assignment that simplifies by unit propagation and splits on one variable at a time.
The story
Before
Early automated theorem provers tested propositional formulas by trying truth assignments exhaustively, whose number doubles with each variable. Even small problems were out of reach of the machines of the day.
What happened, and who
In July 1960 Martin Davis and Hilary Putnam published "A Computing Procedure for Quantification Theory" in the Journal of the ACM. Its propositional part removes clauses that a single-literal clause forces, and eliminates variables. In 1962 Davis, George Logemann and Donald Loveland replaced variable elimination with case splitting and backtracking, giving the form now called DPLL.
Why it mattered
Forcing values by unit propagation prunes most of the search. Modern SAT solvers extend DPLL with clause learning, known as CDCL, and are used to verify hardware and software, plan and schedule, and resolve dependencies in package managers.
Each object counts the references to it and is freed the moment the count reaches zero.
The story
Before
The only automatic reclamation was McCarthy's mark-sweep, described earlier that year. It runs when memory is exhausted and must pause the program to trace the whole heap, so freeing is neither prompt nor predictable.
What happened, and who
In December 1960 George Collins published "A Method for Overlapping and Erasure of Lists" in Communications of the ACM. It stores a count of references in each list cell, raises it when a reference is made, lowers it when one is dropped, and frees the cell and lowers the counts of its children when the count hits zero.
Why it mattered
Memory is returned immediately and in small steps, without a whole-heap pause. CPython, Swift's ARC and Rust's Rc and Arc use it. Its known weakness is that a cycle of objects holds each other's counts above zero, so a separate cycle collector or weak references are needed.
In a stable system, the average number of items inside equals arrival rate times average time spent inside.
The story
Before
Philip Morse had observed in 1958 that this relation held in the queueing models he studied, and treated it as a property of those models. Each model needed its own derivation, and it was not known whether it held in general.
What happened, and who
In 1961 John D. C. Little of MIT published a proof in Operations Research that L = λW holds for any stable queueing system, whatever the arrival pattern, service order or service-time distribution. It relates the mean number in the system L, the mean arrival rate λ and the mean time in the system W.
Why it mattered
Because it needs no assumptions about distributions, it can be used on any real system from two measured numbers to get the third. Engineers use it to size thread pools, connection pools and queues, and to check that measured concurrency, throughput and latency are consistent.
A supercomputer that overlapped fetch, decode and execute, and met the problem of exceptions arriving out of order.
The story
Before
Computers such as the IBM 704 completed each instruction before starting the next, so the fetch, decode and arithmetic units were idle most of the time. Making the machine faster meant making each of those steps faster.
What happened, and who
IBM delivered the first 7030, called Stretch, to the Los Alamos Scientific Laboratory in April 1961, and it was accepted the next month. Designed to run at least 100 times faster than the 704, it prefetched and decoded instructions ahead, interleaved memory banks, and overlapped instructions in a pipeline. In benchmarks it reached only about 30 times the 704.
Why it mattered
Stretch missed its target and IBM cut the price and stopped selling it, but its techniques became standard: pipelining, prefetching and memory interleaving in later supercomputers. It also exposed a problem every pipelined processor faces, that an error in one instruction can be reported after later ones have started, the imprecise interrupt.
Partition around a pivot and sort each side; the companion Find recurses into only the side that holds the answer.
The story
Before
Sorting by merging needed a second area of memory as large as the data, and simple methods such as insertion sort took time proportional to the square of the number of items.
What happened, and who
C. A. R. Hoare devised quicksort in 1959 while a visiting student in Moscow, to sort words for a machine-translation project. In July 1961 Communications of the ACM published his Algorithm 63 (Partition), Algorithm 64 (Quicksort) and Algorithm 65 (Find), which selects the item of a given rank without sorting everything.
Why it mattered
Quicksort sorts in place with little extra memory and is fast in practice, on average about n log n comparisons. It was the model for the C library's qsort and for the sort in many language libraries, and Find, now called quickselect, is the standard way to find medians and top-k items.
The Compatible Time-Sharing System let several people use one IBM 709 at once, swapping each user's program to tape between turns.
The story
Before
Computers ran one batch job at a time. You handed in cards and waited hours or a day for the printout, so a single typing mistake cost a whole day's turnaround.
What happened, and who
In November 1961 a team at MIT led by Fernando Corbató, with Marjorie Merwin Daggett and Robert Daley, first demonstrated CTSS on the institute's modified IBM 709. The supervisor took each user's typed line by interrupt and, when a return character arrived, swapped the running program out to tape and chose what to run next.
Why it mattered
It showed people that one computer could be used interactively by many people, and made programmers expect a responsive machine. Its ideas, a scheduling quantum, queues that favour short jobs, and hardware to protect the supervisor from users, were carried into Multics and Unix and so into every later operating system.
A search tree that repairs itself after each insertion so subtree heights never differ by more than one.
The story
Before
A plain binary search tree is fast only if its shape happens to be balanced; inserting keys that arrive already sorted turns it into a linked list with linear-time search.
What happened, and who
In 1962 the Soviet mathematicians Georgy Adelson-Velsky and Evgenii Landis published "An Algorithm for the Organization of Information" in the Doklady of the Academy of Sciences of the USSR. It kept the heights of the two subtrees of every node within one of each other, restoring that with rotations after an insertion or deletion.
Why it mattered
Keeping the tree shallow makes search, insertion and deletion take time proportional to log n in the worst case. AVL trees are the oldest self-balancing search tree, and later balancing schemes such as red-black trees are usually taught in comparison to them.
The CTSS paper sets out what time-sharing is, a multilevel scheduling scheme, and the hardware a protected supervisor needs.
The story
Before
Batch monitors ran one job to completion and gave no interactivity. On a machine with no memory protection, one user's program could overwrite the supervisor or another user's program, so sharing a machine between users was unsafe.
What happened, and who
In May 1962 F. J. Corbató, M. Merwin Daggett and R. C. Daley presented "An Experimental Time-Sharing System" at the Spring Joint Computer Conference. It described CTSS and argued that a machine for time-sharing needs a special mode with memory protection, dynamic relocation and trapping of a user's input-output instructions. It also gave a multilevel scheduling algorithm.
Why it mattered
It is an early published account of time-sharing, of multilevel feedback scheduling, which favours short interactive jobs, and of the split between a privileged supervisor and unprivileged user programs. Operating systems still rely on that split, in every CPU's user and kernel modes.
Three nested loops, allowing one more intermediate vertex each round, compute shortest paths between every pair of vertices.
The story
Before
To get all-pairs shortest paths you ran a single-source algorithm from each vertex in turn. For reachability alone, the transitive closure of a graph, there was no equally short procedure.
What happened, and who
In January 1962 Stephen Warshall published "A Theorem on Boolean Matrices" in the Journal of the ACM, computing the transitive closure of a graph. In June 1962 Robert Floyd published Algorithm 97, "Shortest Path", in Communications of the ACM, extending the same scheme to weighted edges. Bernard Roy had published a related result in 1959, and Peter Ingerman gave the form with the loops in their present order.
Why it mattered
Its code is only a few lines and it handles negative edges so long as there are no negative cycles, which makes it a compact example of dynamic programming. It is used for transitive closure and for computing distance tables on small dense graphs.
The first machine to give each program one large memory of its own, backed by a small core and a large drum.
The story
Before
Fast core memory was small and the large drum was slow, and programmers had to move code and data between them by hand, using overlays they planned themselves. A program that outgrew core had to be rewritten to fit.
What happened, and who
Atlas was designed at Manchester University with Ferranti, under Tom Kilburn, with David Edwards, Geoff Lanigan and Frank Sumner. It was inaugurated on 7 December 1962. Its one-level store cut memory into fixed-size pages, translated program addresses to core frames through a page table in hardware, and used a usage bit on each page to choose which to send back to the drum.
Why it mattered
Programmers could now write as if memory were as large as the drum and leave the movement of pages to the machine. Paged virtual memory, with the same ingredients of address translation, page faults and replacement, spread to every general-purpose operating system.
Logic gates built from complementary transistor pairs draw almost no current unless they are switching.
The story
Before
Gates built from bipolar transistors, or from single-type MOS transistors with a load, conduct current in at least one of their two output states. A chip full of them draws steady power and gets hot even when nothing changes.
What happened, and who
In February 1963 Frank Wanlass and C. T. Sah of Fairchild Semiconductor presented "Nanowatt Logic Using Field-Effect Metal-Oxide Semiconductor Triodes" at the International Solid-State Circuits Conference. It paired an n-type and a p-type MOS transistor so that one always blocks current in a steady state. Wanlass filed a patent that June.
Why it mattered
Only the moment of switching uses noticeable power, so a chip's consumption scales with how much of it is active. That let designers put millions and then billions of transistors on one chip, and CMOS is the technology of processors, memory and phone chips today.
Six computers, from small to large, that all run the same machine code.
The story
Before
Each IBM line had its own instruction set: a business ran accounting on a 1401 and engineering on a 7090, and moving to a bigger machine meant rewriting its programs. Buying a new computer meant starting the software again.
What happened, and who
On 7 April 1964 IBM announced a family of six models whose speeds differed by a factor of about fifty, all running exactly the same machine code. Gene Amdahl led the machines' design, Fred Brooks the software, and Gerrit Blaauw was the third of the architects who defined its instruction set.
Why it mattered
System/360 separated an architecture, the contract a program sees, from the implementation that carries it out, so software outlived the hardware it was written for. It also made the 8-bit byte and byte-addressed memory the industry's norm. Every instruction-set family since, x86 and Arm included, rests on that separation.
A binary heap held in a plain array gives an in-place sort with a worst case of n log n steps.
The story
Before
Quicksort was fast in practice but could take quadratic time on bad input, and merge sort needed extra memory. No sort was both in-place and guaranteed n log n.
What happened, and who
In June 1964 J. W. J. Williams published Algorithm 232, "Heapsort", in Communications of the ACM. It introduced the binary heap, a tree stored in an array where every parent outranks its children, and sorted by building a heap and repeatedly removing its top. Robert Floyd published a faster way to build the heap in Algorithm 245 that December.
Why it mattered
The binary heap is the standard priority queue, behind schedulers, Dijkstra's algorithm and event simulators. Heapsort's worst-case guarantee makes it the fallback in introsort, the hybrid used by C++ standard library implementations.
Generally called the first successful supercomputer: ten parallel units, instructions that ran out of order, and a load-store design.
The story
Before
Machines of the time, including the IBM 7030 Stretch, ran into limits when they tried to overlap instructions, and most still executed one operation at a time in program order. Their arithmetic units sat idle while one instruction finished.
What happened, and who
Seymour Cray's team at Control Data Corporation introduced the CDC 6600 in September 1964, with first deliveries in 1965. Its processor had ten functional units that could work at once, and a scoreboard that let an instruction start as soon as its operands and unit were free, even if earlier ones were still running. Setting an address register loaded or stored a word in an operand register. It used ones' complement integers.
Why it mattered
It ran about three times faster than the Stretch. Issuing instructions out of order using a scoreboard is the ancestor of the out-of-order cores in today's processors, and the load-store discipline, memory touched only by dedicated instructions, was adopted by the RISC designs of the 1980s.
A one-page memo proposes joining programs end to end like sections of hose, the idea behind the Unix pipe.
The story
Before
Programs on the machines of the time ran as separate steps, passing results to each other through temporary files or cards. Nothing let a program's output flow straight into another program's input.
What happened, and who
On 11 October 1964 Doug McIlroy of Bell Labs wrote an internal memo saying: "We should have some ways of coupling programs like garden hose - screw in another segment when it becomes necessary to massage data in another way." He kept pressing the idea, and in 1973 Ken Thompson added the pipe() system call and the shell's pipe notation to Unix in a single night.
Why it mattered
Pipes let small programs that each do one job be combined with the shell's | into new tools, the composable style that shaped Unix. Every shell pipeline and every command-line filter chain today grows from this memo.
A protected integer with two operations lets processes block and wake each other instead of spinning.
The story
Before
Mutual exclusion between processes needed busy-waiting protocols such as Dekker's algorithm, which spin on shared variables and become awkward for more than two processes. There was no simple general way to make a process wait.
What happened, and who
Edsger Dijkstra introduced the semaphore in his note EWD123, "Cooperating Sequential Processes", which circulated in 1965 and was published in 1968. A semaphore is an integer with two operations: P waits until it is positive and decrements it, and V increments it, waking a waiting process if there is one.
Why it mattered
It was the first general blocking synchronisation primitive: one mechanism that gives mutual exclusion, signalling and counting of resources. Its descendants, counting semaphores, mutexes and condition variables, are how threads are coordinated in every operating system and language runtime.
A matching algorithm for general graphs, whose author argues that "polynomial time" is the right meaning of efficient.
The story
Before
Matching the vertices of a graph in pairs was solved for bipartite graphs, but graphs with odd cycles had no algorithm known to run in polynomial time. "Efficient" also had no agreed definition beyond "it works on the machine".
What happened, and who
In 1965 Jack Edmonds published "Paths, Trees, and Flowers" in the Canadian Journal of Mathematics. It gave a polynomial-time algorithm for maximum matching in any graph, which shrinks odd cycles, the "blossoms", to single vertices. In the same paper he argued that an algorithm whose running time is bounded by a polynomial in the input size should count as good.
Why it mattered
That definition of efficiency became the standard in complexity theory and is the P in the P versus NP question. The blossom algorithm itself is used for matching problems such as assigning workers to tasks and pairing items.
A short note proposes putting a small fast memory in front of a large slow one, so most accesses are fast.
The story
Before
Main memory was much slower than the processor, and a machine's speed was bounded by it. The only remedy was to make the whole memory faster, which was too expensive at any useful size.
What happened, and who
In April 1965 Maurice Wilkes of Cambridge published "Slave Memories and Dynamic Storage Allocation", a two-page note in IEEE Transactions on Electronic Computers. It proposed a small fast core memory acting as a slave to a larger, slower one, holding the most recently used data so that the effective access time is close to that of the fast memory.
Why it mattered
This is the idea of the CPU cache. Nearly every processor today has several levels of cache, and much of the performance of software depends on whether its accesses hit them.
Splitting a transform into two half-size transforms turns an n² computation into one taking about n log n steps.
The story
Before
The discrete Fourier transform, which finds the frequencies in n samples, was computed directly in about n² operations. Carl Friedrich Gauss had found a faster method around 1805, but it was unpublished in a form that reached later engineers.
What happened, and who
In April 1965 James Cooley of IBM and John Tukey of Princeton published "An Algorithm for the Machine Calculation of Complex Fourier Series" in Mathematics of Computation. It computed the transform by repeatedly splitting it into halves of the even and odd samples. The method was wanted for analysing seismic data, including for detecting nuclear tests.
Why it mattered
For a million samples the saving is a factor of tens of thousands, so spectrum analysis, audio and image compression, and radar and communications signal processing became practical. Convolution and multiplication of large numbers also run through the FFT.
A four-point trend line of components per chip projects that their number will double every year.
The story
Before
Integrated circuits were a few years old and expensive, and it was not clear that they would ever replace discrete parts. No one had put a forecast on how quickly they might improve.
What happened, and who
In the 19 April 1965 issue of Electronics, Gordon Moore, then director of research at Fairchild Semiconductor, published "Cramming More Components onto Integrated Circuits". He plotted the number of components on the cheapest chip against year, saw it double every year, and projected about 65,000 components on a chip by 1975. In 1975 he revised the pace to a doubling about every two years.
Why it mattered
The trend held for decades, and the industry used it to plan: chip makers set their research schedules to it and software makers counted on faster hardware to come. "Moore's law", named later by others, is also why its end, in power limits and cost, changed how processors are designed.
A one-page paper states mutual exclusion precisely and gives a correct solution using only ordinary reads and writes.
The story
Before
Programs that shared data between concurrent processes coordinated with ad hoc schemes that were often subtly wrong, and failures that depended on timing were nearly impossible to reproduce or reason about.
What happened, and who
In September 1965 Edsger Dijkstra, then at the Technological University in Eindhoven, published "Solution of a Problem in Concurrent Programming Control" in Communications of the ACM (volume 8, number 9). It stated the mutual-exclusion problem, which is to let only one process at a time run a critical section, and gave a solution for any number of processes. It credits the Dutch mathematician Th. J. Dekker with the two-process solution it generalises.
Why it mattered
It framed the requirements, safety and freedom from deadlock and starvation, that every later lock is judged by, and it started the formal study of concurrent programming. Locks, semaphores and monitors are all answers to this problem.
A plan for a computer utility shared by many users, with segments, dynamic linking and a hierarchical file system.
The story
Before
MIT's CTSS had proved time-sharing but was a batch system adapted to work that way, running on a machine not designed for it, with small memory and a flat file store.
What happened, and who
At the Fall Joint Computer Conference in November 1965, MIT's Project MAC, the General Electric Company and Bell Labs presented a set of papers describing Multics, the Multiplexed Information and Computing Service. Fernando Corbató, Victor Vyssotsky, Robert Daley, Peter Neumann and others described a system where every program and data file is a segment that many users can share, dynamically linked when first used, kept in a tree of directories.
Why it mattered
Multics established segmentation, the directory tree and dynamic linking, and its later versions added hardware protection rings. It was big and late, and Bell Labs withdrew from it in 1969; the researchers who left, Ken Thompson and Dennis Ritchie among them, built Unix as a simpler answer.
Evicting the page whose next use is furthest away is provably the best any replacement policy can do.
The story
Before
When paged memory arrived, designers chose which page to evict by guesswork, and policies were compared on a handful of programs with no yardstick for how much better a policy could be.
What happened, and who
In 1966 Laszlo Bélády of IBM published "A Study of Replacement Algorithms for a Virtual-Storage Computer" in the IBM Systems Journal. It compared policies through simulation and showed that the best possible policy, one that knows the future and evicts the page used furthest ahead, minimises the number of page faults.
Why it mattered
A real system cannot see the future, but the optimum is the ceiling against which LRU, Clock and other policies are measured on recorded traces. The same yardstick is used for caches of every kind, from CPU caches to content-delivery networks, where recent research tries to approximate it.
Four boxes, SISD, SIMD, MISD and MIMD, classify computers by how many instruction and data streams they run.
The story
Before
Designers were building machines with several processors, vector units and pipelines, but there were no shared terms to compare them. Every machine was described in its own words.
What happened, and who
In 1966 Michael Flynn, then at Northwestern University, published a survey, "Very High-Speed Computing Systems", in the Proceedings of the IEEE. It sorted machines by whether they have one or many instruction streams and one or many data streams. He refined the scheme in 1972.
Why it mattered
The four categories are still the first split in any talk about parallelism: a single core is SISD, a vector or GPU instruction is SIMD, and a multicore processor or a cluster is MIMD. MISD is rare, mainly in fault-tolerant designs.
One transistor plus a capacitor stores a bit, the design behind main memory ever since.
The story
Before
Computer memory was either magnetic core, threaded by hand, or semiconductor SRAM, which needs about six transistors to hold one bit. Six transistors per bit made semiconductor memory too large and costly to replace core as main memory.
What happened, and who
In 1966 Robert Dennard at IBM's Thomas J. Watson Research Center asked what a bit of storage minimally needs, and answered with one transistor as a switch and one capacitor holding the charge. IBM filed the patent, US 3,387,286, on 14 July 1967 and it was granted on 4 June 1968.
Why it mattered
Because a cell is a single transistor and a capacitor, memory chips could be packed far more densely than SRAM, and dynamic RAM became the main memory of almost every computer. The price of that economy is that the charge leaks and must be refreshed, and DRAM is slower than the processor that uses it, which is where the memory wall comes from.
Gives each user a full virtual machine, running on a machine with paged memory hardware.
The story
Before
Time-sharing systems such as CTSS and Multics let many users share one operating system, so every user ran inside the same system's rules. A user could not run a different operating system, or test a new one, without taking the whole machine.
What happened, and who
IBM's Cambridge Scientific Center, a research group with ties to MIT, built CP-40 first, which was in production use from early 1967 on a specially modified System/360 Model 40. It then rewrote it as CP-67 for the System/360 Model 67, which had paging hardware, and began testing on a real Model 67 at MIT's Lincoln Laboratory in 1967. CP-67, with its companion single-user system CMS, was made available to customers through IBM's Type-III Library in 1968.
Why it mattered
CP/CMS gave each user a simulated stand-alone System/360, so one machine could run many operating systems at once. It was the first widely available virtual machine system, and IBM's VM/370 of 1972 carried the idea into a product line. The same idea, formalised by Popek and Goldberg in 1974, is what cloud hypervisors rely on today.
The word for microcode that resides in a writable control store.
The story
Before
Programs were either hardware, wired into the machine, or software, loaded and changed by the user. Microcode, the small program that drives a processor's control logic, sat awkwardly between the two, and had no name of its own.
What happened, and who
In the January 1967 issue of the magazine Datamation, Ascher Opler of the Computer Usage Company coined the word firmware. He expected future computers to keep microcode in a writable high-speed control store, so that software functions could move into microcode and instruction sets could be changed by loading new contents.
Why it mattered
The word outlived the idea it was coined for. It now names any software stored in a device to make it work, from the BIOS or UEFI code that starts a computer to the code inside a disk drive or a network card, and 'flash the firmware' is a routine operation.
Asymptotically, the 1-nearest-neighbour error is at most twice the Bayes-optimal error.
The story
Before
The nearest-neighbour rule, which labels a new point with the label of the closest known point, was used as a rule of thumb. Nobody had shown how good it could be compared with the best possible classifier.
What happened, and who
Thomas Cover and Peter Hart, in 'Nearest Neighbor Pattern Classification' in IEEE Transactions on Information Theory, January 1967, proved a bound on the rule's error. As the number of stored examples grows without limit, the error of the 1-nearest-neighbour rule is at most twice the Bayes error, the lowest error any classifier can reach.
Why it mattered
It is one of the few strong guarantees for a method that involves no training at all. It says that a classifier that only stores the data and looks up neighbours is never worse than twice the best possible, and it is still the reference point for how much data a neighbour method needs.
Reservation stations, a common data bus and register renaming let floating-point instructions execute out of order.
The story
Before
The CDC 6600's scoreboard, from 1964, let an instruction start only when its inputs were ready, but it stalled instructions that merely reused a register name another instruction was still using. Those name dependences held back work that had no real data connection.
What happened, and who
Robert Tomasulo of IBM described the scheme in 'An Efficient Algorithm for Exploiting Multiple Arithmetic Units', IBM Journal of Research and Development, January 1967. It was built into the floating-point unit of the System/360 Model 91: instructions wait in reservation stations, results travel on a common data bus, and register tags rename destinations so that independent instructions run at once.
Why it mattered
Register renaming with waiting stations is the core of how out-of-order processors find parallelism in a sequential program. The idea lay largely unused in mainstream designs for decades, then returned with the Pentium Pro in 1995, and today's high-performance cores from x86 to Arm all descend from it.
A three-page paper argues that serial fractions cap the benefit of adding processors.
The story
Before
In the mid-1960s the debate was between building one faster processor and interconnecting many, as in the array-processor designs of the time such as the ILLIAC IV. Advocates of many processors expected speedup to grow with their number.
What happened, and who
Gene Amdahl of IBM presented 'Validity of the Single Processor Approach to Achieving Large Scale Computing Capabilities' at the AFIPS Spring Joint Computer Conference in Atlantic City in April 1967, in three pages. He argued that the part of a job that cannot be parallelised, together with the overhead of coordinating processors, limits what extra processors can give.
Why it mattered
The result, named Amdahl's law by later writers, is the first check made on any plan to scale out: measure the serial fraction, and the speedup can never exceed one over that fraction. It is why engineers look for the serial bottleneck before adding cores or machines, and it frames most later arguments about parallel scaling.
A dynamic-programming decoder for convolutional codes that finds the most likely sequence of states.
The story
Before
Decoding a convolutional code sent over a noisy channel meant comparing the received signal with every possible transmitted sequence, whose number doubles with each extra bit. Sequential decoders were practical but their effort varied with the noise.
What happened, and who
Andrew Viterbi, then at the University of California, Los Angeles, published 'Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm' in IEEE Transactions on Information Theory, April 1967. It gave a decoder that keeps only the best path into each state at each step, so the work grows linearly with the message.
Why it mattered
The same recursion finds the most likely hidden state sequence in any hidden Markov model, so it decodes convolutional codes in mobile phones and modems and aligns sounds to words in speech recognition. It is also used to align sequences in bioinformatics.
The Clock (Second-Chance) Page Replacement Algorithm
An approximation of LRU using a circular list and a reference bit; described in 'A Paging Experiment with the Multics System'.
The story
Before
The best practical policy was to evict the least recently used page, but that meant recording every memory access in order, which hardware of the time could not afford. First-in first-out replacement was cheap but evicted pages still in heavy use.
What happened, and who
In 1968 Fernando Corbató at MIT reported 'A Paging Experiment with the Multics System', issued as MIT Project MAC memo MAC-M-384. Pages sit in a circular list, and a hand sweeps round clearing each page's reference bit and evicting the first page found with the bit already clear. A page used since the last sweep thus gets a second chance.
Why it mattered
One hardware-set bit per page is all the clock needs, and it comes close to LRU at a fraction of the cost. Operating systems still evict pages with variants of it, and the same idea is used in buffer pools and caches.
A small, fast buffer memory sits between the processor and main memory and is named the cache.
The story
Before
Main memory was many times slower than the processor, and a program that fetched every word from it waited for each one. Maurice Wilkes had proposed 'slave memories' in 1965, and a few research machines used such buffers, but no widely sold machine offered the idea as a standard feature.
What happened, and who
IBM announced the System/360 Model 85 on 30 January 1968, with a fast static-RAM buffer of 16 or 32 KB between the processor and main memory. IBM's engineers called it a cache, and J. S. Liptay described its design in the IBM Systems Journal in 1968. The hardware decided automatically which blocks of memory to hold there, with no help from the programmer.
Why it mattered
Programs tend to reuse recent addresses, so a small fast memory answers most requests and the machine behaves as if all memory were fast. The word and the idea spread from CPUs to disk buffers, operating-system page caches, databases and content delivery networks, and every modern processor has several levels of cache.
Organises an operating system as strict layers, each of which can be verified on its own.
The story
Before
Operating systems were large programs whose parts depended on each other in tangled ways. Testing them meant running the whole system and hoping the bugs showed up, and nobody could argue that one part was correct without the rest.
What happened, and who
Edsger W. Dijkstra and his group at the Technische Hogeschool Eindhoven built a small batch operating system for the Electrologica X8 computer, and he described it in 'The Structure of the THE-Multiprogramming System' in Communications of the ACM, May 1968. It was arranged in layers, from processor allocation at the bottom to the operator at the top, each using only the layers beneath it, and it used his semaphores to synchronise processes.
Why it mattered
Each layer could be tested completely before the next was built, which showed that an operating system could be designed for correctness, not only debugged into it. Layered structure, semaphores in a real kernel, and reasoning about a system one level at a time all entered mainstream practice from this paper.
Defines the pages a process needs in a recent window and links memory management to scheduling.
The story
Before
Scheduling and memory management were designed independently. A system admitted as many programs as it could, and when their pages together exceeded memory it spent its time swapping pages in and out and did almost no useful work, a collapse that had no theory to explain it.
What happened, and who
Peter Denning, then at MIT, published 'The Working Set Model for Program Behavior' in Communications of the ACM, May 1968. It defined a process's working set as the pages it referenced in its most recent window of time, and argued that the set is what the process needs in memory in order to run efficiently.
Why it mattered
The model explained thrashing and gave load control a rule: run a process only if its working set fits in memory, otherwise suspend it. That principle underlies how operating systems and databases limit concurrency under memory pressure, and it made locality of reference a precise idea rather than a folk observation.
Turns a regex into an NFA that is simulated in linear time; used in the QED editor and later grep.
The story
Before
Matching a pattern with alternatives and repetition was usually done by trying one choice and backing up on failure. Backtracking can take time exponential in the length of the input on unlucky patterns.
What happened, and who
Ken Thompson, then at Bell Labs, published 'Regular Expression Search Algorithm' in Communications of the ACM, June 1968. It turns a regular expression into a nondeterministic finite automaton and simulates all of its possible states at once as the text is read, so the work grows linearly with the input. He built it into the QED text editor, and the same idea later reached the Unix tool grep.
Why it mattered
The construction gives a guaranteed running time whatever the pattern, which backtracking matchers do not. Google's RE2 library reimplemented that guarantee to protect servers from regular-expression denial of service, and Thompson's construction is still how regular expressions are taught and compiled.
Dijkstra's loop plus an admissible heuristic guides the search toward the goal.
The story
Before
Dijkstra's shortest-path algorithm, from 1959, expands nodes in every direction in order of distance from the start. On a large graph it explores huge regions that lie away from the goal.
What happened, and who
Peter Hart, Nils Nilsson and Bertram Raphael at the Stanford Research Institute published 'A Formal Basis for the Heuristic Determination of Minimum Cost Paths' in IEEE Transactions on Systems Science and Cybernetics, July 1968. They were planning routes for the Shakey robot. The method orders nodes by the cost so far plus an estimate of the cost remaining, and they proved it finds a shortest path when the estimate never overstates the true cost.
Why it mattered
A good heuristic can cut the search to a thin corridor toward the goal without giving up optimality. A* and its variants find routes in maps and games, and plan motions for robots.
A hierarchical database first installed to manage the parts of the Apollo spacecraft.
The story
Before
Applications kept their records in their own files, each laid out by the program that used it. Tracking the bill of materials of something as large as the Saturn V rocket, with parts made of parts, needed one shared structure for many programs.
What happened, and who
IBM, North American Rockwell and Caterpillar Tractor began building it in 1966. The first system ran at North American Rockwell's Downey, California, site on 14 August 1968, keeping the bill of materials for the Saturn V rocket of the Apollo programme. IBM sold it commercially as IMS/360 from 1969.
Why it mattered
IMS stores data as trees of parent and child segments, and applications navigate them record by record. That is the hierarchical data model that Codd's relational model was written to replace, and IMS itself is still in use in large banks and airlines.
A small, general operating system written on a little-used minicomputer, after a big one proved too complex.
The story
Before
Bell Labs had been building Multics, an ambitious time-sharing system, with MIT and General Electric. Frustrated by its size and complexity, Bell Labs pulled out, and its researchers lost the interactive system they had been using.
What happened, and who
In 1969 Ken Thompson, with Dennis Ritchie and others of that group, wrote a new operating system on a little-used DEC PDP-7. It had a hierarchical file system, processes and a shell. In 1973 it was rewritten in C, against the belief of the time that an operating system had to be written in assembly language.
Why it mattered
Unix's few general ideas — everything is a file, a process makes another with fork, programs connect through pipes — became the shape of nearly every later operating system. Because it was written in C, it could be carried to new machines, and its interface became the POSIX standard that Linux and macOS follow today.
FIFO page replacement can fault more when given more frames.
The story
Before
It was taken for granted that giving a program more memory frames could only reduce, or at worst not change, its page faults. Replacement policies were compared on that footing.
What happened, and who
Laszlo Belady, R. A. Nelson and G. S. Shedler of IBM's Watson Research Center reported in Communications of the ACM, June 1969, in 'An Anomaly in Space-Time Characteristics of Certain Programs Running in a Paging Machine', cases where the opposite happened. Under first-in first-out replacement, some reference strings produce more page faults with more frames.
Why it mattered
The anomaly showed that a policy needs a stack property, meaning the pages held with n frames are always among those held with n plus one, for more memory to be safe. LRU and optimal replacement have it and FIFO does not, which is one reason LRU-style policies are the ones analysed and approximated.
Seven multiplications instead of eight for 2x2 blocks give O(n^2.81) matrix multiplication.
The story
Before
The schoolbook method multiplies two n by n matrices in about n3 steps, and it was widely believed that no method could do fundamentally better.
What happened, and who
Volker Strassen published 'Gaussian Elimination Is Not Optimal' in Numerische Mathematik in August 1969. He showed that two 2 by 2 blocks can be multiplied with seven multiplications instead of eight, and applying that recursively multiplies n by n matrices in about n2.81 steps.
Why it mattered
It showed that the obvious algorithm is not optimal and began the search for the true exponent of matrix multiplication, which is still open. Strassen-style methods are used in some numerical libraries for large matrices, but they need subtraction, which is why they do not carry over to shortest-path computations over the min-plus semiring.
The first message over the first large packet-switched network: two letters, "lo", before the receiving computer crashed.
The story
Before
Computers were islands. A researcher who needed another site's machine or programs had to travel there or carry tapes. Paul Baran at RAND and Donald Davies at the UK's National Physical Laboratory had each proposed breaking messages into small independently routed blocks — Davies named them packets — but no large network had been built on the idea.
What happened, and who
At 10:30 pm on 29 October 1969, UCLA student programmer Charley Kline, in Leonard Kleinrock's laboratory, tried to log in from UCLA's SDS Sigma 7 to the SDS 940 at the Stanford Research Institute, where Bill Duvall was watching. The SRI machine crashed after two characters: the first message over the ARPANET was "lo". The full login worked after a fix.
Why it mattered
The ARPANET, funded by the US Defense Department's Advanced Research Projects Agency to let research computers share resources, was the first large packet-switched network. Its protocols and people led directly to TCP/IP and the internet.
Copies live data into a fresh half-space; described in 'A LISP garbage-collector for virtual-memory computer systems'.
The story
Before
Mark-sweep collectors marked live objects in place and swept the rest into a free list. That left free space fragmented and live objects scattered across memory, which was costly on a machine with virtual memory.
What happened, and who
Robert Fenichel and Jerome Yochelson of MIT described 'A LISP Garbage-Collector for Virtual-Memory Computer Systems' in Communications of the ACM, November 1969. Memory is split into two halves, and the collector copies every reachable object from the active half into the other, which then becomes the active half.
Why it mattered
After copying, live data is packed together and free space is one contiguous block, so allocation is just moving a pointer. Copying is also the basis of generational and compacting collectors in managed runtimes today.
Unix, relational databases, the microprocessor and public-key cryptography.
PaperAlgorithms
Dinic's Max-Flow Algorithm
Level graphs and blocking flows give a maximum-flow algorithm running in O(V^2 E).
The story
Before
The Ford-Fulkerson method finds a maximum flow by repeatedly pushing along augmenting paths, but with badly chosen paths its running time depends on the capacities and it may not terminate at all for irrational ones. No bound depending on the graph's size alone was known for it.
What happened, and who
Yefim Dinitz, then a master's student in Moscow, conceived the algorithm in January 1969 and published it in 1970 in Doklady Akademii Nauk SSSR. It builds a level graph of shortest-path distances from the source and pushes a blocking flow through it, then repeats, and the running time is O(V2E).
Why it mattered
The bound depends only on the graph, and it beats the Edmonds-Karp algorithm, found independently in the West two years later. On unit-capacity graphs the method is fast enough to give the Hopcroft-Karp bipartite matching algorithm as a special case, and it is a common baseline in flow solvers and programming contests.
Adding a positive constant to the diagonal of the normal equations gives a stable, biased estimator.
The story
Before
Ordinary least squares fits a line by solving the normal equations. When predictors are strongly correlated, that system is nearly singular and the fitted coefficients swing wildly with small changes in the data.
What happened, and who
Arthur Hoerl and Robert Kennard published two papers in Technometrics 12(1), February 1970, 'Ridge Regression: Biased Estimation for Nonorthogonal Problems' and 'Ridge Regression: Applications to Nonorthogonal Problems'. They added a positive constant to the diagonal of the normal equations, which makes the solution stable at the price of a small bias.
Why it mattered
Ridge is the prototype of regularisation: accept a little bias to cut the variance of the fit. The same penalty on the size of the weights, under the name weight decay, is used to train neural networks, and it is the standard treatment of collinear features.
A four-bit ALU slice sold as a catalogue part, the first ALU on a single chip.
The story
Before
The arithmetic logic unit of a computer, the part that adds and performs logical operations, was assembled from many separate gate chips spread across several circuit boards. Every designer built their own.
What happened, and who
Texas Instruments introduced the 74181 in February 1970 as a standard catalogue part in its 7400 series. It is a four-bit slice that performs 16 arithmetic and 16 logical operations on two four-bit words, and slices can be chained for wider words.
Why it mattered
A designer could now buy the ALU rather than build it, which made minicomputers with 16 or more bits cheap to design. The Data General Nova 1200 of 1970, the PDP-11 family, the Xerox Alto and the VAX-11/780 used it, and it marks the step between boards of small gates and the single-chip microprocessor.
Adds the proposal ratio to the acceptance rule, allowing asymmetric proposals in Markov chain Monte Carlo.
The story
Before
Metropolis and colleagues had published in 1953 a way to sample from a distribution by a random walk that accepts or rejects moves. It required the proposal to be symmetric, so that proposing a move from A to B is as likely as proposing B to A, which limited how the walk could be designed.
What happened, and who
W. K. Hastings, then at the University of Toronto, generalised the method in 'Monte Carlo Sampling Methods Using Markov Chains and Their Applications', Biometrika 57(1), April 1970. He multiplied the acceptance ratio by the ratio of proposal probabilities, so any proposal can be used. The paper also discusses assessing the error in the resulting estimates.
Why it mattered
With that correction, Markov chain Monte Carlo can draw samples from distributions whose normalising constant is unknown, which is the situation in Bayesian statistics. Metropolis-Hastings is the general recipe behind samplers in packages such as Stan and PyMC.
Data as tables, and queries as mathematics rather than pointers a program walks.
The story
Before
Databases were hierarchical, like IBM's IMS, or networks of records, like the CODASYL systems. A program found data by following stored pointers from record to record, so it depended on how the data was laid out: change the storage structure and the programs broke.
What happened, and who
Edgar F. Codd of IBM's San Jose research laboratory published "A Relational Model of Data for Large Shared Data Banks" in Communications of the ACM in June 1970. Data is a set of relations — tables of rows — and a query is an expression over them that says what is wanted, not how to fetch it.
Why it mattered
Codd's data independence let the storage change without touching programs, and a declarative query left the "how" to the system — which is why query optimisers exist. SQL, and every relational database from System R to PostgreSQL, descends from this paper. Codd received the Turing Award in 1981.
Balanced multiway trees with at least 50% node occupancy, built for large ordered indexes on disk.
The story
Before
Balanced binary trees keep search fast in memory, but on disk each level costs a separate access, so a large index needed far too many. Sequential indexed files such as ISAM avoided that but needed periodic reorganisation as records were added.
What happened, and who
Rudolf Bayer and Edward McCreight, at Boeing Scientific Research Laboratories, circulated 'Organization and Maintenance of Large Ordered Indices' as a Boeing report in July 1970. It appeared in Acta Informatica in 1972. Each node holds many keys and fills a disk page, every node except the root is at least half full, and inserts and deletes split and merge nodes so the tree stays balanced.
Why it mattered
A tree with hundreds of keys per node is only a few levels deep, so a lookup in a huge index costs a handful of disk reads. The B-tree, and its B+tree variant, is the default index of relational databases and of many file systems. The authors never said what the 'B' stands for.
A space-efficient hash structure that trades a controlled error rate for memory; the paper is 'Space/time trade-offs in hash coding with allowable errors' (CACM 13(7)).
The story
Before
Testing whether an item was in a set meant storing the items, or at least a code for each. Bloom's motivating example was a hyphenation program whose dictionary of exceptions was too large to hold in memory, while a disk lookup for every word was too slow.
What happened, and who
Burton H. Bloom of the Computer Usage Company published 'Space/Time Trade-offs in Hash Coding with Allowable Errors' in Communications of the ACM 13(7), July 1970. Each item sets several bits in a bit array chosen by hash functions, and a query reports 'present' only if all its bits are set. It may wrongly say yes for an item never added, but never wrongly says no.
Why it mattered
A few bits per item are enough to rule out most absent keys, so a cheap in-memory check avoids many expensive lookups. Log-structured databases use Bloom filters to skip files that cannot hold a key, and web crawlers and caches use them to remember what they have already seen.
Random-access packet radio: terminals transmit whenever they have a frame and retransmit after a random delay on collision.
The story
Before
A shared channel was normally divided in advance, by polling each terminal in turn or by fixed slices of time or frequency. Most terminals send data rarely and in bursts, so most of the reserved capacity sat idle.
What happened, and who
Norman Abramson, at the University of Hawaii, described the system in 'The ALOHA System: Another Alternative for Computer Communications' at the AFIPS Fall Joint Computer Conference, 17 to 19 November 1970. Work had begun in September 1968, and ALOHAnet became operational in June 1971, linking terminals on the islands to a central computer by radio. A terminal sends a frame whenever it has one; if no acknowledgement comes back, it waits a random time and sends again.
Why it mattered
Random access with retransmission showed that a shared channel needs no central scheduler, at the cost of collisions that limit pure ALOHA to about 18 per cent of the channel. Robert Metcalfe's Ethernet, with the addition of listening before transmitting, descends from it, and its slotted variant is the ancestor of the contention schemes in Wi-Fi and cellular access.
Replaces the recursive copy with a scan pointer sweeping the to-space; the paper is 'A nonrecursive list compacting algorithm'.
The story
Before
The Fenichel-Yochelson collector of 1969 copied objects by following pointers recursively. A long chain of objects could make the recursion deeper than any stack the collector could count on, and a collector must not run out of memory in the middle of freeing memory.
What happened, and who
C. J. Cheney published 'A Nonrecursive List Compacting Algorithm' in Communications of the ACM, November 1970. Two pointers move through the new half-space: one marks where the next object will be copied, the other scans the objects already copied and copies whatever they point to. The scan finishes when the two meet.
Why it mattered
The collector needs no stack and no extra memory beyond the two half-spaces, because the copied objects themselves serve as the work queue. Cheney's breadth-first scan is still the usual way a copying collector is written, including the young-generation collectors of managed runtimes.
The optimal root is monotone, which cuts the interval dynamic program from cubic to quadratic time.
The story
Before
Building the binary search tree with the lowest expected search cost, given how often each key is looked up, used a dynamic program over intervals of keys. Gilbert and Moore's 1958 algorithm tried every possible root for every interval and took time proportional to n3.
What happened, and who
Donald Knuth published 'Optimum Binary Search Trees' in Acta Informatica in 1971. He showed that the best root of an interval lies between the best roots of the two intervals obtained by dropping its first or last key, so each interval needs to try only a few roots. The whole computation then takes time proportional to n2.
Why it mattered
The trick of restricting the search for an optimal split point became a general technique, named the Knuth optimisation and later extended by the quadrangle inequality to many interval and partition recurrences. The lesson is that structure in the choices of a dynamic program can remove a whole factor from its running time.
A PROM that can be erased with ultraviolet light and reprogrammed, first sold as the Intel 1702.
The story
Before
A programmable read-only memory could be written once, by the customer, and never changed. A mistake in the program, or a new version of it, meant throwing the chip away.
What happened, and who
Dov Frohman at Intel invented the erasable programmable read-only memory in 1971, and the Intel 1702, a 2048-bit EPROM, was announced that year. Each cell is a transistor with a floating gate that holds charge, and ultraviolet light shining through a quartz window on the package frees the charge and erases the chip. Frohman's patent, 3,660,819, was granted in 1972.
Why it mattered
For the first time firmware could be revised without replacing the chip, which made development cycles for microprocessor systems much shorter. The floating-gate cell it introduced is the basis of EEPROM and flash memory, which store data in phones, SSDs and most devices with firmware.
A single abstract model (domains x objects -> rights) unifies capability lists, rings and access lists.
The story
Before
Each time-sharing system described its protection in its own vocabulary, with its own mechanisms for keeping users' files and processes apart. It was hard to compare them, or to say what any of them guaranteed.
What happened, and who
Butler Lampson presented 'Protection' at the Fifth Princeton Conference on Information Sciences and Systems, Princeton University, in March 1971 (pages 437 to 443 of the proceedings). It models protection with three parts: a set of domains, a set of objects, and an access matrix whose entry gives the rights a domain has over an object. Graham and Denning later refined the model.
Why it mattered
Reading the matrix by column gives an access-control list kept with each object, and reading it by row gives a capability list held by each domain. That is the conceptual root of the choice between ACLs and capabilities, and of most later formal work on what a protection system can guarantee.
Every problem in NP reduces to Boolean satisfiability; the paper is 'The Complexity of Theorem-Proving Procedures'.
The story
Before
The class NP, problems whose solutions can be checked quickly, had been defined, and the question of whether such problems can also be solved quickly was open. There was no specific problem that could stand for the whole class.
What happened, and who
Stephen Cook of the University of Toronto presented 'The Complexity of Theorem-Proving Procedures' at the third ACM Symposium on Theory of Computing (STOC) in May 1971, in the conference's third annual meeting. He proved that any problem in NP can be turned in polynomial time into a question of whether a Boolean formula can be satisfied. Leonid Levin reached an equivalent result independently in the Soviet Union, published in 1973.
Why it mattered
If satisfiability can be solved quickly, so can everything in NP, and if any problem in NP is hard, satisfiability is. Richard Karp's 1972 paper showed 21 other problems to be equally hard, and proving a problem NP-complete became the standard way to say that no fast general solution is expected.
Isolates the four conditions that are necessary for deadlock.
The story
Before
Deadlock, which Dijkstra had called the 'deadly embrace' in the mid-1960s, was known to stop systems dead when processes each held a resource another needed. Each system handled it in its own way, and there was no shared statement of what made it possible.
What happened, and who
E. G. Coffman, M. J. Elphick and A. Shoshani published the survey 'System Deadlocks' in ACM Computing Surveys 3(2), June 1971. It states that deadlock needs four conditions at once: mutual exclusion, holding a resource while waiting for another, no forced removal of resources, and a circular chain of waiting processes.
Why it mattered
Since deadlock needs all four, breaking any one prevents it, which organises the standard remedies: order the resources, acquire them all at once, allow pre-emption, or detect and recover. The four conditions, now called Coffman's conditions, are still how deadlock is taught and how lock-ordering rules are justified in real code.
Two solutions with opposite starvation trade-offs; the paper is 'Concurrent Control with Readers and Writers' (CACM).
The story
Before
A lock let one process at a time into a shared data structure. That is safe, but it forces readers, who change nothing, to queue behind each other.
What happened, and who
P.-J. Courtois, F. Heymans and D. L. Parnas published 'Concurrent Control with "Readers" and "Writers"' in Communications of the ACM, October 1971. They posed the problem of letting many readers share a structure while a writer has it alone, and gave two solutions built from semaphores. One favours readers and can starve writers; the other favours writers and can starve readers.
Why it mattered
The paper made the fairness-versus-throughput choice explicit, and it is the origin of reader-writer locks in operating systems, language libraries and databases. Choosing between the two policies is a decision engineers still face when a workload mostly reads.
The first commercial microprocessor: a whole processor on one chip.
The story
Before
A processor was assembled from many chips, often across several circuit boards. Even electronic calculators needed a set of custom chips designed for each model.
What happened, and who
Intel announced the 4004 on 15 November 1971. Federico Faggin, Marcian Hoff, Stanley Mazor and Masatoshi Shima designed it after the Japanese firm Busicom asked Intel in 1969 for a chip set for calculators. It is a 4-bit processor with about 2,300 transistors running at roughly 740 kHz.
Why it mattered
One programmable chip could replace a board of logic, which put a processor into calculators, cash registers and machines of every kind. The 4004's successors, the 8008 and 8080, led to the 8086 and the processors in today's computers.
The TENEX paper describes 'a unique copy-on-write facility' for the PDP-10.
The story
Before
Giving a process its own copy of memory meant copying all of it up front, even if the process would read most of it and change little. That made creating a process, or sharing a file mapping safely, slow and wasteful.
What happened, and who
Daniel Bobrow, Jerry Burchfiel, Daniel Murphy and Raymond Tomlinson of Bolt, Beranek and Newman described TENEX, a paged time-sharing system for the DEC PDP-10, in Communications of the ACM, March 1972. The paper describes a copy-on-write facility: a page is shared until a process writes to it, and only then is a private copy made.
Why it mattered
Copying only when a write happens makes the common case free. Copy-on-write is how fork stays cheap in Unix-like systems, and it underlies snapshots in file systems and databases and lazy copying of memory in many runtimes.
Augmenting along shortest paths bounds maximum flow at O(VE^2); the same paper introduces the reweighting later used by Johnson.
The story
Before
The Ford-Fulkerson method of 1956 to 1962 leaves the choice of augmenting path open. With unlucky choices it can take a number of steps that depends on the capacities rather than on the size of the graph, and it can fail to finish on irrational capacities.
What happened, and who
Jack Edmonds and Richard Karp published 'Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems' in the Journal of the ACM 19(2), April 1972. They showed that always augmenting along a path with the fewest edges takes O(VE) augmentations, for a total of O(VE2) time. The paper also introduced the technique of reweighting edges by vertex potentials, which Johnson's algorithm later used.
Why it mattered
This was a polynomial bound that depends only on the number of vertices and edges, not on the capacities. Breadth-first search for the augmenting path is a simple rule that is still the usual first implementation of maximum flow.
Low-link values in one depth-first search find strongly connected components and articulation points.
The story
Before
Finding the strongly connected components of a directed graph, or the points whose removal disconnects a graph, needed several passes over the graph, or more than linear time.
What happened, and who
Robert Tarjan published 'Depth-First Search and Linear Graph Algorithms' in the SIAM Journal on Computing 1(2), June 1972. In a single depth-first search, each vertex records the earliest ancestor its subtree can reach, called its low-link value, and comparing that with its own discovery time reveals components and cut points. The running time is linear in the number of vertices plus edges.
Why it mattered
A linear-time algorithm made these structural questions cheap enough to ask of any graph. Strongly connected components are used to order dependencies in build tools and package solvers and to analyse programs in compilers, and finding bridges and articulation points shows where a network has a single point of failure.
Records move to the leaves and the leaves are linked, giving efficient range scans.
The story
Before
In Bayer and McCreight's B-tree, records sit in internal nodes as well as leaves. Internal nodes spend space on records rather than keys, which lowers fanout, and a scan in key order keeps climbing back up the tree.
What happened, and who
The variant that keeps all records in the leaves and links the leaves has no single originating paper. Douglas Comer's survey 'The Ubiquitous B-Tree', in ACM Computing Surveys in June 1979, notes that IBM's VSAM file-access software used it, citing an IBM article of 1973. Comer's survey treated the B+tree in detail and is where many readers first met the name.
Why it mattered
Internal nodes hold only keys, so more fit on a page and the tree is shallower, and a range scan reads along the linked leaves without going back up. The B+tree is the index structure of PostgreSQL, InnoDB and most other relational databases.
Simple utilisation tests that say whether periodic tasks under fixed-priority or earliest-deadline-first scheduling will always meet their deadlines.
The story
Before
Real-time systems, such as those controlling aircraft or industrial plant, were scheduled with priorities chosen by hand and checked by testing. Nobody could prove that every deadline would be met, so a system that passed its tests could still miss one under a rare alignment of tasks.
What happened, and who
C. L. Liu and James Layland published "Scheduling Algorithms for Multiprogramming in a Hard-Real-Time Environment" in the Journal of the ACM, volume 20 issue 1, in January 1973. For periodic tasks they showed that giving shorter-period tasks higher fixed priorities (rate-monotonic) is the best fixed-priority rule, with a guaranteed utilisation bound that falls towards about 0.69 as tasks are added. Running the task with the nearest deadline first (EDF) meets every deadline whenever total utilisation is at most 1.
Why it mattered
It founded real-time scheduling theory: engineers can now check a schedule by arithmetic instead of by hoping. Rate-monotonic analysis is a standard tool for real-time systems, and Linux's SCHED_DEADLINE class is an EDF scheduler that rests on the utilisation result.
A system call that feeds one program's output straight into another's input, letting small tools be chained; reportedly added in a single night.
The story
Before
The Second Edition of Unix, from June 1972, had no way to connect two programs except through a file: run one, save its output to a temporary file, run the next on that file. Doug McIlroy had argued since a 1964 memo that programs should be connectable like garden hoses, but nothing implemented it.
What happened, and who
Urged on by McIlroy, Ken Thompson added the pipe system call and shell support for it to Unix at Bell Labs, reportedly in one night. By 15 January 1973 the feature existed: McIlroy's announcement of a talk that day listed the pipe call and its implementation. It appears in the Third Edition, and the connecting notation, first written with the redirection characters, was later replaced by the | symbol.
Why it mattered
Pipes made small single-purpose programs composable, which is the working style people associate with Unix. Every shell pipeline, and the streams of data flowing between processes in build tools and log processing, descends from this idea.
Measurements show real systems failing on asynchronous inputs.
The story
Before
Engineers who saw a system fail now and then when an outside signal arrived at an awkward moment blamed glitches or noise. It was widely believed that waiting long enough for a flip-flop to settle would make the failure impossible.
What happened, and who
Thomas Chaney of Washington University in St. Louis and Charles Molnar published 'Anomalous Behavior of Synchronizer and Arbiter Circuits' in IEEE Transactions on Computers, April 1973. They reported oscillation and metastable states in flip-flops that receive an input changing at the wrong moment relative to the clock, and noted that significant system failures had resulted.
Why it mattered
A flip-flop caught between its two states can stay undecided for an unbounded time, so no synchroniser is perfectly safe. All a designer can do is wait long enough that failure becomes astronomically unlikely, which is why every clock-domain crossing in a chip uses a chain of flip-flops, sized to a target mean time between failures.
A memo sketching a network in which computers share one cable and listen before they send, the design that became Ethernet.
The story
Before
ALOHAnet, a radio network built by Norman Abramson at the University of Hawaii in the late 1960s, let each station transmit whenever it had data, without listening first. Transmissions that overlapped were lost and had to be repeated, so a busy channel spent much of its capacity on collisions.
What happened, and who
On 22 May 1973 Robert Metcalfe, at Xerox PARC in Palo Alto, wrote a 12-page memo describing a local network of computers on a shared coaxial cable, adapting ALOHAnet's approach to a wire. Working with David Boggs he built it, and the first network ran on 11 November 1973 at 2.94 Mbit/s. The design is published in Metcalfe and Boggs's 1976 paper in Communications of the ACM, which describes listening before sending and detecting collisions.
Why it mattered
Ethernet became the dominant wired local network and its frame format and shared-medium ideas carried into switched Ethernet, which connects almost every server and desktop today. Its listen-then-back-off rule is the standard example of how a shared channel can be used without central control.
Treats every computational object as an actor that owns its state and communicates only by sending messages.
The story
Before
Concurrent programs were described with several unrelated ideas: shared memory, semaphores and processes as separate concepts, each needing its own reasoning. There was no single small model that covered objects, procedures and processes together.
What happened, and who
Carl Hewitt, Peter Bishop and Richard Steiger presented "A Universal Modular ACTOR Formalism for Artificial Intelligence" at the International Joint Conference on Artificial Intelligence (IJCAI) in August 1973. In it every computational entity is an actor that, on receiving a message, can send messages to actors it knows, create new actors, and decide how to handle the next message.
Why it mattered
Actors gave a clean model of concurrency without shared mutable state. Erlang's processes and the Akka library implement the same idea, Akka by name, and the message-passing style now underlies many actor-based and event-driven systems.
Finding many shortest augmenting paths at once makes maximum bipartite matching much faster than adding one path at a time.
The story
Before
The standard method for maximum bipartite matching repeatedly found one augmenting path, a path that lets the matching grow by one pair, using a search that costs O(E) each time. With up to V pairs to add, the total was O(VE), which was slow for large graphs.
What happened, and who
John Hopcroft and Richard Karp published "An n^5/2 algorithm for maximum matchings in bipartite graphs" in the SIAM Journal on Computing in December 1973. Their algorithm works in phases: each phase finds a maximal set of vertex-disjoint shortest augmenting paths in one pass, and they showed that only O(√V) phases are needed.
Why it mattered
The bound O(E√V) is a large saving over O(VE) on big sparse graphs, and it is still the algorithm taught and used for large bipartite matching and assignment problems. The idea of batching all shortest augmenting paths in a phase also became a pattern for other flow algorithms.
The first widely used operating system for microcomputers, whose system call was an ordinary CALL.
The story
Before
Software for early microprocessor computers was written for one machine at a time, with no common way for a program to keep files on a disk.
What happened, and who
Gary Kildall demonstrated the first working prototype of CP/M (Control Program for Microcomputers) in Pacific Grove, California, in 1974, for the Intel 8080. The part that depended on the hardware, the BIOS, was kept separate, so CP/M could be adapted to many machines. A program asked CP/M for a service by putting a function number in register C and executing an ordinary CALL to address 0005H.
Why it mattered
CP/M gave many different 8080 machines one platform for software. It also shows what an operating system without hardware support is: nothing stopped a program from overwriting it, calling into the middle of it, or keeping the processor forever.
A table-filling dynamic program computes the fewest insertions, deletions and substitutions that turn one string into another.
The story
Before
Vladimir Levenshtein had defined the edit distance in 1965, but comparing strings efficiently was done in scattered, field-specific ways. Speech researchers (Taras Vintsyuk, 1968) and biologists aligning sequences (Needleman and Wunsch, 1970) had found the same dynamic program without knowing of each other's work.
What happened, and who
Robert Wagner and Michael Fischer published "The String-to-String Correction Problem" in the Journal of the ACM in January 1974. It presents the dynamic program that fills a table of distances between prefixes of the two strings, in time proportional to the product of their lengths.
Why it mattered
The table-filling method is how edit distance is computed today. It underlies spell-checkers' suggestions, file comparison, fuzzy search and the sequence alignment used in bioinformatics.
An 8-bit processor on one chip, the ancestor of x86 — and a machine with no protection at all.
The story
Before
Intel's first single-chip processor, the 4-bit 4004, was built for calculators and controllers rather than general-purpose computers.
What happened, and who
Intel released the 8080 in April 1974, in a design led by Federico Faggin with Masatoshi Shima. It is an 8-bit processor with seven 8-bit registers, 16-bit addresses that reach 65,536 bytes of memory, and an original clock rate of 2 MHz. It has no privileged mode, no memory protection and no timer.
Why it mattered
The 8080 ran early personal computers such as the Altair 8800 and the CP/M operating system, and the 8086 that followed it was designed so that 8080 programs could be reassembled for it. Because it has none of the hardware an operating system needs to keep programs apart, it is the plainest real example of the bare machine operating systems are built to fix.
A query language written in structured English keywords, so that non-specialists could ask questions of relational data; it became SQL.
The story
Before
Edgar Codd's 1970 relational model came with two mathematical query notations, relational algebra and relational calculus. Both were precise, but neither was something working programmers or business users could be expected to write.
What happened, and who
Donald Chamberlin and Raymond Boyce at IBM's San Jose research laboratory presented "SEQUEL: A Structured English Query Language" at the ACM SIGFIDET workshop in May 1974. It expressed queries with keywords such as SELECT, FROM and WHERE, and was built for IBM's System R relational database prototype.
Why it mattered
The language was later renamed SQL, reportedly because SEQUEL was already a trademark, and it became the standard interface to relational databases. Oracle shipped the first commercial SQL product in 1979, and SQL was standardised in the 1980s; it is still how most applications talk to databases.
A condition on a processor's instructions that decides whether a virtual machine monitor can run guests by trapping and emulating.
The story
Before
IBM's CP-67, running on the System/360 Model 67 from 1967, showed that virtual machines worked. But there was no theory saying which processor designs could support them efficiently and safely, so each design was a matter of trial.
What happened, and who
Gerald Popek and Robert Goldberg published "Formal requirements for virtualizable third generation architectures" in Communications of the ACM in July 1974. They separated privileged instructions, which trap in user mode, from sensitive ones, which touch or depend on machine state, and proved that a trap-and-emulate monitor can be built if every sensitive instruction is privileged.
Why it mattered
The condition explained why the x86 architecture, which had sensitive instructions that did not trap, was hard to virtualise until binary translation in the late 1990s and then the hardware extensions Intel and AMD added in the mid-2000s. It still frames how virtual machine monitors are designed.
The journal paper that described Unix's file system, device files, mounting and inodes to a wide audience.
The story
Before
Unix, written at Bell Labs for the PDP-11 from 1969, was known inside Bell Labs and to a few outside groups. Its design had been shown in talks but was not available as a widely read description.
What happened, and who
Dennis Ritchie and Ken Thompson published "The UNIX Time-Sharing System" in Communications of the ACM in July 1974. It describes the hierarchical file system, files as sequences of bytes, devices treated as files, the mount operation and the i-node structure, and the small size of the system.
Why it mattered
The paper brought Unix's design to the wider computing community, and universities and companies began to license and imitate it. Its ideas, especially the uniform file abstraction and devices as files, remain the basis of Linux, macOS and BSD.
Mutual exclusion for any number of processes using only ordinary reads and writes and numbered tickets, like a bakery queue.
The story
Before
Dekker's algorithm solved mutual exclusion for two processes, and Edsger Dijkstra's 1965 solution for N processes was intricate. Early N-process solutions could let one process be overtaken by others many times, and they relied on variables that every process wrote.
What happened, and who
Leslie Lamport published "A New Solution of Dijkstra's Concurrent Programming Problem" in Communications of the ACM in August 1974. Each process picks a numbered ticket larger than any it sees and waits for lower tickets to be served, breaking ties by process number; each variable is written by one process only.
Why it mattered
It showed that mutual exclusion needs no special atomic instruction, only reads and writes. The ticket idea recurs in ticket spinlocks that serve waiters in order, and the algorithm is a standard case study in reasoning about concurrency.
Shrinking a transistor's dimensions and voltage by the same factor keeps power per unit area constant while making it faster.
The story
Before
Chip makers knew smaller transistors would be faster and denser, but shrinking one dimension alone made devices leak, break down or run hot. There was no recipe saying how all the dimensions and the voltage should change together.
What happened, and who
In October 1974 Robert Dennard, Fritz Gaensslen and four colleagues at IBM published "Design of ion-implanted MOSFET's with very small physical dimensions" in the IEEE Journal of Solid-State Circuits. They showed that scaling length, width, oxide thickness and voltage by one factor, and raising doping by its inverse, keeps the electric fields the same, so the power density stays constant while each transistor gets faster.
Why it mattered
The rule let each generation of chips carry more transistors at higher clock rates without melting, which drove decades of speed gains from process shrinks alone. It broke down around 2005–2007 when leakage stopped scaling with size, which is why clock speeds stalled and processors moved to multiple cores.
Bundles shared data with the only code allowed to touch it, and lets one thread enter at a time.
The story
Before
Semaphores, introduced by Edsger Dijkstra in 1965, could be called from anywhere in a program. A forgotten or misplaced wait or signal caused subtle errors, and the correctness of a shared structure could not be judged from one place.
What happened, and who
C. A. R. (Tony) Hoare published "Monitors: An Operating System Structuring Concept" in Communications of the ACM in October 1974. A monitor collects shared data and the procedures that use it, admits one caller at a time, and uses condition variables for waiting. Per Brinch Hansen's Concurrent Pascal (1975) was the first language to implement it.
Why it mattered
Monitors put the locking next to the data it protects. Java's synchronized methods with wait and notify, and the condition variables of POSIX threads, descend from them.
A processor with no microcode and a small set of simple instructions, built to run faster: the first modern RISC design.
The story
Before
Mainframe processors such as IBM's System/370 offered many complex, variable-length instructions, many implemented in microcode. This made the hardware complicated, and measurements of real programs showed that a handful of simple instructions did most of the work.
What happened, and who
IBM's 801 project began in 1974 at the Thomas J. Watson Research Center, led by John Cocke, as a processor for a telephone switch. The switch project was cancelled in 1975, and in October 1975 IBM chose to continue the processor as a general-purpose design. It used only registers for arithmetic, with separate loads and stores, and a working machine ran by summer 1980.
Why it mattered
The 801 showed that removing complexity could improve speed if compilers made good use of many registers. The term RISC came later from David Patterson's group at Berkeley, but the 801's ideas fed IBM's POWER family and shaped the load/store designs that followed.
When copies of a database receive updates in different orders, the update with the latest timestamp wins everywhere.
The story
Before
When a database was copied to several ARPANET hosts and updated at each, copies could receive the same changes in different orders and end up different. There was no agreed rule for deciding which change should be final.
What happened, and who
Paul Johnson and Robert Thomas, both at BBN, wrote RFC 677, "The Maintenance of Duplicate Databases", dated 27 January 1975. Each modification carries a timestamp combined with the identity of the process that made it, and every copy applies the modification with the latest stamp, so all copies converge to the same state.
Why it mattered
This is the origin of last-write-wins, still the default conflict rule in systems such as Cassandra, and the same idea Thomas later formalised as the Thomas write rule. Its dependence on clocks, and the question of how to order events without one, led to Lamport's logical clocks of 1978.
One automaton with failure links searches for a whole dictionary of patterns in a single pass over the text.
The story
Before
Searching for several keywords meant running a single-pattern search once per keyword, so the cost grew with the number of patterns. Linear-time single-pattern methods such as Knuth-Morris-Pratt did not by themselves help with a dictionary.
What happened, and who
Alfred Aho and Margaret Corasick at Bell Labs published "Efficient string matching: an aid to bibliographic search" in Communications of the ACM in June 1975. It builds a trie of all keywords, adds failure links that say where to resume after a mismatch, and scans the text once, reporting every match. The authors used it to speed up the Unix fgrep command.
Why it mattered
The running time depends on the text length and the number of matches, not on how many patterns there are. Variants are used in fgrep, virus scanners such as ClamAV and network intrusion detection systems such as Snort.
Names eight principles for secure design, among them least privilege, fail-safe defaults and complete mediation.
The story
Before
Experience with protecting information in shared systems such as Multics, built at MIT, was scattered across papers and word of mouth. Designers lacked a common vocabulary for judging whether a protection mechanism was well designed.
What happened, and who
Jerome Saltzer and Michael Schroeder, both of MIT, published "The Protection of Information in Computer Systems" in the Proceedings of the IEEE in September 1975. It surveys protection mechanisms, including access lists and capabilities, and lists design principles: economy of mechanism, fail-safe defaults, complete mediation, open design, separation of privilege, least privilege, least common mechanism and psychological acceptability.
Why it mattered
The principles are still quoted in security design reviews. Least privilege shapes permission systems such as cloud IAM policies, and fail-safe defaults and complete mediation are checked whenever a new access path is added.
Lets a database lock a whole table, a page or a single row, and lets those locks coexist safely.
The story
Before
Database systems locked at one fixed granularity. Locking whole files was simple but blocked unrelated work, while locking single records allowed concurrency but forced a transaction touching many records to take a very large number of locks.
What happened, and who
Jim Gray, Raymond Lorie, Gianfranco Putzolu and Irving Traiger of IBM Research published "Granularity of Locks in a Shared Data Base" at the VLDB conference in September 1975. They arranged lockable items in a hierarchy and added intention modes (IS, IX and SIX) that a transaction places on the ancestors of the item it locks, so that a lock on a large item can be checked without inspecting every item inside.
Why it mattered
Every database lock manager works this way: a transaction can take a row lock cheaply and a table lock when it touches most of the table. The intention modes are visible in the lock tables of MySQL's InnoDB and of other mainstream relational databases.
Argues that a kernel should provide the mechanisms and leave the policies that use them to user-level code.
The story
Before
Operating systems of the time built scheduling and protection policies into the kernel. Changing a policy meant changing the kernel, and one system could not serve users who needed different policies.
What happened, and who
Roy Levin, Ellis Cohen, William Corwin, Fred Pollack and William Wulf, at Carnegie Mellon University, presented "Policy/Mechanism Separation in Hydra" at the Symposium on Operating Systems Principles in November 1975. Hydra was an operating system for the C.mmp multiprocessor, and the paper describes how it kept the mechanisms fixed while policies were supplied outside the kernel.
Why it mattered
The distinction became a standard design principle in operating systems and beyond. It shaped microkernels, pluggable schedulers and security modules, and it is a routine question in design reviews: which part is the mechanism and which the policy.
Several machines commit or abort a transaction together by first agreeing to be ready and only then committing.
The story
Before
A transaction that updated data on several machines could succeed on some and fail on others, for example after a crash between two commits. There was no protocol guaranteeing that all of them would commit or none would.
What happened, and who
Butler Lampson and Howard Sturgis at Xerox PARC described the protocol in a 1976 report, "Crash recovery in a distributed data storage system". Jim Gray at IBM worked out the same protocol independently and spread it through his 1978 notes on database operating systems. In the first phase every participant records that it is prepared to commit; in the second the coordinator tells all to commit if all were prepared, or to abort.
Why it mattered
It is the basis of distributed commit in database systems and of the XA standard for transactions across resource managers. Google's Spanner uses it to commit across groups of replicas. Its weakness is that participants that have prepared must wait if the coordinator fails, which later protocols address.
A supercomputer whose vector instructions each operate on 64 numbers at once, running at 80 MHz.
The story
Before
Scientific computers issued one arithmetic instruction per pair of numbers, so long loops over arrays spent much of their time fetching and decoding instructions. Machines struggled to keep their arithmetic units fed with operands.
What happened, and who
Seymour Cray's company, Cray Research, delivered the first Cray-1 (serial number 001) to Los Alamos National Laboratory in March 1976 for a six-month trial. It had a 12.5 ns clock, eight vector registers of 64 elements each, and pipelined arithmetic units that a single vector instruction kept busy across a whole register.
Why it mattered
The Cray-1 made the vector model the way to reach high floating-point speed for array code. Modern SIMD extensions such as AVX, Arm's SVE and RISC-V's vector extension carry the same idea to ordinary processors.
A diagram notation of entities and relationships for designing a database before deciding how to store it.
The story
Before
Designers thought in terms of one of the network, relational or record-based data models directly, and each biased what they noticed. There was no neutral way to describe what a business needed to store before choosing a model.
What happened, and who
Peter Chen, at MIT's Sloan School of Management, published "The Entity-Relationship Model: Toward a Unified View of Data" in the first issue of ACM Transactions on Database Systems, in March 1976. It describes the world as entities with attributes and the relationships between them, and shows how such a description maps to relational, network and other models.
Why it mattered
Entity-relationship diagrams became the usual first step of schema design and are drawn for the same purpose today. The mapping from entities and relationships to tables is how many relational schemas are derived.
Skips counting references held by the stack and registers, removing most of reference counting's per-assignment cost.
The story
Before
Naive reference counting updates a count on every pointer assignment. Most such updates are for local variables and temporaries that live only briefly, so the bookkeeping cost was large compared with the useful work.
What happened, and who
L. Peter Deutsch and Daniel Bobrow published "An efficient, incremental, automatic garbage collector" in Communications of the ACM in September 1976. Their method does not count references from the stack and registers; it keeps a table of objects whose count has dropped to zero and frees one only after checking that no stack reference to it remains.
Why it mattered
Leaving out or postponing the counts on frequently changing references is the idea behind high-performance reference counting collectors that defer or coalesce updates. It is why a modern collector need not touch a count every time a local variable changes.
One server handles requests and forwards its updates to backups, so a shared resource survives the loss of a machine.
The story
Before
A resource shared across a network, such as a file or database held on one host, was a single point of failure. If that host went down, everything that depended on it stopped.
What happened, and who
Peter Alsberg and J. D. Day presented "A Principle for Resilient Sharing of Distributed Resources" at the Second International Conference on Software Engineering, held in San Francisco on 13–15 October 1976. They declared one server host the primary and the others backups, any of which could take over, so the primary role could move between hosts. They introduced the idea of n-host resiliency and described error detection and recovery for two hosts.
Why it mattered
This is the origin of leader-follower replication. Database primary and replica setups, and the failover that promotes a replica when the primary dies, follow the same pattern.
Two strangers can agree on a shared secret over a channel that an eavesdropper can read in full.
The story
Before
Secret keys had to be delivered in advance by courier or through a trusted key distribution centre. Two parties with no prior contact could not start a private conversation, and the key distribution burden grew with every new pair of users.
What happened, and who
Whitfield Diffie and Martin Hellman, at Stanford, published "New Directions in Cryptography" in IEEE Transactions on Information Theory in November 1976. It proposed public-key cryptography and gave a key agreement method in which each side sends a public value and combines it with a private one, using the fact that modular exponentiation is easy to compute and hard to reverse.
Why it mattered
The method, now often used with elliptic curves, is how TLS, SSH and VPNs establish session keys. British intelligence researchers at GCHQ had found related ideas earlier, but kept them secret until 1997.
Proves that a transaction that takes all its locks before releasing any keeps executions serialisable, and shows that row locks alone are not enough.
The story
Before
Concurrent transactions were controlled by locking rules chosen ad hoc. There was no criterion for saying that a locking scheme was correct, so a scheme could allow interleavings whose results no serial order could produce.
What happened, and who
Kapali Eswaran, Jim Gray, Raymond Lorie and Irving Traiger of IBM San Jose published "The Notions of Consistency and Predicate Locks in a Database System" in Communications of the ACM in November 1976. They defined consistency for transactions, proved that two-phase locking (a growing phase of taking locks, then a shrinking phase of releasing them) guarantees serialisable schedules, and observed that locking existing rows misses rows that match a condition but do not yet exist, which they addressed with predicate locks.
Why it mattered
Two-phase locking is the reference correctness rule for lock-based concurrency control, and its strict variants are what many databases implement. The problem the predicate locks address later became known as the phantom problem.
Replaces repeated text with a pointer to an earlier copy inside a sliding window.
The story
Before
Huffman coding, from 1952, shortened common symbols using their frequencies, but treated each symbol separately. It could not exploit a long phrase that appeared many times in the data.
What happened, and who
Abraham Lempel and Jacob Ziv, at the Technion in Israel, published "A Universal Algorithm for Sequential Data Compression" in IEEE Transactions on Information Theory in May 1977. The encoder looks back over a window of recent text for the longest match with the upcoming text and emits a triple of distance, length and the next literal character.
Why it mattered
LZ77 is the base of DEFLATE, which combines it with Huffman coding and is used in gzip, ZIP and PNG files and in the compression of web traffic. Modern compressors such as LZ4 and Zstandard are direct descendants.
A failure table lets a string search avoid ever moving back in the text, giving linear time.
The story
Before
The naive way to find a pattern of length m in a text of length n tries each starting position and, on a mismatch, backs up and starts again, costing O(nm) in the worst case. It threw away what the matched characters had already revealed.
What happened, and who
Donald Knuth, James Morris and Vaughan Pratt published "Fast Pattern Matching in Strings" in the SIAM Journal on Computing in June 1977. Morris and Pratt had found the method around 1970, and Knuth had reached it independently from a different direction. A table computed from the pattern says how far to shift after a mismatch, so the text pointer never moves back.
Why it mattered
The search takes O(n + m) time in the worst case, and the failure-table idea recurs in later matchers such as Aho-Corasick, which extends it to many patterns. It remains the standard example of using a pattern's own structure to avoid rework.
The name for the limit set by a single channel between processor and memory, which both instructions and data must share.
The story
Before
In the stored-program design John von Neumann described in 1945, instructions and data share one memory reached through one path. Engineers knew the path limited speed, but the limit had no widely used name.
What happened, and who
John Backus of IBM gave his Turing Award lecture at the ACM annual conference in Seattle on 17 October 1977. It was titled "Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs" and was published in Communications of the ACM in August 1978. In it he called the word-at-a-time traffic through that channel the von Neumann bottleneck, both a literal one and an intellectual one for programming style.
Why it mattered
The name stuck for the gap between processor speed and memory speed, the memory wall that caches, prefetching, wide memory buses and accelerators with memory close to the compute all work around. Backus's lecture also promoted functional programming as a way out.
Keeps several versions of each data item so that readers see a consistent snapshot without blocking writers.
The story
Before
Under locking, a reader and a writer of the same item blocked each other, so long read queries stalled updates and updates stalled reads. Reads could only be made consistent by keeping them out of the way of writers.
What happened, and who
David P. Reed described the multiversion approach in his 1978 doctoral dissertation at MIT, "Naming and Synchronization in a Decentralized Computer System". Each write creates a new version stamped with a time, and a read is served from the version that was current at its own time.
Why it mattered
Multiversion concurrency control is the default in PostgreSQL, Oracle and MySQL's InnoDB. The first commercial product to use it was Digital Equipment's VAX Rdb/ELN in 1984, designed by Jim Starkey.
The rule that a change's log record must reach stable storage before the change itself, so a crash can be undone or redone.
The story
Before
Early database recovery used ad hoc schemes, such as keeping shadow copies of pages, and different systems reached correctness by different means. There was no simple statement of the ordering rule between the log and the data that recovery relies on.
What happened, and who
Jim Gray, at IBM, set out the write-ahead log protocol in "Notes on Data Base Operating Systems", published in 1978 in the Springer volume Operating Systems: An Advanced Course. The rule is that a log record describing a change must be written to stable storage before the changed page is, and all of a transaction's records before it commits.
Why it mattered
Write-ahead logging is how databases survive crashes: after a failure, the log is replayed to redo committed changes and undo unfinished ones. PostgreSQL, MySQL's InnoDB and SQLite's WAL mode use it, and the log is also what replication streams from.
The first practical public-key system for both encryption and digital signatures, based on the difficulty of factoring large numbers.
The story
Before
Diffie and Hellman had proposed public-key encryption and signatures in 1976 but had no working function for them, only a key agreement method. Without a practical scheme, a signature that anyone could check but only one person could make remained an idea.
What happened, and who
Ron Rivest, Adi Shamir and Leonard Adleman, at MIT, published "A Method for Obtaining Digital Signatures and Public-Key Cryptosystems" in Communications of the ACM, volume 21 issue 2, in February 1978. A public key is a product of two large primes and an exponent; encrypting raises a message to that power modulo the product, and only someone who knows the primes can reverse it.
Why it mattered
RSA made digital signatures and certificates practical, and they secure TLS certificates, software signing and SSH keys. Cryptographers at GCHQ, including Clifford Cocks, had found the same scheme in 1973, but that stayed classified until 1997.
Copies live objects a little at a time while the program runs, and fixes pointers as they are loaded.
The story
Before
Copying collectors such as Cheney's, from 1970, moved all live objects in one go while the program stopped. The pause grew with the amount of live data, which was too long for real-time use.
What happened, and who
Henry G. Baker published "List Processing in Real Time on a Serial Computer" in Communications of the ACM in April 1978. His collector interleaves copying with the running program: every load of a pointer goes through a check that, if the object still sits in the old space, copies it or follows its forwarding address before returning the pointer.
Why it mattered
That check on loads was the first read barrier. Concurrent compacting collectors such as Azul's C4, the JVM's ZGC and Shenandoah descend from it, and they use it to move objects while the program keeps running.
Timestamp rules decide who waits and who restarts, so a circular wait can never form.
The story
Before
Deadlocks among transactions were handled by detection: building a wait-for graph and looking for cycles. In a distributed database that graph is spread across machines and costly to assemble, and checking it repeatedly adds overhead.
What happened, and who
Daniel Rosenkrantz, Richard Stearns and Philip Lewis published "System Level Concurrency Control for Distributed Database Systems" in ACM Transactions on Database Systems in June 1978. Every transaction gets a timestamp when it starts; under wait-die an older transaction may wait for a younger one but a younger one that would wait for an older one is aborted, and wound-wait reverses the roles by having an older transaction abort the younger one it needs.
Why it mattered
Since waits only ever go one way in age order, no cycle can occur and no detector is needed. Google's Spanner uses wound-wait to avoid deadlock, and the rule of keeping a restarted transaction's original timestamp prevents starvation.
A 16-bit processor that reached a megabyte of memory through segmented 20-bit addresses; the ancestor of the x86 line.
The story
Before
Intel's 8-bit 8080, from 1974, used 16-bit addresses and so could reach only 64 KiB of memory. Programs that needed more had to swap banks of memory in and out by hand.
What happened, and who
Intel released the 8086 on 8 June 1978. It was a 16-bit processor in a 40-pin package with a 20-bit address bus, giving a 1 MiB address space, and it formed each address from a 16-bit segment and a 16-bit offset. A version with an 8-bit external bus, the 8088, went into the IBM PC in 1981.
Why it mattered
The 8086's instruction set is the ancestor of x86, and today's x86-64 processors still start in a mode that behaves like it and still run its instructions. Its segmented addressing and the compatibility that later designs kept shape the layout of early PC memory and the boot sequence.
Defines a partial order of events in a distributed system from message causality, with logical clocks that respect it.
The story
Before
In a distributed system there is no single clock that all machines agree on, so what it means for one event to come before another was unclear. Timestamps from physical clocks could disagree with the actual causes and effects.
What happened, and who
Leslie Lamport published "Time, Clocks, and the Ordering of Events in a Distributed System" in Communications of the ACM in July 1978. He defined event A as happening before B if they are in the same process in that order, if A sends a message that B receives, or by chaining these; he then gave logical clocks that assign numbers consistent with that order.
Why it mattered
Happens-before is the basis of reasoning about causality in distributed systems, and vector clocks extend it. Data race detectors and the memory models of languages such as Java and C++ are defined with happens-before.
Concurrent programs built from sequential processes that synchronise by sending values over named channels.
The story
Before
Concurrent programming was dominated by shared variables guarded by semaphores and monitors. Correctness depended on getting every lock right, and the interactions between processes were hard to reason about.
What happened, and who
C. A. R. Hoare published "Communicating Sequential Processes" in Communications of the ACM in August 1978. It proposed a programming notation in which processes share nothing and interact only by input and output commands, with a sender and receiver waiting until both are ready.
Why it mattered
It is the direct ancestor of the occam language and of Go's channels and goroutines, and its process algebra grew into a formal method for checking concurrent designs. Programs that pass messages over channels instead of sharing memory follow the pattern.
Introduces the tri-colour invariant that lets a collector mark memory while the program keeps changing it.
The story
Before
Mark-and-sweep collectors, in use since John McCarthy's Lisp of 1960, halted the program while they traced everything reachable. On large heaps the pauses grew, and a collector that ran alongside the program risked freeing objects the program was still using.
What happened, and who
Edsger Dijkstra, Leslie Lamport, A. J. Martin, C. S. Scholten and E. F. M. Steffens published "On-the-fly garbage collection: an exercise in cooperation" in Communications of the ACM in November 1978. They coloured objects white, grey and black and proved that keeping black objects from pointing to white ones lets the collector and the program run concurrently.
Why it mattered
Tri-colour marking, and the write barriers that protect its invariant, underlie concurrent collectors such as those of Go and the JVM. The invariant is how such collectors can prove that they never free a live object.
A directory records which caches hold each block of memory, so invalidations go only to those caches rather than to all.
The story
Before
Multiprocessors kept private caches consistent by broadcasting every write to all other caches. The broadcast traffic grew with the number of processors and swamped the shared bus.
What happened, and who
Lucien Censier and Paul Feautrier published "A New Solution to Coherence Problems in Multicache Systems" in IEEE Transactions on Computers, December 1978. Memory keeps, for each block, a presence flag per cache and a bit saying whether the block is modified, so a write asks only the caches whose flag is set to invalidate.
Why it mattered
Directories let coherence scale to machines with many processors and no single shared bus. Multi-socket servers and large shared-memory machines use directory protocols in place of, or beside, snooping.
The first relational database sold commercially with SQL, written in assembly language for a minicomputer.
The story
Before
Relational databases existed as research prototypes: IBM's System R at San Jose and Ingres at Berkeley. Neither could be bought as a product, so companies that read Codd's relational model papers had nothing to run it on.
What happened, and who
In 1979 a small company, Relational Software, Inc., founded by Larry Ellison, Bob Miner and Ed Oates, shipped Oracle Version 2. It was written in PDP-11 assembly language and, according to the Oracle community wiki OraFAQ, was the first commercially available SQL relational database.
Why it mattered
It began the commercial relational database industry: Ellison's company, later renamed Oracle, went on to sell SQL databases to businesses for decades, and the vendors that followed made SQL the common interface to stored data.
Sweeping a line across the plane and handling events in sorted order finds every crossing among a set of segments.
The story
Before
Finding the crossings among n line segments by testing every pair takes time proportional to n squared, even when there are almost no crossings. The earlier Shamos-Hoey algorithm decided whether any crossing exists but did not list them.
What happened, and who
Jon Bentley and Thomas Ottmann published "Algorithms for reporting and counting geometric intersections" in IEEE Transactions on Computers in 1979. Their algorithm sweeps a vertical line left to right, keeping the segments it crosses in order and updating that order at each endpoint and each crossing, and runs in O((n + k) log n) time for k crossings.
Why it mattered
The cost now depends on the answer's size k as well as the input. The sweep-line pattern of events processed in sorted order became a standard technique across computational geometry, from map overlay to hidden-surface removal.
A tree of hashes lets a huge data set be checked against one short root hash, with proofs that grow only logarithmically.
The story
Before
To check a large file or database against a trusted digest you had to hash all of it, and a mismatch did not say where the damage was. Keeping a separate trusted hash for every block avoided rehashing everything but meant storing and protecting as many hashes as there were blocks.
What happened, and who
Ralph Merkle patented the scheme in 1979 (US 4,309,569, granted in 1982) as a way to build digital signatures from hash functions. Each leaf holds the hash of a block of data and each inner node the hash of its children, so one root hash commits to every block beneath it.
Why it mattered
To show that one block belongs to the tree you send that block and the sibling hashes along its path to the root, a number proportional to the logarithm of the block count. Git, ZFS, Apache Cassandra's replica repair, Bitcoin and Ethereum, BitTorrent and Certificate Transparency logs all rely on this structure.
US patent 4,135,240 covers the set-user-ID bit, which lets a program run with its owner's privileges instead of its caller's.
The story
Before
On a multi-user system a file's permissions applied to whoever ran the program. A user who needed a limited view of a protected file had to be given the whole file, or an administrator had to do the work.
What happened, and who
Dennis Ritchie of Bell Labs filed the application on 9 July 1973, and US patent 4,135,240, "Protection of data file contents", was granted on 16 January 1979. It adds a bit to an executable file so that, while the program runs, the effective user becomes the file's owner. The patent's own example is a program that lets employees read only their own entries in a shared accounting file.
Why it mattered
Setuid is how passwd can edit the password file for an ordinary user, and sudo is built on it. Any bug in a setuid program runs with the owner's privileges, which makes these programs a long-standing source of privilege-escalation vulnerabilities.
"Access Path Selection in a Relational Database Management System" shows how to choose scans and join orders by estimating cost.
The story
Before
SQL lets a user say what data is wanted rather than how to fetch it, but a system still has to pick indexes and an order for joins. Early relational systems either executed queries naively or left the choice of access path to the user.
What happened, and who
Patricia Selinger and colleagues Morton Astrahan, Donald Chamberlin, Raymond Lorie and Thomas Price at IBM's San Jose research laboratory presented the optimiser of the System R prototype at SIGMOD 1979 in Boston, where the ACM dates the paper 30 May 1979. It estimates the cost of each access path from statistics and searches join orders by dynamic programming, keeping only the cheapest plan for each set of tables.
Why it mattered
Cost-based optimisation with statistics and dynamic programming over left-deep join orders is still the shape of the planners in PostgreSQL and most other relational engines. The paper is why ANALYZE-style statistics matter to query speed.
A short paper defines the memory model in which a multiprocessor behaves as if all its operations ran in one interleaved order.
The story
Before
Programmers writing for several processors assumed that memory acted like a single interleaving of every thread's steps, but nobody had stated that contract, so hardware designers had no definition to check a machine against.
What happened, and who
Leslie Lamport's "How to Make a Multiprocessor Computer That Correctly Executes Multiprocess Programs" appeared in IEEE Transactions on Computers, September 1979. It requires the result of any execution to be the same as if all processors' operations ran in some sequential order, with each processor's operations in program order.
Why it mattered
Sequential consistency is the yardstick every later memory model is measured against. Real hardware and compilers relax it for speed, and the data-race-free guarantee of modern languages is a promise that programs without races still see this simple behaviour.
Deciding whether a schedule of transactions is equivalent to some serial order is NP-complete, so real engines test something narrower.
The story
Before
A schedule was called serializable if its result matched that of some serial schedule of the same transactions. That definition said what correct meant but gave no quick way to test a given schedule.
What happened, and who
Christos Papadimitriou proved in "The Serializability of Concurrent Database Updates", Journal of the ACM 26(4), October 1979, that recognising serializable histories is NP-complete. The paper also identified subclasses that can be recognised efficiently.
Why it mattered
A database cannot afford an NP-complete check on every schedule, so engines enforce conflict serializability, which has a polynomial test (the precedence graph has no cycle), or use locking and timestamps that guarantee it by construction. This is why the narrower class is the one engines actually implement.
Choosing read and write quorums whose sizes overlap guarantees that a read always meets the latest write.
The story
Before
When a data item is copied to several machines, a read may reach a copy the last write missed. Replica schemes had no rule that guaranteed a read would see the latest write.
What happened, and who
David Gifford of Xerox PARC presented "Weighted Voting for Replicated Data" at the seventh ACM Symposium on Operating Systems Principles (SOSP) in December 1979. Each copy gets a number of votes, and a read collects a read quorum of votes and a write a write quorum, chosen so that the two together exceed the total and any two write quorums overlap.
Why it mattered
The overlap rule is the idea behind quorum reads and writes in Dynamo-style stores such as Cassandra and Riak, where the replica counts R and W are set per request to trade latency against consistency.
Personal computers, RISC, TCP/IP and the theory of distributed systems.
PaperLanguages and runtimes
Mesa-Style Monitors
In Mesa's monitors a signal is only a hint that a condition may hold, so a waiter must check it again.
The story
Before
Hoare's monitors handed the lock straight to the waiting thread on a signal, which guaranteed the condition was still true but forced two extra context switches and awkward rules for the signaller. Simple mistakes, such as signalling too early, broke programs.
What happened, and who
Butler Lampson and David Redell of Xerox PARC described the monitors of the Mesa language in "Experience with Processes and Monitors in Mesa", Communications of the ACM 23(2), February 1980, after presenting it at SOSP in December 1979. A signal moves the waiter to the ready queue but the signaller keeps the lock, so the woken thread must re-check its condition.
Why it mattered
The rule to wait in a loop, while (!cond) wait(), is the standard usage in Java's wait/notify and in POSIX threads condition variables. It also tolerates spurious wakeups, which those APIs allow.
Computers that must agree can be forced to when some of them send arbitrarily wrong messages, but only if enough of them are sound.
The story
Before
Fault-tolerant designs mostly assumed a failed component simply stopped. A component that keeps running and gives different answers to different neighbours had not been formalised, and no one had said how many such faults a group could survive.
What happened, and who
Marshall Pease, Robert Shostak and Leslie Lamport at SRI International published "Reaching Agreement in the Presence of Faults" in the Journal of the ACM, April 1980. The work came from SIFT, a NASA-sponsored project to build aircraft control from several ordinary computers. It showed that agreement needs at least 3n + 1 computers to tolerate n faulty ones.
Why it mattered
This opened the theory of consensus under arbitrary faults. Lamport, Shostak and Pease retold it in 1982 as the Byzantine Generals problem, and today's Byzantine-fault-tolerant protocols and blockchain consensus start from it.
A memo gave the two byte orders their names, borrowing a quarrel over eggs from Gulliver's Travels.
The story
Before
Machines stored a multi-byte number with its most significant byte first or last, and engineers argued about which was right. The argument had no agreed words, which made it hard to state which order a protocol or file format used.
What happened, and who
Danny Cohen wrote "On Holy Wars and a Plea for Peace" as IEN 137, an Internet Engineering Note, dated 1 April 1980, and it was reprinted in IEEE Computer in October 1981. He borrowed from Jonathan Swift's Gulliver's Travels, where two kingdoms fight over which end of a boiled egg to break, and called the orders big-endian and little-endian.
Why it mattered
The words are still how programmers describe byte order. Network protocols fix one order on the wire (big-endian, so-called network byte order) so machines of either kind can talk, and file formats state their own.
"The Case for the Reduced Instruction Set Computer"
A paper argues that a small, simple instruction set can beat a rich one, and names the idea RISC.
The story
Before
Each processor generation added instructions, addressing modes and microcode, as the DEC VAX did, so that one instruction could do more. The complexity cost chip area and design time, and compilers seldom used the most elaborate instructions.
What happened, and who
David Patterson and David Ditzel published "The Case for the Reduced Instruction Set Computer" in ACM SIGARCH Computer Architecture News 8(6), October 1980. It argued that the next generation of VLSI machines might be better built with reduced instruction sets than complex ones.
Why it mattered
The paper gave the movement its name and launched the Berkeley RISC project. Later RISC designs such as SPARC, MIPS and ARM followed the argument, and today's x86 processors decode complex instructions into simple internal operations.
A hash file grows one bucket at a time in a fixed order, with no directory to double and no full rebuild.
The story
Before
A static hash table has a fixed number of buckets and has to be rebuilt as the file grows. Extendible hashing grows smoothly but needs a directory of bucket pointers that periodically doubles.
What happened, and who
Witold Litwin presented "Linear Hashing: A New Tool for File and Table Addressing" at the Sixth International Conference on Very Large Data Bases (VLDB) in Montreal, published in October 1980. Buckets split one at a time in round-robin order, and a pair of hash functions tells any key which bucket it belongs in.
Why it mattered
The table grows in small, predictable steps, so no single insert pays for a whole rebuild. Variants are used in database and storage engines, and Litwin's later distributed version, LH*, applied it across servers.
Corrupted sequence numbers left three routing updates each newer than another, so they circulated forever and choked every router.
The story
Before
The ARPANET's routers, called IMPs, flooded routing updates that carried a sequence number, and a node kept whichever update had the higher number. That comparison was the only safeguard against stale updates.
What happened, and who
On 27 October 1980 a fault at IMP 50 dropped bits in the sequence number of an update, so the network held copies numbered 8, 40 and 44. Each counted as later than another (44 over 40, 40 over 8, and 8 over 44 under the wrap-around comparison), so all three circulated indefinitely and used the routers' CPU and buffers. Eric Rosen analysed it in RFC 789; operators recovered the network only after hours, by patching the IMPs to ignore updates from IMP 50.
Why it mattered
A failure of one small check took down a whole network, and it shaped link-state design. Later protocols such as OSPF and IS-IS use large sequence numbers, ageing of old updates and checksums so that a corrupted update cannot outlive its welcome.
A university design built a working processor from a few dozen simple instructions and overlapping register windows.
The story
Before
Commercial processors of the time, such as the VAX and the Intel 8086, carried hundreds of instructions and much microcode. The RISC argument was still a paper proposal until someone built a chip to test it.
What happened, and who
David Patterson's group at UC Berkeley designed RISC I from 1980, with Carlo Sequin, and published the design at ISCA in 1981. It had 31 instructions and register windows that let a procedure call avoid saving registers to memory. Working silicon arrived in June 1982, and the follow-up RISC II refined the design.
Why it mattered
A small university team built a competitive processor, which persuaded industry that simple instructions could win. Sun's SPARC was commercialised from the Berkeley design, and its register windows were an idea that later designs dropped.
An invited paper presents the transaction as the unit of work that either happens completely or not at all.
The story
Before
Programmers grouped operations that had to succeed or fail together by convention and by ad hoc code in each application. Recovery was worked out one system at a time.
What happened, and who
Jim Gray gave "The Transaction Concept: Virtues and Limitations" as an invited paper at the seventh VLDB conference in Cannes in 1981. It describes a transaction as a transformation of state with the properties of atomicity, durability and consistency, and lists what such a unit cannot do, such as handle very long-lived work.
Why it mattered
The transaction became the basic contract that databases, message queues and payment systems offer applications. Gray received the Turing Award in 1998 for this line of work.
A two-bit counter per branch predicts taken or not-taken and only changes its mind after two misses in a row.
The story
Before
Early pipelines guessed statically, for example that every backward branch is taken, or remembered a single bit of history per branch. A one-bit predictor is wrong twice for each loop exit, because it flips on the exit and flips back on the next entry.
What happened, and who
James E. Smith published "A study of branch prediction strategies" at the International Symposium on Computer Architecture (ISCA) in 1981. It compared several schemes and described a table of small counters that count up when a branch is taken and down when it is not, saturating at both ends.
Why it mattered
A saturating counter tolerates one anomaly, such as a loop exit, without changing its prediction. The idea underlies the bimodal tables and, combined with history, the predictors in modern processors, which make a wrong guess cost a pipeline flush.
Let transactions run without locks and check for conflicts only at commit, restarting one if a conflict is found.
The story
Before
Lock-based concurrency control assumed conflicts were common and paid for locking on every access. When transactions rarely touch the same data, that cost buys little, and waiting for locks wastes time.
What happened, and who
H. T. Kung and John T. Robinson published "On Optimistic Methods for Concurrency Control" in ACM Transactions on Database Systems 6(2), June 1981. A transaction reads and writes private copies in three phases, read, validate and write, and is backed up if validation finds a conflict.
Why it mattered
Validation at commit is the model behind optimistic locking in application code (a version column checked on update) and in many in-memory and multiversion engines. It wins when conflicts are rare and loses to locking under heavy contention, where the restarts pile up.
Two shared variables give mutual exclusion between two processes in a few lines, with a proof anyone can follow.
The story
Before
Dekker's algorithm from the 1960s solved mutual exclusion for two processes using only reads and writes, but it was long and its correctness was hard to follow. Many believed the problem inevitably needed complex solutions.
What happened, and who
Gary Peterson published "Myths About the Mutual Exclusion Problem" in Information Processing Letters 12(3), June 1981. A process sets its own flag to say it wants in, then sets a shared turn variable to the other process and waits while the other wants in and has the turn.
Why it mattered
It is the textbook proof that software alone can provide mutual exclusion. On modern processors it fails as written unless memory barriers stop reads and writes being reordered, which is why real locks rely on atomic instructions.
An open-architecture PC built from off-the-shelf parts set the standard that later machines had to be compatible with.
The story
Before
Earlier personal computers, such as the Apple II and the machines running CP/M, were incompatible with one another. Software written for one did not run on another, and IBM had sold only large systems to businesses.
What happened, and who
IBM announced the IBM Personal Computer, model 5150, on 12 August 1981. It used the Intel 8088 processor at 4.77 MHz and ran PC DOS from Microsoft, and IBM published its technical documentation, including the BIOS listing.
Why it mattered
Other firms could copy the design and sold compatible clones, so the x86 instruction set and the PC's BIOS-based boot and bus layout became the common platform. That compatibility is why today's x86 machines still start in a mode that resembles the 8086.
A right-pointer on each B-tree node lets a reader that lands in a half-split node simply move right.
The story
Before
Concurrent access to a B-tree usually meant lock coupling: hold a lock on a node until the child is locked, going down the tree. Readers then blocked behind writers near the top of the tree, which every operation passes through.
What happened, and who
Philip Lehman and S. Bing Yao published "Efficient Locking for Concurrent Operations on B-Trees" in ACM Transactions on Database Systems 6(4), December 1981. Each node gets a link to its right neighbour and a high key, so a search that arrives at a node that has just split can follow the link to the new sibling.
Why it mattered
Readers need no locks on the way down, and a writer holds few locks at once. PostgreSQL's nbtree index still uses this scheme, and it is the reference design for concurrent B-trees in databases.
Keeping just k counters is enough to find every item that occurs more than 1/k of the time.
The story
Before
Counting how often each value occurs needs a counter per distinct value, memory that grows with the number of distinct keys. For a huge stream, that memory is not available.
What happened, and who
Jayadev Misra and David Gries published "Finding repeated elements" in Science of Computer Programming 2(2), in 1982. It extends the Boyer-Moore majority-vote algorithm: it keeps at most k value-counter pairs and, when a new value finds no free counter, decrements all of them. Any value occurring more than n/k times among n items is guaranteed to survive, and a second pass can confirm which ones do.
Why it mattered
It is one of the first streaming algorithms, using memory that depends on k instead of the stream's length. It is the ancestor of the heavy-hitter sketches that find hot keys and top talkers in monitoring and network traffic.
"On Optimizing an SQL-like Nested Query" turns a correlated subquery into a join the optimiser can plan.
The story
Before
A nested query was executed as a nested loop, re-running the inner query once for every row of the outer one. The cost grew with the product of the two sizes, whatever indexes existed.
What happened, and who
Won Kim published "On Optimizing an SQL-like Nested Query" in ACM Transactions on Database Systems 7(3), in 1982. It classified nested queries and rewrote several kinds into joins that a planner can order freely.
Why it mattered
Subquery unnesting, later called decorrelation, is a step in every serious optimiser. Kim's method gave wrong answers for some queries, a defect known as the count bug that Richard Ganski and Harry Wong identified and repaired in 1987.
Two running sums detect all single-bit errors and many burst errors at much lower cost than a CRC.
The story
Before
A plain sum of the bytes in a block is cheap but blind to reordering, since swapping two bytes leaves the total unchanged. A cyclic redundancy check catches more but was costly to compute in software of the time.
What happened, and who
John G. Fletcher of Lawrence Livermore National Laboratory published "An Arithmetic Checksum for Serial Transmissions" in IEEE Transactions on Communications, January 1982. It keeps a running sum of the data and a second running sum of the first, so position affects the result.
Why it mattered
It offers error detection close to a CRC's at a fraction of the work. The idea lives on in Adler-32, in ISO transport checksums and in the checksums used by ZFS.
Segment selectors, descriptors and privilege levels give x86 a hardware boundary between the operating system and programs.
The story
Before
The 8086 let any program read or write any address, so one program could corrupt another or the operating system. It had no way to keep a program out of memory it did not own.
What happened, and who
Intel introduced the 80286 on 1 February 1982. It was the first x86 processor with a protected mode: segment registers hold selectors that index descriptor tables giving each segment's base, limit and privilege level, and the chip has an on-chip memory management unit and a 24-bit address space of 16 MB.
Why it mattered
Protected mode is where privilege levels and hardware isolation between processes on x86 begin. The 286 could not return from protected to real mode without a reset, which limited its use with DOS, and Intel's 80386 later extended the design to 32 bits with paging.
Generals sending messengers must agree on one plan although some of them may be traitors, an allegory for faulty computers.
The story
Before
Distributed designs mostly considered nodes that fail by stopping. A node that sends conflicting messages to different peers, which is what an arbitrary fault looks like, was described in the 1980 paper on agreement among faulty units in dry, technical terms.
What happened, and who
Leslie Lamport, Robert Shostak and Marshall Pease of SRI International published "The Byzantine Generals Problem" in ACM Transactions on Programming Languages and Systems 4(3), July 1982. It retold the earlier result as commanders of a besieging army. With oral messages the problem is solvable only if more than two thirds of the generals are loyal, while unforgeable signed messages allow it for any number of traitors.
Why it mattered
"Byzantine fault" became the standard term for arbitrary misbehaviour. It underlies Byzantine-fault-tolerant replication protocols and the consensus used by blockchains, where participants are not trusted.
Berkeley's Unix release gave programs one file-descriptor interface to the network, plus select to wait on many at once.
The story
Before
There was no common way for a program to use a network that did not depend on the protocol. Each network stack came with its own calls, so a program written for one had to be changed for another.
What happened, and who
The Computer Systems Research Group at the University of California, Berkeley released 4.2BSD in 1983, funded by DARPA, with a TCP/IP stack and the socket interface, in which a network connection is a file descriptor. It also added select, which suspends a process until one of several descriptors is ready.
Why it mattered
The Berkeley socket API is still how nearly every program reaches the network; Windows' Winsock is a version of it. select was the first tool for one thread to serve many connections, and its successors poll, epoll and kqueue serve today's servers.
A campus computing project meant to share one file system across thousands of workstations produced the Andrew File System.
The story
Before
Sun's NFS kept servers stateless so that a crash was easy to recover from, but each client had to keep asking the server whether its cached copy was still valid. That traffic grows with every client added, so it does not scale to thousands of machines.
What happened, and who
Carnegie Mellon University began the Andrew project around 1983 with IBM's support, aiming at a campus of thousands of workstations sharing files. Its file system, first called Vice and then the Andrew File System (AFS), takes its name from Andrew Carnegie and Andrew Mellon.
Why it mattered
AFS caches whole files on the client, and the server promises to tell the client when a file changes, a mechanism called a callback, so clients stop polling. Callbacks and whole-file caching influenced later distributed file systems, and AFS uses Kerberos for authentication.
Each instruction holds several operations chosen by the compiler, so the chip does not need to find parallelism at run time.
The story
Before
Superscalar-style processors used hardware to search the running instruction stream for operations that could go together. That issue logic grew complicated and took chip area and power as more operations were attempted at once.
What happened, and who
Josh Fisher of Yale developed the idea, working on trace scheduling, a way to schedule code across branches, and on very long instruction word (VLIW) machines. His 1983 paper "Very Long Instruction Word Architectures and the ELI-512", at ISCA, laid out the design. He founded Multiflow in 1984 to build VLIW minicomputers.
Why it mattered
The hardware shrinks and the compiler carries the scheduling. VLIW appears in many digital signal processors, and Intel's Itanium took the idea, though the compiler difficulty of finding enough parallelism held it back in general-purpose computing.
An entire network changed protocols on one day, because the old and new protocols could not coexist.
The story
Before
The ARPANET's hosts spoke NCP, the Network Control Protocol, which leaned on the network's own routers for reliability and was designed for a single network. It could not join together the networks that were being built.
What happened, and who
On 1 January 1983 the ARPANET switched from NCP to the TCP/IP protocol suite. Every host had to convert together or be left communicating by makeshift means, which gave such deadlines the name flag day.
Why it mattered
From then on the ARPANET ran on the protocols the whole internet still uses, and different networks could be joined at the IP layer. The flag day is also a lesson in why changes to a protocol that everyone must share are so hard to roll out.
IBM's second PC came with a hard disk as standard on most models, making disk booting the normal case.
The story
Before
The original IBM PC of 1981 stored programs on floppy disks and cassette tape. Booting a disk with a large amount of data and programs on it was an add-on, not part of the standard machine.
What happened, and who
IBM announced the Personal Computer XT on 8 March 1983. Most models came with a 10 MB hard disk and a disk controller built around Xebec chips.
Why it mattered
A fixed disk as a standard part made a PC into a machine that kept its programs and data on it, and the IBM PC line ran from it for years.
A distributed, hierarchical database replaced one central table of host names and let every organisation manage its own names.
The story
Before
Every host name on the ARPANET was listed in a single file, HOSTS.TXT, maintained at SRI's Network Information Center and copied to each host. As the network grew, the file's size and the rate of change came near the limit of what could be managed.
What happened, and who
Paul Mockapetris of the Information Sciences Institute at USC specified the Domain Name System in RFC 882, "Domain names: Concepts and facilities", and RFC 883, published in November 1983. Names form a tree, each part of which can be delegated to the organisation responsible for it, and servers answer queries about their part.
Why it mattered
DNS has run the internet's naming ever since. Delegation lets the system grow without central coordination, and caching keeps the load down. RFC 1034 and RFC 1035 later replaced the two documents.
A survey gives databases their acronym, ACID, and a vocabulary for how a buffer pool and a log interact.
The story
Before
Recovery techniques were described one system at a time, in each system's own terms. There was no common way to compare two designs or state what a transaction system promised.
What happened, and who
Theo Haerder of the University of Kaiserslautern and Andreas Reuter of IBM Research in San Jose published "Principles of Transaction-Oriented Database Recovery" in ACM Computing Surveys 15(4), December 1983. It names atomicity, consistency, isolation and durability as ACID and classifies buffer management by whether uncommitted pages may be written to disk (steal) and whether committed pages must be (force).
Why it mattered
ACID is how databases and their guarantees are still discussed, and steal/no-force is the policy that write-ahead logging exists to make safe. The vocabulary lets an engineer say exactly what a recovery scheme must undo and redo after a crash.
Every object carries a pointer to itself that the collector can redirect, so a program never has to check whether an object has moved.
The story
Before
Real-time garbage collectors had needed special hardware or a check on every pointer load to see whether the object had been moved. On ordinary hardware that check was expensive.
What happened, and who
Rodney Brooks published "Trading data space for reduced time and code space in real-time garbage collection on stock hardware" in the proceedings of the ACM Symposium on LISP and Functional Programming in 1984. He added one indirection pointer to every heap object, so an access always goes through it and the overhead of most operations falls to a single instruction.
Why it mattered
A collector can move an object and update its one forwarding pointer while the program keeps running. The Shenandoah collector in OpenJDK used this Brooks pointer for concurrent compaction.
Adding an exclusive state lets a core that holds the only copy of a cache line write it without telling the bus.
The story
Before
Snooping caches that wrote every store through to memory filled the shared bus with traffic. A write-back cache that fixed this needed extra bus messages, for example to ask permission to write a line it had just read alone.
What happened, and who
Mark Papamarcos and Janak Patel of the University of Illinois described a write-back coherence protocol at the International Symposium on Computer Architecture (ISCA) in 1984. It is now called the Illinois protocol, and its four states, Modified, Exclusive, Shared and Invalid, give MESI its name.
Why it mattered
The Exclusive state saves a bus transaction when a core reads a line no one else holds and then writes it. MESI and its descendants such as MOESI are the base of cache coherence in nearly every multicore processor.
Translating bytecode to machine code while the program runs, and caching what each call site last found, made a dynamic language fast.
The story
Before
Smalltalk-80 bytecode was interpreted, and every message send had to look up the receiver's class to find the method. In a language where everything is an object, that lookup dominated the running time.
What happened, and who
L. Peter Deutsch and Allan Schiffman presented "Efficient implementation of the Smalltalk-80 system" at POPL in Salt Lake City in January 1984. The paper describes translating bytecode to native code on demand and remembering, at each call site, the class and method last found there, an idea now called an inline cache.
Why it mattered
Dynamic translation and inline caches are found in the just-in-time compilers of JavaScript engines, the JVM and others. Together they show a dynamic language can approach compiled speed when the program's behaviour at each site is stable.
Names congestion collapse and gives TCP a rule that holds back tiny segments while earlier data is still unacknowledged.
The story
Before
TCP sent whatever the application handed it, so a program sending one typed character at a time put a 41-byte packet on the wire (one byte of data, 40 of headers) for every keystroke. When the network was busy, round-trip times grew past the retransmission timer, hosts resent segments that were only late, and the duplicates added still more load.
What happened, and who
John Nagle, an engineer at Ford Aerospace and Communications Corporation, wrote RFC 896, "Congestion Control in IP/TCP Internetworks", dated 6 January 1984. He observed the trouble on Ford's own network, joined to the ARPANET through a gateway, described the stable overloaded state that he called congestion collapse, and proposed the small-packet rule now named after him: while data is unacknowledged, hold back new small segments and send them together.
Why it mattered
The name congestion collapse comes from this note, and the small-packet rule it proposed is still TCP's default behaviour. Together with delayed acknowledgements it shapes the latency of every interactive TCP connection, which is why engineers still meet it when they set TCP_NODELAY on a latency-sensitive socket.
One disk flush of the log can commit many transactions at once.
The story
Before
Each transaction waited for its own synchronous write of the log to disk before it could report success. A disk that can only sync a few hundred times a second capped the commit rate at that figure.
What happened, and who
David DeWitt, Randy Katz, Frank Olken, Leonard Shapiro, Michael Stonebraker and David Wood described committing several transactions with one log write in "Implementation Techniques for Main Memory Database Systems", at SIGMOD in Boston in June 1984.
Why it mattered
Group commit lets a database sustain thousands of commits per second on a device that syncs far fewer times, at the price of a little latency for the first waiter. It is a standard feature of relational engines and a common tuning knob.
A B-tree-like tree of bounding rectangles indexes spatial data and answers "what overlaps this area".
The story
Before
B-trees order keys along one dimension, which does not fit rectangles and points in two or more. Grid files and quadtrees handled dynamic, changing spatial data poorly.
What happened, and who
Antonin Guttman presented "R-trees: A Dynamic Index Structure for Spatial Searching" at SIGMOD in Boston, 18 to 21 June 1984. It builds on an earlier Berkeley technical report by Guttman and Michael Stonebraker in 1983. Each node holds the bounding rectangle of its children, and the tree stays balanced as objects are added.
Why it mattered
Spatial queries can skip whole subtrees whose rectangle misses the query area. R-tree indexes underlie GIS software and databases such as PostGIS, and SQLite offers an R*Tree module.
"A Fast File System for UNIX" introduces cylinder groups and locality-aware allocation, raising disk throughput several-fold.
The story
Before
The original Unix file system kept its inodes in one table apart from the data blocks and used small blocks. As files were created and deleted, a file's blocks ended up scattered, and the disk head kept moving between the inode table and the data, so a file read reached only a small fraction of the disk's raw bandwidth.
What happened, and who
Marshall Kirk McKusick, William Joy, Samuel Leffler and Robert Fabry, at the University of California, Berkeley, published "A Fast File System for UNIX" in ACM Transactions on Computer Systems in August 1984. It splits the disk into cylinder groups, each holding its own inodes and data, places a file's blocks and its inode near each other, and uses larger blocks.
Why it mattered
The idea of keeping related data and metadata physically close became the model for later Unix file systems. The block groups of ext2, ext3 and ext4 descend from cylinder groups, and the UFS family in the BSDs and elsewhere derives from this design.
"A Theory of the Learnable" defines learning with probability and accuracy guarantees.
The story
Before
Machine learning was a collection of methods with no agreed, computational definition of what it meant for a program to have learned something. Whether a learning task was feasible could be argued only case by case.
What happened, and who
Leslie Valiant, then at Harvard, published "A Theory of the Learnable" in Communications of the ACM in November 1984, after presenting it at the ACM Symposium on Theory of Computing earlier that year. It defines learning as producing, from random examples and in polynomial time, a rule that is probably approximately correct: wrong on at most a small fraction of cases, with high probability.
Why it mattered
It founded computational learning theory, the study of which concepts can be learned with how many examples and how much computation. Sample-complexity bounds in machine learning use its vocabulary, and Valiant received the Turing Award in 2010, in part for this work.
A non-volatile memory that can be erased in blocks rather than a byte at a time, the basis of most storage today.
The story
Before
EEPROM kept its contents without power and could be rewritten, but it was erased and programmed a byte at a time, which made it slow and expensive for large amounts of storage. Ultraviolet-erasable EPROM had to be taken out of the machine to be erased.
What happened, and who
Fujio Masuoka, an engineer at Toshiba, had invented the cell around 1980, and he and colleagues presented the design at the International Electron Devices Meeting (IEDM) in December 1984. The name came from a colleague, Shoji Ariizumi, because erasing a block at once reminded him of a camera flash. Toshiba presented the NAND variant at IEDM in 1987.
Why it mattered
NAND flash is the storage in phones, memory cards and solid-state drives, and NOR flash holds firmware that a processor runs directly. Its block-erase behaviour is also why flash storage needs wear levelling and a translation layer, which shapes how engineers design for it.
"Amortized efficiency of list update and paging rules" proves LRU is k-competitive against the offline optimum.
The story
Before
Least-recently-used replacement was known to work well in practice, but the reason rested on assumptions about locality of reference. It had no guarantee for an arbitrary request sequence.
What happened, and who
Daniel Sleator and Robert Tarjan published "Amortized efficiency of list update and paging rules" in Communications of the ACM in February 1985. They compare an online algorithm, which sees requests one at a time, with the best offline algorithm that knows them all in advance, and prove that LRU with a cache of k pages does at most k times as many faults, on any sequence.
Why it mattered
This launched competitive analysis, the standard way to judge online algorithms without assuming anything about the input. The same style of argument is used for caching, scheduling and load balancing.
A way to keep a uniform random sample of k items from a stream whose length is not known in advance.
The story
Before
Drawing a uniform random sample of a data set normally needed its size beforehand, or a pass to count it and a second to select. That is impossible for a stream you can read only once.
What happened, and who
Jeffrey Vitter, at Brown University, published "Random Sampling with a Reservoir" in ACM Transactions on Mathematical Software in March 1985. It describes Algorithm R, which he credits to Alan Waterman: keep the first k items, then replace a random kept item with the n-th item with probability k/n. He then gives faster variants, including Algorithm Z, which skip over items instead of examining each.
Why it mattered
The method uses memory proportional to the sample, however long the stream is, and every item ends up equally likely to be in the sample. It is a standard tool for sampling logs, events and database scans.
Proves that no consistent strategy for the multi-armed bandit problem can have regret that grows more slowly than the logarithm of time.
The story
Before
Herbert Robbins posed the multi-armed bandit problem in 1952: choose repeatedly among options with unknown payoffs, balancing trying new ones against using the best so far. Good strategies existed, but no one knew how well the best possible could do.
What happened, and who
Tze Leung Lai and Herbert Robbins published "Asymptotically efficient adaptive allocation rules" in Advances in Applied Mathematics in March 1985. They show that any consistent strategy must, over time, pull suboptimal arms a number of times that grows at least logarithmically, with a constant given by the Kullback-Leibler divergence between the arms' reward distributions.
Why it mattered
Regret that grows like the logarithm of time is the target that bandit algorithms are judged against. The upper-confidence-bound and Thompson-sampling methods used in online experimentation and recommendation are analysed by how closely they meet this lower bound.
No deterministic protocol can guarantee that nodes agree in an asynchronous system if even one node may crash.
The story
Before
Distributed systems researchers had built agreement protocols for replicated databases and hoped a careful enough design could always reach agreement, even when a node failed. Nobody had shown whether that hope could be met in a system with no bound on message delay.
What happened, and who
Michael Fischer, Nancy Lynch and Michael Paterson proved the result in "Impossibility of Distributed Consensus with One Faulty Process", published in the Journal of the ACM in April 1985 (first presented at the ACM Symposium on Principles of Database Systems in 1983). In a fully asynchronous system, where a slow node cannot be told from a crashed one, no deterministic protocol can guarantee agreement while tolerating even one crash.
Why it mattered
It tells designers what to give up. Practical consensus protocols such as Paxos and Raft add timeouts, randomness or assumptions about timing, and they stay safe when those assumptions fail but may stop making progress. Every failure detector and leader-election timeout in a replicated system is a response to this result.
The first ARM RISC processor, in which almost every instruction can be made conditional.
The story
Before
Acorn Computers, the British maker of the BBC Micro, wanted a processor for its next machine. The 16-bit and 32-bit chips on the market were slow at memory access or complex, and Acorn's small team believed a simple RISC design, as in the Berkeley RISC research, could do better.
What happened, and who
At Acorn in Cambridge, Sophie Wilson designed the instruction set and Steve Furber designed the chip. The first silicon, made by VLSI Technology, arrived on 26 April 1985 and worked first time. The instruction set gives every instruction a condition field, so a short branch can be replaced by conditionally executed instructions.
Why it mattered
The ARM design led to Arm Ltd, founded in 1990 as a joint venture of Acorn, Apple and VLSI Technology. Its low power use made its descendants the processors in most phones and, more recently, in laptops and servers, and its conditional execution is a standard example of how an instruction set can avoid branches.
"A Decomposition Storage Model" stores every attribute as its own column of (id, value) pairs.
The story
Before
Database systems stored a table as rows, all of a record's fields side by side. A query that needed one column still read every field of every record it scanned.
What happened, and who
George Copeland and Setrag Khoshafian, at the Microelectronics and Computer Technology Corporation in Austin, presented "A Decomposition Storage Model" at the ACM SIGMOD conference in May 1985. It splits each relation into one binary relation per attribute, holding pairs of a record identifier and a value.
Why it mattered
This is the ancestor of the column store. Analytical engines that read only the columns a query names, and compress each column well because its values are alike, descend from this layout, which became widely adopted in analytical databases from the 2000s onwards.
Shows how a pipelined processor that finishes instructions out of order can still report an exception as if it ran one at a time.
The story
Before
A pipelined processor could finish instructions in a different order from the program's. When one of them faulted, later instructions might already have changed registers or memory, so the saved state matched no point in the program and the fault could not be cleanly resumed.
What happened, and who
James Smith and Andrew Pleszkun presented "Implementation of Precise Interrupts in Pipelined Processors" at the International Symposium on Computer Architecture in Boston in June 1985. It defines a precise interrupt, where the saved state matches sequential execution, and evaluates several ways to get one, including the reorder buffer, which holds results until earlier instructions have finished.
Why it mattered
The reorder buffer is central to modern out-of-order processors: results become visible in program order, so a mispredicted branch or a fault can be undone by discarding younger entries. This is what makes speculative execution safe to recover from.
"Why Do Computers Stop and What Can Be Done About It?" reports that faults in production software are often transient.
The story
Before
Reliability engineering aimed at removing every fault before release, on the assumption that a failing program would fail the same way again. Little published data existed on why deployed systems actually stopped.
What happened, and who
Jim Gray, at Tandem Computers, wrote "Why Do Computers Stop and What Can Be Done About It?", Tandem Technical Report 85.7, in June 1985. It analyses the failures of Tandem's fault-tolerant systems, finds administration and software to be the major causes, and observes that many software faults are soft: rerunning the operation usually succeeds.
Why it mattered
If most faults are transient, the sensible response is to restart the failed part from a known state and retry instead of trying to repair it in place. That argument underlies process pairs, and later the supervisors of Erlang, whose designers drew on it.
A binary search tree that reshapes itself on every access, with no balance information, yet costs O(log n) amortised per operation.
The story
Before
Balanced search trees such as AVL and red-black trees keep extra balance data in each node and need several rebalancing cases. Their guarantee is per operation, but they gain nothing when some keys are accessed much more often than others.
What happened, and who
Daniel Sleator and Robert Tarjan published "Self-adjusting binary search trees" in the Journal of the ACM in July 1985. After every access the tree moves the accessed node to the root by a sequence of rotations called splaying. A single operation can be slow, but any sequence of m operations on n keys costs O(m log n) in total.
Why it mattered
The paper made amortised analysis a design tool: a structure is judged by the cost of a whole sequence. Splay trees adapt to skewed access patterns, and the idea of moving recently used items closer to the front recurs in caches and lookup structures.
RFC 958 introduces the Network Time Protocol and its four-timestamp exchange.
The story
Before
The Time Protocol (RFC 868, 1983) returned the server's time and nothing else. It ignored the delay of the request and reply, so a client's clock was wrong by however long the message spent in the network.
What happened, and who
David L. Mills, at the University of Delaware, wrote RFC 958, "Network Time Protocol (NTP)", published in September 1985. A client records when it sent a request and when it received the reply, and the server records when it received the request and when it replied. The four timestamps let the client estimate the delay and its own clock offset.
Why it mattered
NTP keeps clocks across the internet within milliseconds of each other. Certificate validity, log correlation, distributed databases and Kerberos tickets all rely on clocks agreeing, and the protocol has been revised several times since.
Lets Ethernet bridges elect a root and switch off redundant links, so a network with loops cannot flood itself.
The story
Before
Bridges join Ethernet segments by learning which addresses are on which side. When the topology contained a loop, a broadcast frame circled it forever and was copied at every pass, a broadcast storm that could bring the network down.
What happened, and who
Radia Perlman, then at Digital Equipment Corporation, published "An Algorithm for Distributed Computation of a Spanning Tree in an Extended LAN" in ACM SIGCOMM Computer Communication Review in September 1985. Each bridge exchanges messages with its neighbours, the bridges elect a root, and every bridge blocks the ports that would form a loop, leaving a tree.
Why it mattered
It allowed bridged networks to be cabled with redundant links and to work without configuration. Spanning tree, in its later standardised forms, is still running in Ethernet switches.
A 32-bit x86 processor with paging and a mode that runs 8086 programs inside a protected system.
The story
Before
The 80286 had a protected mode, but it could only return to real mode by resetting the processor, and it had no paging. Operating systems could not protect programs from each other and also run existing real-mode software.
What happened, and who
Intel released pre-production samples of the 80386 to developers in October 1985; volume production followed in 1986. It was the first 32-bit x86, addressing up to 4 GB, with an on-chip memory management unit for paging and a virtual-8086 mode that runs real-mode programs under a protected-mode operating system.
Why it mattered
Paged virtual memory on x86 makes possible the 32-bit versions of Windows and Linux, and the 386's instruction set (IA-32) is the base that later x86 processors extended. Linus Torvalds wrote Linux for it in 1991.
One standard for floating-point formats, rounding, infinities and NaN.
The story
Before
Each computer maker had its own floating-point format: IBM used a hexadecimal exponent, and CDC and Cray used ones' complement. The same program could give different answers on different machines, and handling of overflow and division by zero varied.
What happened, and who
The IEEE standard for binary floating-point arithmetic, IEEE 754-1985, was published on 12 October 1985. William Kahan of the University of California, Berkeley, was a principal architect of it and of the Intel 8087 coprocessor that put its ideas into hardware. It fixed formats, rounding modes, and the special values infinity and NaN (not a number).
Why it mattered
Almost every processor and language now uses these formats, so a double computes the same result on a laptop, a phone and a server. Behaviour such as 0.1 + 0.2 != 0.3 and the meaning of NaN comes from this standard, which was revised in 2008 and 2019.
IBM had created SQL in the 1970s, and other vendors had built their own versions of it. Queries written for one product did not run unchanged on another.
What happened, and who
The American National Standards Institute adopted the first SQL standard, ANSI X3.135, in 1986, and the International Organization for Standardization adopted it as ISO 9075 in 1987. It was prepared by the ANSI X3H2 database committee.
Why it mattered
It made SQL the common language of relational databases, and later revisions added the features now taken for granted. Vendors still differ in what they implement, so SQL rarely moves between systems without changes.
A stack held in a single word that threads update with compare-and-swap, without locks.
The story
Before
Shared data structures were guarded by locks. If the thread holding a lock was descheduled or delayed, every other thread that needed the structure had to wait.
What happened, and who
R. Kent Treiber, at IBM's Almaden Research Center, described the stack in the IBM research report "Systems Programming: Coping with Parallelism" (RJ 5118), dated April 1986. The stack is reachable from one word, the top pointer. A push or pop reads it and installs a new value with compare-and-swap, retrying if another thread changed it first.
Why it mattered
It is the simplest non-blocking data structure, and it is the standard example of the ABA problem, where a pointer is reused and the swap succeeds although the structure has changed. Lock-free free lists and queues in operating systems and language runtimes follow its pattern.
The first commercial MIPS processor, exposing a branch delay slot and a load delay to software.
The story
Before
Earlier processors hid the pipeline from programs: the hardware stalled or interlocked whenever an instruction needed a result that was not ready. That hardware cost cycle time and design effort.
What happened, and who
MIPS Computer Systems, a company formed from John Hennessy's MIPS research project at Stanford, introduced the R2000 in May 1986. It left some pipeline hazards to the compiler: the instruction after a branch always executes (the branch delay slot), and a loaded value cannot be used by the very next instruction.
Why it mattered
The R2000 helped make RISC a commercial architecture, used in workstations from Silicon Graphics and DECstation. Its exposed hazards are the textbook case of moving pipeline management from hardware to compilers, and later processors reversed that choice by resolving hazards in hardware.
"The Design of POSTGRES" sets out an Ingres successor with extensible types, rules and a no-overwrite storage manager.
The story
Before
Ingres, the University of California's relational database of the 1970s, handled business records well. Adding new data types or complex objects was, in its authors' view, very hard because the code base had been modified so much.
What happened, and who
Michael Stonebraker and Lawrence Rowe at Berkeley presented "The Design of POSTGRES" at the ACM SIGMOD conference in Washington in May 1986; it also appeared in SIGMOD Record that June. It describes a system with user-defined types, a rule system, and a storage manager that does not overwrite data.
Why it mattered
POSTGRES became PostgreSQL in the mid-1990s. Its no-overwrite storage, in which an update adds a new version of a row, remains the basis of PostgreSQL's multiversion concurrency control and of the vacuum process that reclaims old versions.
Lets one kernel use several file-system types through a single interface, so a network file system looks like a local disk.
The story
Before
The Unix kernel of the time was written for one on-disk file-system format. Adding a network file system meant a separate path for remote files, and every part of the kernel that touched files knew about the format.
What happened, and who
Steve Kleiman of Sun Microsystems described the design in "Vnodes: An Architecture for Multiple File System Types in Sun UNIX" at the USENIX Summer conference in June 1986. A vnode represents a file to the kernel, and each file-system type supplies its own operations behind it. Sun had introduced the mechanism in SunOS 2.0 in 1985.
Why it mattered
Because the kernel calls one set of operations, the same system calls work on local disks and on NFS, Sun's network file system. Linux's virtual file system layer follows the same idea, which is how it mounts ext4, NFS and FUSE file systems through one API.
Over a path just 400 yards and two IMP hops long, throughput collapsed from 32 Kbps to 40 bps, a factor of about 800.
The story
Before
TCP answered every late acknowledgement with a retransmission, whatever the reason the acknowledgement was late. It sent as fast as the receiver's window allowed, with no sense of how much the network could carry.
What happened, and who
In October 1986, Van Jacobson of Lawrence Berkeley Laboratory and Mike Karels of the University of California, Berkeley, saw throughput between the two sites fall from 32 Kbps to 40 bps. Their path was 400 yards and two IMP (ARPANET packet-switch) hops long. Jacobson traced the collapse to TCP's behaviour, in which retransmissions added load to a network that was already overloaded.
Why it mattered
This was the first measured congestion collapse on the internet. Jacobson's investigation led to the paper "Congestion Avoidance and Control" (1988), which introduced slow start and other mechanisms that keep TCP stable today.
A counter that drains at a steady rate and rejects arrivals when full, used to police how fast and how bursty a sender may be.
The story
Before
A network that sold or guaranteed bandwidth needed a simple way to check that each user stayed within an agreed rate. Averages over long periods let a sender flood the network for a short time.
What happened, and who
Jonathan Turner, at Washington University in St. Louis, described the leaky bucket in "New Directions in Communications (or Which Way to the Information Age?)", in IEEE Communications Magazine, October 1986. Arriving packets fill a bucket that leaks at a fixed rate; a packet that would overflow it is dropped or delayed.
Why it mattered
The idea bounds a sender's rate and burst with two numbers, and it became a basis for traffic policing in ATM networks. Its close relative, the token bucket, is the model behind rate limiters in API gateways and network devices.
A Nature paper showing that hidden units in a multilayer network learn useful representations when errors are propagated backwards.
The story
Before
Networks with hidden layers were known to be more powerful than single-layer perceptrons, but no practical rule trained them. The mathematics of reverse-mode differentiation had appeared before, in Seppo Linnainmaa's work in 1970 and Paul Werbos's in 1974, but it was not in use for neural networks.
What happened, and who
David Rumelhart, Geoffrey Hinton and Ronald Williams published "Learning representations by back-propagating errors" in Nature on 9 October 1986. They show that applying the chain rule from the output back through the layers gives each weight's gradient, and that the hidden units learn internal representations of the task.
Why it mattered
It reopened neural network research. Training every modern deep network still applies this backward pass, which frameworks now automate as automatic differentiation.
An IBM research report on high-performance RISC processors that gives the name superscalar to machines issuing several instructions per cycle.
The story
Before
Most processors started one instruction per clock cycle, so their speed was limited by the clock rate. Some earlier large machines, such as the CDC 6600, overlapped instructions, but the approach had no common name in the RISC literature.
What happened, and who
Tilak Agerwala and John Cocke of IBM's Thomas J. Watson Research Center wrote an IBM research report on high-performance reduced instruction set processors in 1987, in which the term superscalar is used for processors that can start more than one instruction per cycle from an ordinary sequential program.
Why it mattered
Superscalar is the standard description of modern processor cores. They fetch and start several instructions per cycle and check for dependences in hardware, so a program written for one instruction at a time runs faster without changes.
"The Design of the POSTGRES Storage System": no data is ever overwritten, updates become insertions, and a two-bit-per-transaction log records commit state.
The story
Before
Databases updated data in place, so they needed a write-ahead log with undo and redo records to recover after a crash. Recovery meant replaying and reversing log records.
What happened, and who
Michael Stonebraker described the POSTGRES storage manager in "The Design of the POSTGRES Storage System", presented at the VLDB conference in 1987 (a Berkeley technical report appeared in February of that year). An update writes a new version of a row and leaves the old one, and a small log records only whether each transaction committed.
Why it mattered
PostgreSQL's heap of row versions, multiversion concurrency control and vacuum descend directly from this design, and it lets readers see a past state of the data without blocking writers.
Classifies cache misses as compulsory, capacity or conflict, each with a different remedy.
The story
Before
Cache designers counted misses as a single number. A miss rate did not say whether a bigger cache, more associativity or a different block size would help.
What happened, and who
Mark Hill classified misses in his PhD dissertation at the University of California, Berkeley, in 1987. A compulsory miss is the first reference to a block, a capacity miss occurs because the cache is too small for the working set, and a conflict miss occurs because too many blocks map to one set.
Why it mattered
The three categories tell a designer or performance engineer what to change: a larger cache for capacity, higher associativity for conflict. Textbooks and profiling tools still use them.
"Optimization of Nested SQL Queries Revisited" shows that Kim's transformation gives wrong answers for COUNT and repairs it with an outer join.
The story
Before
Won Kim's 1982 algorithm rewrote a nested SQL query into a join, which is often faster to execute. For a subquery that computes a COUNT, the join dropped outer rows with no matches, though their count should have been zero.
What happened, and who
Richard Ganski and Harry Wong presented "Optimization of Nested SQL Queries Revisited" at the ACM SIGMOD conference in May 1987. They showed the error and repaired it by using an outer join, which keeps the unmatched rows so that their count can be zero.
Why it mattered
The count bug is the classic example of an optimiser rewrite that is right for most inputs and wrong for one. Query optimisers still have to handle it when they decorrelate subqueries.
A priority queue that lazily postpones tidying so that decrease-key costs O(1) amortised.
The story
Before
Binary heaps cost O(log n) for each decrease-key, which dominates Dijkstra's shortest-path and Prim's spanning-tree algorithms on dense graphs. Vuillemin's binomial queues had improved on other operations but not this one.
What happened, and who
Michael Fredman and Robert Tarjan published "Fibonacci heaps and their uses in improved network optimization algorithms" in the Journal of the ACM in July 1987. The structure delays consolidation until a delete-minimum, which gives constant amortised time for insertion and decrease-key.
Why it mattered
It improved the asymptotic bounds for shortest paths and minimum spanning trees. In practice its constants are large, so simpler heaps are often faster, but it is the standard example of amortised analysis meeting graph algorithms.
Randomised gossip for keeping replicated databases consistent, devised after a name service overloaded a network.
The story
Before
The Clearinghouse name service at Xerox replicated its database on servers across the company's internet, and each update was mailed to every server. As the network grew, this deterministic scheme generated a heavy load and left replicas inconsistent.
What happened, and who
Alan Demers and eight colleagues at Xerox's Palo Alto Research Center presented "Epidemic Algorithms for Replicated Database Maintenance" at the ACM Symposium on Principles of Distributed Computing in August 1987. They replaced the mailing with randomised exchanges between pairs of servers, analysed with the mathematics of epidemics.
Why it mattered
Gossip spreads an update to all replicas with high probability, without a central coordinator and despite failures. Cassandra and other distributed databases use it to share membership and repair replicas.
"Hashed and Hierarchical Timing Wheels" gives O(1) timer start and stop for protocol timers.
The story
Before
Timer facilities kept timers in a sorted list or a heap, so starting one cost O(n) or O(log n). A network stack with thousands of retransmission timers spent noticeable time on them.
What happened, and who
George Varghese and Tony Lauck of Digital Equipment Corporation presented "Hashed and Hierarchical Timing Wheels" at the ACM Symposium on Operating Systems Principles in Austin, Texas, in November 1987. A circular array of slots is indexed by expiry time, so starting or stopping a timer is a constant-time insertion or removal.
Why it mattered
Timer wheels handle large numbers of timers cheaply. Variants are used in the Linux kernel's timers, in Apache Kafka's delay queue and in many network stacks.
"Reimplementing the Cedar file system using logging and group commit" journals file-system metadata.
The story
Before
After a crash, a Unix-style file system had to run a consistency check such as fsck, which scanned the whole disk. The time grew with the disk's size.
What happened, and who
Robert Hagmann, at Xerox's Palo Alto Research Center, presented "Reimplementing the Cedar file system using logging and group commit" at the ACM Symposium on Operating Systems Principles in November 1987. It records changes to the file system's metadata in a log and batches several updates into one log write, called group commit.
Why it mattered
Replaying a log after a crash takes time that depends on the log rather than the disk. Journaling, in various forms, is used by ext3 and ext4, XFS and NTFS.
The Head-of-Line Blocking Limit of Input-Queued Switches
Computes the throughput ceiling of head-of-line blocking: input-queued switches saturate at 2-√2, about 58.6%.
The story
Before
A simple input-queued crossbar switch keeps one first-in-first-out queue at each input. It was assumed to approach full line rate under load, as long as the crossbar itself could carry it.
What happened, and who
Mark Karol, Michael Hluchyj and Samuel Morgan published "Input versus output queueing on a space-division packet switch" in IEEE Transactions on Communications in December 1987. A packet at the front of a queue that waits for a busy output blocks the packets behind it, even those destined for idle outputs, and under uniform random traffic this limits throughput to 2-√2 for large switches.
Why it mattered
The result led designers to use virtual output queues, one per output at each input, with a scheduler that matches inputs to outputs, as in the iSLIP algorithm. High-speed routers and switches are built this way.
Represents a joint probability distribution as a graph and gives a graphical test for conditional independence.
The story
Before
Rule-based expert systems handled uncertainty with ad hoc numbers, and a full joint distribution over many variables is too large to write down or reason with.
What happened, and who
Judea Pearl, at the University of California, Los Angeles, published the book Probabilistic Reasoning in Intelligent Systems in 1988. It presents Bayesian networks, in which a directed graph shows which variables depend on which, and d-separation, a test on the graph for whether variables are conditionally independent.
Why it mattered
The graph makes the joint distribution compact and shows how to compute with it. Bayesian networks became the base of probabilistic graphical models, used in diagnosis, spam filtering and causal reasoning.
Two 1988 ideas for replicated systems: a protocol that keeps copies agreeing through failures, and clocks that record which events could have influenced which.
The story
Before
Replicated services needed a way to keep copies in step when nodes or messages failed, and few practical protocols existed. Lamport's logical clocks of 1978 ordered events consistently with causality, but a smaller timestamp did not tell you whether one event could have influenced the other.
What happened, and who
Brian Oki and Barbara Liskov, at MIT, presented "Viewstamped Replication: A New Primary Copy Method to Support Highly-Available Distributed Systems" at the ACM Symposium on Principles of Distributed Computing in August 1988. In the same year Colin Fidge and, independently, Friedemann Mattern published vector clocks, a timestamp with one counter per process that can show whether two events are ordered or concurrent.
Why it mattered
Viewstamped Replication is an early protocol of the family that includes Raft, and Liskov and James Cowling later revisited it. Vector clocks are how systems in the style of Dynamo detect conflicting versions of a value.
A case that arrays of small, cheap disks with redundancy could beat one big expensive disk on speed, cost and reliability.
The story
Before
Mainframes stored data on a single large expensive disk, such as the IBM 3380, which the paper calls a SLED. Disk performance was improving far more slowly than CPU speed, and one disk failure lost its data. Cheap small disks built for personal computers were catching up in capacity per dollar, but a lone small disk was too unreliable to trust.
What happened, and who
David Patterson, Garth Gibson and Randy Katz at UC Berkeley presented "A Case for Redundant Arrays of Inexpensive Disks (RAID)" at the ACM SIGMOD conference in Chicago in June 1988. They showed that an array of many small disks, with a share of the space spent on mirroring or parity, could offer more throughput per dollar than a SLED. They defined five organisations, later called RAID levels 1 to 5.
Why it mattered
The paper's arithmetic is still how storage is sized: the failure rate of an array is the failure rate of one disk multiplied by the number of disks, so redundancy is what makes many disks safe. RAID levels, and the mirroring and parity ideas behind them, appear in hardware controllers, Linux md, ZFS and in the erasure coding of cloud storage.
The fixes to TCP that stopped the Internet collapsing under load: slow start, congestion avoidance and a better retransmit timer.
The story
Before
In October 1986 the Internet had the first of a series of congestion collapses: throughput between Lawrence Berkeley Laboratory and UC Berkeley, 400 yards apart, fell from 32 Kbps to 40 bps. The 4.3BSD TCP of the time sent a full window as soon as a connection opened and retransmitted on a timer built from a fixed multiple of the average round-trip time. When a link filled, delays grew, timers fired early, and the resulting retransmissions made the congestion worse.
What happened, and who
Van Jacobson of Lawrence Berkeley Laboratory and Michael Karels of UC Berkeley presented "Congestion Avoidance and Control" at ACM SIGCOMM in August 1988. It added seven algorithms to 4BSD TCP, among them slow start, congestion avoidance, round-trip-time variance estimation and exponential retransmit backoff, all rooted in a principle of packet conservation: do not put a new packet into the network until an old one has left.
Why it mattered
Every widely used TCP since descends from this paper, and the congestion window it introduced is still the quantity a sender adjusts to share a link fairly. It turned congestion control from something a network did to something each end host did, which is why the Internet scaled without a central controller.
Updates a prediction toward the next prediction, without waiting for the final outcome.
The story
Before
Monte Carlo methods had to wait until an episode ended before they could learn from it. Dynamic programming updated estimates from other estimates but needed a model of the environment.
What happened, and who
Richard Sutton published "Learning to Predict by the Methods of Temporal Differences" in the journal Machine Learning in August 1988. TD(0) moves each prediction toward the reward received plus the next prediction, so it learns step by step from experience.
Why it mattered
Temporal-difference learning is the base of Q-learning and of the value-learning methods used in reinforcement learning, including those in game-playing systems such as TD-Gammon.
Garbage collection for unmodified C programs, by treating any word that could be a pointer as one.
The story
Before
Garbage collectors then needed help from the language or compiler: type information or tagged values told them which words in memory were pointers. C programs had neither, so their authors freed memory by hand, with leaks and dangling pointers as the price of mistakes.
What happened, and who
Hans Boehm and Mark Weiser published "Garbage collection in an uncooperative environment" in Software: Practice and Experience in September 1988. Their collector scans the stack, registers and heap and treats every word that looks like a heap address as a pointer, so it can find live objects in a program compiled without any cooperation. A program could be relinked with it rather than recompiled.
Why it mattered
Conservative scanning showed that automatic memory management does not require a cooperating language, so it can be added to C and C++ programs and to runtimes that have no precise map of their pointers. The Boehm collector was used in real systems such as the GNU compiler's Java runtime, and the idea of a conservative stack scan survives in several engines today.
A cost model that counts block transfers between memory and disk, with a proved lower bound for external sorting.
The story
Before
Algorithm analysis used the RAM model, which charges one unit for every memory access. That is wrong when data lives on disk, where reading one block costs about as much as reading a few bytes and a random access costs thousands of times more than a sequential one. An algorithm with a good operation count could still be far from the best on disk.
What happened, and who
Alok Aggarwal and Jeffrey Vitter published "The Input/Output Complexity of Sorting and Related Problems" in Communications of the ACM in September 1988. They gave a machine with a memory of size M that moves data in blocks of B items, counted only block transfers, and proved a lower bound for sorting N items with matching upper bounds from external merge sort and distribution sort.
Why it mattered
The I/O model gave database and storage designers a way to argue that one algorithm beats another on disk, and it is the basis of the analysis of B-trees, external sorting and joins. Its block size B reappears in cache-aware and cache-oblivious analysis of memory hierarchies.
A short note describes a compiler with special authority, tricked by a caller-supplied output name into overwriting a billing file.
The story
Before
Access control lists tie authority to who is running a program, not to which request it is serving. A program with broad authority used it whenever it acted, for whichever user had called it.
What happened, and who
Norm Hardy published "The Confused Deputy (or why capabilities might have been invented)" in ACM Operating Systems Review in October 1988. A compiler on a time-sharing system was allowed to write a statistics file. A user asked it to write its debugging output to a billing file, and the compiler, using its own authority, did.
Why it mattered
The confused-deputy problem motivated capability systems, where a name and the right to use it travel together. It also describes setuid programs and cross-site request forgery, in which a browser is the deputy.
A self-copying program that used a buffer overflow, a mail-server debug hole and weak passwords to spread across the early Internet in a night.
The story
Before
Networked Unix systems trusted their neighbours: services such as finger and sendmail accepted input from any host and were written in C without bounds checks. Buffer-overflow bugs were known, but nobody had used them at scale, and no organisation coordinated the response to an attack on many sites at once.
What happened, and who
On the evening of 2 November 1988 Robert Tappan Morris, a Cornell graduate student, released a worm from a computer at MIT. It guessed passwords, used a debug mode in sendmail, and overflowed a buffer in the finger daemon on 4BSD systems, then copied itself onward. A bug made it reinfect machines repeatedly, so affected hosts slowed to a crawl; estimates of the number infected range from a couple of thousand to about six thousand.
Why it mattered
It was the first Internet worm to become widely known, and it made memory-safety bugs a security matter rather than a reliability one. DARPA responded by funding the CERT Coordination Center at Carnegie Mellon, the model for incident-response teams since, and Morris was the first person convicted under the 1986 Computer Fraud and Abuse Act.
A geometric analysis showing that additive increase with multiplicative decrease converges to a fair and efficient share of a link.
The story
Before
Congestion control had been designed by experiment. Jacobson's TCP fixes of 1988 worked, and DEC had its own scheme in which routers set a bit to signal congestion, but nobody had shown which increase and decrease rules would reliably share a link among competing senders.
What happened, and who
Dah-Ming Chiu and Raj Jain, at Digital Equipment Corporation, published "Analysis of the Increase and Decrease Algorithms for Congestion Avoidance in Computer Networks" in Computer Networks and ISDN Systems in 1989. They drew each pair of senders' rates as a point in a plane, with lines for full use of the link and for equal shares, and showed that only additive increase with multiplicative decrease moves toward the point where both hold.
Why it mattered
It gave the theory behind TCP's rule of increasing the window by a fixed amount and halving it on loss, and it explains why senders using it converge toward equal shares. Later congestion controllers such as CUBIC and BBR are judged against the same two goals of efficiency and fairness.
A small neural network that steered a real van from camera images, learning to drive by imitation.
The story
Before
Autonomous vehicles before it relied on hand-built vision, in which programmers wrote rules to find road edges and lines and then computed a steering command. Such systems were fragile when lighting or road surface changed, because the rules had to anticipate every case.
What happened, and who
Dean Pomerleau at Carnegie Mellon described ALVINN, "An Autonomous Land Vehicle in a Neural Network", in the first volume of the NIPS proceedings, published in 1989. A network with one hidden layer took a low-resolution camera image and produced a steering direction, and it steered the Navlab vehicle on real roads. The first version learned from simulated road images; later versions learned from watching a person drive.
Why it mattered
ALVINN was the first serious system to learn to drive by copying demonstrations, and it exposed the weakness of that approach: a network trained only on good driving has never seen the situations it drifts into. That distribution-shift problem is the reason later imitation-learning methods train on the states the learner itself visits.
Time-limited grants that let clients cache files safely, because a grant from a vanished client simply expires.
The story
Before
A file server that let clients cache data had to tell those clients when a file changed, or stop them from writing. If a client crashed or became unreachable while holding such a promise, the server could not tell it from a slow one, so it either waited forever or broke consistency.
What happened, and who
Cary Gray and David Cheriton at Stanford introduced leases in a paper at the ACM Symposium on Operating Systems Principles in 1989. A lease is a promise from the server that lasts for a fixed time, so the server can wait out the term before changing the file if the client cannot be reached. Their analysis showed that short terms were enough to keep most of the benefit of caching.
Why it mattered
Leases turned an unanswerable question, whether a client is dead, into a question about clocks, and they became a standard tool for distributed coordination. Google's Chubby lock service grants its clients leases on their sessions, and Kubernetes uses lease objects for leader election and node heartbeats.
A method that learns the value of every action in every state and converges to the best policy while behaving differently.
The story
Before
Temporal-difference learning, introduced by Richard Sutton in 1988, learned the value of the policy the agent was actually following. To learn the value of a better policy, an agent had to switch to following it, and there was no proof that it would find the best one without doing so.
What happened, and who
Chris Watkins described Q-learning in his 1989 Cambridge PhD thesis, "Learning from Delayed Rewards", and Watkins and Peter Dayan published the convergence proof in Machine Learning in 1992. Its update moves the estimate for a state and action toward the reward plus the value of the best action in the next state, whatever action the agent took next, so it learns the optimal values from any exploration.
Why it mattered
Learning from behaviour that is not the policy being evaluated, called off-policy learning, is what lets an agent reuse stored experience. Q-learning with a neural network in place of the table is the Deep Q-Network that DeepMind used to play Atari games in 2013 and 2015.
The path-vector protocol by which independently run networks tell each other which addresses they can reach, sketched on two napkins.
The story
Before
The Exterior Gateway Protocol, EGP, connected the networks of the early Internet, but it assumed they formed a tree. As the Internet grew into a mesh of networks run by different organisations, EGP could not express a choice between routes or avoid loops.
What happened, and who
Yakov Rekhter, Kirk Lougheed and Len Bosack sketched the design of the Border Gateway Protocol on napkins at the IETF meeting in Austin, Texas, in January 1989, and the first specification, RFC 1105, followed in June 1989. Each router announces the list of networks, the AS path, a route has crossed, so a router can reject a route that already contains its own network. BGP version 4 followed in 1994 and 1995.
Why it mattered
BGP is the protocol that joins the tens of thousands of autonomous systems that make up the Internet, so a mistake in its announcements can take down a country or a large service. It lets each operator choose routes by policy rather than only by distance.
The tutorial that became the standard reference for hidden Markov models and their algorithms.
The story
Before
The mathematics of hidden Markov models had been worked out by Leonard Baum and colleagues in the late 1960s and used in speech recognition at IBM and Carnegie Mellon in the 1970s. It was spread across specialist papers in different notations, and engineers who wanted to build a recogniser had to piece it together.
What happened, and who
Lawrence Rabiner of AT&T Bell Laboratories published "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition" in the Proceedings of the IEEE in February 1989. It explains the model and its three problems, evaluating a sequence, finding the best hidden path and training the parameters, together with the forward-backward, Viterbi and Baum-Welch algorithms.
Why it mattered
For a generation it was the paper people learned hidden Markov models from, and HMMs were the core of speech recognisers until neural networks replaced them. The same model and algorithms are still used for sequence labelling in speech, biology and signal processing.
A stateless network file system protocol, written down as an RFC so that other vendors could implement it.
The story
Before
Before network file systems, sharing files between machines meant copying them with tools such as rcp and ftp, so each machine held its own version. Early remote file systems tended to keep state on the server, which made recovery after a crash complicated.
What happened, and who
Sun Microsystems had shipped NFS in 1984, and in March 1989 it published version 2 as RFC 1094, "NFS: Network File System Protocol Specification". Each request carries everything the server needs, including a file handle and an offset, so the server keeps no record of its clients between requests.
Why it mattered
Because the server remembers nothing, it can restart after a crash and clients simply retry their requests, which made NFS simple and robust. It became the standard network file system on Unix, and version 3 (RFC 1813, June 1995) and version 4 still run in data centres today.
How a multiprocessor keeps its address-translation caches consistent: interrupt the other processors so they discard stale entries.
The story
Before
Each CPU caches recent virtual-to-physical translations in its translation lookaside buffer, or TLB. Multiprocessors of the time were built from off-the-shelf microprocessors that had no hardware to keep TLBs consistent, so when the operating system changed a page mapping, other CPUs could go on using the old one.
What happened, and who
David Black, Richard Rashid, David Golub, Charles Hill and Robert Baron at Carnegie Mellon, working on the Mach operating system, presented "Translation Lookaside Buffer Consistency: A Software Approach" at the third ASPLOS conference in April 1989. The processor changing a mapping interrupts the others and makes them flush the affected entries before it goes on.
Why it mattered
This technique, now called a TLB shootdown, is still how Linux and other kernels invalidate mappings on multiple cores, and it is why unmapping memory in a multithreaded program costs more as the core count grows. Its cost shapes the design of memory allocators and of huge-page and batching optimisations.
The first release of the POSTGRES research database to people outside the project that built it.
The story
Before
POSTGRES was Michael Stonebraker's research successor to the Ingres database at Berkeley, built to try out extensible types and rules. Until then it was a prototype run by the group that wrote it.
What happened, and who
The PostgreSQL project's history records that version 1 was released to a few external users in June 1989. Berkeley went on to make further releases until version 4.2 in 1994, after which the project ended.
Why it mattered
That release started the line of code that became PostgreSQL: in 1994 and 1995 two students added an SQL interpreter, and volunteers took over the code and named it PostgreSQL in 1996. It is now one of the most widely used relational databases.
A way for a router to share a link fairly among flows by serving them as if it sent one bit from each in turn.
The story
Before
Routers, then called gateways, queued packets first-come first-served, so a sender that transmitted aggressively took most of the link and starved the rest. Nagle's round-robin among sources, proposed in 1985, gave each source a turn per packet, which favoured senders using large packets.
What happened, and who
Alan Demers, Srinivasan Keshav and Scott Shenker presented "Analysis and Simulation of a Fair Queueing Algorithm" at ACM SIGCOMM in Austin in September 1989. They defined an ideal that serves the flows one bit at a time in round-robin order, and a packet-by-packet algorithm that approximates it by sending packets in order of the time they would finish under the ideal.
Why it mattered
Fair queueing gave a precise meaning to fairness on a shared link. Its descendants include weighted fair queueing, deficit round robin used in switches and routers, and the fq_codel scheduler that Linux uses to control latency on home and data-centre links.
Two memory-management ideas: freeing whole groups of objects at once by lifetime, and a write barrier that lets a collector work while the program runs.
The story
Before
Freeing objects one by one costs time for each object and makes it easy to leak or free early. Incremental garbage collectors, which mark objects while the program is still changing them, could miss an object if the program hid a reference from the marker.
What happened, and who
David Hanson published "Fast Allocation and Deallocation of Memory Based on Object Lifetimes" in Software: Practice and Experience in 1990, describing arenas that hand out memory cheaply and free everything together. In the same year Taiichi Yuasa published a snapshot-at-the-beginning barrier in the Journal of Systems and Software, which records any reference the program overwrites during marking so that the object it pointed to is still seen.
Why it mattered
Arenas are used in compilers and in many servers to free a request's memory in one step. Snapshot-at-the-beginning barriers are used in modern concurrent collectors, such as G1 in the Java virtual machine.
A consensus protocol told as the story of a Greek parliament, which reviewers found baffling and which later became the standard way to reach agreement.
The story
Before
Getting a group of machines that can crash to agree on a value was known to be hard: the FLP result of 1985 had shown that no protocol can guarantee agreement in a fully asynchronous system with even one crash. Viewstamped replication, by Brian Oki and Barbara Liskov in 1988, had solved a similar problem, but it was little known.
What happened, and who
Leslie Lamport wrote "The Part-Time Parliament", in which a Greek island's legislators keep consistent records though they wander in and out, and submitted it in 1990. The referees found the framing a distraction, and it was published only in 1998 in ACM Transactions on Computer Systems. Lamport restated the protocol plainly in "Paxos Made Simple" in 2001.
Why it mattered
Paxos became the reference algorithm for consensus, and Google's Chubby lock service and Spanner database build on it. Raft, published in 2014, was designed as a more teachable alternative to it.
Protocols that bound how long a high-priority task waits on a lock held by a low-priority one.
The story
Before
In a real-time system a high-priority task can block on a lock held by a low-priority task. If medium-priority tasks preempt the lock holder, the high-priority task waits as long as they run, so its wait has no bound and it can miss its deadline.
What happened, and who
Lui Sha, Ragunathan Rajkumar and John Lehoczky at Carnegie Mellon introduced the priority inheritance and priority ceiling protocols in "Priority Inheritance Protocols: An Approach to Real-Time Synchronization", in IEEE Transactions on Computers in 1990. Under inheritance the lock holder runs at the priority of the highest task waiting for the lock.
Why it mattered
The fix was applied to the Mars Pathfinder probe in 1997, when a priority inversion kept resetting its computer and the team enabled priority inheritance on the lock. It is now an option on POSIX mutexes and a standard feature of real-time operating systems.
A small buffer that keeps recently evicted cache lines, so that a line thrown out by a conflict can be recovered cheaply.
The story
Before
A direct-mapped cache is fast and simple, but two addresses that map to the same line keep evicting each other. Making the cache set-associative removed these conflict misses but slowed every access.
What happened, and who
Norman Jouppi, at Digital Equipment Corporation's Western Research Laboratory, proposed the victim cache in "Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers" at the International Symposium on Computer Architecture in 1990. A few fully associative entries hold lines just evicted from the main cache, and a hit there swaps the line back.
Why it mattered
It showed that a tiny structure can remove most conflict misses without slowing hits. Victim caches survive in real processors: the L3 cache in AMD's Zen processors is filled by lines evicted from the L2 cache.
An open/next/close interface for query operators, plus an exchange operator that adds parallelism without changing the operators.
The story
Before
Query executors were written for particular plan shapes, and each operator had its own way to hand rows to the next. Adding parallelism meant changing the operators, so every operator had to be rewritten to run on several processors.
What happened, and who
Goetz Graefe presented "Encapsulation of Parallelism in the Volcano Query Processing System" at the ACM SIGMOD conference in May 1990. Every operator implements the same three calls, open, next and close, so a plan is a tree of operators that each pull rows from their children. An exchange operator can be placed anywhere in the tree to pass rows between processes.
Why it mattered
The pull-based iterator model is how most row-at-a-time executors work, including PostgreSQL's. The exchange operator is the ancestor of the Gather and repartition steps in today's parallel and distributed query engines.
An ordered structure built from layered linked lists whose levels are chosen by coin flips, as an alternative to balanced trees.
The story
Before
Ordered data was kept in balanced trees such as AVL and red-black trees, which stay balanced by rotating nodes on insertion and deletion. The rotation code is intricate and hard to make concurrent.
What happened, and who
William Pugh of the University of Maryland published "Skip Lists: A Probabilistic Alternative to Balanced Trees" in Communications of the ACM in June 1990. Each element is given a random height by repeated coin flips and is linked into that many levels, so a search can skip along the top levels and drop down; the expected cost of search, insertion and deletion is logarithmic.
Why it mattered
Skip lists are used where simple code and easy concurrency matter: Redis uses them for sorted sets, LevelDB and RocksDB for their in-memory tables, and Java has ConcurrentSkipListMap.
A correctness condition for concurrent objects: every operation appears to take effect at a single instant between its call and its return.
The story
Before
Earlier definitions of atomic operations on shared objects were vague about when an operation began and ended. Sequential consistency required a single order of operations but ignored real time, so an object could return results that seemed to come from the past.
What happened, and who
Maurice Herlihy and Jeannette Wing defined linearizability in "Linearizability: A Correctness Condition for Concurrent Objects" in ACM Transactions on Programming Languages and Systems in July 1990. An execution is linearizable if each operation can be given a point between its call and its return so that the results match a sequential execution in that order.
Why it mattered
Linearizability is what engineers mean by strong consistency, and stores advertise it for reads and writes to a single key. Tools such as Jepsen test real systems by checking whether the histories they produce could be linearized, and it is the property against which weaker consistency models are described.
A web page describing the World Wide Web project itself, served from a computer at CERN.
The story
Before
Hypertext already existed, in systems such as Ted Nelson's Xanadu vision, Doug Engelbart's NLS and Apple's HyperCard, but it linked documents on one machine or one network. There was no common way to link documents held on different computers across the Internet.
What happened, and who
Tim Berners-Lee at CERN, the European physics laboratory near Geneva, put the first web server and site online on 20 December 1990. Its pages described the project itself. On 6 August 1991 he announced a summary of the project on the newsgroup alt.hypertext, inviting others to take part.
Why it mattered
That first site demonstrated the combination of URLs, HTTP and HTML on which every website since is built. The announcement of August 1991 brought in the people who wrote the first browsers and servers elsewhere.
Caches at each call site that remember several receiver types, making method dispatch fast in dynamic object-oriented languages.
The story
Before
In languages such as Smalltalk and Self, the method run by a call depends on the type of the object at run time, so each call needs a lookup. An inline cache, introduced by Peter Deutsch and Allan Schiffman for Smalltalk in 1984, remembers the last type seen at a call site, but it thrashes when a site sees several types.
What happened, and who
Urs Hölzle, Craig Chambers and David Ungar presented "Optimizing Dynamically-Typed Object-Oriented Languages With Polymorphic Inline Caches" at ECOOP in 1991, from the Self project at Stanford. A call site gets a small stub that checks a handful of remembered types in turn before falling back to a full lookup.
Why it mattered
Inline caches, and polymorphic ones in particular, are how JavaScript engines such as V8 speed up property access on objects. They are one reason dynamic languages run fast enough to be used for large systems.
Branch prediction that picks its counter by recent branch history, so it can learn repeating patterns.
The story
Before
Processors guessed a branch's direction with a small saturating counter per branch, an idea of James Smith from 1981. Such a counter records only how often a branch was taken and cannot learn patterns such as alternating outcomes or a branch that depends on the one before it.
What happened, and who
Tse-Yu Yeh and Yale Patt of the University of Michigan presented "Two-Level Adaptive Training Branch Prediction" at the International Symposium on Microarchitecture in 1991. A first-level register records the recent outcomes of branches, and that history selects a counter in a second-level table.
Why it mattered
Using history to choose the prediction is the basis of the predictors in modern processors, including gshare and the more elaborate designs that followed. Intel's Pentium Pro of 1995 used a two-level predictor, and accurate prediction is what makes deep pipelines worthwhile.
A theory that ranks hardware synchronisation primitives by what they can build: compare-and-swap can build anything wait-free, plain reads and writes cannot.
The story
Before
Concurrent data structures were built with locks, or by trial and error with atomic instructions. Nobody could say which instructions a processor had to provide, or whether one instruction could do what another did.
What happened, and who
Maurice Herlihy published "Wait-Free Synchronization" in ACM Transactions on Programming Languages and Systems in January 1991. A wait-free operation finishes in a bounded number of its own steps whatever the other threads do. He gave each primitive a consensus number, the number of threads that can use it to agree on a value: read/write registers have 1, test-and-set has 2, and compare-and-swap is universal.
Why it mattered
The result explains why processors provide compare-and-swap or load-linked/store-conditional, and why locks cannot be replaced by reads and writes alone. It underlies lock-free and wait-free data structures in language runtimes and kernels.
A spin lock in which each waiting thread spins on its own memory location, so waiting does not flood the cache-coherence system.
The story
Before
Test-and-set locks and ticket locks make every waiting thread read one shared word. Each release invalidates that word in every waiter's cache, so traffic on the memory system grows with the number of waiters and slows even the thread holding the lock.
What happened, and who
John Mellor-Crummey and Michael Scott of the University of Rochester published "Algorithms for Scalable Synchronization on Shared-Memory Multiprocessors" in ACM Transactions on Computer Systems in February 1991. Waiters join a linked queue, each spinning on a flag in its own queue node, and the releaser sets the flag of the next node.
Why it mattered
The MCS lock gives each waiter a private line and grants the lock in order, so its cost stays flat as cores are added. Linux's queued spinlock, used since version 4.2 in 2015, is built on it.
A free Unix-like kernel, announced as "just a hobby", that became the one under most of the world's servers.
The story
Before
Unix was AT&T's licensed product. MINIX, Andrew Tanenbaum's small Unix-like teaching system, came with its source but could not be freely redistributed. The GNU project had been writing a free Unix since 1983 and had its compiler and tools, but not a finished kernel.
What happened, and who
On 25 August 1991 Linus Torvalds, a student at the University of Helsinki, posted to a newsgroup for MINIX: "I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones." Linux contained no Unix code; it implemented the Unix interface anew.
Why it mattered
Because Linux followed POSIX closely, programs written for Unix could be ported to it, and together with GNU's tools it made a complete free Unix-like system anyone could change. It is now the kernel under Android phones, most cloud servers and the containers that Docker starts.
A file system that writes everything sequentially to a log and reclaims space by cleaning old segments in the background.
The story
Before
Disk seeks dominated small writes, and Unix file systems such as the Berkeley Fast File System scattered metadata and data across the disk. Creating a small file took several separate writes at different places, so disk bandwidth was mostly wasted on seeking.
What happened, and who
Mendel Rosenblum and John Ousterhout at UC Berkeley presented "The Design and Implementation of a Log-Structured File System" at the ACM Symposium on Operating Systems Principles in October 1991, describing Sprite LFS, and published the journal version in 1992. It buffers changes in memory and writes them to disk in large sequential segments; a cleaner then copies live data out of mostly-empty segments.
Why it mattered
Writing only sequentially, and cleaning later, is now used in flash file systems such as F2FS, in the translation layer inside SSDs, and in the log-structured merge trees behind many databases. It also raised the cost of cleaning as the central question of the design.
A neural network that taught itself backgammon by playing against itself, reaching close to the level of the best human players.
The story
Before
Backgammon programs relied on evaluation functions written by hand, or on networks trained on positions that experts had scored. Tesauro's earlier program Neurogammon was of the second kind, so it could not rise above the quality of its teachers' scores.
What happened, and who
Gerald Tesauro of IBM's Thomas J. Watson Research Center trained TD-Gammon by temporal-difference learning, with the network's weights adjusted so that each position's value moved toward that of the position after it. His paper "Practical Issues in Temporal Difference Learning" appeared in Machine Learning in 1992. By 1993 version 2.1, trained on about 1.5 million self-play games, was a little below the best human players.
Why it mattered
It was an early demonstration that temporal-difference learning with a neural network could reach expert play from nothing but the rules and the result of each game. Learning by self-play is the method behind later systems such as AlphaGo and AlphaZero.
A transaction recovery method that repeats history to rebuild the crashed state, then undoes unfinished work, on top of write-ahead logging.
The story
Before
Earlier recovery schemes either forced modified pages to disk at commit, which was slow, or could not combine fine-grained locking with rolling back part of a transaction. They struggled when many transactions were active at a crash.
What happened, and who
C. Mohan, Don Haderle, Bruce Lindsay, Hamid Pirahesh and Peter Schwarz of IBM published "ARIES: A Transaction Recovery Method Supporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging" in ACM Transactions on Database Systems in March 1992. After a crash it runs three passes over the log: analysis, redo, which repeats history exactly, and undo, which reverses unfinished transactions and records each reversal in the log.
Why it mattered
ARIES is the reference design for recovery in commercial engines, and most undo-based systems follow its structure. PostgreSQL's log needs only redo, and its lessons contrast it with ARIES to show what that choice removes.
A 32-bit, 33 MHz expansion bus giving about 133 MB/s, which became the standard way to attach hardware to a computer.
The story
Before
Personal computers used competing expansion buses: the old ISA and its extension EISA, IBM's proprietary Micro Channel, and the VESA Local Bus. ISA was slow, and each faster alternative had drawbacks in cost, licensing or the number of cards it could carry.
What happened, and who
Intel released version 1.0 of the Peripheral Component Interconnect specification on 22 June 1992. A 32-bit bus at 33 MHz can move about 133 MB/s and lets cards be configured automatically. An industry group, the PCI-SIG, later took over maintenance of the standard.
Why it mattered
PCI became the common expansion bus for a decade, and its software model of bus, device and function numbers and configuration space survives in PCI Express, the interconnect used by graphics cards, network cards and NVMe drives today. Operating systems still enumerate devices by walking it.
Maximum-margin classifiers made nonlinear by kernels, which let a linear method draw curved boundaries.
The story
Before
Linear classifiers found a straight boundary between classes and could not separate data that needed a curved one. Mapping the data into a richer space by hand made the computation expensive, and the resulting classifiers overfitted.
What happened, and who
Bernhard Boser, Isabelle Guyon and Vladimir Vapnik, at AT&T Bell Laboratories, presented "A Training Algorithm for Optimal Margin Classifiers" at the Workshop on Computational Learning Theory in July 1992. They combined Vapnik's maximum-margin classifier with the kernel trick, computing similarity in a high-dimensional space without visiting it.
Why it mattered
Support vector machines, extended to noisy data by Corinna Cortes and Vapnik in 1995, were among the leading machine-learning methods until deep networks displaced them on large data around 2012. The kernel idea persists in Gaussian processes and other kernel methods.
A small register-based bytecode filter run inside the kernel, making packet capture up to 100 times faster than the interface before it.
The story
Before
Tools that watched network traffic ran in user space, so every packet had to be copied across the kernel boundary. Filters such as the CMU/Stanford Packet Filter of 1980 discarded unwanted packets early, but they were designed for a 64 KB PDP-11 and used a stack-based evaluator that suited RISC processors poorly. Sun's NIT interface copied and buffered inefficiently.
What happened, and who
Steven McCanne and Van Jacobson of Lawrence Berkeley Laboratory presented "The BSD Packet Filter: A New Architecture for User-Level Packet Capture" at the USENIX Winter conference in January 1993. Its filter language is a small register-based virtual machine that the kernel runs on each packet. The register-based evaluator is up to 20 times faster than the earlier design, and BPF overall up to 100 times faster than NIT on the same hardware.
Why it mattered
Running a checked bytecode program inside the kernel is the pattern that became eBPF, which Linux now uses for tracing, networking and security. tcpdump still compiles its filter expressions to BPF.
The first superscalar x86 processor, able to start two instructions in a single cycle.
The story
Before
The Intel 486 issued at most one instruction per cycle, even though it was pipelined. To run faster, Intel could raise the clock speed or find a way to start more than one instruction at once.
What happened, and who
Intel introduced the Pentium on 22 March 1993. It had two integer pipelines, so it could issue two instructions per cycle when they were independent, and separate 8 KB caches for instructions and data. It was first sold at 60 and 66 MHz.
Why it mattered
The Pentium brought superscalar execution to mass-market computers, and later x86 processors added more units and out-of-order execution. Today's cores issue several instructions per cycle, and software performance depends on how much independent work the processor can find.
A three-line patch that made Linux's load average count tasks stuck waiting on I/O, not only tasks wanting the CPU.
The story
Before
Unix load average was the average number of processes running or waiting to run. Linux's version counted only those, so a machine whose processes waited on a slow swap disk showed a low load even though it was struggling.
What happened, and who
On 29 October 1993 Matthias Urlichs posted a patch that also counted tasks in uninterruptible sleep, usually waiting for disk. It was included in Linux 0.99.14.
Why it mattered
It changed Linux's load average from a measure of CPU demand into a measure of demand for system resources, which still surprises operators who see a high load on an idle CPU. Reading it correctly means asking whether tasks are waiting on CPU or on I/O.
Many servers share one address and the network delivers each packet to the nearest one.
The story
Before
Users who wanted a nearby copy of a service had to find its servers on a list and pick one themselves. RFC 1546 gives the example of choosing the closest archie file-search server by hand.
What happened, and who
Craig Partridge, Trevor Mendez and Walter Milliken of BBN published RFC 1546, "Host Anycasting Service", in November 1993. It proposed that several hosts share an anycast address and that a datagram sent to it be delivered to at least one of them, usually the closest.
Why it mattered
Anycast is now deployed by announcing the same address from many sites through BGP, rather than through the new address class the RFC proposed. Root DNS servers, public resolvers such as 8.8.8.8 and 1.1.1.1, and content delivery networks use it to send each user to a nearby site and to absorb attacks.
Message queuing that connects programs on different systems, so a sender can hand off work without the receiver being ready.
The story
Before
Programs on different machines could not reliably hand work to each other unless both were running and shared a network protocol. IBM's message-switching software of the time, TCAM, ran only on IBM systems, and connecting other platforms meant custom code.
What happened, and who
IBM announced MQSeries in 1992 and first released it in December 1993, extending message queuing from IBM systems to other platforms such as HP and Sun. A program puts a message on a named queue and the queue manager stores it, even across failures, until the receiver takes it.
Why it mattered
MQSeries, now called IBM MQ, established the contract of a durable queue that decouples sender from receiver, on which later message brokers and cloud queueing services are built. It is still widely used to connect banking and other enterprise systems.
An on-policy temporal-difference method that updates from the quintuple state, action, reward, next state, next action.
The story
Before
Q-learning, introduced by Christopher Watkins in 1989, updates its estimate of an action's value using the best action available in the next state, whether or not the agent will actually take it. It therefore learns the value of a greedy policy while behaving exploratorily, which makes its estimates about a different policy from the one being followed.
What happened, and who
In 1994 Gavin Rummery and Mahesan Niranjan of Cambridge University Engineering Department described a modification in the technical report "On-Line Q-Learning Using Connectionist Systems" (CUED/F-INFENG/TR 166). It updates from the action the agent actually chooses next, so the update uses the whole quintuple. They called it Modified Connectionist Q-Learning; the name SARSA, from that quintuple, was suggested by Richard Sutton and stuck.
Why it mattered
SARSA learns the value of the policy actually being followed, exploration included. That makes it more cautious than Q-learning where exploring is dangerous, and it is the standard example of on-policy control. Comparing its update with Q-learning's is how the on-policy and off-policy distinction is usually taught.
Four per-client promises, such as read-your-writes, that let a user make sense of a store whose replicas are allowed to disagree.
The story
Before
Bayou, a replicated store built at Xerox PARC for mobile users, let a client read or write at any replica, even one that had been disconnected for hours. That kept it available, but a client could write at one replica, read from another that had not yet received the write, and watch its own update vanish, or see data go backwards in time.
What happened, and who
In 1994 Douglas Terry, Alan Demers, Karin Petersen, Mike Spreitzer, Marvin Theimer and Brent Welch, all at Xerox PARC, published "Session Guarantees for Weakly Consistent Replicated Data" at the third international conference on Parallel and Distributed Information Systems (PDIS). It defined four guarantees a client's session can ask for: read your writes, monotonic reads, writes follow reads and monotonic writes.
Why it mattered
These four names are still how engineers say what a client of an eventually consistent store is promised. They are cheap to provide, since a session only has to remember how far it has read and written and use a replica at least that current, so systems such as Azure Cosmos DB expose a "session" consistency level. Read-your-writes is the guarantee most often bolted onto a replicated database after users complain that a saved change disappeared.
A plain text file at a site's root that tells well-behaved web crawlers which paths to stay out of.
The story
Before
In 1994 the web was small and crawlers, programs that fetch pages automatically to index them, were new. A badly written one could request pages from a single small server so fast that it stopped serving people, and a site owner had no agreed way to say "not here" or "slow down".
What happened, and who
In February 1994 Martijn Koster, then working at Nexor in the UK, proposed a convention on the www-talk mailing list: a crawler should first fetch /robots.txt and obey the rules it finds there. It is reported that Koster proposed it after another author's crawler accidentally overloaded his own server. By June 1994 it was a de facto standard that crawlers such as WebCrawler, Lycos and AltaVista followed.
Why it mattered
Every search engine and most other crawlers still read robots.txt, and it is the first thing a crawler design has to handle. It is a request, not access control: a crawler that ignores it is not stopped, so anything genuinely private needs authentication. The convention ran on goodwill for 28 years before the IETF published it as RFC 9309 in September 2022.
An x86 processor that turned each instruction into simpler internal operations and executed them out of order, a year before the Pentium Pro.
The story
Before
Intel's 486 and Pentium executed the x86 instruction set, with its variable-length instructions and complex addressing, more or less directly, and the Pentium ran its instructions in order. Making such a design faster meant adding pipelines around a hard-to-decode instruction set.
What happened, and who
On 14 March 1994 NexGen, a small American company, announced the Nx586, sold as a Pentium-compatible chip at 60 and 66 MHz, priced roughly $300 below a comparable Pentium. Its decoder translated each x86 instruction into one or more simpler internal "RISC86" operations, which were renamed, executed speculatively and out of order, and then retired.
Why it mattered
This is the design that every modern x86 core still uses: x86 is the encoding the software sees, and a RISC-like machine underneath does the work. Intel's Pentium Pro reached the market in 1995 with the same approach, and Intel's version became the model. AMD agreed to buy NexGen in October 1995, completed the purchase in January 1996, and its follow-on design shipped as the K6.
Putting each item in the less loaded of two randomly chosen bins cuts the fullest bin's load from logarithmic to doubly logarithmic.
The story
Before
Throw n balls into n bins, each at a uniformly random bin, and the fullest bin ends up with about log n / log log n balls. When the balls are requests and the bins are servers, that means the busiest server carries far more than the average of one, and checking every server's load before each choice does not scale.
What happened, and who
Yossi Azar, Andrei Broder, Anna Karlin and Eli Upfal presented "Balanced Allocations" at the ACM Symposium on Theory of Computing (STOC) in Montreal in May 1994. They proved that if each ball picks two bins at random and goes into the less full one, the maximum load drops to about log log n, an exponential improvement, and that a third choice adds only a constant factor.
Why it mattered
One extra probe buys almost all of the benefit of knowing every server's load. Load balancers use it directly as "pick two servers at random, send the request to the one with fewer active connections": NGINX ships it as its random two least-connections method and HAProxy as a mode of its random algorithm. It works well when each balancer has only a stale or partial view of the fleet.
A compiler that reads the receiver types seen at run time and inlines the calls it now expects, speeding up a dynamic language.
The story
Before
In SELF, a dynamically typed object-oriented language, almost every operation is a message send whose target method depends on the receiver's type at run time. The compiler could not know the callee, so it could not inline it, and each send stayed a slow indirect call.
What happened, and who
Urs Hölzle and David Ungar published "Optimizing Dynamically-Dispatched Calls with Run-Time Type Feedback" at the ACM PLDI conference in Orlando in June 1994. The system records which receiver types each send site has actually seen, recompiles hot code, and inlines the sends that almost always reach the same method behind a cheap type check. On their large SELF programs it cut the number of calls by a factor of 3.6 and improved performance by a factor of 1.7.
Why it mattered
This is speculative optimisation: guess from observed behaviour, guard the guess, and inline. Inlining also lets the compiler optimise across the former call boundary, which is why the speedup exceeds the call overhead removed. The technique passed through Hölzle's later work into Sun's HotSpot Java VM, and the same idea underlies the just-in-time compilers of JavaScript engines.
An argument that a remote call cannot be made to look like a local one, because latency, memory and failure all differ.
The story
Before
Remote procedure calls, and later object systems such as CORBA, tried to let a program call a method on another machine exactly as it called a local one, hiding the network behind a familiar interface. The promise was that a distributed application could be written as if it were one program.
What happened, and who
In November 1994 Jim Waldo, Geoff Wyant, Ann Wollrath and Sam Kendall of Sun Microsystems Laboratories published "A Note on Distributed Computing" as technical report SMLI TR-94-29. It argues that local and remote calls differ in four ways: latency, memory access, partial failure and concurrency. Pretending they do not leads to designs that cannot be reliable.
Why it mattered
Partial failure, where some parts of a system stop while others carry on and the caller cannot tell which, is now the starting assumption of distributed system design. It is why remote calls carry timeouts, retries and idempotency keys, and why API designers avoid interfaces that look local but are not.
Proportional-share CPU scheduling done by randomly drawing a ticket to choose who runs next.
The story
Before
Priority schedulers order jobs but give no simple way to say "this job gets 30 percent of the CPU". Priorities are relative and unit-less, and a low-priority job can starve behind high-priority ones.
What happened, and who
Carl Waldspurger and William Weihl of MIT presented "Lottery Scheduling: Flexible Proportional-Share Resource Management" at the first Symposium on Operating Systems Design and Implementation (OSDI) in Monterey in November 1994. Each process holds tickets, at each scheduling decision the kernel draws one ticket at random, and the holder runs, so a process holding 30 percent of the tickets wins about 30 percent of the time. Tickets can be grouped into currencies, so one user's tickets can be traded among that user's own jobs without disturbing others.
Why it mattered
A share became a number you can set and reason about, and a job with any tickets at all cannot starve. Lottery scheduling is only fair on average, and its short-run variance led the same authors to stride scheduling. The share-based view of CPU time it popularised runs through the fair schedulers of general-purpose kernels.
Five missing entries in a lookup table made some Pentium divisions slightly wrong, and forced a recall that cost Intel $475 million.
The story
Before
Processor errata, mistakes in the silicon, were normally corrected in a later revision of the chip with little public notice. Intel's Pentium, launched in 1993, divided using a radix-4 SRT algorithm that reads its quotient digits from a lookup table held in a programmable logic array.
What happened, and who
Thomas Nicely, a mathematician at Lynchburg College in Virginia, found in 1994 that his Pentium gave slightly wrong results when he computed reciprocals of primes. He confirmed the fault on 19 October, reported it to Intel on 24 October and made it public on 30 October. After IBM stopped shipping Pentium PCs, Intel announced on 20 December 1994 that it would replace affected chips for any owner who asked, and in January 1995 it took a $475 million charge. The cause was five entries in the 1,066-entry table that should have held +2 but held zero.
Why it mattered
It was the first time a CPU arithmetic error became a public consumer crisis, and it showed that a rare error can still be unacceptable and that denying it is costly. Vendors put far more effort into verifying arithmetic units before manufacture and into designs that can be corrected after release by a microcode update.
A proportional-share scheduler that picks, among the tasks entitled to run now, the one with the earliest virtual deadline.
The story
Before
Proportional-share schedulers such as the fair queueing family gave each client its share of the CPU over the long run. They did not let a client with a short request say how soon it needs service, so long-run fairness could still leave a short request waiting.
What happened, and who
Ion Stoica and Hussein Abdel-Wahab of Old Dominion University described the Earliest Eligible Virtual Deadline First algorithm in a 1995 technical report, "Earliest Eligible Virtual Deadline First: A Flexible and Accurate Mechanism for Proportional Share Resource Allocation". A client is eligible once it has not yet received more service than it is owed, and among eligible clients the one whose request would finish earliest in virtual time runs. The report shows the difference between ideal and actual service stays within one time quantum.
Why it mattered
Both latency and fairness come out of one rule. In 2023 Peter Zijlstra proposed it to replace the Completely Fair Scheduler, and EEVDF became the default scheduler in Linux 6.6, so most servers and Android phones now schedule this way.
A step-up procedure that controls the expected proportion of false discoveries among the results declared significant.
The story
Before
When many hypotheses are tested at once, some will look significant by chance. The Bonferroni correction guarantees no false positives at all, but it demands such a small threshold for each test that real effects are missed as the number of tests grows.
What happened, and who
Yoav Benjamini and Yosef Hochberg published "Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing" in the Journal of the Royal Statistical Society, Series B, volume 57, in 1995. Their procedure sorts the p-values and finds the largest one that falls below a threshold that grows with its rank, then declares it and all smaller ones significant, holding the expected share of false discoveries below a chosen rate.
Why it mattered
It suits problems where some errors are tolerable and many tests are run, so it is standard in genomics, where thousands of genes are tested at once. It is also the usual fix when an A/B testing platform reports many metrics or variants and some will seem significant by luck.
A five-page note arguing that because processors improve faster than DRAM, memory access time will come to dominate performance.
The story
Before
Processor speed and DRAM speed had both been improving exponentially for years, but at different rates, so each new generation widened the gap between how fast a core could compute and how fast memory could answer. Caches and clever design were assumed to absorb it.
What happened, and who
William Wulf and Sally McKee of the University of Virginia wrote "Hitting the Memory Wall: Implications of the Obvious" in December 1994; it appeared in ACM SIGARCH Computer Architecture News in 1995. With a simple model of average access time from the cache hit rate, they argued that even a tiny miss rate would eventually make average memory time the limit on speed, and that bigger caches would only postpone it.
Why it mattered
The paper named the memory wall, and the idea it names shapes almost every design decision since: deeper cache hierarchies, prefetching, multithreading to hide latency, and the emphasis on data layout and locality in performance work. A working engineer meets it whenever a profile shows a program waiting on memory rather than computing.
A tiny network with irrational capacities on which the Ford-Fulkerson method can run forever.
The story
Before
The Ford-Fulkerson method of 1956 finds a maximum flow by repeatedly pushing flow along any path with spare capacity. With whole-number capacities each round adds at least one unit, so it must stop. It was known that irrational capacities can make it run forever, but not how small a network could show this.
What happened, and who
Uri Zwick published "The Smallest Networks on Which the Ford-Fulkerson Maximum Flow Procedure May Fail to Terminate" in Theoretical Computer Science in 1995. Textbook accounts such as Jeff Erickson's use his six-node network, in which one edge has capacity (√5 − 1)/2 and a repeating cycle of paths pushes ever smaller amounts, so the flow converges to a value below the true maximum.
Why it mattered
The example shows that Ford-Fulkerson needs a rule for choosing paths to be sure of finishing. Edmonds and Karp's rule of always taking the shortest augmenting path, from 1972, bounds the number of rounds independently of the capacities.
Shows that the SQL standard's isolation definitions are ambiguous and leave out snapshot isolation, which many databases actually use.
The story
Before
The SQL-92 standard defined its isolation levels by three anomalies: dirty reads, non-repeatable reads and phantoms. Read that way, the levels missed real problems such as lost updates and write skew, and it did not describe snapshot isolation, in which each transaction reads from a fixed snapshot of the data.
What happened, and who
Hal Berenson, Philip Bernstein, Jim Gray, Jim Melton, Elizabeth O'Neil and Patrick O'Neil presented "A Critique of ANSI SQL Isolation Levels" at the ACM SIGMOD conference in San Jose in May 1995. They made the phenomena precise, added new ones, and defined snapshot isolation as a level of its own, different from any the standard listed.
Why it mattered
It explains why the same level name means different things in different engines: the standard's wording does not settle it, and vendors filled the gap differently. Reading a database's documentation for what its REPEATABLE READ or SERIALIZABLE permits, rather than trusting the name, follows directly from this paper.
Letting several threads issue instructions in the same cycle so that a wide core's idle execution slots get used.
The story
Before
A superscalar core can issue several instructions per cycle, but one thread rarely has that many independent instructions ready, so most issue slots went empty. Fine-grain multithreading switched between threads cycle by cycle, but each cycle still issued from only one thread.
What happened, and who
Dean Tullsen, Susan Eggers and Henry Levy of the University of Washington presented "Simultaneous Multithreading: Maximizing On-Chip Parallelism" at the International Symposium on Computer Architecture (ISCA) in Italy in June 1995. Using simulation, they showed that mixing instructions from several threads in the same cycle could achieve up to four times the throughput of a single-threaded superscalar. Earlier IBM research had explored similar ideas in the 1960s.
Why it mattered
Intel's Hyper-Threading, introduced in the Pentium 4 in 2002, is an SMT implementation, and Tullsen's work at UC San Diego was used in its development. Every operating system now shows each SMT hardware thread as a logical CPU, which is why scheduling, licensing and performance analysis have to distinguish logical from physical cores.
A deterministic form of proportional-share scheduling that removes lottery scheduling's random short-run error.
The story
Before
Lottery scheduling gave each job CPU time in proportion to its tickets, but only on average, since each choice was a random draw. Over a short window a job could receive noticeably more or less than its share, and response times varied.
What happened, and who
Carl Waldspurger and William Weihl of MIT published "Stride Scheduling: Deterministic Proportional-Share Resource Management" as technical report MIT/LCS/TM-528 in June 1995. Each client has a stride, inversely proportional to its tickets, and a pass value; the scheduler always runs the client with the smallest pass and then advances it by its stride. A client with twice the tickets has half the stride and so runs twice as often.
Why it mattered
Stride scheduling achieved the same shares with much lower error and more predictable response, and the idea of tracking each task's virtual progress and running the one furthest behind is the core of the virtual-time schedulers that followed, including the fair schedulers in Linux.
One extensible tree index in which the B+tree and R-tree are special cases, so a new index type needs only a few functions.
The story
Before
Every kind of database index, from the B+tree for ordered keys to the R-tree for regions, had been built as a separate structure with its own search code, insertion code and concurrency and recovery machinery. Supporting a new data type meant writing a new index from scratch.
What happened, and who
Joseph Hellerstein of the University of Wisconsin, Jeffrey Naughton and Avi Pfeffer of Berkeley presented "Generalized Search Trees for Database Systems" at the VLDB conference in Zurich in September 1995. A GiST keeps the balanced tree, page layout, locking and logging fixed and asks the index designer only for a handful of methods, such as how to test whether a key may match a query and how to split a page.
Why it mattered
PostgreSQL adopted the design as its GiST access method, and range types, exclusion constraints and PostGIS's spatial indexes are built on it. The principle of separating what is a tree from what the keys mean is why Postgres can index geometry, text and ranges with the same machinery.
Replaced the research system's own query language with SQL, turning Berkeley's POSTGRES into a database ordinary programmers could use.
The story
Before
POSTGRES, the Berkeley research database led by Michael Stonebraker, spoke PostQUEL, its own query language, while the rest of the industry was standardising on SQL. It was a vehicle for research ideas, hard to use without learning a language nobody else supported.
What happened, and who
In 1994 two Berkeley graduate students, Andrew Yu and Jolly Chen, replaced the PostQUEL interpreter with one for SQL, and replaced the monitor console program with psql. They announced a 0.01 beta to testers on 5 May 1995 and Postgres95 version 1.0 on 5 September 1995, under a more liberal licence that let anyone modify the software.
Why it mattered
SQL and a permissive licence made the system usable and adoptable by the wider world, and a community of volunteers took it over. The project was renamed PostgreSQL in 1996 and released as version 6.0 in January 1997, and it is now one of the most widely used open-source databases.
Intel's P6 design decoded x86 into micro-operations and ran them out of order, ending the assumption that x86 could not be fast.
The story
Before
Intel's Pentium executed x86 instructions in order, in two pipelines, so one stalled instruction, such as one waiting for memory, held up those behind it. The complexity of x86 instructions made out-of-order execution look impractical.
What happened, and who
On 1 November 1995 Intel released the Pentium Pro, the first processor with its P6 microarchitecture. It decoded x86 instructions into micro-operations, held them in a pool tracked by a 40-entry reorder buffer, executed them as their inputs became ready, and retired them in program order.
Why it mattered
P6 made translate-then-execute-out-of-order the standard way to build an x86 core: the Pentium II and III and the Pentium M descended from it, and Intel's Core line evolved from them. Every high-performance x86 and most other cores today work this way, which is why a program's instructions do not run in the order written even though its results appear to.
Showed one-way message passing on his L3 microkernel costing 250 cycles against 5,750 on Mach, about 22 times faster, on the same hardware.
The story
Before
First-generation microkernels such as Mach moved file systems and drivers into user-level servers, so every system call became messages between address spaces, and those messages were very slow. Unix built on Mach ran markedly slower than monolithic Unix, and many concluded that the microkernel design was impractical.
What happened, and who
Jochen Liedtke of GMD, the German national research centre for information technology, presented "On micro-kernel construction" at the ACM Symposium on Operating Systems Principles (SOSP) in December 1995. Measuring on a 486 at 50 MHz, he found a one-way IPC on his L3 microkernel cost about 250 cycles against 5,750 on Mach, and argued that the gap came from how the kernel was built, not from the design idea.
Why it mattered
The result rehabilitated the microkernel: the L4 family that grew out of it, and later seL4, the first kernel with a machine-checked proof of correctness, and QNX-style systems all rely on a small kernel with cheap IPC. It also changed how kernel performance is argued, by counting cycles and cache and TLB misses.
A kernel that only securely multiplexes hardware, leaving abstractions such as files and virtual memory to libraries in each application.
The story
Before
Conventional kernels, monolithic or microkernel, present every application with the same fixed abstractions: a file system, virtual memory, sockets. An application that could do better with its own page-replacement policy or file layout had no way to change what the kernel provides.
What happened, and who
Dawson Engler, M. Frans Kaashoek and James O'Toole of MIT presented "Exokernel: An Operating System Architecture for Application-Level Resource Management" at the ACM Symposium on Operating Systems Principles (SOSP) in December 1995. The kernel only allocates and protects hardware resources such as disk blocks and physical pages, and an untrusted library operating system linked into each application builds the familiar abstractions on top.
Why it mattered
The exokernel is an ancestor of unikernels and library operating systems, and of modern designs where applications drive hardware directly with the kernel only granting access, such as user-space networking and storage stacks. It is the clearest statement of the idea that a kernel should enforce protection and leave policy to applications.
Training many models on bootstrap resamples of the data and averaging their predictions cuts variance without raising bias much.
The story
Before
Some predictors, such as decision trees, are unstable: a small change to the training set can produce a very different model. A single such model overfits, and it was not clear how to reduce that without collecting more data.
What happened, and who
Leo Breiman of the University of California, Berkeley published "Bagging Predictors" in the journal Machine Learning, volume 24, in 1996. Bagging, short for bootstrap aggregating, draws several resamples of the training set with replacement, fits a model to each and averages their outputs, or takes a majority vote for classification.
Why it mattered
Averaging removes much of the variance of unstable learners, and the idea is one of the basic tools of ensemble methods. Breiman's random forests, which add random feature selection to bagged trees, followed in 2001 and remain a strong default for tabular data.
A patch that removed CPython's global lock by making reference counts atomic and adding fine-grained locks, and made single-threaded code far slower.
The story
Before
CPython protects its interpreter state, and above all its reference counts, with one global interpreter lock (GIL), so only one thread executes Python bytecode at a time. Programs could not use several cores for Python code in one process.
What happened, and who
In 1996 Greg Stein published a patch against Python 1.4 that removed the GIL. It made reference-count updates atomic, using Windows interlocked operations and locks on Unix, and added per-object and per-container locks. Measuring the patched interpreter, David Beazley later reported single-threaded code running several times slower, around four times on the Pystone benchmark.
Why it mattered
The patch was not adopted, apart from some of its thread-state changes, because the cost to ordinary single-threaded programs was too high. It defined the problem later attempts had to beat: PEP 703 cites it, and CPython's free-threaded build, made possible by much cheaper reference-count techniques, came decades later.
A regression penalty on the sum of absolute coefficient values that shrinks coefficients and sets some exactly to zero.
The story
Before
Choosing which variables to keep in a regression by subset selection was unstable, since a small change in the data could change the chosen set, and ridge regression shrank coefficients but never removed any variable.
What happened, and who
Robert Tibshirani of the University of Toronto published "Regression Shrinkage and Selection via the Lasso" in the Journal of the Royal Statistical Society, Series B, volume 58, in 1996. The lasso minimises squared error subject to a bound on the sum of absolute coefficients, an L1 penalty, whose geometry pushes some coefficients exactly to zero.
Why it mattered
Fitting and selecting variables in one step made sparse models routine, and the L1 penalty is used across statistics and machine learning, wherever there are many more candidate features than examples.
Buffers writes in memory and merges them into sorted files on disk in large sequential passes, trading read work for cheap writes.
The story
Before
A B-tree index updates a page in place, so each insert may touch a random page on disk. Where writes are very frequent, as with logging or history tables, a disk head performing one random I/O per update was the bottleneck.
What happened, and who
Patrick O'Neil, Edward Cheng, Dieter Gawlick and Elizabeth O'Neil published "The Log-Structured Merge-Tree (LSM-Tree)" in the journal Acta Informatica in 1996. New entries go into a small in-memory tree and are merged, in batches, into a hierarchy of larger sorted trees on disk, so every disk write is large and sequential.
Why it mattered
The structure underlies the storage engines of Google's Bigtable, Apache Cassandra and RocksDB, and through RocksDB many other databases. It is why write-heavy stores are built the way they are, and why they need compaction and pay a read cost checking several levels.
A busy server that spends all its time handling network interrupts and so delivers no packets to the programs waiting for them.
The story
Before
Operating systems handled arriving packets by interrupt, which cost nothing while the CPU was idle and got the packet handled quickly. Interrupts run at higher priority than ordinary code, so interrupt-driven receive was assumed to be better than polling.
What happened, and who
Jeffrey Mogul of Digital Equipment Corporation's Western Research Laboratory and K. K. Ramakrishnan of AT&T Bell Laboratories presented "Eliminating Receive Livelock in an Interrupt-driven Kernel" at the USENIX Annual Technical Conference in San Diego in January 1996. They showed that above some arrival rate a system spends all of its time in interrupt handlers, so throughput of delivered packets falls towards zero, and they fixed it by polling under load and limiting interrupt work.
Why it mattered
Under overload, more work done early can mean less work finished. Their fix, switching between interrupts when quiet and polling when busy, lives on in interrupt moderation in network cards and NAPI in Linux, and in kernel-bypass network stacks that poll all the time.
A compressed data format that combines LZ77 back-references with Huffman coding and is used by zip, gzip, zlib and PNG.
The story
Before
Popular compressors of the early 1990s used patented methods, such as the LZW algorithm behind Unix compress and GIF, and the RFC itself notes that many variations of LZ77 were patented. Free software needed a format that anyone could implement without a licence.
What happened, and who
In May 1996 L. Peter Deutsch of Aladdin Enterprises published RFC 1951, the specification of the DEFLATE format, version 1.3. It codes data as literal bytes and back-references to earlier bytes, LZ77 style, and then Huffman-codes both. The format originated with Phil Katz's PKZIP, and the RFC says it can be implemented in a way not covered by patents.
Why it mattered
With an open specification and no patent problem, DEFLATE became the common compression format: the ZIP archive, gzip, the zlib library, PNG images and HTTP content-encoding all use it. Most text delivered over the web has been compressed with it.
A shared queue with one lock for the head and another for the tail, so producers and consumers do not block each other.
The story
Before
A queue shared by several threads was usually protected by one lock, so an enqueue and a dequeue, even though they touch opposite ends, had to wait for each other. Every operation on the queue was serialised.
What happened, and who
Maged Michael and Michael Scott of the University of Rochester published "Simple, Fast, and Practical Non-Blocking and Blocking Concurrent Queue Algorithms" at the ACM Symposium on Principles of Distributed Computing (PODC) in May 1996. Its blocking algorithm keeps a head lock and a tail lock and a dummy node, so that one enqueue and one dequeue can proceed at the same time.
Why it mattered
It is the standard example of splitting a lock along the structure of the data rather than locking everything. The same paper's non-blocking queue became known as the Michael-Scott queue and is the basis of Java's ConcurrentLinkedQueue.
The first Linux to run on multiprocessors, protected by one Big Kernel Lock that took fifteen years to remove.
The story
Before
Linux began as a uniprocessor kernel. Its code assumed that only one thing ran in the kernel at a time, apart from interrupts, so nothing inside it was protected against another CPU.
What happened, and who
Linux 2.0, released on 9 June 1996, added support for symmetric multiprocessing on Intel MP-compliant machines and some SPARC systems. To do it safely with little effort, the kernel developers added the Big Kernel Lock, a single lock that let only one processor at a time run kernel code.
Why it mattered
The lock kept the change small and correct, but it limited the benefit of extra CPUs, and it was replaced with finer-grained locks piece by piece until Arnd Bergmann removed the last of it in Linux 2.6.39 in 2011. It is the standard cautionary tale in lock granularity: start coarse to be correct, and expect to pay years to refine it.
A clustering method that grows clusters from dense regions, finds clusters of any shape and labels sparse points as noise.
The story
Before
The k-means algorithm finds clusters by distance to a centre, which suits compact round groups and needs the number of clusters in advance. It forces every point into a cluster and cannot recover crescent or ring-shaped ones.
What happened, and who
Martin Ester, Hans-Peter Kriegel, Jörg Sander and Xiaowei Xu presented "A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise" at the second International Conference on Knowledge Discovery and Data Mining (KDD) in Portland in August 1996. A point with enough neighbours within a set radius is a core point, clusters are chains of connected core points and their neighbours, and everything else is noise.
Why it mattered
DBSCAN needs no cluster count and handles noise and odd shapes, which made it a default for spatial data and anomaly-flavoured problems. Its two parameters, the radius and minimum neighbours, can be hard to choose when density varies.
A step-by-step tutorial on stack buffer overflows and shellcode that became the standard introduction to memory-corruption attacks.
The story
Before
Buffer overflows were known, and the 1988 Morris worm used one, but the technique for turning an overrun buffer into control of a program was folklore held by a few people. Most programmers did not understand how a stray write could redirect a program.
What happened, and who
On 8 November 1996 the pseudonymous Aleph One, later identified as Elias Levy, published "Smashing the Stack for Fun and Profit" in issue 49 of the hacker magazine Phrack. It explains how the stack holds a function's return address, how writing past the end of a local array overwrites it, and how to write and place a small piece of machine code so that the return jumps into it.
Why it mattered
The article made stack smashing widely reproducible, and the years of exploits it enabled drove the defences now standard in compilers and operating systems: stack canaries, non-executable stacks and address space layout randomisation. It is still the first thing security courses point to.
Postgres95 was renamed PostgreSQL and took the Berkeley version numbering forward, giving the project the name it still has.
The story
Before
The name Postgres95 tied the project to a year, and it had passed from the two Berkeley students to a group of outside volunteers. Marc Fournier had provided the project's first external development server in July 1996, and Bruce Momjian and Vadim Mikheev were among those working on the inherited code.
What happened, and who
PostgreSQL 6.0 was released on 29 January 1997. The name PostgreSQL reflected both the project's POSTGRES origin and its SQL support, and the version number resumed the Berkeley sequence rather than restarting at 2.
Why it mattered
The name, and the structure of a volunteer development group working in public, have persisted: PostgreSQL is still developed that way, without a single owning company.
A way of assigning keys to servers so that adding or removing one server moves only a small share of the keys.
The story
Before
The obvious way to spread keys over n cache servers is to put key k on server hash(k) mod n. When n changes, almost every key maps to a different server, so adding a server makes nearly all cached data miss at the same moment.
What happened, and who
David Karger, Eric Lehman, Tom Leighton, Rina Panigrahy, Matthew Levine and Daniel Lewin of MIT presented "Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web" at the ACM Symposium on Theory of Computing (STOC) in El Paso in May 1997. Servers and keys are hashed onto the same circle and each key belongs to the next server round the circle, so a server joining or leaving only takes over or releases the keys next to it. Leighton and Lewin went on to found the content delivery company Akamai.
Why it mattered
Consistent hashing became the standard way to partition data and requests over a changing set of machines. Amazon's Dynamo and Apache Cassandra use it to place data, and it appears in caches, load balancers and content delivery.
A computer won a six-game chess match against the reigning world champion, 3.5 to 2.5, by searching and hand-written evaluation.
The story
Before
In February 1996 in Philadelphia, Garry Kasparov had beaten an earlier version of IBM's Deep Blue 4 to 2. No computer had yet won a match against a reigning world champion under tournament conditions.
What happened, and who
In New York from 3 to 11 May 1997 an improved Deep Blue, built by an IBM team, played a rematch against Kasparov. It won 3.5 to 2.5, and the deciding sixth game finished on 11 May.
Why it mattered
Deep Blue searched enormous numbers of positions with special-purpose hardware and scored them with a function written by hand, so its win showed what search with human-designed knowledge could do. Two decades later AlphaZero reached and passed that level by self-play alone, without a hand-written evaluation.
A robot on Mars kept rebooting because a low-priority task held a lock a high-priority task needed, until engineers patched it from Earth.
The story
Before
Pathfinder's flight computer ran VxWorks, a real-time operating system, whose mutexes could be created with or without priority inheritance, the rule that lets a task holding a lock temporarily take on the priority of the highest task waiting for it. The mutex guarding the shared information bus had been created without it.
What happened, and who
Pathfinder landed on Mars on 4 July 1997, and a few days later, as it began collecting weather data, its computer started resetting. A low-priority weather task held the bus mutex, the high-priority bus task blocked on it, and medium-priority communications work kept the weather task from running, so a watchdog timer saw the bus task stalled and reset the system. Engineers at NASA's Jet Propulsion Laboratory, led by Glenn Reeves, reproduced it on a testbed and uploaded a small program that turned priority inheritance on; Mike Jones wrote up the account.
Why it mattered
It is the standard real-world example of priority inversion, and it shows that priority inheritance, published by Lui Sha, John Lehoczky and Raj Rajkumar years earlier, is needed for real-time locks. It also showed the value of being able to patch running flight software remotely.
Reweights the training examples after each round so that a series of weak classifiers combine into a strong one.
The story
Before
Boosting, turning a learner that is only slightly better than guessing into an accurate one, had been shown possible in theory by Robert Schapire in 1990, but the known constructions were hard to use in practice.
What happened, and who
Yoav Freund and Robert Schapire published "A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting" in the Journal of Computer and System Sciences, volume 55, in August 1997. Their AdaBoost algorithm trains weak classifiers one after another, each time increasing the weight of the examples the previous ones got wrong, and combines them by a weighted vote.
Why it mattered
AdaBoost was simple, needed no tuning of the weak learner, and worked well, and it opened the way to gradient boosting, the basis of the tree ensembles that are still the strongest general method for tabular data. The paper won the Gödel Prize in 2003.
An overflow that reuses code already in the program, calling the C library's system() instead of injected shellcode.
The story
Before
After Aleph One's 1996 article, attacks worked by injecting shellcode onto the stack and jumping to it. Alexander Peslyak, known as Solar Designer, had written a Linux patch that made the stack non-executable, which stopped that kind of attack.
What happened, and who
On 10 August 1997 Peslyak posted "Getting around non-executable stack (and fix)" to the Bugtraq mailing list, with working exploits for lpr and color_xterm. Instead of injecting code, the overflow overwrites the return address with the address of system() in the C library and arranges the stack so that it receives a pointer to the string /bin/sh. He also posted a patch that loads shared libraries at addresses containing zero bytes, to make passing such arguments harder.
Why it mattered
A non-executable stack alone is not enough, because an attacker can reuse code the program already contains. This started the code-reuse line that led to return-oriented programming, published in 2007, and is why address space layout randomisation and control-flow integrity exist alongside non-executable memory.
A name for the trade-off of giving up strict consistency for availability in large services, offered as the opposite of ACID.
The story
Before
Database transactions guarantee the ACID properties: atomicity, consistency, isolation and durability. Holding them across a cluster of machines is expensive and can make a service unavailable when part of it fails, and the big Internet services of the time wanted to stay up.
What happened, and who
Armando Fox, Steven Gribble, Yatin Chawathe, Eric Brewer and Paul Gauthier, at the University of California, Berkeley and Inktomi, presented "Cluster-Based Scalable Network Services" at the ACM Symposium on Operating Systems Principles (SOSP) in October 1997. It describes how to build such services on clusters of commodity machines, and it contrasts the ACID guarantees with the weaker ones such services can live with, which it calls BASE, for basically available, soft state, eventual consistency.
Why it mattered
BASE gave engineers a shorthand for the choice that shaped the NoSQL generation of stores, and the ACID-versus-BASE framing is still how the trade-off is introduced. It is only a slogan, though: the consistency models it stands in for are much more precise.
A recurrent network cell with a self-loop of constant error flow and gates that lets gradients survive over hundreds of steps.
The story
Before
Recurrent networks trained by backpropagation through time lost or blew up the error signal as it was passed back across many steps, so they could not learn dependencies more than a few steps apart. Hochreiter analysed this in his 1991 diploma thesis.
What happened, and who
Sepp Hochreiter and Jürgen Schmidhuber published "Long Short-Term Memory" in Neural Computation, volume 9, issue 8, on 15 November 1997. Their cell has a memory that carries the error back unchanged through a self-loop, with input and output gates that decide when it is written and read; it could bridge time lags of more than 1,000 steps. The forget gate was added in 2000 by Gers, Schmidhuber and Cummins.
Why it mattered
LSTM became the standard sequence model for speech recognition and machine translation, which it remained for about two decades until attention-based Transformers replaced it. Its gating idea lives on in the GRU and in gated state-space models.
A compiler patch plants a guard word beside the return address, so a stack overflow is caught before the function returns.
The story
Before
A stack buffer overflow overwrites the saved return address and hands the attacker control when the function returns, as in the 1988 Morris Worm. The fixes were to audit source code for unchecked copies and patch each hole as it was found. New overflows kept appearing, and existing binaries stayed exposed.
What happened, and who
In the paper "StackGuard: Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks", Crispin Cowan and colleagues at the Oregon Graduate Institute described a small patch to the GCC compiler. It places a canary word between a function's local buffers and its return address and checks it before returning. The paper appeared at the 7th USENIX Security Symposium in San Antonio, held 26 to 29 January 1998.
Why it mattered
An overflow that runs on to the return address must first overwrite the canary, so the program can stop instead of jumping to attacker code. It needs no source changes, only a recompile. Compiler-inserted canaries, such as GCC's -fstack-protector family and Microsoft's /GS, are now enabled by default in many mainstream toolchains and distributions.
Ranks web pages by how often an imaginary surfer who keeps following links would land on each one.
The story
Before
Early web search engines ranked results mostly by matching the query's words against page text, so a page could rise simply by repeating a word. Hand-built directories such as Yahoo! used human judgement, which could not keep up with the growth of the web.
What happened, and who
In April 1998 Larry Page and Sergey Brin, then Stanford graduate students, presented "The Anatomy of a Large-Scale Hypertextual Web Search Engine" at the Seventh International World Wide Web Conference in Brisbane. It describes Google, a prototype search engine, and PageRank, which scores a page by the chance that a random surfer, who follows a random link with some probability and otherwise jumps to a random page, is on it.
Why it mattered
Using the web's link structure as a vote of importance made results far harder to game than text matching, and PageRank became the ranking core of Google. The same computation, the stationary distribution of a Markov chain found by repeated multiplication, is now the standard example of how link-graph importance is calculated.
A company built to turn consistent hashing into a content delivery network that spreads popular web pages across many servers.
The story
Before
When a web page suddenly became popular, every visitor fetched it from the one origin server, which slowed or failed under the flash crowd. Caching servers could help, but spreading load across many caches while keeping each object findable, as caches came and went, had no good solution.
What happened, and who
Akamai was founded on 20 August 1998 by MIT's Tom Leighton and Danny Lewin with Jonathan Seelig, Randall Kaplan and Preetish Nijhawan, after they entered MIT's $50K entrepreneurship competition with a design based on consistent hashing. That idea came from a 1997 paper by Karger, Leighton, Lewin and others, and the commercial service launched in April 1999.
Why it mattered
Akamai was the first large content delivery network: pages are served from a nearby cache, and consistent hashing decides which cache holds which object without remapping everything when a server is added or fails. That basic arrangement now carries a large share of web and video traffic, and consistent hashing is used well beyond CDNs, for example in distributed caches and key-value stores.
Updaters publish a new copy of shared data while readers run without locks; the old copy is freed once no reader can hold it.
The story
Before
Read-mostly kernel data, such as routing tables, was protected by reader-writer locks. Even a reader has to write to the lock word to take it, so the lock's cache line bounces between CPUs and reads stop scaling as cores are added.
What happened, and who
Paul McKenney and John Slingwine of Sequent Computer Systems presented "Read-Copy Update: Using Execution History to Solve Concurrency Problems" at the Parallel and Distributed Computing and Systems conference in October 1998. Sequent had already been granted a patent on the technique in 1995.
Why it mattered
Readers do no writes and take no locks, so their cost does not grow with the number of CPUs, and the updater waits for a grace period before freeing the old copy. RCU was merged into the Linux kernel in October 2002 and is now used throughout it, in the network stack, the directory-entry cache and many other subsystems.
A stack of convolution and subsampling layers, trained end to end by backpropagation, reads handwritten digits and bank cheques.
The story
Before
Recognition systems of the time fed hand-designed feature extractors into a separate classifier. A fully connected network on raw pixels ignored the image's 2D structure, had far more weights than it needed, and did not cope well with shifted or distorted characters.
What happened, and who
In "Gradient-Based Learning Applied to Document Recognition", published in the Proceedings of the IEEE in November 1998, Yann LeCun, Léon Bottou, Yoshua Bengio and Patrick Haffner described LeNet-5, a network of two convolution and subsampling stages followed by fully connected layers. The paper reports it on handwritten digits and on a cheque-reading system deployed by banks.
Why it mattered
Sharing one small filter across the whole image cut the parameter count and built in tolerance to shifts, and the pattern of convolution, pooling and dense layers became the template for later convolutional networks. The same idea, at far larger scale, underlies image models in use today.
Ericsson's fault-tolerant language became freely available just after the company had banned it from new internal products.
The story
Before
Erlang had been built at Ericsson from 1986 by Joe Armstrong, Robert Virding and Mike Williams for telephone switches that had to stay up while being upgraded. It remained proprietary, and in February 1998 Ericsson Radio Systems banned its use in new products in favour of non-proprietary languages.
What happened, and who
In December 1998 Ericsson released the Erlang implementation as open source, and most of the Erlang team left to found a new company, Bluetail. The release included OTP (Open Telecom Platform), the libraries and design patterns for building supervised, fault-tolerant systems.
Why it mattered
Anyone could now use lightweight isolated processes that communicate only by messages, with supervisors that restart failed ones. That model runs the chat service WhatsApp and the message broker RabbitMQ, and it influenced later languages and frameworks.
A web page posed the question of how one ordinary server could hold ten thousand client connections at once.
The story
Before
Servers commonly used one process or thread per connection, so memory and scheduling costs grew with every client. The event-driven alternative used select or poll, which hand the kernel the whole list of descriptors on every call, so the cost grew with the number of connections rather than with the number that were active.
What happened, and who
In 1999 Dan Kegel, a software engineer, published a page named "The C10K problem", argued that hardware costing around $1,200 was no longer the limit, and the software's I/O strategy was. It surveyed the options, from readiness notification to asynchronous I/O and threads. Kegel kept it up to date for years, and the term C10K (ten thousand concurrent connections) passed into common use.
Why it mattered
The page named the goal and catalogued the mechanisms, which pushed the Linux and BSD kernels toward scalable readiness interfaces: epoll on Linux and kqueue on FreeBSD. The event-loop server style of nginx and Node.js grew from the same line of work.
An open instant-messaging system where each client holds one long-lived TCP connection carrying a stream of XML, later standardised as XMPP.
The story
Before
Instant messaging was run by separate closed networks such as ICQ and AOL Instant Messenger. Their protocols were proprietary and mutually incompatible, so a user on one network could not message a friend on another.
What happened, and who
Jeremie Miller began work on Jabber in 1998 and released the first version of the jabberd server on 4 January 1999. The IETF standardised the core protocol as XMPP (Extensible Messaging and Presence Protocol) in RFC 3920 in October 2004.
Why it mattered
Jabber showed chat as an open protocol with a server per domain and gateways to the closed networks. Its pattern, a persistent connection over which the server pushes messages and presence, is how chat systems have worked since, and XMPP is still used in messaging and in some server-to-server signalling.
A purpose-built machine and a volunteer network searched all of DES's key space fast enough to win a public challenge in under a day.
The story
Before
The Data Encryption Standard used a 56-bit key, and critics had long said this was short enough to search by brute force. In 1997 the volunteer DESCHALL project had won RSA's first DES challenge in 96 days, but many people still took that as proof only that it was very slow.
What happened, and who
On 19 January 1999 the Electronic Frontier Foundation's Deep Crack, a machine of custom chips it had built for under $250,000, worked with distributed.net's volunteer computers to win RSA's DES Challenge III in 22 hours 15 minutes.
Why it mattered
The demonstration made the point that a 56-bit key protects little against a determined attacker with modest funds. In October 1999 the US federal standard moved to Triple DES, and the Advanced Encryption Standard later replaced DES as the federal standard.
Ran unmodified Windows and Linux as guests on an ordinary x86 PC by translating the instructions that could not be trapped.
The story
Before
Classic virtual machine monitors relied on trap-and-emulate: the guest runs in user mode and any privileged instruction traps to the monitor. x86 has sensitive instructions that do not trap in user mode, so it failed the Popek-Goldberg conditions, and virtual machines were known mainly from IBM mainframes.
What happened, and who
VMware, a company founded in 1998 whose founders included Diane Greene and Mendel Rosenblum, first released VMware Workstation on 15 May 1999. It ran a guest operating system on top of a host one, using binary translation to rewrite the guest's troublesome instructions on the fly.
Why it mattered
It showed that x86 could be virtualised without changing the guest, which made consolidating servers and running many operating systems on one machine practical. That work led to server virtualisation and the virtual machines that cloud providers sell. Hardware support for virtualisation later arrived in Intel VT-x and AMD-V.
A web server with one event loop for the network and helper processes for the disk, so a slow disk read blocks nobody else.
The story
Before
A single-process event-driven server was fast on cached files. But a disk read, which cannot be waited on with select, stalled the whole server. Multi-process and multi-threaded servers avoided that stall but cost more memory and context switching.
What happened, and who
In "Flash: An Efficient and Portable Web Server", Vivek Pai, Peter Druschel and Willy Zwaenepoel of Rice University described the asymmetric multi-process event-driven (AMPED) architecture. It appeared at the USENIX Annual Technical Conference in Monterey, held 6 to 11 June 1999. One event loop handles the network, and helper processes do blocking disk work and report back.
Why it mattered
The paper showed that an event loop can be kept from blocking by handing unavoidable blocking work to helpers, matching or beating Apache and Zeus on real workloads. The same division of labour appears in libuv, the library under Node.js, which runs blocking file work on a thread pool.
Readers see a consistent older version of a row while writers create a new one, so reads and writes no longer block each other.
The story
Before
PostgreSQL 6.4 and earlier used table-level locking. A transaction that modified a table blocked other users from reading it until it committed.
What happened, and who
PostgreSQL 6.5, released on 9 June 1999, replaced the table locks with multiversion concurrency control (MVCC), implemented by Vadim Mikheev. The release notes say it uses the natural multiversion nature of PostgreSQL: an update leaves the old row version in the table and adds a new one, and each transaction sees only the versions that were committed when it started.
Why it mattered
Readers do not block writers, and writers do not block readers, which is why PostgreSQL copes well with mixed read and write loads. The design has a cost: old row versions pile up and VACUUM has to clean them out, a maintenance task that PostgreSQL still has.
Algorithms whose memory access pattern is efficient at every level of a cache hierarchy without being told the cache's size.
The story
Before
Getting the best from a memory hierarchy meant tuning code to the machine: blocking a matrix multiply to the cache size, or a sort to the page size. The tuning was specific to one machine and had to be redone for the next.
What happened, and who
In "Cache-Oblivious Algorithms", Matteo Frigo, Charles Leiserson, Harald Prokop and Sridhar Ramachandran of MIT described algorithms for matrix transposition, the fast Fourier transform and sorting. They are asymptotically optimal in the number of cache misses without any cache size or line length in the code. It was presented at the IEEE Symposium on Foundations of Computer Science in New York, on 17 and 18 October 1999.
Why it mattered
The recursive divide-and-conquer structure of these algorithms adapts to every level of the hierarchy at once, so one program performs well on caches of any size. Cache-oblivious layouts and algorithms are used in libraries and in data structures such as the van Emde Boas layout of trees.
A compiler analysis proves that some objects never leave their method or thread, so they can live on the stack and skip locks.
The story
Before
In Java every object was allocated on the garbage-collected heap, and synchronised methods paid for locks even on objects only one thread could ever see. The compiler had no way to show that an object stayed local.
What happened, and who
In "Escape Analysis for Java", presented at OOPSLA in Denver in November 1999, Jong-Deok Choi, Manish Gupta, Mauricio Serrano, Vugranam Sreedhar and Sam Midkiff of IBM introduced the connection graph, a structure recording which objects are reachable from which references. From it the compiler decides whether an object escapes its method or thread.
Why it mattered
An object that does not escape can be allocated on the stack, freeing it without the collector, and locks on a thread-local object can be removed. Escape analysis is now a standard optimisation in JIT compilers such as HotSpot and OpenJ9, and Go's compiler uses it to decide what goes on the stack.
Web-scale systems, multicore processors and the start of the cloud.
PaperNetworking
REST
A doctoral dissertation gives a name and a set of constraints to the architectural style that explains why the web scales.
The story
Before
The web had grown quickly, and HTTP and URIs were being standardised, but there was no shared vocabulary for why the design worked. Interfaces built on top of it, such as remote procedure calls tunnelled through HTTP, often ignored the properties that made it scale.
What happened, and who
In 2000 Roy Fielding, one of the authors of the HTTP specification, completed his doctoral dissertation "Architectural Styles and the Design of Network-based Software Architectures" at the University of California, Irvine. It compares architectural styles by the constraints they impose, and Chapter 5 introduces REST (Representational State Transfer), derived from the web's own design.
Why it mattered
REST gave engineers terms for stateless requests, uniform interfaces, cacheable responses and hypermedia. Web APIs are now routinely described as REST or RESTful, often loosely, and the constraints remain the standard way to reason about caching and scaling of HTTP interfaces.
Divide one FreeBSD system into isolated compartments, each with its own users, processes and network address, sharing a single kernel.
The story
Before
The chroot call restricts a process to a subtree of the file system, but nothing else: processes inside could still see and signal others, and root inside a chroot was still root on the machine. Keeping the customers of one shared hosting machine apart therefore needed something more.
What happened, and who
Poul-Henning Kamp added jails to FreeBSD in 1999, out of a hosting provider's need to keep customers apart, and they first shipped in FreeBSD 4.0 on 14 March 2000. Kamp and Robert Watson described the design in the paper "Jails: Confining the Omnipotent Root" at the SANE 2000 conference.
Why it mattered
A jail restricts what a process can do across the system, not only what files it can see, while costing far less than a virtual machine. It was an early operating-system-level container, and Solaris zones and Linux containers later solved the same problem in similar ways.
Seventeen x86 instructions read or change machine state yet never trap in user mode, so plain trap-and-emulate cannot virtualise the Pentium.
The story
Before
Popek and Goldberg had shown in 1974 that trap-and-emulate works only if every sensitive instruction is also privileged. Whether the Pentium met that condition had not been checked instruction by instruction, and products claiming to isolate operating environments on it were taken at their word.
What happened, and who
John Robin of the US Air Force and Cynthia Irvine of the Naval Postgraduate School analysed roughly 250 Pentium instructions in "Analysis of the Intel Pentium's Ability to Support a Secure Virtual Machine Monitor". It was presented at the 9th USENIX Security Symposium in Denver, held 14 to 17 August 2000. They found 17 sensitive instructions, including POPF and SGDT, that are not privileged, and concluded that products then on the market should not be trusted as secure monitors.
Why it mattered
The list explains why VMware had to rewrite guest code with binary translation, and why Intel VT-x and AMD-V (2005) added hardware support to make trap-and-emulate work on x86. Virtualisation-friendly instruction-set design now starts from the Popek-Goldberg condition.
A processor with a 20-stage pipeline, designed so its clock speed could climb far beyond its predecessor's.
The story
Before
Intel's Pentium III, built on the P6 core, had a 10-stage pipeline. Higher clock speed was how Intel and its rivals sold each generation, and each pipeline stage does less work, so a clock can only go faster if the pipeline gets longer.
What happened, and who
Intel launched the Pentium 4 on 20 November 2000 at 1.4 and 1.5 GHz. Its NetBurst microarchitecture doubled the branch-misprediction pipeline to 20 stages, and replaced the instruction cache with a trace cache holding decoded micro-operations. NetBurst chips eventually reached 3.8 GHz.
Why it mattered
A misprediction now cost twice as many cycles, and at higher clocks the chip drew so much power that it could not be cooled. Intel cancelled its planned successors in 2004 and moved to more efficient designs, which is why the Pentium 4 is the standard example of pipeline depth pushed too far.
The kernel declined one built-in security policy and chose pluggable hooks instead, which became the Linux Security Modules framework.
The story
Before
Linux had only the traditional Unix permission bits, which the owner of a file can change at will. Mandatory access control, in which a system-wide policy decides regardless of the owner, existed only in separate kernel patch sets.
What happened, and who
In March 2001 the US National Security Agency presented Security-Enhanced Linux (SELinux) at the Linux Kernel Summit and proposed including it in Linux 2.5. Linus Torvalds declined to choose a single model and asked for a general framework of hooks so that different security policies could be loaded as modules. Crispin Cowan of WireX led the resulting Linux Security Modules (LSM) project, which entered the mainline kernel with Linux 2.6 in December 2003.
Why it mattered
That is why Linux ships hooks rather than one policy: SELinux, AppArmor, Smack, TOMOYO and Landlock all sit on the same framework. The choice explains why Linux distributions differ in which mandatory access control they enable by default.
A write-ahead log lets a commit flush one sequential log instead of every changed data page, and TOAST stores oversized values out of line.
The story
Before
Before 7.1, PostgreSQL forced every modified data page to disk before a transaction could commit, which made commits slow. A row also had a compiled-in maximum length, typically between 8 KB and 32 KB.
What happened, and who
PostgreSQL 7.1, released on 13 April 2001, added the write-ahead log (WAL): changes are recorded in a log that is flushed at commit, and the data pages can be written later. It also introduced TOAST (The Oversized-Attribute Storage Technique), which lets rows of any length be stored with good performance.
Why it mattered
A commit became one sequential log write, which made PostgreSQL much faster for writes. The same log is the basis of crash recovery, and later of point-in-time recovery and streaming replication, so WAL remains central to how PostgreSQL survives failure and copies data to replicas.
Intel and HP's IA-64 chip bet that the compiler, not the hardware, should schedule parallel instructions; it lasted twenty years and never won.
The story
Before
Out-of-order superscalar processors find independent instructions at run time with large, power-hungry hardware structures, and Intel and HP expected that approach to run out of room. Very long instruction word designs shift the work to the compiler but had stayed niche.
What happened, and who
HP and Intel began designing IA-64 together in 1994, and the first chip, Itanium (codenamed Merced), launched on 29 May 2001, with systems from HP, IBM and Dell shipping in June. It called its style EPIC (explicitly parallel instruction computing): the compiler marks which instructions are independent, in bundles of three.
Why it mattered
Performance was disappointing and existing x86 software ran slowly, and AMD's 64-bit x86 extension gave buyers a smoother path. Itanium became the best-known commercial attempt at compiler-scheduled parallelism: its main lesson is that run-time behaviour, such as cache misses, cannot be predicted at compile time. Intel stopped shipping it in July 2021.
Each run of a program places its stack, heap and libraries at random addresses, so an exploit cannot rely on fixed ones.
The story
Before
Programs loaded at the same addresses every time. An attacker who found a vulnerability could hard-code the address of the stack or of a library function, and one working exploit then worked on every machine running that program.
What happened, and who
In July 2001 the PaX Team, an anonymous group writing Linux kernel hardening patches, released the first implementation of what it called address space layout randomisation (ASLR). It shifts the executable, the heap, the memory-mapped libraries and the stack by random amounts at each program start.
Why it mattered
An attacker now needs a way to learn an address first, usually an information leak, which raises the cost of exploiting many memory bugs. ASLR entered the mainline Linux kernel in 2.6.12 in 2005 and is enabled by default on Windows Vista, macOS, Android and other systems.
A router marks a packet when its queue is filling instead of dropping it, so the sender can slow down without losing data.
The story
Before
TCP treated a lost packet as the only sign of congestion. The sender backed off only after a packet had been dropped and had to be retransmitted, which added delay for the very traffic being protected.
What happened, and who
K. Ramakrishnan, S. Floyd and D. Black specified ECN in RFC 3168, "The Addition of Explicit Congestion Notification (ECN) to IP", published in September 2001. Two bits of the IP header let a sender declare that it understands ECN and let a router set a congestion-experienced mark, and two TCP flags carry the echo back to the sender.
Why it mattered
A mark says the queue is building without discarding anything, so the sender can react earlier and avoid a retransmission. ECN was slow to deploy, but the idea of marking underlies data-centre protocols such as DCTCP and the L4S work at the IETF.
Builds a model as a sum of small trees, each fitted to the errors the model still makes.
The story
Before
Boosting, as in AdaBoost, combined weak learners by reweighting the training examples, and was tied to one particular loss function. It was not clear how to apply the idea to other kinds of error, such as absolute or robust losses.
What happened, and who
Jerome Friedman of Stanford published "Greedy Function Approximation: A Gradient Boosting Machine" in The Annals of Statistics, volume 29, issue 5, in October 2001. It treats boosting as gradient descent in function space: each new tree is fitted to the negative gradient of the loss, which for squared error is the residual. Leo Breiman had earlier noted that boosting can be seen as optimisation, and Mason and colleagues developed a similar view.
Why it mattered
Any differentiable loss can now be used, so one method handles regression, classification and ranking. Gradient boosting is the basis of libraries such as XGBoost and LightGBM, which often win competitions on tabular data.
A trie indexed by pieces of a hash, with a bitmap and a population count keeping each node small.
The story
Before
Persistent maps, which keep every older version intact after an update, were built from balanced trees with O(log n) depth and pointer-heavy nodes. Hash tables were faster but had to be resized wholesale, which suits mutable maps only.
What happened, and who
In October 2001 Phil Bagwell of EPFL published the technical report "Ideal Hash Trees". Each trie node holds a bitmap saying which of its 32 slots are used, plus an array of only the used slots, found by counting the set bits with a population-count instruction. The result is called a hash array mapped trie (HAMT).
Why it mattered
A lookup takes a few steps regardless of size, nodes stay compact, and an update copies only one path, so old versions are shared. The HAMT became the basis for the built-in immutable maps of Clojure and Scala.
IBM's server processor was the first non-embedded chip with two complete cores on one die.
The story
Before
Processors had one core per die, and performance came from higher clock speeds and wider single cores. Multiprocessor servers needed several separate chips wired together.
What happened, and who
IBM described the POWER4 in detail at the Microprocessor Forum in San Jose, which began on 4 October 2001. The chip put two full cores on a single die, sharing an on-chip L2 cache and running at 1.1 and 1.3 GHz.
Why it mattered
Two cores that share a cache and sit on one die are cheaper and faster to link than two chips, so POWER4 opened the multicore era in servers. Intel and AMD followed with dual-core x86 processors in 2005, and multicore is now how nearly every processor gains performance.
Averages many decision trees, each grown on a random sample and choosing splits from random subsets of features, to lower variance.
The story
Before
A single decision tree is unstable: small changes in the data change the tree, and it overfits. Leo Breiman's own bagging averaged trees grown on resampled data, but the trees stayed similar to one another because the same strong features dominated every tree, limiting the gain.
What happened, and who
Leo Breiman of the University of California, Berkeley, published "Random Forests" in the journal Machine Learning in October 2001. At each split each tree may choose only from a random subset of the features, on top of bagging. Tin Kam Ho had used random feature subsets in 1995.
Why it mattered
Trees that differ more make errors that differ more, so averaging them reduces variance further. Random forests need little tuning and work on mixed kinds of data, which is why they remain a default baseline for tabular problems.
A free software load balancer for TCP and HTTP traffic that spreads requests across a pool of servers.
The story
Before
Balancing traffic across web servers usually meant buying a dedicated hardware appliance. These were expensive and closed, so a small site had few options for keeping a site running when one server failed.
What happened, and who
Willy Tarreau wrote HAProxy, and version 1.0 was released on 16 December 2001. It runs as an ordinary program on a commodity server, forwarding TCP and HTTP connections to a set of back ends and stopping traffic to ones that fail health checks.
Why it mattered
Anyone could run a load balancer without special hardware, and HAProxy became one of the standard layer-7 proxies. It has since shipped with most mainstream Linux distributions and is common in front of busy websites.
A lock whose uncontended path stays entirely in user space and calls the kernel only to sleep or wake waiters.
The story
Before
Locks in the kernel-based style needed a system call for every lock and unlock, even when nobody else wanted the lock. A system call costs far more than the few instructions that an uncontended lock really needs.
What happened, and who
Hubertus Franke, Rusty Russell and Matthew Kirkwood presented "Fuss, Futexes and Furwocks: Fast Userlevel Locking in Linux" at the Ottawa Linux Symposium in 2002. A futex, short for fast user-space mutex, is a word of memory updated with atomic instructions, with a kernel service that sleeps a caller only if the word still has an expected value and wakes waiters on request.
Why it mattered
Uncontended locking costs a couple of atomic operations and no system call, and the kernel is involved only under contention. The futex became part of mainline Linux with the 2.6 series in December 2003 and is what POSIX mutexes and condition variables on Linux are built on.
Linux's first namespace gave each process group its own view of the mounted file systems, and began the series that makes containers possible.
The story
Before
The chroot call could confine a process to one directory tree but did not give it its own set of mounts, and did nothing for process IDs, network interfaces or user IDs. Processes started under it still saw and shared the rest of the system.
What happened, and who
Mount namespaces, mainly written by Al Viro, appeared in Linux 2.4.19 in 2002, and took the generic flag name CLONE_NEWNS because, as Michael Kerrisk wrote in his 2013 LWN series, no one then seems to have expected other kinds. Later kernels added UTS and IPC namespaces (2.6.19), PID namespaces (2.6.24), network namespaces (finished around 2.6.29) and user namespaces (3.8).
Why it mattered
A container is a process started with a private set of namespaces, so it sees its own mounts, processes and network. Docker, Kubernetes and Podman are built on this together with control groups, and the flag name NEWNS is still in the kernel interface as a reminder that it came first.
A stable merge sort that finds runs already in the data and merges them, so nearly sorted input sorts in nearly linear time.
The story
Before
Typical library sorts of the time, quicksort variants, took no advantage of data that was already partly in order, and did not keep equal items in their original order. Real data is often sorted in stretches, such as appended logs or merged lists.
What happened, and who
Tim Peters wrote Timsort for Python in 2002. It finds existing ascending or descending runs, extends short ones with insertion sort, and merges runs under rules that keep the merges balanced. The run-based idea draws on Peter McIlroy's work on adaptive sorting.
Why it mattered
Timsort is stable and takes about O(n) time on sorted or nearly sorted input while remaining O(n log n) in the worst case. It became the standard sort for objects in Python, Java (from version 7), Android and the V8 JavaScript engine (from 2018); Python later moved to a derived algorithm, Powersort, in version 3.11.
Each CPU gets its own run queue and lock, so the scheduler no longer serialises every CPU behind one lock.
The story
Before
The Linux 2.4 scheduler kept one global run queue protected by a single lock, and picking the next task meant scanning the tasks on it. On a multiprocessor every CPU queued for that lock, so scheduling stopped scaling as CPUs were added.
What happened, and who
In January 2002 Ingo Molnár announced on the Linux kernel mailing list a rewritten scheduler, the "O(1)" scheduler, saying there was no big runqueue_lock any more, only per-CPU run queues and locks. His test of processes repeatedly yielding the CPU on an 8-CPU machine rose from about 108,000 context switches per second on a 2.5.2-pre6 kernel to over 6 million.
Why it mattered
Picking the next task took constant time and CPUs no longer contended for a lock. The O(1) scheduler shipped in Linux 2.6 in 2003, and its per-CPU run queues remain the design of Linux's schedulers, which replaced it with the Completely Fair Scheduler in 2007.
One physical core presents itself as two logical processors and interleaves two threads, using slots that one thread would leave idle.
The story
Before
A superscalar core can issue several instructions per cycle, but a single thread often cannot supply them because of cache misses and dependencies. Much of the core's execution hardware sat idle while one thread waited.
What happened, and who
Intel announced Hyper-Threading Technology on its Xeon processors on 25 February 2002, in a new dual-processor server platform at 1.8 to 2.2 GHz. The operating system sees each physical processor as two logical ones, and the core interleaves instructions from both threads. Intel claimed up to 80 per cent better performance for two-way systems than previous platforms.
Why it mattered
It was the first widely sold simultaneous multithreading, gaining throughput for little added silicon. It also changed how operating systems schedule threads and count CPUs, since two logical processors share one core's caches and units, and it later became a subject of side-channel attacks.
A paper showed that the argument over events versus threads mixed up two different questions and named the chore that makes callbacks hard.
The story
Before
Programmers argued about threads versus event-driven programs as if the choice were one thing. Event-driven code is hard to write, but the reason was not clear: was it the lack of preemption, the explicit scheduling, or something else?
What happened, and who
Atul Adya, Jon Howell, Marvin Theimer, Bill Bolosky and John Douceur of Microsoft Research published "Cooperative Task Management without Manual Stack Management" at the USENIX Annual Technical Conference in Monterey, held 10 to 15 June 2002. It separates task management, which can be cooperative or preemptive, from stack management, and names the burden of splitting one logical task across callbacks manual stack management.
Why it mattered
Because the stack disappears whenever a task waits, the programmer must save local state by hand and split the code into pieces. That is the cost callbacks impose, and later designs such as promises and async/await remove it by letting the language restore the stack.
A formal proof that a system cut in two by a network partition cannot be both linearizable and available.
The story
Before
Eric Brewer of the University of California, Berkeley had put forward, in a keynote at the Symposium on Principles of Distributed Computing in 2000, the idea that a networked data service cannot provide consistency, availability and partition tolerance together. It was a conjecture, without agreed definitions of the three terms.
What happened, and who
Seth Gilbert and Nancy Lynch of MIT published "Brewer's Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services" in ACM SIGACT News in June 2002. They defined the properties precisely and proved that in an asynchronous network no service can be both atomic and always available if the network can lose messages.
Why it mattered
That turned CAP from a slogan into a theorem with stated assumptions, which is why it can be argued about precisely. It underlies how engineers choose between consistency and availability in replicated databases, though the popular "pick two" summary hides that the choice matters only when a partition happens.
A kernel object holds a persistent list of watched descriptors and reports only the ready ones, so the cost tracks activity rather than connection count.
The story
Before
The select and poll calls take the full set of descriptors on every call, and the kernel and program both walk it. With tens of thousands of mostly idle connections, most of the time goes to rescanning descriptors that have nothing to report.
What happened, and who
Davide Libenzi wrote epoll, first as a /dev/epoll device and then, at Linus Torvalds's request, as three system calls, epoll_create, epoll_ctl and epoll_wait. It was merged into the development kernel 2.5.44 in October 2002.
Why it mattered
A program registers each descriptor once and each wait returns only the ready ones, so a server can hold very many connections cheaply. That answered the C10K problem on Linux, and epoll is what nginx and Node.js use there.
Read-copy update entered the Linux development kernel as a way to read shared data without locks.
The story
Before
Linux protected read-mostly data with reader-writer locks or with special-purpose schemes. As CPU counts rose, even taking a read lock became a scaling problem because of cache-line traffic.
What happened, and who
Dipankar Sarma of IBM added RCU to the Linux 2.5.43 development kernel in October 2002, working with Paul McKenney, who had co-invented it at Sequent. The technique itself dated from the 1998 paper and a 1995 patent.
Why it mattered
RCU's use spread from a handful of places to thousands in the kernel, including routing tables and directory-entry lookups. It is the reason many kernel read paths take no lock, and it is now also available to user-space programs through libraries.
A thesis sets out the let-it-crash approach: isolated processes that fail alone and are restarted by supervisors.
The story
Before
Reliable software was usually pursued by defensive programming: catching every error where it happened and trying to carry on. Erlang, created at Ericsson in 1986 for telephone switches, had used a different style for years, but it had not been written down as a design method.
What happened, and who
Joe Armstrong, one of Erlang's creators, submitted his doctoral thesis Making Reliable Distributed Systems in the Presence of Software Errors at the Royal Institute of Technology (KTH) in Stockholm in 2003. It describes isolated processes that communicate only by messages, and supervisors that watch them and restart them when they crash.
Why it mattered
Let-it-crash and supervision trees became a shared vocabulary for fault-tolerant design, far beyond Erlang. The same shape appears in Akka's actors, in process supervisors and in restart policies for services.
A rewrite of Linux threads that let each thread be a light kernel-scheduled task and built its locks on futexes.
The story
Before
LinuxThreads, the original Linux threading library, only partly followed the POSIX threads standard, particularly in signal handling, scheduling and synchronisation between processes. It also implemented each thread as a separate process, which made creating and managing them costly.
What happened, and who
Ulrich Drepper and Ingo Molnar, working at Red Hat, wrote the Native POSIX Thread Library (NPTL) in competition with IBM's NGPT project. It first shipped in Red Hat Linux 9 in 2003, alongside the futex and scheduler work in the Linux 2.6 kernel, and later became a full part of the GNU C Library.
Why it mattered
NPTL made Linux threads cheap to create and standard-conforming, and its user-space locks use futexes so an uncontended lock never enters the kernel. Every threaded program on Linux today, from a database to a language runtime, runs on it.
Fast serial links, each joining one device directly to the chipset, replaced the shared parallel PCI bus.
The story
Before
PCI, PCI-X and AGP connected devices over wide parallel buses that they shared or that each served only one kind of card. Pushing a parallel bus faster ran into signal skew between its wires, and every added device slowed the whole bus.
What happened, and who
PCI-SIG, the industry body that maintains the PCI standards, introduced PCI Express 1.0a in 2003. Each device gets its own point-to-point connection made of one or more serial lanes, each carrying 2.5 gigatransfers per second with 8b/10b encoding.
Why it mattered
PCI Express is the connection between the processor and graphics cards, network cards and NVMe storage in nearly every server and PC. Its lane-based design lets a link's width scale from one lane to sixteen, and later generations have kept doubling the per-lane rate.
AMD extended the x86 instruction set to 64 bits, keeping every 32-bit program running at full speed.
The story
Before
Intel's route to 64-bit computing was Itanium, a new instruction set that ran existing x86 software only with significant slowdown. Servers also reached memory through a separate northbridge chip, which added latency.
What happened, and who
AMD released the first Opteron on 22 April 2003, the first processor to implement the x86-64 instruction set, which AMD called AMD64. It ran 32-bit and 64-bit programs without penalty, put a memory controller on the chip, and linked multiprocessor systems with HyperTransport.
Why it mattered
AMD64 became the 64-bit x86 standard, and Intel later adopted a compatible version. Nearly every server and PC runs its long mode today, and the on-chip memory controller made multi-socket machines the NUMA systems that operating systems still schedule around.
Spare memory on many servers is pooled into one network-wide cache that sits in front of the database.
The story
Before
A busy web site answered the same database queries again and again. When the database became the bottleneck, the only remedy was to buy a bigger database server or to split the data, and each web server had only its own small local cache.
What happened, and who
Brad Fitzpatrick wrote memcached for his blogging site LiveJournal, with an initial release on 22 May 2003. It was first written in Perl and later rewritten in C by Anatoly Vorobey, also of LiveJournal. Servers hold key-value pairs in memory, and the client library picks a server by hashing the key.
Why it mattered
Cached results can be shared by every web server, and the cache tier can grow by adding machines. Memcached became a model for the cache layer of large web sites and is used by Wikipedia, Facebook, YouTube and Twitter, among others.
Choosing keys that all collide turns a hash table's constant-time lookups into linear scans, so a few packets can exhaust a server.
The story
Before
Hash tables were analysed by their average case, which assumes keys spread evenly. Designers used fixed, published hash functions and treated the worst case as too unlikely to matter.
What happened, and who
Scott Crosby and Dan Wallach of Rice University presented "Denial of Service via Algorithmic Complexity Attacks" at the 12th USENIX Security Symposium in August 2003. They showed that an attacker who knows the hash function can send keys that land in one bucket, and demonstrated slowdowns against real software.
Why it mattered
It made the worst case a security concern for any table that holds attacker-chosen keys. Language runtimes and libraries now key their hash functions with a per-process secret, often using SipHash, so an outsider cannot predict collisions.
Split a network daemon into a small privileged part and an unprivileged part so a bug in the exposed code gains little.
The story
Before
The OpenSSH daemon ran with root privilege throughout, including while parsing untrusted input from the network. Any bug in that code, such as a buffer overflow, therefore gave an attacker full control of the machine.
What happened, and who
Niels Provos, Markus Friedl and Peter Honeyman published "Preventing Privilege Escalation" at the 12th USENIX Security Symposium in August 2003. They split OpenSSH into a small privileged monitor and an unprivileged process that does the exposed work and asks the monitor to perform the few operations that need privilege.
Why it mattered
The paper showed the technique prevented known OpenSSH vulnerabilities while cutting the code that runs with privilege. The pattern of a small trusted core and a sandboxed, untrusted worker is now standard, in browsers and in many other network daemons.
A file system that keeps petabytes across thousands of cheap machines, using one master and very large chunks.
The story
Before
Most file systems assumed reliable hardware, small files and fairly random access. Google's workload was different: multi-gigabyte files, mostly appended to and read sequentially, on commodity machines that fail every day.
What happened, and who
Sanjay Ghemawat, Howard Gobioff and Shun-Tak Leung of Google presented "The Google File System" at the ACM Symposium on Operating Systems Principles (SOSP) in October 2003. Files are split into large chunks replicated on several servers, and a single master holds the metadata.
Why it mattered
GFS set the template for storage that treats failure as normal, replicating data and repairing it automatically. It inspired the Hadoop Distributed File System, and the same design choices, a small metadata service over a large pool of data servers, recur in many storage systems.
A hypervisor that asks the guest operating system to cooperate, avoiding x86's awkward instructions and running near native speed.
The story
Before
Certain x86 instructions behave differently in user mode without trapping to the hypervisor, so a virtual machine monitor could not simply let a guest kernel run. VMware's approach was to rewrite the guest's code on the fly by binary translation, which is complex and adds overhead.
What happened, and who
Paul Barham and colleagues at the University of Cambridge published "Xen and the Art of Virtualization" at the ACM Symposium on Operating Systems Principles (SOSP) in October 2003. Xen modified the guest kernel to call the hypervisor explicitly instead of executing the troublesome instructions.
Why it mattered
Xen ran several operating systems on one machine with small overhead, and it powered Amazon's early EC2 virtual machines. Its split between a thin hypervisor and a privileged control domain shaped later hypervisor design, and paravirtual drivers remain common in virtual machines today.
A collector that splits the heap into regions and collects the most garbage-filled ones first, to keep pauses within a target.
The story
Before
The Concurrent Mark-Sweep (CMS) collector in Java did much of its work alongside the program, but it left the heap fragmented and could pause unpredictably, especially on large heaps. Users could not ask it for a pause-time limit.
What happened, and who
David Detlefs, Christine Flood, Steven Heller and Tony Printezis of Sun Microsystems presented "Garbage-First Garbage Collection" at the International Symposium on Memory Management (ISMM) in 2004. It divides the heap into regions, estimates how much live data each holds, and collects those with the least live data first.
Why it mattered
G1 gave users a way to state a pause target, and it compacts as it collects. It became an experimental option in Java SE 6 Update 14 and the default collector in Java 9.
A failure detector that reports a continuous suspicion level instead of a yes or no.
The story
Before
Failure detectors gave a binary answer: a node was trusted until a fixed timeout expired, then suspected. A short timeout falsely accused slow nodes, while a long one was slow to notice real crashes.
What happened, and who
Naohiro Hayashibara, Xavier Défago, Rami Yared and Takuya Katayama published "The φ Accrual Failure Detector" at the 23rd IEEE Symposium on Reliable Distributed Systems (SRDS) in 2004. The detector outputs a suspicion value that grows as heartbeats grow late, and each application picks its own threshold.
Why it mattered
Facebook's Cassandra adopted a modified version, and its paper reports that failure detection in a 100-node experiment fell from about two minutes to about fifteen seconds. Akka's cluster failure detector is also an implementation of it.
A small table of counters that estimates how often each item appeared in a stream, never undercounting.
The story
Before
Counting each distinct item exactly needs a table that grows with the number of distinct items, which is too much memory for a high-volume stream. Earlier approximate methods were more complicated or answered fewer kinds of query.
What happened, and who
Graham Cormode and S. Muthukrishnan presented "An Improved Data Stream Summary: The Count-Min Sketch and its Applications" at the LATIN conference in Buenos Aires in April 2004. Several hash functions each pick a counter to increment, and an item's estimate is the smallest of its counters.
Why it mattered
The sketch finds heavy hitters and approximates frequencies in space that does not grow with the number of items. Systems use it for cache admission such as TinyLFU, for network monitoring, and in database and stream-processing engines.
Intel dropped its planned Pentium 4 successor, whose clock speed would have needed too much power, and turned to multiple cores.
The story
Before
For decades each processor generation ran at a higher clock speed than the last. Intel's Pentium 4 line, built for frequency, already ran hot, and its planned successor, code-named Tejas, was designed for much higher clocks still.
What happened, and who
On 7 May 2004 Intel cancelled Tejas and its server counterpart Jayhawk, because of concerns about heat and power consumption. Intel moved its desktop, mobile and server designs to dual-core chips built on the more efficient Pentium M approach.
Why it mattered
The cancellation marked the end of the race for clock speed and the start of the multicore era, which is why software now has to be parallel to get faster. It led to Intel's Core processors.
Search completions that appear as you type, predicting the query from what other people have searched.
The story
Before
Users typed each whole query and pressed search, and a misspelling or a poor guess cost a round trip to see. There was no help while typing.
What happened, and who
Kevin Gibbs, an engineer at Google, built Google Suggest and released it as an experiment on Google Labs in December 2004. It stayed an opt-in feature for years and was made the default on google.com in August 2008.
Why it mattered
Autocomplete became an expected part of every search box. Serving it means answering a prefix lookup within a few tens of milliseconds for every keystroke, so it is a standard system-design problem.
A programming model where you write a map and a reduce function and the system handles distribution and failures.
The story
Before
Google engineers wrote many programs that processed huge inputs, each hand-coding how to split the work across machines, schedule it and recover from failures. The distribution code was larger than the computation.
What happened, and who
Jeffrey Dean and Sanjay Ghemawat of Google presented "MapReduce: Simplified Data Processing on Large Clusters" at the Symposium on Operating Systems Design and Implementation (OSDI) in December 2004. A program supplies a map function and a reduce function, and the runtime partitions the input, schedules tasks and re-runs those that fail.
Why it mattered
MapReduce led to Hadoop and the era of batch data processing that came before Spark and cloud data warehouses. Its idea, that the framework hides distribution and the user writes only the computation, remains the model for data-parallel systems.
A process can lock itself down so that it may only read, write, exit and return from signals.
The story
Before
On Linux, a process could make any system call its user allowed, so running someone else's code meant trusting it or giving it a separate machine. There was no built-in way to narrow a process's system calls.
What happened, and who
Andrea Arcangeli devised seccomp in 2005, originally to let people sell spare CPU time by safely running untrusted, compute-only programs. It was merged into Linux 2.6.12, and in strict mode a process may call only read and write on descriptors already open, exit and sigreturn.
Why it mattered
Strict mode was too narrow for most programs, but it began the idea of a per-process system-call allowlist. Its extension, seccomp-bpf, is used by Chrome, Docker, Firefox, systemd and Android.
A query engine that passes blocks of values between operators instead of one row at a time, keeping the processor busy.
The story
Before
Database engines pulled one tuple at a time through a tree of operators. Each call did little useful work, so most of the time went to interpretation overhead, and modern out-of-order processors were left idle.
What happened, and who
Peter Boncz, Marcin Zukowski and Niels Nes of CWI presented "MonetDB/X100: Hyper-Pipelining Query Execution" at the Conference on Innovative Data Systems Research (CIDR) in January 2005. Operators process vectors of a few hundred or thousand values in tight loops that fit in cache.
Why it mattered
Vectorised execution is the design of DuckDB and ClickHouse, among others, and is the main alternative to compiling queries into code. It is a large part of why analytical databases run far faster than row-at-a-time engines.
A release that added point-in-time recovery, savepoints and a native Windows port.
The story
Before
Before 8.0, PostgreSQL could restore only to the moment of its last backup, and it lacked savepoints inside a transaction. On Windows it needed the Cygwin compatibility layer.
What happened, and who
The PostgreSQL Global Development Group released version 8.0 on 19 January 2005. It added a native Windows port, point-in-time recovery from a base backup plus archived write-ahead log, subtransactions used for savepoints, and tablespaces.
Why it mattered
Point-in-time recovery from a base backup and archived WAL is the basis of PostgreSQL backups, and the same log replay later drove replication and standby servers. Subtransactions are also behind the exception blocks and savepoints that applications use today.
An essay arguing that programs no longer get faster on their own, because performance now comes from concurrency.
The story
Before
Programmers had come to expect that their software would run faster on each new processor without any change. The hardware supplied the speed-up through rising clock frequency.
What happened, and who
Herb Sutter, a C++ expert then at Microsoft, published "The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software" in Dr. Dobb's Journal in March 2005. He argued that clock speeds had stopped climbing and that gains would now come from more cores, which software must be written to use.
Why it mattered
The essay put the multicore turn in front of ordinary software developers and helped make concurrency skills a common requirement. It is still the standard reference for why programs are written as parallel today.
How two hosts behind different NAT devices open a direct connection to each other, with measurements of how often it works.
The story
Before
Network address translators let many machines share one public address, but they drop incoming traffic that no inside host asked for. Two peers behind separate NATs therefore could not connect directly and had to relay through a public server.
What happened, and who
Bryan Ford, Pyda Srisuresh and Dan Kegel presented "Peer-to-Peer Communication Across Network Address Translators" at the USENIX Annual Technical Conference in April 2005. They described and named the technique of hole punching and tested it on real devices: it worked with 82% of NATs for UDP and 64% for TCP.
Why it mattered
Hole punching is the basis of NAT traversal in the ICE protocol used by WebRTC and in voice-over-IP and peer-to-peer applications. It is why video calls can often go directly between two homes without passing through a server.
Copy a running machine's memory to another host while it keeps working, then switch over in a fraction of a second.
The story
Before
Moving a virtual machine meant suspending it, copying its state and resuming it, which froze the guest for as long as the copy took. That was too long for a machine with users connected.
What happened, and who
Christopher Clark, Keir Fraser and colleagues at the University of Cambridge presented "Live Migration of Virtual Machines" at the Symposium on Networked Systems Design and Implementation (NSDI) in May 2005. They copied memory in rounds while the guest ran, and demonstrated moving a Quake 3 game server with six players mid-game with about 60 ms of downtime.
Why it mattered
Live migration lets a cloud provider drain a host for maintenance or rebalance load without restarting the tenants' machines. It is a routine operation in virtualisation platforms today.
Two processor cores in one chip package reached ordinary desktops from both Intel and AMD within a week.
The story
Before
Desktop computers had a single processing core. To get more speed, vendors had raised clock frequency, which had run into limits of power and heat.
What happened, and who
Intel launched the Pentium D on 25 May 2005, and AMD followed with the Athlon 64 X2 on 31 May. The first Pentium D put two cores on one die, and later versions put two dies in one package.
Why it mattered
Software could now run two threads truly at the same time on a home PC, and multicore chips became the norm. Every processor since has added cores rather than relying on frequency alone.
A database design that stores each column separately and compresses it, which suits analytical queries.
The story
Before
Data warehouses stored data by rows and tried to speed up analytics with bitmap indexes and materialised views. Queries that read a few columns of a wide table still had to read whole rows from disk.
What happened, and who
Michael Stonebraker and colleagues from several universities published "C-Store: A Column-oriented DBMS" at the VLDB conference in 2005. It argued for column storage, compression and sorted projections, plus separate write and read stores.
Why it mattered
C-Store became the commercial system Vertica, and the paper renewed interest in column stores. Column-oriented storage is now the basis of most analytical databases and file formats.
A program with no special privileges can recover an encryption key by timing how its own memory accesses evict another program's cache lines.
The story
Before
Side channels, which learn secrets from physical effects such as timing, were mostly seen as a concern for smart cards. Memory protection and process isolation were assumed to keep a secret safe from other programs on the same machine.
What happened, and who
Dag Arne Osvik, Adi Shamir and Eran Tromer posted "Cache Attacks and Countermeasures: the Case of AES" to the IACR ePrint archive on 17 August 2005. Their attacks exploit the CPU cache's state, and they recovered the full key of a Linux dm-crypt encrypted partition in 65 milliseconds after 800 writes.
Why it mattered
It showed cache timing to be a practical attack between programs on one machine, and it is an ancestor of FLUSH+RELOAD and of later speculative-execution attacks such as Spectre. It is why cryptographic libraries avoid table lookups that depend on secret data.
Check every indirect jump against the set of targets the program could legitimately use, so hijacked code cannot go elsewhere.
The story
Before
Defences at the time targeted specific memory errors, such as stack canaries for buffer overflows. An attacker who found another way to overwrite a return address or function pointer could still redirect execution.
What happened, and who
Martin Abadi, Mihai Budiu, Úlfar Erlingsson and Jay Ligatti published "Control-Flow Integrity" at the ACM Conference on Computer and Communications Security (CCS) in November 2005. They defined the property and built and measured a software implementation.
Why it mattered
Control-flow integrity is used in LLVM/Clang's CFI, Windows Control Flow Guard and hardware features such as shadow stacks. It limits what an attacker gains even after finding a memory-corruption bug.
A predictor that consults several tables indexed by different history lengths and uses the longest one that matches.
The story
Before
Earlier predictors such as gshare hashed the branch address and recent history into one table, where unrelated branches could collide and share an entry. A fixed history length was either too short to catch long patterns or too long to learn quickly.
What happened, and who
André Seznec and Pierre Michaud published "A case for (partially) tagged geometric history length branch prediction" in 2006 in the Journal of Instruction-Level Parallelism. Tables use history lengths that grow geometrically, and tags in each entry reduce false sharing.
Why it mattered
TAGE and its successors won branch-prediction championships, and predictors in this family are found in modern processor cores. Branch prediction accuracy directly limits how deep a pipeline can usefully be.
Use a bandit rule at each node of a search tree to decide which move to sample next.
The story
Before
Go programs relied on alpha-beta search with hand-written evaluation knowledge, which worked poorly because Go positions are hard to score and the branching factor is large. Plain random playouts did not focus on the promising moves.
What happened, and who
Levente Kocsis and Csaba Szepesvári presented "Bandit Based Monte-Carlo Planning" at the European Conference on Machine Learning (ECML) in Berlin in September 2006. Their UCT algorithm applies the UCB1 bandit rule at every node, balancing trying promising moves against exploring others.
Why it mattered
UCT is the core of Monte Carlo tree search, which drove the first strong computer Go programs. It was later combined with neural networks in AlphaGo.
An open-source implementation of MapReduce and a distributed file system, freely available to anyone.
The story
Before
Google's GFS and MapReduce papers described how to process huge datasets on cheap machines, but the software itself was only available inside Google. Outside it, there was no open version to run.
What happened, and who
Doug Cutting and Mike Cafarella had implemented these ideas inside Apache Nutch, an open-source web crawler. Development moved to a new Hadoop subproject in January 2006, with Cutting by then working at Yahoo, and the first 0.1.0 release followed in April 2006.
Why it mattered
Hadoop put large-scale batch processing within reach of any company, and its file system and job model founded a whole ecosystem. Later systems such as Spark grew up alongside it or replaced parts of it.
Storage you rent by the gigabyte through a web API: 15 cents a month.
The story
Before
A company that needed storage bought disks, arrays and file servers, sized for its peak, and ran them itself. There was no way to rent storage that grew and shrank with use and was reached over the web.
What happened, and who
On 14 March 2006 Amazon Web Services launched the Simple Storage Service, "a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web", at $0.15 per gigabyte per month stored and $0.20 per gigabyte transferred.
Why it mattered
S3 made infrastructure a metered web service, and with EC2's beta later that year it started the public cloud. Its object interface — whole objects under keys, reached over HTTP — became the contract a whole ecosystem of object stores copies.
The line that finds the middle of a range, (low+high)/2, overflows for large arrays, and it sat unnoticed in a famous algorithm for years.
The story
Before
Binary search was the textbook example of a proven algorithm, first published in 1946, with a correct version for all sizes appearing only in 1962. The proofs assumed integers that never overflow, which machine integers do.
What happened, and who
Joshua Bloch of Google wrote on the Google Research blog on 2 June 2006 that his implementation of binary search in Java's libraries had a bug for about nine years. Computing (low + high) / 2 in 32-bit int overflows once an array has around 230 elements.
Why it mattered
The post showed that proving an algorithm correct does not prove that its implementation on fixed-width machine integers is. The safer form low + (high - low) / 2 is now the standard idiom.
Virtual servers rented by the hour over the internet, with no contract or hardware to buy.
The story
Before
Getting a server meant buying it, or leasing it for months, and then waiting for it to be installed. Capacity had to be planned for peak demand, and unused machines still cost money.
What happened, and who
Amazon Web Services announced the Elastic Compute Cloud in a limited beta on 25 August 2006, in a post by Jeff Barr. It offered a single instance type, the m1.small, in a single region, US East, at ten cents an hour, and was open only to existing AWS customers at first.
Why it mattered
Compute became something you could request and release in minutes and pay for by the hour, and with S3 it started public cloud computing. Auto scaling, serverless functions and much of modern infrastructure practice depend on that elasticity.
A personalised stream of friends' activity that replaced having to visit each profile.
The story
Before
On Facebook you had to visit each friend's profile to see what had changed. The site had no single place that gathered updates for you.
What happened, and who
Facebook launched News Feed on 5 September 2006, with Ruchi Sanghvi as product manager. It showed each user a personalised list of stories about friends' activity, and provoked an immediate wave of user complaints about privacy.
Why it mattered
News Feed became Facebook's most influential feature and set the pattern for social media. Building it forced the choice between assembling each feed when it is written or when it is read, the fan-out problem that feed designs still face.
The first processors with hardware virtualisation support were often no faster than software binary translation.
The story
Before
Intel and AMD had just added hardware support so a hypervisor could run a guest kernel directly, and most people expected it to beat VMware's software approach. Software had to translate guest code on the fly, which looked slow by comparison.
What happened, and who
Keith Adams and Ole Agesen of VMware published "A Comparison of Software and Hardware Techniques for x86 Virtualization" at ASPLOS in October 2006. They measured the primitives, finding for example that a VM entry cost 2,409 cycles on a Pentium 4, and found that software translation often kept up.
Why it mattered
It is a lasting example of measuring instead of assuming. Hardware support later improved, notably with nested page tables, and became the basis of virtualisation, but the paper explains why the first generation disappointed.
An open competition with a million-dollar prize for predicting film ratings 10% more accurately than Netflix could.
The story
Before
Recommendation research had no large public dataset to compare methods on. Companies kept their ratings private, so results from different groups could not be compared.
What happened, and who
Netflix launched the Netflix Prize on 2 October 2006, offering US$1 million to any team that beat its Cinematch algorithm by 10% in prediction error. On 21 September 2009 the team BellKor's Pragmatic Chaos achieved a 10.06% improvement and won.
Why it mattered
The contest brought matrix factorisation, which describes users and films by short vectors of learned factors, to wide attention as a recommendation method. It also showed how a public dataset and a leaderboard can drive research.
A sorted table spread over thousands of machines, built on the Google File System and the Chubby lock service.
The story
Before
Google's products stored huge structured datasets, such as web indexes and satellite images, that a single relational database could not hold or serve. The company needed something with a simpler model that could spread across many machines.
What happened, and who
Fay Chang and colleagues at Google presented "Bigtable: A Distributed Storage System for Structured Data" at the Symposium on Operating Systems Design and Implementation (OSDI) in November 2006. Data is held in a sparse, sorted table divided into tablets, written first to a log and an in-memory table and then to immutable files on GFS.
Why it mattered
Bigtable's design of a memtable, immutable sorted files and compaction became the model for many storage engines. It directly inspired HBase and Cassandra, and it is the starting point for later systems such as Spanner.
A small cluster that uses consensus to offer locks and files, so other services need not run their own.
The story
Before
Each Google service that needed a single leader or agreement on a small fact tended to build its own fragile election code. Getting consensus right in each one was hard and error-prone.
What happened, and who
Mike Burrows of Google presented "The Chubby Lock Service for Loosely-Coupled Distributed Systems" at the Symposium on Operating Systems Design and Implementation (OSDI) in November 2006. Chubby is a replicated service, built on Paxos consensus, that offers coarse-grained locks and small files.
Why it mattered
Chubby made consensus something a system could call as a service, and it was the model for Apache ZooKeeper and etcd. Leader election and configuration stores in many distributed systems still work the way it did.
A graphics chip built from many identical processors that could run ordinary programs, not just draw triangles.
The story
Before
Earlier graphics chips had a fixed pipeline with separate hardware for vertex and pixel shading, and they were hard to use for anything but graphics. Programmers had to disguise computations as drawing operations.
What happened, and who
NVIDIA released the GeForce 8800 on 8 November 2006, the first product based on its Tesla architecture. It had unified shader processors that run threads in groups, an execution style NVIDIA calls SIMT.
Why it mattered
The design let programmers use GPUs for general computation, and NVIDIA's CUDA followed in 2007. The same model now trains and runs neural networks.
A way to choose the starting centres for k-means that gives a provable guarantee on how good the result is.
The story
Before
The k-means algorithm needs starting centres, and choosing them uniformly at random could leave the algorithm stuck in a poor solution, with no bound on how far it was from the best.
What happened, and who
David Arthur and Sergei Vassilvitskii published "k-means++: The Advantages of Careful Seeding" at the ACM-SIAM Symposium on Discrete Algorithms (SODA) in January 2007. Each new centre is chosen with probability proportional to its squared distance from the centres already picked, which gives an expected O(log k) approximation to the optimal cost.
Why it mattered
k-means++ is the default initialisation in libraries such as scikit-learn. It usually gives better clusterings and fewer iterations than random seeding.
Field data on about 100,000 disks showed they fail much more often than their datasheets suggest, and in patterns not independent of each other.
The story
Before
Storage designers, and RAID reliability models in particular, assumed disk failures were independent and constant in rate, at the level the manufacturer's mean-time-to-failure figure implied. That figure suggested about 0.88% of disks would fail per year.
What happened, and who
Bianca Schroeder and Garth Gibson of Carnegie Mellon University presented "Disk Failures in the Real World: What Does an MTTF of 1,000,000 Hours Mean to You?" at the USENIX Conference on File and Storage Technologies (FAST) in February 2007. Studying about 100,000 disks in the field, they found replacement rates of 2-4% common and up to 13% in some systems, rising with age.
Why it mattered
Failures were correlated, not independent, and rates did not differ much between SCSI, Fibre Channel and SATA drives. Designers therefore plan RAID rebuild windows and replication with more caution than a simple model suggests.
An open-source message broker that implements the AMQP protocol and is easy to run yourself.
The story
Before
Message brokers such as IBM's MQSeries were proprietary and expensive, and each had its own protocol. Applications that used one were locked to its vendor.
What happened, and who
RabbitMQ was developed by Rabbit Technologies, a joint venture of LShift and CohesiveFT, and launched in February 2007. It is written in Erlang and implemented AMQP, a protocol whose development was led by JPMorgan Chase and whose specification was published in 2006.
Why it mattered
RabbitMQ made queues, routing and acknowledgements available to any team, and it became one of the most widely used message brokers. Its model of exchanges and queues is still how many engineers first learn asynchronous messaging.
Turns the Linux kernel itself into a hypervisor, so a virtual machine is just a process that opens /dev/kvm.
The story
Before
Xen, the leading open-source hypervisor of the time, ran underneath the operating system as its own small kernel and demoted Linux to a management guest, and its guests had to be modified to cooperate. It also had to duplicate what Linux already did well: scheduling, memory management and power handling. Full virtualisation of unmodified guests on x86 had been hard because some privileged instructions could not be safely trapped.
What happened, and who
Avi Kivity of the Israeli company Qumranet announced KVM (Kernel-based Virtual Machine) on 19 October 2006. It uses the new hardware virtualisation extensions, Intel VT-x and AMD-V, and appears to programs as the device /dev/kvm. It was merged into the kernel and shipped in Linux 2.6.20, released on 4 February 2007.
Why it mattered
A virtual machine became an ordinary Linux process, scheduled, limited and accounted for by the same kernel code as everything else, and every improvement to Linux benefited KVM for free. It is now the hypervisor under most public clouds, and tools such as QEMU, libvirt and the microVM runtimes behind sandboxed containers all build on it.
A tiny proxy that lets thousands of clients share a few dozen PostgreSQL connections.
The story
Before
PostgreSQL starts one operating-system process for every client connection, and each carries memory and scheduling cost whether or not it is busy. An application with thousands of mostly idle connections therefore either exhausted the server or had to be restructured around a small, hand-managed pool.
What happened, and who
PgBouncer 1.0, the first public release of the pooler, appeared on 13 March 2007. Marko Kreen wrote it for use at Skype, the internet-telephony company, and released it after it had run there in production.
Why it mattered
Putting a pooler between the application and the database is still the standard answer to PostgreSQL's connection limit. PgBouncer, and later poolers modelled on it, sit in front of most busy PostgreSQL deployments, including managed services, and it explains why connection counts are a capacity setting rather than something an application can ignore.
Counts the distinct items in a huge stream to within a couple of percent using a few kilobytes.
The story
Before
Counting distinct items exactly needs memory proportional to the number of distinct items, since each must be remembered to avoid counting it twice. Flajolet and Durand's earlier LogLog algorithm got by with tiny memory but had a standard error of about 1.30/√m for m registers, which was still larger than it needed to be.
What happened, and who
Philippe Flajolet, Éric Fusy, Olivier Gandouet and Frédéric Meunier published “HyperLogLog: the analysis of a near-optimal cardinality estimation algorithm” at the Analysis of Algorithms (AofA) conference in June 2007. It keeps m small registers and combines them with a harmonic mean, which brings the standard error down to about 1.04/√m.
Why it mattered
Approximate distinct counts became cheap enough to compute for every page, user or query. HyperLogLog, and Google's 2013 refinement HyperLogLog++, are behind Redis's PFCOUNT, approximate distinct counts in BigQuery and many other analytics engines, and the registers of two sketches can be merged, which suits distributed aggregation.
A large field study of disk sectors that silently become unreadable and are discovered only when read.
The story
Before
Disk failure was pictured as a whole drive dying, which a RAID array masks. A latent sector error is different: one sector becomes unreadable and nobody knows until the data is next read, which may be during the very reconstruction that is meant to recover from another failure. Earlier field studies were too small to say how common this was or how the errors cluster.
What happened, and who
Lakshmi Bairavasundaram, Garth Goodson, Shankar Pasupathy and Jiri Schindler of NetApp and the University of Wisconsin–Madison published “An Analysis of Latent Sector Errors in Disk Drives” at SIGMETRICS in June 2007. They studied about 1.53 million disks in NetApp's production systems over 32 months and found that the errors are not independent events, differ by drive type and cluster in space and time.
Why it mattered
The study made latent errors a measured, common risk rather than a rare accident, and showed why a drive cannot be trusted to report its own damage. It supports the practice of periodic scrubbing, reading every sector while redundancy still exists, and of checksumming data end to end, as file systems such as ZFS and btrfs do.
Argues that the one-size-fits-all relational database is obsolete and that OLTP wants main-memory, partitioned engines.
The story
Before
Relational systems such as Oracle, DB2 and PostgreSQL descended from System R, built in the 1970s for small memories and slow disks. Every transaction therefore paid for a disk-oriented buffer pool, record-level locking and a write-ahead log, even when the whole database would now fit in memory.
What happened, and who
Michael Stonebraker, Samuel Madden, Daniel Abadi, Stavros Harizopoulos, Nabil Hachem and Pat Helland published “The End of an Architectural Era (It’s Time for a Complete Rewrite)” at VLDB 2007 in Vienna, in September 2007. It argued for specialised engines and described a main-memory, partitioned, single-threaded prototype called H-Store.
Why it mattered
The paper gave the in-memory and NewSQL lines their argument: remove the buffer pool, locks and heavyweight logging, and transaction throughput rises by a large factor. H-Store became the commercial VoltDB, and its ideas of partitioned, serial execution reappear in later in-memory and deterministic databases.
A key-value store that never refuses a write, trading consistency for availability.
The story
Before
Amazon's services such as the shopping cart mostly needed simple lookups by key, not the full relational model, and could not tolerate refused writes. Relational databases with a single primary or strict synchronous replication stopped accepting updates during failures and partitions, which meant lost carts and lost sales.
What happened, and who
Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall and Werner Vogels of Amazon presented “Dynamo: Amazon's Highly Available Key-value Store” at SOSP in October 2007. Any replica can accept a write, consistent hashing spreads keys across nodes, sloppy quorums and hinted handoff keep writes flowing during failures, and vector clocks record conflicting versions for the application to merge.
Why it mattered
The paper made eventual consistency and the N, R and W replication knobs standard vocabulary. It directly inspired Cassandra and Riak, and Voldemort at LinkedIn, and set the template for leaderless databases. Amazon DynamoDB, launched in 2012, borrowed the name and the key-value model but not the leaderless design.
Schedules by giving every task an equal share of the CPU, tracked as one number, instead of by heuristics.
The story
Before
Linux's previous scheduler, the O(1) scheduler, picked tasks in constant time from priority queues but had to guess which tasks were interactive from sleep behaviour. Those heuristics were intricate and could misjudge a task, leaving desktop programs sluggish.
What happened, and who
Ingo Molnar wrote the Completely Fair Scheduler (CFS), which shipped in Linux 2.6.23, released on 9 October 2007. It keeps each runnable task in a red-black tree ordered by virtual runtime, the CPU time it has received scaled by its weight, and always runs the task that has received the least.
Why it mattered
CFS replaced the guesswork with a single fairness rule that a nice value or a cgroup share can weight directly, and that made cgroup CPU limits practical. It was the default scheduler for sixteen years, until the EEVDF scheduler replaced it in Linux 6.6 in 2023, and virtual-runtime fairness remains the reference design for general-purpose schedulers.
A long paper that explains caches, TLBs and NUMA, and measures their costs, including false sharing.
The story
Before
CPUs had grown far faster than DRAM, so the cache hierarchy decided real performance, yet most programmers wrote as if memory were a flat array with uniform cost. The details were spread over vendor manuals and folklore.
What happened, and who
Ulrich Drepper, then maintaining the GNU C library at Red Hat, wrote “What Every Programmer Should Know About Memory”, published in instalments on LWN.net in autumn 2007 and issued as a single paper in November 2007. It covers how DRAM and caches work, virtual memory costs and NUMA, and measures effects such as cache-line contention between threads.
Why it mattered
It became the standard long-form explanation of why data layout, access order and cache-line sharing dominate performance. Engineers still cite it when explaining why an array beats a linked list, why threads that write adjacent variables slow each other down, and why NUMA placement matters.
An article for practitioners that names the consistency choices in replicated systems and their trade-offs.
The story
Before
Consistency models were discussed in the research literature on distributed systems and weak consistency, but most engineers building web services thought only in terms of a database that was either correct or down. There was no shared, practical vocabulary for the middle ground.
What happened, and who
Werner Vogels, Amazon's chief technology officer, published “Eventually Consistent” in ACM Queue in October 2008 (volume 6, issue 6), building on a blog post of the same theme a year earlier. It lays out the consistency variants a client can be given, such as read-your-writes and monotonic reads, and the N, R and W replica settings that select them.
Why it mattered
It gave industry a common set of terms for what a replicated store promises, and framed the choice between consistency and availability as a decision made per application. The phrase and its variants are now standard in the documentation of systems from Cassandra to DynamoDB.
Lets the kernel limit, account for and prioritise the resources used by a whole group of processes.
The story
Before
Linux limits applied to a single process, such as ulimit and nice values. There was no way to say that all the processes of one job or customer together may use at most a given share of CPU and memory, so on a shared server one greedy job could starve the others.
What happened, and who
Paul Menage and Rohit Seth at Google developed “process containers”, which were renamed control groups to avoid confusion with other uses of “container”. They were merged into Linux 2.6.24, released on 24 January 2008.
Why it mattered
Control groups are the resource half of what is now called a container: the kernel enforces each group's CPU, memory and I/O limits and reports its usage. Docker's limit flags, Kubernetes requests and limits, and systemd's resource control are all built on cgroups.
Lets an update reuse space and skip index changes when no indexed column changes.
The story
Before
In PostgreSQL an update writes a new version of the row, and every index on the table needed a new entry pointing at it, even when the changed column was not indexed. The dead versions and their index entries had to wait for vacuum, so update-heavy tables bloated and wrote far more than they changed.
What happened, and who
PostgreSQL 8.3, released on 4 February 2008, added heap-only tuples (HOT), credited in the release notes to Pavan Deolasee with ideas from many others. When an update changes no indexed column and the new version fits on the same page, it needs no new index entries and old versions can be cleaned up by ordinary page pruning. The same release also shrank tuple headers and moved full-text search into the core.
Why it mattered
HOT reduced write amplification and table bloat for the common case of updating non-indexed columns, and it is a reason why leaving a column out of an index can make an update workload faster. It is also why PostgreSQL tables keep some free space on each page.
One ISP's attempt to block a site inside its country made the site unreachable for much of the world.
The story
Before
BGP, the protocol by which networks announce which address blocks they can reach, believes what it is told: a neighbour's announcement is accepted and passed on without checking that the announcer owns the addresses. Routers also prefer the most specific matching prefix, so a narrower announcement wins over a wider one.
What happened, and who
On 24 February 2008 Pakistan Telecom, following an order to block YouTube inside Pakistan, announced a more specific route for part of YouTube's address space to divert local traffic to a dead end. Its upstream provider, PCCW, passed the announcement to the rest of the Internet, and because it was more specific, networks around the world began sending YouTube traffic to Pakistan, where it was dropped.
Why it mattered
It is the standard example of how one mistaken announcement can take a major service offline globally, and it made the case for cryptographic proof of who may originate an address block. That led to RPKI origin validation, now used by many large networks to reject announcements from unauthorised origins.
A switch exposes its forwarding table so an outside controller can decide where packets go.
The story
Before
Switches and routers were closed boxes: forwarding logic and control logic lived together in the vendor's firmware, so researchers could not try new routing or forwarding ideas on real networks at scale, and operators could not change behaviour except through the vendor's configuration options.
What happened, and who
Nick McKeown, Tom Anderson, Hari Balakrishnan, Guru Parulkar, Larry Peterson, Jennifer Rexford, Scott Shenker and Jonathan Turner published “OpenFlow: Enabling Innovation in Campus Networks” in ACM SIGCOMM Computer Communication Review in April 2008. It proposed a small standard interface for adding, removing and reading flow-table entries in a switch, so that a separate controller program decides the rules.
Why it mattered
OpenFlow is the origin of software-defined networking: separating the controlling software from the forwarding hardware. The idea underlies the centrally controlled networks of large data centres and cloud providers and the network virtualisation used in cloud platforms.
Makes snapshot isolation truly serializable by watching for a specific dangerous pattern of read-write dependencies.
The story
Before
Snapshot isolation gave readers a consistent view without blocking writers, but it allows anomalies such as write skew, so it is not serializable. Strict two-phase locking is serializable, but it makes readers block writers and lowers concurrency.
What happened, and who
Michael Cahill, Uwe Röhm and Alan Fekete published “Serializable Isolation for Snapshot Databases” at SIGMOD in June 2008. They showed that every snapshot-isolation anomaly involves a transaction with both an incoming and an outgoing read-write dependency, and proposed aborting one transaction in such a pivot, without taking read locks.
Why it mattered
It gave a route to serializability at close to snapshot-isolation cost. PostgreSQL 9.1 used it, following work by Dan Ports and Kevin Grittner, as its SERIALIZABLE level, and it is why serializable transactions in PostgreSQL can fail with a serialisation error that the application must retry.
Combines Dynamo's distribution with Bigtable's on-disk storage design.
The story
Before
Facebook needed to store and search users' inboxes across many servers. Dynamo, described in 2007, showed how to replicate a key-value store without a single master but offered only simple keys and values, and Google's Bigtable, described in 2006, offered a richer column model but relied on a master and a distributed file system.
What happened, and who
Avinash Lakshman, one of Dynamo's authors, and Prashant Malik built Cassandra at Facebook for inbox search, and Facebook released it as open-source software on Google Code in July 2008. It became an Apache incubator project in March 2009 and a top-level Apache project in February 2010.
Why it mattered
Cassandra brought leaderless replication and log-structured storage to a widely used open-source database. It became a leading choice for write-heavy, always-available workloads, and the trade-offs it exposes, such as tunable consistency and compaction strategy, are ones engineers now meet in many systems.
A compact binary format described by a schema, so services can add fields without breaking old readers.
The story
Before
XML and other text formats were verbose and slow to parse, and passing loosely structured messages between services made it hard to change a message safely over time.
What happened, and who
Google used Protocol Buffers internally from about 2001 and released it as open-source software on 7 July 2008. A .proto file declares each message and gives every field a number, and generated code in each language reads and writes a compact binary encoding keyed by those numbers.
Why it mattered
Because fields are identified by number, old and new programs can exchange messages while the schema evolves, which is the point of a schema-evolution discipline in RPC systems. Protocol Buffers is the default message format of gRPC and is widely used for storage and configuration.
Asking a resolver for made-up names lets an attacker race it again and again until a forged answer sticks.
The story
Before
An attacker who could guess a resolver's 16-bit query ID could forge a reply and get a false address cached. But a fair attempt failed as soon as the real answer arrived and was cached for its time-to-live, so the attacker had to wait for it to expire before trying again, which made the attack impractical.
What happened, and who
Dan Kaminsky of IOActive found that asking for random non-existent names under a target domain, such as 1.example.com, gives an unlimited series of fresh races, since none of those names is cached. A forged reply can carry extra records that take over the whole domain. He coordinated the disclosure with CERT/CC and vendors, who released patches on 8 July 2008 (VU#800113, CVE-2008-1447).
Why it mattered
The fix, randomising the UDP source port of each query, added about 16 bits of unpredictability to what an attacker has to guess, and every major resolver adopted it. The episode also revived interest in DNSSEC, which authenticates DNS answers rather than making forgery harder.
Showed how to reroute traffic through an attacker's machine while it still arrives at its destination.
The story
Before
BGP hijacks such as the YouTube incident earlier that year had simply dropped the traffic they diverted, which made them obvious to the victim. It was assumed that a hijack could disrupt but not quietly eavesdrop.
What happened, and who
Tony Kapela and Alex Pilosov presented “Stealing the Internet: An Internet-Scale Man in the Middle Attack” at DEF CON 16 in Las Vegas on 10 August 2008. They announced a route for the target's addresses while carrying the legitimate networks in the AS path, so a working path onward to the real destination remained, and intercepted traffic bound for the conference network before passing it on.
Why it mattered
It turned BGP hijacking from an outage risk into a surveillance risk, since the victim sees no failure. It is cited in the case for authenticating routes and, with later leaks, for monitoring routing paths as well as origins.
A JavaScript engine that compiles only the hot loops it observes running, after specialising them to the types it saw.
The story
Before
SpiderMonkey, Firefox's JavaScript engine, was a bytecode interpreter. Every operation had to check the types of its operands at run time, so loops that did the same work with the same types each time still paid the checking cost on every iteration.
What happened, and who
Andreas Gal, with Brendan Eich and colleagues at Mozilla and UC Irvine, announced on 22 August 2008 that TraceMonkey, a tracing just-in-time compiler for JavaScript, had landed in Firefox's development tree. It records the path through a hot loop, compiles that trace to machine code with type-specialised operations, and falls back to the interpreter when a guard fails. It shipped to users in Firefox 3.5 in June 2009.
Why it mattered
Tracing gave large speedups on loop-heavy code, up to about 20 times faster than the interpreter on some benchmarks, but performance was uneven when code left the traced paths. Mozilla removed it in 2011 in favour of a method-based JIT, but the idea lived on in tracing compilers such as PyPy's and LuaJIT.
A JavaScript engine that compiles straight to machine code, using hidden classes to make property access fast.
The story
Before
Browser JavaScript engines of the time were mostly bytecode interpreters. JavaScript objects behave like dictionaries whose properties can change at any time, so looking up obj.x meant a hash lookup at every access, which made web applications slow.
What happened, and who
Google released Chrome on 2 September 2008 with V8, an engine led by Lars Bak that compiled JavaScript directly to machine code. V8 gives objects that have been built the same way a shared hidden class describing their layout, and uses inline caches at each property access to jump straight to the property's offset.
Why it mattered
V8 made large JavaScript applications feasible and started a speed race among browser engines. It also powers Node.js, which brought JavaScript to servers, and its later tiered compiler, which starts with an interpreter and optimises hot code, is a widely copied design.
Intel moved the memory controller onto the processor and replaced the shared bus with point-to-point links.
The story
Before
Intel's processors before Nehalem reached memory and each other over the front-side bus, a single shared channel through a separate memory-controller chip. As core counts grew, all cores and processors competed for that one bus, which limited memory bandwidth and made latency uniform but high.
What happened, and who
Intel launched the Nehalem microarchitecture in November 2008, starting with the Core i7. It integrated the memory controller on the processor die and, in higher-end models, replaced the front-side bus with the QuickPath Interconnect (QPI), a set of point-to-point links between processors.
Why it mattered
With each processor owning its local memory, multi-socket x86 servers became NUMA machines: memory attached to another socket is slower to reach than local memory. That is why operating systems and databases on x86 servers now care where threads and their memory are placed. AMD had taken the same step with HyperTransport in 2003.
Draws high-dimensional data on a plane so that points that were close stay close.
The story
Before
Principal component analysis (PCA) projects data along straight directions of greatest variance, so it cannot unfold data lying on a curved surface, and it tends to pile up points that are near neighbours in the original space. Stochastic Neighbor Embedding, proposed by Hinton and Roweis in 2002, tried to preserve neighbourhoods but was hard to optimise and crowded points together.
What happened, and who
Laurens van der Maaten and Geoffrey Hinton published “Visualizing Data using t-SNE” in the Journal of Machine Learning Research, volume 9, in November 2008. It compares neighbour probabilities in the original space with those in the map and minimises their divergence, using a heavy-tailed Student t distribution in the map to relieve the crowding.
Why it mattered
t-SNE became the usual way of drawing a picture of learned representations, such as word embeddings or the internal features of a neural network. Its maps show cluster structure well, but distances between clusters and cluster sizes can mislead, which practitioners have to keep in mind.
Keeps messages in a partitioned, append-only log that readers can replay, rather than deleting them once read.
The story
Before
Message queues such as RabbitMQ and ActiveMQ removed a message once a consumer had taken it, and were not designed for the volume of activity data, such as page views and searches, that a large website produces. Feeding the same data to several systems, such as Hadoop and search, meant separate pipelines.
What happened, and who
Jay Kreps, Neha Narkhede and Jun Rao began building Kafka at LinkedIn from about 2009, with a first open-source release in early 2011. Messages are appended to a partitioned log on disk and kept for a set time, and each consumer tracks its own position, so many consumers can read the same stream at their own pace and replay it.
Why it mattered
Treating the log as the central data structure made it a shared backbone between systems. Kafka is now the standard platform for event streaming, change-data-capture pipelines and event-driven architectures, and it popularised the idea of the log as a source of truth.
Google took down a lock service on purpose because it had become too reliable for its own good.
The story
Before
Chubby, Google's distributed lock service, was so dependable that outages were rare. Teams began to assume it never failed and built hard dependencies on it, so a real outage would have caused failures far beyond what its owners expected.
What happened, and who
From 2009, Chubby's site reliability engineers began forcing periodic, planned outages when the service had exceeded its availability target, so that teams depending on it would find and remove their hidden assumptions. Eric Brewer describes the practice in his note “Spanner, TrueTime and the CAP Theorem”, and Google's Site Reliability Engineering book describes it as the global Chubby planned outage.
Why it mattered
It is an early and widely cited example of injecting failure deliberately to keep dependants honest, and it shows that an availability target is a ceiling as well as a floor. The practice foreshadows chaos engineering, and the argument applies to any internal service that others treat as infallible.
Generates a tracing compiler automatically from an interpreter, rather than writing one by hand for each language.
The story
Before
Just-in-time compilers were built by hand for one language at a time, at great cost, and each new language needed a new one. TraceMonkey and LuaJIT, for example, were specific to JavaScript and Lua.
What happened, and who
Carl Friedrich Bolz, Antonio Cuni, Maciej Fijalkowski and Armin Rigo published “Tracing the meta-level: PyPy's tracing JIT compiler” at the ICOOOLPS workshop in 2009. The PyPy project writes a Python interpreter in a restricted subset of Python, and its toolchain traces that interpreter as it runs a program, so the compiler comes from the interpreter with hints from its author.
Why it mattered
The approach, called meta-tracing, lets a language implementer get a fast JIT by writing only an interpreter. PyPy became a faster Python for many workloads, and related ideas, partial evaluation in Truffle and Graal, apply the same principle to other languages.
A server that keeps data structures such as lists, sets and counters in memory and lets clients operate on them.
The story
Before
Memcached, released in 2003, held opaque strings in memory with no persistence or structure, so a program had to fetch a value, change it and write it back for even a counter or a list. Relational databases were too slow for the real-time analytics that Salvatore Sanfilippo was building.
What happened, and who
Salvatore Sanfilippo, known as antirez, wrote Redis in 2009 for a real-time web log analysis service he was building. It began as a prototype in Tcl and was rewritten in C, and it offers lists, sets, sorted sets and hashes with atomic operations, and optional persistence to disk.
Why it mattered
Because operations such as increment or push are atomic on the server, Redis is widely used for counters, rate limiters, queues, leaderboards and session stores as well as for caching. It is one of the most deployed data stores in web applications.
A precise, machine-checked model of how x86 processors reorder memory operations.
The story
Before
Intel and AMD described x86 memory ordering in prose, and the manuals were ambiguous and at times contradicted what real processors did. Programmers writing lock-free code could not tell which reorderings were allowed.
What happened, and who
Scott Owens, Susmit Sarkar and Peter Sewell of the University of Cambridge presented “A Better x86 Memory Model: x86-TSO” in 2009, and Sewell and co-authors published the fuller “x86-TSO: A Rigorous and Usable Programmer's Model for x86 Multiprocessors” in Communications of the ACM in July 2010. The model is a processor with a FIFO store buffer in front of shared memory, defined in the HOL4 proof assistant and checked against hardware behaviour.
Why it mattered
It gave a short, exact account of the one reordering x86 allows, a later load overtaking an earlier store to a different address, and thereby of when a fence is needed. It is the model that lock-free algorithms and compiler and language memory models for x86 are checked against.
Streams video as a series of small files fetched over ordinary HTTP, so plain web servers and CDNs can serve it.
The story
Before
Streaming video used protocols such as RTSP and Adobe's RTMP, which keep a connection to a special streaming server and are often blocked by firewalls. They could not use the ordinary web caches and content delivery networks that already carried everything else.
What happened, and who
Roger Pantos of Apple posted the first Internet-Draft for HTTP Live Streaming (HLS) on 1 May 2009. A stream is a playlist that lists short media segments, which the client downloads in turn; variant playlists at different bitrates let it switch quality as the network changes. The draft was later published as RFC 8216.
Why it mattered
Since delivery is ordinary HTTP, video could use existing caches and CDNs, and the client could adapt to its bandwidth. HLS and the similar MPEG-DASH are how most internet video is delivered, and their segment-and-playlist design shapes how video services are built.
A database of millions of labelled photographs organised by the WordNet hierarchy, built by crowdsourcing.
The story
Before
Vision datasets of the time, such as Caltech 101 and PASCAL VOC, held thousands to tens of thousands of images in a modest number of categories. That was too little to train or fairly compare methods on recognising the variety of objects in the world.
What happened, and who
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li and Fei-Fei Li of Princeton University published “ImageNet: A Large-Scale Hierarchical Image Database” at CVPR in June 2009. Its images were collected from the web and labelled by workers on Amazon Mechanical Turk, organised by WordNet's noun hierarchy.
Why it mattered
ImageNet supplied the data that made large neural networks worth training. Its annual challenge, ILSVRC, was won in 2012 by AlexNet, and models pre-trained on ImageNet became the starting point for transfer learning across computer vision.
A proof, checked by machine, that an operating-system kernel's C code does what its specification says.
The story
Before
Kernels were tested and reviewed but never proven correct, and a single kernel bug can break every security guarantee above it. Earlier formal verification of operating systems had covered only small or simplified systems.
What happened, and who
Gerwin Klein and colleagues at NICTA in Australia presented “seL4: Formal Verification of an OS Kernel” at SOSP in October 2009. Using the Isabelle/HOL proof assistant they proved that about 8,700 lines of C and 600 lines of assembler in the seL4 microkernel implement its abstract specification.
Why it mattered
It showed that a real, general-purpose kernel could be fully verified, and ruled out whole classes of bugs such as buffer overflows and null-pointer errors in that code. seL4 is used in defence, aviation and automotive systems, and it strengthens the case for small kernels whose trusted code is small enough to verify.
Deep learning, containers and cloud-native infrastructure.
DiscoveryLanguages and runtimes
The GIL on Multicore
Measurements showed that Python threads doing CPU work on several cores ran slower than on one.
The story
Before
CPython's global interpreter lock lets only one thread run Python code at a time. That was thought to limit threads to one core's worth of speed on multicore machines, but not to make them worse than a single core.
What happened, and who
David Beazley presented “Understanding the Python GIL” at PyCon in Atlanta on 20 February 2010, backed by measurements on an instrumented interpreter. On a multicore machine, two CPU-bound threads ran about twice as slow as running sequentially, since the threads repeatedly fought over the lock, and I/O threads were badly delayed by CPU-bound ones.
Why it mattered
The talk exposed a concrete flaw in how the old GIL handed off between threads. Antoine Pitrou's redesigned GIL, with a switch interval and a request to drop it, shipped in Python 3.2, and Beazley's numbers explain why Python programs use processes rather than threads for CPU-bound parallelism.
A tracing system that follows one request across many services, sampling a fraction of them to keep the cost low.
The story
Before
A user request at Google might touch dozens of services. Engineers could see that overall latency was high, or which service was slow on average, but not where a particular slow request had spent its time.
What happened, and who
Benjamin Sigelman and colleagues at Google published the technical report “Dapper, a Large-Scale Distributed Systems Tracing Infrastructure” in April 2010. Each request carries a trace identifier, each service records timed spans linked to their parent, and the system samples only a small fraction of requests.
Why it mattered
Dapper defined the vocabulary of trace and span and showed that sampling keeps tracing cheap enough to leave on. Twitter's Zipkin, Uber's Jaeger and the OpenTelemetry standard descend from it, and tracing is now a routine part of debugging microservices.
For about fifteen minutes, an ISP announced many thousands of address blocks it did not own, pulling in traffic for them.
The story
Before
BGP accepts announcements from a neighbour at face value, and network operators filter them only by convention and by their own configuration. A mistaken or malicious announcement could therefore propagate widely before anyone noticed.
What happened, and who
On 8 April 2010 China Telecom's AS23724, which normally announces only a few dozen prefixes, announced tens of thousands of address blocks belonging to other networks. Roughly 37,000 unique prefixes by one measurement, and about 50,000 in a US government report, were affected for roughly fifteen minutes, some of them belonging to US government and military networks.
Why it mattered
The incident is a standard example of BGP's lack of origin checks, and, when a network keeps a working path onward, of how traffic can be observed without an outage. It fed the debate about routing security in the US and became an argument for filtering customer announcements and for RPKI origin validation.
Extends the CAP theorem to normal operation: even without a partition, a replicated system trades latency against consistency.
The story
Before
The CAP theorem says that during a network partition a system must choose between consistency and availability, but says nothing about how a system behaves the rest of the time, which is almost always. Its two-way framing hid the fact that replicas trade consistency for speed even when nothing is broken.
What happened, and who
Daniel Abadi, a Yale professor, proposed PACELC in a blog post, “Problems with CAP, and Yahoo's little known NoSQL system”, on 23 April 2010. It reads: if there is a partition (P), choose availability (A) or consistency (C); else (E), choose latency (L) or consistency (C).
Why it mattered
PACELC explains why a system like Yahoo's PNUTS gives up consistency without any partition, since waiting for distant replicas costs latency. It is now the usual way of classifying replicated databases, and Abadi elaborated it in an IEEE Computer article in 2012.
A free, open instruction set that anyone may implement without a licence.
The story
Before
Widely used instruction sets such as x86 and Arm were proprietary and licensed, so building a processor meant permission and fees. Earlier open designs, such as OpenRISC, had not become widely adopted, and academic projects lacked a modern free ISA to build on.
What happened, and who
Krste Asanović, Yunsup Lee and Andrew Waterman began RISC-V at the University of California, Berkeley's Par Lab in May 2010, initially as a short project to give their group a clean instruction set to use. It was designed as a small base set of instructions with optional standard extensions.
Why it mattered
Because the specification is open and now managed by RISC-V International, anyone can build processors, compilers and tools for it. It is used for embedded and accelerator cores and in research, and it is the instruction set that many computer-architecture courses now teach.
A 64-bit identifier made of a timestamp, a machine number and a counter, so many servers can create sortable unique IDs without coordinating.
The story
Before
Twitter stored tweets in MySQL with auto-incrementing IDs, which needs a single database to hand out numbers. As Twitter moved to Cassandra, which has no built-in unique ID generator, it needed IDs that many machines could create independently and that still sorted roughly by time.
What happened, and who
Twitter's engineers announced Snowflake in a post on the company's engineering blog on 1 June 2010. Each ID is 64 bits: a 41-bit millisecond timestamp, a 10-bit machine number and a 12-bit sequence number. It began serving production IDs in September 2010.
Why it mattered
Snowflake IDs need no central counter, sort by creation time and fit in a 64-bit integer. Discord, Instagram and Mastodon use variants of the layout, and it is the standard answer to “design a distributed unique ID generator”.
Adds capabilities to Unix, so a sandboxed process can use only the file descriptors it has been handed.
The story
Before
Unix processes run with ambient authority: a compromised process can open any file or connect to any address its user could. Sandboxing a component such as an image decoder therefore needed awkward tools such as chroot or complex access-control policies.
What happened, and who
Robert Watson, Jonathan Anderson, Ben Laurie and Kris Kennaway presented “Capsicum: Practical Capabilities for UNIX” at USENIX Security in August 2010. A process enters capability mode, after which it cannot open new global resources and can use only the descriptors it already holds, each with limited rights. It was integrated into FreeBSD.
Why it mattered
Capsicum showed how to retrofit capability-based sandboxing into an existing Unix without a new operating system. FreeBSD ships it with the Casper helper for services needing outside access, and its approach has been compared with Linux's seccomp filters.
A TCP variant that shrinks its window in proportion to how many packets were marked congested, not in half on any mark.
The story
Before
Standard TCP treats congestion as a yes or no signal and halves its sending window on any sign of it. In a data centre, where many short and long flows share shallow switch buffers, this either left queues long, delaying short flows, or lost throughput.
What happened, and who
Mohammad Alizadeh and colleagues at Microsoft Research and Stanford presented “Data Center TCP (DCTCP)” at ACM SIGCOMM in August 2010. Switches mark packets with ECN when their queue passes a small threshold, and the sender estimates the fraction of marked packets and cuts its window by that fraction.
Why it mattered
DCTCP keeps switch queues short while holding high throughput, which lowers latency for small flows in a data centre. It was added to Windows Server and Linux and standardised as RFC 8257, and it is the basis of later data-centre congestion-control work.
A query system that scans nested records column by column across thousands of machines, answering aggregations over trillion-row tables in seconds.
The story
Before
Google's data was mostly nested records, such as protocol buffers with repeated and optional fields. Analysing it meant running MapReduce jobs that took minutes to hours, or flattening the records into rows for a relational engine and losing their structure. Columnar storage already sped up scans of flat tables, but no one had a way to store nested fields as columns.
What happened, and who
Sergey Melnik and six colleagues at Google described Dremel in “Dremel: Interactive Analysis of Web-Scale Datasets”, presented at the 36th International Conference on Very Large Data Bases (VLDB) in September 2010 and published in PVLDB. The paper combines a tree of servers that fans a query out and merges partial answers with a columnar layout for nested data. Each value is stored with a repetition level and a definition level, which are enough to rebuild the original records or to skip straight to the fields a query needs.
Why it mattered
Dremel showed that interactive, ad hoc analysis over huge nested datasets was possible. Google's BigQuery grew out of it, and Apache Parquet adopted its repetition and definition levels as the way to store nested columns, so the encoding sits inside most modern data lakes.
A standby streams the primary's write-ahead log over a connection and can answer read-only queries while it replays it.
The story
Before
PostgreSQL replicas were fed by shipping whole write-ahead log (WAL) files, so a standby lagged the primary by up to a file's worth of changes. A standby also could not be queried while it was recovering, so it sat idle until a failover.
What happened, and who
The PostgreSQL Global Development Group released PostgreSQL 9.0 on 20 September 2010. It added streaming replication, in which a standby receives WAL records over a network connection as they are written, and hot standby, which lets the standby run read-only queries while it replays them.
Why it mattered
This is the built-in physical replication that most PostgreSQL deployments still use. It made read replicas and a warm failover target part of the database itself rather than a scripted add-on, and later releases built cascading replication, synchronous modes and replication slots on top of it.
Compile each query pipeline into one tight machine-code loop, so a tuple stays in CPU registers instead of passing through interpreted operators.
The story
Before
Most engines used the iterator model, where every operator calls a next() function on its child once per tuple. Those virtual calls, and the tuple data written back to memory between operators, cost far more than the arithmetic. Vectorised engines like MonetDB/X100 amortised the calls over batches of values but still moved results between operators.
What happened, and who
Thomas Neumann of the Technical University of Munich published “Efficiently Compiling Efficient Query Plans for Modern Hardware” in PVLDB volume 4, issue 9, in 2011, describing the query engine of the HyPer database. It cuts the plan into pipelines between operators that must hold state, and generates one loop per pipeline using the LLVM compiler framework. Data flows through the loop in registers, and the operator code is inlined into it.
Why it mattered
It made query compilation the main alternative to vectorisation. Umbra, the successor to HyPer, extended the design, and compiled or hybrid engines are now weighed against vectorised ones whenever a new analytical engine is built.
A distributed commit log built for high-volume event data, released free for anyone to run.
The story
Before
Kafka was an internal LinkedIn system, created by Jay Kreps, Neha Narkhede and Jun Rao to move activity and operational data between services. Traditional message queues of the time were built for reliable delivery of individual messages and handled sustained high throughput poorly.
What happened, and who
LinkedIn released Kafka as open source in January 2011. In July 2011 it was accepted into the Apache Incubator, the Apache Software Foundation's route for new projects.
Why it mattered
Opening the code let other companies adopt the append-only, partitioned log as the backbone between services, and it became the reference design for event streaming. Much of the modern streaming ecosystem, and the practice of treating a log as the source of truth, grew from this release.
The Internet's central pool of unallocated IPv4 addresses ran out, and every regional registry received its last block.
The story
Before
IPv4 addresses are 32 bits, so there are about 4.3 billion of them. The Internet Assigned Numbers Authority (IANA) handed them out in blocks of 16.7 million, called /8s, to the five regional Internet registries, which passed them on to providers. Classless addressing and NAT had stretched the supply for years, but the arithmetic had only one direction.
What happened, and who
On 31 January 2011 IANA allocated its last two unreserved /8 blocks to APNIC, the Asia-Pacific registry. On 3 February 2011, at a press conference and ceremony, it gave one of the five remaining /8 blocks to each regional registry, which emptied the central pool.
Why it mattered
From then on a registry could only allocate from what it already held, and each ran dry in turn. That moved IPv6 from something to plan for to something to deploy, and it is why carrier-grade NAT, address markets and dual-stack networks are everyday operations work.
A register interface and command set for SSDs on PCI Express, with up to tens of thousands of deep command queues.
The story
Before
SSDs were attached through SATA and the Advanced Host Controller Interface (AHCI), which was designed for spinning disks. AHCI offers one command queue holding at most 32 commands, and each command costs several slow register accesses. A drive that could serve many requests in parallel was fed through a single narrow door.
What happened, and who
The NVM Express work group, chaired by Amber Huffman of Intel, released version 1.0 of the specification on 1 March 2011. It defines paired submission and completion queues in host memory, up to 65,535 of them with up to 65,536 commands each, so that each CPU core can have its own queue and a device needs only a doorbell write to be told about new work.
Why it mattered
It exposed flash parallelism to software instead of hiding it behind a disk-era protocol. NVMe is now the standard interface for SSDs in servers and laptops, and the Linux multi-queue block layer and later kernel interfaces such as io_uring were shaped by what it made possible.
A mis-routed network change set off mass volume re-mirroring that overloaded a shared control plane; full recovery took about three days.
The story
Before
Amazon Elastic Block Store (EBS) volumes are each mirrored on a second node, and when a node loses its mirror it looks for free space to make a new one. That recovery logic ran only after failures, so it had never met a failure of this size. The control plane that created volumes and handled API calls was shared across the whole US East Region, so trouble in one Availability Zone could reach the others.
What happened, and who
At 12:47 AM PDT on 21 April 2011, during a network upgrade in one Availability Zone of us-east-1, traffic was routed onto a lower-capacity network instead of onto the backup router. EBS nodes lost contact with their mirrors, and when connectivity returned many of them searched for space at once and exhausted it. Backed-up create-volume requests starved the control plane's threads, so API calls failed across the Region. At the peak about 13% of the volumes in the zone were stuck, and most were recovered by 24 April; AWS published its account as a post-event summary.
Why it mattered
It became the textbook case of a recovery path that sits idle on good days and overwhelms the system on the worst one. AWS added capacity buffers, backed off retries more aggressively and isolated the control plane by Availability Zone, and the episode fed its later cell-based designs such as Physalia and the wider habit of designing for blast radius.
Serializable snapshot isolation made SERIALIZABLE mean serialisable, instead of a label for snapshot isolation.
The story
Before
Before 9.1, asking PostgreSQL for SERIALIZABLE gave a single MVCC snapshot for the whole transaction, which is snapshot isolation. That level permits anomalies such as write skew, where two transactions each read what the other writes and both commit, so a database could break an invariant while claiming the strongest isolation level.
What happened, and who
PostgreSQL 9.1 was released on 12 September 2011 with a new SERIALIZABLE level, implemented by Kevin Grittner and Dan Ports. It is serializable snapshot isolation (SSI): transactions still run on snapshots, but the system watches for the read-write dependencies that make a cycle and aborts one transaction when it finds one. The old behaviour stayed available as REPEATABLE READ.
Why it mattered
Applications can now rely on the database to enforce a cross-row invariant without explicit locks, at the price of retrying transactions that abort with a serialisation failure. It showed that serialisability need not mean two-phase locking's blocking, and it is the reason PostgreSQL's isolation levels are described the way they are today.
Replicated data types built so that any two replicas that have seen the same updates converge, with no coordination between them.
The story
Before
Replicas that accept writes independently have to reconcile them somehow. Operational transformation, the technique behind early collaborative editors, rewrote concurrent operations against each other, and its correctness proofs proved hard and error-prone. The alternative was to pick a winner with a timestamp and lose updates.
What happened, and who
Marc Shapiro, Nuno Preguiça, Carlos Baquero and Marek Zawirski gave the idea its name in “Conflict-free Replicated Data Types”, presented at the Symposium on Stabilization, Safety, and Security of Distributed Systems in Grenoble in October 2011. A companion INRIA research report from January 2011 worked out the underlying convergent and commutative designs in detail. The point of the definition is a condition on the data type: if merging is commutative, associative and idempotent, or updates commute, replicas converge without any coordination.
Why it mattered
CRDTs turned convergence from a property proved case by case into something a data type's design guarantees. They are the basis of the counters and sets in several databases, and of the text-editing libraries behind collaborative and local-first software.
Answer every query from a slow, exact batch layer plus a fast layer for the newest data, merged at read time.
The story
Before
Batch systems such as Hadoop computed exact answers but hours late. Real-time systems answered immediately from incrementally updated state, which was complex to keep correct and could drift after failures or bugs. Teams had to choose between fresh and right.
What happened, and who
On 13 October 2011 Nathan Marz, then at Twitter and the creator of the Storm stream processor, posted “How to beat the CAP theorem” on his blog. He proposed keeping immutable, append-only data in a batch layer that recomputes views from scratch, a real-time layer that covers only the most recent data, and a serving layer that merges the two. The post does not use the name; the pattern was later called the lambda architecture.
Why it mattered
It gave teams a vocabulary for combining batch and stream processing, and because recomputation from raw data repairs mistakes, a bug could be fixed by re-running the batch. Its cost, writing and maintaining the same logic twice, is what Jay Kreps' later critique and the log-based designs that followed set out to remove.
A standard way for a browser and a server to keep one connection open and send messages both ways.
The story
Before
HTTP is request and response: the server can only speak when spoken to. To push data to a browser, pages polled on a timer, or used long polling, where a request is held open until there is news. Both spent a full set of HTTP headers on every exchange and added latency.
What happened, and who
The Internet Engineering Task Force (IETF) published RFC 6455, “The WebSocket Protocol”, in December 2011, written by Ian Fette and Alexey Melnikov. A WebSocket starts as an ordinary HTTP request that asks to upgrade, then becomes a framed, two-way message channel over the same TCP connection.
Why it mattered
It made chat, live dashboards, collaborative editors and multiplayer games practical in a browser without polling. Servers had to change too: a connection now lives for hours, so capacity is counted in open sockets, and load balancers and deployments have to cope with long-lived connections.
Stack samples drawn as stacked boxes whose width is time, so the hot code paths are visible at a glance.
The story
Before
A sampling profiler records the call stack many times a second, which produces thousands of stack traces. Tools printed them as long text listings or as call trees, and finding the code that used the most CPU meant reading and summing by hand.
What happened, and who
Brendan Gregg, a performance engineer then at Joyent, published “Flame Graphs” on his blog on 16 December 2011. He had been profiling a MySQL performance problem and wanted to see many stack samples at once. Each box is a function, its width is the share of samples that include it, and its parent below is its caller.
Why it mattered
The flame graph became the standard way to read a CPU profile, and the same idea now covers memory allocation, off-CPU time and differences between two profiles. Most profilers, from Linux perf to browser developer tools, can emit one.
A few hundred kilobytes of crafted request parameters can keep a web server's CPU busy for minutes or hours.
The story
Before
Hash tables are fast on average because keys spread evenly. Scott Crosby and Dan Wallach had shown in 2003 that if the hash function is known and fixed, an attacker can choose keys that all collide and turn each lookup into a linear scan, but most language runtimes went on using fixed hash functions, so the warning had little effect.
What happened, and who
Alexander Klink and Julian Wälde presented “Effective Denial of Service Attacks Against Web Application Platforms” at the 28th Chaos Communication Congress on 28 December 2011. They crafted HTTP POST parameters whose names collide in the hash tables that web frameworks use to store them, and showed that one request could occupy a CPU for a long time on PHP, Java, Python, Ruby 1.8 and ASP.NET. The oCERT advisory the same day listed the affected platforms.
Why it mattered
Vendors patched within weeks. Language runtimes introduced randomised hash seeds, later adding stronger keyed hashes such as SipHash, and web servers began to cap the number of request parameters. Any structure keyed by attacker-chosen input now has to assume the attacker can pick colliding keys.
One FreeBSD server held 2,277,845 open TCP connections while more than 40% of its CPU sat idle.
The story
Before
Holding a connection open per user was assumed to be costly, and “the C10K problem”, serving ten thousand clients on one machine, had defined a hard target for years. WhatsApp had reached one million connections on a server some months earlier and treated that as its milestone.
What happened, and who
WhatsApp engineers posted “1 million is so 2011” on 6 January 2012, with a screen capture from one production chat server. It showed 2,277,845 open sockets on a machine with 24 cores and 103 GB of memory, running FreeBSD 8.2 and Erlang R14B03, with about 42% of CPU idle.
Why it mattered
It showed how far one machine goes when each connection is a few kilobytes of a lightweight process on an event-driven kernel interface, and it fixed the picture of a chat server as memory-bound and not thread-bound. Sizing a connection-heavy service still starts from a per-connection memory budget like this.
A fully managed key-value and document database where you buy throughput, not servers, with single-digit-millisecond latency at any size.
The story
Before
Amazon's 2007 Dynamo paper described a highly available store, but running it meant operating the cluster yourself and having applications reconcile conflicting versions of an item. Amazon SimpleDB was managed, but limited each domain in size and had unpredictable performance.
What happened, and who
Amazon Web Services launched DynamoDB on 18 January 2012, announced by its chief technology officer Werner Vogels. Customers state the read and write capacity they need, called provisioned throughput, and the service spreads and replicates the data over SSD-backed storage to deliver it, with average service-side latency in the single-digit milliseconds.
Why it mattered
It set the pattern for serverless, predictable-latency databases: capacity and latency are the contract, and replication and partitioning are the provider's choice. Despite sharing a name with Dynamo, it dropped the design's client-side version reconciliation, and it now underlies large parts of AWS and many customers' applications.
Trying random settings finds good hyperparameters faster than a grid, because only a few of the settings really matter.
The story
Before
Hyperparameters such as the learning rate or the layer size were commonly tuned by grid search, which tries every combination on a fixed lattice. A grid spends its budget on repeating the same few values of the important parameters while it sweeps the unimportant ones.
What happened, and who
James Bergstra and Yoshua Bengio published “Random Search for Hyper-Parameter Optimization” in volume 13 of the Journal of Machine Learning Research in February 2012. They showed empirically and in theory that random trials beat grid trials for neural networks when only a few hyperparameters affect a given dataset, since each random trial tries a fresh value of every parameter.
Why it mattered
Random search became the default baseline, and the argument about effective dimensionality carried into later tuning methods such as Bayesian optimisation and Hyperband. A team tuning a model without a specific reason to use a grid should start here.
Agree on the order of transactions first, then run them deterministically on every replica, so no distributed commit protocol is needed.
The story
Before
A transaction that touches several partitions normally ends with two-phase commit, and it holds its locks while the partitions exchange messages. Over a network, and especially between data centres, those locks are held for the length of several round trips, which limits throughput on contended data.
What happened, and who
Alexander Thomson, Thaddeus Diamond, Shu-Chun Weng, Kun Ren, Philip Shao and Daniel Abadi presented “Calvin: Fast Distributed Transactions for Partitioned Database Systems” at SIGMOD in Scottsdale, Arizona, in May 2012. A sequencing layer batches incoming transactions into a single agreed global order, and every replica then executes them in that order deterministically, so all replicas reach the same state without exchanging commit votes. The paper reported about half a million TPC-C transactions per second on a hundred commodity nodes.
Why it mattered
Calvin is the reference for a different answer from Spanner's: pay for ordering up front rather than for clocks or commit rounds. Its ideas live on in FaunaDB and in later deterministic and log-first database designs.
A web page that shows, next to your source code, the assembly a compiler generates for it, updating as you type.
The story
Before
To see what a compiler produced, you compiled with an option to keep the assembly, or disassembled the object file, and then searched the output for your function. It was slow enough that most programmers argued about what the compiler would do instead of looking.
What happened, and who
Matt Godbolt, then at the trading firm DRW, wrote the first version over a week or so of spare time after an argument with a colleague about range-based for loops, and released it as open source in May 2012 as GCC Explorer, with the first commit on 22 May. On 30 April 2014 it was renamed Compiler Explorer as it grew beyond GCC.
Why it mattered
Looking at the compiler's output became a thirty-second habit. It is used in code reviews, bug reports and talks to show what a construct costs, and it supports dozens of languages and compilers.
Give each processor its own run queue and let idle ones steal work, instead of every thread sharing one locked queue.
The story
Before
Go's original scheduler kept all runnable goroutines behind one global lock and central state. Worker threads often handed goroutines off to one another, adding latency, and with more cores the lock became the bottleneck, so Go programs stopped speeding up as processors were added.
What happened, and who
Dmitry Vyukov, a Google engineer, posted the “Scalable Go Scheduler Design Doc” to the golang-dev mailing list on 2 May 2012. It adds a new resource, the P (processor), that a thread must hold to run Go code, with a local run queue per P and stealing from other Ps' queues when its own is empty. The design shipped in Go 1.1 in May 2013.
Why it mattered
This is the G, M and P model that Go still uses. It is why a program can create hundreds of thousands of goroutines and have them spread across cores, and why understanding Go performance means understanding local queues, stealing and blocking system calls.
The same loop over the same numbers runs several times faster once the data is sorted, though sorting does none of the work.
The story
Before
Most programmers reasoned about speed by counting operations, and an if inside a loop looked like one comparison whatever the data. The cost of a mispredicted branch, when the processor has guessed the wrong way and must throw away work already begun, was invisible in the source code.
What happened, and who
In June 2012 a Stack Overflow user asked why a C++ loop that summed the elements of an array above a threshold ran much faster when the array was sorted. The top answer, from a user called Mysticial, explained branch prediction: on sorted data the test is false for a long run and then true for a long run, so the processor predicts it almost every time, while on random data it guesses wrong about half the time.
Why it mattered
It became one of the most widely read questions on the site and the usual first demonstration that data can change speed through the branch predictor. It is the standard example for why data-dependent branches in hot loops are worth removing or making predictable.
Randomly switching off half the hidden units during training stops units relying on each other and acts like averaging many networks.
The story
Before
Large neural networks trained on small datasets overfit: each hidden unit could rely on the others to fix its mistakes, so features co-adapted to the training set. The main defences were weight decay, which penalises large weights, and stopping training early.
What happened, and who
Geoffrey Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever and Ruslan Salakhutdinov posted “Improving neural networks by preventing co-adaptation of feature detectors” to arXiv on 3 July 2012. In each training step every hidden unit is omitted with some probability, so no unit can count on a particular partner. The longer JMLR paper of 2014 gave the technique its usual name, dropout.
Why it mattered
Dropout is a single line in most frameworks and became the standard regulariser for large networks in the early 2010s, used in AlexNet's fully connected layers. It also gave a cheap picture of model averaging that shaped later ideas such as ensembles and stochastic depth.
Compressing a secret together with attacker-chosen text leaks the secret through the size of the encrypted result.
The story
Before
TLS could compress data before encrypting it, and SPDY, the protocol Google had designed that HTTP/2 grew from, always compressed request headers with zlib to save bandwidth. Encryption hides content but not length, and nobody had shown that a shrinking length could give away a cookie.
What happened, and who
Security researchers Juliano Rizzo and Thai Duong demonstrated CRIME at the ekoparty security conference in September 2012, building on a hypothesis by Adam Langley of Google. They made a victim's browser send requests containing guesses next to the secret session cookie; when the guess matched, the compressed request came out shorter, and watching the encrypted length let them recover the cookie a byte at a time. The flaw was catalogued as CVE-2012-4929.
Why it mattered
Browsers and servers turned off TLS compression, and it is now almost never used. It also pushed the designers of HTTP/2 to replace zlib with HPACK, which avoids compressing with a general-purpose context, and the design carried into HTTP/3's QPACK. Later attacks such as BREACH used the same length side channel.
A query can be answered from an index alone when the visibility map says every row on a heap page is visible to everyone.
The story
Before
PostgreSQL's multiversion storage keeps row visibility in the heap, not in the index. Even when an index held every column a query needed, each matching entry needed a visit to the heap page to check that the row was visible, so a covering index saved little I/O.
What happened, and who
PostgreSQL 9.2 was released on 10 September 2012 with index-only scans. A per-table visibility map records which heap pages contain only rows visible to all transactions; for those pages the executor returns column values straight from the index entry and skips the heap fetch.
Why it mattered
Covering indexes now pay off for read-heavy queries, sometimes by large factors, provided VACUUM keeps the visibility map current. That dependence on vacuuming is why a table's maintenance state can change a query plan.
A cluster manager crashed while automatic failover was disabled, two halves each elected a leader, and an out-of-date node briefly became primary.
The story
Before
GitHub's MySQL databases used Pacemaker, a cluster manager, with the Percona Replication Manager to promote a standby automatically when the primary failed its health checks. The design assumed that a promoted node would be healthy and up to date, and that the manager itself would not fail during an operation.
What happened, and who
On 10 September 2012 a schema migration raised load, the health checks failed and the primary was failed over to a standby with a cold buffer pool, which then struggled and failed over again; operators put the cluster in maintenance mode, which left the standby's replication out of step. On the morning of 11 September, while they were restoring normal operation, a Pacemaker crash split the cluster, and at 8:19 AM a node known to be out of date was elected primary. Operators powered it off at 8:26 AM, and GitHub published its account on 14 September.
Why it mattered
Sixteen private repositories were visible to the wrong users for those seven minutes, and data in MySQL and Redis disagreed afterwards. GitHub turned off automatic failover of its main database so that a person now starts one. The incident is the usual illustration of why a promotion needs a check that the new primary is current, and fencing of the old one.
A deep convolutional network trained on two gaming GPUs wins the ImageNet challenge by more than ten points.
The story
Before
From about 1990 to 2010, neural networks did no better than other methods such as support vector machines. Image recognition was built from hand-engineered features, like SIFT and HOG, fed to shallow classifiers, and error on the ImageNet benchmark fell slowly.
What happened, and who
Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton of the University of Toronto, as team SuperVision, submitted AlexNet to the ImageNet Large Scale Visual Recognition Challenge on 30 September 2012. Trained for five to six days on two Nvidia GTX 580 GPUs, it reached 15.3% top-5 error; the best other team scored 26.2%.
Why it mattered
The margin made the field's mind up: a deep network, given enough data and GPU compute, beat hand-designed features. Within a few years nearly all computer-vision work — and then speech and language — moved to deep learning trained on GPUs.
A database replicated across the globe that gives externally consistent transactions by making clock uncertainty explicit.
The story
Before
Global systems had to choose. Bigtable and Dynamo-style stores scaled across data centres but offered weak or single-row guarantees, while sharded SQL databases could not run transactions across shards without heavy coordination. Agreeing on the order of events across continents was assumed to need constant messaging.
What happened, and who
James Corbett, Jeffrey Dean and 23 other Google authors presented “Spanner: Google's Globally-Distributed Database” at the OSDI conference in Hollywood, California, in October 2012, where it won a best paper award. Spanner replicates data with Paxos and runs cross-shard transactions with two-phase commit, and it uses an API called TrueTime, built from GPS receivers and atomic clocks, that returns an interval bounding the clock's error. A transaction waits out that uncertainty before it commits, so its timestamp order matches real-time order.
Why it mattered
It showed that bounded clock uncertainty could replace much coordination, and that a database could offer SQL, transactions and global replication together. It started the NewSQL line, including CockroachDB and YugabyteDB, and cloud offerings such as Cloud Spanner and Aurora DSQL follow its central idea.
Two cheap training tricks give every word a vector in which directions carry meaning: king minus man plus woman lands near queen.
The story
Before
Word representations came from counting co-occurrences and compressing the matrix with singular value decomposition, or from neural language models such as Bengio's 2003 model, which were too slow to train on large text. Words were otherwise atomic symbols with no notion of similarity.
What happened, and who
Tomas Mikolov, Kai Chen, Greg Corrado and Jeffrey Dean of Google posted “Efficient Estimation of Word Representations in Vector Space” to arXiv on 16 January 2013. It introduced two shallow models: continuous bag-of-words, which predicts a word from its context, and skip-gram, which predicts the context from a word. Training them on billions of words is cheap enough that the learned vectors also capture relations, such as the king and queen arithmetic.
Why it mattered
Pretrained word vectors became a standard input to almost every language model for the next several years, and the paper popularised learning representations from raw text. Its idea of predicting context is the ancestor of the objectives used to pretrain today's language models.
When one request fans out to many servers, rare slow responses become the common case, and hedging requests can hide them.
The story
Before
Services were judged by their average or median response time. But if a request must wait for the slowest of a hundred servers, and each is slow one time in a hundred, most requests wait on a slow one. The average said nothing about what users actually saw.
What happened, and who
Jeffrey Dean and Luiz André Barroso of Google published “The Tail at Scale” in the February 2013 issue of Communications of the ACM. It explains why latency variability is amplified by fan-out, and describes techniques that tolerate it instead of removing every cause, including hedged requests sent to a second replica after a short delay, and tied requests that cancel each other.
Why it mattered
It made the 99th-percentile latency a number that teams design against and put on dashboards. Hedging, request cancellation and per-request deadlines, common in RPC frameworks and load balancers, come straight from this paper.
An application compiled together with only the operating-system code it uses into a sealed virtual machine image: a DNS server of 183.5 kB.
The story
Before
A cloud virtual machine ran a full general-purpose operating system, with drivers, a file system and a shell, to host one service. That meant hundreds of megabytes to store and boot, and a large attack surface of code the service never needed.
What happened, and who
Anil Madhavapeddy, Richard Mortier, Jon Crowcroft and colleagues at the University of Cambridge presented “Unikernels: Library Operating Systems for the Cloud” at ASPLOS in Houston in March 2013. Their MirageOS builds an application and just the library operating-system components it needs into a single-purpose image that runs directly on the Xen hypervisor. Their DNS server image was 183.5 kB, against 462 MB for BIND on a Linux virtual machine, and it answered queries faster.
Why it mattered
It revived the library-operating-system idea of the 1990s, now with a hypervisor to supply hardware compatibility. The unikernel approach shaped MirageOS and later minimal virtual machines and microVMs, and it remains the reference for how small and specialised a cloud workload's operating system can be.
A five-minute lightning talk showed an app packaged into a container image that runs the same anywhere.
The story
Before
Linux containers, built from namespaces and control groups, had existed for years through LXC, but using them meant assembling the pieces by hand. Applications were still shipped as installers, scripts or whole virtual machine images, which were heavy and behaved differently from machine to machine.
What happened, and who
On 15 March 2013 Solomon Hykes of the platform company dotCloud gave the first public demonstration of Docker, in a five-minute lightning talk titled “The Future of Linux Containers” at PyCon US in Santa Clara. He showed how to build and run an isolated container from a single command line. dotCloud open-sourced the project and later renamed itself Docker, Inc.
Why it mattered
Docker made the container image a standard unit of packaging and deployment: build once, run the same thing on a laptop, in CI and in production. That set the stage for container registries, orchestrators such as Kubernetes, and the way most services are now shipped.
Let several schedulers place jobs on one shared view of the cluster at once, and settle collisions the way an optimistic database does.
The story
Before
Google's Borg used one monolithic scheduler for every job, which made each new scheduling policy a change to one increasingly complex program and made quick jobs wait behind slow placements. Two-level designs such as Mesos let frameworks choose from resources offered to them, but gave each framework only a partial view of the cluster.
What happened, and who
Malte Schwarzkopf, Andy Konwinski, Michael Abd-El-Malek and John Wilkes presented “Omega: flexible, scalable schedulers for large compute clusters” at EuroSys in April 2013. Each scheduler in Omega sees the whole cell state, decides freely, and commits its placement as a transaction; if another scheduler has claimed the same resources in the meantime, the commit fails and is retried. Their study used the workload of three Google clusters.
Why it mattered
Omega's central idea, a shared store of state that independent components read and change under optimistic concurrency, is the shape of Kubernetes: the API server holds the cluster state and separate controllers and schedulers act on it. It also moved the question of scheduler design from raw speed to how often schedulers conflict.
A B-tree that never updates a page in place: changes are prepended as deltas and swung in with one atomic compare-and-swap.
The story
Before
A conventional B-tree updates pages in place under latches, short locks on a page. On many-core machines, hot pages make threads queue on those latches, and in-place writes are also a poor fit for flash storage, which prefers appending to overwriting.
What happened, and who
David Lomet, Justin Levandoski and Sudipta Sengupta of Microsoft Research presented “The Bw-Tree: A B-tree for New Hardware Platforms” at the IEEE International Conference on Data Engineering in April 2013. A mapping table turns page identifiers into memory addresses, so a page can be replaced by changing one table entry. An update prepends a delta record to the page's chain and installs it with a single compare-and-swap, and the design writes to flash in a log-structured way.
Why it mattered
Microsoft used the Bw-tree as the ordered index of SQL Server's in-memory Hekaton engine and in Azure Cosmos DB. It is also a reference point for a caution: an independent 2018 evaluation by Wang and colleagues found a carefully engineered latch-based B+tree faster on the workloads they measured, so latch-free describes a progress guarantee, not speed.
Maximum flow in a network of n nodes and m edges can be computed in O(nm) time, whatever the density of the graph.
The story
Before
The maximum-flow problem, sending as much as possible from a source to a sink through capacitated edges, has been studied since Ford and Fulkerson in the 1950s. King, Rao and Tarjan's 1994 algorithm reached O(nm) time only for graphs dense enough, with m at least about n^(1+ε); for sparser graphs the best known bounds stayed above it.
What happened, and who
James Orlin of MIT presented “Max flows in O(nm) time, or better” at the ACM Symposium on Theory of Computing (STOC) in Palo Alto in June 2013. He applies an algorithm of Goldberg and Rao to a compressed version of the network, kept up to date with dynamic transitive-closure data structures, for a running time of O(nm + m^(31/16) log² n). Together with the earlier bound this gives O(nm) for all values of n and m.
Why it mattered
It settled the question of how fast max flow could be solved in terms of n and m alone, a landmark of algorithms theory. It is not the algorithm practitioners use, which is usually Dinic's or push-relabel; its value is in what it shows the problem's complexity to be.
Give each CPU its own staging queue for disk requests, mapped onto the device's own queues, so cores stop fighting over one lock.
The story
Before
The Linux block layer put every I/O request through a single queue guarded by one lock. That was fine for hard disks, which manage a few hundred operations a second, but with fast SSDs the lock became the limit: the paper measured a ceiling of around 800,000 I/Os per second however many cores submitted, and multi-socket machines did worse because the lock's cache line bounced between sockets.
What happened, and who
Matias Bjørling, Jens Axboe (the Linux block-layer maintainer), David Nellans and Philippe Bonnet presented “Linux Block IO: Introducing Multi-queue SSD Access on Multi-core Systems” at the SYSTOR conference in Haifa in June 2013. Their design, blk-mq, has per-CPU software queues that map onto hardware dispatch queues. It was merged for Linux 3.13, released on 19 January 2014, with only the virtio block driver converted at first.
Why it mattered
It let Linux drive NVMe devices at millions of I/Os per second, and it matches the multi-queue model NVMe defines. Drivers were converted over the following releases, and the old single-queue path was eventually removed.
For one second, 143,199 tweets arrived, about 25 times the steady rate, because a film reached a famous line.
The story
Before
Twitter's traffic was steady enough to plan for an average, with occasional spikes around events. Its earlier infrastructure had shown strain under bursts, and capacity had to be sized for the peak second, not the average one.
What happened, and who
At 11:21:50 JST on 3 August 2013 (7:21:50 PDT on 2 August), viewers in Japan watching a broadcast of Studio Ghibli's Castle in the Sky tweeted the word “balse”, a spell spoken in the film, at the same instant, as fans do each time it airs. Twitter reported a peak of 143,199 tweets in one second, roughly 25 times its average of about 5,700 a second, and said users noticed nothing.
Why it mattered
It is the standard example for capacity planning: load is set by its peak second, and a synchronised audience can produce that peak with no warning. Twitter's engineers used it to show what its rebuilt infrastructure could absorb.
A load generator that waits for each slow reply skips the requests it should have sent, so measured tail latency looks far better than real.
The story
Before
Benchmark tools typically ran a fixed number of threads, each sending a request, waiting for the reply and sending the next. When the system stalled, those threads simply stopped sending, so the requests that would have queued during the stall were never issued or timed. The worst delays were left out of the results, and reported percentiles stayed low.
What happened, and who
Gil Tene, chief technology officer of the Java runtime company Azul Systems, described the flaw as “coordinated omission” in a post to the Mechanical Sympathy mailing list on 4 August 2013, pointing to the same argument in his talk “How NOT to Measure Latency”. He recommended recording latency against the time each request should have been sent, and built the correction into HdrHistogram and the wrk2 load generator.
Why it mattered
The name gave engineers a way to challenge a latency number: did the tool keep sending while the system stalled? Load generators such as wrk2 and fixes to benchmarks like YCSB followed, and open-loop testing at a fixed arrival rate is now the accepted way to measure tail latency.
One neural network, given only screen pixels and the score, learns to play several Atari games, some better than a human expert.
The story
Before
Reinforcement learning worked from hand-designed features of the state, and combining Q-learning with a neural network was known to be unstable: successive training samples are strongly correlated and the targets move as the network changes, so learning often diverged.
What happened, and who
Volodymyr Mnih and colleagues at DeepMind, then a London start-up, posted “Playing Atari with Deep Reinforcement Learning” to arXiv on 19 December 2013, for the NIPS Deep Learning Workshop. A convolutional network reads raw frames and estimates the value of each action, trained by Q-learning with experience replay, which samples random past transitions to break their correlation. It was applied to seven Atari 2600 games with no game-specific tuning, beat all earlier methods on six and passed a human expert on three.
Why it mattered
It showed that deep learning could turn raw perception into control, and started the field of deep reinforcement learning. Google bought DeepMind in 2014, and the same recipe led to the agents that followed, from AlphaGo to systems tuned with reinforcement learning today.
Passing gradients through random sampling lets an encoder and decoder train together, giving a generative model with a smooth latent space.
The story
Before
Variational inference approximates an intractable posterior over hidden variables, but the standard methods either optimised an approximation separately for every data point or used gradient estimates of very high variance. That made it too slow to train on large datasets.
What happened, and who
Diederik Kingma and Max Welling of the University of Amsterdam posted “Auto-Encoding Variational Bayes” to arXiv on 20 December 2013; it appeared at ICLR 2014. They train an encoder network that outputs a distribution over the latent variables and a decoder that reconstructs the data, by maximising the evidence lower bound (ELBO). The reparameterisation trick writes the sample as a deterministic function of the parameters plus independent noise, so ordinary gradient descent applies.
Why it mattered
The paper began deep latent-variable generative modelling. Its reparameterisation trick and amortised encoder run through later work, including the image autoencoders in latent diffusion models.
A missing length check in a TLS keep-alive feature let anyone read chunks of a server's memory, private keys included.
The story
Before
Most of the web's HTTPS traffic was secured by OpenSSL, an open-source TLS library maintained by a handful of volunteers. In 2012 it added the TLS heartbeat extension, a keep-alive in which one side sends a small payload and the other echoes it back. The code trusted the length field in the request and never checked it against the payload actually sent.
What happened, and who
Neel Mehta of Google's security team and, independently, the Finnish firm Codenomicon found that a request with a tiny payload and a huge claimed length made the server return up to 64 KB of adjacent memory. The bug, CVE-2014-0160, affected OpenSSL 1.0.1 to 1.0.1f. It was reported to OpenSSL in early April and made public on 7 April 2014, the day the fixed release 1.0.1g shipped.
Why it mattered
Memory that could hold session cookies, passwords and the server's private keys could be read without leaving a trace, so affected sites had to patch, revoke and reissue certificates and rotate credentials. It became the standard example of a buffer over-read in a security-critical parser, and it pushed funding for OpenSSL and renewed interest in memory-safe languages for protocol code.
A consensus algorithm designed first for being understood, then proved as safe as Paxos.
The story
Before
Paxos, described by Leslie Lamport in "The Part-Time Parliament" (published 1998), had been the standard way to make a group of machines agree on one sequence of commands. It was famously hard to grasp, and real systems that used it needed large, subtle extensions to reach a usable replicated log, so implementations often diverged from the proof.
What happened, and who
Diego Ongaro and John Ousterhout of Stanford presented "In Search of an Understandable Consensus Algorithm" at the USENIX Annual Technical Conference in Philadelphia in June 2014, where it won best paper. It splits consensus into leader election, log replication and safety, and a study of 43 students found Raft easier to answer questions about than Paxos.
Why it mattered
Engineers could finally implement consensus from a paper they understood. Raft is the replication layer of etcd, which holds Kubernetes' cluster state, and of Consul, and many distributed databases use it for the same job.
Reading the same DRAM row over and over can flip bits in the rows next to it, breaking memory isolation from ordinary software.
The story
Before
Operating systems and hypervisors assumed that memory returns what was written, and that a program can only change memory it has been given access to. As DRAM cells shrank and were packed closer, the electrical coupling between neighbouring rows grew, and manufacturers' testing had not examined this failure.
What happened, and who
Yoongu Kim, Ross Daly, Jeremie Kim, Chris Fallin, Ji Hye Lee, Donghyuk Lee, Chris Wilkerson, Konrad Lai and Onur Mutlu, from Carnegie Mellon University and Intel, presented “Flipping Bits in Memory Without Accessing Them: An Experimental Study of DRAM Disturbance Errors” at ISCA in Minneapolis in June 2014. They tested commercial DRAM modules and found that more than 80% of them could be made to flip bits by repeatedly activating a row, with a simple user-level program on real systems.
Why it mattered
It turned a reliability flaw into a security problem: later work, first by Google's Project Zero, used it to escape sandboxes and gain kernel privileges without any software bug. Mitigations such as target row refresh and error-correcting memory have followed, and none has fully closed the hole, so it remains a standing design constraint for memory hardware and the systems above it.
A lighter gated recurrent cell, and a two-network design that compresses a phrase into a vector and decodes it.
The story
Before
Long short-term memory cells kept information over long spans with a separate memory cell and three gates. Machine translation was still dominated by phrase-based statistical systems, which scored phrase pairs with hand-engineered features and count tables.
What happened, and who
Kyunghyun Cho, Yoshua Bengio and colleagues at the University of Montréal, with collaborators at Le Mans and Jacobs universities, posted "Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation" to arXiv on 3 June 2014; it appeared at EMNLP that year. One recurrent network reads a phrase into a fixed-length vector and a second generates the target phrase from it; the paper adds a simpler cell with two gates, now called the GRU. Here the pair re-scored phrases inside a statistical translation system.
Why it mattered
The GRU became a standard, cheaper alternative to the LSTM. The encoder-decoder split, translation as reading into a vector and writing out of it, was the frame that attention was soon added to and that sequence-to-sequence models built on.
Two networks trained against each other, one forging data and one spotting forgeries, learn to generate realistic samples.
The story
Before
Deep generative models had to model the probability of the data directly, which meant intractable likelihoods, approximations, or Markov chain sampling that was slow and mixed badly in high dimensions. Samples were blurry or limited to small problems.
What happened, and who
Ian Goodfellow, then at the Université de Montréal, and colleagues including Yoshua Bengio posted "Generative Adversarial Nets" to arXiv on 10 June 2014, and it appeared at NIPS that year. A generator turns noise into samples and a discriminator tries to tell them from real data; each improves against the other, a minimax game whose optimum has the generator reproducing the data distribution.
Why it mattered
Training needs only gradients through the two networks, with no likelihood. GANs produced the first sharp, photo-like synthetic images and dominated image generation for several years, until diffusion models overtook them. Their unstable training also became a case study in how optimisation of a game differs from minimising a loss.
An open-source container orchestrator descended from Google's Borg, which became the default way to run containers.
The story
Before
Google ran everything on Borg, an internal cluster manager, but nobody outside could use it. Docker, released in 2013, made containers easy to build, yet running many of them across many machines meant writing your own scheduling, restarts and service discovery, or adopting a tool that was not widely used.
What happened, and who
Google engineers Craig McLuckie, Joe Beda and Brendan Burns, among others, started the project in the open; the first commit landed on 6 June 2014. Eric Brewer, Google's vice-president of infrastructure, announced Kubernetes in his keynote at DockerCon on 10 June 2014. Version 1.0 followed on 21 July 2015, when Google donated it to the newly formed Cloud Native Computing Foundation.
Why it mattered
Users describe the state they want and controllers keep reconciling the cluster towards it, an idea taken from Borg. Kubernetes gave every cloud vendor a common target, so deployments became portable across clouds. It is now the base layer that much production infrastructure is built on.
Keep events in a log and reprocess history with one stream job, instead of maintaining separate batch and real-time code.
The story
Before
Nathan Marz's Lambda architecture answered the need for both fresh and accurate results with two systems: a batch layer that recomputed everything and a speed layer that produced approximate results quickly. The two had to compute the same answer, so the same logic was written twice, in two complex distributed systems.
What happened, and who
Jay Kreps, who had helped build Kafka at LinkedIn, published "Questioning the Lambda Architecture" on O'Reilly Radar on 2 July 2014. He argued that a log with long retention lets a stream processor replay history through an improved job and switch over to its output, and half-jokingly suggested calling this the Kappa architecture.
Why it mattered
The essay named the cost of duplicated pipelines and the alternative. It fed the case for stream processors that handle both live and historical data, and the retained, replayable log became the core of many event-driven data platforms.
A decoder that looks back over every encoder state at each step, instead of relying on one summary vector.
The story
Before
Encoder-decoder translators squeezed a whole source sentence into one fixed-length vector before writing a word of the translation. That vector had to hold everything, and translation quality fell off as sentences got longer.
What happened, and who
Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio, at the University of Montréal and Jacobs University Bremen, posted "Neural Machine Translation by Jointly Learning to Align and Translate" to arXiv on 1 September 2014; it appeared at ICLR 2015. The encoder keeps a state for every source word, and at each output step the decoder scores its own state against all of them, then takes a weighted average with the resulting softmax weights.
Why it mattered
Alignment was learned as part of translation, and long sentences stopped degrading. This score-and-average step is the attention mechanism, which the Transformer of 2017 kept and made the centre of the architecture, so every current large language model descends from it.
One deep LSTM reads a sentence into a vector and another writes the translation, with no hand-built pipeline.
The story
Before
The best machine translation systems were phrase-based statistical ones, assembled from separately trained alignment, phrase-table and language-model components. Neural networks had been used only to re-score their output, since standard networks needed inputs and outputs of fixed size.
What happened, and who
Ilya Sutskever, Oriol Vinyals and Quoc Le at Google posted "Sequence to Sequence Learning with Neural Networks" to arXiv on 10 September 2014, and it appeared at NIPS that year. A four-layer LSTM encoded the English sentence and a second one decoded the French. It scored 34.8 BLEU on the WMT'14 English-to-French task against 33.3 for a phrase-based baseline, helped by reading the source sentence in reverse.
Why it mattered
It showed that translation could be one trained network, and framed many tasks as turning one sequence into another. That framing, extended by attention and then the Transformer, is the template behind later language models.
Databases and version-control tools quietly rely on file-system ordering behaviour that no specification promises.
The story
Before
Applications that promise durability, such as databases and version control, write files in a careful order and call fsync at chosen points. Programmers assumed that this recipe survived a crash on any file system, although POSIX says almost nothing about what order writes reach disk.
What happened, and who
Thanumalayan Sankaranarayana Pillai, Vijay Chidambaram, Ramnatthan Alagappan, Samer Al-Kiswany and Andrea and Remzi Arpaci-Dusseau of the University of Wisconsin-Madison presented "All File Systems Are Not Created Equal: On the Complexity of Crafting Crash-Consistent Applications" at OSDI in Broomfield, Colorado, in October 2014. Their tool BOB tested six Linux file systems and ALICE checked eleven applications; ALICE found 60 crash vulnerabilities, many with severe consequences.
Why it mattered
It showed that data loss after a crash could come from an application depending on ordering that one file system happens to give and another does not. Storage engines now reason about crash states explicitly, and it explains why the same database can corrupt on one file system and not another.
Upload a function, and the cloud runs it when an event arrives, with no server for the customer to manage.
The story
Before
Running a small piece of event-driven code, such as resizing an image when it landed in storage, meant renting a virtual machine that sat mostly idle and was billed for the hours it existed. The customer also had to patch it, scale it and keep it alive.
What happened, and who
Amazon Web Services announced Lambda in preview at its re:Invent conference on 13 November 2014. The first version ran Node.js functions triggered by events from S3, Kinesis and DynamoDB, and charged only for the time the code ran. It became generally available in April 2015.
Why it mattered
It launched function-as-a-service: the provider owns the machine, and the customer pays per invocation. Running many customers' code on shared hosts with fast start-up demanded lightweight isolation, which led AWS to build the Firecracker microVM manager that now runs Lambda and Fargate.
Rust dropped its built-in lightweight-thread runtime and tied its standard library to native operating-system threads.
The story
Before
Early Rust shipped green threads, lightweight tasks scheduled by the language runtime, alongside native threads. The standard library had to give both the same I/O API, which made every abstraction a trait object, added overhead and binary size, and complicated calling C code and embedding Rust in other programs.
What happened, and who
Aaron Turon of the Rust core team opened RFC 230, "Remove the runtime", on 16 September 2014. The change that carried it out, "Finish runtime removal" (pull request 18967), was merged on 21 November 2014. It deleted the runtime trait and the libgreen and libnative libraries and bound scheduling and I/O directly to native threads.
Why it mattered
It removed the cost that made Rust unsuitable for embedding and for zero-overhead code. Without a runtime in the language, asynchronous programming moved into libraries built on futures and async/await, chosen by the program, which is how Rust's async model works today.
A membership filter storing fingerprints in a cuckoo hash table supports deletion and, at low error rates, beats Bloom filters on space.
The story
Before
The Bloom filter, invented by Burton Bloom in 1970, answers "is this key possibly in the set?" in a few bits per key. It cannot delete an entry without corrupting others, and at a false-positive rate ε it needs about 1.44 log2(1/ε) bits per key. Counting variants allow deletion at several times the space.
What happened, and who
Bin Fan, David Andersen, Michael Kaminsky and Michael Mitzenmacher presented "Cuckoo Filter: Practically Better Than Bloom" at CoNEXT 2014 in Sydney, held 2–5 December. They adapted cuckoo hashing, which resolves collisions by evicting and relocating entries, so that it moves fingerprints between two candidate buckets computed from the fingerprint itself.
Why it mattered
Deletion and lookups that touch at most two buckets made approximate membership practical where the set changes, such as caches, packet processing and storage indexes. The paper reports better space than a Bloom filter when the target false-positive rate is below about 3%.
A music video's view count reached 2,147,483,647, the largest value a signed 32-bit integer can hold.
The story
Before
YouTube stored each video's view count as a signed 32-bit integer, whose largest value is 231 − 1, or 2,147,483,647. When the site was built, no video looked likely to be watched that many times.
What happened, and who
PSY's "Gangnam Style", posted in July 2012, closed in on that limit in early December 2014. On 3 December, news outlets reported that YouTube had confirmed the count exceeded what its counter could hold and that it had moved view counts to a signed 64-bit integer.
Why it mattered
The largest signed 64-bit value is about 9.2 × 1018, so the counter is no longer the constraint. The episode is a well-known concrete case of integer overflow: nothing was wrong until a value crossed the width of its type.
PostgreSQL gains a binary JSON type that can be indexed for containment queries.
The story
Before
PostgreSQL 9.2 added a json type, but it stored the text exactly as given, whitespace and duplicate keys included, and re-parsed it on each access. It could not be indexed to answer "which rows contain this key and value?", so document-style queries scanned tables.
What happened, and who
PostgreSQL 9.4 was released on 18 December 2014 by the PostgreSQL Global Development Group. Its jsonb type stores a parsed binary form, and it supports GIN indexes and containment and existence operators such as @>.
Why it mattered
One database could hold relational tables and flexible documents in the same transaction, with indexes on both. That made PostgreSQL a credible replacement for a separate document store in many systems, and jsonb is now the usual choice for semi-structured columns.
Each parameter gets its own step size, computed from running averages of the gradient and its square.
The story
Before
Stochastic gradient descent with momentum needed a learning rate tuned by hand for each problem. AdaGrad and RMSProp adapted the step per parameter, but RMSProp's averages started at zero and were biased towards it early in training, and it had no momentum term of its own.
What happened, and who
Diederik Kingma of the University of Amsterdam and Jimmy Ba of the University of Toronto posted "Adam: A Method for Stochastic Optimization" to arXiv on 22 December 2014; it appeared at ICLR 2015. Adam keeps exponential moving averages of the gradient and of its square, corrects the bias from starting at zero, and divides one by the square root of the other.
Why it mattered
It worked with little tuning across very different models, and it became the default optimiser for deep learning. Its variant with decoupled weight decay, AdamW, is what most transformer training runs use.
A blog post shows how splitting functions into synchronous and asynchronous kinds divides a whole language in two.
The story
Before
JavaScript programmers wrote asynchronous code with callbacks and then promises, while async/await, already in C#, was on its way to JavaScript. A function that waited on I/O had to be written differently from one that did not, and the difference spread to everything that called it.
What happened, and who
Bob Nystrom, an engineer on Google's Dart team, published "What Color is Your Function?" on his blog on 1 February 2015. He imagines a language where every function is red or blue, red ones can only be called from red ones, and shows that asynchronous functions behave exactly like the red ones.
Why it mattered
"Function colouring" became the shared name for the cost of the split. It frames the design choice between async/await and runtimes that hide it, such as goroutines in Go and virtual threads in Java, and engineers cite it when they argue about which to adopt.
Normalising each layer's activations over the mini-batch lets networks train with much higher learning rates.
The story
Before
Deep networks needed careful weight initialisation and small learning rates, because a layer's inputs shifted as the layers before it changed during training. Saturating activations made that worse and slowed training.
What happened, and who
Sergey Ioffe and Christian Szegedy of Google posted "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift" to arXiv on 11 February 2015 (ICML 2015). Each layer subtracts the mini-batch mean and divides by its standard deviation, then applies a learned scale and shift. On ImageNet, the same accuracy took far fewer training steps.
Why it mattered
Very deep convolutional networks became easy to train, and batch normalisation was standard in vision models for years. Later work by Santurkar and colleagues in 2018 found the explanation in the title, internal covariate shift, does not hold up, and that a smoother loss surface is the better account. Transformers replaced it with layer normalisation, which does not depend on batch size.
Limiting how far each update moves the policy, measured by KL divergence, gives a guarantee that performance never gets worse.
The story
Before
Policy-gradient methods took steps sized by a learning rate with no link to how much the policy's behaviour changed. One large step could ruin a policy, and the data it then collected came from the ruined policy.
What happened, and who
John Schulman, Sergey Levine, Philipp Moritz, Michael Jordan and Pieter Abbeel, at UC Berkeley, posted "Trust Region Policy Optimization" to arXiv on 19 February 2015 (ICML 2015). It bounds the new policy's return by the old one's plus a term that depends on the KL divergence between them, so a step inside the trust region cannot hurt. The practical algorithm approximates this with conjugate gradients.
Why it mattered
Policy optimisation became stable enough to train neural-network controllers for locomotion and games. Its constraint was too costly for large models, and the simpler clipped objective of Proximal Policy Optimisation (PPO, 2017) replaced it in practice, including for tuning language models from human feedback.
Human-Level Control Through Deep Reinforcement Learning
One agent, learning from screen pixels with one set of hyperparameters, reaches a professional tester's level across 49 Atari games.
The story
Before
In 2013 DeepMind had shown a deep Q-network learning seven Atari games from pixels. Combining neural networks with Q-learning was known to be unstable, because consecutive frames are correlated and the network chases a target that moves as it learns.
What happened, and who
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Demis Hassabis and colleagues at DeepMind published "Human-level control through deep reinforcement learning" in Nature on 26 February 2015. It added a target network, a periodically frozen copy used to compute learning targets, to the replay buffer of the 2013 version. One agent with fixed hyperparameters learned 49 games from pixels and score alone, reaching a level comparable to a professional games tester.
Why it mattered
It put deep reinforcement learning on the front page, and replay and target networks became standard tools. It also convinced Google-scale investment in the approach that produced AlphaGo the following year.
A small network trained to copy the temperature-softened outputs of a large one keeps most of its accuracy.
The story
Before
Averaging an ensemble of networks improved accuracy but was too costly to run for every user request. Buciluǎ, Caruana and Niculescu-Mizil had shown in 2006 that a small model could be trained to mimic a large one, but the approach was not widely used for neural networks.
What happened, and who
Geoffrey Hinton, Oriol Vinyals and Jeff Dean of Google posted "Distilling the Knowledge in a Neural Network" to arXiv on 9 March 2015. The student learns from the teacher's probabilities after a softmax with a raised temperature, which reveals how the teacher ranks the wrong classes, information they called dark knowledge.
Why it mattered
Distillation became the standard way to compress a big model into one cheap enough to serve. It also underlies teacher-student training of language models, where a strong model's outputs teach a smaller one.
The first public account of Borg, the system that schedules Google's jobs across clusters of tens of thousands of machines.
The story
Before
Publicly known cluster managers were Mesos, described in 2011, and Hadoop YARN. Google had run Borg in production for over a decade without a public description of its design, so its lessons on packing jobs, priorities and failures were unavailable to others.
What happened, and who
Abhishek Verma, Luis Pedrosa, Madhukar Korupolu, David Oppenheimer, Eric Tune and John Wilkes of Google presented "Large-scale cluster management at Google with Borg" at EuroSys in Bordeaux, held 21–24 April 2015. It describes cells of machines, jobs made of tasks, priority and preemption, sharing machines between production and batch work, and reclaiming resources that jobs reserve but do not use.
Why it mattered
Kubernetes had been announced a year earlier, and the paper explained the ideas it inherited: declarative job specifications, labels, and controllers that keep state converging. It is the main reference for how a large cluster scheduler is designed, and for why sharing machines raises utilisation.
One TCP connection carries many interleaved request-response streams, with compressed headers.
The story
Before
HTTP/1.1 sent one request at a time per connection, so a slow response blocked those behind it. Browsers worked around this by opening about six connections per host, and sites sharded assets across hostnames. Each request also repeated verbose headers in plain text.
What happened, and who
The IETF HTTP working group published RFC 7540, "Hypertext Transfer Protocol Version 2 (HTTP/2)", in May 2015, edited by Mike Belshe, Roberto Peon and Martin Thomson. It grew out of SPDY, an experimental protocol Google had begun in 2009. HTTP/2 splits messages into frames on independent streams and compresses headers with HPACK (RFC 7541).
Why it mattered
Browsers and servers could load a page over one connection without head-of-line blocking between requests. TCP still delays every stream when one packet is lost, and that remaining problem motivated HTTP/3, which runs over QUIC instead.
A Malaysian ISP announced about 179,000 prefixes it should not have, and for two hours traffic to major sites took a detour.
The story
Before
The Border Gateway Protocol (BGP), which networks use to tell each other which addresses they can reach, accepts what a neighbour announces. Nothing in the protocol checks that a network is entitled to announce a route, so safety depends on each operator filtering what it sends and accepts.
What happened, and who
On 12 June 2015 Telekom Malaysia (AS4788) announced roughly 176,000 to 179,000 prefixes to Level 3 (AS3549), which accepted them and passed them on to its peers and customers. The BGPmon monitoring service saw it begin at about 08:43 UTC. Traffic for Google, Microsoft and many others was drawn towards Telekom Malaysia's network, which could not carry it; the effects lasted about two hours and were worst in Australia and New Zealand.
Why it mattered
It is the standard example of a route leak: no attacker was needed, and a single missing filter at one network and one provider that accepted the routes hurt users worldwide. It strengthened the case for prefix filtering, and later for the RPKI and the MANRS operator norms.
Docker, CoreOS and others agreed to standardise the format of container images and how they are run.
The story
Before
Docker's image format and runtime were a de facto standard controlled by one company. CoreOS had announced a rival container format, rkt and the appc specification, in 2014, and there was a risk of incompatible images and runtimes.
What happened, and who
The Open Container Initiative was launched on 22 June 2015 by Docker, CoreOS and other container-industry companies, under the Linux Foundation. Docker contributed its runtime, runC, as the reference implementation. The image specification reached version 1.0 in 2017.
Why it mattered
Any compliant tool can build an image and any compliant runtime can run it, so Kubernetes, Podman, containerd and Docker share the same images and registries. The layered image format that overlay file systems make cheap became an open standard, not one vendor's.
A busy PostgreSQL database refused all writes for most of a working day because it ran out of transaction IDs.
The story
Before
PostgreSQL numbers transactions with a 32-bit counter that wraps around, so old rows must periodically be marked frozen by vacuum or they would seem to lie in the future. If vacuum falls too far behind, the database stops accepting writes rather than risk corrupting data. Sentry's largest tables, under heavy writes, had outpaced its autovacuum settings.
What happened, and who
Sentry, the error-tracking service, hit this limit on Monday 20 July 2015, and its hosted product stopped accepting writes for most of the US working day. The team watched vacuum work through the tables and, after about three hours, truncated a non-critical mapping table, after which writes resumed within minutes. David Cramer published the postmortem on 23 July.
Why it mattered
It is the widely cited public warning about transaction-ID wraparound. It shows that autovacuum needs monitoring and headroom, and that turning it off or running it only at quiet hours leads to this outage. Teams running PostgreSQL now alert on the age of their oldest transaction ID.
Go's collector began running alongside the program, cutting pauses by a factor of ten and later to under a millisecond.
The story
Before
Go's collector up to version 1.4 stopped the whole program while it marked live memory, so the pause grew with the heap. The Go team feared that unless this was fixed, Go would not succeed for servers where latency matters.
What happened, and who
Go 1.5, released on 19 August 2015, introduced a concurrent tri-colour mark-sweep collector designed by Rick Hudson, Austin Clements and colleagues, with a pacer that decides when to start each cycle. On a Twitter production server, pauses fell from around 300–400 ms to 30–40 ms. Go 1.8, in February 2017, removed the remaining stop-the-world stack scanning and brought pauses under a millisecond.
Why it mattered
Pauses stopped growing with heap size, making Go a practical choice for latency-sensitive services. The collector also has a single tuning knob, GOGC, which keeps operating it simple.
Building a vocabulary of subword pieces by merging frequent pairs lets a model handle words it has never seen.
The story
Before
Translation models used a fixed vocabulary of the most frequent words, and mapped everything else to one unknown-word token or a dictionary lookup. Names, compounds and rare inflections were lost. Character-level models covered every word but made sequences much longer and slower.
What happened, and who
Rico Sennrich, Barry Haddow and Alexandra Birch of the University of Edinburgh posted "Neural Machine Translation of Rare Words with Subword Units" to arXiv on 31 August 2015; it appeared at ACL 2016. They adapted byte-pair encoding, a compression method described by Philip Gage in 1994, to start from characters and repeatedly merge the most frequent adjacent pair into a new symbol.
Why it mattered
Any word can be written as pieces, and common words stay single tokens. Byte-pair encoding and its variants became the tokenisation of nearly every large language model, and they fix the vocabulary size and the cost per word.
DQN's replay buffer and target networks applied to continuous actions, using an actor network that outputs the action directly.
The story
Before
DQN picks an action by taking a maximum over all the actions' values, which is easy for a few joystick moves and impossible to do exactly for a continuous torque or steering angle. Discretising the action finely made the choice space explode.
What happened, and who
Timothy Lillicrap, Jonathan Hunt and colleagues at DeepMind, including David Silver and Daan Wierstra, posted "Continuous control with deep reinforcement learning" to arXiv on 9 September 2015 (ICLR 2016). It builds on the deterministic policy gradient theorem of Silver et al. (2014): an actor network outputs an action, and a critic network scores it, with replay and slowly updated target copies keeping learning stable.
Why it mattered
Deep reinforcement learning could control simulated robots and vehicles with continuous actions from raw observations. Its actor-critic structure is the base for later continuous-control methods such as TD3 and SAC.
Choosing an action with one set of weights and scoring it with another reduces Q-learning's tendency to overestimate values.
The story
Before
Q-learning bootstraps from the maximum of its own noisy value estimates, and the maximum of noisy numbers is biased upwards. Hado van Hasselt had described this and proposed Double Q-learning in 2010 for small tabular problems, but DQN still used the plain maximum.
What happened, and who
Hado van Hasselt, Arthur Guez and David Silver at DeepMind posted "Deep Reinforcement Learning with Double Q-learning" to arXiv on 22 September 2015 (AAAI 2016). It showed that DQN overestimates values on Atari games, and used the online network to choose the best next action and the target network to evaluate it.
Why it mattered
It is a one-line change that lowers the bias and improved scores on several Atari games. It became a routine component of later value-based agents, including Rainbow.
A measurement study finds that wrong row-count estimates, not the cost model, are what most often produce bad query plans.
The story
Before
Optimiser research had long concentrated on cost models and on how to enumerate join orders. A query optimiser picks a plan by estimating how many rows each step produces, but those estimates had not been tested against realistic data with many correlated columns.
What happened, and who
Viktor Leis, Andrey Gubichev, Atanas Mirchev, Peter Boncz, Alfons Kemper and Thomas Neumann published "How Good Are Query Optimizers, Really?" in Proceedings of the VLDB Endowment, volume 9, issue 3, in November 2015. They built the Join Order Benchmark from the IMDB dataset and compared PostgreSQL and other systems, finding estimation errors of several orders of magnitude on multi-join queries.
Why it mattered
It showed that a better cost model or a smarter search cannot repair plans built on bad row estimates. Research on cardinality estimation, including learned estimators, follows from it, and the Join Order Benchmark is still used to test optimisers.
Replaying the experiences a value network is most wrong about, instead of random ones, speeds up learning.
The story
Before
DQN stored past transitions in a buffer and sampled them uniformly. Most were already predicted well and taught the network little, while rare surprising transitions were replayed no more often than the rest.
What happened, and who
Tom Schaul, John Quan, Ioannis Antonoglou and David Silver at DeepMind posted "Prioritized Experience Replay" to arXiv on 18 November 2015 (ICLR 2016). Transitions are sampled in proportion to their temporal-difference error, and importance-sampling weights correct the bias that non-uniform sampling introduces.
Why it mattered
Agents reached the same scores in far fewer updates on many Atari games. It is one of the six components combined in Rainbow (2017), and prioritised sampling recurs in later replay-based methods.
A Q-network split into a state-value stream and an action-advantage stream learns which states matter without trying every action.
The story
Before
DQN's network had one output per action and each was learned separately. In many states the choice of action barely matters, so the network spent effort learning near-identical values for every action, and only the taken action's value was updated.
What happened, and who
Ziyu Wang, Tom Schaul, Matteo Hessel, Hado van Hasselt, Marc Lanctot and Nando de Freitas at DeepMind posted "Dueling Network Architectures for Deep Reinforcement Learning" to arXiv on 20 November 2015; it won a best paper award at ICML 2016. After shared layers, one stream estimates the state's value and another the advantage of each action, and they are combined into Q-values.
Why it mattered
A single update improves the value of a state for all its actions, which helps most when actions have similar outcomes. The architecture was one of the components combined in Rainbow (2017), and needs no change to the learning algorithm.
Adding a layer's input to its output lets networks more than a hundred layers deep train, and win ImageNet.
The story
Before
Very deep plain networks trained worse than shallower ones, and not from overfitting: a 56-layer network had higher training error than a 20-layer one. This degradation problem capped how deep networks could usefully go, in the 20 to 30 layer range of VGG and GoogLeNet.
What happened, and who
Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun of Microsoft Research Asia posted "Deep Residual Learning for Image Recognition" to arXiv on 10 December 2015 (best paper at CVPR 2016). Each block computes a change to its input and adds it back through an identity shortcut, so a block can start as close to doing nothing. A 152-layer ensemble won ILSVRC 2015 classification with 3.57% top-5 error.
Why it mattered
Gradients and signals travel through the shortcuts, so depth stopped being the barrier. Residual connections now surround the layers of almost every deep architecture, including every block of a transformer.
Congestion control driven by a model of bottleneck bandwidth and round-trip time instead of packet loss.
The story
Before
TCP congestion control such as Reno and CUBIC treated packet loss as the signal that the network was full. On paths with large buffers this fills the buffers before any loss appears, adding delay, and on paths with shallow buffers or random loss it backs off far too soon.
What happened, and who
Neal Cardwell, Yuchung Cheng, C. Stephen Gunn, Soheil Hassas Yeganeh and Van Jacobson at Google published BBR: Congestion-Based Congestion Control in ACM Queue, in its September-October 2016 issue. BBR measures the path's bottleneck bandwidth and round-trip propagation time and paces its sending to match them. The article reports it running on Google's B4 backbone and rolling out on Google.com and YouTube.
Why it mattered
BBR was the first widely deployed alternative to loss-based control, and it is now an option in Linux and used by large services. It also reopened the question of what the right congestion signal is, which is why a choice between CUBIC and BBR is now a real tuning decision for an engineer.
A garbage collector for OpenJDK that moves live objects while the program runs, so pauses do not grow with the heap.
The story
Before
OpenJDK's default collector G1 compacted the heap by copying objects while the application was stopped, so its pauses depended on how much live data it had to move. Moving objects concurrently is hard because every reference to a moved object must be redirected at once.
What happened, and who
Christine Flood, Roman Kennke, Andrew Dinn, Andrew Haley and Roland Westrelin of Red Hat published "Shenandoah: An open-source concurrent compacting garbage collector for OpenJDK" at PPPJ in Lugano in 2016. Each object carries an extra forwarding pointer, so the collector can copy an object and switch every reader over to the new copy without stopping the program. It shipped in mainline OpenJDK 12 in March 2019, as an experimental feature.
Why it mattered
Pauses became short and independent of live-data size, so a large heap no longer means long pauses. It gave Java services a choice of low-latency collector, alongside the Z garbage collector (ZGC), when pause time matters more than throughput.
Neural networks that suggest moves and judge positions, guiding tree search, beat a professional Go player 5–0.
The story
Before
Go has about 250 legal moves per turn over roughly 150 turns, too many for the exhaustive search that let programs beat chess champions, and positions are hard to score by hand. The strongest programs used Monte Carlo tree search, which plays random games to the end, and reached strong amateur level.
What happened, and who
David Silver, Demis Hassabis and colleagues at DeepMind published "Mastering the game of Go with deep neural networks and tree search" in Nature on 27 January 2016. A policy network suggests moves and a value network scores positions, trained first on human expert games and then by self-play. The system beat the European champion Fan Hui 5–0 in October 2015, the first time a program beat a professional on a full board without handicap.
Why it mattered
Go had been expected to stay out of reach for years, and learned evaluation plus search settled it. The combination of search guided by learned networks, then self-play without human games in AlphaGo Zero and AlphaZero, became a template for game-playing and planning systems.
A blog post shows why a lock with a timeout is unsafe unless the resource it guards checks a number that only increases.
The story
Before
Distributed locks were often built as a key with an expiry in a store such as Redis, and the Redlock algorithm proposed running it across several Redis nodes. The assumption was that a lock holder that pauses, for a garbage collection or a slow network, would simply notice it had lost the lock. A paused client wakes up still believing it holds the lock.
What happened, and who
Martin Kleppmann, a researcher at the University of Cambridge and the author of "Designing Data-Intensive Applications", published "How to do distributed locking" on 8 February 2016. He showed that a stalled holder can write after its lease has expired and someone else holds the lock, and proposed that each grant carry a fencing token, a number that increases with every grant, which the storage service checks and uses to reject stale writers. Chubby, Google's lock service, had described the same idea as a sequencer.
Why it mattered
It settled the argument about lease-based locks: mutual exclusion has to be enforced by the resource, not trusted to the client's clock. Fencing tokens are the standard answer to "is this lock safe?" and appear in the leader-election and locking designs of systems built on ZooKeeper or etcd.
A program won a five-game Go match 4–1 against one of the world's best players, who won game 4 with a move nobody expected.
The story
Before
Go had resisted computers because of its enormous number of possible positions and the difficulty of judging a position by rule. After Deep Blue beat the chess champion Garry Kasparov in 1997, many experts still expected Go to take about another decade.
What happened, and who
DeepMind's AlphaGo played Lee Sedol, an 18-time world champion, in Seoul from 9 to 15 March 2016 for a prize of one million dollars. AlphaGo won the first three games and the fifth. Lee won game 4 with move 78, a wedge play that AlphaGo had judged very unlikely and then misplayed against.
Why it mattered
For many people outside computer science it was the moment they took modern AI seriously, and it drew wide attention to learned systems. The match showed a system that had trained itself, and professional players began studying its moves.
A layered graph of near neighbours, searched from a sparse top layer downwards, finds similar vectors in logarithmic time.
The story
Before
Exact nearest-neighbour search over high-dimensional vectors degrades to scanning everything, because tree and hash indexes lose their ability to prune. The authors' own earlier navigable-small-world graphs found neighbours by hopping along edges, but a search could wander through many nodes before it got close.
What happened, and who
Yury Malkov and Dmitry Yashunin posted "Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs" to arXiv on 30 March 2016; it appeared in IEEE TPAMI in 2020. Each point is placed in a random number of layers, as in a skip list. A query enters at the sparse top layer and moves greedily towards the query, dropping down a layer at each local minimum.
Why it mattered
Search time grows roughly logarithmically with the number of vectors, with good recall. HNSW is the default index in many vector databases and libraries, the machinery behind semantic search and retrieval for language models.
A chip built for one job, the matrix arithmetic of neural networks, run in Google's data centres for a year before anyone outside knew.
The story
Before
Neural-network inference ran on CPUs and GPUs. Both are general-purpose designs that spend silicon and energy on flexibility, such as caches, branch prediction and high-precision arithmetic, that a network's fixed multiply-and-add pattern does not need.
What happened, and who
Sundar Pichai announced the Tensor Processing Unit at the Google I/O developer conference in Mountain View on 18 May 2016. A blog post the same day by Norm Jouppi explained that Google had been running these custom chips in its data centres for over a year, for services such as Search and Street View and in the AlphaGo match against Lee Sedol. The first generation is an 8-bit matrix-multiplication engine aimed at inference.
Why it mattered
It made a hyperscaler's custom accelerator public and showed that a chip designed around one workload could beat general-purpose parts on cost and energy. Later generations added training and were offered to outside users through Google Cloud, and other cloud providers and chip makers followed with their own neural-network accelerators.
An investigation found that a widely used criminal risk score made different kinds of mistakes for Black and white defendants.
The story
Before
US courts and parole boards used risk-assessment software such as COMPAS, sold by Northpointe, to score how likely a defendant was to reoffend. A score computed by a formula was assumed to be neutral, and its error rates were not examined by race.
What happened, and who
On 23 May 2016 the news organisation ProPublica published Machine Bias, by Julia Angwin, Jeff Larson, Surya Mattu and Lauren Kirchner. They analysed more than 7,000 COMPAS scores given in Broward County, Florida, in 2013 and 2014. Among defendants who did not go on to reoffend, 44.9% of Black defendants had been labelled higher risk against 23.5% of white defendants; among those who did reoffend, 47.7% of white defendants had been labelled low risk against 28.0% of Black defendants. Northpointe disputed the analysis.
Why it mattered
The dispute forced a question that had no agreed answer: what does it mean for a score to be fair? Researchers soon showed that several natural fairness measures cannot all hold at once when groups differ in base rates, and the article became the standard example in the fairness literature and in debates about algorithmic decisions in courts, credit and hiring.
A cloud data warehouse that keeps data in object storage and lets compute be resized independently of it.
The story
Before
Data warehouses of the time, such as Teradata and Amazon Redshift, were shared-nothing: each node stored a slice of the data on its own disks and processed that slice. Adding or removing nodes meant moving data, and a heavy query and a light one competed for the same machines.
What happened, and who
Benoit Dageville and colleagues at Snowflake published The Snowflake Elastic Data Warehouse at the ACM SIGMOD conference in June 2016. It described a system built from scratch for the cloud: tables stored as immutable files in object storage (Amazon S3), and queries run by separate clusters of compute nodes, called virtual warehouses, that can be created, resized and shut down independently.
Why it mattered
Separating storage from compute became the default shape of cloud analytical databases: many teams can query the same data with different amounts of compute, and idle compute costs nothing. The same idea underlies the table formats that let several engines share files in object storage.
One comment containing 20,000 spaces made a text-trimming regular expression backtrack until the whole site fell over for 34 minutes.
The story
Before
Regular expressions were treated as cheap, constant-cost string helpers. Most engines match by trying alternatives and backing up when one fails, so a pattern that looks harmless can take time that grows with the square of the input on the wrong text.
What happened, and who
On 20 July 2016 a post on Stack Overflow's home page list contained about 20,000 consecutive whitespace characters. The site trimmed whitespace with the pattern ^[\s\u200c]+|[\s\u200c]+$; on that line the engine retried the match from each space in turn, roughly 200 million checks. Every home-page request burned CPU, and because the load balancer used the home page as its health check, it took the web servers out of rotation. Stack Overflow was unavailable for 34 minutes, and the fix replaced the regex with a substring function.
Why it mattered
It became the best-known example of regular-expression denial of service, and it moved backtracking cost from a textbook remark to a code-review question about any pattern that runs on user input. It also shows two lasting lessons: a health check that exercises expensive code can turn one bad input into a full outage, and linear-time matching engines exist for this reason.
A blog post blamed write amplification, physical replication and replica conflicts for Uber leaving PostgreSQL.
The story
Before
Uber ran its main data on PostgreSQL 9.2. A PostgreSQL update writes a new row version and, unless it can be done within the same page, a new entry in every index, because indexes point at a row's physical location. Replication shipped the physical write-ahead log, so every one of those small physical changes crossed the network.
What happened, and who
On 26 July 2016 Uber Engineering published Why Uber Engineering Switched from Postgres to MySQL, written by Evan Klitzke. It listed the design costs the team had met: write amplification from updates, verbose physical replication between data centres, replicas that cancel long queries to keep applying the log, upgrades that could not be done between versions on a live replica set, and a replication bug in 9.2 that duplicated rows after a failover. Uber's newer storage layer, Schemaless, ran on MySQL with InnoDB.
Why it mattered
The post was widely debated, and PostgreSQL developers answered several of its points, but it gave engineers a concrete checklist of what the append-only heap, physical replication and MVCC cost under an update-heavy workload. The same questions come up in any comparison of PostgreSQL and MySQL today.
The Challenge-ACK Off-Path TCP Attack (CVE-2016-5696)
A shared global rate limit leaked whether spoofed guesses hit a live TCP connection.
The story
Before
RFC 5961 (2010) hardened TCP against blind injection by answering a suspicious segment with a challenge ACK instead of resetting the connection. Linux capped these replies at a global 100 per second so they could not be used to flood a host. The cap was shared by every connection on the machine.
What happened, and who
Yue Cao, Zhiyun Qian, Zhongjie Wang, Tuan Dao and Srikanth Krishnamurthy of UC Riverside, with Lisa Marvel of the US Army Research Laboratory, presented Off-Path TCP Exploits: Global Rate Limit Considered Dangerous at USENIX Security in August 2016. An attacker who can send spoofed packets and also hold one connection of its own can count how many challenge ACKs it receives. The count reveals whether its guess for a victim's connection was right, and so lets it find sequence numbers, reset the connection or inject data. The attack typically finished in 40 to 60 seconds. It affected Linux from 3.6, and the kernel was patched in July 2016.
Why it mattered
It showed that a defence's own state can be a side channel: a counter shared between connections told an outsider about a connection it could not see. Linux made challenge-ACK behaviour unpredictable to outsiders, and the lesson, that shared limits leak, carried into later side-channel work on kernels and CPUs.
A gradient-boosted tree system engineered to handle sparse data and billions of examples on modest hardware.
The story
Before
Gradient boosting, which builds trees one at a time to correct the errors of the ones before, dated from Jerome Friedman's work around 2000. It was accurate on tabular data, but common implementations were slow and did not handle sparse inputs or datasets larger than memory well.
What happened, and who
Tianqi Chen and Carlos Guestrin of the University of Washington published XGBoost: A Scalable Tree Boosting System at the ACM KDD conference in San Francisco in August 2016 (a preprint had appeared in March). It added a sparsity-aware split-finding algorithm, a weighted quantile sketch for approximate split candidates, and system work on cache access patterns, compression and sharding.
Why it mattered
XGBoost became the default tool for tabular prediction and won a large share of machine-learning competitions on Kaggle. Its design, and the libraries that followed it such as LightGBM and CatBoost, are why boosted trees remain hard to beat on structured data that deep networks handle less well.
PostgreSQL 9.6 lets one query use several CPU cores for scans, joins and aggregates.
The story
Before
Every PostgreSQL query ran in a single process on a single core. A large analytical scan could leave most of a many-core server idle, and the only way to use the rest was to run more queries at once.
What happened, and who
The PostgreSQL Global Development Group released PostgreSQL 9.6 on 29 September 2016. Its headline feature was parallel query: a leader process starts worker processes that share a sequential scan, and can also run joins and aggregates in parallel. It is controlled by settings such as max_parallel_workers_per_gather and was not enabled by default. The earlier 9.5 release had added BRIN indexes for very large tables.
Why it mattered
It started PostgreSQL's parallel execution, which later releases extended to index scans, more join types and parallel maintenance work. It also changed what the planner has to decide, because every plan now weighs the cost of starting workers against the work they share.
A botnet of hijacked cameras and routers knocked a DNS provider offline and, with it, much of the popular web.
The story
Before
Large botnets had mostly been built from infected desktop computers. Home cameras, routers and recorders shipped with factory-default passwords and were rarely patched, but few people had turned them into a weapon at scale.
What happened, and who
Mirai scans the internet for Linux-based devices, tries a table of more than sixty factory-default usernames and passwords, and enlists those that answer. Its source code was published on a hacking forum in September 2016. On 21 October 2016 Mirai-based traffic hit the DNS provider Dyn in several waves; sites that depended on Dyn, including GitHub, Twitter, Reddit, Netflix and Airbnb, became unreachable for many users.
Why it mattered
It showed that one DNS provider can be a single point of failure for many unrelated services, and that insecure consumer devices are a source of attack traffic no defender controls. It pushed both DNS redundancy, with several providers per domain, and pressure on device makers to drop default passwords.
A compact, portable bytecode that browsers run at near-native speed, designed as a compilation target rather than a language to write by hand.
The story
Before
Mozilla's asm.js, from 2013, was a subset of JavaScript that compilers could target and engines could optimise. It still shipped as text that had to be downloaded and parsed, and it was limited by what JavaScript could express. Google's Native Client ran native code in Chrome but was not portable across browsers.
What happened, and who
In 2017 the first version of WebAssembly, the minimum viable product, was finished and shipped in browsers: Firefox and Chrome in March, Safari in September, and Edge later in the year. Andreas Haas and colleagues from Google, Mozilla, Microsoft and Apple described its design and formal semantics in Bringing the Web up to Speed with WebAssembly at PLDI 2017.
Why it mattered
It gave every browser one binary format that compiles quickly, validates before it runs and runs in a sandbox. That made it a target for C, C++, Rust and Go, and it has since spread beyond the browser to plugin systems, edge computing and serverless runtimes.
A leap second made the clock step backwards, a negative duration reached a random-number call, and DNS servers panicked for nearly seven hours.
The story
Before
Cloudflare's DNS software, RRDNS, written in Go, measured how well each upstream resolver was doing by subtracting one wall-clock reading from another. The code assumed that time never goes backwards, which the wall clock does not guarantee.
What happened, and who
At 00:00 UTC on 1 January 2017 a leap second was added and the clock briefly moved backwards. An elapsed time in RRDNS became negative, passed through a smoothing step into the code that picks a resolver, and reached rand.Int63n, which does not accept a negative argument and panicked. Customers whose domains used CNAME records were affected: at its peak about 0.2% of Cloudflare's DNS queries failed. The fix, deployed by 01:48 UTC, was a one-character check; the impact ended at 06:45 UTC.
Why it mattered
It is the standard example of why code that measures durations should use a monotonic clock, which only moves forward, and keep the wall clock for timestamps. Go 1.9, released in 2017, made time.Time carry a monotonic reading for this reason.
A gated layer sends each token to a few of thousands of experts, so parameters grow to 137 billion while compute per token barely moves.
The story
Before
Conditional computation, using only part of a network for each input, had been an idea since the mixture of experts of Jacobs and colleagues in 1991. Nobody had a practical recipe for training such a network at scale: the gating tended to favour a few experts, and splitting a batch among experts left each with too little work to keep hardware busy.
What happened, and who
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton and Jeff Dean at Google Brain posted Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer to arXiv on 23 January 2017. A trainable gate picks the top few of up to thousands of feed-forward experts for each input, and extra loss terms keep the load balanced. Placed between stacked LSTM layers, it reached 137 billion parameters and improved language modelling and translation at low extra compute.
Why it mattered
Capacity now grows with the number of experts while the cost per token grows only with the number used. The idea returned in Transformer form in GShard and the Switch Transformer and is the basis of open mixture-of-experts language models such as Mixtral and DeepSeek's.
An engineer wiped the wrong PostgreSQL server, and the backups that should have saved it turned out not to work.
The story
Before
GitLab.com had several protections: replication, nightly pg_dump backups and disk snapshots. The dump job ran PostgreSQL 9.2 tools against a 9.6 server and failed, and the alert emails were rejected by the receiving mail server, so nobody noticed. The backups had never been tested by restoring them.
What happened, and who
On 31 January 2017, while trying to resync a lagging replica, a GitLab engineer ran a command to wipe the PostgreSQL data directory and ran it on the primary, db1, instead. About 300 GB was removed before it was stopped. The team recovered from a disk snapshot taken about six hours earlier, which lost roughly five thousand projects, five thousand comments and seven hundred new accounts created in that window. Restoration finished on 1 February, and GitLab published a candid post-mortem and streamed the recovery live.
Why it mattered
It is the reference case for the rule that a backup you have not restored is not a backup. Teams now schedule restore tests, monitor that backup jobs succeed rather than merely that they ran, and treat the openness of GitLab's post-mortem as a model for incident reports.
Two different PDF files with the same SHA-1 hash: the first practical collision in a hash function still trusted in the wild.
The story
Before
SHA-1, published by the US government in 1995, was known to be weaker than its 160-bit size suggests: since 2005 researchers had shown attacks on paper that needed fewer than the 280 operations of a brute-force collision search. It was nevertheless still used in certificates, version control and signatures, and no collision had been produced.
What happened, and who
On 23 February 2017 researchers at CWI Amsterdam and Google (Marc Stevens, Pierre Karpman, Elie Bursztein, Ange Albertini and Yarik Markov) published two visibly different PDFs with the same SHA-1 hash. It took roughly nine quintillion, about 263, SHA-1 computations, or about 6,500 CPU-years and 110 GPU-years, thousands of times fewer than brute force.
Why it mattered
It turned SHA-1's weakness from theory into a file anyone could download. Browsers and certificate authorities finished retiring SHA-1 certificates, Git added collision detection, and NIST set a deadline of the end of 2030 for removing SHA-1 entirely.
One mistyped input removes far more servers than intended, and much of the web stops for four hours.
The story
Before
Thousands of sites and many AWS services — EC2 launches, EBS snapshots, Lambda — depended on S3 in the us-east-1 region. So did AWS's own Service Health Dashboard, which is why it could not report the outage.
What happened, and who
At 9:37 am PST an S3 engineer debugging the billing system ran a command meant to remove a few servers, and entered one input incorrectly. It removed servers from two core subsystems: the index, which knows where every object is, and placement, which allocates space for new ones. Both had to restart in full; placement finished recovering at 1:54 pm.
Why it mattered
It showed how far one region's storage reached into everything built on it, and how a tool that can remove capacity quickly can remove too much. AWS changed the tool to remove capacity slowly and refuse to go below a minimum, and sped up splitting the index into smaller cells so a failure touches less.
Redo processing moves into a multi-tenant storage service so only redo log records cross the network.
The story
Before
Running a mirrored MySQL on Amazon's network disks (EBS) meant every write sent data pages, redo log, binary log and metadata through several synchronous network hops to mirrored copies. Throughput and latency were dominated by that network traffic, not by the database's own work.
What happened, and who
Alexandre Verbitski, Anurag Gupta, Debanjan Saha, Murali Brahmadesam and colleagues at Amazon presented Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases at SIGMOD in Chicago, 14-19 May 2017. In Aurora the database instance sends only redo log records to a storage service that keeps six copies of the data across three Availability Zones and applies the log itself, reaching a write quorum of four of the six.
Why it mattered
The paper made "the log is the database", with a quorum-replicated storage layer that materialises pages on demand, the reference design for cloud-native relational engines. Later systems that separate compute from storage, including other cloud-managed PostgreSQL and MySQL variants, follow the same idea.
Shapley values from game theory, used to split a model's prediction fairly among its input features.
The story
Before
Explaining a single prediction of a complex model relied on separate methods, such as LIME, DeepLIFT and layer-wise relevance propagation, each with its own heuristics. There was no argument for why one attribution should be preferred to another.
What happened, and who
Scott Lundberg and Su-In Lee of the University of Washington posted A Unified Approach to Interpreting Model Predictions to arXiv on 22 May 2017; it appeared at NIPS 2017. They defined a class of additive feature-attribution methods, showed that six existing methods belong to it, and proved that only one attribution, the Shapley value from Lloyd Shapley's cooperative game theory, satisfies the properties they require. They called the resulting values SHAP.
Why it mattered
SHAP gave practitioners one principled way to answer why a model made a particular prediction, and its library became a standard tool for explaining tree ensembles and neural networks. It is widely used in credit, healthcare and model-debugging work, alongside the caveat that Shapley values depend on how missing features are modelled.
A reward model trained from people choosing between pairs of clips drives reinforcement learning, with no hand-written reward.
The story
Before
A reinforcement-learning agent needs a reward function, and for many goals, such as doing a backflip or writing a good summary, nobody can write one down. A badly specified reward is exploited by the agent in ways its designers did not intend.
What happened, and who
Paul Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg and Dario Amodei, at OpenAI and DeepMind, posted Deep Reinforcement Learning from Human Preferences to arXiv on 12 June 2017. A person is shown two short clips of the agent and picks the better one; a network learns a reward that predicts those choices, and the agent is trained against it. Feedback on less than one percent of the agent's interactions was enough for tasks in Atari games and simulated robotics.
Why it mattered
This is the direct ancestor of reinforcement learning from human feedback for language models: comparisons, a learned reward model and a policy-gradient step. Fine-tuning language models from human preferences followed in 2019, and the recipe underlies today's instruction-following chat models.
A sequence model with no recurrence at all: only attention and feed-forward layers.
The story
Before
Translation and other sequence tasks used recurrent networks, which read one token after another. That made training hard to parallelise, and information from far back in a sentence had to survive many steps to be used.
What happened, and who
Eight Google authors — Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Łukasz Kaiser and Illia Polosukhin — posted the paper on 12 June 2017. Every token attends to every other in one step, so a whole sequence trains in parallel. It set a new English-to-French record after 3.5 days on eight GPUs.
Why it mattered
Because the Transformer trains in parallel on GPUs, it could be scaled far beyond recurrent networks, and it became the architecture behind nearly every modern language, vision and multimodal model, including the GPT family.
Idempotent producers and transactions arrive in Kafka, so a stream job can read, process and write without duplicates or losses.
The story
Before
Kafka guaranteed each message at least once. If a producer retried after a lost acknowledgement, or a broker failed over, a message could be written twice, and a stream-processing job had to tolerate or remove the duplicates itself.
What happened, and who
Apache Kafka 0.11.0.0 was released on 28 June 2017 with the changes designed in KIP-98. The idempotent producer attaches sequence numbers so a broker can discard a retried duplicate within a partition, and transactions let a producer write to several partitions atomically. Confluent, the company founded by Kafka's original authors, explained the design in a post on 30 June by Neha Narkhede, Guozhang Wang and colleagues. Kafka Streams gained an exactly-once processing option built on them.
Why it mattered
It made exactly-once processing possible for the read-process-write pattern inside Kafka, at a reported cost of about three percent of producer throughput. The guarantee stops at Kafka's edge: a side effect in an external database or an email still needs its own idempotence.
A clipped objective keeps each policy update small, so one batch of experience can safely be reused for several training epochs.
The story
Before
Trust Region Policy Optimization (TRPO), from John Schulman and colleagues in 2015, kept updates safe by constraining how far the policy could move, but it needed a second-order optimisation step that was complex to implement and did not fit architectures with shared layers or noise. Plain policy gradient was simple but unstable and used each batch of experience once.
What happened, and who
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford and Oleg Klimov at OpenAI posted Proximal Policy Optimization Algorithms to arXiv on 20 July 2017. PPO replaces the constraint with a clipped surrogate objective: the ratio between the new and old policy's action probabilities is clipped, so a step that would move it too far earns no extra reward. That allows several epochs of minibatch updates on the same data with only first-order optimisation.
Why it mattered
PPO was simple, robust and cheap enough to become the default policy-gradient method in robotics and games. It is also the optimiser used in the reinforcement-learning step of RLHF for language models, so most chat assistants have been shaped by it.
Google reports that its encrypted UDP-based transport already carried about 7% of Internet traffic.
The story
Before
HTTPS ran over TCP with TLS on top. A new connection spent a round trip on the TCP handshake and one or two more on TLS before the first request, and on HTTP/2 a single lost packet stalled every multiplexed stream. TCP lives in operating-system kernels and is inspected by middleboxes, so changing it takes years.
What happened, and who
Adam Langley and colleagues at Google presented The QUIC Transport Protocol: Design and Internet-Scale Deployment at ACM SIGCOMM in August 2017. QUIC, begun at Google around 2012, runs over UDP in user space, encrypts nearly all of its headers, multiplexes independent streams and combines the transport and cryptographic handshakes. The paper describes its design, its use in Chrome and Google's servers, and estimates that about 7% of Internet traffic was QUIC.
Why it mattered
It showed that a new transport could be deployed and improved at Internet scale without waiting for kernels and middleboxes. The IETF standardised the design as QUIC in RFC 9000, and it underlies HTTP/3.
Random seeds and implementation details were enough to reverse which reinforcement-learning algorithm looked better.
The story
Before
Papers on deep reinforcement learning usually compared algorithms using a handful of training runs, reporting a single curve or the best run. Because training is noisy, a difference between two methods could be a difference between two lucky seeds.
What happened, and who
Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup and David Meger posted Deep Reinforcement Learning that Matters to arXiv on 19 September 2017. They ran the same algorithm with different random seeds, split the runs into two groups and found the groups could differ significantly. They also showed that hyperparameters, reward scaling and the choice of code base changed the ranking of methods.
Why it mattered
The paper changed evaluation practice in the field: reporting many seeds, confidence intervals and full hyperparameters, and releasing code, became expected. For an engineer reading a benchmark table, it is the reason to ask how many runs stand behind each number.
A hash table that keeps one byte of metadata per slot in a separate array and checks sixteen slots at once with SIMD instructions.
The story
Before
The C++ standard library's std::unordered_map uses chaining: each bucket points at a linked list of separately allocated nodes. Every lookup follows pointers, which misses the CPU cache, and the interface promises that element addresses never change, which rules out most faster layouts.
What happened, and who
Matt Kulukundis of Google gave the talk Designing a Fast, Efficient, Cache-friendly Hash Table, Step by Step at CppCon 2017 on 27 September. It walked from std::unordered_map to the design Google called SwissTable: open addressing in a flat array, with a byte per slot holding a few bits of the hash and the slot's state, so that a group of slots can be compared with one SSE instruction. Google released it in its open-source Abseil library.
Why it mattered
Abseil's flat_hash_map is built on it, and Rust's standard HashMap uses the hashbrown port of the same design. The default hash table in a large share of new C++ and Rust code is now cache-conscious in this way.
Logical Replication and Partitioning in PostgreSQL 10
PostgreSQL 10 adds built-in publish and subscribe replication, declarative partitioning, quorum commit and stronger password authentication.
The story
Before
Replicating selected tables, or between major versions, needed third-party tools built on triggers, such as Slony and Londiste. Partitioning meant hand-built table inheritance with triggers or rules to route rows and constraints for the planner to prune.
What happened, and who
The PostgreSQL Global Development Group released PostgreSQL 10 on 5 October 2017. It added logical replication using publications and subscriptions, declarative partitioning with automatic routing of rows, quorum synchronous commit (ANY in synchronous_standby_names), SCRAM-SHA-256 password authentication and ICU collation support. It also renamed pg_xlog to pg_wal.
Why it mattered
Logical replication is the basis of near-zero-downtime major upgrades and of selective replication between databases. Declarative partitioning is how large tables are managed and pruned, and SCRAM replaced the weaker MD5 password scheme.
Six separate improvements to the DQN algorithm, combined in one agent, gain more together than any does alone.
The story
Before
DQN, DeepMind's 2013-2015 deep Q-network that learned Atari games from pixels, had attracted many fixes: double Q-learning, prioritised replay, dueling networks, multi-step returns, distributional value estimates and noisy exploration. Each had been shown to help on its own, but nobody knew whether the gains overlapped or stacked.
What happened, and who
Matteo Hessel and colleagues at DeepMind posted Rainbow: Combining Improvements in Deep Reinforcement Learning to arXiv on 6 October 2017. They combined the six extensions in a single agent and ran it on the Atari 2600 benchmark, where it beat each ingredient in both data efficiency and final score. Ablation runs measured what each part contributed.
Why it mattered
It showed that the improvements were largely complementary, and Rainbow became the standard strong baseline for value-based deep reinforcement learning on Atari. It is also the model example of an ablation study that separates the components of a combined method.
Train in 16-bit floating point, keep a 32-bit master copy of the weights, and scale the loss so small gradients survive.
The story
Before
Networks were trained in 32-bit floating point, which used twice the memory and memory bandwidth of 16-bit numbers. Converting everything to 16 bits was faster but failed: many gradient values are smaller than the smallest 16-bit number and round to zero, and small weight updates vanish when added to large weights.
What happened, and who
Paulius Micikevicius and colleagues at NVIDIA and Baidu posted Mixed Precision Training to arXiv on 10 October 2017. Weights, activations and gradients are held in 16-bit; an FP32 master copy accumulates the weight updates; and the loss is multiplied by a constant before back-propagation so gradients stay in range, then divided out. Networks for image classification, detection, translation and speech matched their 32-bit accuracy.
Why it mattered
Half-precision arithmetic runs several times faster on GPU tensor cores and halves memory, so the recipe became standard for training large models and is built into the major frameworks as automatic mixed precision. The 16-bit formats used today extend the idea.
A Go program trained by self-play alone, with no human games, beat the version that had defeated Lee Sedol.
The story
Before
AlphaGo, DeepMind's program that beat the European champion in 2015 and Lee Sedol in 2016, was first trained to imitate human expert games and only then improved by self-play. It also used hand-designed input features and separate policy and value networks.
What happened, and who
David Silver and colleagues at DeepMind published Mastering the game of Go without human knowledge in Nature on 19 October 2017. AlphaGo Zero started from random play, was told only the rules, and learned from games against itself, using one network and Monte Carlo tree search to generate better moves and then training on them. After about three days of training it beat the Lee Sedol version of AlphaGo by 100 games to 0.
Why it mattered
It showed that search can act as a policy-improvement step, so an agent can bootstrap its own training data and exceed the human knowledge it was once thought to need. The same idea generalised at once to chess and shogi in AlphaZero and inspired later self-play and search-based methods.
The hidden processor inside most Intel PCs was found to run MINIX, a teaching microkernel, and its author wrote to Intel about it.
The story
Before
Intel chipsets contain a separate small processor, the Management Engine, that runs its own firmware whenever the machine has power. Users and most engineers could not see it and did not know what software it ran.
What happened, and who
Researchers had found that from version 11 of the Management Engine, used with Skylake and later processors, its operating system is a version of MINIX 3, the microkernel OS that Andrew Tanenbaum's group at Vrije Universiteit Amsterdam built for teaching and reliability. In November 2017 Tanenbaum published An Open Letter to Intel. He wrote that he had learned about this only from a press report and that it would have been courteous of Intel to tell him.
Why it mattered
Intel's choice made MINIX, in Tanenbaum's words, probably the most widely used operating system on x86 computers, a real-world footprint of the microkernel idea: a small, modular kernel that fits in a chip. It also raised the case for auditing firmware that users cannot inspect or turn off.
A proposal to treat an index as a model that predicts where a key sits, instead of searching a tree.
The story
Before
A B-tree finds a key by comparing it against separator keys at each level, and it makes no use of how the keys happen to be distributed. If the keys were the consecutive integers 1 to n, a lookup could be a plain array offset, but the tree would still descend its levels.
What happened, and who
Tim Kraska, Alex Beutel, Ed Chi, Jeffrey Dean and Neoklis Polyzotis of Google posted “The Case for Learned Index Structures” to arXiv on 4 December 2017; it appeared at SIGMOD in June 2018. It views a B-tree as a function from key to position, replaces it with a hierarchy of small neural networks plus a fallback search, and reports up to 70% faster lookups than cache-optimised B-trees with an order of magnitude less memory on some datasets.
Why it mattered
The paper opened the field of learned indexes and drew a line of follow-up structures and critiques, including work showing that simpler non-neural models can do much of the same job. It made database researchers ask what a data structure gains from knowing its data’s distribution.
One algorithm, given only the rules, taught itself chess, shogi and Go in hours and beat the strongest programs in each.
The story
Before
The strongest chess programs, such as Stockfish, searched millions of positions per second with alpha-beta search and evaluation functions tuned by hand and refined over decades. Each game needed its own engine, and the strongest Go program, AlphaGo Zero, was built for Go alone.
What happened, and who
David Silver and colleagues at DeepMind posted Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm to arXiv on 5 December 2017. AlphaZero applied the AlphaGo Zero method, self-play with a neural network guiding Monte Carlo tree search, to chess, shogi and Go with no game-specific knowledge beyond the rules. It surpassed Stockfish after about four hours of training on chess and went on to beat Stockfish, the shogi program Elmo and AlphaGo Zero.
Why it mattered
It was the first strong evidence that one general learning method could reach superhuman play across several games from rules alone. It made learned evaluation plus search the model for game-playing systems and led to later work that learns without even being given the rules.
CPUs leak data through speculative execution and the cache: Meltdown reads kernel memory from user space, Spectre tricks other code into leaking.
The story
Before
Processors run instructions speculatively, guessing which branch will be taken and executing ahead. The assumption was that if the guess was wrong the results were discarded, so speculation left no visible trace. Operating systems relied on the CPU's permission check to keep kernel memory safe even though the kernel was mapped into every process.
What happened, and who
On 3 January 2018 the flaws were made public, in a Google Project Zero post by Jann Horn and in papers from researchers including Paul Kocher and a group at Graz University of Technology (Moritz Lipp, Daniel Gruss and others). Several teams had found them independently, and vendors had been told in 2017. Meltdown lets user code read kernel memory on affected Intel processors by touching it speculatively and recovering the value through cache timing. Spectre coaxes a program into speculatively performing an access that leaks its own data, and affects Intel, AMD and Arm processors.
Why it mattered
It broke a basic assumption of computer design and led to kernel page-table isolation (shipped in Linux 4.15), retpolines, microcode updates and browser changes. Every processor since is designed with speculative side channels in mind, and the mitigations still cost performance on many workloads.
An off-policy actor-critic method that also rewards the policy for staying random, which stabilises learning in continuous control.
The story
Before
Off-policy methods for continuous control such as DDPG, from 2015, reuse old experience and so need few samples, but they learn a deterministic policy and proved brittle to hyperparameter choices. On-policy methods such as PPO were more stable but needed far more samples.
What happened, and who
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel and Sergey Levine at UC Berkeley posted Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor to arXiv on 4 January 2018; it appeared at ICML 2018. Its objective adds the policy's entropy to the reward, so the agent learns to succeed while acting as randomly as possible, which encourages exploration and smooths learning.
Why it mattered
SAC became a default sample-efficient algorithm for continuous control, including robotics, where each real-world sample is costly. The maximum-entropy idea also reappears in later reinforcement-learning methods.
A proposed PostgreSQL table format with in-place updates and an undo log, meant to end table bloat and most need for vacuum.
The story
Before
PostgreSQL's heap keeps every row version in the table and never overwrites in place, so an update behaves like a delete plus an insert. Old versions pile up as bloat until vacuum removes them, and long-running transactions stop that cleanup.
What happened, and who
On 30 January 2018 Robert Haas of EnterpriseDB described zheap on his blog in DO or UNDO - there is no VACUUM: new versions replace the old in place and old versions move to an undo log. On 1 March 2018 Amit Kapila of EnterpriseDB posted a proposal with a design and early benchmarks to the PostgreSQL developers' list. The project is not part of PostgreSQL.
Why it mattered
zheap was never merged, but it made concrete how much of PostgreSQL rests on having no undo log: hint bits, visibility, index maintenance and vacuum all assume it. The trade-offs it exposed, cheaper updates against costlier aborts and deletes, are the choice InnoDB and Oracle make the other way.
Tiny spoofed requests to open memcached servers returned responses up to 51,000 times larger, peaking at 1.35 Tbps.
The story
Before
Reflection attacks send small requests with the victim's forged address to servers that answer with much larger replies. Those using DNS or NTP amplified traffic by tens to a few hundred times, and memcached's UDP mode, which many operators had left open to the internet, had not been used at scale.
What happened, and who
On 28 February 2018 GitHub was hit by a memcached-based attack that peaked at 1.35 terabits per second and 126.9 million packets per second from over a thousand networks. GitHub.com was unavailable from 17:21 to 17:26 UTC and degraded until 17:30. GitHub withdrew its routes from its transit providers and sent traffic through Akamai's Prolexic scrubbing service, which filtered the attack.
Why it mattered
It was the largest attack on record at the time and showed the danger of an unauthenticated, connectionless service reachable from the internet. Operators were urged to block or disable memcached's UDP port, and the incident is the standard example of amplification.
CVE-2018-1058: The Public-Schema Search-Path Attack
Any user could plant lookalike objects in the shared public schema and hijack queries run by more privileged users.
The story
Before
In PostgreSQL every new database had a public schema that any user could create objects in, and it sat on everyone's default search_path. Queries that named a function or table without a schema were resolved by walking that path, so a hostile user's objects could be found first.
What happened, and who
On 1 March 2018 the PostgreSQL project released 10.3 and matching updates for older branches, fixing CVE-2018-1058. A user with rights to create objects in a schema on someone else's search path could capture queries and run code with that person's privileges, for example a superuser running a dump. The release added documentation on secure schema usage and made pg_dump, pg_upgrade, vacuumdb and autovacuum use only pg_catalog in their search path.
Why it mattered
The fix was mostly guidance for administrators, and the hazard stayed in the defaults. PostgreSQL 15 finally removed the ability of ordinary users to create objects in public, and secure search-path practice is now standard advice for functions and extensions.
PostgreSQL could lose committed data after a failed fsync, because the kernel reported the error once and then let later fsyncs succeed.
The story
Before
PostgreSQL, like most databases, assumed that when fsync fails the pages stay dirty in the kernel's cache, so the call can be retried and the data will eventually reach disk. A successful fsync was taken to mean everything written since the last one was durable.
What happened, and who
On 28 March 2018 Craig Ringer posted to the pgsql-hackers mailing list that PostgreSQL's handling of fsync() errors was unsafe and risked data loss, at least on XFS. On Linux, after a writeback error the kernel could mark the failed pages clean and discard the data, and the error might be reported to only one caller. A retried fsync then returned success while the data was gone. Developers on the list, including Tom Lane and Robert Haas, reacted with dismay at the kernel's behaviour.
Why it mattered
It showed that the durability contract between a database and the operating system was weaker than the database believed. PostgreSQL now treats an fsync failure as fatal rather than retrying, and the Linux kernel changed how writeback errors are recorded and reported so that every open file sees them.
A blog post argued that spawning a task is a hidden goto, and proposed scopes that wait for every task started inside them.
The story
Before
Spawning a thread, a goroutine or a callback let a task outlive the code that started it. Nothing waited for it, errors it raised had nowhere to go, and cancelling the caller did not cancel it, much as goto broke the structure of functions before Dijkstra's critique.
What happened, and who
On 25 April 2018 Nathaniel J. Smith, the author of the Python library Trio, published Notes on structured concurrency, or: Go statement considered harmful. He argued that the go statement breaks control flow like goto and proposed the nursery: a block that cannot exit until every task started in it has finished, and that passes errors and cancellation along.
Why it mattered
It gave the idea a name and an argument that spread widely. Scoped tasks now appear as coroutine scopes in Kotlin, task groups in Swift and structured-concurrency scopes in Java, and code that must clean up after cancelled work is easier to reason about.
A kernel written in Go that runs in user space and answers a container's system calls, so the container never talks to the host kernel.
The story
Before
Containers share the host's kernel, so an application can reach hundreds of system calls and a single kernel bug is a possible container escape. Virtual machines isolate better but are heavier to start and run.
What happened, and who
Google open-sourced gVisor on 2 May 2018 and described it on its Cloud blog. It has an application kernel called the Sentry, written in Go and running as an unprivileged user-space process, which intercepts the container's system calls and implements them itself, and a helper, the Gofer, that mediates file access. It plugs into Docker and Kubernetes through the runsc runtime and at launch implemented over 200 system calls.
Why it mattered
It made a middle path between a plain container and a virtual machine practical: much of the kernel's attack surface is replaced by a small, memory-safe reimplementation. It is used to sandbox untrusted code in Google's cloud products such as Cloud Run and GKE Sandbox, at the price of some compatibility and speed.
OpenBSD switched off simultaneous multithreading by default, judging its side-channel risk not worth the speed-up.
The story
Before
Simultaneous multithreading, which Intel sells as Hyper-Threading, lets one core run two threads at once and was on by default everywhere. The two threads share the core's caches and TLBs, which was accepted as a performance trade-off but not treated as a security boundary.
What happened, and who
On 19 June 2018 the OpenBSD developer Mark Kettenis committed a hw.smt control and disabled SMT by default on Intel processors on amd64. His commit message noted that SMT threads typically share TLBs and L1 caches, which makes cache-timing attacks easier and might allow Spectre-class attacks, and that the benefit depends on the workload.
Why it mattered
It was an early and blunt example of a system dropping a hardware feature for security after Spectre and Meltdown. SMT has since become a setting security-conscious operators weigh, and other systems, including Linux with its nosmt option, later added similar controls.
A routine update to the C library’s sorting rules silently left text indexes in the wrong order after operating-system upgrades.
The story
Before
PostgreSQL asks the operating system’s C library, glibc, how strings sort, and stores B-tree indexes in that order on disk. It recorded no version of those rules, so nothing could notice if the library later changed them under an existing database.
What happened, and who
glibc 2.28, released on 1 August 2018, carried a major update to its locale data, bringing it in line with current international standards. Distributions that shipped it, such as Debian 10 “buster”, therefore sorted some strings differently from their predecessors, and Debian’s PostgreSQL maintainer, Christoph Berg, reported the consequence for indexes on the debian-glibc list in March 2019.
Why it mattered
An index built under the old rules is no longer sorted by the new ones, so lookups can miss rows that are present and uniqueness checks can pass duplicates. After a major operating-system upgrade every index on a text column, and any partition bound depending on collation, has to be rebuilt. The episode made the risk widely known, and it is why teams now plan a reindex into every operating-system upgrade and why collations provided by PostgreSQL itself, such as ICU (which was not affected), are considered for stable ordering.
A rewrite of the protocol behind HTTPS that removes old cryptography and finishes the handshake in one round trip.
The story
Before
TLS 1.2 (RFC 5246, 2008) needed two round trips before encrypted data could flow. It also still allowed static RSA key exchange, which gives no forward secrecy, and a long list of legacy ciphers and options that attacks such as BEAST and CRIME had exploited.
What happened, and who
Eric Rescorla of Mozilla was the editor of RFC 8446, which the IETF published on 10 August 2018 after four years of work in its TLS working group. It removes the weak options, encrypts most of the handshake, makes forward-secret key exchange mandatory, and lets a client send data after one round trip, or none when resuming a session.
Why it mattered
Every modern browser and most servers now negotiate TLS 1.3, so a new HTTPS connection costs one round trip fewer and cannot be downgraded to the removed algorithms. QUIC, the transport under HTTP/3, builds its handshake directly on TLS 1.3.
A transformer pretrained by guessing hidden words from both sides, then briefly fine-tuned for almost any language task.
The story
Before
OpenAI’s GPT-1 read text left to right only, so each word saw no context to its right. ELMo (2018) combined a left-to-right and a right-to-left model, but only shallowly, joining them at the top. Most language tasks still needed their own purpose-built architecture.
What happened, and who
Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova of Google posted “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” to arXiv on 11 October 2018. It hides about 15% of the input tokens and trains a transformer encoder to predict them from context on both sides, which avoids each word indirectly seeing itself.
Why it mattered
BERT set new records on eleven language tasks, and one pretrained model plus a small task-specific layer replaced many bespoke systems. Pretrain-then-fine-tune became the default recipe, and BERT-style encoders still power search ranking and text classification.
Forty-three seconds of lost connectivity triggered an automatic failover that took a day to unpick.
The story
Before
GitHub’s MySQL clusters had their primaries on the US East Coast, with replicas in the US West Coast data centre. Orchestrator, an open-source failover tool that agrees on a leader by Raft consensus, watched them and could promote a replica when it lost sight of a primary.
What happened, and who
At 22:52 UTC on 21 October 2018, maintenance replacing failing optical equipment cut connectivity between the East Coast network hub and the primary data centre for 43 seconds. Orchestrator, by then unable to see the East Coast primaries, promoted primaries on the West Coast, but the East Coast clusters had accepted writes the West Coast never received. The two sides had diverged, and GitHub’s service was degraded for 24 hours 11 minutes.
Why it mattered
GitHub chose to preserve every user’s data rather than fail back quickly, so it spent the day reconciling the two histories. The post-incident analysis is now a standard example of why automatic failover needs a fencing story and why cross-region failover is a consistency decision as much as an availability one.
A small Rust virtual-machine monitor that boots isolated microVMs in about 125 milliseconds with very little memory overhead.
The story
Before
Lambda’s first isolation design ran containers per function inside EC2 instances dedicated to each customer. Containers share the host kernel, and ordinary virtual machines, driven by QEMU, boot slowly and cost a lot of memory. Neither gave both strong isolation and the density serverless economics need.
What happened, and who
On 26 November 2018, at re:Invent, Amazon Web Services announced and open-sourced Firecracker under the Apache 2.0 licence. It started from crosvm, the virtual-machine monitor of Chromium OS (Google’s ChromeOS), was written in Rust, and used Linux’s KVM with a minimal device model. AWS said it already ran Lambda and Fargate, and gave under 5 MiB of memory per microVM and start-up in as little as 125 ms.
Why it mattered
Hardware-virtualised isolation became cheap enough to give each function or task its own microVM. Beginning in 2018 Lambda moved its customers to Firecracker on EC2 bare-metal instances, and Fargate runs on it too. Firecracker is the reference design for a minimal VMM, and it underlies several other sandboxing platforms.
One shard of a transactional-email service ran out of transaction IDs and refused all writes for more than a day.
The story
Before
PostgreSQL numbers transactions with 32-bit IDs and must periodically “freeze” old rows, through vacuum, so the counter can wrap safely. If vacuum falls too far behind, the database stops accepting writes rather than risk corrupting visibility.
What happened, and who
Mandrill, Mailchimp’s transactional-email service, kept its data in several PostgreSQL shards. The busiest one hit the limit at 05:35 UTC on 4 February 2019 after a write spike outpaced autovacuum, and the shard shut down to protect itself. A full vacuum would have taken days, so engineers instead truncated two large tables, freeing enough IDs to resume; sending from the backlog began at 21:36 UTC on 5 February.
Why it mattered
It is a well-known public case of transaction-ID wraparound in production, alongside Sentry’s in 2015, and the write-up is the standard reference for why autovacuum lag has to be monitored as a distance to wraparound, not as a maintenance detail.
A 1.5-billion-parameter language model that OpenAI declined to release in full at first, citing possible misuse.
The story
Before
GPT-1 (2018) had shown that pretraining a transformer on unlabelled text helps downstream tasks, but each task still needed fine-tuning. No model had yet shown that a single one could attempt many tasks from a prompt alone.
What happened, and who
OpenAI announced GPT-2 on 14 February 2019 in the post “Better Language Models and Their Implications”, with the paper by Alec Radford and colleagues. It was trained on WebText, about 8 million web pages linked from Reddit, and the largest version had 1.5 billion parameters. OpenAI released only smaller models first and published the full weights in November 2019.
Why it mattered
GPT-2 showed that a language model could attempt translation, summarisation and question answering with no fine-tuning, and it started the public argument over whether to withhold generative models. Its byte-level byte-pair encoding and layer-norm-before-sublayer design were adopted by many later models.
An algorithm that multiplies two n-bit integers in O(n log n) steps, the bound believed to be the best possible.
The story
Before
The schoolbook method takes n² steps. Schönhage and Strassen (1971) reached n log n log log n with Fourier transforms over modular rings, and Fürer (2007) lowered the extra factor to one that grows only like an iterated logarithm. The best unconditional bound still carried such a factor.
What happened, and who
David Harvey of the University of New South Wales and Joris van der Hoeven of the French research agency CNRS posted “Integer multiplication in time O(n log n)” in March 2019; it was later published in the Annals of Mathematics (2021). It multiplies through multidimensional Fourier transforms, and its constants are so large that it wins only for astronomically big inputs.
Why it mattered
It settles the complexity as far as it can be settled: Schönhage and Strassen conjectured that n log n is optimal, and any lower bound would now have to match. It has no practical use itself, since real libraries such as GMP still use Karatsuba, Toom-Cook and FFT-based methods, but it closed a fifty-year search.
The Linux scheduler can now weigh the energy cost of running a task on each core, not just how busy the cores are.
The story
Before
Mainline Linux’s scheduler, CFS, spread tasks by load. On phones with big.LITTLE chips, which pair fast power-hungry cores with slow frugal ones, that put light work on big cores and wasted battery. Android kept its own energy-aware patches out of tree.
What happened, and who
Energy-aware scheduling, developed largely by Arm engineers and based on the version in Android’s common kernel, was merged in Linux 5.0, released on 3 March 2019. It gives the scheduler an energy model of each CPU’s power at each speed, and lets a waking task go to the core where it costs least energy without hurting performance.
Why it mattered
Mainline Linux gained a phone-driven design that Android systems no longer had to carry as a patch set. It also pushed the scheduler to treat power as a resource alongside throughput, on asymmetric CPUs that are now common in phones and laptops.
Five neural-network agents trained only by playing themselves beat the reigning Dota 2 world champions two games to none.
The story
Before
Deep reinforcement learning had mastered Atari games and Go, which are perfectly observed or have few players. Dota 2 is a five-against-five game with hidden information, hundreds of decisions per minute, and it was not clear that existing methods could cope.
What happened, and who
On 13 April 2019 in San Francisco, OpenAI Five defeated OG, the winners of The International 2018, 2–0 in a best-of-three exhibition, under a restriction to 17 heroes. The agents were trained with Proximal Policy Optimisation, a policy-gradient method, entirely by self-play, on a large fleet of GPUs and CPU cores that produced about 180 years of gameplay per day.
Why it mattered
The result was evidence that a simple algorithm such as PPO can scale with compute to long-horizon, multi-agent problems, and it encouraged large-scale self-play in later systems. PPO itself became a workhorse, and is the algorithm behind much reinforcement learning from human feedback.
Two rings of memory shared between an application and the kernel let programs queue many I/O operations with almost no system calls.
The story
Before
Linux’s native asynchronous I/O (AIO), added around kernel 2.5, worked only with files opened with O_DIRECT, had no support for sockets, and still needed a system call to submit work and another to collect results. Other kernel-bypass approaches gave up the kernel’s file systems and network stack.
What happened, and who
Jens Axboe, the Linux block-layer maintainer, wrote io_uring, and it was merged into Linux 5.1, released on 5 May 2019. The application writes requests to a submission queue and reads results from a completion queue, both in memory shared with the kernel, and can optionally have the kernel poll the queue so no submission call is needed. The liburing library wraps the details.
Why it mattered
io_uring is now the interface for high-performance asynchronous file and network I/O on Linux, used by databases, storage systems and runtimes. Its wide reach also made it a large attack surface: Google reported in 2023 that most submissions to its kernel bug bounty exploited it, and it disabled io_uring on Android and ChromeOS.
Google’s single global service that answers “may this user do this to that object?” for its biggest products, consistently and in milliseconds.
The story
Before
Each Google product kept its own access-control lists with its own semantics. Sharing a document from one product to another was awkward, and none of them could promise that a permission change would be seen before the content it protected was next read.
What happened, and who
Ruoming Pang, Ramon Caceres, Mike Burrows and colleagues at Google described Zanzibar in “Zanzibar: Google’s Consistent, Global Authorization System” at the USENIX Annual Technical Conference in Renton, Washington, in July 2019. It stores trillions of ACLs as relation tuples, and its “zookies”, opaque consistency tokens, let a client ask for a check no older than a given change.
Why it mattered
The paper reports 95th-percentile latency under 10 ms and availability above 99.999% over three years, at millions of requests a second. Its tuple model inspired open-source systems such as OpenFGA and SpiceDB, and relationship-based access control became a mainstream way to build authorisation.
One firewall rule with a backtracking regular expression used all the CPU on Cloudflare’s edge servers worldwide for 27 minutes.
The story
Before
Cloudflare’s web application firewall (WAF) runs customer-facing rules on every edge server, and its rules were deployed to the whole network at once. Its regex engine backtracks, so some patterns take time exponential in the input, and a protection against runaway CPU use had been removed in an earlier refactoring.
What happened, and who
At 13:42 UTC on 2 July 2019 Cloudflare deployed a new managed rule intended to detect cross-site scripting. Its pattern contained .*(?:.*=.*), which backtracks catastrophically, so CPU on HTTP servers worldwide rose to nearly 100% and customers got 502 errors until a global kill switch disabled the WAF; it was re-enabled at 14:52 UTC.
Why it mattered
Cloudflare restored the CPU limiter, moved the WAF to the RE2 or Rust regex engines, whose run time cannot blow up, and required staged rollouts for rule changes, keeping a global kill switch. It is the standard example of a regex denial of service in production and of why configuration changes deserve the same staging as code.
A rewrite that gave every worker thread its own run queue and made spawning a task up to twelve times faster.
The story
Before
Tokio, the most widely used asynchronous runtime for Rust, had a work-stealing scheduler in its 0.1 series (March 2018) built on the general-purpose Chase-Lev deque from the crossbeam library. That queue was not tuned for many small independent tasks, and the scheduler had accumulated inefficiency.
What happened, and who
Carl Lerche, Tokio’s maintainer, described the rewrite in a post on 13 October 2019, ahead of Tokio 0.2. Each processor owns a fixed-size queue that other processors steal from when idle, and only about half of them may be searching for work at once. On micro-benchmarks task spawning was up to twelve times faster, and the post reports 34% more throughput for the Hyper HTTP library.
Why it mattered
That design carried through to Tokio 1.0 and is the multi-threaded runtime under much of the Rust server ecosystem. Its combination of per-thread queues, throttled stealing and testing with the loom permutation checker is a widely copied reference for async runtimes.
An agent that learns the rules of a game by itself, then plans with them, and matches AlphaZero without being told the rules.
The story
Before
AlphaZero (2017) played Go, chess and shogi at superhuman level by searching a tree, but it needed a perfect simulator of the game to know what each move does. Earlier model-based methods that learned a model tried to reproduce the next observation, which is hard and mostly wasteful.
What happened, and who
Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, David Silver and colleagues at DeepMind posted “Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model” on 19 November 2019. MuZero learns a model that predicts only what planning needs, the reward, the value and the policy, and searches with it.
Why it mattered
MuZero reached AlphaZero’s strength in board games and set a record on Atari without knowing the rules, which removed the perfect-simulator requirement from tree-search reinforcement learning. Its idea of learning a value-equivalent model, not a reconstructive one, shaped later model-based agents.
Container CPU limits were throttling services that used far less CPU than their quota allowed.
The story
Before
The Linux scheduler enforces a cgroup’s CPU limit by handing out its quota in 5 ms slices to each CPU’s run queue, and each slice expired at the end of the period. A change in Linux 4.18 (commit 512ac999d275) made expiry stricter to fix a clock-skew problem, so a slice left partly unused on one CPU was simply lost.
What happened, and who
Dave Chiluk and colleagues at Indeed, the job-search company, saw high response times on multithreaded services that showed low average CPU use and heavy throttling, and wrote a reproducer, Fibtest, that mimics threads that do a little work and then sleep. Chiluk wrote the fix, which removes slice expiry altogether. It was merged in Linux 5.4, released in November 2019, and he described the investigation on Indeed’s engineering blog on 5 December 2019.
Why it mattered
Unused slices now persist, so a service is held to its quota on average, with at most about 1 ms of overshoot per CPU, and in the reproducer performance improved almost thirty-fold. Kubernetes CPU limits on kernels since 5.4 behave much better for many-threaded services, and later kernels added a separate burst allowance, cpu.cfs_burst_us, in Linux 5.14.
Large language models, and the systems built to train and serve them.
PaperAI
Neural Scaling Laws for Language Models
Language-model test loss falls as a smooth power law in model size, data and compute, over many orders of magnitude.
The story
Before
How big to make a model and how long to train it was decided by trial, experience and budget. There was no quantitative rule to predict what a model ten times larger, or given ten times more data, would gain.
What happened, and who
Jared Kaplan, Sam McCandlish and colleagues at OpenAI and Johns Hopkins University posted “Scaling Laws for Neural Language Models” on 23 January 2020. Training transformer language models across a wide range of sizes, they found loss follows power laws in parameters, dataset size and compute, and that shape matters little.
Why it mattered
The curves let labs forecast a large run’s loss from small ones, and they shaped the push to train ever larger models. Later work (Hoffmann et al., 2022) revised how to split a compute budget between parameters and data, but predicting from small runs remains how frontier training is planned.
A configuration store for Amazon’s block-storage service that runs as millions of small databases, so a failure touches only a few volumes.
The story
Before
On 21 April 2011 a network change made 13% of the volumes in one Availability Zone of Amazon EBS, Amazon’s network block-storage service, unavailable. The volumes’ replication configuration lived in a control-plane database shared with API traffic, which made it a single failure domain.
What happened, and who
Marc Brooker, Tao Chen and Fan Ping of AWS described Physalia in “Millions of Tiny Databases” at NSDI 2020 in February 2020. The team began building it in 2013. Each small database, a cell, is a Paxos group of seven nodes holding one partition of the configuration data, placed near the clients that use it.
Why it mattered
Physalia is a canonical account of cell-based architecture: keep strong consistency but make the unit that can fail small, so an outage affects few customers. It is the store that EBS uses for volume configuration, and the pattern is now applied across AWS control planes.
The published design rationale for Firecracker: why so little hardware is emulated, and how it lets Lambda pack functions densely.
The story
Before
Lambda’s first isolation design used containers per function inside EC2 instances dedicated to each customer. Resources were hard allocations fixed when a VM launched, so memory and CPU could not be oversubscribed across customers.
What happened, and who
Alexandru Agache, Marc Brooker, Andreea Florescu, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka and Diana-Maria Popa of AWS presented “Firecracker: Lightweight Virtualization for Serverless Applications” at NSDI 2020 in Santa Clara (25–27 February 2020). It explains keeping KVM but replacing QEMU with a small monitor and device model, and reports moving Lambda customers to it from 2018.
Why it mattered
The paper is the public reference for microVM isolation with soft allocation: it reports testing oversubscription ratios above 20 times and running production at up to 10 times. It is the source for the boot-time and overhead figures that other sandbox runtimes are compared with.
A flaw in the checker that vets eBPF programs let an ordinary user read and write kernel memory.
The story
Before
eBPF lets users load small programs into the Linux kernel, and a verifier statically checks every memory access before the program runs. The kernel’s security here rests entirely on the verifier’s tracking of what values each register can hold being sound.
What happened, and who
Manfred Paul of RedRocket CTF used the bug, tracked as CVE-2020-8835, at Pwn2Own Vancouver in March 2020, to go from an unprivileged user to root and won $30,000. The verifier’s bounds tracking for 32-bit operations assumed that if the minimum and maximum of a range both ended in the same low bits, so did every value between them. Fixes went into stable kernels on 30 March 2020, when the CVE was published.
Why it mattered
The bug showed that the verifier is a security boundary: a soundness error in its arithmetic reasoning gives kernel read and write access. It is one of several verifier flaws that led many systems to restrict or disable unprivileged eBPF.
A language model that first fetches relevant documents with a trained retriever, then writes its answer conditioned on them.
The story
Before
Language models such as GPT-2 and BART kept their knowledge in their weights. It could not be updated without retraining, could not be traced to a source, and the models sometimes produced fluent but false statements.
What happened, and who
Patrick Lewis and colleagues at Facebook AI Research, University College London and New York University posted “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” on 22 May 2020. It pairs a dense retriever over Wikipedia with a sequence-to-sequence generator and trains them together, and it set the state of the art on several open-domain question-answering benchmarks.
Why it mattered
The paper named the technique. Retrieval-augmented generation is now the standard way to give a model private or current data without retraining it, and it is the design behind many document-search and chat-with-your-data products.
A 175-billion-parameter model that performs new tasks from a few examples in its prompt, with no weight updates.
The story
Before
Applying a pretrained model to a task meant fine-tuning a copy on thousands of labelled examples, giving a specialised model for each task. Humans, by contrast, can pick up a task from a sentence of instruction or a couple of examples.
What happened, and who
Tom Brown and colleagues at OpenAI posted “Language Models are Few-Shot Learners” on 28 May 2020. GPT-3, a 175-billion-parameter transformer trained on hundreds of billions of tokens, performed translation, question answering and arithmetic given only examples in its prompt, and its few-shot quality improved with model size.
Why it mattered
Prompting became an interface: the same model does many tasks by changing its input. GPT-3 was offered through an API from mid-2020, and its scale and behaviour set off the wave of large language models that followed.
Learn to remove a little noise from an image at a time, and generate pictures by repeating that from pure noise.
The story
Before
Sohl-Dickstein and colleagues had proposed diffusion models in 2015, learning to reverse a gradual noising process, but their samples lagged behind generative adversarial networks (GANs), which dominated image generation.
What happened, and who
Jonathan Ho, Ajay Jain and Pieter Abbeel of UC Berkeley posted “Denoising Diffusion Probabilistic Models” on 19 June 2020. They trained a network to predict the noise added at each step, with a simple loss, and it produced high-quality samples and reached a competitive score on CIFAR-10 image generation.
Why it mattered
Diffusion models proved easier to train than GANs and scaled well, and became the basis of image generators such as DALL·E 2, Stable Diffusion and Imagen, and of audio and video generators.
A table format that keeps a transaction log next to the data files in cloud storage to give ACID transactions on top of it.
The story
Before
Cloud object stores such as Amazon S3 hold immutable objects and could not update several of them atomically, and listing objects was not guaranteed to be consistent then. Data lakes made of thousands of files therefore exposed half-finished writes and were slow to list.
What happened, and who
Michael Armbrust and colleagues at Databricks published “Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores” in the Proceedings of the VLDB Endowment, volume 13, in August 2020. A table is a set of Parquet files plus an ordered log of JSON actions, and a commit is a single atomic object write, while checkpoints keep the log short.
Why it mattered
The log-based design gives snapshots, time travel and concurrent writers on plain object storage. Apache Iceberg and Apache Hudi took the same approach with different metadata layouts, and open table formats are now the base of most lakehouse systems.
Cut an image into patches, treat them like words and feed them to a plain transformer; with enough data it matches convolutional networks.
The story
Before
Since AlexNet (2012) computer vision had been built on convolutional networks, whose structure assumes that nearby pixels matter and that a pattern means the same wherever it appears. Earlier attempts to use attention in vision mixed it into convolutional designs.
What happened, and who
Alexey Dosovitskiy and colleagues at Google posted “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale” on 22 October 2020. The Vision Transformer splits an image into 16×16-pixel patches, embeds each as a token, and runs a standard transformer encoder on them.
Why it mattered
With pretraining on large datasets, ViT matched or beat the best convolutional networks at lower training cost. It showed that one architecture could serve both language and vision, and it underlies most current vision and multimodal models.
Fedora stopped creating a swap partition by default and used a compressed block device in RAM instead.
The story
Before
Linux installers created swap as a partition or file on disk. When memory ran out, pages were written to the disk, which is thousands of times slower than RAM, so a system in trouble slowed to a crawl.
What happened, and who
Fedora 33, released on 27 October 2020, made swap on zram the default for new installations. Zram, first written by Nitin Gupta as “compcache” and in mainline Linux since 3.14 (2014), compresses pages in memory. Fedora sizes the device at half of RAM, capped at 4 GiB, using the zram-generator package.
Why it mattered
Under memory pressure a machine compresses cold pages rather than write them to disk, which is faster and spares flash storage. Android and ChromeOS also use zram for swap.
An Arm-based chip for Macs whose CPU cores can decode eight instructions per cycle, about twice what typical x86 cores managed.
The story
Before
Macs had used Intel x86 processors since 2006. x86’s variable-length instructions are hard to decode in parallel, and contemporary x86 cores decoded around four instructions a cycle.
What happened, and who
Apple announced the M1 on 10 November 2020. It is a system-on-a-chip with 16 billion transistors, four high-performance and four high-efficiency CPU cores, an up-to-eight-core GPU and a 16-core Neural Engine, and its performance cores decode eight instructions per cycle, helped by Arm’s fixed-width instruction set.
Why it mattered
The M1 showed that an Arm design could beat contemporary x86 laptops in speed and battery life, and Apple moved the whole Mac line to its own chips. It reopened the argument over how much the instruction set matters, and put wide-decode Arm cores in competitors’ roadmaps.
Lambda functions can be packaged as container images up to 10 GB, about forty times the old zip limit.
The story
Before
Lambda functions were uploaded as zip archives limited to 250 MiB unzipped, which was too small for machine-learning libraries or large dependencies, and packaging did not match the container workflows many teams already used.
What happened, and who
On 1 December 2020, at re:Invent, AWS announced container image support for Lambda: images up to 10 GB built from AWS base images or any image implementing the Lambda runtime API, with a Runtime Interface Emulator for local testing. It was announced together with 1 ms billing and up to 10 GB of memory.
Why it mattered
A 10 GB image cannot be downloaded in full at every cold start. Lambda therefore loads images lazily in blocks, deduplicates them across customers and caches them in tiers, and this machinery is what makes large images start quickly.
After fourteen years, S3 guarantees that a read right after a write, overwrite or delete sees the change, at no extra cost.
The story
Before
S3 launched in 2006 and gave read-after-write consistency only for new objects. Overwrites and deletes were eventually consistent, so a read could return old data or a list could omit a new key. Big-data users added layers such as S3Guard for Hadoop or EMRFS Consistent View to work around it.
What happened, and who
On 1 December 2020 Amazon announced that S3 was strongly consistent for all GET, PUT and LIST operations, as well as for tags, ACLs and metadata, at no additional charge and without performance impact. It did this by making the metadata layer itself strongly consistent, with a cache-coherence protocol.
Why it mattered
Applications and data lakes can rely on what they write being what they read, and the workarounds became unnecessary. Conditional writes were added in 2024, and open table formats use object-store atomic commits built on this.
A mixture-of-experts transformer that sends each token to just one expert, scaling to a trillion parameters.
The story
Before
Mixture-of-experts models, introduced for language by Shazeer and colleagues in 2017, route each token to a few of many expert networks, so parameters grow without more compute per token. It was thought that routing to at least two experts was needed for stable training.
What happened, and who
William Fedus, Barret Zoph and Noam Shazeer of Google posted “Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity” on 11 January 2021. Its router picks one expert per token, with a load-balancing loss, and it trained models up to 1.6 trillion parameters, speeding pretraining by up to seven times over a dense T5 baseline.
Why it mattered
Top-1 routing made sparse models cheaper and simpler to run, and it showed that parameter count and compute per token can be separated. Mixture-of-experts is now a common design for large open and closed language models.
Train an image encoder and a text encoder so that matching pictures and captions land close together; it classifies images it was never trained on.
The story
Before
Vision models learned from datasets with fixed labels, such as the 1,000 ImageNet classes, and needed new labelled data to recognise anything else. Labelling was slow and expensive.
What happened, and who
Alec Radford and colleagues at OpenAI posted “Learning Transferable Visual Models From Natural Language Supervision” on 26 February 2021. CLIP was trained on 400 million image-text pairs from the internet, using a contrastive loss that pulls each image toward its own caption. At test time, class names are written as sentences, and the closest one is the prediction, with no training on that dataset.
Why it mattered
CLIP provided a shared space for images and text. It steers text-to-image systems, from DALL·E to Stable Diffusion, is used for image search, and is the vision component of many multimodal models.
An open-source PostgreSQL extension that stores embedding vectors and finds the nearest ones to a query vector.
The story
Before
Similarity search over embeddings was done in dedicated libraries and databases such as Faiss, which kept vectors outside the application’s main database, alongside its data and its transactions.
What happened, and who
Andrew Kane tagged version 0.1.0 of pgvector, an open-source extension for PostgreSQL, on 20 April 2021. It adds a vector column type and distance operators, so a query can order rows by closeness to a vector.
Why it mattered
Vectors could now live in the same table as the data they describe, with the same backups and transactions. Later releases added approximate indexes, including HNSW, and pgvector became a common way to build retrieval for language-model applications.
A transport protocol over UDP with encryption built in, which sets up connections faster and keeps streams from blocking each other.
The story
Before
TCP needs its own handshake and then a separate TLS handshake. A lost packet stalls every stream sharing the connection, and because TCP’s headers are visible to routers and middleboxes, changes to it are blocked by boxes that expect the old behaviour, so it could hardly evolve.
What happened, and who
Google began QUIC around 2012, and the IETF QUIC working group standardised it as RFC 9000, edited by Jana Iyengar of Fastly and Martin Thomson of Mozilla, published on 27 May 2021. It runs over UDP, embeds TLS 1.3, encrypts most header fields, and multiplexes independent streams.
Why it mattered
QUIC is the transport for HTTP/3 (RFC 9114, 2022), which a large share of web traffic now uses. Connections survive a change of network, and because the headers are encrypted the protocol can be updated without waiting for middleboxes.
A test message meant for engineers was sent to real subscribers.
The story
Before
Email systems are commonly tested by sending messages through the same sending service that delivers real mail, so one wrong recipient list or one wrong environment setting separates a test from a mass mailing.
What happened, and who
On the evening of 17 June 2021, HBO Max subscribers received an email with the subject “Integration Test Email #1”, whose only content was “This template is used by integration tests only.” HBO Max said on Twitter that it had mistakenly sent an empty test email to a portion of its mailing list, joking that an intern was to blame.
Why it mattered
Nothing was lost, but it is a widely told case for keeping test and production sending paths separate, with allow-lists, sandbox recipients and a check that a bulk send cannot target a real list from a test.
Freeze a pretrained model and train only a small low-rank update to each weight matrix.
The story
Before
Fine-tuning a large model changes all its weights, so each task needs a full copy: 175 billion parameters for GPT-3, and optimiser state several times that in memory. Adapter layers added inference latency.
What happened, and who
Edward Hu, Yelong Shen and colleagues at Microsoft posted “LoRA: Low-Rank Adaptation of Large Language Models” on 17 June 2021. LoRA learns the change to a weight matrix as the product of two thin matrices of rank r, so it trains about 10,000 times fewer parameters for GPT-3 175B, and the update can be merged into the weights, adding no latency.
Why it mattered
Fine-tuning a large model became possible on modest hardware, and many task-specific adapters can share one base model. QLoRA (2023) combined it with 4-bit quantisation, and LoRA is now the usual way to fine-tune open models.
Adding SAVEPOINTs looked harmless, but past a threshold they stalled replicas for minutes at a time.
The story
Before
In PostgreSQL a SAVEPOINT starts a subtransaction, which has its own transaction ID. Application frameworks use them freely, on the assumption that they cost little.
What happened, and who
From June 2021, GitLab.com had unexplained database stalls that lasted minutes and produced 500 errors, and on 29 September 2021 GitLab published “Why we spent the last month eliminating PostgreSQL subtransactions”. It found that queries waited on the subtransaction cache, a small in-memory cache holding about 65,000 entries, which GitLab measured filling in about 18 seconds on a busy system; past that point replicas fall off a performance cliff.
Why it mattered
GitLab removed every SAVEPOINT from its code and the stalls stopped. The write-up is a reference for the hidden cost of subtransactions.
Amazon S3's Rust storage node was checked against small executable models instead of full formal proofs, and the method found real bugs.
The story
Before
ShardStore is the new storage node that keeps S3's data on disk, and its code was under constant feature development. Hand-written tests missed the rare crash and concurrency interleavings that lose data, and full formal verification was too slow to keep up with the pace of change.
What happened, and who
Amazon engineers, including James Bornholt, Rajeev Joshi and Vytautas Astrauskas, described their approach in “Using Lightweight Formal Methods to Validate a Key-Value Storage Node in Amazon S3”, presented at the ACM Symposium on Operating Systems Principles (SOSP) in October 2021. They wrote executable reference models of ShardStore's behaviour and checked the Rust code against them with property-based testing and model checking of concurrent runs, so engineers could run the checks as ordinary tests. At the time the node was being rolled out gradually and held hundreds of petabytes of customer data.
Why it mattered
It is an industrial case showing that formal-style checking can live inside a fast-moving production system, because the models are ordinary code that engineers keep up to date. It also showed that crash consistency and concurrency, the hardest properties to test by hand, can be covered by generating and exploring many executions.
Generate machine code by copying precompiled templates and filling in their gaps, which makes a fast JIT compiler easy to build.
The story
Before
A just-in-time compiler that produces good code is hard to write, and a baseline JIT that emits machine code by hand needs an assembler for each instruction and each architecture, which is laborious to maintain. Interpreters are easier but slower.
What happened, and who
Haoran Xu and Fredrik Kjolstad of Stanford published “Copy-and-Patch Compilation” in the Proceedings of the ACM on Programming Languages (OOPSLA 2021) on 15 October 2021. Templates for each operation are compiled ahead of time with an ordinary compiler, and the JIT copies them into memory and patches in constants and addresses.
Why it mattered
The technique compiles far faster than a conventional optimising compiler at a modest cost in code quality. The JIT compiler added to CPython 3.13, experimental at first, is built on it.
A state space layer, initialised with a special matrix, models sequences tens of thousands of steps long with cost that grows only linearly.
The story
Before
Transformers compare every position with every other, so their cost grows with the square of the sequence length, and recurrent networks tend to forget distant inputs. Earlier state space models could in principle remember a long past but were too slow and numerically unstable to train on long sequences.
What happened, and who
Albert Gu, Karan Goel and Christopher Ré at Stanford posted “Efficiently Modeling Long Sequences with Structured State Spaces” to arXiv on 31 October 2021 (published at ICLR 2022). Their Structured State Space (S4) layer keeps the HiPPO initialisation, Gu's earlier matrix designed to remember the history of an input, and restructures it so it can be computed quickly as a convolution. It was the first model to solve the Path-X task of the Long Range Arena benchmark.
Why it mattered
S4 made state space models a serious alternative to attention for long sequences. It is the direct ancestor of Mamba, and of the hybrid models that now mix state space layers with attention.
Running diffusion in an autoencoder's compressed latent space cut the cost enough for a single consumer GPU.
The story
Before
Diffusion models made the best images of their day, but they ran in pixel space, so training took hundreds of GPU-days and every sampling step processed the full image. Only a few labs could afford to train or run them.
What happened, and who
Robin Rombach, Andreas Blattmann, Björn Ommer and colleagues at LMU Munich and Runway posted “High-Resolution Image Synthesis with Latent Diffusion Models” to arXiv on 20 December 2021. They first compress images with an autoencoder, then run diffusion on the much smaller latent representation. Stability AI, working with the same researchers, released a model built on it, Stable Diffusion, with public weights on 22 August 2022.
Why it mattered
Because the weights were open and the model fitted on one consumer graphics card, anyone could run, fine-tune and build on it. That set off the ecosystem of open image-generation tools, and latent-space diffusion is now the standard design for image and video generators.
Showing worked reasoning steps in the prompt lifts a large model's accuracy on arithmetic and reasoning problems.
The story
Before
Few-shot prompting, popularised by GPT-3, gave the model examples of question and answer only. On multi-step problems such as grade-school arithmetic word problems, making the model bigger helped little when it had to jump straight to the answer.
What happened, and who
Jason Wei, Denny Zhou and colleagues at Google Brain posted “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” to arXiv on 28 January 2022. They put step-by-step worked solutions into the few-shot examples, and large models then wrote out their own steps before answering. With eight such examples, the 540-billion-parameter PaLM model set a new best result on the GSM8K maths benchmark.
Why it mattered
The paper showed that a model's reasoning depends on how much computation it is allowed to spend in its output. That idea leads to zero-shot “think step by step” prompts, self-consistency voting and, later, reasoning models trained to produce long chains of thought.
A 1.3-billion-parameter model tuned with human feedback was preferred by people to the 175-billion-parameter base GPT-3.
The story
Before
Base language models such as GPT-3 were trained only to continue text, so they often ignored what a user asked and produced toxic or untruthful output. Prompt tricks could coax them, but the behaviour was not what the model was trained for.
What happened, and who
Long Ouyang and colleagues at OpenAI posted “Training language models to follow instructions with human feedback” to arXiv on 4 March 2022. They fine-tuned GPT-3 on human-written demonstrations, trained a reward model from people's rankings of outputs, and then optimised the model against that reward with the PPO reinforcement-learning algorithm, keeping it close to the original with a KL penalty.
Why it mattered
This three-stage recipe (supervised fine-tuning, reward model, RL) became the standard way to turn a base model into an assistant. OpenAI described ChatGPT as a sibling of InstructGPT, and later alignment methods are defined by how they simplify or replace parts of this pipeline.
For a fixed compute budget, model size and training tokens should grow together; a 70B model on 1.4T tokens beat the 280B Gopher.
The story
Before
The 2020 scaling laws of Kaplan and colleagues at OpenAI suggested spending most extra compute on more parameters. Large models such as GPT-3 (175 billion parameters) and Gopher (280 billion) were accordingly trained on only about 300 billion tokens, so they were large but under-trained.
What happened, and who
Jordan Hoffmann and colleagues at DeepMind posted “Training Compute-Optimal Large Language Models” to arXiv on 29 March 2022. Training over 400 models of different sizes, they found that parameters and tokens should scale in equal proportion. To test it they trained Chinchilla, with 70 billion parameters on 1.4 trillion tokens and the same compute as Gopher, and it outperformed the much larger model.
Why it mattered
After Chinchilla, labs trained smaller models on far more data, and the number of tokens became a headline figure for a model. Since a smaller model is cheaper to run for every request, later work went on to train well past the compute-optimal point to save inference cost.
A driver notices when a virtual machine is cloned or restored from a snapshot and forces the kernel's random-number generator to reseed.
The story
Before
A virtual machine restored from a snapshot resumes with the same random-number-generator state it had when the snapshot was taken. Every clone therefore produced the same “random” values, which can repeat keys and nonces in cryptographic protocols.
What happened, and who
Jason A. Donenfeld, the maintainer of the Linux kernel's random-number generator, wrote a driver for VM Generation ID, a 16-byte value that the hypervisor changes whenever a machine is cloned or restored. The driver mixes that value into the generator's entropy pool. It shipped in Linux 5.18, released on 22 May 2022, and works with QEMU, Hyper-V and VMware.
Why it mattered
Snapshot-based start-up is how serverless platforms launch microVMs quickly, and it is only safe if each copy gets fresh randomness. The driver closed the hazard for hypervisors that supply the ID, and it gave in-kernel users such as WireGuard a way to clear their keys after a fork.
Exact attention computed in tiles that stay in fast on-chip memory, never writing the full T × T score matrix out to slow memory.
The story
Before
Standard attention computed the full matrix of scores for every pair of positions and wrote it to the GPU's high-bandwidth memory (HBM) and read it back several times. On a GPU the time went into moving those bytes rather than into arithmetic, and the memory needed grew with the square of the sequence length.
What happened, and who
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra and Christopher Ré at Stanford and the University at Buffalo posted “FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness” to arXiv on 27 May 2022 (NeurIPS 2022). The kernel splits the inputs into tiles that fit in on-chip SRAM and combines partial softmax results as it goes, so the answer is exact but the large matrix is never stored.
Why it mattered
It made long-context training and inference practical, and it showed that a GPU algorithm should be designed around memory traffic rather than operation counts. Fused, tiled attention kernels in the major training and serving frameworks descend from it.
Some abilities look absent in small models and then appear abruptly once a model passes a scale threshold.
The story
Before
Scaling laws showed the training loss of language models falling smoothly and predictably as models grew. Nothing in them said whether a specific skill, such as multi-digit arithmetic, would also improve smoothly or switch on at some size.
What happened, and who
Jason Wei and colleagues, mostly at Google, posted “Emergent Abilities of Large Language Models” to arXiv on 15 June 2022 (published in Transactions on Machine Learning Research). They defined an emergent ability as one not present in smaller models but present in larger ones, and collected examples where accuracy stayed near chance until a model crossed a size, then rose sharply.
Why it mattered
The paper gave the field a name for that behaviour and made unpredictable jumps a central worry of scaling. It also started a debate: a 2023 study by Schaeffer and colleagues argued that many apparent jumps come from all-or-nothing evaluation metrics, not from the models.
Ten years of DynamoDB's evolution: admission control, partition splits, adaptive capacity and what the service learned from operating at scale.
The story
Before
DynamoDB's internals had been described only in blog posts and talks, and the 2007 Dynamo paper from Amazon described a different, internal system. Engineers had no authoritative account of how the managed service actually works.
What happened, and who
Fifteen Amazon engineers, led by Mostafa Elhemali, presented “Amazon DynamoDB: A Scalable, Predictably Performant, and Fully Managed NoSQL Database Service” at the USENIX Annual Technical Conference in Carlsbad, California, in July 2022. It covers the service since its 2012 launch, including how it keeps performance predictable under uneven load, and reports that on Prime Day 2021 it peaked at 89.2 million requests per second.
Why it mattered
It is the authoritative account of how the managed service differs from its namesake: throughput is provisioned and enforced by admission control, and partitions split and rebalance without the customer's involvement. Engineers designing a multi-tenant data store use it as a reference for fairness and predictable latency.
A plain chat window on top of an RLHF-tuned GPT-3.5 model, free to the public, turned language models into a consumer product.
The story
Before
The instruction-following models OpenAI had trained with human feedback, such as InstructGPT, were reachable only through an API meant for developers. Using them meant writing code or a playground prompt, and they had no memory of a conversation.
What happened, and who
OpenAI released ChatGPT on 30 November 2022 as a free research preview. It was fine-tuned from a model in the GPT-3.5 series using reinforcement learning from human feedback, with the same methods as InstructGPT and some differences in how the training conversations were collected.
Why it mattered
A chat box needed no technical skill, and it reached a mass audience within weeks. It set off the race among labs to ship assistants and made fine-tuning for conversation, safety and tool use a central engineering task.
Kernel code can now be written in Rust, a language that rules out whole classes of memory-safety bugs at compile time.
The story
Before
For three decades the Linux kernel was written in C and assembly. C leaves memory management to the programmer, and use-after-free and out-of-bounds bugs have made up a large share of kernel vulnerabilities.
What happened, and who
Miguel Ojeda and the Rust for Linux contributors developed the infrastructure, and Linus Torvalds accepted it. Linux 6.1, released on 11 December 2022, merged the initial support: the build machinery and a first layer of kernel abstractions for writing modules in Rust. The release notes call it initial support, to be improved in later releases.
Why it mattered
Rust is the first language other than C and assembly in the mainline kernel. Drivers and other new kernel code can now be written with compiler-checked memory safety, and the same approach is spreading to other operating-system code.
Instructions hidden in a web page or document can take over an LLM application that reads it, such as Bing Chat or Copilot.
The story
Before
Prompt injection was understood as a user typing an adversarial instruction to make a model misbehave, so the attacker and the user were the same person. Nobody had shown an attack where a third party plants the instruction in data the application fetches on the user's behalf.
What happened, and who
Kai Greshake and colleagues at CISPA, Saarland University and Sequire Technology posted “Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection” to arXiv on 23 February 2023. They demonstrated attacks on synthetic GPT-4 applications, on the GPT-4-powered Bing Chat, and on GitHub Copilot, where the malicious text sat in content the model retrieved.
Why it mattered
A model cannot reliably tell the developer's instructions from instructions inside the data it reads. That makes any agent that browses, reads mail or calls tools a target, and it is why prompt injection is treated as a core risk for LLM applications.
Intel proposed an x86 that boots straight into 64-bit mode and drops the legacy modes, then abandoned the idea.
The story
Before
Every x86 processor still starts in 16-bit real mode, as the 8086 did, and carries 16-bit and 32-bit protected modes, virtual-8086 mode and rings 1 and 2. Operating systems use these only to boot, yet the hardware and its validation cost stay.
What happened, and who
In May 2023 Intel published a white paper and draft specification for X86S, a simplified architecture that resets directly into 64-bit operation. It would remove real mode, 16-bit protected mode, virtual-8086 mode, 32-bit ring 0 and rings 1 and 2, while 32-bit applications kept running under a 64-bit kernel. The draft reached version 1.2 by June 2024 and was abandoned in December 2024, after Intel and AMD formed the x86 Ecosystem Advisory Group.
Why it mattered
It shows how far the accumulated legacy of x86 is load-bearing: firmware, boot loaders and old software all depend on it. The cancellation left the question of trimming x86 to be settled jointly by the ecosystem rather than by one vendor.
Groups of query heads share one key and value head, shrinking the KV cache with little loss of quality.
The story
Before
During generation a model keeps the keys and values of every earlier token in a KV cache, and with multi-head attention each head has its own, which makes the cache large. Noam Shazeer's 2019 multi-query attention gave all heads a single key-value head, which cut memory but reduced quality and made training less stable.
What happened, and who
Joshua Ainslie and colleagues at Google Research posted “GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints” to arXiv on 22 May 2023. Grouped-query attention sits between the two extremes, with each key-value head shared by a group of query heads. The paper also showed how to convert an existing multi-head checkpoint to it with a small fraction of the original training compute.
Why it mattered
A smaller KV cache means more concurrent requests per GPU and longer contexts within the same memory. Llama 2's larger models and Llama 3 use grouped-query attention, and most open models since have followed.
Preference learning with a closed-form loss, so no separate reward model and no reinforcement-learning loop.
The story
Before
Reinforcement learning from human feedback needed a reward model trained from preference pairs and then a PPO loop, with the policy, a frozen reference model, the reward model and a value model all in memory. The loop was expensive and sensitive to hyperparameters.
What happened, and who
Rafael Rafailov, Archit Sharma, Eric Mitchell and colleagues at Stanford posted “Direct Preference Optimization: Your Language Model is Secretly a Reward Model” to arXiv on 29 May 2023 (NeurIPS 2023). They showed that the RLHF objective can be rewritten so the policy is trained directly on preference pairs with a simple classification-style loss, with the reference model still supplying the KL anchor.
Why it mattered
It cut alignment training to a single supervised-style stage that fits ordinary fine-tuning tooling. DPO and its variants became a default method for preference tuning of open models.
A mailing-list proposal titled “Let's make PostgreSQL multi-threaded” reopened the database's founding process-per-connection design.
The story
Before
PostgreSQL gives every client connection its own operating-system process, a design that goes back to its Berkeley origins. Processes isolate a crash and avoid shared-memory races, but each one costs memory and start-up time, and a great deal of state is held in per-process global variables.
What happened, and who
On 5 June 2023 Heikki Linnakangas of Neon posted “Let's make PostgreSQL multi-threaded” to the pgsql-hackers list. He reported a strong sense at that year's PGCon that a move to one thread per connection in a single process was worth making, and listed the obstacles: about 1,666 global variables, signal-based communication, process IDs exposed to users, crash recovery and extensions.
Why it mattered
It put a threaded PostgreSQL on the community's roadmap for discussion instead of leaving it a perennial wish. It also showed how deeply the process model runs through the code, since the proposal itself was a list of what would have to change.
Most of the kernel exploits Google's reward programme received targeted io_uring, so Google limited or switched it off in its own products.
The story
Before
io_uring, added to Linux by Jens Axboe in 2019, lets programs batch and complete I/O without a system call per operation, and it is very fast. It also exposes a large and fast-growing amount of kernel code to unprivileged programs, and that code had many bugs.
What happened, and who
On 14 June 2023 Google's security team published “Learnings from kCTF VRP's 42 Linux kernel exploits submissions” on its security blog. It reported that 60% of the exploits submitted to kernelCTF, the kernel part of Google's vulnerability rewards programme, targeted io_uring. As a result Google restricted or disabled io_uring on ChromeOS, in Android apps and on its production servers.
Why it mattered
It is a measured example of a performance interface carrying a security cost, and of a vendor giving up speed to shrink the attack surface. Container runtimes and distributions have since made io_uring something to be enabled deliberately, not left on by default.
ACID multi-item transactions added to DynamoDB by ordering them with timestamps, without slowing ordinary reads and writes.
The story
Before
Customers wanted ACID transactions across items and partitions, but DynamoDB updated items in place without keeping versions, and no single node led a table. The usual answers, locking and two-phase commit with a coordinator, would have cost the single-item operations their predictable latency.
What happened, and who
Joseph Idziorek, Doug Terry, Akshat Vig and colleagues at Amazon presented “Distributed Transactions at Scale in Amazon DynamoDB” at the USENIX Annual Technical Conference in Boston in July 2023. Their design assigns each transaction a timestamp and orders transactions by it, using a two-phase protocol between coordinator nodes and the partitions holding the items.
Why it mattered
It showed that a large-scale key-value store can offer serialisable transactions without touching the cost of its simple operations, because only transactional requests pay for the coordination. Applications that had built their own compensation logic could drop it.
The KV cache is stored in small fixed-size blocks, like virtual-memory pages, so almost none of GPU memory is wasted.
The story
Before
Serving systems reserved one contiguous slab of GPU memory per request, sized for the longest output it might produce. Fragmentation and over-reservation meant that in the paper's measurements only 20.4% to 38.2% of the KV-cache memory held actual token state.
What happened, and who
Woosuk Kwon and colleagues at UC Berkeley posted “Efficient Memory Management for Large Language Model Serving with PagedAttention” to arXiv on 12 September 2023 (SOSP 2023). PagedAttention stores each request's cache in fixed-size blocks that need not be contiguous, mapped through a table, as an operating system pages memory. They built the vLLM serving engine on it, which reached two to four times the throughput of the systems they compared it with.
Why it mattered
More requests fit on each GPU, so serving costs less, and blocks can be shared between requests with a common prefix. vLLM became the standard open-source LLM serving engine, and paged KV caches are now common across serving stacks.
The Completely Fair Scheduler is replaced by EEVDF, which gives each task a virtual deadline and lets short-slice tasks run sooner.
The story
Before
The Completely Fair Scheduler (CFS), written by Ingo Molnar and merged in 2007, shared CPU time fairly by always running the task that had received the least. It had no direct notion of latency, so responsiveness was tuned with a pile of heuristics and special cases.
What happened, and who
Peter Zijlstra adapted “Earliest Eligible Virtual Deadline First” (EEVDF), described by Ion Stoica and Hussein Abdel-Wahab in 1995, for Linux. It gives each task a lag showing how much CPU it is owed and a virtual deadline, and runs the eligible task with the earliest deadline. It merged into Linux 6.6, released on 30 October 2023.
Why it mattered
After sixteen years it is the first replacement of the Linux general-purpose scheduler's core. From Linux 6.12 tasks can also request a time slice with sched_setattr(), so latency-sensitive work is expressed directly and no longer inferred from how a task slept.
A state space layer whose parameters depend on the input, computed with a hardware-aware scan, matches transformers at linear cost.
The story
Before
State space models such as S4 applied the same fixed dynamics to every token, so they could not choose to remember one input and ignore another. On language they lagged behind transformers, and the fast convolution trick that made them efficient no longer applied once the dynamics depended on the input.
What happened, and who
Albert Gu and Tri Dao posted “Mamba: Linear-Time Sequence Modeling with Selective State Spaces” to arXiv on 1 December 2023. Its selective layers let the model decide, token by token, what to write into or forget from its state. Because that breaks the convolution, they computed it with a parallel scan written to keep the state in fast on-chip memory.
Why it mattered
Mamba showed that a non-attention model can match transformers of its size on language while using time linear in the sequence length and a fixed-size state at inference. It is the leading non-attention alternative, and many newer models mix Mamba-style layers with attention.
Group Relative Policy Optimization scores each answer against others sampled for the same prompt, removing the need for a value model.
The story
Before
PPO needs a critic, a value network about as large as the policy, to estimate a baseline for each token's advantage. For language models that roughly doubles the memory and adds another model to train.
What happened, and who
Zhihong Shao and colleagues at DeepSeek introduced GRPO in “DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models”, posted to arXiv on 5 February 2024. The method samples a group of answers to each prompt and uses each answer's reward relative to the group's average as its advantage. With it, a 7-billion-parameter model reached 51.7% on the MATH benchmark without external tools.
Why it mattered
GRPO makes reinforcement learning with checkable rewards much cheaper. It is the optimiser DeepSeek used for its R1 reasoning models, and it is now a standard choice for reasoning training.
Users vote between two anonymous models, and a Bradley-Terry rating turns the votes into a ranking.
The story
Before
Models were compared on fixed benchmarks with known answers. Those saturated as models improved, leaked into training data, and did not measure how helpful an open-ended reply is to a person.
What happened, and who
Wei-Lin Chiang and colleagues at LMSYS and UC Berkeley described the platform in “Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference”, posted to arXiv on 7 March 2024. Visitors ask a question, get answers from two unnamed models and pick the better one. The paper analyses more than 240,000 votes and ranks models with the Bradley-Terry model of pairwise comparisons.
Why it mattered
It became the de facto public leaderboard for chat models, and a high rank there is a common headline for a launch. It also raised the question of what such votes reward, such as length and style, which later analyses tried to control for.
Each token caches one small low-rank latent vector, from which keys and values are rebuilt, instead of the full keys and values.
The story
Before
Grouped-query attention shrank the KV cache by sharing key-value heads, but paid for it in model quality, and the cache still grew with the number of heads kept. Long-context serving remained limited by cache memory.
What happened, and who
DeepSeek-AI introduced Multi-head Latent Attention (MLA) in the DeepSeek-V2 paper, “DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model”, posted to arXiv on 7 May 2024. MLA compresses keys and values into a low-rank latent per token and expands them when needed. The paper reports a 93.3% smaller KV cache than DeepSeek's previous 67B model and 5.76 times the generation throughput.
Why it mattered
It cut KV-cache cost far below what grouped-query attention reaches, with quality reported at least as good as standard attention. It made very long contexts cheap to serve, and other labs have adopted or adapted it.
S3 can refuse a write if an object already exists under the key, which turns object storage into a coordination point.
The story
Before
Two writers racing to create the same S3 key both succeeded, and the last one silently overwrote the first. Coordinating concurrent writers, for a lock or a leader, needed an external service such as DynamoDB or a consensus system.
What happened, and who
On 20 August 2024 Amazon Web Services announced conditional writes for S3. A request carrying the header If-None-Match: * on PutObject or CompleteMultipartUpload succeeds only if no object exists under that key, and otherwise fails.
Why it mattered
Create-if-absent is enough to build a lock, claim a job or elect a leader directly on object storage, with no separate coordination service. Data systems built on S3 now use it to commit table metadata and write-ahead logs.
An experimental build of CPython runs without the global interpreter lock, so Python threads can use several cores at once.
The story
Before
CPython's global interpreter lock (GIL) lets only one thread run Python bytecode at a time, a design from the early 1990s that made the interpreter simple and safe. CPU-bound Python programs could not use several cores from threads and had to use multiple processes instead.
What happened, and who
Python 3.13, released on 7 October 2024, included an optional free-threaded build, run as python3.13t, that can disable the GIL. It followed PEP 703 by Sam Gross, which the Python Steering Council accepted in 2023. The documentation calls the mode experimental and warns of bugs and a substantial single-threaded slowdown.
Why it mattered
It is the first official step in removing the GIL, after decades of failed attempts. Extension authors have to make their code thread-safe, and later releases aim to make free-threading a normal option.
Real-time preemption, developed out of tree for about twenty years, became part of the mainline Linux kernel.
The story
Before
Standard Linux could delay a high-priority task for a long time when the kernel was inside a non-preemptible section, such as a spinlock or the printk logging code. Low, bounded latency needed the separate PREEMPT_RT patch set, which users had to apply and keep in step with each release.
What happened, and who
Thomas Gleixner, Ingo Molnar and the real-time community developed PREEMPT_RT from the early 2000s. It became part of mainline with Linux 6.12, released on 17 November 2024. The last missing piece was a rewrite of printk, the kernel's logging, so that it no longer blocks preemption.
Why it mattered
Latency-critical systems such as industrial control, robotics and audio can now use a stock kernel with real-time preemption instead of a patched fork. Bounded latency still depends on the hardware and on the code running in the system.
S3 can refuse an overwrite unless the object's ETag still matches, which gives object storage compare-and-swap.
The story
Before
S3 could refuse a write to an existing key, but only when creating one. Updating an existing object still had a lost-update race, since two clients could read the same version and the second write silently replaced the first.
What happened, and who
On 25 November 2024 Amazon Web Services added If-Match conditional writes to S3. A write carrying the ETag, the identifier of the object version the client read, succeeds only if the object is still that version. It extended the create-only If-None-Match check announced that August.
Why it mattered
Together the two checks give a compare-and-set on a single key. Table formats and metadata logs can now commit safely with S3 alone, with no separate lock service.
A serverless, PostgreSQL-compatible database for several regions, built so reads need no coordination and conflicts are settled at commit.
The story
Before
Amazon Aurora has one writer region, so a multi-region application could read elsewhere but write in only one place. Spanner-style systems allow writes in several regions but can pay a wide-area round trip for reads inside a read-write transaction.
What happened, and who
Amazon Web Services announced Aurora DSQL at its re:Invent conference on 3 December 2024, in preview, and it became generally available in May 2025. Marc Brooker explained the design in a blog post the same day: reads run against a consistent snapshot without coordinating, and transactions use optimistic concurrency, with conflicts checked at commit time.
Why it mattered
It is a third design point for distributed SQL, between single-writer Aurora and lock-based systems such as Spanner. It relies on precise clocks to keep reads coordination-free, and it puts the cost of coordination at commit rather than at every read.
Reinforcement learning with rule-checked rewards taught a model to reason; on AIME 2024 pass@1 rose from 15.6% to 71.0%.
The story
Before
Reasoning models such as OpenAI's o1, released in September 2024, were closed, and their training method was not published. Nobody outside those labs knew whether long chains of thought could be trained without human-written step-by-step examples.
What happened, and who
DeepSeek-AI posted “DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning” to arXiv on 22 January 2025, two days after releasing the model weights. Its R1-Zero model was trained from a base model with GRPO and rewards checked by rules, such as a correct final answer, with no supervised reasoning data. Its pass@1 on the AIME 2024 maths contest rose from 15.6% to 71.0% over training.
Why it mattered
It gave the open community a working recipe for reasoning by reinforcement learning, and showed that verifiable rewards are enough to make long reasoning appear. It led to a wave of open reasoning models and made training on checkable rewards a standard technique.
A release with asynchronous I/O, time-ordered uuidv7() keys, temporal constraints and data checksums on by default.
The story
Before
PostgreSQL read data pages with synchronous calls, so a scan waited for each read before issuing the next, and apps that used random UUIDs as keys scattered inserts across the whole index. Data checksums, which detect silent disk corruption, had to be asked for when a cluster was created.
What happened, and who
The PostgreSQL Global Development Group released PostgreSQL 18 on 25 September 2025. It added an asynchronous I/O subsystem, a uuidv7() function for time-ordered UUIDs, temporal PRIMARY KEY and UNIQUE constraints using WITHOUT OVERLAPS, and data checksums enabled by default in initdb. pg_upgrade also keeps the planner's statistics.
Why it mattered
Asynchronous I/O lets the database keep several reads in flight, which helps scans and vacuum on fast storage. Time-ordered UUIDs keep new keys near each other in an index, so they can replace random ones without giving up locality.
Two DNS automation processes raced, leaving DynamoDB's regional endpoint with no addresses, and the failure spread to EC2, Lambda and load balancers.
The story
Before
Many AWS services, and AWS's own control planes, depend on DynamoDB in us-east-1, the region's main data centre cluster. DynamoDB kept its DNS records up to date with automation, in which independent “enactors” apply plans, and it contained a latent race that no one had triggered.
What happened, and who
Beginning at 11:48 pm PDT on 19 October 2025, which was 20 October in UTC, two DNS Enactors interacted so that an older plan overwrote a newer one and was then deleted, leaving dynamodb.us-east-1.amazonaws.com with no IP addresses. DynamoDB recovered by 2:40 am PDT, but EC2's droplet workflow manager had lost its leases on servers, and Network Load Balancer health checks failed. AWS declared the whole event over at 2:20 pm PDT on 20 October.
Why it mattered
It showed that the automation that manages a data-plane dependency is part of its blast radius, and that recovery can be slower than the original failure when the systems needed to recover depend on the failed one. AWS disabled the DNS automation worldwide while it fixed the race and added protections against bad plans and limits on how fast health-check failovers can act.
Taught in
Not yet taught in a course.
PaperData
The Aurora DSQL Paper
A design paper for Aurora DSQL covers its query processors, adjudicators, journal and storage, and how they combine into a multi-region SQL database.
The story
Before
Aurora DSQL's design had been explained only in blog posts and conference talks. Engineers could read what it promised but not how its parts fit together or where its limits were.
What happened, and who
Marc Brooker, Marc Bowes, Mike Hershey, Zak van der Merwe, James Morle and Matthys Strydom of AWS posted “Aurora DSQL: Scalable, Multi-Region OLTP” to arXiv on 14 July 2026. It describes a disaggregated design in which PostgreSQL-compatible query processors run in Firecracker microVMs, reads use multiversion snapshots with precise timestamps, and writes use optimistic concurrency with distributed adjudicators settling conflicts at commit.
Why it mattered
It is the first primary source for DSQL's mechanisms, so its design can be reasoned about and compared with other distributed SQL systems on its stated assumptions. It replaces reading the design out of blog posts.
A model that takes unstructured input and a schema and returns typed values with calibrated confidence, without generating text.
The story
Before
Large language models generate text token by token, so a program that wants a decision has to prompt for it and parse the reply. That is slow and costly for the many small classification, routing and scoring decisions inside software, and the output can fall outside the expected values.
What happened, and who
TypeSafe AI, founded by Diogo Almeida, Erik Gafni and Sasha Sheng, launched Jev on 15 September 2026, in early access, alongside a $40 million seed round. The company calls it the first “System One” model, after Kahneman's term for fast intuitive thinking. Given input and a schema of typed questions, it answers all of them in parallel and returns each value with a probability, and it cannot return a value outside the schema.
Why it mattered
It proposes a class of decision models for use inside software, in which the output is typed and calibrated and one call answers many questions at once. Whether the class proves its worth against small fine-tuned classifiers and prompted LLMs is not yet settled.
Learning