Ordering and repetition do not change a set. Use a tuple when positions matter, or a list when repetition and sequence matter.

An element and a singleton containing it are different objects. Membership is not the same relation as being a subset.

The subset relation permits equality; a proper subset excludes equality. The empty set is a subset of every set.

Changing the universe can change the complement even when the original set stays fixed. Negated selection rules must identify their population.

Mathematical either includes both. De Morgan’s laws exchange the two operations when taking complements.

Difference is directional. A minus B and B minus A generally select different elements.

The bars around a set count elements, not the size of the values inside it. The empty set has cardinality zero; a set containing the empty set has cardinality one.

Square brackets include endpoints; parentheses exclude them. Intersect with the integers when only whole-number values are allowed.

For finite sets the cardinalities multiply. Restrictions coupling the choices define a subset of the product and can reduce the count.

Corresponding coordinates must agree for two tuples to be equal. A pair (1,2) differs from (2,1), unlike sets containing those numbers.

Specify the domain of a predicate. Quantifying its inputs turns a statement template into a claim about a population.

It is true when its premise is false. This is a statement about permitted truth assignments, not a claim of causation.

The converse reverses premise and conclusion. The contrapositive also negates both. Proving the converse does not establish the original implication.

Coincidental truth of two claims is insufficient. A truth table exhausts a finite Boolean domain and can establish an equivalence.

The domain and nesting order are part of the claim. A later existential witness may depend on earlier universal choices.

Existence does not imply uniqueness. Verify both domain membership and the claimed property.

There is no premise-satisfying violation. An existential over the empty domain is false because it has no witness.

Match the obligation to the claim’s logical form. Arithmetic transformations need their conditions, such as nonzero divisors.

Represent arbitrary objects using their defining properties, then identify why the derived form establishes the conclusion.

For a conditional, assume both its premise and the negation of its conclusion. A surprising result is not necessarily a contradiction.

Cases may overlap, but no permitted input may be omitted. Positive and negative cases alone omit zero.

A conditional counterexample must satisfy the premise and fail the conclusion. Finite successful tests cannot replace an all-input argument.

A composite integer exceeds one and has a factorisation into two smaller factors greater than one. One is neither prime nor composite.

The least-failure argument derives induction from this foundational property. Positive reals do not have this property.

The induction hypothesis is temporary while proving an implication. Check the smallest index for which the successor step is claimed.

Equivalent to ordinary induction applied to the proposition that an entire prefix holds. Especially useful when an object splits into smaller pieces.

Supply enough base values to determine every later case. A recursive definition supplies objects; induction proves properties of them.

Initialisation and preservation correspond to induction. Termination needs a separate argument that the loop reaches its stopping condition.

Rename every bound occurrence together. Reindexing that changes values also changes endpoints and the summand.

These identity values make splitting and appending rules continue to work at length zero.

Coefficients must not depend on the summation index. A constant added inside the sum appears once per term.

Use a one-to-one correspondence, translate bounds, and substitute in each summand. For nested sums, reconstruct dependent bounds from the index region.

Consecutive differences sum to final value minus initial value. Establish the exact algebraic cancellation before applying it to rounded computations.

Pairing a finite progression with its reverse derives its total. The formula requires a constant increment and a specified term count.

A finite total follows by multiplying and subtracting shifted terms. Ratio one has its own branch; infinite totals require convergence theory.

Absolute difference measures distance with the same physical unit as the original quantities. A signed bound alone is not a two-sided tolerance.

Opposite signs can cancel; equal signs can attain the bound. Repeated application controls finite sums without probability assumptions.

Adding absolute-error bounds protects against every permitted sign combination. Observing cancellation in one run does not tighten that guarantee.

Domain and codomain are part of the function, not just the rule. The image may sit strictly inside the declared codomain.

Defined only when f’s codomain is g’s domain. Associative always, commutative almost never.

Both properties depend on the declared sets, not only the rule. On finite same-size sets they stand or fall together; on infinite sets they need not.

An inverse exists exactly for bijections, and composites invert in reverse order. The superscript is not a reciprocal.

Strict monotonicity implies injectivity, a sufficient but not necessary test. A strictly monotone map inverts onto its image.

Degrees add under multiplication and stay under the maximum under addition. The zero polynomial has no degree.

By the factor theorem a root a is exactly a factor x−a, so a nonzero degree-n polynomial has at most n roots.

Long division terminates because the degree strictly decreases. The remainder at x−a is the value p(a).

Angle is arc divided by radius, so it is a ratio of lengths. Calculus formulas assume radians, not degrees.

The period is 1/f. Phase shifts the waveform in time; amplitude scales the peaks.

Multiplication rotates and scales in the complex plane. Every polynomial over the reals splits over the complex numbers.

The product z·z̄ = |z|² is real, which is what makes division by a complex number possible.

Stated, not proved, at this level; repeated factoring then gives exactly n roots counted with multiplicity.

Real exponents are filled in by continuity, not defined by a new counting rule. The laws bˣ⁺ʸ = bˣbʸ and (bˣ)ʸ = bˣʸ survive the extension.

Needs b > 0 and b ≠ 1. Change of base makes every logarithm proportional to every other; base 2 counts bits and doublings.

On a log axis equal ratios are equal distances, so a doubling anywhere is the same step. Check the axis before comparing bar lengths.

The slope reads the exponent and the intercept reads log c. Curvature on a log–log plot is itself evidence of another mechanism.

Positions and repetition matter, unlike a set. Convergence is a property of the tail, so finitely many edits change nothing.

The ε–N contract: for every positive tolerance, some index threshold leaves all later terms within it. Checking a finite window is evidence, not proof.

The limit is the supremum of the terms. This proves convergence before the limit is identified — the route taken by Heron's iteration.

Approximately 2.718281828, irrational. It is the base whose exponential and logarithm have the simplest slopes in calculus.

A series converges or diverges exactly as its sequence of partial sums does. Terms tending to zero is necessary but not sufficient.

The value is 1/(1−r) for a start of 1. Discounted rewards and retry backoffs are geometric series in disguise.

Grouping into blocks each contributing at least a half proves divergence. It grows without bound, but only logarithmically fast.

Comparison bounds against a known series; the ratio test inspects successive-term ratios. Each test has cases where it is silent.

Absolutely convergent series may be rearranged safely; conditionally convergent ones can be reordered to reach any limit.

The product rule needs each stage's options not to depend on earlier choices; the sum rule needs disjointness. Most counting errors are one of these assumed without checking.

Choosing k of n in order gives n!/(n−k)! arrangements. Order matters means positions are distinct.

Derived by dividing ordered arrangements by the k! orderings of each subset — the division rule removes systematic overcounting.

Stars and bars counts multisets by matching selections with binary strings. The bijection must be total in both directions.

The count is the number of binary strings with k stars and n−1 bars. Identical items with distinct boxes is the dual picture.

Stated and checked numerically, not derived, at this level. It is how astronomically large counts are estimated and compared.

The distinguished-element split makes it a theorem, not a pattern. It builds a whole row by addition alone — no factorials needed.

Specialising the identity produces counting facts: all-ones gives 2ⁿ, a minus gives the equal even/odd split. The theorem also feeds the binomial distribution later.

Each element is counted exactly once because the alternating sum of subsets containing it telescopes to one. Truncating the sum alternates over- and under-estimates.

The first truncation of inclusion–exclusion. Crude but always true, which is why it bounds failure chances under arbitrary overlap.

The count follows from inclusion–exclusion and approaches n!/e. It is the canonical case of the alternating sum doing real work.

The quantifier order is the contract: ε is challenged first, δ is the response. Equivalent to convergence along every sequence approaching a.

Compositions and arithmetic combinations of continuous functions stay continuous. Continuity is what makes bisection and limit interchange legal.

IVT brackets roots (a sign change forces a crossing); EVT guarantees a maximiser exists — both stated with proof ideas at this level.

Equivalently the coefficient of the best linear approximation f(x+h) = f(x) + f′(x)h + o(h). Differentiability implies continuity, not conversely.

Predicting a function near a point by its tangent is the workhorse behind linearisation, Newton steps and gradient descent.

(fg)′ = f′g + fg′; the quotient rule follows from product + reciprocal. The inverse rule (f⁻¹)′ = 1/f′ yields (ln x)′ = 1/x.

Composing the linear approximations proves it — the proof must handle inner increments of zero. Values flow forwards; correction factors flow backwards.

Fermat → Rolle → MVT is the proof chain. Its corollaries — positive slope means increasing, |f′| ≤ L bounds change — convert local derivatives into global facts.

The MVT turns a derivative bound into a bound on how far outputs can separate — the seed of every smoothness and stability claim later.

Derived from Cauchy's mean value theorem. It orders growth rates — logarithmic < polynomial < exponential — and evaluates expressions like x ln x that drive entropy formulas.

A continuum has no index set — the sum exists only after a partition supplies one. Linear, additive over intervals, monotone in the integrand.

For continuous f it is attained somewhere inside — the mean value theorem for integrals, which is the bridge in the FTC proof.

Part 1 makes every continuous f the derivative of its own integral; part 2 turns most evaluation into finding one antiderivative.

The '+C' family is exactly what 'two antiderivatives differ by a constant' buys — an MVT corollary. Indefinite integrals name the whole family.

Limits transform too: ∫f(g(t))g′dt over [a,b] equals ∫f over [g(a),g(b)]. The |g′| factor is the local axis stretch — the 1-D seed of change of variables.

Choose u by what simplifies on differentiating. Repeated application gives the reduction Iₙ = nIₙ₋₁ behind ∫xⁿe⁻ˣ = n!.

∫₁^∞ x⁻ᵖ converges iff p > 1; comparison and the integral test tie it to series. One-sided limits — symmetric pairing is a separate, stronger claim.

Matching forces coefficients f⁽ᵏ⁾(x)/k!; the Lagrange remainder f⁽ⁿ⁺¹⁾(ξ)hⁿ⁺¹/(n+1)! is proved by a telescoping auxiliary function through Cauchy's MVT.

R = 1/lim|cₖ₊₁/cₖ| by the ratio test. Convergence of the series and equality to f are separate facts; smooth but non-analytic functions exist.

Series substitution motivates it as the unique extension consistent with exp law; then e^{i(α+β)} = e^{iα}e^{iβ} recovers the angle-sum identities.

Representable numbers are finite and unevenly spaced — the gap doubles at each power of two — so most reals are rounded to a neighbour rather than stored.

binary32 halves the storage at a range and precision cost; bfloat16 keeps the exponent width and cuts the mantissa, trading precision for range.

It bounds the relative error of one rounded operation, not of a whole computation; accumulated error and conditioning are separate questions.

The standard model is relative and per-operation, with ties breaking to even; it justifies first-order error analysis but says nothing about error compounding.

Subnormals make underflow gradual — precision drains as the exponent bottoms out — instead of flushing abruptly to zero.

NaN propagates through arithmetic and compares unequal to everything, including itself, so x ≠ x is the portable test for it.

A property of the problem, not the algorithm; a stable formula inherits it but cannot beat it.

Small relative errors in the operands become a large relative error in the difference; rearranging the formula, not extra precision, is the fix.

A thousand factors of 0.01 underflow to zero while their log sum stays finite; likelihoods are handled in log space for this reason.

Shifting by the largest term caps the biggest exponential at one, removing overflow; softmax and normalising constants are computed this way.

The naive forms cancel catastrophically for tiny x; these library functions evaluate the Taylor series directly instead.

A data row and a geometric displacement are the same object; positions matter, unlike in a set.

The two readings agree via the law of cosines; the scalar projection of x on a unit u is x·u, so the dot product measures alignment rather than magnitude.

Proved from the nonnegative quadratic ‖x−ty‖² ≥ 0 in t; equality holds exactly for parallel vectors, which licenses cosine as a similarity.

Works in any dimension where no picture exists; nonzero orthogonal vectors are automatically linearly independent.

Magnitude-free, so proportional vectors of different lengths score one; it is the default similarity for feature vectors.

The axioms capture what every notion of vector size shares; norms on ℝⁿ differ by at most dimension factors, ‖x‖∞ ≤ ‖x‖₂ ≤ ‖x‖₁ ≤ √n‖x‖₂.

L₂ is smooth and rotation-invariant; L₁ weighs small entries relatively more and promotes sparsity; L∞ sees only the worst coordinate.

Determined entirely by the images of the basis vectors, which is why a finite grid of numbers can store it.

Ax reads two ways: a combination of the columns with weights x, or dot products of x with the rows; both pictures are used constantly.

The rule is derived from composing maps, not chosen; column by column, AB applies A to each column of B.

Associativity A(BC) = (AB)C always holds — bracketing is free, ordering is not, as a rotation–shear pair shows.

As a map it sends v to x(y·v); rank-one pieces of this shape are the terms the SVD decomposes a matrix into.

Its defining identity is (Ax)·y = x·(Aᵀy) — the transpose moves a matrix across the dot product; it reverses products, (AB)ᵀ = BᵀAᵀ.

Covariance matrices, Hessians and AᵀA all qualify; the spectral theorem later converts the property into real eigenvalues and an orthonormal eigenbasis.

Symmetric and positive semidefinite since xᵀAᵀAx = ‖Ax‖² ≥ 0; it is the matrix the normal equations solve.

Row operations act on whole equations; the appended column carries b through the same transformations without writing the variables.

Each preserves the solution set and equals left multiplication by an elementary matrix, so elimination is secretly a factorisation.

Pivot columns hold the determined variables while free columns parameterise the rest; a row reading 0 = c signals inconsistency.

Ax = b asks whether b is a combination of the columns of A; the unknowns are the coefficients.

Always a subspace, and the smallest one containing the set; membership tests reduce to solving a system.

It must contain the origin; the column space and null space are the working examples behind every existence and uniqueness statement.

Ax = b is solvable exactly when b ∈ C(A); its dimension is the rank.

Equivalently Ax = 0 has only the zero solution, so representations are unique; more than n vectors in ℝⁿ are always dependent.

Gives every vector of the space unique coordinates; the exchange lemma proves all bases of a space share one size.

dim ℝⁿ = n; a plane through the origin is two-dimensional wherever it sits inside a larger space.

The same vector has different coordinate lists in different bases; converting between them is change of basis.

Row rank equals column rank — a theorem, not a definition; rank(AB) ≤ min(rank A, rank B) since composing can only destroy directions.

The general solution of Ax = b is one particular solution plus all of the null space; a trivial null space means uniqueness.

Every input direction is either kept or destroyed and the count balances exactly; it is why a wide matrix cannot have a trivial null space.

Exists exactly when the rank is full, the null space trivial and the determinant nonzero; Gauss–Jordan on [A | I] computes it, but solving Ax = b directly is cheaper and stabler than forming it.

One property among many equivalent tests — a matrix failing one fails them all; inverses of products reverse order, (AB)⁻¹ = B⁻¹A⁻¹.

Fixed uniquely by multilinearity, alternation and det I = 1; cofactor expansion and the product of pivots are consequences, not definitions.

Composing maps multiplies volume scaling; it yields det A⁻¹ = 1/det A and det(P⁻¹AP) = det A — the determinant belongs to the map, not the coordinates.

Coordinates are basis-dependent while the vector is not; P translates the description, it does not move the object.

Similar matrices share rank, determinant, trace and eigenvalues — the quantities belonging to the map itself rather than to a coordinate choice.

Always a subspace; the pairings N(A) = C(Aᵀ)⊥ and C(A)⊥ = N(Aᵀ) organise the four fundamental subspaces into two orthogonal decompositions.

Minimising distance and enforcing b−p ⊥ a are the same computation; every later projection is this argument done on a subspace.

Idempotent (P² = P) and symmetric; the residual b − Pb is orthogonal to the space by construction, giving the Pythagorean split ‖b‖² = ‖p‖² + ‖b−p‖².

Coordinates come free as cᵢ = qᵢᵀv — no system to solve, because each basis vector already accounts for one direction.

So Q⁻¹ = Qᵀ and ‖Qx‖ = ‖x‖ — it preserves lengths and angles; rotations and reflections are the examples, with determinant ±1.

v − Σ(qᵢᵀv)qᵢ removes the accounted components before normalising; numerically the modified ordering is the stable variant.

Gram–Schmidt repackaged; R's triangularity records that each column was projected onto the span of its predecessors, and it replaces the normal equations in stable least squares.

The minimiser's residual is orthogonal to the column space — geometry supplies the equation before calculus does; the fitted values Ax̂ are the projection of b.

Unique when the columns are independent; analytically tidy but numerically it squares the condition number, which is why QR is the practical route.

Repeated application is trivial in the eigenbasis — that is the entire point; eigenspaces, the null spaces of A−λI, group an eigenvalue's directions.

Roots may be complex or repeated; a shear has a repeated eigenvalue with too few eigenvectors — the non-diagonalisable case.

Powers collapse to Aᵏ = VΛᵏV⁻¹ and det A = Πλᵢ; distinct eigenvalues give independent eigenvectors, which is the typical case.

Real eigenvalues and orthogonal eigenvectors remove both failure modes of general matrices; the rank-one sum reading is the seed of low-rank approximation.

Maximised over unit vectors by the top eigenvector at λ_max; it is how eigenvalues get re-read as the answer to an optimisation.

Cyclic — tr(AB) = tr(BA) — hence similarity-invariant; xᵀAx = tr(Axxᵀ) turns quadratic forms into traces, the move matrix calculus repeats.

Only the symmetric part (A+Aᵀ)/2 matters; in the eigenbasis it reads Σλᵢyᵢ², so the eigenvalue signs classify its level sets as ellipses, hyperbolas or saddles.

Equivalent tests include positive pivots and positive leading minors; it guarantees a strict minimum of the quadratic form and the existence of a Cholesky factorisation.

Covariance and kernel matrices must be PSD; Gram matrices BᵀB qualify automatically, and congruences CᵀAC preserve the property.

Turns a symmetric solve into two triangular solves at half the cost, and generates correlated Gaussians as μ + Lz from independent noise.

A partial order — two symmetric matrices can be incomparable; curvature bounds like ∇²f ⪰ mI are written in it.

Derived from the spectral theorem on AᵀA; σᵢ = √λᵢ(AᵀA) with Avᵢ = σᵢuᵢ, and U and V supply orthonormal bases for all four fundamental subspaces.

The expansion A = Σσᵢuᵢvᵢᵀ orders the matrix's content by importance and the count of nonzero σᵢ is the rank; decay of the σᵢ is what makes low-rank summaries possible.

Returns the minimum-norm least-squares solution even for rank-deficient systems — what a numerical solve quietly computes when no unique answer exists.

Equals σ_max in the 2-norm and is submultiplicative, ‖AB‖ ≤ ‖A‖‖B‖; the Frobenius norm √(Σaᵢⱼ²) instead measures total energy.

‖δx‖/‖x‖ ≤ κ(A)·‖δb‖/‖b‖ — roughly log₁₀κ digits are lost; since κ(AᵀA) = κ(A)² the normal equations are twice as ill-conditioned as the problem.

Eckart–Young proves it optimal with error ‖A−A_k‖₂ = σ_{k+1}; the kept share Σ_{i≤k}σᵢ²/Σσᵢ² is the energy fraction that picks k.

A loss is the first, a layer map the second; the graph of a scalar field is a surface one dimension above its domain.

Contours of a quadratic form are ellipses or hyperbolas; gradients turn out perpendicular to them.

xy/(x²+y²) takes different limits along different lines, so checking the axes proves nothing; norms replace absolute value in the definition.

A coordinatewise rate only; having every partial is weaker than differentiability, which demands the whole linear approximation.

This, not the existence of partials, is the property later rules are proved from; a function can have all partials and still fail it.

Cauchy–Schwarz makes u = ∇f/‖∇f‖ the maximiser of the directional derivative; ∇f is orthogonal to the level set through the point.

One dot product answers every direction question; the partials are the special cases along the axes.

F(x+h) ≈ F(x) + J(x)h; rows are gradients of outputs, columns sensitivities to inputs — a linear map's Jacobian is the matrix itself.

Composing the linear approximations proves it; for scalar output ∇ₓ(f∘g) = J_gᵀ∇_y f, and the order the product is evaluated in is the cost question of automatic differentiation.

Symmetric by Clairaut's theorem for smooth f, and equal to the Jacobian of the gradient; uᵀHu is the curvature along u and its eigenvalues are the principal curvatures.

Derived by restricting to the line through h and applying the one-variable theorem; the bound f ≤ f(x) + ∇fᵀh + (L/2)‖h‖² when ‖∇²f‖ ≤ L is what a gradient step minimises.

The two conventions transpose every answer; the course fixes denominator layout, under which ∇ₓaᵀx = a and ∇ₓxᵀAx = (A+Aᵀ)x.

Differentials track the first-order perturbation directly and survive where componentwise bookkeeping collapses; ∇_A log det A = A⁻ᵀ falls out of eigenvalue perturbation.

Accumulates a density over area rather than length; a joint probability mass is exactly this object.

Iterated integration is the computable form; on non-rectangular regions the order fixes the bounds, and swapping can turn an impossible integral into an easy one.

The Jacobian determinant is the local volume scale, generalising the one-variable |g′| factor; polar coordinates contribute the extra r, and squaring the Gaussian integral into a polar one evaluates ∫e^{−x²/2}dx = √2π.

The scalar ẏ = ay gives y = e^{at}y₀; eigenvalues with negative real part decay, positive diverge, imaginary oscillate — the spectrum is the stability certificate of the flow.

Stable only while |1+λΔ| < 1, so a stiff eigenvalue caps the step; global error is O(Δ) versus O(Δ⁴) for classical Runge–Kutta.

Ā = e^{AΔ} and B̄ = A⁻¹(e^{AΔ}−I)B discretise ḣ = Ah + Bx exactly — the form controllers and samplers actually implement.

Set algebra — unions, intersections, complements — becomes event algebra; continuous outcome sets need measure-theoretic care, named but not taught.

Complements, monotonicity, inclusion–exclusion and the union bound are theorems from these; the axioms deliberately assume nothing about equally likely outcomes.

B becomes the new sample space; conditioning is a different probability measure, not a filtered count of the old one.

Splits a hard probability into cases; the Bᵢ must partition — overlapping or incomplete cases silently corrupt the total.

Prior, likelihood and posterior are the named parts; a 95%-sensitive test on a 1%-prevalence condition still yields a posterior near 16% — the base rate is part of the answer.

Pairwise independence is weaker than mutual — three events can satisfy every pair and fail the triple; the product form, not intuition, is the checkable definition.

Can hold conditionally but not marginally, or the reverse; it is the assumption behind factorised feature models.

The distribution is the induced law on values, not the map itself; different mechanisms can induce the same distribution.

Always defined, nondecreasing and right-continuous; discrete jumps and continuous slopes both live inside this one object.

A density is not a probability and may exceed 1 — probabilities are its integrals, and P(X = x) = 0 at every point.

Defined only when the weighting converges absolutely — the Cauchy distribution has no mean at all.

The distribution of g(X) is never needed; every variance, moment and Monte Carlo estimate uses this form.

Holds through arbitrary dependence — the strongest of the easy rules, and the reason indicator decompositions count complicated events without untangling them.

Scales quadratically, Var(aX+b) = a²Var(X); the standard deviation √Var returns to the original units.

P(A) = E[1_A] converts probabilities to expectations; expected counts become sums of easy indicators, as in counting a permutation's fixed points.

The atom of discrete probability; indicators, coin flips and single classification outcomes are all Bernoulli.

Mean np by linearity of expectation; the PMF sums to one by the binomial theorem — where counting meets probability.

Mean 1/p from a series computation; it is the discrete waiting-time law.

The model for rare events in a large population; mean and variance both equal λ.

The many-label generalisation of Bernoulli; its one-hot encoding is the vector form classifiers emit.

Characterises the geometric among discrete distributions and the exponential among continuous ones; it is what makes Markov models tractable.

Probability is proportional to interval length; pseudo-random generators emit it and every other distribution is built from it.

Mean 1/λ by integration by parts; the continuous counterpart of the geometric and the summand of the gamma family.

A flexible family on the positive reals; the gamma function's defining integral is integration by parts applied repeatedly.

Mean α/(α+β); the natural prior for an unknown probability because Bernoulli data just increments the parameters.

Heavier tails than the Gaussian at equal variance; as a prior its log-density contributes an L1 penalty.

Closed under affine maps — aX+b stays Gaussian; the tails decay like e^{−t²/2}, faster than any exponential.

Tables and software give probabilities only for the standard normal Φ; the 68–95–99.7 rule reads mass off it.

The whole story of how the variables relate; marginals, conditionals and independence are all derived from it.

Sum or integrate the other variables out; the name survives from the margin of a printed joint table.

A renormalised slice of the joint; Bayes' rule for densities, f(θ|x) ∝ f(x|θ)f(θ), is the same move with a parameter.

Implies zero covariance but not conversely; the factorisation is the checkable definition, not the intuition.

Bilinear and symmetric; positive when the variables tend to sit on the same side of their means, and zero under independence.

Cauchy–Schwarz gives the bound, attained exactly when one variable is affine in the other; zero correlation does not imply independence — X and X² are uncorrelated for symmetric X.

Independence kills the cross terms, giving Var(X̄ₙ) = σ²/n — the identity the law of large numbers spends.

Always PSD, since aᵀΣa = Var(aᵀX) ≥ 0; transforms as Cov(AX) = AΣAᵀ, the rule behind Gaussian affine closure.

A function of X rather than a number, minimising mean squared error among all predictors of that form; functions of X pull out, E[g(X)Y|X] = g(X)E[Y|X].

The tower property; it turns an average into a case analysis and runs through every variance argument.

Splits spread into unexplained and explained parts — the decomposition behind variance-attribution arguments.

The reliable route when g is not monotone; inverting the event into a statement about X is where the work sits.

The derivative factor is the local stretch — a density is probability per unit length and g rescales length; in ℝⁿ the Jacobian determinant takes over.

Contours are ellipsoids along Σ's eigenvectors; affine maps preserve the family, and for jointly Gaussian variables uncorrelated does imply independence — unlike in general.

Half its square is the density's exponent; it is Euclidean distance in coordinates whitened by Σ^{-1/2}.

The conditional mean is a linear regression on the observed block and the covariance shrinks — observing is information, computed not asserted.

Appears both in the conditional law and in block inversion of Σ; it is variance explained away, in matrix form.

Proved by comparing X with the indicator a·1_{X≥a}; weak but assumption-free, and the parent of every stronger tail bound.

Markov applied to (X−μ)²; distribution-free, so tight in the worst case and loose for named distributions.

The weak law follows from Var(X̄ₙ) = σ²/n via Chebyshev — convergence in probability; the strong law's almost-sure convergence is stated, and the Cauchy counterexample shows the finite-mean hypothesis is load-bearing.

M^{(k)}(0) = E[Xᵏ] by differentiating the series, and M_{X+Y} = M_XM_Y under independence — convolutions become products; existing near zero is itself a tail condition, and the Gaussian MGF e^{μt+σ²t²/2} falls out by completing the square.

The MGF's second-order expansion is the proof idea; it gives the shape of the error — with a continuity correction for lattice distributions — but no finite-sample bound, which is what Berry–Esseen prices.

Exponentiating tilts weight toward the tail and t is then optimised; any MGF becomes an exponential tail bound this way.

Needs boundedness alone — no variance knowledge; its lemma bounds e^{sx} by the chord on [a,b].

The property Hoeffding-style bounds actually use; bounded variables qualify, and the class is closed under independent sums.

Concentration plus a union bound covers many hypotheses at once — the price of breadth is logarithmic, not linear.

Chebyshev, CLT and Hoeffding give different answers under different assumptions — finite variance, asymptotic normality, boundedness; the honest choice matches what is actually known.

A modelling assumption with computational payoff — the whole state is summarised by one probability vector.

n-step probabilities are matrix powers, (Pᵗ)_{ij} — the Chapman–Kolmogorov equations; state distributions update as π_{t+1} = π_tP.

Periodicity — returns locked to multiples of d > 1 — is the remaining obstruction to convergence; the class structure decides which behaviours persist.

Gambler's ruin and expected hitting times reduce to solving systems; it is the recursion behind every Markov-chain calculation.

A left eigenvector with eigenvalue 1; it exists for every finite chain and is unique when the chain is irreducible.

Sufficient for stationarity and stronger — it forbids circulation; checking it is the design recipe behind Metropolis–Hastings.

The law of large numbers for dependent samples; it licenses estimating expectations from a single long chain.

For reversible chains the second eigenvalue |λ₂| sets the geometric decay of convergence; burn-in is the wait it prices.

Necessary, not sufficient — maxima and saddles satisfy it too; it filters candidates rather than certifying them.

Positive Hessian eigenvalues mean upward curvature in every direction; a zero eigenvalue leaves the test silent, as x⁴ versus x³ shows.

A minimum along some directions and a maximum along others; in high dimension saddles outnumber minima and are what descent slows around.

Algorithms certify the first; only extra structure — convexity — promotes local to global.

Intersections, affine images, half-spaces and norm balls are convex; convex combinations Σθᵢxᵢ with θ ≥ 0, Σθ = 1 generate the convex hull, and the PSD matrices form a convex cone.

Stated geometrically at this level; it is the engine behind duality and the existence of subgradients.

Equivalently the epigraph is a convex set; nonnegative sums, affine composition and pointwise maxima all preserve convexity.

One inequality turns ∇f = 0 into a global guarantee — the tangent is a lower bound everywhere, not just nearby.

Upgrades the qualitative guarantee to a rate; the ratio L/m is the condition number that descent's geometric bound spends.

The theorem deciding whether a found minimum can be trusted; strict convexity adds uniqueness on top.

The subdifferential ∂f(x) is the set of them — ∂|x|(0) = [−1,1]; the optimality condition becomes 0 ∈ ∂f(x*).

The subdifferential condition produces it in three cases; it is the atomic step inside L1-penalised methods.

One tangent line at E[X] proves it; E[log X] ≤ log E[X] and AM ≥ GM are the corollaries that keep paying off.

At the optimum ∇f is perpendicular to the constraint surface, hence parallel to ∇g; λ also reads as the sensitivity of the optimal value to the constraint level.

Its unconstrained stationary points are the constrained candidates; maximising xᵀAx subject to ‖x‖ = 1 lands on the top eigenvector.

μᵢ ≥ 0 with μᵢhᵢ = 0 encodes bind-or-slack per inequality; under a constraint qualification like Slater's they are necessary, and for convex problems sufficient.

Identifies which constraints matter at the optimum; solving the cases is how small KKT problems are done by hand.

Always concave — an infimum of affine functions — so the dual problem is convex whatever the primal is.

The gap p*−d* certifies how far a feasible point sits from optimal; strong duality is what makes solving the dual a route to the primal.

Derived as minimising the quadratic upper bound rather than adopted as a heuristic; the step of maximal guaranteed progress is η = 1/L.

Yields the descent lemma f(x−η∇f) ≤ f − η(1−Lη/2)‖∇f‖²; steps beyond 2/L diverge even on a quadratic.

Rates convert curvature assumptions into iteration counts; the condition number L/m decides how geometric the geometric rate is.

In the Hessian's eigenbasis the error in coordinate i multiplies by 1−ηλᵢ per step; the steepest direction caps η while the flattest crawls.

The optimal step is 2/(λ_max+λ_min); at κ = 10⁴, six digits of accuracy costs tens of thousands of iterations — the motivation for curvature-aware methods.

Even a diagonal reparameterisation removes most zigzag; momentum, stated rather than proved, improves the dependence to √κ.

Solves a quadratic exactly in one step and is affine-invariant — rescaling coordinates changes nothing; the per-step cost is a linear solve, not an inverse.

Local only — far from a minimum, or with an indefinite Hessian, the step can climb; damping and line search supply the global behaviour.

Unbiasedness legitimises substituting it for the true gradient; its variance shrinks like 1/b in batch size.

Expected progress is descent minus an η²σ²/b penalty; the noise term balances the gain at a floor proportional to ησ²/b.

η_k ~ 1/k is the canonical choice; the two conditions trade total travel against variance decay.

Each step is low-dimensional — a soft threshold or a small solve; monotone decrease is free, but convergence needs smoothness or separable non-smoothness, and a non-separable kink can stall every coordinate away from the minimum.

Robbins–Monro targets a root of h rather than a minimum — α_k = 1/k on h(x) = μ−x recovers the running average; the ODE method reads the iterates as following ẋ = h(x), and constant steps track a moving target at the price of residual noise.

GD pays O(nd) per step for κ log(1/ε) steps, Newton pays O(nd²+d³) for log log(1/ε), SGD achieves O(d/ε); dimension, data size and target accuracy pick the winner.

Judged by its distribution over resampled datasets, not by one answer; bias, variance and consistency are the criteria.

The 1/n sample variance undershoots by exactly (n−1)/n — Bessel's correction divides by n−1 to fix it; a little bias can buy a lot of variance.

The split has no residual term; a biased shrinkage estimator can beat an unbiased one outright on MSE.

The law of large numbers supplies it for the sample mean; it constrains the limit only, saying nothing about finite n.

The unit of estimation uncertainty; when σ is estimated from the same data, the t-distribution prices the extra uncertainty.

Not a probability over θ — it need not integrate to one; the log-likelihood Σlog p is what gets maximised, both for calculus and because the raw product underflows.

p̂ = k/n for Bernoulli, μ̂ = x̄ and the 1/n variance for Gaussians; invariant under reparameterisation — the MLE of g(θ) is g(θ̂).

The general form most training losses take; per sample it is the average code length that cross-entropy formalises.

Bernoulli, Gaussian, Poisson and gamma all fit the form; the natural parameter η and sufficient statistic T(x) — the only channel through which data enters the likelihood — are what make inference uniform.

∇A = E[T] and ∇²A = Cov[T] ⪰ 0 — convexity for free, which is why the MLE condition ∇A(η̂) = (1/n)ΣT(xᵢ) is moment matching.

Has mean zero — the fact that powers baselines in gradient estimators; curvature of the log-likelihood is measured through it.

The two forms agree by differentiating ∫p = 1 twice; for Bernoulli, I = 1/(p(1−p)) — most information where the outcome is least predictable.

Also the asymptotic variance of the MLE, √n(θ̂−θ) ⇒ 𝒩(0,I⁻¹); Fisher information is the local curvature of KL divergence.

Combines likelihood and prior as probability laws; the normalising constant p(D) is usually the computational bottleneck.

Beta on Bernoulli data returns Beta(α+k, β+n−k) — data just increments hyperparameters; conjugacy is a property of a likelihood–prior pair.

Integrates θ out instead of plugging in a point estimate; for Beta–Bernoulli it is simply the updated success fraction.

The direct reading a confidence interval does not permit — here the parameter is random and the interval fixed; the two notions must not be conflated.

log p(θ) is the penalty — a Gaussian prior adds an L2 term, a Laplace prior an L1; unlike the MLE it is not invariant to reparameterisation.

95% refers to the procedure's long-run capture rate — the parameter is fixed and the interval random; a pivot like (X̄−μ)/(s/√n) is inverted into the endpoints, with Student's t for small n.

Inverts the score test rather than plugging p̂ into the standard error; the symmetric ±z·SE form breaks down at extreme p.

The test computes evidence against H₀, never for it — the asymmetry is structural, not a rhetorical habit.

It is not P(H₀|D) — conditioning runs on the hypothesis, not the data; the test–interval duality means a level-α test rejects exactly the values outside the (1−α) confidence interval.

The significance level α controls the first; power trades against the effect size Δ and sets the required sample size.

Grows like 1−(1−α)ᵐ under independence; at m = 20 and α = 0.05 it is already near two-thirds.

The union bound gives FWER control under arbitrary dependence; the price is power, so it suits confirmatory rather than exploratory testing.

The step-up rule compares sorted p-values against iα/m; tolerating a controlled fraction of false finds buys much more power than FWER control.

The universal method when the quantile function is computable; −ln(U)/λ produces an exponential in one line.

The acceptance rate is 1/M and collapses in high dimension — the limit that motivates Markov chain methods.

A change-of-variables construction; combined with μ + Lz from a Cholesky factor it generates arbitrary Gaussian vectors.

Unbiased by linearity, with standard error σ_f/√n independent of dimension — the reason grids lose and sampling wins in high dimension.

The optimal c = Cov(f,g)/Var(g) removes the correlated share of the variance without touching the mean — the simplest variance reduction.

Unbiased whenever q > 0 where f·p ≠ 0; the optimal proposal q* ∝ |f|p concentrates samples where the integrand is large.

The unknown normaliser cancels in the ratio at the cost of a small bias — the usable form for unnormalised posteriors.

One dominant weight collapses the ESS toward one; weight degeneracy is the failure mode this diagnostic catches.

Needs only a differentiable density, so it works for discrete x where sampling itself is not differentiable; E[∇ log p] = 0 means subtracting any baseline stays unbiased while removing variance.

Moves θ out of the distribution into the function — x = μ + σε for Gaussians; far lower variance than the score estimator whenever it applies.

The empirical distribution F̂ₙ plays the role of the true F; the spread of resampled estimates gives a standard error and the percentile interval — failing at extremes and under dependence, where resamples no longer mimic the mechanism.

Derived from detailed balance, which is why the unknown normalising constant cancels; the chain's stationary distribution is the target by construction.

ESS = n/(1+2Σρ_k): correlated draws are worth less than independent ones, and the proposal scale trades acceptance against step size.

Metropolis–Hastings with the conditional as proposal has acceptance one; it needs tractable conditionals, which conjugate structure supplies.

The √ε noise must balance the ε drift; a discretised diffusion whose stationary law approximates p with bias vanishing as ε→0 — wrapping the proposal in a Metropolis correction (MALA) removes the bias exactly.

Derived from requiring rarer events to surprise more and independent events to add; the log base sets the unit — bits for 2, nats for e.

Between 0 and log n with equality iff uniform by Jensen; 0 log 0 = 0 by convention, the binary case H_b(p) peaks at p = ½, and the source-coding theorem makes entropy the compression limit.

Can be negative and changes under rescaling — it is not a limit of the discrete quantity; the Gaussian maximises it at fixed variance.

The chain rule H(X,Y) = H(X) + H(Y|X) decomposes it, and conditioning reduces entropy on average: H(Y|X) ≤ H(Y).

Splits as H(p) + D_KL(p‖q); minimising average cross-entropy to the empirical distribution is exactly maximum likelihood.

Gibbs' inequality gives D_KL ≥ 0 with equality iff p = q; asymmetric — its two directions produce mode-seeking versus mass-covering fits, and its local curvature is Fisher information.

Equals D_KL(p(x,y)‖p(x)p(y)) — dependence as distance from independence; symmetric, nonnegative, zero iff independent, and it detects dependence correlation misses.

Downstream variables know no more about X than the intermediate does; no clever feature rescues information the sensor never captured.

Closed under integer combinations — a|b and a|c give a|(xb+yc); the gcd and prime arguments are built on this closure.

Existence follows from well-ordering of the nonnegative candidates; the remainder is the object modular arithmetic computes with.

Euclid: the product of any finite list plus one has a new prime factor; the prime number theorem π(x) ~ x/ln x prices key sizes.

Satisfies gcd(a,b) = gcd(b, a mod b) — the recursion Euclid's algorithm runs; gcd = 1 is coprimality.

The gcd is the least positive integer combination, by well-ordering; the coefficients are exactly what a modular inverse needs.

Existence is strong induction; uniqueness runs through Euclid's lemma p|ab ⇒ p|a or p|b, which Bézout supplies — factorisations differing only in order are the same.

An equivalence relation respecting addition and multiplication, so remainders can be taken before or after operating — the licence for residue arithmetic.

A finite number system; composite n brings zero divisors — 2·3 ≡ 0 (mod 6) with neither factor zero — which prime moduli avoid.

Read off Bézout coefficients ax + ny = 1; the invertible residues are exactly the units of ℤₙ.

Solutions mod mn correspond bijectively to pairs of small solutions; it splits big-modulus computation into independent pieces.

ℤₙ under addition and the units ℤₙˣ under multiplication are the running examples; the axioms prove each theorem once for all of them.

Cosets are equal or disjoint, so they partition the group; counting them gives the index [G:H] = |G|/|H|.

The coset partition is the whole proof; the order of an element and Fermat's and Euler's theorems are its corollaries.

Every element is a power of g, and the order of an element divides |G|; the units mod a prime are cyclic — the existence of primitive roots — and solving gˣ = h is the discrete logarithm problem Diffie–Hellman rests on.

φ(n) = nΠ_{p|n}(1−1/p) by inclusion–exclusion; φ(p) = p−1 for primes, and |ℤₙˣ| = φ(n) in general.

Lagrange applied to the group of units — the order of a divides φ(n), so exponents reduce mod φ(n).

The prime case of Euler's theorem — the correctness basis of RSA and of Fermat's primality test, which Carmichael numbers like 561 fool despite being composite.

ℤₙ is always a ring and a field exactly when n is prime — Bézout supplies the missing inverses.

The polynomial analogue of a prime; quotienting by one makes every nonzero remainder invertible, producing a field.

Elements are remainder polynomials multiplied modulo m — 𝔽_4 = 𝔽_2[x]/(x²+x+1) is the smallest beyond the prime fields; 𝔽_{2⁸} with x⁸+x⁴+x³+x+1 is the field where bytes live, and every finite field's multiplicative group is cyclic.

Choose the number system first; then give the membership condition. Preserve order with tuples when a set would discard it.

Identify the forbidden case of an implication. Distinguish one fixed witness for everyone from a witness chosen separately for each input.

Try definitions directly; use the contrapositive when its premise is easier to represent. One in-domain counterexample refutes a universal, while finite favourable tests do not establish it.

Check the first transition and all required bases. A program invariant still needs a separate termination argument.

Arithmetic totals require a fixed increment; geometric totals a fixed ratio. Changed bounds, caps or weights may require the original finite sum.

Rectangles swap directly; triangles need dependent bounds rewritten. Exact real equalities do not guarantee identical floating-point rounding.

Add same-unit bounds for a deterministic guarantee. Use exact signed arithmetic when errors are known; tighter probabilistic guarantees require later probability assumptions.

Check both halves: no shared outputs and no missed targets. A pipeline undoes its stages in reverse order.

A degree-n polynomial has at most n roots; divide out each found factor before seeking the next. No real root may mean complex roots instead.

Amplitude scales peaks, frequency sets the period as its reciprocal, phase slides in time. Library trig functions assume radians.

Divide by multiplying with the conjugate over the squared modulus. Polar form makes powers and roots of unity routine.

Choose the base by the unit of the answer: 2 for bits and doublings, e for calculus. A power law reads as a straight line on log–log axes.

A finite run is evidence, not proof; a shrinking gap between neighbours is not convergence. Monotone plus bounded needs no candidate limit in advance.

A fixed ratio below one sums to 1/(1−r); terms shrinking to zero is necessary but not sufficient. Reorder only an absolutely convergent series.

Order matters gives factorials and k-permutations; order irrelevant divides by k!. A bijection to an easy set is often the cleanest count.

Build a row by Pascal's rule when many coefficients are needed. Substituting values specialises the theorem into counting identities.

Alternate the signs as intersection sizes grow. With many sets, the union bound is the cheap over-estimate and the first truncation.

Bisection turns IVT into an algorithm. Neither theorem locates the point — it certifies existence, which is what a bracket needs.

Differentiability gives f(x+h) ≈ f(x) + f′(x)h. A finite-difference table estimates f′ but degrades when h hits rounding noise.

Match the combination's shape first — the quotient rule needs a nonzero denominator, the product rule both factors' derivatives. Reserve the limit definition for functions no rule covers.

Forward-store intermediate values, then multiply correction factors backwards. This is scalar backpropagation.

MVT also turns f′ > 0 into strict increase and f′ = 0 into constancy. The second-derivative sign classifies interior extrema.

Reorder the form until a determinate limit appears. Repeating xᵏ/eˣ shows any fixed power loses to exponential growth.

Distance from speed, energy from power and probability from density are the same object. Set up the slices first; evaluating the resulting integral is the fundamental theorem's job.

Linearity, additivity and |∫f| ≤ ∫|f| govern manipulation. A variable upper limit differentiates back to the integrand — the CDF–density relation.

Spot the inner function's derivative beside it for substitution; differentiate the factor that simplifies for parts. Limits transform under substitution.

Logs and powers differentiate down; exponentials and trig integrate without damage. The move handles x·e^(−λx) — expected waits — and powers stripped one degree at a time.

∫₁^∞ x⁻ᵖ converges exactly when p > 1. The integral test converts series questions into integral ones and back.

The bound |Rₙ| ≤ M|h|ⁿ⁺¹/(n+1)! is the usable form — ξ is never located. The series equals f only where the remainder vanishes.

n-th roots of unity are e^{2πik/n}. A planar rotation of (x, y) is multiplication by e^{iθ} — matrices generalise it to n dimensions.

Each operation returns the exact result rounded once, and the unit roundoff bounds that single error. Equality tests between computed floats are unreliable — compare within a tolerance scaled to the operands.

A thousand probabilities can multiply to zero although each operation kept its promise; the formula spent the accuracy, not the hardware. Subtracting near-equal numbers and exponentiating unshifted scores are the classic leaks.

u·v = ‖u‖‖v‖cos θ turns similarity, orthogonality and projection into one computation. The bound |u·v| ≤ ‖u‖‖v‖ is the estimate most later inequalities stand on.

Pick the norm by the question being asked: the largest single miss, the total absolute miss or the typical miss are different numbers. In fixed dimension the norms bound each other within constants, so the choice is about meaning.

Rescaling, rotating and mixing channels are linear; thresholds and products of inputs are not, and no matrix expresses them. Check linearity before reaching for the entries.

One combined pass replaces three sweeps over the data, and associativity lets you pick the cheapest multiplication order. AB ≠ BA — stage order is part of the semantics.

(AB)ᵀ = BᵀAᵀ reverses products. Symmetry looks like bookkeeping here but is the hypothesis the spectral theorem spends later.

A zero row facing a nonzero constant means inconsistent; free variables mean a whole family of solutions. Elimination reports which case you are in, not just an answer.

Span turns one solvability verdict into a description of every reachable right-hand side. It is closed under addition and scaling — a subspace, not just a list.

Independent lists give unique coordinates; dependent ones carry waste. A total column that merely sums other features is the canonical redundancy.

Any list longer than the dimension is dependent; a shorter one cannot span. Dimension is the invariant that survives every choice of basis.

Rank equals the number of pivots. A nontrivial null space means information loss — many inputs landing on one output — and non-unique solutions.

Invertibility, a trivial null space and a nonzero determinant are the same fact restated. For a single solve, elimination alone is cheaper than forming the inverse.

det A = 0 is the singular case — the yes-or-no behind invertibility. It also rescales areas and volumes under a change of variables and records orientation flips.

The map is one object; only its numbers change with the names of inputs and outputs. Determinant and trace survive — choose the basis that makes the map simple.

When Ax = b has no solution, the projection satisfies Aᵀ(b − p̂) = 0. Geometry turns impossible into best approximation.

Gram–Schmidt orthogonalises any basis; packaged as A = QR it is the numerically stable route to projections and least squares.

It minimises ‖Ax − b‖² exactly. Reach for the QR route rather than forming AᵀA when conditioning matters, and remember squared error lets outliers dominate.

Long-run behaviour is governed by the largest |λ|. Diagonalisation is a gift, not a guarantee — shears and rotations can fail to provide a real eigenbasis.

This is the hypothesis behind principal curvatures, quadratic-form geometry and the SVD's construction. Without symmetry even real eigenvalues can fail to exist.

tr(AB) = tr(BA) lets products rotate inside a trace — the identity that makes Frobenius norms and matrix-calculus gradients cheap.

All-positive eigenvalues give elliptic bowls, mixed signs give saddles, zeros give flat-bottomed troughs. One expression covers Hessian losses, Gaussian exponents and transformed lengths.

Strict positivity gives a strict bowl and a strict local minimum; semidefinite allows flat directions. Covariance matrices and Gram matrices BᵀB are PSD by construction.

Eigendecomposition demands a square self-map; the SVD demands nothing. Rank, pseudo-inverse, conditioning and best low-rank fit all read off the same factorisation.

A floating-point solve returns the exact answer to a slightly different problem; κ decides whether that matters. ‖A‖₂ is the largest singular value, so the SVD already priced it.

The error is the first dropped singular value in the spectral norm, or the tail sum of squares in Frobenius. When the spectrum decays fast, a handful of terms carries the structure.

Limits and continuity carry over, but approach can come from infinitely many directions — that is where differentiability fails even when every partial exists.

The directional derivative along u is ∇f·u, so Cauchy–Schwarz makes the gradient itself the maximiser. Coordinate slopes alone cannot see diagonal movement.

Nudging one input moves every output, so mn rates live in one matrix. Its determinant is the volume rescale used by change of variables and density transforms.

Store the forward intermediates, then propagate the correction matrices backwards: this is backpropagation in its general form.

Two surfaces can share a gradient while one bowls upward, one caps over and one saddles — only second derivatives separate them, and they set the safe step length.

This quadratic model is what gradient descent bounds from above and what Newton's method minimises outright — the workhorse of optimisation analysis.

∇w‖Xw − y‖² = 2Xᵀ(Xw − y) falls out in a few lines where entrywise bookkeeping drowns. The differential method beats memorised tables on anything new.

Joint probabilities and masses over areas are double integrals. Swapping integration order means re-deriving the bounds from the same region — the two-dimensional reindexing of a double sum.

Round regions want polar coordinates, and the same rule transforms probability densities. Forgetting the determinant factor silently changes the answer.

The law gives slopes, not positions; integration turns them back into a path. Euler's local error is second order in the step and accumulates to a first-order global error.

P(A ∪ B) ≤ P(A) + P(B) holds without disjointness — the cheap over-estimate used everywhere. Counting faces is only the equally likely special case.

P(H|E) = P(E|H)P(H)/P(E): base rates dominate, so a sensitive test on a rare condition still returns mostly false positives. Condition on the evidence actually observed.

It is a property of the distribution, not of the story — check it rather than assume it. Zero correlation is weaker and does not imply it.

The CDF F(x) = P(X ≤ x) works for every type, including mixed ones; the density is its derivative where one exists. Events say what happened; variables say how much.

Expectation is linear even without independence; variances add only under uncorrelatedness. These two moments are all that the cheap tail bounds consume.

Choose by the story — fixed trial count versus unbounded rate — not by the shape of the PMF. Each name is a short exact answer to a recurring counting question.

A density is not a probability and can exceed 1; only its integrals are probabilities. Pick the family whose generating story matches the mechanism.

The bell is justified by the central limit theorem — a theorem about sums, not a universal shape. Check that accumulation or measurement error really is the mechanism before defaulting to it.

Marginals alone cannot reconstruct the joint — the dependence lives between them. Bayes' rule for densities is this same slice-and-renormalise.

Covariance sees only linear co-movement — X and X² can be uncorrelated yet fully dependent. Any covariance matrix must be positive semidefinite.

The tower property E[E[Y | X]] = E[Y] prices averages in stages — take the inner expectation first, then average over what was observed.

f_Y(y) is not f_X evaluated at a new point — the volume rescale is essential. For sums, convolve directly or switch to generating functions.

Independent per-coordinate bells only paint axis-aligned ellipses; correlation needs the full Σ. The Cholesky factor Σ = LLᵀ is how you sample and evaluate it.

Marginals come free. This one formula powers Kalman filtering, Gaussian processes and Bayesian linear regression — linear-Gaussian worlds never leave the family.

They are assumption-light and therefore weak — polynomial decay only. When independence supplies a moment generating function, Chernoff and Hoeffding buy exponential tails instead.

It certifies that the average settles, not how fast — a guarantee at fixed n still needs a tail bound, and an approximate distribution needs the central limit theorem.

The MGF determines the distribution and powers the Chernoff method — bound P(X ≥ a) by minimising e^(−ta)M(t) over t. Existence on a neighbourhood of 0 is the hypothesis to check.

It is an approximation with no guaranteed error at fixed n — fine for intervals and quick estimates. When the requirement is a hard probability guarantee, a concentration bound is the honest tool.

The price is a hypothesis: boundedness or a sub-Gaussian proxy such as Hoeffding's lemma supplies. A union bound over m questions multiplies the failure probability by m.

Each tool prices the same (ε, δ) requirement differently — Chebyshev pays 1/δ where Hoeffding pays log(1/δ). The bound you are willing to assume decides the n you must collect.

Check the memoryless property first — if history matters, enlarge the state until it does not. Multi-step distributions are matrix powers; hitting times are linear systems.

The second eigenvalue controls the mixing speed, and time averages along one run obey a law of large numbers — the fact every MCMC estimate is spending.

A vanishing gradient alone cannot separate the three cases. An inconclusive Hessian sends you to higher-order terms or directional checks.

Norm balls, half-spaces and the PSD cone qualify; intersections preserve convexity while unions break it. Non-convex regions are where local methods stall.

Convexity upgrades every local minimum to global and every KKT point to a solution. Least squares and norms qualify; deep training losses do not.

Hinge losses and L1 penalties bend exactly where they are not differentiable. Any subgradient gives a descent direction; optimality needs zero inside the set.

Curvature alone decides the direction — this one inequality sits under the ELBO, entropy bounds and AM ≥ GM. Check which way f bends before moving an expectation inside it.

Probability budgets and unit-length directions are surface constraints. The method returns candidate points; classifying them is a separate step.

A constraint is either active, biting like an equality, or silent with multiplier zero. For convex problems a KKT point is a global minimum.

The dual variables are few, real and merely sign-constrained. A nonzero gap between the two values certifies how far from optimal you still are.

One step size must serve every curvature at once, so the rate is governed by the conditioning. Cheap per step, many steps — the baseline every other method is priced against.

A step safe along the steepest direction barely moves along the flattest. Rescale coordinates to shrink κ, or spend a curvature-aware method.

Each step is a linear solve in n variables and descends only when ∇²f ≻ 0. It wins when few expensive steps beat many cheap ones.

Cost per step falls by the batch factor, but a constant step parks the iterate at a noise floor and rates degrade toward 1/k. The default when n is huge.

A scalar quadratic has a closed-form vertex and a quadratic plus |xᵢ| has soft thresholding — no gradient or step size needed. It stalls when the variables genuinely couple.

SGD is the special case with an objective behind it. Use this when there is no loss at all — running means, fixed points, Bellman-style updates.

Ask four questions — how big is n, is the Hessian available and positive definite, does the objective split by blocks, is it smooth — and the conditioning prices what is left.

The sample mean is unbiased with variance σ²/n. A biased recipe can still win on mean-squared error — the decomposition decides, not the label.

Compute in log-space — products of probabilities underflow. MLE is consistent and asymptotically efficient, but small samples can return degenerate certainty like p̂ = 1.

The log-partition A(θ) is convex and generates moments by differentiation; the sufficient statistics T(x) are all the data need to supply. MLE inside a family is moment matching.

A sharper likelihood means more information and a lower achievable variance. The MLE attains the bound asymptotically — that is what efficiency purchases.

Conjugate priors make the update a closed-form parameter change. The cost is a prior you must defend and a normalising integral that usually refuses closed form.

Cheaper than carrying a full posterior, safer than MLE on small samples. It is still a point estimate: the posterior's uncertainty is thrown away.

Coverage is a property of the recipe over resampling, not a probability that this interval holds the value. The usual ±1.96 standard-error endpoints come from the central limit theorem.

A small p-value rejects; a large one fails to reject and never accepts. The p-value is not the probability the null is true — that inversion is Bayes' job.

m honest level-α tests still yield up to mα expected false positives — each test kept its promise while the collection broke it. The union bound is the cheap fix.

Rejection sampling needs an envelope and degrades fast with dimension. When no direct sampler applies, the fallback is a Markov chain.

You need actual draws from p and finite variance; the error bar comes from the central limit theorem. Correlated draws silently inflate the effective variance.

Unbiased when q covers p's support and the weights are exact; self-normalised weights handle targets known only up to a constant. A badly matched q lets a few huge weights dominate and the variance explodes.

The score function needs only a differentiable density but pays in variance; reparameterisation needs the distribution expressible as a noise transformation. Choose by which you have.

Medians and odd statistics get a sampling distribution without a pivot or closed-form standard error. The resample is only as representative as the original sample — dependence and tiny n break it.

You trade independent draws for tractability: burn-in precedes usable samples and correlation makes the effective sample size smaller than n. The missing envelope stops mattering.

Gradients remove the blindness that stalls a symmetric random walk in high dimensions. The unadjusted chain is biased at finite step size; a Metropolis correction restores exactness.

The same quantity prices compression length and forecast vagueness. The 0 log 0 term is set to 0 by its L'Hôpital limit.

The chain rule splits joint uncertainty, and conditioning never raises entropy on average. This is the term mutual information subtracts.

D(p‖q) is asymmetric and not a distance — check which argument is the truth. Minimising cross-entropy over q is exactly maximum likelihood.

Correlation inspects second moments only — X and X² can be uncorrelated yet fully dependent, while mutual information catches it. It is symmetric and equals D(p_{XY}‖p_Xp_Y).

Primality testing and factoring split apart: testing is cheap while factoring is the hardness RSA banks on. A checksum is a divisibility question.

A multiplicative inverse mod n exists exactly when the gcd is 1, and Bézout's x is that inverse — the subroutine behind modular division.

Hashing, clocks, checksums and RSA live here. ℤₙ is a field exactly when n is prime — otherwise nonzero elements can multiply to zero and division fails where a protocol needs it.

Clock arithmetic, modular units, rotations and permutations are the same object. Lagrange's theorem — subgroup size divides group size — is what Fermat's and Euler's theorems spend.

This is RSA's correctness core and a compositeness certificate — a^(n−1) mod n differing from 1 proves composite without factoring.

A byte has 2⁸ values and 256 is not prime, so GF(2⁸) rather than mod-256 arithmetic is what erasure codes and AES compute in.