The Transform That Was Too Simple to Fund

A rejection letter, and two trillion pictures

In 1972 Nasir Ahmed, an electrical engineer at Kansas State University, asked the National Science Foundation to fund a summer of work on a new way to compress digital signals. The reviewers turned him down. As Ahmed later told it, they said the idea was too simple, so they rejected the proposal.¹ He paid for the summer himself, and in January 1974 he and two collaborators published a four-page paper introducing the discrete cosine transform.¹

Consider what that too-simple idea now carries. In 2025 people took roughly 2.1 trillion photographs, about 94 percent of them on smartphones.² Almost every one is stored and sent as a JPEG or a close relative, and the step that makes a JPEG small is a discrete cosine transform. The same transform sits inside the video in a call, the frames of a streamed film, and the audio of nearly every compressed song. It is, by a wide margin, the most widely used linear transform in data compression.³

Ahmed's own verdict, decades on, was mild: it is gratifying, he said, to see the transform become a standard, adding that little did they realize it would be used the way it is.⁴ The interesting question is not why a good idea took hold. It is why this particular idea, which really is close to trivial to describe, turned out to be almost the best possible thing to do. That is not luck. It is a fact about the mathematics, and it explains both why the reviewer was wrong and, in a sense, why he was right.

What the transform actually does

Start with the concrete case the transform was built for. The JPEG baseline process cuts an image into blocks of eight pixels by eight, and runs a forward DCT on each block on its own.⁵ A block is 64 brightness values. The DCT rewrites those 64 numbers as 64 different numbers, the amounts of 64 fixed cosine patterns that, added together, reconstruct the block exactly. The first pattern is a flat gray, the block's average. The next few are broad light-to-dark gradients. The later ones are fine checkerboards, the rapid ripples that capture edges and texture.

No information is lost in that rewrite; it is a change of coordinates, fully reversible. The compression happens next. For a typical photographic block, the energy piles up in the first few coefficients and the fine-ripple coefficients come out near zero. That property has a name.

JPEG cashes that in by dividing the coefficients by a quantization table and rounding, which sends most of the small high-frequency terms to zero, and then storing the survivors compactly. Undo the steps and you get a block that looks almost like the original from far fewer bits. The whole scheme leans on one assumption: that when you express a natural image in cosines, the important part collapses into a handful of numbers. Fabian Giesen, who has spent a career building codecs, puts the engineering verdict plainly: after decades of DCT-based coders, the transform is known to sit quite close to the optimal choice of basis for certain classes of signals.⁶ That word optimal is doing real work, and it is worth taking literally.

The exact answer it stands in for

There is a provably best linear transform for this job, and it is not the DCT. Machine-learning readers already know it under another name. Take a large collection of image blocks, compute the covariance matrix of the pixels, and find its eigenvectors. Those eigenvectors are the basis that decorrelates the data perfectly and, among all linear transforms, concentrates the most energy into the fewest coefficients. Statisticians call it the Karhunen–Loève transform; practitioners call the same construction principal component analysis.

The DCT is not that basis. It is a fixed set of cosines, chosen before any image is seen, identical for every block in every picture ever coded. So how can a fixed basis come close to the data-optimal one? Ahmed's 1974 paper already reported the answer empirically: the DCT's performance compares closely to that of the Karhunen–Loève transform, which is known to be optimal, including on the rate-distortion measure that governs compression.⁷

Five years later Anil Jain gave the reason. Model a row of pixels as a first-order Markov process, where each pixel is a noisy copy of its neighbor with correlation close to one. That is a crude but decent model of a natural image, whose neighboring pixels are nearly always alike. Jain showed that the discrete Fourier, cosine, and sine transforms and the Karhunen–Loève transform of such a process are all members of one family of sinusoidal transforms, and that they are asymptotically equivalent.⁸ As the correlation between neighbors approaches one, the KLT of the Markov model converges to the DCT. The cosine basis is not an approximation someone tuned to look like the optimum. It is the exact limit the optimum runs toward for the signal statistics that real images happen to have.

This inverts the reviewer's complaint. The DCT is simple because the right answer, for this signal, is simple. A blunt objection that the idea was too plain to matter missed that the plainness was the whole result: a basis you can write down with no data at all lands a hair away from the basis you would need a mountain of data to compute.

Why the imitation beat the original

If the KLT is optimal, why did the world standardize on the approximation instead? Because optimal on paper and usable in a chip are different tests, and the DCT wins the second decisively. The KLT is data-dependent, which is fatal in a codec. You would have to estimate a covariance matrix, compute its eigenvectors, and then ship those eigenvectors alongside the image so the decoder could invert them; the basis itself becomes part of the payload. The DCT carries none of that cost. It is fixed, so both ends already know it. It is separable, so a two-dimensional block is just the one-dimensional transform applied to rows and then columns. And Ahmed's original paper gave an algorithm to compute it through the fast Fourier transform, which put it within reach of the hardware of the day.⁷ The one real friction, as Giesen notes, is that the cosine coefficients are irrational, so bit-exact implementations across devices need careful integer approximations.⁶ That is a solvable engineering problem, not a barrier.

A fixed basis that is nearly optimal and nearly free to compute is the kind of thing that spreads. It did.

1. 1974

   The DCT is defined⁷

   Ahmed, Natarajan, and Rao publish the transform and show it tracks the optimal KLT for correlated signals.

2. 1979

   The reason is proved⁸

   Jain places the DCT and the KLT in one asymptotically equivalent family.

3. 1992

   JPEG ships⁵

   The still-image standard builds its baseline on the 8×8 DCT.

4. 1990s–2010s

   Everything moving³

   MPEG video, H.264 and H.265, and, through the modified DCT, MP3, AAC, Dolby Digital, Vorbis, and Opus.

The audio branch is the clearest sign of reach. A modified DCT, applied to overlapping windows of sound rather than square blocks of pixels, became the compression core of MP3, AAC, Dolby Digital, Vorbis, and Opus.³ A transform designed for still pictures turned out to be the right tool for music it was never built for, because sound, like an image row, is a signal whose samples are strongly correlated with their neighbors. Touradj Ebrahimi, who chaired the JPEG committee, has said flatly that only a few image-compression standards not using the DCT exist today.¹

The loop closes

There is a final turn, and it is the one that makes the whole story more than a piece of trivia about compression. The DCT won because, fifty years ago, computing the true optimum for each signal was out of the question. That constraint is lifting. The newest generation of codecs is beginning to abandon the fixed cosine basis for the very thing the cosines were imitating all along.

AV2, the successor codec now under development at the Alliance for Open Media, still keeps the DCT and its sine-based cousins as core kernels. But it introduces data-driven transforms for certain block sizes, trained on prediction residuals using the Karhunen–Loève transformation itself.⁹ The optimal, data-dependent basis that was too expensive to ship in 1974 is now cheap enough to learn from data and embed in a standard. The field spent five decades using a closed-form stand-in for the KLT, and is now circling back to compute the KLT directly, because the arithmetic finally fits.

That is the shape worth carrying away. The discrete cosine transform was never really the answer; it was a near-perfect placeholder for an answer nobody could afford, and the placeholder was so good that it quietly defined how digital images and sound look and feel for two generations. The grant reviewer who called it too simple was reading the wrong signal. The idea was simple because it had found, in closed form and with no data, a result that the optimum only reaches in the limit. Simplicity that close to the exact answer is not a weakness to be funded around. It is the rarest thing in engineering, and this time it happened to be sitting in a rejected summer proposal.