AlphaFold2 folds proteins by learning a geometry it can differentiate

A fifty-year problem meets a blind contest

A protein is a chain. Each link is a residue, one of twenty amino acids, and the chain is the same one whether it is stretched out or folded. What a cell uses is the folded shape: the single compact arrangement the chain settles into, with each residue at a definite point in space. That shape decides what the protein binds and what it does. A genome hands you the chain for free; reading the shape meant growing a crystal or freezing a sample and firing radiation at it, and turning the chain into the shape by calculation had stayed open for more than fifty years.¹

The field kept score through CASP, a contest run every two years in which organizers release sequences whose structures have been solved but not published and let every group predict them blind. Accuracy is measured mostly by GDT_TS, the share of a model's residues that land within a set of distance cutoffs of the true positions, scaled from 0 to 100. For most of CASP's history the hard targets, the ones with no solved relative to copy from, sat far down that scale. In 2020 one entry did not.

AlphaFold2's models had a median backbone error of 0.96 Å against the experimental structures, where the next best method managed 2.8 Å, and its all-atom error was 1.5 Å against the field's 3.5 Å.¹ The unit is worth holding onto: a carbon atom is about 1.4 Å wide, so a median backbone within one Ångström is a prediction wrong by less than the width of an atom.

The rest is how a chain of letters became coordinates that accurate.

The two things the network keeps

Start with the first idea, because without it the geometry has nothing to work from. A single sequence is thin evidence about a shape. The same protein in a mouse, a fish, and a bacterium has drifted in sequence while keeping its fold, and evolution has quietly recorded which residues are free to change and which are not. When two residues touch in the folded structure, a mutation at one that would break the contact tends to be answered by a compensating mutation at the other, so the two positions vary together across species. Stack up the homologous sequences into a multiple sequence alignment, the MSA, one row per relative with the columns lined up by position, and those coupled columns are a readout of which residues sit close in space: the structural signal is there before any structure exists.¹

AlphaFold2 turns that observation into architecture by keeping two separate tensors and refusing to collapse them into one. The first is the MSA representation: the alignment itself, every residue of every relative as a vector the network can revise. The second is the pair representation, a grid with one vector for every ordered pair of residues in the target, meant to hold what the network believes about the relationship between position i and position j: how close they are, how they are oriented. The paper reframes structure prediction as inference on a graph whose nodes are residues and whose edges are exactly those pairs, defined by residues in proximity.¹

One representation is evidence, the coevolution the MSA carries; the other is geometry, a claim about distances and angles. The whole trunk of the network is machinery for letting those two talk without merging them, so evidence can update geometry and a hardening geometry can tell the network what to look for back in the evidence.

Forcing every triangle to close

The trunk is a stack of 48 identical blocks called the Evoformer, and each block updates both representations in turn.² On the MSA side the update is attention run along two axes: across a row, so a residue attends to other residues in the same relative, and down a column, so a position attends to how that same position varies across species. That column pass is where coevolution enters, one alignment column reading its own statistics. Crucially, the MSA's row attention is biased by the pair representation: the current belief about which residues are close is added into the attention logits, so geometry steers where the evidence looks.² The two stacks trade through an outer-product-mean step that writes MSA statistics back onto the pair grid, closing the loop the other way.

The pair representation gets the harder job, and here the biophysics dictates the operation rather than the other way around. A pair grid is a table of pairwise relationships, but not every table of distances describes a real arrangement of points. If i and k are close and k and j are close, then i and j cannot be far: the three distances have to close into a consistent triangle. A network free to set every edge independently will happily write down triangles that violate that and never notice. AlphaFold2 builds the constraint into the update.

Two operations do it. The triangular multiplicative update treats each edge ij as something to be recomputed from the other two edges of every triangle it belongs to, combining ik and jk to update ij, in an outgoing and an incoming variant so both orientations of the triangle get their say.² The triangular self-attention does the same in attention's language: whether edge ij attends to edge ik is modulated by information drawn from the third edge jk, so an update between two edges is always refereed by the one that closes the triangle.² Neither operation imposes the triangle inequality as a hard rule. Both give the network a strong, built-in pressure to keep its pairwise beliefs geometrically consistent, which is what lets a grid of independent edges behave like a set of points in space.

A gas of frames becomes a backbone

The Evoformer never emits coordinates. It hands off two abstractions: the pair grid, and a single representation, one 384-dimensional vector per residue read off the first row of the refined MSA.² Turning those into atoms is the second idea, the one that makes the geometry differentiable end to end.

The structure module represents the protein as what the paper calls a residue gas: every residue is a free-floating rigid body with its own local coordinate frame, a rotation and a translation that place that residue's backbone somewhere in space, initially all stacked at the origin with the same orientation, which the authors name black-hole initialization.² Eight layers, sharing one set of weights, repeatedly move those frames. The bond lengths and bond angles inside a residue are held rigid and the side chains are placed by a handful of torsion angles the module also predicts, so the network never relearns chemistry it can be handed and spends its capacity on where each rigid piece goes.²

The operation that moves the frames is Invariant Point Attention, and its name is its design. A structure has no privileged pose: rotate the whole protein and it is the same protein, so the answer must never depend on the arbitrary global orientation. IPA is attention that, alongside the usual queries and keys, produces points in each residue's local frame and compares them in a shared frame. Because the comparison rests on distances between points, and a distance is unchanged by any global rotation or translation, the computation is invariant to global pose by construction rather than by training.² Twelve heads, four query points each, working in nanometres. The module reads the pair representation as an attention bias, so every constraint the Evoformer negotiated flows into where the atoms go.

The residue gas is a second decoupling. Classical folding moved a connected chain, where nudging one atom drags its neighbours and the model spends its effort keeping the chain intact. Cutting the residues loose lets the network place each one by its own evidence and repair the chain afterward, so the optimization is about geometry, not about holding a string together.

Scoring geometry with one differentiable loss

A pile of free-floating frames only becomes a structure if the training signal rewards the right geometry, and this is the piece that ties the method together. The main term is the Frame Aligned Point Error. FAPE takes the predicted atoms, expresses them in the local frame of each residue in turn, and measures the distance to the true atoms under that alignment, then averages over all the frames.² Scoring under every local frame, not one global superposition, forces atoms to be correct relative to their neighbours rather than only on average, and it hands the network a smooth gradient that flows back through the frames, through IPA, and into the Evoformer.

FAPE also fixes what a simpler distance loss cannot. Distances alone are blind to handedness: a structure and its mirror image have identical pairwise distances, so a model trained on distance r.m.s.d. frequently produces mirror structures. Scoring points inside oriented frames, FAPE sees the difference, and the authors prove it obeys the triangle inequality.²

FAPE does not train alone. The total loss adds auxiliary, distogram, masked-MSA, and confidence terms, and the weights record a candid piece of the story.

The masked-MSA loss, at weight 2.0, hides entries in the alignment and asks the network to fill them back in, the BERT trick applied to evolution, and it outweighs FAPE fourfold. Here an honest reading has to separate what is measured from what is understood. The authors say the masked term is meant to force the network to read coevolutionary relationships, then add, in as many words, that this is the intention but they only observe the outcome that it increases accuracy.² The loss weights were hand-selected and only lightly tuned. A clean story in which every component is derived from principle is not the one the paper tells about itself: several choices are justified by ablation after the fact, and it says so.

Two more mechanisms sharpen the result without changing its nature. Recycling runs the whole network four times, feeding each pass's output back in as the next pass's input from a zero start, for a marked accuracy gain at little extra training cost.² Self-distillation grows the data: the trained network predicts structures for 355,993 sequences with no experimental structure, the confident ones are kept, and a fresh network is trained from scratch on the mix.² Both are amplifiers; neither is why the method works. The evolutionary input and the differentiable geometry are.

The same training produces the confidence outputs that made AlphaFold2 usable in practice. A pLDDT head predicts each residue's own local accuracy, projected into 50 bins and reported as an expected score, so a model arrives annotated with where to trust it.² A separate head predicts the aligned error between every pair of residues across 64 distance bins, telling a user whether two confidently-built domains are confidently placed relative to each other. The second head exists because the first cannot see that question: pLDDT is a local score and, by the paper's own statement, is not sensitive to whether the whole structure aligns under one global rotation, so a model can be uniformly high-pLDDT and still pack its domains wrongly against each other.²

What the referees and a rival found

A method paper grades its own homework, so the result is worth checking against parties with no stake in it. The CASP14 organizers, who ran the blind contest and assessed every group, are the outside referee. Their assessment confirms the headline in their own words: one group, AlphaFold2, submitted models competitive with experimental accuracy for at least two-thirds of the targets, and for only four targets did any other group score higher.³ That is independent certification that the jump was real, not an artifact of self-report.

The same referees mark where it thinned. Their accuracy trend starts near a GDT_TS of 95 and falls to about 85 for the hardest targets, so hardness still costs something. Three targets solved by NMR, a technique that reports an ensemble rather than one crystal structure, scored very low. One flagellar subunit was predicted poorly because it makes extensive contacts with other subunits it was modelled without. And on multi-protein assemblies the round's progress over the previous CASP was small.³ Each of those is the same limitation seen from outside: a network trained to place one chain, at rest, cannot know what it was not shown.

The rival is the second check. Within weeks of AlphaFold2, the Baker lab published RoseTTAFold, an independently designed network that reached, in its authors' own framing, accuracy approaching DeepMind's while remaining, on the CASP14 targets, not as good as AlphaFold2.⁴ Its architecture converged on the same core intuition from a different direction: a three-track network in which information flows back and forth between the 1-D sequence, the 2-D distance map, and the 3-D coordinates, letting the three levels reason jointly. Two groups reaching that design at once is evidence that coupling evidence and geometry was an idea whose time had come, not a trick only one lab could find, and RoseTTAFold's end-to-end run in about ten minutes on a desktop GPU showed it was practical, not just accurate.⁴

What a single resting chain cannot know

Every well-known limit of AlphaFold2 follows from a design choice already described.

Start with the one the paper measures itself. The whole first idea was that structure is read from coevolution, so a protein with no relatives has nothing to read. The paper shows accuracy falling off sharply once the alignment holds fewer than roughly thirty sequences.¹ Orphan proteins, designed sequences, and fast-evolving regions live in that thin-alignment regime, and the model is weakest exactly where evolution left the least signal. The dependence is not a bug to be patched; it is the cost of the idea that made the method work.

Point mutations expose the same root differently. A single amino-acid change can destabilize a protein, and it is tempting to read a mutant structure straight off AlphaFold2. When Buel and Walters tried, the predicted mutant and wild-type backbones differed by less than about one Ångström, and pLDDT did not track the measured change in stability.⁵ The model looks stable because it was trained to reproduce one settled structure per sequence; it learned the fold a family adopts, not the energetics that a mutation perturbs, so it reports the wild-type answer with confidence and misses the effect entirely.

The remaining gaps line up the same way. A review by Agarwal and McShan collects them: AlphaFold2 struggles to produce multiple conformations for one sequence and to model antigen–antibody interactions, and it does not handle protein–DNA or protein–RNA complexes, bound ligands and ions, post-translational modifications, or the membrane plane.⁶ Read against the architecture, the list is one sentence. The network was trained on single protein chains, in one resting conformation, with nothing but amino acids in the frame. Dynamics need more than one structure; complexes and ligands need partners the model never saw; disordered regions have no single fold to find. Each failure is the shadow of the training distribution.

The clearest confirmation that these are limits of the design, not of the ambition, is what changed when the design did. DeepMind's own AlphaFold3, its later work rather than an independent check, replaced the structure module with a diffusion-based generator that predicts the joint structure of proteins together with nucleic acids, small molecules, ions, and modified residues, and reported markedly better accuracy on exactly the protein–ligand and protein–nucleic-acid complexes AlphaFold2 could not touch.⁷ The successor's whole target list is AlphaFold2's list of failures, and the fix was to change the training distribution and the generator, not to tune the 2021 network harder.