AntiBERTy, another model based on the BERT architecture, was proposed by Ruffolo and colleagues (Ruffolo etal. 2021). It was trained on 558million sequences from OAS with MLM objective. The team analyzed repertoires from donors with HIV‑1 neutral‑ izing VRC01 antibodies. For each sample, they created a k bearers neighbour (kNN) graph using model embeddings and visualized it in two‑dimensions using Uniform Manifold Approximation and Projection (UMAP). Using these plots, they observed tra‑ jectories from germline sequences and mutated derivatives corresponding to sequence changes in the afnity maturation process. With repertoire data, individual sequences are not labeled. Hence–relying on clonal expansion–the team produced noisy labels, and frequently observed sequences were assumed to be binders. Next, they applied mul‑ tiple instance learning (MIL) to predict whether the sets of sequences contain binding antibodies. They created single‑instance bags of sequences from known VRC01 anti‑ bodies, conrming that the model produces the correct positive predictions. Finally, they annotated each antibody structure with attention, and in most cases (7 out of 10 sequences), attention‑pointed binding residues.