If You Give a Cofolding Model a CCD ID
SMILES and CCD IDs describe the same molecule, but cofolding models don't always treat them that way. For Boltz-2, giving it a CCD ID raised its success rate significantly, with the gain largely disappearing for structures it hasn't seen before.
Authors: Kate Huddleston and Pat Walters
DOI: https://doi.org/10.5281/zenodo.24326120
Over the past year, I have worked almost exclusively with cofolding models, scrutinizing them for ADMET target prediction. Even after a year of this intense scrutiny, I ran into something fundamental and unexpected: my results changed depending on how I described the ligand in the model input file.
Cofolding models take a ligand descriptor as either a SMILES string or a PDB Chemical Component (CCD) ID. Both describe the same molecule, so the model should not care which one you use (Figure 2 shows an example). We tested four models, Boltz-2, AlphaFold3, OpenFold3 (OF3), and OpenFold-OpenBindv0 (OBv0), across different input formats, using two difficult metabolic targets (PXR and CYP3A4) and a random set of PDB complexes. Across every test arm, Boltz-2 was highly sensitive to the ligand input format on both targets, while the picture for the other models was less clear.
Throughout this post, a successful prediction is one in which the predicted ligand pose lies within 2 Å RMSD of the experimental ligand pose. Each model was run with five seeds, and each seed produced five samples (25 poses per complex per model). For each seed, we took the pose with the lowest ligand RMSD and counted that seed as either a success (1) or a failure (0). A complex’s score is the fraction of its five seeds that succeed. For example, three successes out of five seeds is 60%. Unless otherwise stated, a model’s success rate is the average of these scores across all complexes.
Same molecule, different answer.
SMILES and CCD IDs should be interchangeable, but the models do not treat them that way. On the PDB structures for which every model generated a prediction (49 PXR and 85 CYP3A4 structures, five seeds each), CCD input gave a higher average score for every model, but the size of the gain varied widely by model and target.
Figure 1.a shows that Boltz-2 had the largest effect from using CCD IDs, going from 41% success rate with SMILES to 63% with CCD on PXR, and from 42% to 65% on CYP3A4. AlphaFold3 rose from 45% to 59% on CYP3A4, but was essentially unchanged for PXR, with only a one-percentage-point increase. Both OpenFold models (OF3 and OBv0) gained less, between 4 and 12 percentage points depending on the model and target.
A gap in average success does not tell us whether the gain is consistent or driven by a few complexes, or whether this is noise from a large seed-to-seed variation (Figure 1.b). To determine if the differences in performance between CCD and SMILES inputs were statistically significant across complexes, we ran a two-sided paired Wilcoxon signed-rank test on the per-complex success rates (averaged over the five seeds and dropping complexes with identical scores under both inputs) and applied a Benjamini–Hochberg correction across the eight model–target comparisons. Where padj < 0.05, the performance shift cannot be explained by seed-to-seed noise alone. These are shown in bold in Figure 1.a.

Boltz-2 shows a large gain on both targets, about 22 percentage points, and it is consistent across complexes (p = 0.002 on PXR, p < 0.001 on CYP3A4). AlphaFold3 is a mixed case. On CYP3A4 the CCD seeds sit clearly above the SMILES seeds (p = 0.008), but on PXR there is no significant difference (p = 0.78). The OpenFold models are less clear-cut. OpenFold-OpenBindv0 shows a smaller but consistent gain on both targets (p = 0.021 on PXR, p = 0.002 on CYP3A4), while OpenFold3 shows no statistically significant difference between input types on either target (p = 0.057 on PXR, p = 0.13 on CYP3A4), although CCD scored higher on average.
Where does the gain come from?
To identify the source of the gain, we mapped how each codebase converts a ligand into model input. In all three model families, when a SMILES string is provided, the ligand’s conformation is generated with RDKit and its atoms are assigned generic labels that carry no information. A CCD ID provides the PDB entry’s full stereochemistry and deposited atom names. As an example, Figure 2 shows oxadiazon, CCD ID S6Z, from PXR structure 7AXE. With SMILES, the atoms get generated labels such as C1, C2, and O1. With the CCD code, the atoms keep the names deposited with the PDB entry, which are the same in every entry that contains this ligand.

Boltz-2 pulls a precomputed conformer from the CCD entry, while AlphaFold3 and OpenFold still generate theirs with RDKit. All three model families pass the atom names to the network as an input feature. Because the deposited names are the same in every PDB entry containing the ligand, they could serve as an identifier for ligands seen during training, like an atomic name tag.
To separate the two ingredients, we gave each model the CCD molecule with its atom names swapped for generic ones. This changes only the names, so any remaining gain is likely due to the conformer or stereochemistry. For Boltz-2, we keep the stored conformer, so the names are the only difference from the full CCD input. This also indicates whether the stored conformer gives the model information the other models aren’t seeing. This test uses a single seed, so baseline numbers differ from Figure 1.
For Boltz-2, replacing the deposited names with the generic ones removes the gain (Figure 3). On PXR, success drops from 60% to 33%, and on CYP3A4 from 62% to 48%. This change is also one-directional. PXR has 14 structures that improve with deposited names and 1 that worsens, and CYP3A4 has 16 that improve and 4 that worsen (Figure 3). Even with the stored conformers, generic names leave Boltz-2 near or below its SMILES level. So the conformer alone does not explain the boost in success rate.
The other models tell a different story. For AlphaFold3, the names do nothing overall. Generic and deposited names both score 50% on PXR and 54% on CYP3A4, though individual structures flip in both directions. Since the gain survives with generic names, something other than the names must drive it for this model. For the OpenFold models, deposited names add 11 to 13 percentage points on PXR, and the mixed CYP3A4 results can’t be separated from noise with a single seed.

For Boltz-2, the gain diminishes at the training cutoff.
If the deposited names tag ligands seen in training, the gain should vanish for structures released after the training cutoff. We tested this by splitting the PXR and CYP3A4 results at each model’s cutoff date (Figure 4): June 1, 2023 for Boltz-2 and September 30, 2021 for AlphaFold3 and OpenFold3. OpenFold-OpenBindv0 is excluded from this test because its cutoff falls after all but one of these structures.

For Boltz-2, the gain is concentrated before the cutoff. On PXR, CCD input raises success from 47% to 80% before the cutoff but lowers it from 24% to 20% after. On CYP3A4, success rate rises from 42% to 68% before the cutoff but shrinks to 40% to 45% after. AlphaFold3 and OpenFold3 do not follow this pattern. AlphaFold3’s CYP3A4 gain appears on both sides of its cutoff (46% to 58% before, 40% to 60% after), and its PXR results are flat on both sides. OpenFold3’s modest gain does not shrink after the cutoff for either target. The after-cutoff groups are smaller, so these numbers are not as precise, but the contrast for Boltz-2 is stark.
To check whether this trend is specific to our “hard” targets, we ran the same test on a random selection of PDB complexes (Figure 5). We selected 50 complexes released between 2017 and 2021, before each model’s cutoff, and 50 more from a separate pool released after each model’s cutoff. For the OpenFold family, we are showing OpenFold-OpenBindv0, which has the most up-to-date weights. As a result, we selected the “after training” set so that every structure was released after this model's June 2025 cutoff. We ran each complex with a single seed (five samples). With 50 complexes and one seed, each complex is theoretically worth 2 percentage points, so small differences should be interpreted as suggestive.

Boltz-2 shows the same pattern on the random complexes as on PXR and CYP3A4, going from 84% to 92% before the cutoff, and actually decreasing from 50% to 46% after the cutoff. AlphaFold3 shows a 10 percentage point increase before the cutoff and no increase after, but because its PXR and CYP3A4 results conflict with this, we read it as suggestive. OpenFold-OpenBindv0 shows no CCD benefit on either side.
Boltz-2 is the only model whose gain largely disappears after its cutoff on both PXR and CYP3A4, which is consistent with a gain that appears only for structures it could have seen. AlphaFold3’s gain is inconsistent, since its CYP3A4 gain does not depend on the cutoff, so we cannot say that anything from training explains it. For the OpenFold models, we see no evidence to claim data leakage from training, and OpenFold-OpenBindv0’s gain on PXR and CYP3A4 remains unexplained.
In summary
Memorization in cofolding models is a known problem. Here we show evidence pointing to a source of it for Boltz-2: the CCD input gain is consistent with memorization of atom labels. It largely disappears for structures released after the training cutoff. For Boltz-2, a CCD code is not only a shorter way to describe a ligand, but it is also a SMILES plus a name tag the model may already know.
AlphaFold3 and the OpenFold models don’t show this pattern. AlphaFold3’s gain doesn’t depend on the atom names or the cutoff, and the OpenFold gains are modest. While we might see some significant gain for these models, we don’t know where it comes from. One candidate is something else the CCD molecule carries, such as its stereochemistry, which requires more digging.
The implication, as I see it, is that Boltz-2’s scores on PDB structures are inflated when ligands are given as CCD IDs, both relative to its own SMILES results and relative to models that show a smaller CCD effect. This inflation misleads users and benchmark readers. A new compound only has a SMILES, so the performance you see in a CCD-based benchmark may overpromise what you’ll get in practice.
More broadly, we need to take stock of what these models see and learn from their training sets and be vigilant about determining what, if any, data leakage we are letting through. This is paramount in measuring the model’s full potential and improving upon current baselines.
Reproducibility
All data and notebooks needed to reproduce the figures in this post are available in the repo linked below. Only data files are provided. No prediction mmCIFs are within this repo. If you would like the actual files or model inputs please reach out to Kate Huddleston at kate.huddleston@omsf.io.
Repo: https://github.com/OpenADMET/cofolding-ligand-input-study
Acknowledgements
We would like to thank our funders for their support of OpenADMET, in particular ARPA-H, Radial (part of the Astera Institute (https://ror.org/00ydx1s47)), Schrödinger Inc., the Gates Foundation and the OpenAI Foundation. We would also like to thank our partners Enamine, Hugging Face, OpenEye, CDD Vault, Discovery Life Sciences, and the beamline staff at NSLS-II for their support.
This work is supported by the Advanced Research Projects Agency for Health (ARPA-H) under AVOID-OME, and Award Number 1AY1AX000035. The contents are those of the authors. They may not reflect the policies of the Department of Health and Human Services or the U.S. government. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Advanced Research Projects Agency for Health.