The applicability domain problem: why AI drug discovery models fail exactly where novelty lives

The applicability domain problem in AI drug discovery is easy to state and unpleasant to confront: a model is accurate on molecules that resemble its training data, and the whole point of generative design is to produce molecules that do not. Accuracy and novelty pull against each other, and most published accuracy figures are measured where the tension is absent.

We ran into this on our own output. A recovery task that scores 0.945 on a conventional split scored 0.62 when the queries were molecules our own engine had designed. Same model, same target, same metric. The only thing that changed was where in chemical space the questions came from.

Why the number moves that far

The collapse is not mysterious once you measure the distances involved. Take a target with a known set of active compounds, remove one, and ask how similar it is to the rest. For the targets we work on, that leave-one-out similarity lands between 0.41 and 0.53. Known binders look like other known binders, which is exactly why a model trained on them can find them.

Now ask the same question of a molecule our engine designed. The answer is 0.14 to 0.17.

Our molecules sit about three times further from binder space than binders sit from each other. That distance is not a defect to be tuned away. It is the definition of a novel chemotype, and it is what a company is paying for when it commissions de novo design. The prediction problem and the novelty goal are the same axis, pointed in opposite directions.

This is not a quirk of our pipeline

The obvious first suspicion is that we built something unusual. It has been reported elsewhere. A 2026 preprint on structural-frontier evaluation found the same class of failure across six public ADMET tasks, with a taskwise median error inflation of 87% once the evaluation moved to the frontier, and reported that the training penalties it tested did not resolve it.

What we tried, and what lost

The intuitive fix is more physics. If two-dimensional fingerprints fail on unfamiliar scaffolds, a shape-and-electrostatics comparison should do better, because it describes what a molecule is rather than which substructures it happens to contain.

We built that arm and it lost. Shape overlays combined with electroshape descriptors and a desolvation term scored 0.572 on the novel slice. Plain two-dimensional Tanimoto scored 0.610 on the same molecules.

Several other levers went the same way. Down-weighting common fingerprint bits made things measurably worse, because the frequent bits carry the scaffold class, which is most of what a nearest-neighbour method has left when overlap is low. A heavy-tailed kernel produced a paired interval of exactly zero, which turned out to be a proof rather than a result: it is a monotone function of the same distance, and a ranking metric reads only order.

The lever that works is detection, not accuracy

After enough failed accuracy work, the useful question changes. If nobody's numbers are good in this regime, then a model that quietly reports a confident score for a molecule it cannot judge is worse than one that says so. Calibration becomes the deliverable.

Distance to the nearest neighbour is blind to direction

Almost every applicability-domain measure in common use is a distance to the closest known compound. That answers "is anything nearby" and cannot answer "is this molecule pointing somewhere the reference set has never been". Those are different questions, and the second one is the one that matters for novel matter.

Treating the reference set as a subspace rather than a point cloud gives a second reading: the fraction of a query that lies outside the space the references span. On our own molecules that fraction runs 0.56 to 0.64. On held-out real actives it runs 0.18 to 0.30. About 60% of a generated molecule points where the reference set has no mass, against about 20% for a real binder.

What to do about it

• ·Measure on your own output, not on a benchmark split. A scaffold-disjoint split of a public actives set is far easier than the real problem, because two known actives resemble each other much more than a generated molecule resembles either. Until your queries are your own molecules, your accuracy number describes a different task.
• ·Publish the confidence, not just the score. A prediction with a disclosed applicability flag is actionable. The same prediction without one is a liability that surfaces months later.
• ·Do not loosen the domain threshold to make the interface look confident. Ours is set from measurement. Moving it would improve exactly one thing, which is how the product looks.
• ·Treat a score of 0.0 as suspicious. It has twice meant "could not compute" in our own code rather than "scored badly", once when a conformer embedding failed and once for a target with no model at all. Both looked like chemistry and were arithmetic.

Where this leaves us

We do not claim better prediction accuracy on novel chemotypes, because we have not demonstrated it and the published attempts we can check have not either. What we sell is different: a candidate arrives with a synthesis route that exists by construction, a genotoxicity screen run on the reagents rather than only the product, and a stated distance from the region where our numbers can be trusted.

If you want the full evidence table, including the results that went against us, it is on the benchmarks page. The design pipeline behind these molecules is described in Methods, and the companion piece on synthesizability by construction explains the part of the problem we can solve rather than only measure.