Net Good IndexSubmit a correction

Benefit Ledger · provisional · Biology

Cell-level random splits leak group-owned answers in single-cell benchmarks

Machine learning models in single-cell biology increasingly forecast differentiation, reprogramming and therapeutic response from early transcriptomic profiles. Testing whether a model has learned real biology requires held-out cells. Single-cell data, however, are grouped: cells from the same clone, patient or batch share the same label. A random split therefore places relatives of each test cell, carrying its label, in the training set, and a model can score well by memorizing a relative instead of learning a transferable rule. Grouped validation removes this leakage but leaves far fewer ind

Tier 1 UsefulMethodology 0.2
Submit a correction

Current score

+0.00

1 base · Useful (tier 1 of 5, 1 pts)
× 0.1000 attribution · Minor documented assistance
× 0.1000 evidence · Firsthand or social claim
× 0.2000 realization · Proposed
× 0.5000 durability · Medium-term
Event-level product before credit split: 0.00

Auto-published from news ingest as a provisional placeholder. Score is conservative until a named release is identified and the record is rescored.

What happened

Machine learning models in single-cell biology increasingly forecast differentiation, reprogramming and therapeutic response from early transcriptomic profiles. Testing whether a model has learned real biology requires held-out cells. Single-cell data, however, are grouped: cells from the same clone, patient or batch share the same label. A random split therefore places relatives of each test cell, carrying its label, in the training set, and a model can score well by memorizing a relative instead of learning a transferable rule. Grouped validation removes this leakage but leaves far fewer independent units behind each error bar. Here we show how to estimate this leakage before training any model, from two properties of the data: exposure, the fraction of test cells with relatives in training, and retrievability, how often a nearest-neighbor search returns such a relative rather than an unrelated cell. Across lineage-barcoded and patient data, exposure determines whether a random split opens a leakage channel, and retrievability determines how much it can inflate the score. The inflation is negligible where cell state has decoupled from ancestry, much larger where clonal sisters remain close in expression space, and in a patient cohort large enough to overturn a clinical conclusion. We also provide eakcheck, which computes both properties in seconds, before the outcome model is fitted.

Model attribution

Unspecified AI system
Version unspecified
+0.00

Unspecified

Unspecified system mentioned or implied by a news item. Remap to a named release when identified.

News ingest does not infer a named model from the publisher alone. Attribution stays unspecified until a release is identified.

Attribution 0.1000 · Credit share 100% · Unknown

Claims

  • Cell-level random splits leak group-owned answers in single-cell benchmarks

    outcome · supported

Sources

Revision history

  • 16 Sep 2026 · 0.00 → 0.00

    Auto-published from news ingest.