What happened
Deep-learning methods for antibody structure prediction, antibody-antigen interaction modelling and design are advancing rapidly. However, comparisons across studies remain difficult because training and test sets are often constructed independently, and a temporal cutoff alone does not prevent train-test leakage. We present SABLE (Structural Antibody Benchmark for deep-Learning Evaluation), a versioned structural antibody resource that couples a fixed training collection with a leakage-controlled held-out test set for reproducible machine-learning development and evaluation. SABLE combines 16,511 experimental training entries with 327 manually reviewed test entries and 3,274 high-confidence, patent-derived AlphaFold3 models spanning 476 antigens. Candidate test structures were selected after the AlphaFold3 temporal cutoff and filtered against the training set using antibody and antigen sequence similarity filters. Each test entry records its nearest training set neighbour, allowing users to quantify remaining relatedness and stratify performance by similarity. Versioned releases provide metadata, processed structures, CDR annotations, redundancy labels and model-confidence fields. A Python/PyTorch API and standardised benchmark metrics provide reproducible database access and evaluation code without requiring additional antibody-structure processing. Update (2026-01-12): Why we are excited about confessions. Deeper analysis of confession training and comparisons to chain-of-thought monitoring. Update (2026-09-17): Inoculation Prompting: Instructing LLMs to misbehave at train-time improves test-time alignment For example, we train on demonstrations of test case hacking with training prompts that request hacks. This model does not learn to hack.. Inoculation Prompting: Instructing LLMs to misbehave at train-time improves test-time alignment For example, we train on demonstrations of test case hacking with training prompts that request hacks. This model does not learn to hack. Update (2026-09-22): Show HN: JevBench, a reproducible benchmark for typed decision models. Hi HN! I built JevBench because Jev kicks ass, and the world deserves to know how the serious open source and fake lookalike projects really perform in comparison. Jev-class models return bounded choices and probabilities instead of text, and are disruptively faster and cheaper than LLMs, while being similarly intelligent on the text input they operate on. JevBench allows looking at accuracy, latency and price all at once, in a weighted way - you can even configure the weighting. A full run asks 534 English decisions. The v1.3 score combines chance-corrected Intelligence, Calibration, Speed and Cost. Leaderboard right now: #1 - Jev 74.4 #2 - SemIf 73.1 #3 - djev 73.0 #4 - Winnow-12B Q8 71.2 #5 reflex 4B 70.3. https://github.com/fstandhartinger/jevbench Two no-signup demos: https://who-is-right.app.mintapis.com https://is-it-ai-slop.app.mintapis.com Limitations: English-only; latency from one German server; local/demo latency gets a disclosed ×2 adjustment (+150 ms on my servers) which is an informed assumption; held-out prompts still reach e Update (2026-09-23): Leakage-controlled benchmarking reveals generalization limits of deep learning for protein-ligand binding affinity prediction . To address widespread data leakage and inconsistent evaluation in protein-ligand affinity prediction, we introduce PLABench, a leakage-controlled and target-centric benchmarking framework that enables standardized comparison across sequence- and structure-based models. We benchmark nine deep learning methods across blind CASP16 targets, leakage-controlled ChEMBL35 sets, and established Davis and KIBA datasets under rigorous data split settings, standardizing structural input via AlphaFold3 to ensure fair comparison. Although pretrained structure-based models achieve the highest overall accuracy, they show severe target-dependent variability, and increasing structural fidelity from predicted to experimental conformations yields no consistent gains. Meanwhile, sequence-based approaches surpass some structure-based methods on select targets, and protein family-level evaluations reveal uneven performance across families and substantial inter-model complementarity obscured by aggregate metrics. These findings demonstrate that training