{"data":{"event":{"id":"084c846f-7dec-401c-8137-6e28c8d27c7c","slug":"regularized-emphatic-temporal-difference-learning-stability-under-consta-c4b68703ac","title":"Regularized Emphatic Temporal-Difference Learning: Stability under Constant Stepsizes","short_summary":"arXiv:2609.19170v1 Announce Type: new \nAbstract: Emphatic temporal-difference learning (ETD) stabilizes the expected off-policy TD update and changes its projection geometry, but neither property determines constant-stepsize sampled dynamics. We construct an ergodic two-state counterexample in which the ETD mean map contracts while the sampled product has a positive top Lyapunov exponent. Regenerative-cycle analysis separates this sign from the infinite variance of the follow-on trace. We introduce regularized emphatic TD (RETD), a normalized first-order post-shock repair that leaves the trace","full_description":"arXiv:2609.19170v1 Announce Type: new \nAbstract: Emphatic temporal-difference learning (ETD) stabilizes the expected off-policy TD update and changes its projection geometry, but neither property determines constant-stepsize sampled dynamics. We construct an ergodic two-state counterexample in which the ETD mean map contracts while the sampled product has a positive top Lyapunov exponent. Regenerative-cycle analysis separates this sign from the infinite variance of the follow-on trace. We introduce regularized emphatic TD (RETD), a normalized first-order post-shock repair that leaves the trace and importance ratios unchanged, stores the emphatic TD signal in a leaky scalar state, and releases a delayed correction. RETD's raw equilibrium is an affine shift of the ETD equilibrium; single- and two-regularization readouts recover the ETD fixed point exactly. We prove almost-sure convergence for harmonic diminishing stepsizes and a conditional constant-stepsize moment-contraction result from a Markovian random-product bound. RETD has certified negative exponents on the two-state construction and one Baird point, whereas the positive Baird ETD sign remains numerical. Paired 10,000-run experiments validate both separations, fixed-point recovery, a nonmonotone stability region, and task dependence. RETD changes post-shock dynamics; it does not reduce the shared follow-on-trace variance.","ledger_type":"harm","primary_domain_id":"8f1af1b9-7302-4b51-aae9-fdfac04d158a","event_status":"provisional","event_date":"2026-09-18T00:00:00.000Z","discovery_date":"2026-09-18T00:00:00.000Z","first_published_date":"2026-09-18T00:00:00.000Z","last_reviewed_date":"2026-09-18T00:00:00.000Z","geographic_scope":"International","affected_population":null,"base_impact_tier":1,"base_score":"1.00","attribution_multiplier":"0.1000","evidence_multiplier":"0.1000","realization_multiplier":"0.2000","durability_multiplier":"0.5000","current_event_score":"0.001000","confidence_level":"low","score_explanation":"Auto-published from news ingest as a provisional placeholder. Score is conservative until a named release is identified and the record is rescored.","methodology_version_id":"8745541b-0f8e-4d5a-9b47-5291d3813a8d","original_methodology_version_id":"8745541b-0f8e-4d5a-9b47-5291d3813a8d","published_at":"2026-09-18T04:00:52.206Z","created_at":"2026-09-18T04:00:52.206Z","updated_at":"2026-09-18T04:00:52.206Z","flags":[],"domain_name":"Biology","domain_slug":"biology","methodology_version":"0.2"},"contributions":[{"id":"eb59ef08-3ee0-4a3a-a848-9e58a97ecfa5","event_id":"084c846f-7dec-401c-8137-6e28c8d27c7c","model_id":"535014f7-c941-4d39-aa9b-8f9c1d3d07b7","role_description":"Unspecified system mentioned or implied by a news item. Remap to a named release when identified.","attribution_multiplier":"0.1000","credit_share":"1.0000","contribution_score":"0.001000","attribution_rationale":"News ingest does not infer a named model from the publisher alone. Attribution stays unspecified until a release is identified.","attribution_confidence":"medium","first_used_date":"2026-09-18T00:00:00.000Z","model_version_if_known":null,"review_status":"approved","created_at":"2026-09-18T04:00:52.217Z","model_slug":"unspecified-ai-system","model_name":"Unspecified AI system","identity_class":"unknown","is_internal":false,"family_name":"Unspecified","family_slug":"unknown-unspecified","organization_name":"Unknown","organization_slug":"unknown"}],"sources":[{"id":"b3e3936c-abe6-45c9-a502-be0d0629827b","event_id":"084c846f-7dec-401c-8137-6e28c8d27c7c","url":"https://arxiv.org/abs/2609.19170","canonical_url":"https://arxiv.org/abs/2609.19170","source_type":"preprint","publisher":"arXiv cs.AI","author":null,"publication_date":"2026-09-18T00:00:00.000Z","retrieved_at":"2026-09-18T04:00:52.226Z","title":"Regularized Emphatic Temporal-Difference Learning: Stability under Constant Stepsizes","excerpt":"arXiv:2609.19170v1 Announce Type: new \nAbstract: Emphatic temporal-difference learning (ETD) stabilizes the expected off-policy TD update and changes its projection geometry, but neither property determines constant-stepsize sampled dynamics. We construct an ergodic two-state counterexample in which the ETD mean map contracts while the sampled product has a positive top Lyapunov exponent. Regenerative-cycle analysis separates this sign from the infinite variance of the follow-on trace. We introd","content_hash":null,"source_reliability_class":"medium","is_primary_source":true,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-18T04:00:52.226Z"}],"claims":[{"id":"74e52c61-f429-4d42-9f82-ffa721e1acbf","event_id":"084c846f-7dec-401c-8137-6e28c8d27c7c","claim_text":"Regularized Emphatic Temporal-Difference Learning: Stability under Constant Stepsizes","claim_type":"outcome","claim_status":"supported","confidence_score":null,"created_at":"2026-09-18T04:00:52.234Z","updated_at":"2026-09-18T04:00:52.234Z"}],"revisions":[{"id":"a3cf4cce-f8e9-4a5c-882b-9460c0106484","event_id":"084c846f-7dec-401c-8137-6e28c8d27c7c","model_id":null,"previous_score":"0.000000","new_score":"0.001000","previous_factors":{},"new_factors":{"tier":1,"evidence":0.1,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Auto-published from news ingest.","trigger_type":"news_ingest","trigger_source_ids":null,"reviewer_id":null,"review_status":"published","created_at":"2026-09-18T04:00:52.241Z"}],"secondary":[]},"methodology_version":"0.1"}