{"data":{"event":{"id":"a2b1345f-daae-459d-964c-66336eb7c8ba","slug":"from-intent-to-action-benchmarking-llm-safety-in-vehicle-voice-command-a-65f9641f23","title":"From Intent to Action: Benchmarking LLM Safety in Vehicle Voice Command Authorization","short_summary":"arXiv:2609.19630v1 Announce Type: new \nAbstract: Large language models (LLMs) are increasingly integrated into vehicle voice assistants. But linking natural-language requests to vehicle functions creates a safety-critical authorization problem. Before executing a command, the system must choose whether to execute, refuse, clarify, require confirmation, defer to manual control, trigger an emergency response, or make no tool call. To our knowledge, prior evaluations do not isolate this pre-action decision across speaker role, authentication status, vehicle state, and tool availability. We introd","full_description":"arXiv:2609.19630v1 Announce Type: new \nAbstract: Large language models (LLMs) are increasingly integrated into vehicle voice assistants. But linking natural-language requests to vehicle functions creates a safety-critical authorization problem. Before executing a command, the system must choose whether to execute, refuse, clarify, require confirmation, defer to manual control, trigger an emergency response, or make no tool call. To our knowledge, prior evaluations do not isolate this pre-action decision across speaker role, authentication status, vehicle state, and tool availability. We introduce a 202-scenario benchmark with Reference Decisions under a seven-class taxonomy. We evaluate two local open-weight models and three API-based LLMs using Decision Alignment and safety-specific error metrics. Alignment ranges from 40.1% for Llama 3.2 3B to 89.1% for Gemini 3.1 Pro Preview. The API-based models score between 83.2% and 89.1%, with no statistically significant differences among them. Even these models produce two to three False Executes among 161 non-execution scenarios, and persistent errors remain in confirmation and manual-control decisions. A controlled Llama 3.2 3B ablation increases alignment to 40.1% under the structured authorization policy, versus 28.2-29.2% under schema-only and generic-safety baselines, but it does not eliminate False Executes. Structured LLM decisions are therefore insufficient as a standalone safety mechanism, and deployment requires an independent enforcement layer that verifies tool permissions and vehicle-state constraints before invoking any vehicle function.\n\nUpdate (2026-09-18): Reach or Solve? Attributing Agentic RL Gains with Checkpoint Handoffs. arXiv:2609.19636v1 Announce Type: new \nAbstract: Reinforcement learning now trains language-model agents that act over dozens of steps in live environments. The gains are large, and they are read as better decision-making. An agent in a closed loop writes its own inputs. Each observation follows from its own earlier actions, so the states it meets late in an episode are partly of its own making. An SFT checkpoint and an RL checkpoint are then scored from different states, even on identical tasks. Endpoint success mixes two changes: where the agent arrives, and what it does once it is there. Restricting the comparison to states both policies reach does not separate them. That restriction selects on an outcome, and in our data it flips the sign of the effect. We introduce checkpoint handoff, an evaluation protocol that clones a state one released checkpoint reached and hands it to another, with no retraining. Crossing a reacher role and a solver role over SFT and RL splits an endpoint gain into REACH and SOLVE. REACH is how often a policy arrives at a state the environment confirms is a fixe\n\nUpdate (2026-09-22): Splitting Documents at Lower Cost: Multi-Split Boundary Decisions for LLM-Based Page Stream Segmentation. arXiv:2609.22620v1 Announce Type: new \nAbstract: Scanned mail, uploaded PDFs, and consolidated attachments often arrive as page streams that must be split into individual documents before downstream classification, extraction, or routing. Zero-shot large language models can detect document boundaries without task-specific training, but standard Page Classification (PC) and Boundary Decision (BD) formulations resolve only one boundary per model call. We introduce Multi-Split Boundary Decision (MSBD), which predicts multiple boundaries within a page window in a single call, reducing the number of inference requests. We evaluate MSBD across multiple language models, document collections, input modalities, and window sizes. The results reveal a model- and corpus-dependent operating range in which MSBD preserves strong segmentation accuracy while substantially improving inference efficiency, followed by a sharp decline at larger windows. MSBD provided the strongest overall accuracy--efficiency trade-off, while large windows expose distinct over- and under-segm\n\nUpdate (2026-09-23): Do Synthetic Personas Predict Real Audience Response? A Sim-to-Real Study Where a No-Persona Baseline Beats Persona-Based Copy Simulation. arXiv:2609.25010v1 Announce Type: new \nAbstract: Marketers increasingly use large language models (LLMs) as \"synthetic personas\" to predict how an audience will react to a piece of copy before it ships, encouraged by evidence that profile-conditioned LLMs mimic human samples. But is that prediction actually valid against real behaviour - and does the persona machinery help? We present a sim-to-real validity study using the Upworthy Research Archive - thousands of headline A/B tests on shared real traffic, with measured click-through - as held-out ground truth. We compare a ten-persona panel, grounded in the real audience's demographics, against a no-persona zero-shot baseline that simply asks the model how likely a typical reader is to click. Two findings stand out. First, ground-truth reliability is the binding constraint: most A/B tests have no statistically distinguishable winner, so validity can only be measured on the reliable subset (n = 399). Second, and counter to the persona-simulation premise, persona conditionin\n\nUpdate (2026-09-24): Strengthening Red Teams: A Modular Scaffold for Control Evaluations We decompose sabotage into constituent skills and use synthetic simulations to strengthen attacks in complex environments.. Strengthening Red Teams: A Modular Scaffold for Control Evaluations We decompose sabotage into constituent skills and use synthetic simulations to strengthen attacks in complex environments.\n\nUpdate (2026-09-25): Modifying LLM Beliefs with Synthetic Document Finetuning We study whether we can modify the beliefs of LLMs and investigate whether doing so could decrease risk from advanced AI systems.. Modifying LLM Beliefs with Synthetic Document Finetuning We study whether we can modify the beliefs of LLMs and investigate whether doing so could decrease risk from advanced AI systems.\n\nUpdate (2026-09-25): Introducing Anthropic's Safeguards Research Team We're launching a new research team focused on mitigating the post-deployment risks of AI systems.. Introducing Anthropic's Safeguards Research Team We're launching a new research team focused on mitigating the post-deployment risks of AI systems.","ledger_type":"benefit","primary_domain_id":"8f1af1b9-7302-4b51-aae9-fdfac04d158a","event_status":"provisional","event_date":"2026-09-18T00:00:00.000Z","discovery_date":"2026-09-18T00:00:00.000Z","first_published_date":"2026-09-18T00:00:00.000Z","last_reviewed_date":"2026-09-25T00:00:00.000Z","geographic_scope":"International","affected_population":null,"base_impact_tier":1,"base_score":"1.00","attribution_multiplier":"0.1000","evidence_multiplier":"0.5000","realization_multiplier":"0.2000","durability_multiplier":"0.5000","current_event_score":"0.005000","confidence_level":"low","score_explanation":"Ingest update: independent source raised evidence; added a follow-up source. Score remains provisional until review. Follow-up ingest: independent source raised evidence; added a follow-up source. Follow-up ingest: independent source raised evidence; added a follow-up source. Follow-up ingest: independent source raised evidence; added a follow-up source. Follow-up ingest: added a follow-up source. Follow-up ingest: added a follow-up source.","methodology_version_id":"8745541b-0f8e-4d5a-9b47-5291d3813a8d","original_methodology_version_id":"8745541b-0f8e-4d5a-9b47-5291d3813a8d","published_at":"2026-09-18T04:00:58.767Z","created_at":"2026-09-18T04:00:58.767Z","updated_at":"2026-09-25T02:00:54.928Z","flags":[],"domain_name":"Biology","domain_slug":"biology","methodology_version":"0.2"},"contributions":[{"id":"79113ee6-ba87-4392-8eb3-dabc427055ad","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","model_id":"535014f7-c941-4d39-aa9b-8f9c1d3d07b7","role_description":"Unspecified system mentioned or implied by a news item. Remap to a named release when identified.","attribution_multiplier":"0.1000","credit_share":"1.0000","contribution_score":"0.005000","attribution_rationale":"News ingest does not infer a named model from the publisher alone. Attribution stays unspecified until a release is identified.","attribution_confidence":"medium","first_used_date":"2026-09-18T00:00:00.000Z","model_version_if_known":null,"review_status":"approved","created_at":"2026-09-18T04:00:58.777Z","model_slug":"unspecified-ai-system","model_name":"Unspecified AI system","identity_class":"unknown","is_internal":false,"family_name":"Unspecified","family_slug":"unknown-unspecified","organization_name":"Unknown","organization_slug":"unknown"}],"sources":[{"id":"a93d793a-88d5-45f4-9bf1-f226e90ec287","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","url":"https://arxiv.org/abs/2609.19630","canonical_url":"https://arxiv.org/abs/2609.19630","source_type":"preprint","publisher":"arXiv cs.AI","author":null,"publication_date":"2026-09-18T00:00:00.000Z","retrieved_at":"2026-09-18T04:00:58.785Z","title":"From Intent to Action: Benchmarking LLM Safety in Vehicle Voice Command Authorization","excerpt":"arXiv:2609.19630v1 Announce Type: new \nAbstract: Large language models (LLMs) are increasingly integrated into vehicle voice assistants. But linking natural-language requests to vehicle functions creates a safety-critical authorization problem. Before executing a command, the system must choose whether to execute, refuse, clarify, require confirmation, defer to manual control, trigger an emergency response, or make no tool call. To our knowledge, prior evaluations do not isolate this pre-action ","content_hash":null,"source_reliability_class":"medium","is_primary_source":true,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-18T04:00:58.785Z"},{"id":"9b594089-c2fb-49e6-bcaa-0ae3b022053b","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","url":"https://arxiv.org/abs/2609.19636","canonical_url":"https://arxiv.org/abs/2609.19636","source_type":"preprint","publisher":"arXiv cs.AI","author":null,"publication_date":"2026-09-18T00:00:00.000Z","retrieved_at":"2026-09-18T04:00:59.030Z","title":"Reach or Solve? Attributing Agentic RL Gains with Checkpoint Handoffs","excerpt":"arXiv:2609.19636v1 Announce Type: new \nAbstract: Reinforcement learning now trains language-model agents that act over dozens of steps in live environments. The gains are large, and they are read as better decision-making. An agent in a closed loop writes its own inputs. Each observation follows from its own earlier actions, so the states it meets late in an episode are partly of its own making. An SFT checkpoint and an RL checkpoint are then scored from different states, even on identical tasks","content_hash":null,"source_reliability_class":"medium","is_primary_source":false,"is_independent":false,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-18T04:00:59.030Z"},{"id":"bd637de7-b964-4967-a609-036b91a99b35","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","url":"https://arxiv.org/abs/2609.22620","canonical_url":"https://arxiv.org/abs/2609.22620","source_type":"preprint","publisher":"arXiv cs.AI","author":null,"publication_date":"2026-09-22T00:00:00.000Z","retrieved_at":"2026-09-23T03:01:07.174Z","title":"Splitting Documents at Lower Cost: Multi-Split Boundary Decisions for LLM-Based Page Stream Segmentation","excerpt":"arXiv:2609.22620v1 Announce Type: new \nAbstract: Scanned mail, uploaded PDFs, and consolidated attachments often arrive as page streams that must be split into individual documents before downstream classification, extraction, or routing. Zero-shot large language models can detect document boundaries without task-specific training, but standard Page Classification (PC) and Boundary Decision (BD) formulations resolve only one boundary per model call. We introduce Multi-Split Boundary Decision (MS","content_hash":null,"source_reliability_class":"medium","is_primary_source":false,"is_independent":false,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-23T03:01:07.174Z"},{"id":"c6a0afb4-8f1a-49bb-bf6d-e3d34867ce7b","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","url":"https://arxiv.org/abs/2609.25010","canonical_url":"https://arxiv.org/abs/2609.25010","source_type":"preprint","publisher":"arXiv cs.AI","author":null,"publication_date":"2026-09-23T00:00:00.000Z","retrieved_at":"2026-09-23T04:01:02.819Z","title":"Do Synthetic Personas Predict Real Audience Response? A Sim-to-Real Study Where a No-Persona Baseline Beats Persona-Based Copy Simulation","excerpt":"arXiv:2609.25010v1 Announce Type: new \nAbstract: Marketers increasingly use large language models (LLMs) as \"synthetic personas\" to predict how an audience will react to a piece of copy before it ships, encouraged by evidence that profile-conditioned LLMs mimic human samples. But is that prediction actually valid against real behaviour - and does the persona machinery help? We present a sim-to-real validity study using the Upworthy Research Archive - thousands of headline A/B tests on shared rea","content_hash":null,"source_reliability_class":"medium","is_primary_source":false,"is_independent":false,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-23T04:01:02.819Z"},{"id":"415dede3-f0ae-449a-96ae-ded7f9857c19","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","url":"https://alignment.anthropic.com/2025/strengthening-red-teams/","canonical_url":"https://alignment.anthropic.com/2025/strengthening-red-teams/","source_type":"developer_announcement","publisher":"Anthropic Alignment Science","author":null,"publication_date":"2026-09-24T00:00:00.000Z","retrieved_at":"2026-09-24T23:00:54.604Z","title":"Strengthening Red Teams: A Modular Scaffold for Control Evaluations We decompose sabotage into constituent skills and use synthetic simulations to strengthen attacks in complex environments.","excerpt":"Strengthening Red Teams: A Modular Scaffold for Control Evaluations We decompose sabotage into constituent skills and use synthetic simulations to strengthen attacks in complex environments.","content_hash":null,"source_reliability_class":"medium","is_primary_source":false,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-24T23:00:54.604Z"},{"id":"30bae3ff-fd7a-4a43-a017-b577be6fb4db","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","url":"https://alignment.anthropic.com/2025/modifying-beliefs-via-sdf/","canonical_url":"https://alignment.anthropic.com/2025/modifying-beliefs-via-sdf/","source_type":"developer_announcement","publisher":"Anthropic Alignment Science","author":null,"publication_date":"2026-09-25T00:00:00.000Z","retrieved_at":"2026-09-25T00:00:54.381Z","title":"Modifying LLM Beliefs with Synthetic Document Finetuning We study whether we can modify the beliefs of LLMs and investigate whether doing so could decrease risk from advanced AI systems.","excerpt":"Modifying LLM Beliefs with Synthetic Document Finetuning We study whether we can modify the beliefs of LLMs and investigate whether doing so could decrease risk from advanced AI systems.","content_hash":null,"source_reliability_class":"medium","is_primary_source":false,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-25T00:00:54.381Z"},{"id":"e0bd8f18-36fa-45b9-ab11-3e5641786994","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","url":"https://alignment.anthropic.com/2025/introducing-safeguards-research-team/index.html","canonical_url":"https://alignment.anthropic.com/2025/introducing-safeguards-research-team/index.html","source_type":"developer_announcement","publisher":"Anthropic Alignment Science","author":null,"publication_date":"2026-09-25T00:00:00.000Z","retrieved_at":"2026-09-25T02:00:54.907Z","title":"Introducing Anthropic's Safeguards Research Team We're launching a new research team focused on mitigating the post-deployment risks of AI systems.","excerpt":"Introducing Anthropic's Safeguards Research Team We're launching a new research team focused on mitigating the post-deployment risks of AI systems.","content_hash":null,"source_reliability_class":"medium","is_primary_source":false,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-25T02:00:54.907Z"}],"claims":[{"id":"b2bc6148-16a0-4975-98ee-4870f5984d0e","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","claim_text":"From Intent to Action: Benchmarking LLM Safety in Vehicle Voice Command Authorization","claim_type":"outcome","claim_status":"supported","confidence_score":null,"created_at":"2026-09-18T04:00:58.792Z","updated_at":"2026-09-18T04:00:58.792Z"}],"revisions":[{"id":"93562be8-7fba-429e-b1a0-6afac3ff2bf6","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","model_id":null,"previous_score":"0.000000","new_score":"0.001000","previous_factors":{},"new_factors":{"tier":1,"evidence":0.1,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Auto-published from news ingest.","trigger_type":"news_ingest","trigger_source_ids":null,"reviewer_id":null,"review_status":"published","created_at":"2026-09-18T04:00:58.799Z"},{"id":"79fa9afb-9e90-41e4-bd6e-3aaeced72270","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","model_id":null,"previous_score":"0.001000","new_score":"0.002000","previous_factors":{"tier":1,"evidence":0.1,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"new_factors":{"tier":1,"models":["535014f7-c941-4d39-aa9b-8f9c1d3d07b7"],"evidence":0.2,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Ingest update: independent source raised evidence; added a follow-up source.","trigger_type":"ingest_update","trigger_source_ids":["9b594089-c2fb-49e6-bcaa-0ae3b022053b"],"reviewer_id":null,"review_status":"published","created_at":"2026-09-18T04:00:59.051Z"},{"id":"38584b99-2964-4675-ae7f-58a1e9ccff30","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","model_id":null,"previous_score":"0.002000","new_score":"0.003000","previous_factors":{"tier":1,"evidence":0.2,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"new_factors":{"tier":1,"models":["535014f7-c941-4d39-aa9b-8f9c1d3d07b7"],"evidence":0.3,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Ingest update: independent source raised evidence; added a follow-up source.","trigger_type":"ingest_update","trigger_source_ids":["bd637de7-b964-4967-a609-036b91a99b35"],"reviewer_id":null,"review_status":"published","created_at":"2026-09-23T03:01:07.208Z"},{"id":"c9e546be-0fa0-4d12-948e-36a4c8ca119b","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","model_id":null,"previous_score":"0.003000","new_score":"0.004000","previous_factors":{"tier":1,"evidence":0.3,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"new_factors":{"tier":1,"models":["535014f7-c941-4d39-aa9b-8f9c1d3d07b7"],"evidence":0.4,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Ingest update: independent source raised evidence; added a follow-up source.","trigger_type":"ingest_update","trigger_source_ids":["c6a0afb4-8f1a-49bb-bf6d-e3d34867ce7b"],"reviewer_id":null,"review_status":"published","created_at":"2026-09-23T04:01:02.891Z"},{"id":"59d2be45-a8ba-4042-83f7-ba86a0a3faaa","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","model_id":null,"previous_score":"0.004000","new_score":"0.005000","previous_factors":{"tier":1,"evidence":0.4,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"new_factors":{"tier":1,"models":["535014f7-c941-4d39-aa9b-8f9c1d3d07b7"],"evidence":0.5,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Ingest update: independent source raised evidence; added a follow-up source.","trigger_type":"ingest_update","trigger_source_ids":["415dede3-f0ae-449a-96ae-ded7f9857c19"],"reviewer_id":null,"review_status":"published","created_at":"2026-09-24T23:00:54.625Z"},{"id":"3cfbb175-bb22-45c0-adf5-3e2af0371f68","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","model_id":null,"previous_score":"0.005000","new_score":"0.005000","previous_factors":{"tier":1,"evidence":0.5,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"new_factors":{"tier":1,"models":["535014f7-c941-4d39-aa9b-8f9c1d3d07b7"],"evidence":0.5,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Ingest update: added a follow-up source.","trigger_type":"ingest_update","trigger_source_ids":["30bae3ff-fd7a-4a43-a017-b577be6fb4db"],"reviewer_id":null,"review_status":"published","created_at":"2026-09-25T00:00:54.395Z"},{"id":"da8cdab1-faf9-492c-a1be-ae1e1d1e412d","event_id":"a2b1345f-daae-459d-964c-66336eb7c8ba","model_id":null,"previous_score":"0.005000","new_score":"0.005000","previous_factors":{"tier":1,"evidence":0.5,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"new_factors":{"tier":1,"models":["535014f7-c941-4d39-aa9b-8f9c1d3d07b7"],"evidence":0.5,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Ingest update: added a follow-up source.","trigger_type":"ingest_update","trigger_source_ids":["e0bd8f18-36fa-45b9-ab11-3e5641786994"],"reviewer_id":null,"review_status":"published","created_at":"2026-09-25T02:00:54.947Z"}],"secondary":[]},"methodology_version":"0.1"}