{"data":{"event":{"id":"6f79fa85-fddf-4344-9b3f-590996006abb","slug":"tau-elicitation-benchmarking-multi-turn-entity-extraction-in-voice-agent-281e2c825c","title":"$\\tau$-Elicitation: Benchmarking multi-turn entity extraction in voice agents","short_summary":"arXiv:2609.13602v1 Announce Type: new \nAbstract: Voice agents often need to collect names, addresses, identifiers, dates, and times exactly, yet end-to-end benchmarks obscure where capture fails. We introduce $\\tau$-Elicitation, a 200-task voice benchmark spanning 10 entity types, controlled difficulty, caller realisms, and three environments. A matched text agent passes all tasks, but four voice configurations achieve robust exact success from 0.14 to 0.41. Agents increase verification for hard and unfamiliar entities and sometimes for incorrect captures, but not for their weakest caller voic","full_description":"arXiv:2609.13602v1 Announce Type: new \nAbstract: Voice agents often need to collect names, addresses, identifiers, dates, and times exactly, yet end-to-end benchmarks obscure where capture fails. We introduce $\\tau$-Elicitation, a 200-task voice benchmark spanning 10 entity types, controlled difficulty, caller realisms, and three environments. A matched text agent passes all tasks, but four voice configurations achieve robust exact success from 0.14 to 0.41. Agents increase verification for hard and unfamiliar entities and sometimes for incorrect captures, but not for their weakest caller voice; only 24 to 37 percent of verified errors are repaired. A scaffold that prescribes spelling, read-back, correction, and confirmation raises robust Pass$^3$ by 14 to 31 points, at a cost of 21 to 28 seconds per call. Realisms such as spelling variations and restarts do not detectably affect exact success; mispronunciation increases repair effort. These results identify strategy selection and successful recovery as the central bottlenecks in exact spoken entity collection.\n\nUpdate (2026-09-19): Google’s Gemini hacked three companies in new AI safety incident - Financial Times. Google’s Gemini hacked three companies in new AI safety incident  Financial Times\n\nUpdate (2026-09-21): GameASG-Bench: Benchmarking Autonomous Software Generation for Game Development. arXiv:2609.21293v1 Announce Type: new \nAbstract: Autonomous software generation (ASG) aims to turn human requirements into executable applications, but delivering these applications does not necessarily establish that their interacting components satisfy the specified behavioral requirements. We introduce GameASG-Bench, a benchmark that makes behavioral testability part of the generation task for game development. Our design declares an evaluation interface specification before generation, fixing legal starting scenarios, player-level actions, stable snapshots, rejection behavior, and invariants while leaving private implementations open. Concretely, we include: (i) static L1 checks that assess source-level compliance; and (ii) browser-executed L2 checks that combine semantic observations with real input and runtime evidence. We implement this protocol as 47 browser-native game-generation tasks spanning 12 primary genres and both 2D and 3D interaction, each with executable checks and an independently verified reference implementation. Our experiments answer four key questions abo","ledger_type":"benefit","primary_domain_id":"fc7d617f-c9df-4b9f-b595-4408915ea380","event_status":"provisional","event_date":"2026-09-16T00:00:00.000Z","discovery_date":"2026-09-16T00:00:00.000Z","first_published_date":"2026-09-16T00:00:00.000Z","last_reviewed_date":"2026-09-21T00:00:00.000Z","geographic_scope":"International","affected_population":null,"base_impact_tier":2,"base_score":"3.00","attribution_multiplier":"0.7500","evidence_multiplier":"0.3000","realization_multiplier":"0.5500","durability_multiplier":"0.5000","current_event_score":"0.185600","confidence_level":"low","score_explanation":"Ingest update: raised magnitude to tier 2; raised attribution; independent source raised evidence; raised realization; added a follow-up source. Score remains provisional until review. Follow-up ingest: independent source raised evidence; added a follow-up source.","methodology_version_id":"8745541b-0f8e-4d5a-9b47-5291d3813a8d","original_methodology_version_id":"8745541b-0f8e-4d5a-9b47-5291d3813a8d","published_at":"2026-09-17T01:00:49.066Z","created_at":"2026-09-17T01:00:49.066Z","updated_at":"2026-09-21T04:01:05.031Z","flags":[],"domain_name":"Other","domain_slug":"other","methodology_version":"0.2"},"contributions":[{"id":"42c66fea-b696-4b03-bbb0-880cd77d1fe9","event_id":"6f79fa85-fddf-4344-9b3f-590996006abb","model_id":"535014f7-c941-4d39-aa9b-8f9c1d3d07b7","role_description":"Unspecified system mentioned or implied by a news item. Remap to a named release when identified.","attribution_multiplier":"0.7500","credit_share":"1.0000","contribution_score":"0.185600","attribution_rationale":"News ingest does not infer a named model from the publisher alone. Attribution stays unspecified until a release is identified.","attribution_confidence":"medium","first_used_date":"2026-09-16T00:00:00.000Z","model_version_if_known":null,"review_status":"approved","created_at":"2026-09-17T01:00:49.075Z","model_slug":"unspecified-ai-system","model_name":"Unspecified AI system","identity_class":"unknown","is_internal":false,"family_name":"Unspecified","family_slug":"unknown-unspecified","organization_name":"Unknown","organization_slug":"unknown"}],"sources":[{"id":"5050d24c-8db0-4456-8505-5b7b43b9e897","event_id":"6f79fa85-fddf-4344-9b3f-590996006abb","url":"https://arxiv.org/abs/2609.13602","canonical_url":"https://arxiv.org/abs/2609.13602","source_type":"preprint","publisher":"arXiv cs.AI","author":null,"publication_date":"2026-09-16T00:00:00.000Z","retrieved_at":"2026-09-17T01:00:49.083Z","title":"$\\tau$-Elicitation: Benchmarking multi-turn entity extraction in voice agents","excerpt":"arXiv:2609.13602v1 Announce Type: new \nAbstract: Voice agents often need to collect names, addresses, identifiers, dates, and times exactly, yet end-to-end benchmarks obscure where capture fails. We introduce $\\tau$-Elicitation, a 200-task voice benchmark spanning 10 entity types, controlled difficulty, caller realisms, and three environments. A matched text agent passes all tasks, but four voice configurations achieve robust exact success from 0.14 to 0.41. Agents increase verification for hard","content_hash":null,"source_reliability_class":"medium","is_primary_source":true,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-17T01:00:49.083Z"},{"id":"409bf615-099c-4404-9bca-9af3aa682ee8","event_id":"6f79fa85-fddf-4344-9b3f-590996006abb","url":"https://news.google.com/rss/articles/CBMihAFBVV95cUxQWS1LS1lPeHExdWFnaXFfQmtzM0ZNWjVDbEhRTUxua1o5TXJwMk9ma1JManNHa2JTQldXU3FQc0ZoT29OYlp2MnhsbzdZdXhJT2YtUmpxODJMaWdDTGJlQ3NGam1XRHdWZmh3Q3RiN083el9jcDhPakxRdTFEdEw0NF9ZdTQ?oc=5","canonical_url":"https://news.google.com/rss/articles/CBMihAFBVV95cUxQWS1LS1lPeHExdWFnaXFfQmtzM0ZNWjVDbEhRTUxua1o5TXJwMk9ma1JManNHa2JTQldXU3FQc0ZoT29OYlp2MnhsbzdZdXhJT2YtUmpxODJMaWdDTGJlQ3NGam1XRHdWZmh3Q3RiN083el9jcDhPakxRdTFEdEw0NF9ZdTQ?oc=5","source_type":"journalism","publisher":"Frontier lab incident news","author":null,"publication_date":"2026-09-19T00:00:00.000Z","retrieved_at":"2026-09-19T05:00:58.447Z","title":"Google’s Gemini hacked three companies in new AI safety incident - Financial Times","excerpt":"Google’s Gemini hacked three companies in new AI safety incident  Financial Times","content_hash":null,"source_reliability_class":"medium","is_primary_source":false,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-19T05:00:58.447Z"},{"id":"33995247-2174-401e-aef5-3b6917c6a959","event_id":"6f79fa85-fddf-4344-9b3f-590996006abb","url":"https://arxiv.org/abs/2609.21293","canonical_url":"https://arxiv.org/abs/2609.21293","source_type":"preprint","publisher":"arXiv cs.AI","author":null,"publication_date":"2026-09-21T00:00:00.000Z","retrieved_at":"2026-09-21T04:01:05.018Z","title":"GameASG-Bench: Benchmarking Autonomous Software Generation for Game Development","excerpt":"arXiv:2609.21293v1 Announce Type: new \nAbstract: Autonomous software generation (ASG) aims to turn human requirements into executable applications, but delivering these applications does not necessarily establish that their interacting components satisfy the specified behavioral requirements. We introduce GameASG-Bench, a benchmark that makes behavioral testability part of the generation task for game development. Our design declares an evaluation interface specification before generation, fixin","content_hash":null,"source_reliability_class":"medium","is_primary_source":false,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-21T04:01:05.018Z"}],"claims":[{"id":"d0867fb1-e2c5-42a2-8484-64c6783e4719","event_id":"6f79fa85-fddf-4344-9b3f-590996006abb","claim_text":"$\\tau$-Elicitation: Benchmarking multi-turn entity extraction in voice agents","claim_type":"outcome","claim_status":"supported","confidence_score":null,"created_at":"2026-09-17T01:00:49.091Z","updated_at":"2026-09-17T01:00:49.091Z"}],"revisions":[{"id":"2f916b3c-aaaa-4e7c-a41d-3d017ab249f9","event_id":"6f79fa85-fddf-4344-9b3f-590996006abb","model_id":null,"previous_score":"0.000000","new_score":"0.001000","previous_factors":{},"new_factors":{"evidence":0.1,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Auto-published from news ingest.","trigger_type":"news_ingest","trigger_source_ids":null,"reviewer_id":null,"review_status":"published","created_at":"2026-09-17T01:00:49.100Z"},{"id":"a7eb84c5-1485-49ed-8698-f7adfd5498a6","event_id":"6f79fa85-fddf-4344-9b3f-590996006abb","model_id":null,"previous_score":"0.001000","new_score":"0.123800","previous_factors":{"tier":1,"evidence":0.1,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"new_factors":{"tier":2,"models":["535014f7-c941-4d39-aa9b-8f9c1d3d07b7"],"evidence":0.2,"base_score":3,"durability":0.5,"attribution":0.75,"realization":0.55},"change_reason":"Ingest update: raised magnitude to tier 2; raised attribution; independent source raised evidence; raised realization; added a follow-up source.","trigger_type":"ingest_update","trigger_source_ids":["409bf615-099c-4404-9bca-9af3aa682ee8"],"reviewer_id":null,"review_status":"published","created_at":"2026-09-19T05:00:58.483Z"},{"id":"68ac3390-4a6b-4869-b38a-96dfc6716c4c","event_id":"6f79fa85-fddf-4344-9b3f-590996006abb","model_id":null,"previous_score":"0.123800","new_score":"0.185600","previous_factors":{"tier":2,"evidence":0.2,"base_score":3,"durability":0.5,"attribution":0.75,"realization":0.55},"new_factors":{"tier":2,"models":["535014f7-c941-4d39-aa9b-8f9c1d3d07b7"],"evidence":0.3,"base_score":3,"durability":0.5,"attribution":0.75,"realization":0.55},"change_reason":"Ingest update: independent source raised evidence; added a follow-up source.","trigger_type":"ingest_update","trigger_source_ids":["33995247-2174-401e-aef5-3b6917c6a959"],"reviewer_id":null,"review_status":"published","created_at":"2026-09-21T04:01:05.038Z"}],"secondary":[]},"methodology_version":"0.1"}