{"data":{"event":{"id":"867a3ff2-d431-4653-b202-6998143b2abd","slug":"closed-world-resolution-against-tool-hallucination-in-llm-agents-5cd3037856","title":"Closed-World Resolution Against Tool Hallucination in LLM Agents","short_summary":"arXiv:2609.19425v1 Announce Type: new \nAbstract: Tool-augmented large language model (LLM) agents fail in a way no tool-selection or tool-security method addresses: they call tools that do not exist and pass arguments no schema declares. Existing defenses either pick the right tool (selection) or constrain what an agent may do with real tools (gating), both of which presuppose the emitted call refers to a real tool at all. We show this is a structural blind spot: a hallucinated call is by construction not a decision any gate made, so no gate can reject it. This paper is primarily a measurement","full_description":"arXiv:2609.19425v1 Announce Type: new \nAbstract: Tool-augmented large language model (LLM) agents fail in a way no tool-selection or tool-security method addresses: they call tools that do not exist and pass arguments no schema declares. Existing defenses either pick the right tool (selection) or constrain what an agent may do with real tools (gating), both of which presuppose the emitted call refers to a real tool at all. We show this is a structural blind spot: a hallucinated call is by construction not a decision any gate made, so no gate can reject it. This paper is primarily a measurement and benchmark study. We give a five-class taxonomy of tool hallucination (H1-H5) and, as a reference point, the Resolution Rung: a training-free, closed-world resolver (registry membership plus a signature check) whose interest is where it must sit, not what it computes. We prove hallucination defense must precede any causal gate, and characterize the one irreducible residue (borrowed arguments schema-indistinguishable from a valid call). Across ten hosted models under two invocation surfaces we measure 322 genuine hallucinations; fabricated-tool calls concentrate on the unconstrained raw-JSON surface (34 vs. 3), and model scale does not help (a 675B model matches a 7-8B one). We then extend to the Model Context Protocol, where merging several servers into one namespace creates hallucination surfaces a single registry cannot express (a second taxonomy, M1-M5); on the live MCP surface we measure 154 hallucinations, including from frontier models that were clean on the single-registry surface, because collisions and shadowing are structural to the merge. We release the versioned Hallucinated-Tools Benchmark (HTB) so any resolver is comparable across submissions.\n\nUpdate (2026-09-22): Show HN: AI·rete·RAG – a Rete rule engine decides, RAG explains why. Hi HN, I built ai·rete·rag because I kept seeing teams put an LLM in charge of decisions that need to be auditable (lending, fraud, clinical triage), then bolt on \"guardrails\" after the fact.\nIt runs the two in series instead:\n1. A pure-Python Rete engine evaluates YAML rules against your facts. The verdict comes only from here. Same facts, same verdict, every time, with salience-based conflict resolution.\n2. RAG retrieves passages from your own policy documents, and an LLM writes a plain-English explanation of the decision that was already made, citing those passages. It can't change the verdict.\nA few things that went further than I expected:\n- Rules are a graph, not flat lists: nested all/any/not, and rules can assert facts that other rules consume (forward chaining). The decision trace shows the causal chain.\n- Audit mode records every rule evaluated, including the ones that didn't fire, condition by condition, with a snapshot of the rule set for replay.\n- Rules can steer retrieval (a fired rule narrows which documents get searched), and retrieved text can be turned into facts for the en","ledger_type":"benefit","primary_domain_id":"fc7d617f-c9df-4b9f-b595-4408915ea380","event_status":"provisional","event_date":"2026-09-18T00:00:00.000Z","discovery_date":"2026-09-18T00:00:00.000Z","first_published_date":"2026-09-18T00:00:00.000Z","last_reviewed_date":"2026-09-22T00:00:00.000Z","geographic_scope":"International","affected_population":null,"base_impact_tier":1,"base_score":"1.00","attribution_multiplier":"0.1000","evidence_multiplier":"0.2000","realization_multiplier":"0.2000","durability_multiplier":"0.5000","current_event_score":"0.002000","confidence_level":"low","score_explanation":"Ingest update: independent source raised evidence; added a follow-up source. Score remains provisional until review.","methodology_version_id":"8745541b-0f8e-4d5a-9b47-5291d3813a8d","original_methodology_version_id":"8745541b-0f8e-4d5a-9b47-5291d3813a8d","published_at":"2026-09-18T04:00:54.437Z","created_at":"2026-09-18T04:00:54.437Z","updated_at":"2026-09-22T17:01:04.144Z","flags":[],"domain_name":"Other","domain_slug":"other","methodology_version":"0.2"},"contributions":[{"id":"95c36a0a-0960-4888-bbeb-ce0504850941","event_id":"867a3ff2-d431-4653-b202-6998143b2abd","model_id":"535014f7-c941-4d39-aa9b-8f9c1d3d07b7","role_description":"Unspecified system mentioned or implied by a news item. Remap to a named release when identified.","attribution_multiplier":"0.1000","credit_share":"1.0000","contribution_score":"0.002000","attribution_rationale":"News ingest does not infer a named model from the publisher alone. Attribution stays unspecified until a release is identified.","attribution_confidence":"medium","first_used_date":"2026-09-18T00:00:00.000Z","model_version_if_known":null,"review_status":"approved","created_at":"2026-09-18T04:00:54.446Z","model_slug":"unspecified-ai-system","model_name":"Unspecified AI system","identity_class":"unknown","is_internal":false,"family_name":"Unspecified","family_slug":"unknown-unspecified","organization_name":"Unknown","organization_slug":"unknown"}],"sources":[{"id":"4440fe9d-eb7c-4529-89e7-59aa0c5bc5b8","event_id":"867a3ff2-d431-4653-b202-6998143b2abd","url":"https://arxiv.org/abs/2609.19425","canonical_url":"https://arxiv.org/abs/2609.19425","source_type":"preprint","publisher":"arXiv cs.AI","author":null,"publication_date":"2026-09-18T00:00:00.000Z","retrieved_at":"2026-09-18T04:00:54.464Z","title":"Closed-World Resolution Against Tool Hallucination in LLM Agents","excerpt":"arXiv:2609.19425v1 Announce Type: new \nAbstract: Tool-augmented large language model (LLM) agents fail in a way no tool-selection or tool-security method addresses: they call tools that do not exist and pass arguments no schema declares. Existing defenses either pick the right tool (selection) or constrain what an agent may do with real tools (gating), both of which presuppose the emitted call refers to a real tool at all. We show this is a structural blind spot: a hallucinated call is by constr","content_hash":null,"source_reliability_class":"medium","is_primary_source":true,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-18T04:00:54.464Z"},{"id":"82f4b41f-b7ff-48f5-a59d-179ef959843b","event_id":"867a3ff2-d431-4653-b202-6998143b2abd","url":"https://ai-rete-rag.com/","canonical_url":"https://ai-rete-rag.com/","source_type":"journalism","publisher":"Hacker News","author":null,"publication_date":"2026-09-22T00:00:00.000Z","retrieved_at":"2026-09-22T17:01:04.092Z","title":"Show HN: AI·rete·RAG – a Rete rule engine decides, RAG explains why","excerpt":"Hi HN, I built ai·rete·rag because I kept seeing teams put an LLM in charge of decisions that need to be auditable (lending, fraud, clinical triage), then bolt on \"guardrails\" after the fact.\nIt runs the two in series instead:\n1. A pure-Python Rete engine evaluates YAML rules against your facts. The verdict comes only from here. Same facts, same verdict, every time, with salience-based conflict resolution.\n2. RAG retrieves passages from your own policy documents, and an LLM writes a plain-Englis","content_hash":null,"source_reliability_class":"medium","is_primary_source":false,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-22T17:01:04.092Z"}],"claims":[{"id":"8de76ce5-c22a-4d48-b903-aea9120de015","event_id":"867a3ff2-d431-4653-b202-6998143b2abd","claim_text":"Closed-World Resolution Against Tool Hallucination in LLM Agents","claim_type":"outcome","claim_status":"supported","confidence_score":null,"created_at":"2026-09-18T04:00:54.486Z","updated_at":"2026-09-18T04:00:54.486Z"}],"revisions":[{"id":"72a34ef8-8c5b-43e6-8840-b66ba9cceac6","event_id":"867a3ff2-d431-4653-b202-6998143b2abd","model_id":null,"previous_score":"0.000000","new_score":"0.001000","previous_factors":{},"new_factors":{"tier":1,"evidence":0.1,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Auto-published from news ingest.","trigger_type":"news_ingest","trigger_source_ids":null,"reviewer_id":null,"review_status":"published","created_at":"2026-09-18T04:00:54.496Z"},{"id":"3e7f4c21-513c-46a3-b209-c2b52d073890","event_id":"867a3ff2-d431-4653-b202-6998143b2abd","model_id":null,"previous_score":"0.001000","new_score":"0.002000","previous_factors":{"tier":1,"evidence":0.1,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"new_factors":{"tier":1,"models":["535014f7-c941-4d39-aa9b-8f9c1d3d07b7"],"evidence":0.2,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Ingest update: independent source raised evidence; added a follow-up source.","trigger_type":"ingest_update","trigger_source_ids":["82f4b41f-b7ff-48f5-a59d-179ef959843b"],"reviewer_id":null,"review_status":"published","created_at":"2026-09-22T17:01:04.163Z"}],"secondary":[]},"methodology_version":"0.1"}