Net Good IndexSubmit a correction

Benefit Ledger · provisional · Biology

ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software

arXiv:2609.17885v1 Announce Type: new Abstract: Computer-use agents that operate through screenshots and simulated actions are advancing rapidly, yet their evaluation remains anchored to general desktop and web tasks. Enterprise Resource Planning (ERP) systems run the finance, procurement, inventory, and customer operations of organizations worldwide, and pose distinct challenges for computer-use agents: dense interfaces, coordinated multi-step interactions, and errors that alter persistent business records rather than surfacing on screen. Existing enterprise benchmarks rely on proprietary pl

Tier 1 UsefulMethodology 0.2
Submit a correction

Current score

+0.00

1 base · Useful (tier 1 of 5, 1 pts)
× 0.1000 attribution · Minor documented assistance
× 0.2000 evidence · Developer announcement
× 0.2000 realization · Proposed
× 0.5000 durability · Medium-term
Event-level product before credit split: 0.00

Ingest update: independent source raised evidence; named GPT-4o; added a follow-up source. Score remains provisional until review.

What happened

arXiv:2609.17885v1 Announce Type: new Abstract: Computer-use agents that operate through screenshots and simulated actions are advancing rapidly, yet their evaluation remains anchored to general desktop and web tasks. Enterprise Resource Planning (ERP) systems run the finance, procurement, inventory, and customer operations of organizations worldwide, and pose distinct challenges for computer-use agents: dense interfaces, coordinated multi-step interactions, and errors that alter persistent business records rather than surfacing on screen. Existing enterprise benchmarks rely on proprietary platforms or on simulated approximations of such software. We introduce ERPBench, a benchmark that evaluates screenshot-only agents on a live and reproducible ERP system and scores each task against ground-truth values in its database. Beyond the benchmark, we present a production-grade harness that gates agent actions behind human approval for safe deployment, which ERPBench runs autonomously. Evaluating six closed and open-source agents, we demonstrate that strong general GUI performance does not transfer to enterprise reliability. Even when an agent reaches the right form and saves it, the stored record is often wrong: some agents save in up to 85% of runs but write the correct value in as few as 3%. We further characterize failure modes specific to enterprise workflows. Update (2026-09-17): When to Call an LLM: A Confidence-Gated Hybrid for Cost-Effective Emotion Recognition in Conversational AI. arXiv:2609.17977v1 Announce Type: new Abstract: Emotion recognition in conversation (ERC) is a production capability behind agent-assist prompts, escalation routing, and post-call analytics in contact-center-as-a-service (CCaaS) platforms, where cost and latency constraints matter as much as accuracy. We report a systems-level comparison of three deployment options for dialogue-contextual ERC: a low-cost stacked ensemble (sentence embeddings, windowed context, RandomForest/XGBoost/logistic-regression stacking), off-the-shelf LLM prompting (GPT-4o-mini; zero-shot, few-shot, chain-of-thought), and a confidence-gated hybrid that escalates only the ensemble's least-confident predictions to the LLM - modeled on IVA-to-human-agent escalation policies used in production contact centers. On IEMOCAP, the ensemble significantly outperforms every LLM configuration (0.595 vs. 0.460-0.536 weighted F1, p < 0.0001) at a fraction of the cost and sub-10ms latency; on MELD and CMU-MOSI the ranking reverses, showing neither pure system is a safe default. The confidence-g

Model attribution

+0.00

GPT

Remapped from unspecified after ingest named GPT-4o.

Follow-up source named GPT-4o.

Attribution 0.1000 · Credit share 100% · OpenAI

Claims

  • ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software

    outcome · supported

Sources

Revision history

  • 17 Sep 2026 · 0.00 → 0.00

    Auto-published from news ingest.

  • 17 Sep 2026 · 0.00 → 0.00

    Ingest update: independent source raised evidence; named GPT-4o; added a follow-up source.