Zishan Xu1*, Enjun Du2,3*, Mingqi Gao4*, Cheng Yang3, Junhao Shen5, Zirong Chen4, Zhiyuan Yao6, Yifu Guo7, Yuquan Lu7, Jinyang Huang8, Zhengxi Lu6, Siyi Liu3, Ziyu Zheng2, Yongqi Zhang3†, Difan Zou2†, Weiwen Liu1†
1Shanghai Jiao Tong University 2The University of Hong Kong 3The Hong Kong University of Science and Technology (Guangzhou) 4Tsinghua University
5The Chinese University of Hong Kong 6Zhejiang University 7Sun Yat-sen University 8Central South University
*Equal contribution †Corresponding authors
Work Agents are AI systems that combine foundation models with runtime harnesses to carry out delegated digital work in stateful workspaces. They use tools, interact with applications, and coordinate workflows to produce answers, editable artifacts, and changes to workspace state.
This repository is the companion reading list for Work Agents: Systems, Capabilities, and Optimization. It brings together research across office productivity, enterprise operations, software engineering, scientific research, and computer use:
- Agents, methods, and systems — architectures, training methods, environments, memory, and agent harnesses.
- Benchmarks and evaluation — task suites and evaluation methods for digital work.
- Surveys and resources — foundational studies, open-source tools, and official documentation.
Read the survey on arXiv, and visit the project homepage for the project overview.
- Agents, Methods & Systems
- Benchmarks & Evaluation
- Surveys & Foundations
- Resources
- Citation
- Contributing
Agents, learning methods, training environments, and runtime systems for producing work outcomes and improving capability. 278 entries.
| Paper | Year | Focus | Links |
|---|---|---|---|
| ACE: Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models [236] | 2026 | Learning & agent harnesses | Paper · Code · Publication |
| ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization [257] | 2026 | Learning & agent harnesses | Paper |
| Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams [258] | 2026 | Learning & agent harnesses | Paper · Code |
| ADMIRE: Adaptive Milestone Reward for GUI Agents [205] | 2026 | Web, desktop & mobile | Paper |
| AeSlides: Incentivizing Aesthetic Layout in LLM-Based Slide Generation via Verifiable Rewards [384] | 2026 | Spreadsheets, slides & documents | Paper · Code · Data |
| AEvo: Harnessing Agentic Evolution [249] | 2026 | Learning & agent harnesses | Paper · Publication |
| Agent Lightning v1.0: Towards Harnessed Agentic RL [260] | 2026 | Learning & agent harnesses | Paper |
| AgentMercury: Your Agent Can Synthesize Verifiable Environments for Business Scenarios at scale [163] | 2026 | Tools & business workflows | Paper · Data · Website |
| Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning [162] | 2026 | Tools & business workflows | Paper · Code · Publication · Data |
| Anchor: Mitigating Artifact Drift in Agent Benchmark Generation [174] | 2026 | Verification & recovery | Paper · Code · Website |
| Atomix: Timely, Transactional Tool Use for Reliable Agentic Workflows [382] | 2026 | Verification & recovery | Paper · Code · Publication |
| AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design [232] | 2026 | Spreadsheets, slides & documents | Paper · Code · Data · Website |
| AutoRPA: Efficient GUI Automation through LLM-Driven Code Synthesis from Interactions | 2026 | Learning & agent harnesses | Paper |
| Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation [327] | 2026 | Learning & agent harnesses | Paper |
| ChronoMem: Version Control and Semantic Rollback for Large Language Model Agent Memory [359] | 2026 | Memory & persistent work | Paper |
| CODESKILL: Learning Self-Evolving Skills for Coding Agents [239] | 2026 | Software, data & research | Paper |
| CogEvol: Towards Efficient and Reliable Learning Environment Generation [179] | 2026 | Spreadsheets, slides & documents | Paper · Code |
| Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents [263] | 2026 | Learning & agent harnesses | Paper |
| Component Routing: Not All Experience Belongs in the Weights: Component Routing for Self-Improving GUI Agents [274] | 2026 | Web, desktop & mobile | Paper |
| CompoWorld: Compositional Environment Scaling for General Agents [165] | 2026 | Learning & agent harnesses | Paper |
| ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents [350] | 2026 | Web, desktop & mobile | Paper · Publication |
| CorpGen: Simulating Corporate Environments with Autonomous Digital Employees in Multi-Horizon Task Environments | 2026 | Memory & persistent work | Paper |
| CoworkEnv: Synthesizing Training Environments for Long-Horizon Cowork Agents [166] | 2026 | Workspaces & occupational work | Paper · Code |
| Cowork-Gym: Synthesizing Interactive Work Environments at Scale [167] | 2026 | Workspaces & occupational work | Paper |
| CurateEvo: Data-Curation Evolving for Agentic Post-Training [173] | 2026 | Learning & agent harnesses | Paper |
| C-World: A Computer Use Agent Environment Creator | 2026 | Learning & agent harnesses | Paper · Code |
| daVinci-Dev: Agent-native Mid-training for Software Engineering [169] | 2026 | Software, data & research | Paper · Code · Publication · Data |
| daVinci-Env / OpenSWE: daVinci-Env: Open SWE Environment Synthesis at Scale [158] | 2026 | Software, data & research | Paper · Code |
| DeepPresenter: Environment-Grounded Reflection for Agentic Presentation Generation [58] | 2026 | Spreadsheets, slides & documents | Paper · Code · Publication |
| DMRL: Document-Mediated Reinforcement Learning for Skill Optimization in Advertising Recommendation [268] | 2026 | Tools & business workflows | Paper |
| Dream-RSI: Recursive Self-Improvement through Evolving Worlds [2] | 2026 | Learning & agent harnesses | Paper |
| DSec: DeepSeek Elastic Compute (DSec): A Sandbox Infrastructure for Effective Agentic Training at Scale [352] | 2026 | Learning & agent harnesses | Paper |
| EnterpriseLab: A Full-Stack Platform for developing and deploying agents in Enterprises [171] | 2026 | Tools & business workflows | Paper · Code · Website |
| EnvCraft: Synthesizing Executable Environments in Agentic RL for Claw-like Agent [164] | 2026 | Workspaces & occupational work | Paper · Code · Data |
| EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis | 2026 | Tools & business workflows | Paper · Code |
| FST: Learning, Fast and Slow: Towards LLMs That Adapt Continually [262] | 2026 | Learning & agent harnesses | Paper · Publication · Website |
| Function-Aware FIM: Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models [191] | 2026 | Software, data & research | Paper · Code |
| GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning [229] | 2026 | Learning & agent harnesses | Paper · Code · Publication · Website |
| GHD / The Next Screenshot Knows: The Next Screenshot Knows: Gated Hindsight Distillation for Mobile GUI Agents [216] | 2026 | Web, desktop & mobile | Paper |
| GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis [143] | 2026 | Workspaces & occupational work | Paper · Data |
| Guided-OPD: On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents [211] | 2026 | Learning & agent harnesses | Paper · Code |
| HiPER: Hierarchical Plan--Execute RL for Multi-Turn LLM Agents [206] | 2026 | Learning & agent harnesses | Paper · Code · Publication · Website |
| InfiAgent: An Infinite-Horizon Framework for General-Purpose Autonomous Agents [55] | 2026 | Memory & persistent work | Paper · Code · Publication |
| JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution [233] | 2026 | Learning & agent harnesses | Paper · Code · Data · Website |
| JTPRO: A Joint Tool-Prompt Reflective Optimization Framework for Language Agents [227] | 2026 | Learning & agent harnesses | Paper · Publication · Cited edition |
| LiteGUI: Lightweight GUI Agents via Multi-Solution Guided Distillation and Dual-Level Reinforcement Learning [215] | 2026 | Web, desktop & mobile | Paper |
| Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning [363] | 2026 | Memory & persistent work | Paper |
| MESH-Harness: Self-Improving Agent Harnesses via Bandit-Guided Compositional Evolution [242] | 2026 | Learning & agent harnesses | Paper |
| Meta-Harness: End-to-End Optimization of Model Harnesses [231] | 2026 | Learning & agent harnesses | Paper · Code · Website |
| MiLDEdit / MiLDEAgent: MiLDEdit: Reasoning-Based Multi-Layer Design Document Editing [113] | 2026 | Spreadsheets, slides & documents | Paper |
| Nemotron midtraining: Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe [189] | 2026 | Learning & agent harnesses | Paper |
| NexForge: Scaling Agent Capabilities through Requirement-Driven Task Synthesis for LLMs [159] | 2026 | Workspaces & occupational work | Paper · Website |
| OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution | 2026 | Web, desktop & mobile | Paper |
| On-Policy Distillation: Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe | 2026 | Memory & persistent work | Paper · Code |
| OpenAgent: Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use | 2026 | Tools & business workflows | Paper · Code |
| OpenThoughts-Agent: Data Recipes for Agentic Models [172] | 2026 | Learning & agent harnesses | Paper · Code · Data |
| Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewards [273] | 2026 | Learning & agent harnesses | Paper · Publication |
| PRISM / Beyond Prompts: Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses [228] | 2026 | Learning & agent harnesses | Paper · Code · Data |
| Process Reward Models That Think [181] | 2026 | Verification & recovery | Paper · Publication |
| Provider Digital Twins: Developing provider digital twins for personalized provider-patient communication via a RAG-based conversational framework [340] | 2026 | Personal agents & digital twins | Paper |
| RADAR: Automating Low-Risk Code Review at Meta: RADAR, Risk Calibration, and Review Efficiency | 2026 | Verification & recovery | Paper |
| ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory [235] | 2026 | Learning & agent harnesses | Paper · Code · Publication |
| RePro: Retrospective Progress-Aware Self-Refinement for LLM Agent Training [223] | 2026 | Learning & agent harnesses | Paper |
| ReTool: Reinforcement Learning for Strategic Tool Use in LLMs [200] | 2026 | Tools & business workflows | Paper · Publication · Cited edition |
| Revisiting DAgger in the Era of LLM-Agents [213] | 2026 | Learning & agent harnesses | Paper |
| RHO: Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference [256] | 2026 | Learning & agent harnesses | Paper · Code |
| Rollback-Induced Reflection: Rollback the World, Keep the Reflection: Rollback-Induced Reflection for Long-Horizon LLM Agents | 2026 | Verification & recovery | Paper |
| SDAR: Self-Distilled Agentic Reinforcement Learning [222] | 2026 | Learning & agent harnesses | Paper |
| SDPO: Reinforcement Learning via Self-Distillation [220] | 2026 | Learning & agent harnesses | Paper · Code · Publication · Data · Website |
| SEE: Structure-aware Exploring & Exploiting for Long-horizon GUI Agent Trajectory Synthesis [177] | 2026 | Web, desktop & mobile | Paper · Code |
| Self-Healing Orchestrators: Self-Healing Agentic Orchestrators for Reliable Tool-Augmented Large Language Model Systems | 2026 | Verification & recovery | Paper |
| SIA: Self Improving AI with Harness & Weight Updates [267] | 2026 | Learning & agent harnesses | Paper · Code |
| Skill2Env: Capability-Oriented Environment Synthesis from Skills for General Agents [355] | 2026 | Workspaces & occupational work | Paper · Code · Data |
| Skill-Alpha: Progressive Agent Skill Generation via Reinforcement Learning [238] | 2026 | Learning & agent harnesses | Paper · Code |
| SkillOpt: Executive Strategy for Self-Evolving Agent Skills [237] | 2026 | Learning & agent harnesses | Paper · Publication · Code · Website |
| SkillOS: Learning Skill Curation for Self-Evolving Agents | 2026 | Learning & agent harnesses | Paper |
| SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning [265] | 2026 | Learning & agent harnesses | Paper · Code · Publication · Data |
| Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents [221] | 2026 | Learning & agent harnesses | Paper · Publication |
| SkillSmith: Co-Evolving Skills and Tools for Self-Improving Agent Systems [240] | 2026 | Learning & agent harnesses | Paper |
| SlideRL / Learning to Present: Learning to Present: Inverse Specification Rewards for Agentic Slide Generation [134] | 2026 | Spreadsheets, slides & documents | Paper · Code · Data |
| SLIM: Dynamic Skill Lifecycle Management for Agentic Reinforcement Learning [266] | 2026 | Learning & agent harnesses | Paper · Code |
| SMITH: Joint Optimization of Tool Creation and Use for Large Language Model Agents [271] | 2026 | Learning & agent harnesses | Paper · Code · Publication |
| Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills [269] | 2026 | Software, data & research | Paper |
| SOD: Step-wise On-policy Distillation for Small Language Model Agents [214] | 2026 | Learning & agent harnesses | Paper · Code |
| SpreadsheetAgent: Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format Reasoning [41] | 2026 | Reasoning, evidence & work impact | Paper · Code · Publication |
| Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learning [129] | 2026 | Spreadsheets, slides & documents | Paper · Code · Data · Website |
| SRD: Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight [224] | 2026 | Learning & agent harnesses | Paper · Code |
| SSR (Self-play SWE-RL): Toward Training Superintelligent Software Agents through Self-Play SWE-RL | 2026 | Learning & agent harnesses | Paper · Publication |
| StagedWorkspace: A Versioned Workspace for Knowledge-Work Agents | 2026 | Workspaces & occupational work | Paper |
| StateGuard: Analytical-State Management with Validity-Aware Intervention for Long-Horizon Data Agents [375] | 2026 | Memory & persistent work | Paper |
| TACO: A Self-Evolving Framework for Efficient Terminal Agents via Observational Context Compression [225] | 2026 | Learning & agent harnesses | Paper · Code · Publication |
| TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agents [210] | 2026 | Learning & agent harnesses | Paper · Code · Publication |
| TDScaling: Beyond Quantity: Trajectory Diversity Scaling for Code Agents [176] | 2026 | Tools & business workflows | Paper · Publication |
| Tool Description Rewriter: Learning to Rewrite Tool Descriptions for Reliable LLM-Agent Tool Use [226] | 2026 | Learning & agent harnesses | Paper |
| Trace2Skill (Office): Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills | 2026 | Learning & agent harnesses | Paper · Code |
| TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Training [190] | 2026 | Learning & agent harnesses | Paper |
| UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization [259] | 2026 | Web, desktop & mobile | Paper · Publication · Code |
| UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations [198] | 2026 | Web, desktop & mobile | Paper |
| UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agents [264] | 2026 | Web, desktop & mobile | Paper · Website |
| UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning [70] | 2026 | Web, desktop & mobile | Paper |
| VERA: Scaling Verifiable Environments for Agentic co-Evolution [175] | 2026 | Verification & recovery | Paper · Code |
| Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failures [377] | 2026 | Verification & recovery | Paper |
| VERSE: Verified Self-Evolving Optimizer for Agent Harnesses [270] | 2026 | Learning & agent harnesses | Paper · Code |
| VIGA: Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning [130] | 2026 | CAD & visual design | Paper · Publication · Code · Data · Website |
| Who Verifies the Verifier? Co-Evolving Inspectable Graders with Self-Improving Agents [385] | 2026 | Verification & recovery | Paper |
| WorkBorne: Discovering Agentic Tasks from Real-World Artifact Transitions [161] | 2026 | Workspaces & occupational work | Paper |
| WorkForge: Scaling Verifiable Environments for Long-horizon Work Agents [160] | 2026 | Workspaces & occupational work | Paper |
| WorkGenesis: Building the Worlds That Teach Agents to Work [144] | 2026 | Workspaces & occupational work | Paper |
| WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution [168] | 2026 | Workspaces & occupational work | Paper |
| Workspace Optimization / DreamTeam: Workspace Optimization: How to Train Your Agent | 2026 | Learning & agent harnesses | Paper |
| ADAS: Automated Design of Agentic Systems [248] | 2025 | Learning & agent harnesses | Paper · Code · Publication |
| AFlow: Automating Agentic Workflow Generation [230] | 2025 | Learning & agent harnesses | Paper · Code · Publication |
| Agent-as-a-Judge: Evaluate Agents with Agents [15] | 2025 | Verification & recovery | Paper · Publication · Code · Data |
| AgentDebug: Where LLM Agents Fail and How They can Learn From Failures | 2025 | Verification & recovery | Paper · Code |
| AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation | 2025 | Learning & agent harnesses | Paper · Publication |
| AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning [356] | 2025 | Learning & agent harnesses | Paper · Code · Publication |
| Agent Lightning: Train ANY AI Agents with Reinforcement Learning [351] | 2025 | Learning & agent harnesses | Paper · Code |
| Agent-R1: A Unified and Modular Framework for Agentic Reinforcement Learning | 2025 | Learning & agent harnesses | Paper · Code |
| Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents [52] | 2025 | Web, desktop & mobile | Paper · Code · Publication |
| AgentSquare: Automatic LLM Agent Search in Modular Design Space [241] | 2025 | Learning & agent harnesses | Paper · Code · Publication |
| AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials [170] | 2025 | Web, desktop & mobile | Paper |
| A-MEM: Agentic Memory for LLM Agents [362] | 2025 | Memory & persistent work | Paper |
| Authenticated Delegation and Authorized AI Agents [341] | 2025 | Safety & authorization | Paper |
| AutoPresent / SlidesBench: AutoPresent: Designing Structured Visuals from Scratch [120] | 2025 | Spreadsheets, slides & documents | Paper · Code · Publication · Cited edition |
| AWM: Agent Workflow Memory [369] | 2025 | Learning & agent harnesses | Paper · Code · Publication |
| BrowserGym: The BrowserGym Ecosystem for Web Agent Research [320] | 2025 | Learning & agent harnesses | Paper · Code · Publication · Website |
| CoAct-1: Computer-using Multi-Agent System with Coding Actions | 2025 | Learning & agent harnesses | Paper · Code |
| ColPali / ViDoRe: ColPali: Efficient Document Retrieval with Vision Language Models [147] | 2025 | Reasoning, evidence & work impact | Paper · Code |
| DART-GUI: Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation | 2025 | Learning & agent harnesses | Paper |
| Data Interpreter: An LLM Agent For Data Science [51] | 2025 | Software, data & research | Paper · Code · Publication |
| DGAP: Discriminator-Guided Embodied Planning for LLM Agent | 2025 | Reasoning, evidence & work impact | Paper |
| Docling Toolkit: Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion | 2025 | Tools & business workflows | Paper |
| DynaSaur: Large Language Agents Beyond Predefined Actions | 2025 | Learning & agent harnesses | Paper · Code · Publication |
| GiGPO: Group-in-Group Policy Optimization for LLM Agent Training [207] | 2025 | Learning & agent harnesses | Paper |
| GoalAct: Enhancing LLM-Based Agents via Global Planning and Hierarchical Execution | 2025 | Reasoning, evidence & work impact | Paper |
| GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents [68] | 2025 | Web, desktop & mobile | Paper · Code · Publication |
| HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model | 2025 | Reasoning, evidence & work impact | Paper |
| LocAgent: Graph-Guided LLM Agents for Code Localization [151] | 2025 | Software, data & research | Paper · Publication |
| MAP: A brain-inspired agentic architecture to improve planning with LLMs | 2025 | Reasoning, evidence & work impact | Paper |
| Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory [360] | 2025 | Memory & persistent work | Paper · Code |
| MLGym: A New Framework and Benchmark for Advancing AI Research Agents | 2025 | Software, data & research | Paper · Code |
| OpenHands: An Open Platform for AI Software Developers as Generalist Agents [152] | 2025 | Learning & agent harnesses | Paper · Code · Publication |
| Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks | 2025 | Reasoning, evidence & work impact | Paper |
| PosterGen: Aesthetic-Aware Multi-Modal Paper-to-Poster Generation via Multi-Agent LLMs | 2025 | Spreadsheets, slides & documents | Paper |
| PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slides [5] | 2025 | Spreadsheets, slides & documents | Paper · Code · Publication |
| PPTArena: A Benchmark for PowerPoint Editing | 2025 | Spreadsheets, slides & documents | Paper · Code |
| PreGenie: An Agentic Framework for High-quality Visual Presentation Generation | 2025 | Spreadsheets, slides & documents | Paper · Publication |
| Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance [332] | 2025 | Memory & persistent work | Paper · Code |
| R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning | 2025 | Learning & agent harnesses | Paper · Code |
| R2E-Gym: Procedural Environment Generation and Hybrid Verifiers for Scaling Open-Weights SWE Agents [354] | 2025 | Software, data & research | Paper · Publication · Publication |
| RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph | 2025 | Software, data & research | Paper · Publication |
| Routine: A Structural Planning Framework for LLM Agent System in Enterprise [48] | 2025 | Tools & business workflows | Paper |
| SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planning [381] | 2025 | Memory & persistent work | Paper · Publication |
| Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning [201] | 2025 | Learning & agent harnesses | Paper · Code · Publication |
| SheetAgent: Towards A Generalist Agent for Spreadsheet Reasoning and Manipulation via Large Language Models [45] | 2025 | Spreadsheets, slides & documents | Paper · Publication · Cited edition |
| SheetMind: Actions Set Accuracy, Agents Set the Failure Mode | 2025 | Spreadsheets, slides & documents | Paper |
| SlideTailor: Personalized Presentation Slide Generation for Scientific Papers | 2025 | Spreadsheets, slides & documents | Paper |
| SWE-Gym: Training Software Engineering Agents and Verifiers with SWE-Gym [353] | 2025 | Software, data & research | Paper · Code · Publication |
| SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution [204] | 2025 | Learning & agent harnesses | Paper · Code · Publication |
| SWE-smith: Scaling Data for Software Engineering Agents | 2025 | Learning & agent harnesses | Paper · Publication |
| SyncMind: Measuring Agent Out-of-Sync Recovery in Collaborative Software Engineering | 2025 | Software, data & research | Paper · Code · Data |
| Talk to Your Slides: High-Efficiency Slide Editing via Language-Driven Structured Data Manipulation | 2025 | Spreadsheets, slides & documents | Paper |
| TextGrad: Optimizing generative AI by backpropagating language model feedback [251] | 2025 | Learning & agent harnesses | Paper · Code · Publication · Cited edition |
| ToolMaker: LLM Agents Making Agent Tools [272] | 2025 | Learning & agent harnesses | Paper |
| Tool-to-Agent Retrieval: Bridging Tools and Agents for Scalable LLM Multi-Agent Systems | 2025 | Tools & business workflows | Paper |
| ToRL: Scaling Tool-Integrated RL | 2025 | Learning & agent harnesses | Paper · Code |
| UFO2: The Desktop AgentOS | 2025 | Web, desktop & mobile | Paper · Code |
| UGround: Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents [67] | 2025 | Web, desktop & mobile | Paper · Publication |
| UI-S1: Advancing GUI Automation via Semi-Online Reinforcement Learning [71] | 2025 | Web, desktop & mobile | Paper |
| UI-TARS: Pioneering Automated GUI Interaction with Native Agents [197] | 2025 | Web, desktop & mobile | Paper · Code |
| UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action | 2025 | Learning & agent harnesses | Paper |
| WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning | 2025 | Learning & agent harnesses | Paper · Code · Publication |
| WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning [202] | 2025 | Web, desktop & mobile | Paper · Code · Publication |
| WorkTeam: Constructing Workflows from Natural Language with Multi-Agents [49] | 2025 | Learning & agent harnesses | Paper · Publication |
| Zep: A Temporal Knowledge Graph Architecture for Agent Memory [358] | 2025 | Memory & persistent work | Paper · Code |
| Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models [194] | 2024 | Learning & agent harnesses | Paper · Code · Publication |
| AgentGym: Evolving Large Language Model-based Agents across Diverse Environments | 2024 | Learning & agent harnesses | Paper · Code |
| AgentInstruct (Microsoft): AgentInstruct: Toward Generative Teaching with Agentic Flows | 2024 | Learning & agent harnesses | Paper |
| Agentless: Demystifying LLM-based Software Engineering Agents | 2024 | Software, data & research | Paper · Code |
| AgentOptimizer: Offline Training of Language Model Agents with Functions as Learnable Weights | 2024 | Learning & agent harnesses | Paper · Publication |
| Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents [219] | 2024 | Web, desktop & mobile | Paper |
| Agent S: An Open Agentic Framework that Uses Computers Like a Human | 2024 | Web, desktop & mobile | Paper · Code |
| AgentTuning: Enabling Generalized Agent Abilities for LLMs [193] | 2024 | Learning & agent harnesses | Paper · Code · Publication |
| AsyncLM: Asynchronous LLM Function Calling [365] | 2024 | Learning & agent harnesses | Paper |
| AutoCodeRover: Autonomous Program Improvement [43] | 2024 | Software, data & research | Paper · Publication |
| AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversations [53] | 2024 | Learning & agent harnesses | Paper · Code · Publication |
| BetterTogether: Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together [261] | 2024 | Learning & agent harnesses | Paper · Publication |
| BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement [85] | 2024 | CAD & visual design | Paper · Code · Data |
| BPO: Black-Box Prompt Optimization: Aligning Large Language Models without Model Training [245] | 2024 | Learning & agent harnesses | Paper · Code · Publication |
| Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding | 2024 | Tools & business workflows | Paper |
| CodeAct: Executable Code Actions Elicit Better LLM Agents [46] | 2024 | Tools & business workflows | Paper · Code · Publication |
| CogAgent: A Visual Language Model for GUI Agents [8] | 2024 | Web, desktop & mobile | Paper |
| CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing [57] | 2024 | Verification & recovery | Paper · Code |
| Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflow | 2024 | Tools & business workflows | Paper · Publication |
| DesiGen: A Pipeline for Controllable Design Template Generation | 2024 | Spreadsheets, slides & documents | Paper |
| DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning [203] | 2024 | Web, desktop & mobile | Paper · Code · Publication · Cited edition |
| Docling Technical Report | 2024 | Reasoning, evidence & work impact | Paper · Code |
| DocLLM: A Layout-Aware Generative Language Model for Multimodal Document Understanding [183] | 2024 | Reasoning, evidence & work impact | Paper |
| DSPy: Compiling Declarative Language Model Calls into State-of-the-Art Pipelines [246] | 2024 | Learning & agent harnesses | Paper · Code · Publication |
| EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers [253] | 2024 | Learning & agent harnesses | Paper · Code · Publication |
| ExpeL: LLM Agents Are Experiential Learners [367] | 2024 | Memory & persistent work | Paper · Code · Publication · Cited edition |
| GKD: On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes [209] | 2024 | Learning & agent harnesses | Paper · Publication |
| Gorilla: Large Language Model Connected with Massive APIs [26] | 2024 | Learning & agent harnesses | Paper · Publication |
| Graph-enhanced Asynchronous Planning: Graph-enhanced Large Language Models in Asynchronous Plan Reasoning [366] | 2024 | Tools & business workflows | Paper |
| Human Digital Twin Architecture: A Human Digital Twin Architecture for Knowledge-based Interactions and Context-Aware Conversations [337] | 2024 | Personal agents & digital twins | Paper |
| LATS: Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models [61] | 2024 | Verification & recovery | Paper · Code · Publication |
| Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks [335] | 2024 | Learning & agent harnesses | Paper · Code |
| mEdIT: Multilingual Text Editing via Instruction Tuning | 2024 | Learning & agent harnesses | Paper |
| MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework [54] | 2024 | Software, data & research | Paper · Code · Publication |
| MinerU: An Open-Source Solution for Precise Document Content Extraction [89] | 2024 | Reasoning, evidence & work impact | Paper · Code |
| MIPRO: Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs [247] | 2024 | Learning & agent harnesses | Paper · Publication |
| M-LongDoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Framework | 2024 | Reasoning, evidence & work impact | Paper |
| mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding | 2024 | Reasoning, evidence & work impact | Paper |
| mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding | 2024 | Reasoning, evidence & work impact | Paper · Code |
| NL2Formula: Generating Spreadsheet Formulas from Natural Language Queries [106] | 2024 | Spreadsheets, slides & documents | Paper |
| OmniParser for Pure Vision Based GUI Agent [69] | 2024 | Web, desktop & mobile | Paper |
| OPRO: Large Language Models as Optimizers [244] | 2024 | Learning & agent harnesses | Paper · Code · Publication |
| OS-Copilot: Towards Generalist Computer Agents with Self-Improvement [74] | 2024 | Web, desktop & mobile | Paper · Publication |
| PDFTriage: Question Answering over Long, Structured Documents [97] | 2024 | Reasoning, evidence & work impact | Paper · Publication |
| Prometheus: Inducing Fine-grained Evaluation Capability in Language Models [319] | 2024 | Verification & recovery | Paper |
| PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization | 2024 | Learning & agent harnesses | Paper · Code · Publication |
| Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution [254] | 2024 | Learning & agent harnesses | Paper · Publication |
| SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents [72] | 2024 | Web, desktop & mobile | Paper · Code |
| ShowUI: One Vision-Language-Action Model for GUI Visual Agent | 2024 | Web, desktop & mobile | Paper |
| SpreadsheetLLM: Encoding Spreadsheets for Large Language Models [42] | 2024 | Spreadsheets, slides & documents | Paper · Publication |
| STORM: Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models | 2024 | Spreadsheets, slides & documents | Paper |
| SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering [3] | 2024 | Software, data & research | Paper · Publication · Cited edition · Code · Website |
| TableGPT2: A Large Multimodal Model with Tabular Data Integration | 2024 | Learning & agent harnesses | Paper |
| TableLlama: Towards Open Large Generalist Models for Tables [88] | 2024 | Learning & agent harnesses | Paper |
| Text2Chart31: Instruction Tuning for Chart Generation with Automatic Feedback | 2024 | Spreadsheets, slides & documents | Paper |
| ToolLLM / ToolBench: ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs | 2024 | Tools & business workflows | Paper · Code · Publication |
| Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMs [255] | 2024 | Learning & agent harnesses | Paper · Publication |
| UFO: A UI-Focused Agent for Windows OS Interaction | 2024 | Web, desktop & mobile | Paper · Code |
| Voyager: An Open-Ended Embodied Agent with Large Language Models [368] | 2024 | Memory & persistent work | Paper · Publication · Cited edition |
| WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models [73] | 2024 | Web, desktop & mobile | Paper · Publication |
| Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examples | 2023 | Spreadsheets, slides & documents | Paper |
| CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society | 2023 | Learning & agent harnesses | Paper |
| ChatDev: Communicative Agents for Software Development | 2023 | Software, data & research | Paper |
| CoEdIT: Text Editing by Task-Specific Instruction Tuning | 2023 | Spreadsheets, slides & documents | Paper |
| Dater: Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning | 2023 | Reasoning, evidence & work impact | Paper |
| FireAct: Toward Language Agent Fine-tuning [195] | 2023 | Tools & business workflows | Paper |
| FormaT5: Abstention and Examples for Conditional Table Formatting with Natural Language [107] | 2023 | Spreadsheets, slides & documents | Paper · Publication |
| Generative Agents: Interactive Simulacra of Human Behavior [361] | 2023 | Memory & persistent work | Paper · Publication |
| G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment [318] | 2023 | Verification & recovery | Paper · Publication |
| Hi-VT5 / MP-DocVQA: Hierarchical Multimodal Transformers for Multipage DocVQA [96] | 2023 | Reasoning, evidence & work impact | Paper |
| HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face | 2023 | Tools & business workflows | Paper · Code |
| LayoutGPT: Compositional Visual Planning and Generation with Large Language Models | 2023 | Reasoning, evidence & work impact | Paper · Code |
| LayoutPrompter: TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering | 2023 | Spreadsheets, slides & documents | Paper |
| LLM+P: Empowering Large Language Models with Optimal Planning Proficiency [47] | 2023 | Tools & business workflows | Paper |
| Medprompt: Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine | 2023 | Learning & agent harnesses | Paper |
| MemGPT: Towards LLMs as Operating Systems [234] | 2023 | Memory & persistent work | Paper · Code |
| mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding | 2023 | Reasoning, evidence & work impact | Paper |
| PAL: Program-aided Language Models [99] | 2023 | Tools & business workflows | Paper |
| Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks [100] | 2023 | Tools & business workflows | Paper |
| ProTeGi: Automatic Prompt Optimization with “Gradient Descent” and Beam Search [250] | 2023 | Learning & agent harnesses | Paper · Code · Publication |
| ReAct: Synergizing Reasoning and Acting in Language Models [13] | 2023 | Tools & business workflows | Paper · Code · Publication · Publication |
| Reflexion: Language Agents with Verbal Reinforcement Learning [60] | 2023 | Verification & recovery | Paper · Code · Publication |
| RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting | 2023 | Learning & agent harnesses | Paper |
| RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback | 2023 | Learning & agent harnesses | Paper |
| Self-Refine: Iterative Refinement with Self-Feedback [59] | 2023 | Verification & recovery | Paper · Code · Publication |
| SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models [4] | 2023 | Spreadsheets, slides & documents | Paper · Code · Publication · Cited edition |
| StructGPT: A General Framework for Large Language Model to Reason over Structured Data | 2023 | Tools & business workflows | Paper |
| Table-GPT: Table-tuned GPT for Diverse Table Tasks | 2023 | Learning & agent harnesses | Paper |
| ToolEmu: Identifying the Risks of LM Agents with an LM-Emulated Sandbox | 2023 | Safety & authorization | Paper |
| Toolformer: Language Models Can Teach Themselves to Use Tools [24] | 2023 | Tools & business workflows | Paper · Publication · Cited edition |
| BINDER: Binding Language Models in Symbolic Languages | 2022 | Tools & business workflows | Paper |
| DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents [119] | 2022 | Spreadsheets, slides & documents | Paper · Publication |
| LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking | 2022 | Reasoning, evidence & work impact | Paper · Code |
| PEER: A Collaborative Language Model | 2022 | Spreadsheets, slides & documents | Paper |
| RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning [252] | 2022 | Learning & agent harnesses | Paper · Code · Publication |
| TAPEX: Table Pre-training via Learning a Neural SQL Executor [184] | 2022 | Learning & agent harnesses | Paper |
| Codex / HumanEval: Evaluating Large Language Models Trained on Code | 2021 | Learning & agent harnesses | Paper |
| D2S: Document-to-Slide Generation Via Query-Based Text Summarization | 2021 | Spreadsheets, slides & documents | Paper |
| LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding | 2021 | Reasoning, evidence & work impact | Paper |
| SpreadsheetCoder: Formula Prediction from Semi-structured Context [105] | 2021 | Spreadsheets, slides & documents | Paper |
| TAPAS: Weakly Supervised Table Parsing via Pre-training [87] | 2020 | Reasoning, evidence & work impact | Paper |
| NLyze: Interactive Programming by Natural Language for SpreadSheet Data Analysis and Manipulation [110] | 2014 | Spreadsheets, slides & documents | Paper · Paper |
| Flash Fill: Automating String Processing in Spreadsheets using Input-Output Examples [109] | 2011 | Spreadsheets, slides & documents | Paper · Publication |
Benchmarks, datasets, evaluation environments, and protocols for measuring task completion, work quality, reliability, and cost. 190 entries.
| Paper | Year | Focus | Links |
|---|---|---|---|
|
τ²-Bench: |
2026 | Tools & business workflows | Paper · Code · Publication · Data |
| AgentIF-OneDay: A Task-level Instruction-Following Benchmark for General AI Agents in Daily Scenarios [306] | 2026 | Workspaces & occupational work | Paper · Data · Website |
| Agents’ Last Exam: Agents' Last Exam [285] | 2026 | Workspaces & occupational work | Paper · Data |
| AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs | 2026 | Learning & agent harnesses | Paper · Code |
| AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning [155] | 2026 | Workspaces & occupational work | Paper |
| AIDABench: AI Data Analytics Benchmark | 2026 | Workspaces & occupational work | Paper · Code |
| APEX-Agents [157] | 2026 | Workspaces & occupational work | Paper · Data · Code |
| Auditing Financial Agent Self-Evolution: Auditing Self-Evolution in Financial Agents: Capability Gains, Security Drift, and Execution-Interface Mismatch [387] | 2026 | Safety & authorization | Paper |
| AutomationBench (Zapier): AutomationBench [140] | 2026 | Tools & business workflows | Paper · Code · Website |
| BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows [281] | 2026 | Workspaces & occupational work | Paper · Code · Publication · Data |
| BenchCAD: A Comprehensive, Industry-Standard Benchmark for Programmatic CAD [283] | 2026 | Software, data & research | Paper · Code · Data · Website |
| BlueFin: Benchmarking LLM Agents on Financial Spreadsheets | 2026 | Spreadsheets, slides & documents | Paper |
| CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation [86] | 2026 | CAD & visual design | Paper · Code · Data · Website |
| CentaurBench: Benchmarking LLM Capabilities on Augmenting vs. Automating Real-World Work Tasks [314] | 2026 | Workspaces & occupational work | Paper · Code · Data · Website |
| CHI-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows? [138] | 2026 | Tools & business workflows | Paper · Publication |
| CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agents [303] | 2026 | Safety & authorization | Paper · Code · Publication · Code |
| CL-bench: A Benchmark for Context Learning [315] | 2026 | Reasoning, evidence & work impact | Paper · Code · Data · Website |
| Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration [141] | 2026 | Human-agent collaboration | Paper · Code · Publication |
| ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox | 2026 | Tools & business workflows | Paper |
| CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions [302] | 2026 | Tools & business workflows | Paper · Code · Data · Publication |
| CXBench: A Benchmark for Process-Compliant LLM Agents in Regulated Customer-Experience Workflows [131] | 2026 | Tools & business workflows | Paper |
| DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle [137] | 2026 | Software, data & research | Paper · Publication · Cited edition |
| DataSciBench: An LLM Agent Benchmark for Data Science [118] | 2026 | Software, data & research | Paper · Code · Publication |
| DELEGATE-52: LLMs Corrupt Your Documents When You Delegate [104] | 2026 | Spreadsheets, slides & documents | Paper · Code |
| DRBench: A Realistic Benchmark for Enterprise Deep Research [146] | 2026 | Workspaces & occupational work | Paper · Publication |
| Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing | 2026 | Reasoning, evidence & work impact | Paper · Code |
| DRFLOW: A Deep Research Benchmark for Personalized Workflow Prediction [357] | 2026 | Tools & business workflows | Paper |
| DrugDiscoveryBench: Can Coding Agents Assist Early-Stage Drug Discovery? | 2026 | Software, data & research | Paper · Code · Data |
| E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios | 2026 | Tools & business workflows | Paper |
| E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation [33] | 2026 | Tools & business workflows | Paper · Code · Data · Website |
| EntCollabBench: Beyond the All-in-One Agent: Benchmarking Role-Specialized Multi-Agent Collaboration in Enterprise Workflows [38] | 2026 | Tools & business workflows | Paper |
| EnterpriseClawBench: Benchmarking Agents from Real Workplace Sessions | 2026 | Tools & business workflows | Paper |
| EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings [139] | 2026 | Tools & business workflows | Paper · Publication · Code · Data · Website |
| EXP-Bench: Can AI Conduct AI Research Experiments? [135] | 2026 | Software, data & research | Paper · Publication |
| Facts Without Rules: Boundary Metadata Collapse in Multi-Agent LLM Handoffs [342] | 2026 | Safety & authorization | Paper · Code |
| FieldWorkArena: Agentic AI Benchmark for Real Field Work Tasks [305] | 2026 | Web, desktop & mobile | Paper · Code · Data |
| FrontierFinance: A Challenging Benchmark for Measuring Frontier Intelligence of Finance Agents [282] | 2026 | Finance & professional research | Paper · Code · Data · Website |
| GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents [91] | 2026 | Reasoning, evidence & work impact | Paper · Data · Code · Website |
| GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks [279] | 2026 | Workspaces & occupational work | Paper · Publication · Publication · Data · Website |
| Harness-Bench: Evaluating Agent Harnesses across Models and Realistic Workflows [325] | 2026 | Evaluation infrastructure | Paper |
| HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents [294] | 2026 | Tools & business workflows | Paper · Code · Data · Website |
| HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help? | 2026 | Software, data & research | Paper · Code · Data |
| Holistic Agent Leaderboard (HAL): Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation [324] | 2026 | Evaluation infrastructure | Paper |
| τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge [39] | 2026 | Tools & business workflows | Paper |
| LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growth [150] | 2026 | Memory & persistent work | Paper · Code · Website |
| LongDS-Bench: On the Failure of Long-Horizon Agentic Data Analysis | 2026 | Memory & persistent work | Paper · Code |
| MASBench: Benchmarking LLM-based Multi-Agent Collaboration under Partial Observability | 2026 | Learning & agent harnesses | Paper · Code |
| MBABench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance | 2026 | Spreadsheets, slides & documents | Paper |
| MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers [307] | 2026 | Tools & business workflows | Paper · Publication · Code · Data |
| MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers [124] | 2026 | Tools & business workflows | Paper · Code · Publication · Cited edition |
| Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks [310] | 2026 | Web, desktop & mobile | Paper · Website |
| Office Comprehension Benchmark | 2026 | Reasoning, evidence & work impact | Paper · Code · Data |
| OfficeEval: Mind the Gap: Can Frontier LLMs Pass a Standardized Office Proficiency Exam? | 2026 | Workspaces & occupational work | Paper |
| OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoning [145] | 2026 | Reasoning, evidence & work impact | Paper · Code · Data · Website |
| OmegaUse-OfficeVal: Benchmarking LLM Agents on Long-Horizon Office-Suite Tasks with Economic Grounding [301] | 2026 | Workspaces & occupational work | Paper · Code · Data |
| OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios [317] | 2026 | Tools & business workflows | Paper · Code · Data · Website |
| $OneMillion-Bench: How Far are Language Agents from Human Experts? [313] | 2026 | Workspaces & occupational work | Paper · Publication · Code · Data |
| OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks [32] | 2026 | Web, desktop & mobile | Paper · Code · Data · Website |
| PAST-Bench / Hermes+: PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents [334] | 2026 | Memory & persistent work | Paper · Code |
| PPT-Eval: A Benchmark for Computer-Use Agents on PowerPoint Tasks | 2026 | Spreadsheets, slides & documents | Paper · Code |
| PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning [80] | 2026 | Reasoning, evidence & work impact | Paper · Publication · Code · Data · Website |
| PresentBench: A Fine-Grained Rubric-Based Benchmark for Slide Generation [17] | 2026 | Spreadsheets, slides & documents | Paper · Code · Data · Website |
| PSEBench: A Controllable and Verifiable Benchmark for Evaluating LLMs in Patient Safety Event Triage [298] | 2026 | Reasoning, evidence & work impact | Paper |
| READY: READY or Not: Reliable Enterprise Agent Deployment [330] | 2026 | Verification & recovery | Paper · Website |
| SCUBA: Salesforce Computer Use Benchmark | 2026 | Tools & business workflows | Paper · Publication |
| SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks | 2026 | Workspaces & occupational work | Paper |
| SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows [121] | 2026 | Spreadsheets, slides & documents | Paper · Code · Data · Accepted · Publication |
| StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows [280] | 2026 | Workspaces & occupational work | Paper · Website |
| StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance [364] | 2026 | Memory & persistent work | Paper · Code |
| SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution | 2026 | Software, data & research | Paper · Code |
| SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks? | 2026 | Software, data & research | Paper · Publication |
| SWE-INTERACT: Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding Sessions | 2026 | Software, data & research | Paper · Code |
| Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces [122] | 2026 | Web, desktop & mobile | Paper · Code · Publication · Cited edition · Website |
| Toolathlon: The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution [149] | 2026 | Tools & business workflows | Paper · Code · Publication |
| ToolBench-X: Beyond Function Calling: Benchmarking Tool-Using Agents under Tool-Environment Unreliability | 2026 | Verification & recovery | Paper · Code |
| TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation [339] | 2026 | Personal agents & digital twins | Paper |
| VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World? [34] | 2026 | Memory & persistent work | Paper |
| VibeSearchBench: Benchmarking Long-horizon Proactive Search in the Wild | 2026 | Reasoning, evidence & work impact | Paper |
| WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces | 2026 | Web, desktop & mobile | Paper · Code · Data |
| WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation [311] | 2026 | Web, desktop & mobile | Paper · Code · Data · Website |
| WorkBuddy Bench: Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction | 2026 | Workspaces & occupational work | Paper |
| Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies [36] | 2026 | Workspaces & occupational work | Paper · Code · Publication · Data · Website |
| WTM-Bench: Back to the Future: A workbook time machine for spread sheet creation benchmarks [108] | 2026 | Spreadsheets, slides & documents | Paper · Code · Data · Publication |
| XL-DocBench: Benchmarking Evidence-Grounded Extra-Long Document Understanding [95] | 2026 | Reasoning, evidence & work impact | Paper · Publication |
| YC-Bench: Benchmarking AI Agents for Long-Term Planning and Consistent Execution | 2026 | Memory & persistent work | Paper · Code |
| AAAR-1.0: Assessing AI’s Potential to Assist Research [78] | 2025 | Software, data & research | Paper · Publication |
| AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories [370] | 2025 | Verification & recovery | Paper · Publication |
| AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds [123] | 2025 | Software, data & research | Paper |
| AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents [27] | 2025 | Web, desktop & mobile | Paper · Code · Publication · Data · Website |
|
τ-bench: |
2025 | Tools & business workflows | Paper · Code · Publication |
| BFCL: The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models [35] | 2025 | Tools & business workflows | Paper · Code |
| BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents | 2025 | Reasoning, evidence & work impact | Paper |
| ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering [90] | 2025 | Reasoning, evidence & work impact | Paper · Publication |
| CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments [148] | 2025 | Tools & business workflows | Paper · Code · Publication · Data |
| DABstep: Data Agent Benchmark for Multi-step Reasoning [126] | 2025 | Software, data & research | Paper |
| DiscoveryBench: Towards Data-Driven Discovery with Large Language Models [287] | 2025 | Software, data & research | Paper · Code · Data · Publication |
| Document Haystack: A Long Context Multimodal Image/Document Understanding Vision LLM Benchmark | 2025 | Reasoning, evidence & work impact | Paper · Code · Data |
| DSBench: How Far Are Data Science Agents from Becoming Data Science Experts? [286] | 2025 | Software, data & research | Paper · Code · Publication · Data · Website |
| EnterpriseBench: Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments [304] | 2025 | Tools & business workflows | Paper · Publication · Code · Data · Website |
| Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Tasks [289] | 2025 | Reasoning, evidence & work impact | Paper · Benchmark v2 · Code · Website |
| FINCH / FinWorkBench: Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflows | 2025 | Workspaces & occupational work | Paper · Code · Data |
| FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering [79] | 2025 | Reasoning, evidence & work impact | Paper · Publication |
| Flex-TravelPlanner: A Benchmark for Flexible Planning with Language Agents | 2025 | Tools & business workflows | Paper |
| Hell or High Water: Evaluating Agentic Recovery from External Failures | 2025 | Verification & recovery | Paper · Code |
| HERB: Benchmarking Deep Search over Heterogeneous Enterprise Data [156] | 2025 | Workspaces & occupational work | Paper · Publication |
| Idea2Plan: Exploring AI-Powered Research Planning [82] | 2025 | Software, data & research | Paper |
| LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks [92] | 2025 | Reasoning, evidence & work impact | Paper · Publication |
| LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory [65] | 2025 | Memory & persistent work | Paper · Code · Publication |
| MAST: Why Do Multi-Agent LLM Systems Fail? [343] | 2025 | Verification & recovery | Paper · Code · Data |
| MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers [125] | 2025 | Tools & business workflows | Paper |
| MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents [116] | 2025 | Tools & business workflows | Paper · Code |
| METR time horizon: Measuring AI Ability to Complete Long Software Tasks [63] | 2025 | Memory & persistent work | Paper · Publication |
| MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering | 2025 | Software, data & research | Paper · Publication |
| MMDocIR: Benchmarking Multimodal Retrieval for Long Documents | 2025 | Reasoning, evidence & work impact | Paper · Code |
| MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly [81] | 2025 | Reasoning, evidence & work impact | Paper · Publication |
| MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark [77] | 2025 | Reasoning, evidence & work impact | Paper · Publication |
| MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents [309] | 2025 | Learning & agent harnesses | Paper · Code · Publication · Data |
| Multi-Mission Tool Bench: Assessing the Robustness of LLM based Agents through Related and Dynamic Missions [308] | 2025 | Tools & business workflows | Paper · Code · Data · Website |
| OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflows [300] | 2025 | Workspaces & occupational work | Paper |
| PaperBench: Evaluating AI's Ability to Replicate AI Research [136] | 2025 | Software, data & research | Paper · Code · Publication · Code · Data |
| Ready Jurist One: Benchmarking Language Agents for Legal Intelligence in Dynamic Environments | 2025 | Tools & business workflows | Paper |
| RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts [64] | 2025 | Software, data & research | Paper · Code · Publication |
| Remote Labor Index: Measuring AI Automation of Remote Work [288] | 2025 | Workspaces & occupational work | Paper · Code · Data · Website |
| ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents | 2025 | Reasoning, evidence & work impact | Paper |
| ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery [10] | 2025 | Software, data & research | Paper · Code · Data · Publication · Code · Website |
| ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use [115] | 2025 | Web, desktop & mobile | Paper · Code · Data · Publication |
| Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows [154] | 2025 | Tools & business workflows | Paper · Code · Publication · Website |
| SWE-bench Goes Live! | 2025 | Software, data & research | Paper · Publication |
| SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering? [153] | 2025 | Software, data & research | Paper · Publication |
| TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks [31] | 2025 | Tools & business workflows | Paper · Code · Publication · Cited edition · Website |
| ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities [127] | 2025 | Tools & business workflows | Paper · Code · Publication · Code |
| Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale [312] | 2025 | Web, desktop & mobile | Paper · Code · Publication · Website |
| WorFBench: Benchmarking Agentic Workflow Generation [84] | 2025 | Learning & agent harnesses | Paper · Publication |
| AgentBench: Evaluating LLMs as Agents [275] | 2024 | Tools & business workflows | Paper · Publication |
| AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents [276] | 2024 | Tools & business workflows | Paper · Publication |
| AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents | 2024 | Safety & authorization | Paper · Code |
| AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents | 2024 | Safety & authorization | Paper |
| AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents [37] | 2024 | Tools & business workflows | Paper · Code · Publication · Website |
| AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks? | 2024 | Reasoning, evidence & work impact | Paper |
| DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models | 2024 | Workspaces & occupational work | Paper |
| DocBench: A Benchmark for Evaluating LLM-based Document Reading Systems | 2024 | Reasoning, evidence & work impact | Paper · Code |
| GAIA: a benchmark for General AI Assistants [40] | 2024 | Workspaces & occupational work | Paper · Publication · Data |
| InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents | 2024 | Safety & authorization | Paper |
| LawBench: Benchmarking Legal Knowledge of Large Language Models [292] | 2024 | Reasoning, evidence & work impact | Paper · Code · Publication · Data |
| LoCoMo: Evaluating Very Long-Term Conversational Memory of LLM Agents [66] | 2024 | Memory & persistent work | Paper · Code · Data · Publication |
| LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding [94] | 2024 | Workspaces & occupational work | Paper · Publication |
| LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locating | 2024 | Reasoning, evidence & work impact | Paper · Code |
| MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback [28] | 2024 | Tools & business workflows | Paper · Code · Data · Publication |
| MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark | 2024 | Verification & recovery | Paper |
| MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizations [299] | 2024 | Reasoning, evidence & work impact | Paper · Code · Data · Publication · Data · Website |
| OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation [76] | 2024 | Workspaces & occupational work | Paper · Code · Data |
| OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations | 2024 | Reasoning, evidence & work impact | Paper · Code |
| OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments [7] | 2024 | Web, desktop & mobile | Paper · Code · Data · Publication · Website |
| PPTC Benchmark: Evaluating Large Language Models for PowerPoint Task Completion [112] | 2024 | Spreadsheets, slides & documents | Paper · Code · Publication · Data |
| RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems [44] | 2024 | Software, data & research | Paper · Code · Publication |
| SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation [56] | 2024 | Spreadsheets, slides & documents | Paper · Code · Data · Publication · Website |
| SWE-bench: Can Language Models Resolve Real-World GitHub Issues? [16] | 2024 | Software, data & research | Paper · Publication · Cited edition · Code · Data · Website |
| SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents | 2024 | Software, data & research | Paper |
| TableBench: A Comprehensive and Complex Benchmark for Table Question Answering | 2024 | Workspaces & occupational work | Paper |
| TimeArena: Shaping Efficient Multitasking Language Agents in a Time-Aware Simulation [128] | 2024 | Memory & persistent work | Paper |
| TravelPlanner: A Benchmark for Real-World Planning with Language Agents [83] | 2024 | Tools & business workflows | Paper · Code · Data · Publication |
| VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks | 2024 | Web, desktop & mobile | Paper |
| VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks [284] | 2024 | Web, desktop & mobile | Paper · Publication · Code · Data · Website |
| WebArena: A Realistic Web Environment for Building Autonomous Agents [6] | 2024 | Web, desktop & mobile | Paper · Publication · Code · Data · Website |
| WebLINX: Real-World Website Navigation with Multi-Turn Dialogue [278] | 2024 | Web, desktop & mobile | Paper · Publication |
| WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks? [9] | 2024 | Web, desktop & mobile | Paper · Publication · Code · Data · Website |
| WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks [133] | 2024 | Web, desktop & mobile | Paper · Code · Publication · Cited edition · Data · Website |
| WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting [114] | 2024 | Tools & business workflows | Paper · Code · Publication · Data |
| API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs [25] | 2023 | Tools & business workflows | Paper · Code · Publication · Data |
| BIRD: Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs | 2023 | Workspaces & occupational work | Paper |
| FinanceBench: A New Benchmark for Financial Question Answering [93] | 2023 | Reasoning, evidence & work impact | Paper · Code · Data |
| InstructExcel: A Benchmark for Natural Language Instruction in Excel [111] | 2023 | Spreadsheets, slides & documents | Paper · Code |
| LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models [101] | 2023 | Reasoning, evidence & work impact | Paper · Publication · Cited edition · Code · Data |
| Mind2Web: Towards a Generalist Agent for the Web [277] | 2023 | Web, desktop & mobile | Paper · Publication |
| MT-Bench / Chatbot Arena: Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena [371] | 2023 | Verification & recovery | Paper · Code · Publication · Code · Data |
| ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning [23] | 2022 | Reasoning, evidence & work impact | Paper · Code · Publication |
| ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering [98] | 2022 | Reasoning, evidence & work impact | Paper · Code · Publication · Data |
| DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation | 2022 | Software, data & research | Paper |
| PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change | 2022 | Reasoning, evidence & work impact | Paper · Code |
| ScienceQA: Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering [103] | 2022 | Reasoning, evidence & work impact | Paper · Code · Data · Publication · Cited edition |
| WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents [117] | 2022 | Web, desktop & mobile | Paper · Publication |
| DocVQA: A Dataset for VQA on Document Images [22] | 2021 | Reasoning, evidence & work impact | Paper · Publication · Cited edition · Data · Website |
| FinQA: A Dataset of Numerical Reasoning over Financial Data [21] | 2021 | Reasoning, evidence & work impact | Paper · Code · Publication · Data |
| TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance [20] | 2021 | Reasoning, evidence & work impact | Paper · Code · Publication · Code · Data · Website |
| PubMedQA: A Dataset for Biomedical Research Question Answering [102] | 2019 | Reasoning, evidence & work impact | Paper · Code · Publication |
| HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering [19] | 2018 | Reasoning, evidence & work impact | Paper · Code · Publication |
| Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task | 2018 | Workspaces & occupational work | Paper |
| SQuAD: 100,000+ Questions for Machine Comprehension of Text [18] | 2016 | Reasoning, evidence & work impact | Paper · Publication |
| JobBench: Aligning Agent Work With Human Will [316] | 2026 | Workspaces & occupational work | Paper · Code · Data |
| Legal Research Bench: Measuring End-to-End Reliability in Long-Horizon Legal Research Agents [293] | 2026 | Reasoning, evidence & work impact | Paper · Code · Benchmark |
Research syntheses, foundation-model reports, conceptual foundations, reasoning and verification studies, and empirical work on productivity and collaboration. 92 entries.
| Paper | Year | Focus | Links |
|---|---|---|---|
| Agentic ROI: Position: The Real Barrier to LLM Agent Usability is Agentic ROI [329] | 2026 | Reasoning, evidence & work impact | Paper · Paper |
| Agent Skills Survey: Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward | 2026 | Reasoning, evidence & work impact | Paper |
| AI Agent Systems Survey: AI Agent Systems: Architectures, Applications, and Evaluation | 2026 | Learning & agent harnesses | Paper |
| AI Petition Platform Study: Introducing AI to an online petition platform changed outputs but not outcomes [374] | 2026 | Work impact & productivity | Paper |
| Code as Agent Harness | 2026 | Learning & agent harnesses | Paper |
| Coding Assistants Longitudinal Study: The Impact of AI Coding Assistants on Software Engineering: A Longitudinal Study | 2026 | Reasoning, evidence & work impact | Paper |
| Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design | 2026 | Memory & persistent work | Paper |
| DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence [182] | 2026 | Foundation models & agentic training | Paper |
| Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering | 2026 | Learning & agent harnesses | Paper |
| From Question Answering to Task Completion: A Survey on Agent System and Harness Design | 2026 | Learning & agent harnesses | Paper · Code |
| Generative Agent Simulations: LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals [338] | 2026 | Personal agents & digital twins | Paper · Paper |
| GLM-5: from Vibe Coding to Agentic Engineering [187] | 2026 | Reasoning, evidence & work impact | Paper |
| Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems | 2026 | Learning & agent harnesses | Paper |
| How Well Does Agent Development Reflect Real-World Work? [328] | 2026 | Reasoning, evidence & work impact | Paper · Code |
| Intelligent AI Delegation | 2026 | Safety & authorization | Paper |
| Kimi K2.5: Visual Agentic Intelligence [186] | 2026 | Foundation models & agentic training | Paper |
| On-Policy Distillation Survey: A Survey of On-Policy Distillation for Large Language Models | 2026 | Reasoning, evidence & work impact | Paper |
| Open-World Evaluations for Measuring Frontier AI Capabilities | 2026 | Reasoning, evidence & work impact | Paper |
| Proactivity-Gym: Foundations of Proactive Agents: Principles, Technical Layers, and Proactivity-Gym [333] | 2026 | Memory & persistent work | Paper |
| Qwen3.8-Next: On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [192] | 2026 | Foundation models & agentic training | Paper |
| Reliable Self-Evolving Agents: Diving into Reliable Self-Evolving Agents: A Survey | 2026 | Memory & persistent work | Paper · Code |
| Self-Improvements in Modern Agentic Systems: A Survey | 2026 | Memory & persistent work | Paper |
| Shift to Agentic AI: The Shift to Agentic AI: Evidence from Codex | 2026 | Reasoning, evidence & work impact | Paper |
| Sim2Real User Simulation: Mind the Sim2Real Gap in User Simulation for Agentic Tasks [349] | 2026 | Verification & recovery | Paper |
| Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters [188] | 2026 | Foundation models & agentic training | Paper |
| The Evolution of AI Office: From Passive Assistants to Goal-Driven Agents | 2026 | Spreadsheets, slides & documents | Paper |
| Towards Long-Horizon Agents: A Survey --- Foundation, Evolution, Harness, Optimization, Application, and Frontier [62] | 2026 | Memory & persistent work | Paper · Code |
| A Comprehensive Survey of Agents for Computer Use: Foundations, Challenges, and Future Directions | 2025 | Web, desktop & mobile | Paper |
| Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems | 2025 | Learning & agent harnesses | Paper · Code |
| Agent Evaluation and Benchmarking Survey: Evaluation and Benchmarking of LLM Agents: A Survey | 2025 | Verification & recovery | Paper |
| Anthropic Economic Index: Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations | 2025 | Reasoning, evidence & work impact | Paper |
| A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence | 2025 | Memory & persistent work | Paper |
| A Survey on Latent Reasoning | 2025 | Reasoning, evidence & work impact | Paper |
| DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning | 2025 | Reasoning, evidence & work impact | Paper · Publication |
| Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforce [346] | 2025 | Reasoning, evidence & work impact | Paper · Paper |
| Great Models Think Alike and this Undermines AI Oversight [373] | 2025 | Verification & recovery | Paper · Code |
| Kimi K2: Open Agentic Intelligence [185] | 2025 | Foundation models & agentic training | Paper |
| Levels of Autonomy for AI Agents | 2025 | Reasoning, evidence & work impact | Paper |
| Lifelong Learning of LLM Agents: Lifelong Learning of Large Language Model based Agents: A Roadmap | 2025 | Memory & persistent work | Paper |
| LLM-KG4QA: Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunities | 2025 | Reasoning, evidence & work impact | Paper |
| Memory in the Age of AI Agents | 2025 | Memory & persistent work | Paper |
| METR Developer Productivity RCT: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity [389] | 2025 | Software, data & research | Paper |
| Multimodal Document RAG Survey: Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding | 2025 | Reasoning, evidence & work impact | Paper |
| OS Agents Survey: OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use | 2025 | Reasoning, evidence & work impact | Paper |
| Production-Grade Agentic Workflows: A Practical Guide for Designing, Developing, and Deploying Production-Grade Agentic AI Workflows | 2025 | Reasoning, evidence & work impact | Paper |
| Qwen2.5-VL Technical Report | 2025 | Reasoning, evidence & work impact | Paper · Code |
| Rethinking Agent Memory: Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topics | 2025 | Memory & persistent work | Paper · Code |
| Scaling Agent Systems: Towards a Science of Scaling Agent Systems [344] | 2025 | Learning & agent harnesses | Paper |
| Survey on Agent Evaluation: Survey on Evaluation of LLM-based Agents | 2025 | Verification & recovery | Paper |
| Table Processing Survey: Large language model for table processing: a survey | 2025 | Reasoning, evidence & work impact | Paper |
| The Cybernetic Teammate: A Field Experiment on Generative AI Reshaping Teamwork and Expertise [388] | 2025 | Reasoning, evidence & work impact | Paper |
| Tool Learning Survey: LLM-Based Agents for Tool Learning: A Survey | 2025 | Reasoning, evidence & work impact | Paper |
| Visually Rich Document Understanding Survey: A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trends | 2025 | Reasoning, evidence & work impact | Paper |
| Agent Memory Mechanism Survey: A Survey on the Memory Mechanism of Large Language Model based Agents | 2024 | Memory & persistent work | Paper · Code |
| CoALA: Cognitive Architectures for Language Agents [14] | 2024 | Learning & agent harnesses | Paper · Code |
| DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models | 2024 | Reasoning, evidence & work impact | Paper |
| Eureka: Human-Level Reward Design via Coding Large Language Models | 2024 | Learning & agent harnesses | Paper · Publication |
| GPTs are GPTs: Labor market impact potential of LLMs [348] | 2024 | Work impact & productivity | Paper · Publication |
| GUI Agents Survey: GUI Agents: A Survey | 2024 | Reasoning, evidence & work impact | Paper |
| Let's Verify Step by Step [180] | 2024 | Learning & agent harnesses | Paper · Publication |
| LLM Agent Planning Survey: Understanding the planning of LLM agents: A survey | 2024 | Reasoning, evidence & work impact | Paper |
| LLM Autonomous Agents Survey: A Survey on Large Language Model based Autonomous Agents | 2024 | Learning & agent harnesses | Paper · Publication |
| Self-Preference in LLM Evaluators: LLM Evaluators Recognize and Favor Their Own Generations [372] | 2024 | Verification & recovery | Paper |
| Self-Rewarding LM: Self-Rewarding Language Models [218] | 2024 | Learning & agent harnesses | Paper · Publication |
| Tool Learning Survey: Tool Learning with Large Language Models: A Survey | 2024 | Tools & business workflows | Paper · Code |
| APE: Large Language Models Are Human-Level Prompt Engineers [243] | 2023 | Learning & agent harnesses | Paper · Code · Publication |
| Gemini: A Family of Highly Capable Multimodal Models | 2023 | Reasoning, evidence & work impact | Paper |
| GPT-4 Technical Report | 2023 | Reasoning, evidence & work impact | Paper |
| Productivity Effects of Generative AI: Experimental evidence on the productivity effects of generative artificial intelligence [390] | 2023 | Work impact & productivity | Paper |
| Reward Model Overoptimization: Scaling Laws for Reward Model Overoptimization [383] | 2023 | Verification & recovery | Paper |
| Self-Instruct: Aligning Language Models with Self-Generated Instructions [196] | 2023 | Learning & agent harnesses | Paper · Code · Publication |
| Sparks of AGI: Sparks of Artificial General Intelligence: Early experiments with GPT-4 | 2023 | Reasoning, evidence & work impact | Paper |
| The Rise and Potential of LLM Agents: The Rise and Potential of Large Language Model Based Agents: A Survey | 2023 | Learning & agent harnesses | Paper |
| Emergent Abilities of Large Language Models | 2022 | Reasoning, evidence & work impact | Paper |
| STaR: Bootstrapping Reasoning With Reasoning [217] | 2022 | Learning & agent harnesses | Paper · Publication |
| Foundation Models: On the Opportunities and Risks of Foundation Models [1] | 2021 | Reasoning, evidence & work impact | Paper |
| Productivity J-Curve: The Productivity J-Curve: How Intangibles Complement General Purpose Technologies [391] | 2021 | Work impact & productivity | Paper |
| GPT-3: Language Models are Few-Shot Learners | 2020 | Reasoning, evidence & work impact | Paper |
| Scaling Laws for Neural Language Models | 2020 | Reasoning, evidence & work impact | Paper |
| Gradient Episodic Memory for Continual Learning [386] | 2017 | Reasoning, evidence & work impact | Paper |
| PPO: Proximal Policy Optimization Algorithms [199] | 2017 | Reasoning, evidence & work impact | Paper |
| RL from Human Preferences: Deep reinforcement learning from human preferences [178] | 2017 | Reasoning, evidence & work impact | Paper · Publication |
| Life Beyond Distributed Transactions [378] | 2016 | Reasoning, evidence & work impact | Paper |
| DAgger: A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning [212] | 2011 | Learning & agent harnesses | Paper · Publication |
| Provenance in Databases: Why, How, and Where [379] | 2009 | Reasoning, evidence & work impact | Paper |
| Case Handling: a new paradigm for business process support [376] | 2005 | Reasoning, evidence & work impact | Paper |
| Ten Challenges for Automation Team Players: Ten Challenges for Making Automation a “Team Player” in Joint Human-Agent Activity [345] | 2004 | Reasoning, evidence & work impact | Paper |
| Humans and Automation: Use, Misuse, Disuse, Abuse [331] | 1997 | Reasoning, evidence & work impact | Paper |
| Interdisciplinary Study of Coordination: The interdisciplinary study of coordination [50] | 1994 | Human-agent collaboration | Paper |
| IT Productivity Paradox: The Productivity Paradox of Information Technology [392] | 1993 | Reasoning, evidence & work impact | Paper |
| Sagas [380] | 1987 | Reasoning, evidence & work impact | Paper |
| Ironies of Automation [393] | 1983 | Reasoning, evidence & work impact | Paper |
Official product pages, engineering reports, documentation, evaluation infrastructure, and project resources. These entries are excluded from the research-paper count. 59 entries.
| Resource | Year | Focus | Links |
|---|---|---|---|
| Agentic Capabilities in Word, Excel, and PowerPoint | 2026 | Workspaces & occupational work | Source |
| Claude Duet: Introducing Claude Duet [336] | 2026 | Personal agents & digital twins | Source |
| Finance Agent v2: Leaderboard and Methodology [290] | 2026 | Finance & professional research | Source · Code |
| Harbor: A Framework for Evaluating and Improving Agents [321] | 2026 | Evaluation infrastructure | Source · Code |
| HarnessTax: How Much Does Harness Matter for Coding Agents? [326] | 2026 | Evaluation infrastructure | Source |
| Harvey Legal Agent Benchmark [291] | 2026 | Workspaces & occupational work | Source · Code · Data |
| Inspect AI: Inspect: A Framework for Large Language Model Evaluations [322] | 2026 | Evaluation infrastructure | Source · Code |
| Medical Long Context Reasoning (MLCR-AA) [297] | 2026 | Reasoning, evidence & work impact | Source · Code · Data |
| Mercor / SkyRL training guide: Training Frontier Knowledge Work Agents: A 397B RL Training Guide with SkyRL [208] | 2026 | Learning & agent harnesses | Source · Code |
| Microsoft Copilot architecture and how it works [142] | 2026 | Learning & agent harnesses | Source · Architecture |
| OfficeACE: Huawei Cloud Office Agent Invitation Testing [75] | 2026 | Spreadsheets, slides & documents | Source |
| Pi: A Minimal, Extensible Coding Agent Harness [323] | 2026 | Learning & agent harnesses | Source |
| Anthropic Research system: How we built our multi-agent research system [132] | 2025 | Tools & business workflows | Source |
| Microsoft 365 Copilot Tuning and Multi-Agent Orchestration | 2025 | Learning & agent harnesses | Source |
| AA-AnalystAgent | 2026 | Reasoning, evidence & work impact | Source |
| AA-Briefcase v1.1: Agentic Knowledge Work Benchmark | 2026 | Workspaces & occupational work | Source · Code · Data |
| Adobe Acrobat AI Assistant | 2024 | Workspaces & occupational work | Source |
| Airtable Omni: AI-Native App Builder and Assistant | 2025 | Workspaces & occupational work | Source |
| Announcing the Agent2Agent Protocol (A2A) | 2025 | Learning & agent harnesses | Source |
| BigLaw Bench | 2024 | Reasoning, evidence & work impact | Source |
| Claude Code Overview [12] | 2025 | Learning & agent harnesses | Source · Code |
| Claude Cowork: Agentic AI for Knowledge Work | 2026 | Workspaces & occupational work | Source |
| Codex Manual [11] | - | Learning & agent harnesses | Source · Code |
| Computer-Using Agent | 2025 | Workspaces & occupational work | Source |
| Copilot for Power BI Overview | 2023 | Workspaces & occupational work | Source |
| Copilot in Excel | 2023 | Workspaces & occupational work | Source |
| Dynamic Workflows: Orchestrate Subagents at Scale | 2026 | Learning & agent harnesses | Source |
| Equipping Agents for the Real World with Agent Skills | 2025 | Learning & agent harnesses | Source · Code |
| Excel Modeling Benchmark | 2026 | Spreadsheets, slides & documents | Source |
| Function Calling | 2023 | Learning & agent harnesses | Source |
| GLM Slide and Poster Agent Documentation | - | Workspaces & occupational work | Source |
| Google Workspace Studio: Agents for Everyday Work | 2025 | Workspaces & occupational work | Source |
| How the Microsoft 365 Copilot Orchestrator Chooses Actions | 2023 | Learning & agent harnesses | Source |
| Introducing ChatGPT Agent: Bridging Research and Action | 2025 | Workspaces & occupational work | Source |
| Introducing Claude Design by Anthropic Labs | 2026 | Workspaces & occupational work | Source |
| Introducing Computer Use | 2024 | Workspaces & occupational work | Source |
| Introducing the Model Context Protocol | 2024 | Learning & agent harnesses | Source |
| Loop Engineering: Designing Agentic Loops | 2026 | Learning & agent harnesses | Source |
| MedCode [295] | 2026 | Reasoning, evidence & work impact | Source |
| MedScribe [296] | 2026 | Spreadsheets, slides & documents | Source |
| Microsoft 365 Copilot Tuning Overview | 2025 | Learning & agent harnesses | Source |
| MiniMax Skills: Office Artifact Skills for Coding Agents | 2026 | Learning & agent harnesses | Source |
| New Tools for Building Agents | 2025 | Learning & agent harnesses | Source |
| NotebookLM: AI-Powered Research and Note-Taking | 2023 | Workspaces & occupational work | Source |
| OpenCode: The Open-Source Coding Agent | 2025 | Workspaces & occupational work | Source |
| PinchBench | 2026 | Workspaces & occupational work | Source |
| PptxGenJS Documentation | 2016 | Learning & agent harnesses | Source |
| Quadratic: The AI Spreadsheet | 2023 | Workspaces & occupational work | Source |
| Rows AI: Spreadsheet and Data Analyst | 2023 | Workspaces & occupational work | Source |
| SpreadsheetBench v2: Business Spreadsheet Workflows | 2026 | Workspaces & occupational work | Source |
| Tax Agent Bench | 2026 | Reasoning, evidence & work impact | Source · Code |
| The Next Era of Knowledge Work | 2026 | Workspaces & occupational work | Source |
| The O*NET Content Model [347] | 1995 | Reasoning, evidence & work impact | Source |
| The Ralph Wiggum Technique | 2026 | Learning & agent harnesses | Source |
| We are Changing our Developer Productivity Experiment Design | 2026 | Reasoning, evidence & work impact | Source |
| What 81,000 People Want from AI | 2026 | Workspaces & occupational work | Source |
| WorkBuddy: A Desktop Agent for Office Work | 2026 | Workspaces & occupational work | Source |
| Wukong: An AI-Native Agentic Platform for Enterprises | 2026 | Workspaces & occupational work | Source |
| Zapier Agents: Autonomous AI Teammates | 2025 | Workspaces & occupational work | Source |
If you find this survey or reading list useful, please cite:
@misc{workagents2026,
author = {Zishan Xu and Enjun Du and Mingqi Gao and
Cheng Yang and Junhao Shen and Zirong Chen and
Zhiyuan Yao and Yifu Guo and Yuquan Lu and
Jinyang Huang and Zhengxi Lu and Siyi Liu and
Ziyu Zheng and Yongqi Zhang and Difan Zou and
Weiwen Liu},
title = {Work Agents: Systems, Capabilities,
and Optimization},
year = {2026},
eprint = {2610.35845},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2610.35845}
}Contributions are welcome! Open an issue or submit a pull request to suggest papers, resources, or corrections.
