Skip to content

About

No description, website, or topics provided.

Resources

Stars

62 stars

Watchers

0 watching

Forks

Latest commit

 

History

30 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 

Repository files navigation

Work Agents: Systems, Capabilities, and Optimization

Zishan Xu1*, Enjun Du2,3*, Mingqi Gao4*, Cheng Yang3, Junhao Shen5, Zirong Chen4, Zhiyuan Yao6, Yifu Guo7, Yuquan Lu7, Jinyang Huang8, Zhengxi Lu6, Siyi Liu3, Ziyu Zheng2, Yongqi Zhang3†, Difan Zou2†, Weiwen Liu1†

1Shanghai Jiao Tong University   2The University of Hong Kong   3The Hong Kong University of Science and Technology (Guangzhou)   4Tsinghua University
5The Chinese University of Hong Kong   6Zhejiang University   7Sun Yat-sen University   8Central South University

*Equal contribution   †Corresponding authors

arXiv Project homepage Awesome Work Agents Contributions welcome

Work Agents are AI systems that combine foundation models with runtime harnesses to carry out delegated digital work in stateful workspaces. They use tools, interact with applications, and coordinate workflows to produce answers, editable artifacts, and changes to workspace state.

This repository is the companion reading list for Work Agents: Systems, Capabilities, and Optimization. It brings together research across office productivity, enterprise operations, software engineering, scientific research, and computer use:

  • Agents, methods, and systems — architectures, training methods, environments, memory, and agent harnesses.
  • Benchmarks and evaluation — task suites and evaluation methods for digital work.
  • Surveys and resources — foundational studies, open-source tools, and official documentation.

Read the survey on arXiv, and visit the project homepage for the project overview.

Work Agents: Systems, Capabilities, and Optimization

Contents

Agents, Methods & Systems

Agents, learning methods, training environments, and runtime systems for producing work outcomes and improving capability. 278 entries.

Paper Year Focus Links
ACE: Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models [236] 2026 Learning & agent harnesses Paper · Code · Publication
ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization [257] 2026 Learning & agent harnesses Paper
Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams [258] 2026 Learning & agent harnesses Paper · Code
ADMIRE: Adaptive Milestone Reward for GUI Agents [205] 2026 Web, desktop & mobile Paper
AeSlides: Incentivizing Aesthetic Layout in LLM-Based Slide Generation via Verifiable Rewards [384] 2026 Spreadsheets, slides & documents Paper · Code · Data
AEvo: Harnessing Agentic Evolution [249] 2026 Learning & agent harnesses Paper · Publication
Agent Lightning v1.0: Towards Harnessed Agentic RL [260] 2026 Learning & agent harnesses Paper
AgentMercury: Your Agent Can Synthesize Verifiable Environments for Business Scenarios at scale [163] 2026 Tools & business workflows Paper · Data · Website
Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning [162] 2026 Tools & business workflows Paper · Code · Publication · Data
Anchor: Mitigating Artifact Drift in Agent Benchmark Generation [174] 2026 Verification & recovery Paper · Code · Website
Atomix: Timely, Transactional Tool Use for Reliable Agentic Workflows [382] 2026 Verification & recovery Paper · Code · Publication
AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design [232] 2026 Spreadsheets, slides & documents Paper · Code · Data · Website
AutoRPA: Efficient GUI Automation through LLM-Driven Code Synthesis from Interactions 2026 Learning & agent harnesses Paper
Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation [327] 2026 Learning & agent harnesses Paper
ChronoMem: Version Control and Semantic Rollback for Large Language Model Agent Memory [359] 2026 Memory & persistent work Paper
CODESKILL: Learning Self-Evolving Skills for Coding Agents [239] 2026 Software, data & research Paper
CogEvol: Towards Efficient and Reliable Learning Environment Generation [179] 2026 Spreadsheets, slides & documents Paper · Code
Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents [263] 2026 Learning & agent harnesses Paper
Component Routing: Not All Experience Belongs in the Weights: Component Routing for Self-Improving GUI Agents [274] 2026 Web, desktop & mobile Paper
CompoWorld: Compositional Environment Scaling for General Agents [165] 2026 Learning & agent harnesses Paper
ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents [350] 2026 Web, desktop & mobile Paper · Publication
CorpGen: Simulating Corporate Environments with Autonomous Digital Employees in Multi-Horizon Task Environments 2026 Memory & persistent work Paper
CoworkEnv: Synthesizing Training Environments for Long-Horizon Cowork Agents [166] 2026 Workspaces & occupational work Paper · Code
Cowork-Gym: Synthesizing Interactive Work Environments at Scale [167] 2026 Workspaces & occupational work Paper
CurateEvo: Data-Curation Evolving for Agentic Post-Training [173] 2026 Learning & agent harnesses Paper
C-World: A Computer Use Agent Environment Creator 2026 Learning & agent harnesses Paper · Code
daVinci-Dev: Agent-native Mid-training for Software Engineering [169] 2026 Software, data & research Paper · Code · Publication · Data
daVinci-Env / OpenSWE: daVinci-Env: Open SWE Environment Synthesis at Scale [158] 2026 Software, data & research Paper · Code
DeepPresenter: Environment-Grounded Reflection for Agentic Presentation Generation [58] 2026 Spreadsheets, slides & documents Paper · Code · Publication
DMRL: Document-Mediated Reinforcement Learning for Skill Optimization in Advertising Recommendation [268] 2026 Tools & business workflows Paper
Dream-RSI: Recursive Self-Improvement through Evolving Worlds [2] 2026 Learning & agent harnesses Paper
DSec: DeepSeek Elastic Compute (DSec): A Sandbox Infrastructure for Effective Agentic Training at Scale [352] 2026 Learning & agent harnesses Paper
EnterpriseLab: A Full-Stack Platform for developing and deploying agents in Enterprises [171] 2026 Tools & business workflows Paper · Code · Website
EnvCraft: Synthesizing Executable Environments in Agentic RL for Claw-like Agent [164] 2026 Workspaces & occupational work Paper · Code · Data
EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis 2026 Tools & business workflows Paper · Code
FST: Learning, Fast and Slow: Towards LLMs That Adapt Continually [262] 2026 Learning & agent harnesses Paper · Publication · Website
Function-Aware FIM: Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models [191] 2026 Software, data & research Paper · Code
GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning [229] 2026 Learning & agent harnesses Paper · Code · Publication · Website
GHD / The Next Screenshot Knows: The Next Screenshot Knows: Gated Hindsight Distillation for Mobile GUI Agents [216] 2026 Web, desktop & mobile Paper
GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis [143] 2026 Workspaces & occupational work Paper · Data
Guided-OPD: On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents [211] 2026 Learning & agent harnesses Paper · Code
HiPER: Hierarchical Plan--Execute RL for Multi-Turn LLM Agents [206] 2026 Learning & agent harnesses Paper · Code · Publication · Website
InfiAgent: An Infinite-Horizon Framework for General-Purpose Autonomous Agents [55] 2026 Memory & persistent work Paper · Code · Publication
JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution [233] 2026 Learning & agent harnesses Paper · Code · Data · Website
JTPRO: A Joint Tool-Prompt Reflective Optimization Framework for Language Agents [227] 2026 Learning & agent harnesses Paper · Publication · Cited edition
LiteGUI: Lightweight GUI Agents via Multi-Solution Guided Distillation and Dual-Level Reinforcement Learning [215] 2026 Web, desktop & mobile Paper
Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning [363] 2026 Memory & persistent work Paper
MESH-Harness: Self-Improving Agent Harnesses via Bandit-Guided Compositional Evolution [242] 2026 Learning & agent harnesses Paper
Meta-Harness: End-to-End Optimization of Model Harnesses [231] 2026 Learning & agent harnesses Paper · Code · Website
MiLDEdit / MiLDEAgent: MiLDEdit: Reasoning-Based Multi-Layer Design Document Editing [113] 2026 Spreadsheets, slides & documents Paper
Nemotron midtraining: Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe [189] 2026 Learning & agent harnesses Paper
NexForge: Scaling Agent Capabilities through Requirement-Driven Task Synthesis for LLMs [159] 2026 Workspaces & occupational work Paper · Website
OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution 2026 Web, desktop & mobile Paper
On-Policy Distillation: Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe 2026 Memory & persistent work Paper · Code
OpenAgent: Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use 2026 Tools & business workflows Paper · Code
OpenThoughts-Agent: Data Recipes for Agentic Models [172] 2026 Learning & agent harnesses Paper · Code · Data
Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewards [273] 2026 Learning & agent harnesses Paper · Publication
PRISM / Beyond Prompts: Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses [228] 2026 Learning & agent harnesses Paper · Code · Data
Process Reward Models That Think [181] 2026 Verification & recovery Paper · Publication
Provider Digital Twins: Developing provider digital twins for personalized provider-patient communication via a RAG-based conversational framework [340] 2026 Personal agents & digital twins Paper
RADAR: Automating Low-Risk Code Review at Meta: RADAR, Risk Calibration, and Review Efficiency 2026 Verification & recovery Paper
ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory [235] 2026 Learning & agent harnesses Paper · Code · Publication
RePro: Retrospective Progress-Aware Self-Refinement for LLM Agent Training [223] 2026 Learning & agent harnesses Paper
ReTool: Reinforcement Learning for Strategic Tool Use in LLMs [200] 2026 Tools & business workflows Paper · Publication · Cited edition
Revisiting DAgger in the Era of LLM-Agents [213] 2026 Learning & agent harnesses Paper
RHO: Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference [256] 2026 Learning & agent harnesses Paper · Code
Rollback-Induced Reflection: Rollback the World, Keep the Reflection: Rollback-Induced Reflection for Long-Horizon LLM Agents 2026 Verification & recovery Paper
SDAR: Self-Distilled Agentic Reinforcement Learning [222] 2026 Learning & agent harnesses Paper
SDPO: Reinforcement Learning via Self-Distillation [220] 2026 Learning & agent harnesses Paper · Code · Publication · Data · Website
SEE: Structure-aware Exploring & Exploiting for Long-horizon GUI Agent Trajectory Synthesis [177] 2026 Web, desktop & mobile Paper · Code
Self-Healing Orchestrators: Self-Healing Agentic Orchestrators for Reliable Tool-Augmented Large Language Model Systems 2026 Verification & recovery Paper
SIA: Self Improving AI with Harness & Weight Updates [267] 2026 Learning & agent harnesses Paper · Code
Skill2Env: Capability-Oriented Environment Synthesis from Skills for General Agents [355] 2026 Workspaces & occupational work Paper · Code · Data
Skill-Alpha: Progressive Agent Skill Generation via Reinforcement Learning [238] 2026 Learning & agent harnesses Paper · Code
SkillOpt: Executive Strategy for Self-Evolving Agent Skills [237] 2026 Learning & agent harnesses Paper · Publication · Code · Website
SkillOS: Learning Skill Curation for Self-Evolving Agents 2026 Learning & agent harnesses Paper
SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning [265] 2026 Learning & agent harnesses Paper · Code · Publication · Data
Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents [221] 2026 Learning & agent harnesses Paper · Publication
SkillSmith: Co-Evolving Skills and Tools for Self-Improving Agent Systems [240] 2026 Learning & agent harnesses Paper
SlideRL / Learning to Present: Learning to Present: Inverse Specification Rewards for Agentic Slide Generation [134] 2026 Spreadsheets, slides & documents Paper · Code · Data
SLIM: Dynamic Skill Lifecycle Management for Agentic Reinforcement Learning [266] 2026 Learning & agent harnesses Paper · Code
SMITH: Joint Optimization of Tool Creation and Use for Large Language Model Agents [271] 2026 Learning & agent harnesses Paper · Code · Publication
Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills [269] 2026 Software, data & research Paper
SOD: Step-wise On-policy Distillation for Small Language Model Agents [214] 2026 Learning & agent harnesses Paper · Code
SpreadsheetAgent: Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format Reasoning [41] 2026 Reasoning, evidence & work impact Paper · Code · Publication
Spreadsheet-RL: Advancing Large Language Model Agents on Realistic Spreadsheet Tasks via Reinforcement Learning [129] 2026 Spreadsheets, slides & documents Paper · Code · Data · Website
SRD: Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight [224] 2026 Learning & agent harnesses Paper · Code
SSR (Self-play SWE-RL): Toward Training Superintelligent Software Agents through Self-Play SWE-RL 2026 Learning & agent harnesses Paper · Publication
StagedWorkspace: A Versioned Workspace for Knowledge-Work Agents 2026 Workspaces & occupational work Paper
StateGuard: Analytical-State Management with Validity-Aware Intervention for Long-Horizon Data Agents [375] 2026 Memory & persistent work Paper
TACO: A Self-Evolving Framework for Efficient Terminal Agents via Observational Context Compression [225] 2026 Learning & agent harnesses Paper · Code · Publication
TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agents [210] 2026 Learning & agent harnesses Paper · Code · Publication
TDScaling: Beyond Quantity: Trajectory Diversity Scaling for Code Agents [176] 2026 Tools & business workflows Paper · Publication
Tool Description Rewriter: Learning to Rewrite Tool Descriptions for Reliable LLM-Agent Tool Use [226] 2026 Learning & agent harnesses Paper
Trace2Skill (Office): Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills 2026 Learning & agent harnesses Paper · Code
TrajLong: Co-Designing Agentic and Long-Context Supervision for Mid-Training [190] 2026 Learning & agent harnesses Paper
UI-Copilot: Advancing Long-Horizon GUI Automation via Tool-Integrated Policy Optimization [259] 2026 Web, desktop & mobile Paper · Publication · Code
UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations [198] 2026 Web, desktop & mobile Paper
UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agents [264] 2026 Web, desktop & mobile Paper · Website
UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning [70] 2026 Web, desktop & mobile Paper
VERA: Scaling Verifiable Environments for Agentic co-Evolution [175] 2026 Verification & recovery Paper · Code
Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failures [377] 2026 Verification & recovery Paper
VERSE: Verified Self-Evolving Optimizer for Agent Harnesses [270] 2026 Learning & agent harnesses Paper · Code
VIGA: Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning [130] 2026 CAD & visual design Paper · Publication · Code · Data · Website
Who Verifies the Verifier? Co-Evolving Inspectable Graders with Self-Improving Agents [385] 2026 Verification & recovery Paper
WorkBorne: Discovering Agentic Tasks from Real-World Artifact Transitions [161] 2026 Workspaces & occupational work Paper
WorkForge: Scaling Verifiable Environments for Long-horizon Work Agents [160] 2026 Workspaces & occupational work Paper
WorkGenesis: Building the Worlds That Teach Agents to Work [144] 2026 Workspaces & occupational work Paper
WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution [168] 2026 Workspaces & occupational work Paper
Workspace Optimization / DreamTeam: Workspace Optimization: How to Train Your Agent 2026 Learning & agent harnesses Paper
ADAS: Automated Design of Agentic Systems [248] 2025 Learning & agent harnesses Paper · Code · Publication
AFlow: Automating Agentic Workflow Generation [230] 2025 Learning & agent harnesses Paper · Code · Publication
Agent-as-a-Judge: Evaluate Agents with Agents [15] 2025 Verification & recovery Paper · Publication · Code · Data
AgentDebug: Where LLM Agents Fail and How They can Learn From Failures 2025 Verification & recovery Paper · Code
AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation 2025 Learning & agent harnesses Paper · Publication
AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning [356] 2025 Learning & agent harnesses Paper · Code · Publication
Agent Lightning: Train ANY AI Agents with Reinforcement Learning [351] 2025 Learning & agent harnesses Paper · Code
Agent-R1: A Unified and Modular Framework for Agentic Reinforcement Learning 2025 Learning & agent harnesses Paper · Code
Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents [52] 2025 Web, desktop & mobile Paper · Code · Publication
AgentSquare: Automatic LLM Agent Search in Modular Design Space [241] 2025 Learning & agent harnesses Paper · Code · Publication
AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials [170] 2025 Web, desktop & mobile Paper
A-MEM: Agentic Memory for LLM Agents [362] 2025 Memory & persistent work Paper
Authenticated Delegation and Authorized AI Agents [341] 2025 Safety & authorization Paper
AutoPresent / SlidesBench: AutoPresent: Designing Structured Visuals from Scratch [120] 2025 Spreadsheets, slides & documents Paper · Code · Publication · Cited edition
AWM: Agent Workflow Memory [369] 2025 Learning & agent harnesses Paper · Code · Publication
BrowserGym: The BrowserGym Ecosystem for Web Agent Research [320] 2025 Learning & agent harnesses Paper · Code · Publication · Website
CoAct-1: Computer-using Multi-Agent System with Coding Actions 2025 Learning & agent harnesses Paper · Code
ColPali / ViDoRe: ColPali: Efficient Document Retrieval with Vision Language Models [147] 2025 Reasoning, evidence & work impact Paper · Code
DART-GUI: Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation 2025 Learning & agent harnesses Paper
Data Interpreter: An LLM Agent For Data Science [51] 2025 Software, data & research Paper · Code · Publication
DGAP: Discriminator-Guided Embodied Planning for LLM Agent 2025 Reasoning, evidence & work impact Paper
Docling Toolkit: Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion 2025 Tools & business workflows Paper
DynaSaur: Large Language Agents Beyond Predefined Actions 2025 Learning & agent harnesses Paper · Code · Publication
GiGPO: Group-in-Group Policy Optimization for LLM Agent Training [207] 2025 Learning & agent harnesses Paper
GoalAct: Enhancing LLM-Based Agents via Global Planning and Hierarchical Execution 2025 Reasoning, evidence & work impact Paper
GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents [68] 2025 Web, desktop & mobile Paper · Code · Publication
HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model 2025 Reasoning, evidence & work impact Paper
LocAgent: Graph-Guided LLM Agents for Code Localization [151] 2025 Software, data & research Paper · Publication
MAP: A brain-inspired agentic architecture to improve planning with LLMs 2025 Reasoning, evidence & work impact Paper
Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory [360] 2025 Memory & persistent work Paper · Code
MLGym: A New Framework and Benchmark for Advancing AI Research Agents 2025 Software, data & research Paper · Code
OpenHands: An Open Platform for AI Software Developers as Generalist Agents [152] 2025 Learning & agent harnesses Paper · Code · Publication
Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks 2025 Reasoning, evidence & work impact Paper
PosterGen: Aesthetic-Aware Multi-Modal Paper-to-Poster Generation via Multi-Agent LLMs 2025 Spreadsheets, slides & documents Paper
PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slides [5] 2025 Spreadsheets, slides & documents Paper · Code · Publication
PPTArena: A Benchmark for PowerPoint Editing 2025 Spreadsheets, slides & documents Paper · Code
PreGenie: An Agentic Framework for High-quality Visual Presentation Generation 2025 Spreadsheets, slides & documents Paper · Publication
Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance [332] 2025 Memory & persistent work Paper · Code
R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning 2025 Learning & agent harnesses Paper · Code
R2E-Gym: Procedural Environment Generation and Hybrid Verifiers for Scaling Open-Weights SWE Agents [354] 2025 Software, data & research Paper · Publication · Publication
RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph 2025 Software, data & research Paper · Publication
Routine: A Structural Planning Framework for LLM Agent System in Enterprise [48] 2025 Tools & business workflows Paper
SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planning [381] 2025 Memory & persistent work Paper · Publication
Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning [201] 2025 Learning & agent harnesses Paper · Code · Publication
SheetAgent: Towards A Generalist Agent for Spreadsheet Reasoning and Manipulation via Large Language Models [45] 2025 Spreadsheets, slides & documents Paper · Publication · Cited edition
SheetMind: Actions Set Accuracy, Agents Set the Failure Mode 2025 Spreadsheets, slides & documents Paper
SlideTailor: Personalized Presentation Slide Generation for Scientific Papers 2025 Spreadsheets, slides & documents Paper
SWE-Gym: Training Software Engineering Agents and Verifiers with SWE-Gym [353] 2025 Software, data & research Paper · Code · Publication
SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution [204] 2025 Learning & agent harnesses Paper · Code · Publication
SWE-smith: Scaling Data for Software Engineering Agents 2025 Learning & agent harnesses Paper · Publication
SyncMind: Measuring Agent Out-of-Sync Recovery in Collaborative Software Engineering 2025 Software, data & research Paper · Code · Data
Talk to Your Slides: High-Efficiency Slide Editing via Language-Driven Structured Data Manipulation 2025 Spreadsheets, slides & documents Paper
TextGrad: Optimizing generative AI by backpropagating language model feedback [251] 2025 Learning & agent harnesses Paper · Code · Publication · Cited edition
ToolMaker: LLM Agents Making Agent Tools [272] 2025 Learning & agent harnesses Paper
Tool-to-Agent Retrieval: Bridging Tools and Agents for Scalable LLM Multi-Agent Systems 2025 Tools & business workflows Paper
ToRL: Scaling Tool-Integrated RL 2025 Learning & agent harnesses Paper · Code
UFO2: The Desktop AgentOS 2025 Web, desktop & mobile Paper · Code
UGround: Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents [67] 2025 Web, desktop & mobile Paper · Publication
UI-S1: Advancing GUI Automation via Semi-Online Reinforcement Learning [71] 2025 Web, desktop & mobile Paper
UI-TARS: Pioneering Automated GUI Interaction with Native Agents [197] 2025 Web, desktop & mobile Paper · Code
UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action 2025 Learning & agent harnesses Paper
WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning 2025 Learning & agent harnesses Paper · Code · Publication
WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning [202] 2025 Web, desktop & mobile Paper · Code · Publication
WorkTeam: Constructing Workflows from Natural Language with Multi-Agents [49] 2025 Learning & agent harnesses Paper · Publication
Zep: A Temporal Knowledge Graph Architecture for Agent Memory [358] 2025 Memory & persistent work Paper · Code
Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models [194] 2024 Learning & agent harnesses Paper · Code · Publication
AgentGym: Evolving Large Language Model-based Agents across Diverse Environments 2024 Learning & agent harnesses Paper · Code
AgentInstruct (Microsoft): AgentInstruct: Toward Generative Teaching with Agentic Flows 2024 Learning & agent harnesses Paper
Agentless: Demystifying LLM-based Software Engineering Agents 2024 Software, data & research Paper · Code
AgentOptimizer: Offline Training of Language Model Agents with Functions as Learnable Weights 2024 Learning & agent harnesses Paper · Publication
Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents [219] 2024 Web, desktop & mobile Paper
Agent S: An Open Agentic Framework that Uses Computers Like a Human 2024 Web, desktop & mobile Paper · Code
AgentTuning: Enabling Generalized Agent Abilities for LLMs [193] 2024 Learning & agent harnesses Paper · Code · Publication
AsyncLM: Asynchronous LLM Function Calling [365] 2024 Learning & agent harnesses Paper
AutoCodeRover: Autonomous Program Improvement [43] 2024 Software, data & research Paper · Publication
AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversations [53] 2024 Learning & agent harnesses Paper · Code · Publication
BetterTogether: Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together [261] 2024 Learning & agent harnesses Paper · Publication
BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement [85] 2024 CAD & visual design Paper · Code · Data
BPO: Black-Box Prompt Optimization: Aligning Large Language Models without Model Training [245] 2024 Learning & agent harnesses Paper · Code · Publication
Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding 2024 Tools & business workflows Paper
CodeAct: Executable Code Actions Elicit Better LLM Agents [46] 2024 Tools & business workflows Paper · Code · Publication
CogAgent: A Visual Language Model for GUI Agents [8] 2024 Web, desktop & mobile Paper
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing [57] 2024 Verification & recovery Paper · Code
Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflow 2024 Tools & business workflows Paper · Publication
DesiGen: A Pipeline for Controllable Design Template Generation 2024 Spreadsheets, slides & documents Paper
DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning [203] 2024 Web, desktop & mobile Paper · Code · Publication · Cited edition
Docling Technical Report 2024 Reasoning, evidence & work impact Paper · Code
DocLLM: A Layout-Aware Generative Language Model for Multimodal Document Understanding [183] 2024 Reasoning, evidence & work impact Paper
DSPy: Compiling Declarative Language Model Calls into State-of-the-Art Pipelines [246] 2024 Learning & agent harnesses Paper · Code · Publication
EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers [253] 2024 Learning & agent harnesses Paper · Code · Publication
ExpeL: LLM Agents Are Experiential Learners [367] 2024 Memory & persistent work Paper · Code · Publication · Cited edition
GKD: On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes [209] 2024 Learning & agent harnesses Paper · Publication
Gorilla: Large Language Model Connected with Massive APIs [26] 2024 Learning & agent harnesses Paper · Publication
Graph-enhanced Asynchronous Planning: Graph-enhanced Large Language Models in Asynchronous Plan Reasoning [366] 2024 Tools & business workflows Paper
Human Digital Twin Architecture: A Human Digital Twin Architecture for Knowledge-based Interactions and Context-Aware Conversations [337] 2024 Personal agents & digital twins Paper
LATS: Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models [61] 2024 Verification & recovery Paper · Code · Publication
Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks [335] 2024 Learning & agent harnesses Paper · Code
mEdIT: Multilingual Text Editing via Instruction Tuning 2024 Learning & agent harnesses Paper
MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework [54] 2024 Software, data & research Paper · Code · Publication
MinerU: An Open-Source Solution for Precise Document Content Extraction [89] 2024 Reasoning, evidence & work impact Paper · Code
MIPRO: Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs [247] 2024 Learning & agent harnesses Paper · Publication
M-LongDoc: A Benchmark For Multimodal Super-Long Document Understanding And A Retrieval-Aware Tuning Framework 2024 Reasoning, evidence & work impact Paper
mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding 2024 Reasoning, evidence & work impact Paper
mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding 2024 Reasoning, evidence & work impact Paper · Code
NL2Formula: Generating Spreadsheet Formulas from Natural Language Queries [106] 2024 Spreadsheets, slides & documents Paper
OmniParser for Pure Vision Based GUI Agent [69] 2024 Web, desktop & mobile Paper
OPRO: Large Language Models as Optimizers [244] 2024 Learning & agent harnesses Paper · Code · Publication
OS-Copilot: Towards Generalist Computer Agents with Self-Improvement [74] 2024 Web, desktop & mobile Paper · Publication
PDFTriage: Question Answering over Long, Structured Documents [97] 2024 Reasoning, evidence & work impact Paper · Publication
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models [319] 2024 Verification & recovery Paper
PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization 2024 Learning & agent harnesses Paper · Code · Publication
Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution [254] 2024 Learning & agent harnesses Paper · Publication
SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents [72] 2024 Web, desktop & mobile Paper · Code
ShowUI: One Vision-Language-Action Model for GUI Visual Agent 2024 Web, desktop & mobile Paper
SpreadsheetLLM: Encoding Spreadsheets for Large Language Models [42] 2024 Spreadsheets, slides & documents Paper · Publication
STORM: Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models 2024 Spreadsheets, slides & documents Paper
SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering [3] 2024 Software, data & research Paper · Publication · Cited edition · Code · Website
TableGPT2: A Large Multimodal Model with Tabular Data Integration 2024 Learning & agent harnesses Paper
TableLlama: Towards Open Large Generalist Models for Tables [88] 2024 Learning & agent harnesses Paper
Text2Chart31: Instruction Tuning for Chart Generation with Automatic Feedback 2024 Spreadsheets, slides & documents Paper
ToolLLM / ToolBench: ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs 2024 Tools & business workflows Paper · Code · Publication
Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMs [255] 2024 Learning & agent harnesses Paper · Publication
UFO: A UI-Focused Agent for Windows OS Interaction 2024 Web, desktop & mobile Paper · Code
Voyager: An Open-Ended Embodied Agent with Large Language Models [368] 2024 Memory & persistent work Paper · Publication · Cited edition
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models [73] 2024 Web, desktop & mobile Paper · Publication
Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examples 2023 Spreadsheets, slides & documents Paper
CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society 2023 Learning & agent harnesses Paper
ChatDev: Communicative Agents for Software Development 2023 Software, data & research Paper
CoEdIT: Text Editing by Task-Specific Instruction Tuning 2023 Spreadsheets, slides & documents Paper
Dater: Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning 2023 Reasoning, evidence & work impact Paper
FireAct: Toward Language Agent Fine-tuning [195] 2023 Tools & business workflows Paper
FormaT5: Abstention and Examples for Conditional Table Formatting with Natural Language [107] 2023 Spreadsheets, slides & documents Paper · Publication
Generative Agents: Interactive Simulacra of Human Behavior [361] 2023 Memory & persistent work Paper · Publication
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment [318] 2023 Verification & recovery Paper · Publication
Hi-VT5 / MP-DocVQA: Hierarchical Multimodal Transformers for Multipage DocVQA [96] 2023 Reasoning, evidence & work impact Paper
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face 2023 Tools & business workflows Paper · Code
LayoutGPT: Compositional Visual Planning and Generation with Large Language Models 2023 Reasoning, evidence & work impact Paper · Code
LayoutPrompter: TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering 2023 Spreadsheets, slides & documents Paper
LLM+P: Empowering Large Language Models with Optimal Planning Proficiency [47] 2023 Tools & business workflows Paper
Medprompt: Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine 2023 Learning & agent harnesses Paper
MemGPT: Towards LLMs as Operating Systems [234] 2023 Memory & persistent work Paper · Code
mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding 2023 Reasoning, evidence & work impact Paper
PAL: Program-aided Language Models [99] 2023 Tools & business workflows Paper
Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks [100] 2023 Tools & business workflows Paper
ProTeGi: Automatic Prompt Optimization with “Gradient Descent” and Beam Search [250] 2023 Learning & agent harnesses Paper · Code · Publication
ReAct: Synergizing Reasoning and Acting in Language Models [13] 2023 Tools & business workflows Paper · Code · Publication · Publication
Reflexion: Language Agents with Verbal Reinforcement Learning [60] 2023 Verification & recovery Paper · Code · Publication
RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting 2023 Learning & agent harnesses Paper
RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback 2023 Learning & agent harnesses Paper
Self-Refine: Iterative Refinement with Self-Feedback [59] 2023 Verification & recovery Paper · Code · Publication
SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models [4] 2023 Spreadsheets, slides & documents Paper · Code · Publication · Cited edition
StructGPT: A General Framework for Large Language Model to Reason over Structured Data 2023 Tools & business workflows Paper
Table-GPT: Table-tuned GPT for Diverse Table Tasks 2023 Learning & agent harnesses Paper
ToolEmu: Identifying the Risks of LM Agents with an LM-Emulated Sandbox 2023 Safety & authorization Paper
Toolformer: Language Models Can Teach Themselves to Use Tools [24] 2023 Tools & business workflows Paper · Publication · Cited edition
BINDER: Binding Language Models in Symbolic Languages 2022 Tools & business workflows Paper
DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents [119] 2022 Spreadsheets, slides & documents Paper · Publication
LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking 2022 Reasoning, evidence & work impact Paper · Code
PEER: A Collaborative Language Model 2022 Spreadsheets, slides & documents Paper
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning [252] 2022 Learning & agent harnesses Paper · Code · Publication
TAPEX: Table Pre-training via Learning a Neural SQL Executor [184] 2022 Learning & agent harnesses Paper
Codex / HumanEval: Evaluating Large Language Models Trained on Code 2021 Learning & agent harnesses Paper
D2S: Document-to-Slide Generation Via Query-Based Text Summarization 2021 Spreadsheets, slides & documents Paper
LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding 2021 Reasoning, evidence & work impact Paper
SpreadsheetCoder: Formula Prediction from Semi-structured Context [105] 2021 Spreadsheets, slides & documents Paper
TAPAS: Weakly Supervised Table Parsing via Pre-training [87] 2020 Reasoning, evidence & work impact Paper
NLyze: Interactive Programming by Natural Language for SpreadSheet Data Analysis and Manipulation [110] 2014 Spreadsheets, slides & documents Paper · Paper
Flash Fill: Automating String Processing in Spreadsheets using Input-Output Examples [109] 2011 Spreadsheets, slides & documents Paper · Publication

Benchmarks & Evaluation

Benchmarks, datasets, evaluation environments, and protocols for measuring task completion, work quality, reliability, and cost. 190 entries.

Paper Year Focus Links
τ²-Bench: $tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment [30] 2026 Tools & business workflows Paper · Code · Publication · Data
AgentIF-OneDay: A Task-level Instruction-Following Benchmark for General AI Agents in Daily Scenarios [306] 2026 Workspaces & occupational work Paper · Data · Website
Agents’ Last Exam: Agents' Last Exam [285] 2026 Workspaces & occupational work Paper · Data
AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs 2026 Learning & agent harnesses Paper · Code
AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning [155] 2026 Workspaces & occupational work Paper
AIDABench: AI Data Analytics Benchmark 2026 Workspaces & occupational work Paper · Code
APEX-Agents [157] 2026 Workspaces & occupational work Paper · Data · Code
Auditing Financial Agent Self-Evolution: Auditing Self-Evolution in Financial Agents: Capability Gains, Security Drift, and Execution-Interface Mismatch [387] 2026 Safety & authorization Paper
AutomationBench (Zapier): AutomationBench [140] 2026 Tools & business workflows Paper · Code · Website
BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows [281] 2026 Workspaces & occupational work Paper · Code · Publication · Data
BenchCAD: A Comprehensive, Industry-Standard Benchmark for Programmatic CAD [283] 2026 Software, data & research Paper · Code · Data · Website
BlueFin: Benchmarking LLM Agents on Financial Spreadsheets 2026 Spreadsheets, slides & documents Paper
CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation [86] 2026 CAD & visual design Paper · Code · Data · Website
CentaurBench: Benchmarking LLM Capabilities on Augmenting vs. Automating Real-World Work Tasks [314] 2026 Workspaces & occupational work Paper · Code · Data · Website
CHI-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows? [138] 2026 Tools & business workflows Paper · Publication
CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agents [303] 2026 Safety & authorization Paper · Code · Publication · Code
CL-bench: A Benchmark for Context Learning [315] 2026 Reasoning, evidence & work impact Paper · Code · Data · Website
Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration [141] 2026 Human-agent collaboration Paper · Code · Publication
ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox 2026 Tools & business workflows Paper
CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions [302] 2026 Tools & business workflows Paper · Code · Data · Publication
CXBench: A Benchmark for Process-Compliant LLM Agents in Regulated Customer-Experience Workflows [131] 2026 Tools & business workflows Paper
DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle [137] 2026 Software, data & research Paper · Publication · Cited edition
DataSciBench: An LLM Agent Benchmark for Data Science [118] 2026 Software, data & research Paper · Code · Publication
DELEGATE-52: LLMs Corrupt Your Documents When You Delegate [104] 2026 Spreadsheets, slides & documents Paper · Code
DRBench: A Realistic Benchmark for Enterprise Deep Research [146] 2026 Workspaces & occupational work Paper · Publication
Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing 2026 Reasoning, evidence & work impact Paper · Code
DRFLOW: A Deep Research Benchmark for Personalized Workflow Prediction [357] 2026 Tools & business workflows Paper
DrugDiscoveryBench: Can Coding Agents Assist Early-Stage Drug Discovery? 2026 Software, data & research Paper · Code · Data
E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios 2026 Tools & business workflows Paper
E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation [33] 2026 Tools & business workflows Paper · Code · Data · Website
EntCollabBench: Beyond the All-in-One Agent: Benchmarking Role-Specialized Multi-Agent Collaboration in Enterprise Workflows [38] 2026 Tools & business workflows Paper
EnterpriseClawBench: Benchmarking Agents from Real Workplace Sessions 2026 Tools & business workflows Paper
EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings [139] 2026 Tools & business workflows Paper · Publication · Code · Data · Website
EXP-Bench: Can AI Conduct AI Research Experiments? [135] 2026 Software, data & research Paper · Publication
Facts Without Rules: Boundary Metadata Collapse in Multi-Agent LLM Handoffs [342] 2026 Safety & authorization Paper · Code
FieldWorkArena: Agentic AI Benchmark for Real Field Work Tasks [305] 2026 Web, desktop & mobile Paper · Code · Data
FrontierFinance: A Challenging Benchmark for Measuring Frontier Intelligence of Finance Agents [282] 2026 Finance & professional research Paper · Code · Data · Website
GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents [91] 2026 Reasoning, evidence & work impact Paper · Data · Code · Website
GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks [279] 2026 Workspaces & occupational work Paper · Publication · Publication · Data · Website
Harness-Bench: Evaluating Agent Harnesses across Models and Realistic Workflows [325] 2026 Evaluation infrastructure Paper
HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents [294] 2026 Tools & business workflows Paper · Code · Data · Website
HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help? 2026 Software, data & research Paper · Code · Data
Holistic Agent Leaderboard (HAL): Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation [324] 2026 Evaluation infrastructure Paper
τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge [39] 2026 Tools & business workflows Paper
LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growth [150] 2026 Memory & persistent work Paper · Code · Website
LongDS-Bench: On the Failure of Long-Horizon Agentic Data Analysis 2026 Memory & persistent work Paper · Code
MASBench: Benchmarking LLM-based Multi-Agent Collaboration under Partial Observability 2026 Learning & agent harnesses Paper · Code
MBABench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance 2026 Spreadsheets, slides & documents Paper
MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers [307] 2026 Tools & business workflows Paper · Publication · Code · Data
MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers [124] 2026 Tools & business workflows Paper · Code · Publication · Cited edition
Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks [310] 2026 Web, desktop & mobile Paper · Website
Office Comprehension Benchmark 2026 Reasoning, evidence & work impact Paper · Code · Data
OfficeEval: Mind the Gap: Can Frontier LLMs Pass a Standardized Office Proficiency Exam? 2026 Workspaces & occupational work Paper
OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoning [145] 2026 Reasoning, evidence & work impact Paper · Code · Data · Website
OmegaUse-OfficeVal: Benchmarking LLM Agents on Long-Horizon Office-Suite Tasks with Economic Grounding [301] 2026 Workspaces & occupational work Paper · Code · Data
OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios [317] 2026 Tools & business workflows Paper · Code · Data · Website
$OneMillion-Bench: How Far are Language Agents from Human Experts? [313] 2026 Workspaces & occupational work Paper · Publication · Code · Data
OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks [32] 2026 Web, desktop & mobile Paper · Code · Data · Website
PAST-Bench / Hermes+: PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents [334] 2026 Memory & persistent work Paper · Code
PPT-Eval: A Benchmark for Computer-Use Agents on PowerPoint Tasks 2026 Spreadsheets, slides & documents Paper · Code
PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning [80] 2026 Reasoning, evidence & work impact Paper · Publication · Code · Data · Website
PresentBench: A Fine-Grained Rubric-Based Benchmark for Slide Generation [17] 2026 Spreadsheets, slides & documents Paper · Code · Data · Website
PSEBench: A Controllable and Verifiable Benchmark for Evaluating LLMs in Patient Safety Event Triage [298] 2026 Reasoning, evidence & work impact Paper
READY: READY or Not: Reliable Enterprise Agent Deployment [330] 2026 Verification & recovery Paper · Website
SCUBA: Salesforce Computer Use Benchmark 2026 Tools & business workflows Paper · Publication
SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks 2026 Workspaces & occupational work Paper
SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows [121] 2026 Spreadsheets, slides & documents Paper · Code · Data · Accepted · Publication
StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows [280] 2026 Workspaces & occupational work Paper · Website
StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance [364] 2026 Memory & persistent work Paper · Code
SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution 2026 Software, data & research Paper · Code
SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks? 2026 Software, data & research Paper · Publication
SWE-INTERACT: Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding Sessions 2026 Software, data & research Paper · Code
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces [122] 2026 Web, desktop & mobile Paper · Code · Publication · Cited edition · Website
Toolathlon: The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution [149] 2026 Tools & business workflows Paper · Code · Publication
ToolBench-X: Beyond Function Calling: Benchmarking Tool-Using Agents under Tool-Environment Unreliability 2026 Verification & recovery Paper · Code
TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation [339] 2026 Personal agents & digital twins Paper
VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World? [34] 2026 Memory & persistent work Paper
VibeSearchBench: Benchmarking Long-horizon Proactive Search in the Wild 2026 Reasoning, evidence & work impact Paper
WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces 2026 Web, desktop & mobile Paper · Code · Data
WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation [311] 2026 Web, desktop & mobile Paper · Code · Data · Website
WorkBuddy Bench: Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction 2026 Workspaces & occupational work Paper
Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies [36] 2026 Workspaces & occupational work Paper · Code · Publication · Data · Website
WTM-Bench: Back to the Future: A workbook time machine for spread sheet creation benchmarks [108] 2026 Spreadsheets, slides & documents Paper · Code · Data · Publication
XL-DocBench: Benchmarking Evidence-Grounded Extra-Long Document Understanding [95] 2026 Reasoning, evidence & work impact Paper · Publication
YC-Bench: Benchmarking AI Agents for Long-Term Planning and Consistent Execution 2026 Memory & persistent work Paper · Code
AAAR-1.0: Assessing AI’s Potential to Assist Research [78] 2025 Software, data & research Paper · Publication
AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories [370] 2025 Verification & recovery Paper · Publication
AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds [123] 2025 Software, data & research Paper
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents [27] 2025 Web, desktop & mobile Paper · Code · Publication · Data · Website
τ-bench: $tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains [29] 2025 Tools & business workflows Paper · Code · Publication
BFCL: The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models [35] 2025 Tools & business workflows Paper · Code
BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents 2025 Reasoning, evidence & work impact Paper
ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering [90] 2025 Reasoning, evidence & work impact Paper · Publication
CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments [148] 2025 Tools & business workflows Paper · Code · Publication · Data
DABstep: Data Agent Benchmark for Multi-step Reasoning [126] 2025 Software, data & research Paper
DiscoveryBench: Towards Data-Driven Discovery with Large Language Models [287] 2025 Software, data & research Paper · Code · Data · Publication
Document Haystack: A Long Context Multimodal Image/Document Understanding Vision LLM Benchmark 2025 Reasoning, evidence & work impact Paper · Code · Data
DSBench: How Far Are Data Science Agents from Becoming Data Science Experts? [286] 2025 Software, data & research Paper · Code · Publication · Data · Website
EnterpriseBench: Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments [304] 2025 Tools & business workflows Paper · Publication · Code · Data · Website
Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Tasks [289] 2025 Reasoning, evidence & work impact Paper · Benchmark v2 · Code · Website
FINCH / FinWorkBench: Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflows 2025 Workspaces & occupational work Paper · Code · Data
FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering [79] 2025 Reasoning, evidence & work impact Paper · Publication
Flex-TravelPlanner: A Benchmark for Flexible Planning with Language Agents 2025 Tools & business workflows Paper
Hell or High Water: Evaluating Agentic Recovery from External Failures 2025 Verification & recovery Paper · Code
HERB: Benchmarking Deep Search over Heterogeneous Enterprise Data [156] 2025 Workspaces & occupational work Paper · Publication
Idea2Plan: Exploring AI-Powered Research Planning [82] 2025 Software, data & research Paper
LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks [92] 2025 Reasoning, evidence & work impact Paper · Publication
LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory [65] 2025 Memory & persistent work Paper · Code · Publication
MAST: Why Do Multi-Agent LLM Systems Fail? [343] 2025 Verification & recovery Paper · Code · Data
MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers [125] 2025 Tools & business workflows Paper
MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents [116] 2025 Tools & business workflows Paper · Code
METR time horizon: Measuring AI Ability to Complete Long Software Tasks [63] 2025 Memory & persistent work Paper · Publication
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering 2025 Software, data & research Paper · Publication
MMDocIR: Benchmarking Multimodal Retrieval for Long Documents 2025 Reasoning, evidence & work impact Paper · Code
MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly [81] 2025 Reasoning, evidence & work impact Paper · Publication
MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark [77] 2025 Reasoning, evidence & work impact Paper · Publication
MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents [309] 2025 Learning & agent harnesses Paper · Code · Publication · Data
Multi-Mission Tool Bench: Assessing the Robustness of LLM based Agents through Related and Dynamic Missions [308] 2025 Tools & business workflows Paper · Code · Data · Website
OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflows [300] 2025 Workspaces & occupational work Paper
PaperBench: Evaluating AI's Ability to Replicate AI Research [136] 2025 Software, data & research Paper · Code · Publication · Code · Data
Ready Jurist One: Benchmarking Language Agents for Legal Intelligence in Dynamic Environments 2025 Tools & business workflows Paper
RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts [64] 2025 Software, data & research Paper · Code · Publication
Remote Labor Index: Measuring AI Automation of Remote Work [288] 2025 Workspaces & occupational work Paper · Code · Data · Website
ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents 2025 Reasoning, evidence & work impact Paper
ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery [10] 2025 Software, data & research Paper · Code · Data · Publication · Code · Website
ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use [115] 2025 Web, desktop & mobile Paper · Code · Data · Publication
Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows [154] 2025 Tools & business workflows Paper · Code · Publication · Website
SWE-bench Goes Live! 2025 Software, data & research Paper · Publication
SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering? [153] 2025 Software, data & research Paper · Publication
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks [31] 2025 Tools & business workflows Paper · Code · Publication · Cited edition · Website
ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities [127] 2025 Tools & business workflows Paper · Code · Publication · Code
Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale [312] 2025 Web, desktop & mobile Paper · Code · Publication · Website
WorFBench: Benchmarking Agentic Workflow Generation [84] 2025 Learning & agent harnesses Paper · Publication
AgentBench: Evaluating LLMs as Agents [275] 2024 Tools & business workflows Paper · Publication
AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents [276] 2024 Tools & business workflows Paper · Publication
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents 2024 Safety & authorization Paper · Code
AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents 2024 Safety & authorization Paper
AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents [37] 2024 Tools & business workflows Paper · Code · Publication · Website
AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks? 2024 Reasoning, evidence & work impact Paper
DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models 2024 Workspaces & occupational work Paper
DocBench: A Benchmark for Evaluating LLM-based Document Reading Systems 2024 Reasoning, evidence & work impact Paper · Code
GAIA: a benchmark for General AI Assistants [40] 2024 Workspaces & occupational work Paper · Publication · Data
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents 2024 Safety & authorization Paper
LawBench: Benchmarking Legal Knowledge of Large Language Models [292] 2024 Reasoning, evidence & work impact Paper · Code · Publication · Data
LoCoMo: Evaluating Very Long-Term Conversational Memory of LLM Agents [66] 2024 Memory & persistent work Paper · Code · Data · Publication
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding [94] 2024 Workspaces & occupational work Paper · Publication
LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locating 2024 Reasoning, evidence & work impact Paper · Code
MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback [28] 2024 Tools & business workflows Paper · Code · Data · Publication
MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark 2024 Verification & recovery Paper
MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizations [299] 2024 Reasoning, evidence & work impact Paper · Code · Data · Publication · Data · Website
OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation [76] 2024 Workspaces & occupational work Paper · Code · Data
OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations 2024 Reasoning, evidence & work impact Paper · Code
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments [7] 2024 Web, desktop & mobile Paper · Code · Data · Publication · Website
PPTC Benchmark: Evaluating Large Language Models for PowerPoint Task Completion [112] 2024 Spreadsheets, slides & documents Paper · Code · Publication · Data
RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems [44] 2024 Software, data & research Paper · Code · Publication
SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation [56] 2024 Spreadsheets, slides & documents Paper · Code · Data · Publication · Website
SWE-bench: Can Language Models Resolve Real-World GitHub Issues? [16] 2024 Software, data & research Paper · Publication · Cited edition · Code · Data · Website
SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents 2024 Software, data & research Paper
TableBench: A Comprehensive and Complex Benchmark for Table Question Answering 2024 Workspaces & occupational work Paper
TimeArena: Shaping Efficient Multitasking Language Agents in a Time-Aware Simulation [128] 2024 Memory & persistent work Paper
TravelPlanner: A Benchmark for Real-World Planning with Language Agents [83] 2024 Tools & business workflows Paper · Code · Data · Publication
VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks 2024 Web, desktop & mobile Paper
VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks [284] 2024 Web, desktop & mobile Paper · Publication · Code · Data · Website
WebArena: A Realistic Web Environment for Building Autonomous Agents [6] 2024 Web, desktop & mobile Paper · Publication · Code · Data · Website
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue [278] 2024 Web, desktop & mobile Paper · Publication
WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks? [9] 2024 Web, desktop & mobile Paper · Publication · Code · Data · Website
WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks [133] 2024 Web, desktop & mobile Paper · Code · Publication · Cited edition · Data · Website
WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting [114] 2024 Tools & business workflows Paper · Code · Publication · Data
API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs [25] 2023 Tools & business workflows Paper · Code · Publication · Data
BIRD: Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs 2023 Workspaces & occupational work Paper
FinanceBench: A New Benchmark for Financial Question Answering [93] 2023 Reasoning, evidence & work impact Paper · Code · Data
InstructExcel: A Benchmark for Natural Language Instruction in Excel [111] 2023 Spreadsheets, slides & documents Paper · Code
LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models [101] 2023 Reasoning, evidence & work impact Paper · Publication · Cited edition · Code · Data
Mind2Web: Towards a Generalist Agent for the Web [277] 2023 Web, desktop & mobile Paper · Publication
MT-Bench / Chatbot Arena: Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena [371] 2023 Verification & recovery Paper · Code · Publication · Code · Data
ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning [23] 2022 Reasoning, evidence & work impact Paper · Code · Publication
ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering [98] 2022 Reasoning, evidence & work impact Paper · Code · Publication · Data
DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation 2022 Software, data & research Paper
PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change 2022 Reasoning, evidence & work impact Paper · Code
ScienceQA: Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering [103] 2022 Reasoning, evidence & work impact Paper · Code · Data · Publication · Cited edition
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents [117] 2022 Web, desktop & mobile Paper · Publication
DocVQA: A Dataset for VQA on Document Images [22] 2021 Reasoning, evidence & work impact Paper · Publication · Cited edition · Data · Website
FinQA: A Dataset of Numerical Reasoning over Financial Data [21] 2021 Reasoning, evidence & work impact Paper · Code · Publication · Data
TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance [20] 2021 Reasoning, evidence & work impact Paper · Code · Publication · Code · Data · Website
PubMedQA: A Dataset for Biomedical Research Question Answering [102] 2019 Reasoning, evidence & work impact Paper · Code · Publication
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering [19] 2018 Reasoning, evidence & work impact Paper · Code · Publication
Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task 2018 Workspaces & occupational work Paper
SQuAD: 100,000+ Questions for Machine Comprehension of Text [18] 2016 Reasoning, evidence & work impact Paper · Publication
JobBench: Aligning Agent Work With Human Will [316] 2026 Workspaces & occupational work Paper · Code · Data
Legal Research Bench: Measuring End-to-End Reliability in Long-Horizon Legal Research Agents [293] 2026 Reasoning, evidence & work impact Paper · Code · Benchmark

Surveys & Foundations

Research syntheses, foundation-model reports, conceptual foundations, reasoning and verification studies, and empirical work on productivity and collaboration. 92 entries.

Paper Year Focus Links
Agentic ROI: Position: The Real Barrier to LLM Agent Usability is Agentic ROI [329] 2026 Reasoning, evidence & work impact Paper · Paper
Agent Skills Survey: Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward 2026 Reasoning, evidence & work impact Paper
AI Agent Systems Survey: AI Agent Systems: Architectures, Applications, and Evaluation 2026 Learning & agent harnesses Paper
AI Petition Platform Study: Introducing AI to an online petition platform changed outputs but not outcomes [374] 2026 Work impact & productivity Paper
Code as Agent Harness 2026 Learning & agent harnesses Paper
Coding Assistants Longitudinal Study: The Impact of AI Coding Assistants on Software Engineering: A Longitudinal Study 2026 Reasoning, evidence & work impact Paper
Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design 2026 Memory & persistent work Paper
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence [182] 2026 Foundation models & agentic training Paper
Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering 2026 Learning & agent harnesses Paper
From Question Answering to Task Completion: A Survey on Agent System and Harness Design 2026 Learning & agent harnesses Paper · Code
Generative Agent Simulations: LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals [338] 2026 Personal agents & digital twins Paper · Paper
GLM-5: from Vibe Coding to Agentic Engineering [187] 2026 Reasoning, evidence & work impact Paper
Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems 2026 Learning & agent harnesses Paper
How Well Does Agent Development Reflect Real-World Work? [328] 2026 Reasoning, evidence & work impact Paper · Code
Intelligent AI Delegation 2026 Safety & authorization Paper
Kimi K2.5: Visual Agentic Intelligence [186] 2026 Foundation models & agentic training Paper
On-Policy Distillation Survey: A Survey of On-Policy Distillation for Large Language Models 2026 Reasoning, evidence & work impact Paper
Open-World Evaluations for Measuring Frontier AI Capabilities 2026 Reasoning, evidence & work impact Paper
Proactivity-Gym: Foundations of Proactive Agents: Principles, Technical Layers, and Proactivity-Gym [333] 2026 Memory & persistent work Paper
Qwen3.8-Next: On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [192] 2026 Foundation models & agentic training Paper
Reliable Self-Evolving Agents: Diving into Reliable Self-Evolving Agents: A Survey 2026 Memory & persistent work Paper · Code
Self-Improvements in Modern Agentic Systems: A Survey 2026 Memory & persistent work Paper
Shift to Agentic AI: The Shift to Agentic AI: Evidence from Codex 2026 Reasoning, evidence & work impact Paper
Sim2Real User Simulation: Mind the Sim2Real Gap in User Simulation for Agentic Tasks [349] 2026 Verification & recovery Paper
Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters [188] 2026 Foundation models & agentic training Paper
The Evolution of AI Office: From Passive Assistants to Goal-Driven Agents 2026 Spreadsheets, slides & documents Paper
Towards Long-Horizon Agents: A Survey --- Foundation, Evolution, Harness, Optimization, Application, and Frontier [62] 2026 Memory & persistent work Paper · Code
A Comprehensive Survey of Agents for Computer Use: Foundations, Challenges, and Future Directions 2025 Web, desktop & mobile Paper
Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems 2025 Learning & agent harnesses Paper · Code
Agent Evaluation and Benchmarking Survey: Evaluation and Benchmarking of LLM Agents: A Survey 2025 Verification & recovery Paper
Anthropic Economic Index: Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations 2025 Reasoning, evidence & work impact Paper
A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence 2025 Memory & persistent work Paper
A Survey on Latent Reasoning 2025 Reasoning, evidence & work impact Paper
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning 2025 Reasoning, evidence & work impact Paper · Publication
Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforce [346] 2025 Reasoning, evidence & work impact Paper · Paper
Great Models Think Alike and this Undermines AI Oversight [373] 2025 Verification & recovery Paper · Code
Kimi K2: Open Agentic Intelligence [185] 2025 Foundation models & agentic training Paper
Levels of Autonomy for AI Agents 2025 Reasoning, evidence & work impact Paper
Lifelong Learning of LLM Agents: Lifelong Learning of Large Language Model based Agents: A Roadmap 2025 Memory & persistent work Paper
LLM-KG4QA: Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunities 2025 Reasoning, evidence & work impact Paper
Memory in the Age of AI Agents 2025 Memory & persistent work Paper
METR Developer Productivity RCT: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity [389] 2025 Software, data & research Paper
Multimodal Document RAG Survey: Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding 2025 Reasoning, evidence & work impact Paper
OS Agents Survey: OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use 2025 Reasoning, evidence & work impact Paper
Production-Grade Agentic Workflows: A Practical Guide for Designing, Developing, and Deploying Production-Grade Agentic AI Workflows 2025 Reasoning, evidence & work impact Paper
Qwen2.5-VL Technical Report 2025 Reasoning, evidence & work impact Paper · Code
Rethinking Agent Memory: Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topics 2025 Memory & persistent work Paper · Code
Scaling Agent Systems: Towards a Science of Scaling Agent Systems [344] 2025 Learning & agent harnesses Paper
Survey on Agent Evaluation: Survey on Evaluation of LLM-based Agents 2025 Verification & recovery Paper
Table Processing Survey: Large language model for table processing: a survey 2025 Reasoning, evidence & work impact Paper
The Cybernetic Teammate: A Field Experiment on Generative AI Reshaping Teamwork and Expertise [388] 2025 Reasoning, evidence & work impact Paper
Tool Learning Survey: LLM-Based Agents for Tool Learning: A Survey 2025 Reasoning, evidence & work impact Paper
Visually Rich Document Understanding Survey: A Survey on MLLM-based Visually Rich Document Understanding: Methods, Challenges, and Emerging Trends 2025 Reasoning, evidence & work impact Paper
Agent Memory Mechanism Survey: A Survey on the Memory Mechanism of Large Language Model based Agents 2024 Memory & persistent work Paper · Code
CoALA: Cognitive Architectures for Language Agents [14] 2024 Learning & agent harnesses Paper · Code
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models 2024 Reasoning, evidence & work impact Paper
Eureka: Human-Level Reward Design via Coding Large Language Models 2024 Learning & agent harnesses Paper · Publication
GPTs are GPTs: Labor market impact potential of LLMs [348] 2024 Work impact & productivity Paper · Publication
GUI Agents Survey: GUI Agents: A Survey 2024 Reasoning, evidence & work impact Paper
Let's Verify Step by Step [180] 2024 Learning & agent harnesses Paper · Publication
LLM Agent Planning Survey: Understanding the planning of LLM agents: A survey 2024 Reasoning, evidence & work impact Paper
LLM Autonomous Agents Survey: A Survey on Large Language Model based Autonomous Agents 2024 Learning & agent harnesses Paper · Publication
Self-Preference in LLM Evaluators: LLM Evaluators Recognize and Favor Their Own Generations [372] 2024 Verification & recovery Paper
Self-Rewarding LM: Self-Rewarding Language Models [218] 2024 Learning & agent harnesses Paper · Publication
Tool Learning Survey: Tool Learning with Large Language Models: A Survey 2024 Tools & business workflows Paper · Code
APE: Large Language Models Are Human-Level Prompt Engineers [243] 2023 Learning & agent harnesses Paper · Code · Publication
Gemini: A Family of Highly Capable Multimodal Models 2023 Reasoning, evidence & work impact Paper
GPT-4 Technical Report 2023 Reasoning, evidence & work impact Paper
Productivity Effects of Generative AI: Experimental evidence on the productivity effects of generative artificial intelligence [390] 2023 Work impact & productivity Paper
Reward Model Overoptimization: Scaling Laws for Reward Model Overoptimization [383] 2023 Verification & recovery Paper
Self-Instruct: Aligning Language Models with Self-Generated Instructions [196] 2023 Learning & agent harnesses Paper · Code · Publication
Sparks of AGI: Sparks of Artificial General Intelligence: Early experiments with GPT-4 2023 Reasoning, evidence & work impact Paper
The Rise and Potential of LLM Agents: The Rise and Potential of Large Language Model Based Agents: A Survey 2023 Learning & agent harnesses Paper
Emergent Abilities of Large Language Models 2022 Reasoning, evidence & work impact Paper
STaR: Bootstrapping Reasoning With Reasoning [217] 2022 Learning & agent harnesses Paper · Publication
Foundation Models: On the Opportunities and Risks of Foundation Models [1] 2021 Reasoning, evidence & work impact Paper
Productivity J-Curve: The Productivity J-Curve: How Intangibles Complement General Purpose Technologies [391] 2021 Work impact & productivity Paper
GPT-3: Language Models are Few-Shot Learners 2020 Reasoning, evidence & work impact Paper
Scaling Laws for Neural Language Models 2020 Reasoning, evidence & work impact Paper
Gradient Episodic Memory for Continual Learning [386] 2017 Reasoning, evidence & work impact Paper
PPO: Proximal Policy Optimization Algorithms [199] 2017 Reasoning, evidence & work impact Paper
RL from Human Preferences: Deep reinforcement learning from human preferences [178] 2017 Reasoning, evidence & work impact Paper · Publication
Life Beyond Distributed Transactions [378] 2016 Reasoning, evidence & work impact Paper
DAgger: A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning [212] 2011 Learning & agent harnesses Paper · Publication
Provenance in Databases: Why, How, and Where [379] 2009 Reasoning, evidence & work impact Paper
Case Handling: a new paradigm for business process support [376] 2005 Reasoning, evidence & work impact Paper
Ten Challenges for Automation Team Players: Ten Challenges for Making Automation a “Team Player” in Joint Human-Agent Activity [345] 2004 Reasoning, evidence & work impact Paper
Humans and Automation: Use, Misuse, Disuse, Abuse [331] 1997 Reasoning, evidence & work impact Paper
Interdisciplinary Study of Coordination: The interdisciplinary study of coordination [50] 1994 Human-agent collaboration Paper
IT Productivity Paradox: The Productivity Paradox of Information Technology [392] 1993 Reasoning, evidence & work impact Paper
Sagas [380] 1987 Reasoning, evidence & work impact Paper
Ironies of Automation [393] 1983 Reasoning, evidence & work impact Paper

Resources

Official product pages, engineering reports, documentation, evaluation infrastructure, and project resources. These entries are excluded from the research-paper count. 59 entries.

Resource Year Focus Links
Agentic Capabilities in Word, Excel, and PowerPoint 2026 Workspaces & occupational work Source
Claude Duet: Introducing Claude Duet [336] 2026 Personal agents & digital twins Source
Finance Agent v2: Leaderboard and Methodology [290] 2026 Finance & professional research Source · Code
Harbor: A Framework for Evaluating and Improving Agents [321] 2026 Evaluation infrastructure Source · Code
HarnessTax: How Much Does Harness Matter for Coding Agents? [326] 2026 Evaluation infrastructure Source
Harvey Legal Agent Benchmark [291] 2026 Workspaces & occupational work Source · Code · Data
Inspect AI: Inspect: A Framework for Large Language Model Evaluations [322] 2026 Evaluation infrastructure Source · Code
Medical Long Context Reasoning (MLCR-AA) [297] 2026 Reasoning, evidence & work impact Source · Code · Data
Mercor / SkyRL training guide: Training Frontier Knowledge Work Agents: A 397B RL Training Guide with SkyRL [208] 2026 Learning & agent harnesses Source · Code
Microsoft Copilot architecture and how it works [142] 2026 Learning & agent harnesses Source · Architecture
OfficeACE: Huawei Cloud Office Agent Invitation Testing [75] 2026 Spreadsheets, slides & documents Source
Pi: A Minimal, Extensible Coding Agent Harness [323] 2026 Learning & agent harnesses Source
Anthropic Research system: How we built our multi-agent research system [132] 2025 Tools & business workflows Source
Microsoft 365 Copilot Tuning and Multi-Agent Orchestration 2025 Learning & agent harnesses Source
AA-AnalystAgent 2026 Reasoning, evidence & work impact Source
AA-Briefcase v1.1: Agentic Knowledge Work Benchmark 2026 Workspaces & occupational work Source · Code · Data
Adobe Acrobat AI Assistant 2024 Workspaces & occupational work Source
Airtable Omni: AI-Native App Builder and Assistant 2025 Workspaces & occupational work Source
Announcing the Agent2Agent Protocol (A2A) 2025 Learning & agent harnesses Source
BigLaw Bench 2024 Reasoning, evidence & work impact Source
Claude Code Overview [12] 2025 Learning & agent harnesses Source · Code
Claude Cowork: Agentic AI for Knowledge Work 2026 Workspaces & occupational work Source
Codex Manual [11] - Learning & agent harnesses Source · Code
Computer-Using Agent 2025 Workspaces & occupational work Source
Copilot for Power BI Overview 2023 Workspaces & occupational work Source
Copilot in Excel 2023 Workspaces & occupational work Source
Dynamic Workflows: Orchestrate Subagents at Scale 2026 Learning & agent harnesses Source
Equipping Agents for the Real World with Agent Skills 2025 Learning & agent harnesses Source · Code
Excel Modeling Benchmark 2026 Spreadsheets, slides & documents Source
Function Calling 2023 Learning & agent harnesses Source
GLM Slide and Poster Agent Documentation - Workspaces & occupational work Source
Google Workspace Studio: Agents for Everyday Work 2025 Workspaces & occupational work Source
How the Microsoft 365 Copilot Orchestrator Chooses Actions 2023 Learning & agent harnesses Source
Introducing ChatGPT Agent: Bridging Research and Action 2025 Workspaces & occupational work Source
Introducing Claude Design by Anthropic Labs 2026 Workspaces & occupational work Source
Introducing Computer Use 2024 Workspaces & occupational work Source
Introducing the Model Context Protocol 2024 Learning & agent harnesses Source
Loop Engineering: Designing Agentic Loops 2026 Learning & agent harnesses Source
MedCode [295] 2026 Reasoning, evidence & work impact Source
MedScribe [296] 2026 Spreadsheets, slides & documents Source
Microsoft 365 Copilot Tuning Overview 2025 Learning & agent harnesses Source
MiniMax Skills: Office Artifact Skills for Coding Agents 2026 Learning & agent harnesses Source
New Tools for Building Agents 2025 Learning & agent harnesses Source
NotebookLM: AI-Powered Research and Note-Taking 2023 Workspaces & occupational work Source
OpenCode: The Open-Source Coding Agent 2025 Workspaces & occupational work Source
PinchBench 2026 Workspaces & occupational work Source
PptxGenJS Documentation 2016 Learning & agent harnesses Source
Quadratic: The AI Spreadsheet 2023 Workspaces & occupational work Source
Rows AI: Spreadsheet and Data Analyst 2023 Workspaces & occupational work Source
SpreadsheetBench v2: Business Spreadsheet Workflows 2026 Workspaces & occupational work Source
Tax Agent Bench 2026 Reasoning, evidence & work impact Source · Code
The Next Era of Knowledge Work 2026 Workspaces & occupational work Source
The O*NET Content Model [347] 1995 Reasoning, evidence & work impact Source
The Ralph Wiggum Technique 2026 Learning & agent harnesses Source
We are Changing our Developer Productivity Experiment Design 2026 Reasoning, evidence & work impact Source
What 81,000 People Want from AI 2026 Workspaces & occupational work Source
WorkBuddy: A Desktop Agent for Office Work 2026 Workspaces & occupational work Source
Wukong: An AI-Native Agentic Platform for Enterprises 2026 Workspaces & occupational work Source
Zapier Agents: Autonomous AI Teammates 2025 Workspaces & occupational work Source

Citation

If you find this survey or reading list useful, please cite:

@misc{workagents2026,
  author       = {Zishan Xu and Enjun Du and Mingqi Gao and
                  Cheng Yang and Junhao Shen and Zirong Chen and
                  Zhiyuan Yao and Yifu Guo and Yuquan Lu and
                  Jinyang Huang and Zhengxi Lu and Siyi Liu and
                  Ziyu Zheng and Yongqi Zhang and Difan Zou and
                  Weiwen Liu},
  title        = {Work Agents: Systems, Capabilities,
                  and Optimization},
  year         = {2026},
  eprint       = {2610.35845},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  url          = {https://arxiv.org/abs/2610.35845}
}

Contributing

Contributions are welcome! Open an issue or submit a pull request to suggest papers, resources, or corrections.

About

No description, website, or topics provided.

Resources

Stars

62 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors