Anatomy of a workflow audit: deciding what an AI agent should and should not do

In August, trade-press coverage of an MIT Project NANDA report, "The GenAI Divide: State of AI in Business 2025," put a number on a frustration many executives already felt: heavy spending on generative AI and little measurable return. The report is preliminary and built on a modest sample, so the headline "95%" should be read with care, but its diagnosis is worth acting on. This post describes the step that addresses it: a workflow audit before any agent is built.

What the data says

According to the report, despite $30 to 40 billion in enterprise generative AI investment, 95% of organizations are getting no return, and just 5% of integrated AI pilots are extracting millions in value ^[1]. For task-specific tools, it charts 60% of organizations investigating, 20% piloting and 5% successfully implementing ^[1]. The report's own limitations section says its percentages reflect an interview sample of 52 organizations and may not represent broader market patterns ^[1]; the study combined a review of more than 300 public initiatives, those 52 interviews and 153 survey responses from senior leaders ^[1]. Treat the figures as indicative, not as a measured base rate for any given company.

The explanation that matters for design: the report said the core barrier is not infrastructure, regulation or talent but learning, and that most generative AI systems "do not retain feedback, adapt to context, or improve over time" ^[1]. It also found that workers from over 90% of surveyed companies reported regular use of personal AI tools, while only 40% of companies said they had purchased an official LLM subscription ^[1].

What a workflow audit is

A workflow audit is a structured walk through how a process actually runs today, step by step, before deciding what to automate. Our process for AI agents and automation work covers this kind of analysis alongside building task automation, decision logic, tool and API integration, and human-in-the-loop handoffs. The audit's job is to decide which of those pieces each step needs.

1. Map the real process

Write down each step as people actually perform it, including exceptions, handoffs and the systems touched. Record volume, time per step, and how errors surface.

2. Classify each step

Anthropic distinguishes workflows, where "LLMs and tools are orchestrated through predefined code paths," from agents, where LLMs "dynamically direct their own processes and tool usage" ^[2]. Its guidance: find "the simplest solution possible, and only increasing complexity when needed," because agentic systems often trade latency and cost for performance, and many tasks only need a single well-optimized model call with retrieval ^[2].

The last two rows follow Anthropic's advice: agents suit open-ended problems where steps cannot be hardcoded, but autonomy raises cost and the risk of compounding errors, so extensive testing in sandboxed environments with guardrails is advised ^[2].

3. Define ground truth and checkpoints

Anthropic notes agents should gather "ground truth" from the environment at each step, such as tool results or code execution, and can pause for human feedback at checkpoints or blockers ^[2]. In the audit, specify for each automated step what counts as correct and who reviews exceptions. Coding agents are an example of a measurable task because tests can check results; customer support is another where success can be defined as resolution ^[2].

4. Decide the feedback loop

The "learning gap" finding makes this step non-optional ^[1]. Decide how corrections reach the system: updated instructions, new retrieved reference material, or reviewed examples. A pilot with no path for feedback will plateau.

5. Choose the first slice

Pick one process with enough volume to matter and an unambiguous success measure. The report says top-performing mid-market firms reported about 90 days from pilot to full implementation, versus nine months or longer for large enterprises, which argues for a narrow first scope ^[1].

Where usage is heading

Anthropic's September 15 Economic Index found 77% of enterprise API transcripts show automation patterns, mostly full task delegation, and 97% of API tasks are automation-dominant ^[3]. Delegation is the dominant mode in production use of that vendor's API, which raises the stakes of getting the checkpoints in step 3 right ^[3]. This is one vendor's customer base, not the market.

Practical implication

Do the audit before choosing a platform: the output is a short list of steps with a named automation approach, a correctness check and a human owner for each. If you want help structuring one, talk to us: sales@sinthetix.com.