Before expanding an AI prototype, connect one workflow to real business data

Mocked data is forgiving; your ERP is not. A prototype on invented records proves an idea can be built, not that your business can run on it. Before you fund wider AI automation, require one workflow to run end to end against your real records, permissions and exceptions. Expand only after the process owner accepts its results against an outcome agreed in advance.

What one engineering team reported

On 26 August 2026, Atlassian published an account of a team of 5 taking a weekend prototype to production. About 15 hours, ~250 agent sessions and 100+ review rejections produced “a running application with a real database,” built on “acceptable mocking for real integrations, authentication, and persona derivation” (Atlassian).

The trouble started when the mocks were replaced. “The real schema differed from the mock,” the team wrote, and the data could not simply be transformed because the prototype “had baked in incorrect assumptions and a lot of shortcuts.” Every pull request also changed the end-to-end tests meant to lock in existing behaviour, “which defeated the whole point of having them.” The team's conclusion: “The bottleneck is human context” (Atlassian).

It recovered by making the problem smaller: one feature it could ship in a couple of weeks, with edge-case questions settled in planning, “with no AI crutch,” before any agent touched it (Atlassian). The team reports reaching production in about 5 months at what it describes as roughly 5x the output per engineer of a comparable traditional build (Atlassian).

What this shows, and what it does not

This is one team building software with AI, not an agent inside an ERP, and not a failure rate. The lesson transfers because the break happens at the same point: when a model of the business meets the actual business. TechTarget reported in February 2026 that some AI projects “falter when exposed to real-world data.” In one drive-thru example, a customer ordered 18,000 cups of water, “which the AI system dutifully entered” (TechTarget).

Manish Sood, CEO of SAP-owned Reltio, said in an SAP News feature published on 23 September 2026: “An AI demo can tolerate a lot of imperfections and gaps in the data. An AI agent operating inside a real business context cannot” (SAP News). The article cites Harvard Business Review Analytic Services research: 94% of organisations surveyed are exploring or implementing agentic AI, while 15% believe their data foundation is truly ready (SAP News). We have not seen that survey's sample or method.

The expansion gate we recommend

These five checks are our recommendation, drawn from the accounts above. The first workflow passes all of them before a second gets budget.

1. Real records. The workflow reads and writes the actual ERP tables, customer records and item codes it will use in production. An export or sample schema does not count.
2. Real permissions. It runs with the access a real user in that role has. If the prototype needed administrator rights, it has not worked yet.
3. Named exceptions. The team has listed the cases the workflow must refuse or pass to a person, and tested each one.
4. An accepted outcome. Before the run, the process owner agreed what a correct result looks like, such as a matched invoice or a confirmed order line, then approved a representative batch.
5. A stated human step. Anything that moves money or stock, or commits you to a customer, waits for a named person to approve it. As Sood puts it: “Define where an agent can act, where a human needs to intervene, how decisions are traced” (SAP News).

Measure rework, not output

Atlassian's prototype ran after about 15 hours; most of the roughly 5 months came afterwards (Atlassian). Track two numbers instead of code volume:

• Integration rework: the changes needed to fit the workflow to your real data and access rules.
• Accepted outcomes: the share of results the process owner approves without correction.

If rework does not fall from run to run, the prototype's picture of your business is probably wrong; fix that before adding agents.

The strongest objection

A gate slows a team with momentum, and cheap prototypes expose requirements early. But the gate does not stop anyone building prototypes. It stops the company funding several of them on assumptions that only one real integration can test.

How we approach it

We describe one workflow's actual records, permissions and exceptions in a shared model of the business, and connect the agent to that model, not to a mock. Actions that move money or reputation stay under human approval. Expansion rests on evidence from that first run.

Your first week

• Days 1–2: Pick one stalled prototype and the ERP step it is meant to feed. Write down the outcome the process owner will accept.
• Day 3: List the real tables and user roles the workflow needs.
• Days 4–5: Collect last month's exceptions from the people who handle them; decide which must go to a person.

This quarter, approve one end-to-end integration, not a rollout. If it passes the five checks, with rework falling and accepted outcomes rising, fund the next workflow. If it fails, you have found the problem at the cost of one workflow instead of many.