Most large organisations have had an AI transformation consultant in the building. Most have nothing to show for it.
The advice wasn't wrong. It just skipped the part that matters. Picking OpenAI over Anthropic is an afternoon's decision, and it's usually the decision that gets the most airtime.
What separates the organisations that actually ship AI from the ones still running pilots comes down to ten things, and none of them show up in a strategy deck.
A data platform your agents can actually query
An agent is only as good as the data underneath it. Spread across twelve systems with no connective tissue, AI won't fix that. It'll surface the inconsistency faster and sound more confident doing it.
Forget "single source of truth" if you're picturing one database. That's not how a real enterprise works. What you need is one governed set of data products agents can read, with pipelines that run on a schedule you can state out loud, lineage back to the source system, and data quality checks written as code. The day an agent can reach data nobody's governing, you've built a fast way to generate confident, wrong answers.
A semantic layer so a metric means one thing
Ask five people what "active customer" means and you'll get five answers. Ask an AI agent and you'll get a sixth, generated on the spot, delivered with total confidence.
A semantic layer fixes each definition once, versions it, and gives it a named owner. Building it is the easy bit. The real fight is getting finance, commercial and operations to agree on one definition when their bonuses have depended on three different ones. Budget for that fight. Get it right and the agent becomes auditable: someone challenges a number, you point to the definition and the query.
A context layer, and a way to keep it alive
Definitions tell an agent what a metric is. Context tells it what's actually going on: which promotion ran last month, why a supplier changed terms, what a segment gets called by the people who run it. None of that lives in your warehouse. It lives in people's heads and a wiki nobody's opened since Christmas. Without it an agent gives you technically correct answers that anyone who knows the business can spot as wrong in four seconds.
Most organisations treat this as a documentation project: stand up a wiki, run a content sprint, declare victory, watch it go stale in a quarter. Context needs a home built into the agent, a way to update it no harder than leaving a comment, and someone's name against it. Treat it like code. Version it. Own it. Keep fixing it.
Governance across data, models and tools
In most enterprises the governance already exists, several times over. An AI governance forum, an architecture council, privacy impact assessments, cybersecurity review, legal review, procurement. Each is there for a good reason and each has a legitimate claim on the decision. What's missing is a single agreed path through them: who goes first, what evidence each one needs, what they can reopen once another group has signed off. Without that, projects bounce between councils for months answering slightly different versions of the same question, and nobody is actually saying no.
Agree the process once, with all six parties in the room, before you need it. Loose governance doesn't buy you speed. Neither do six rigorous processes that have never been introduced to each other.
Deployment and sovereignty
People conflate governance and sovereignty. Governance is who approves the model. Sovereignty is where the computing actually happens, and whether your data leaves the building to get an answer.
For health, research, financial services, anyone holding data they can't afford to leak, the deciding factor is usually egress rather than model quality. A public API sends your data out of the building. Your own cloud tenancy keeps it inside your boundary. On-premise or air-gapped and it never leaves. Pick one before you design anything else, because retrofitting a public-API build into an air-gapped environment is basically starting over.
Evaluation: proving the agent is right
This is the step everyone skips, and the one that decides whether anyone trusts the system. Approval happens once. The model, the data and the prompts keep moving underneath it, and most organisations never check whether the answers are still right six months later.
Build a golden set: a hundred or so real business questions with verified answers, the kind people actually ask. Run it automatically every time a prompt or model changes. Sample live answers for drift. Give users a real way to flag a wrong one. Skip this and you can't defend an accuracy number, and you can't tell whether the agent is improving or degrading. You're just hoping.
Token economics is an engineering problem
Cost is where most AI programmes blow up their own second-year business case. The pilot ran for a few thousand a month and nobody blinked. Roll it out to two thousand people and the bill climbs faster than the value.
The fixes are mostly mundane engineering. Retrieve the right hundred rows instead of dumping ten thousand into the context window. Route the boring tasks, classification, extraction, routine summarising, to a cheap model and save your best for actual reasoning. Cache what doesn't change: system prompts, schema definitions, semantic layer context. Filter and aggregate in the database before the model sees the data. At real volume, dedicated GPU infrastructure can beat per-token pricing.
People and ownership
"We'll hire some AI engineers" is not a capability plan. They don't turn up and dissolve your data quality problems or make sense of business logic nobody wrote down. The teams that pull this off are more balanced than people expect:
- Data engineers, because the platform is usually the real constraint
- Solution and data architects to decide how the pieces fit, where the boundaries sit, and what the agent can reach
- Analytics engineers who own the semantic layer and can hold their ground against finance
- Software engineers to build the agents and context layer, integrate them with the systems you already run, and set up the security model. This is production software: testing, version control, error handling, someone on call when it breaks.
- AI and ML engineers for agent design, orchestration and evaluation
- Domain experts embedded in the build from day one
Software engineering gets underestimated most. A demo and a system a thousand people rely on daily are different problems, and the integration and security work usually dwarfs the agent logic.
Then give the outcome a named owner in the business who sets the success metric and answers for it at their review. Watch for an owner in name only with no real stake, and for joint ownership between business and technology, which sounds collaborative and means nobody's accountable. One name.
Adoption and trust
Building the agent is the easy part. Getting people to open it, ask it something real, and believe what comes back takes a lot longer, and most programmes never budget the time.
Trust gets earned. Show the working behind every answer: what definition it used, what data it queried, what period it covered. Let the agent say "I don't know" instead of guessing with a straight face, because one confident wrong answer costs you a user for good. Make sure a flagged mistake visibly changes something, or people stop flagging. Start where being wrong is embarrassing rather than expensive.
Roll it out small. Get one group to real daily use and let them tell the next group. A mandate and a training deck won't get you there. Weekly active users and repeat usage tell you the truth.
Redesigning the workflow around the agent
Almost everyone bolts the agent onto the process they already have. Step three gets faster. Steps one, two, four and five stay exactly as they were, and so does the cost base. Then the business case comes up for review and nobody can find the money.
We did this to ourselves. When we first rolled AI tools across our delivery teams we measured roughly a 60% productivity improvement. Good result. But we'd handed people better tools inside the same workflow. When we redesigned the workflow around what the agents could actually do, who does what, in what order, which review steps only existed because a human used to do the work, the number went to about 5x. Same tools, same people, different process.
That gap is an operating model problem, and closing it needs someone with authority to redraw the process rather than just approve the software. Go looking for the review steps that exist because the work used to be slow, the handoffs that exist because two teams couldn't see the same data, the reports somebody asked for in 2019. Most can go. Until they do, you're paying for an agent and running the old business.
Where to start
There's no universal order to these, and anyone who tells you there is hasn't seen enough organisations. Where you start depends on what you already have.
So skip the sequencing debate and do an audit. Go through the ten and mark which you actually have, which you have on paper only, and which nobody owns. Most organisations find they're missing four or five, and that two they thought were done are a wiki page and a good intention. That list is your plan.
I'm Lawrence, founder of Xephyr. We build sovereign AI agents and the data platforms that power them. If you're working through any of the above, book a call.