Every founder I talk to is trying to build their first AI agent.
Almost none of them are going to ship one that works — because almost none of them have built the AI agent knowledge base the agent needs to operate.
Not because the agent frameworks are broken. They’re better than they’ve ever been. Not because the models are too dumb. The models are extraordinary. The reason the projects fail is older than agents and older than AI itself: the team is trying to build a tool to do work that the team has never written down.
You cannot give an agent a job your humans can’t describe. You cannot automate a process that exists only in someone’s head. You cannot deploy a Kate, a customer service rep, a follow-up coordinator, an inbox triager — until you have written down, in some structured way, what the human version of that job actually does, decides, and notices.
The agent is the easy part. The knowledge base is where the real company lives.
The demo-to-deploy gap
Here is the failure pattern, played out in three acts.
Act one. The team watches a demo. The demo is impressive. A vendor’s agent answers a customer email, books an appointment, drafts a marketing post, all with the elegance of a well-rested intern. The team thinks: we can do this in our company. We have agents to build.
Act two. The team picks a real workflow and tries to deploy. The agent works for three turns and then falls off a cliff. It doesn’t know which prices are negotiable. It doesn’t know which clients have a long-running issue with shipping. It doesn’t know that calls from area code 781 are usually a partner, not a prospect. It doesn’t know that Tuesday is the worst day to schedule a follow-up because everyone is in the operations review. It doesn’t know any of the unwritten things that the human doing the job knows in their bones.
Act three. The team blames the agent framework. They try a different framework. Same result. They try a bigger model. Same result. They eventually conclude that AI agents aren’t ready for their business, when in fact the agent has been ready for two years. What isn’t ready is the knowledge.
I have watched this play out at least a hundred times. It is so reliable I now use it as a test of how much real engineering a team has done. If the team can describe the agent they want, in detail, but can’t describe how the human currently does the job, in equal detail, the project will fail. Always.
What a knowledge base actually is
Most teams I meet think they have a knowledge base. They don’t.
A knowledge base is not a Notion workspace. It’s not a Google Drive folder. It’s not a SharePoint. Those are document graveyards. They are where information goes to be filed, not where it goes to be operated on.
A real knowledge base for an AI agent has four properties.
It is decision-grained, not document-grained. The unit is “what does the team do when X happens” — not “we wrote a doc about X once.” A document is too coarse. It says many things, some current, some stale, some contradictory. A decision is atomic. It has a trigger, a context, a choice, and a rationale. An agent can act on a decision. It cannot act on a 14-page internal wiki.
It is updated where the work happens. If the only way the KB gets updated is “someone remembers to log into the wiki on Friday,” it dies. The successful pattern is to put the capture inside the workflow. When a customer service rep handles a case in a way that’s worth teaching, the system asks them — in the moment — what they did and why, and stores the answer as a reusable trace. The capture is a byproduct of doing the work, not a separate task that gets bumped.
It is structured enough for retrieval, loose enough for reality. Pure free text is too hard for an agent to reason over. Pure schema is too rigid for the messy reality of how decisions actually get made. The right shape is what we often call “structured prose”: short, named fields (trigger, context, choice, why) holding short paragraphs of natural language. Agents can search it, humans can write it, and neither side has to fight the format.
It contains the why, not just the what. A KB that says “we offer 10% discounts to repeat customers” is half-built. A KB that says “we offer 10% discounts to repeat customers because retention is more profitable than acquisition above the second order, but we never offer them in the first call because it trains the customer to wait” is the kind of thing an agent can actually reason about. The “why” is what lets the agent generalize. Without it, the agent is a parrot.
Why this is mostly a humans problem
Here is the part that founders don’t want to hear.
Building the knowledge base is not a model problem. It is not a retrieval problem. It is not even, at root, a tooling problem. It is a humans problem. Specifically, it is a problem of getting the people who actually know how the company works to write down what they know in a form the agent can use. And those people don’t want to.
They don’t want to because writing it down is annoying. They don’t want to because the way they actually do the job is messier than the way they’d describe it on demand. They don’t want to because they suspect — correctly — that documenting their work is a step toward automating it, and they are not sure how they feel about that. They don’t want to because they have been asked to update wikis a hundred times before and it never went anywhere.
Most companies try to solve this by hiring a “documentation lead” or a “knowledge ops” person. That person produces a beautifully structured wiki that nobody reads and nobody updates. The KB looks complete. It isn’t. It’s a museum.
The pattern that actually works is to embed the capture into the work itself. When a senior team member resolves a hard case, the system — agent or coworker — asks them three questions: what was the trigger, what did you do, why did you do that and not the alternative. The answer takes them ninety seconds to give. The system files it as a structured trace. The next time a similar case comes up, the agent can lean on that trace. The senior team member gets to see — in real numbers — that their writing-it-down is paying off, because the agent is now handling the easy version of the case they used to handle.
That last part is the only thing that makes the loop sustainable. Knowledge capture has to feed back into the workload of the person doing the capturing. Otherwise it dies.
The order of operations
If I were starting an AI implementation today, knowing what I know now, I would do it in this order.
Step 1: pick one workflow. Not the whole company. One workflow. The workflow should have three properties: high-volume (so improvements compound), high-judgment (so it actually needs reasoning, not just templating), and owned by one or two senior people who are willing to be involved.
Step 2: shadow the humans for two weeks. Not interview them. Shadow them. Watch what they actually do. Ask “why” every time you see a decision. Take notes — the kind that capture trigger / context / choice / rationale, not the kind that capture summary. By the end of two weeks you should have somewhere between forty and a hundred decision traces. That is your seed KB.
Step 3: build the simplest possible agent on top. Not a multi-agent orchestration. Not a fancy reasoning loop. A single agent that retrieves from the seed KB and produces a draft response or a draft action. Have a human review every output for the first month. Treat the agent as an apprentice, not as a deployment.
Step 4: capture the corrections. Every time the human edits the agent’s draft, the system stores both the original and the correction, structured. That is your richest training signal. Most teams throw this away. Don’t.
Step 5: only after this is humming, expand. A second workflow. Then a third. The second one will go faster than the first because you’ll have the muscle. The third will go faster than the second because you’ll have the tooling. By the fifth or sixth, you’ll be the company in your industry that has an actual operating system for AI work, and your competitors will still be on act two of the demo-to-deploy gap.
The real moat
The thing that almost nobody is saying about agentic AI is that the moat is not the agent. The moat is the captured reasoning of your best people, in a form your agents can use, accumulated over time.
Companies that figure this out early will compound. Every senior decision becomes reusable. Every escalation becomes training data. Every correction tightens the loop. Five years in, the company has an institutional memory the size of which their competitors literally cannot match, because the competitors didn’t start the capture five years ago.
Companies that skip this step will look fast for a while, ship some agent demos, claim some wins, and discover three years in that their agents are still hallucinating their way through cases that their senior people would have handled blindfolded — because the senior people’s knowledge never got into the system.
Build the knowledge base. The agent is the easy part. The knowledge is where the company actually is.
Ahmed Reza is the founder of Yobi (yobi.com), an agentic AI platform that runs as the AI employee for hundreds of small businesses. He spent his earlier career as a software engineer at NASA and JetBlue before founding companies in the dental and small-business AI space. He writes about building practical AI here. If this resonated, reply directly.