Intelligence Debt.

In the late 1800s, factories ran on steam. A single engine in the basement spun a central drive shaft that snaked through the building. Pulleys, belts, and gears delivered power to every machine, all tethered to that one humming heart.

When electric motors arrived, factory owners swapped the steam engine for an electric one and kept the same shaft-and-belt setup. The results were disappointing. The technology improved, but the results stayed the same. Nobody was rethinking how factories operated — they simply upgraded the engine and left everything else unchanged.

That continued for over twenty years. Things shifted in the 1920s with the "unit drive," when each machine got its own motor. Factories no longer had to organize around a central shaft. Machines could be placed based on how materials moved, turned on only when needed, the entire floor plan redesigned. The assembly line emerged from those changes. Henry Ford said he couldn't have built it without electrification — but in the 1880s, no one making electric motors imagined moving assembly lines.

The opportunity wasn't in doing the same work faster but in working backward from the job to be done, the desired outcome, to redesign the entire operation around capabilities that were previously impossible.

We're at a similar inflection point with AI. But the world is more malleable than ever. Projects that used to need fifty engineers and months can be finished in weeks. We can just do things now that were too expensive to even consider before.

When you can build almost anything, the hardest part is deciding what to build. Most companies take the familiar path of making what they already do faster and cheaper. Just swapping out the motor. The factories that thrived didn't predict the assembly line. They solved practical problems — put this machine closer to that material, rearrange the floor so parts flow better — and the assembly line emerged. They built capabilities, and the capabilities created options that didn't exist before.

In the 1960s, Robert McNamara brought the statistical methods that had made him successful at Ford Motor Company to his role as Secretary of Defense. He applied them to the Vietnam War with a rigour that impressed nearly everyone in the room. Body counts became the primary metric of progress. If the numbers said America was winning, America was winning.

The sociologist Daniel Yankelovich described what went wrong as a four-step descent. First, you measure whatever can be easily measured. This is reasonable. Second, you disregard what cannot be easily measured. Third, you presume that what cannot be measured is not important. This is dangerous. Fourth, you presume that what cannot be measured does not exist. This, Yankelovich wrote, is suicide.

Unfortunately the McNamara Fallacy seems to be alive and well in how many companies are approaching AI implementations.

You can measure time saved per task, and how many processes were automated, how many roles eliminated. These numbers show up in quarterly reviews, vendor case studies, and ROI reports. But the judgment your best people carry — your product team's instinct for what customers actually want, the workarounds and exceptions and philosophy that make your company yours — none of that appears on a dashboard.

If you follow the McNamara logic too far, you end up cutting out the very things that make your company unique, or worse, you automate them out of existence.

Rory Sutherland coined the Doorman Fallacy. Consultants come in, see the doorman's salary, run the ROI numbers, and recommend automated doors. You fire the doorman, see an immediate margin gain on the P&L, and pay the consultants. But what's missing from the P&L or ROI analysis is the lost customer touchpoint, the human presence at your business's entrance, and the warmth every guest feels when greeted by the doorman.

This is happening everywhere with AI adoption. Every vendor promises speed. Every case study measures percentage improvements. In those meetings, nobody is asking: what are we giving up?

Klarna answered that question the hard way. In 2023, the Swedish fintech replaced roughly 700 customer service workers with an AI agent built on OpenAI's models. For a while, the metrics looked excellent. The bot handled two-thirds of all customer conversations, resolution times improved, labor costs dropped. Then the complaints started. Customers reported robotic responses, inflexible scripts, the maddening loop of having to repeat their problem to a human after the bot failed. By early 2025, satisfaction scores had deteriorated, and CEO Sebastian Siemiatkowski admitted publicly that cost had been "a too predominant evaluation factor." The company began rehiring.

What Klarna automated away was the institutional knowledge its agents carried — the ability to read when a routine query is actually a deeper problem, to handle edge cases, to exercise judgment in the moment. That knowledge vanished when the team did, and it couldn't be quickly rebuilt. The savings showed up immediately on the P&L. The damage showed up later, in churn, in brand erosion, in the cost of unwinding a strategy that had already been celebrated.

Just as developers speak of technical debt — the cost of moving fast today by writing messy code that must be fixed later — companies are now accruing Intelligence Debt.

Intelligence Debt is what happens when you automate a process you no longer fully understand. You swap the doorman for a sensor, the support team for an LLM wrapper, and you gain an immediate speed boost. But you've also severed the feedback loop. You've traded the reason for the result. The interest on this debt doesn't show up on a dashboard. It shows up in the maddening loops Klarna's customers experienced. It shows up when a market shift happens and your automated systems keep sprinting in the wrong direction because nobody remembers the why behind the what. Eventually, the interest gets so high — in the form of brand erosion, lost taste, and rigid systems — that the only way to pay it back is to start over from scratch.

You can't automate what you don't understand without eventually becoming a commodity. To avoid the debt, you do the thinking up front.

Many companies that succeed with AI will still face a quieter risk: they may lose the very essence of what makes them valuable today. We've seen this pattern before at industrial scale. For decades, Western firms treated manufacturing as a pure cost center to be minimized. Outsourcing to Asia made perfect economic sense — until it didn't. Companies (and in some cases entire countries) quietly lost the ability to make things. The fragility only became obvious during COVID, but the hollowing out had been underway for years.

Apple offers the clearest cautionary tale. Its aggressive outsourcing of manufacturing to China didn't just slash costs; it systematically exported the company's deepest expertise. Thousands of Apple's top manufacturing engineers embedded themselves in Chinese factories, the company poured hundreds of billions into tooling and training, and it turned suppliers into the world's most sophisticated electronics producers.

The short-term payoff was spectacular: Apple became the most valuable company on earth. But that decision also created a deeper structural vulnerability. By handing over the "how" of production, Cupertino surrendered control over a core capability. Today Apple is far more exposed to geopolitical shifts, tariff changes, supply disruptions, and sudden policy moves than it would have been had it kept more of that knowledge and manufacturing muscle in-house.

The parallel to AI is direct: when you outsource your thinking to a model you don't own, you aren't just being efficient, you're building your future on a rented foundation. Think about what we trade off when we outsource jobs at individual level or collective. What does it mean for an employee to not internalize the memo that was written by Claude? Or the customer comms that humans don't experience? Dashboards and ROI reports will only ever show you the gains. They can't show you what you're giving up.

Your company is the knowledge your people have built up over years. Your AI strategy should treat these tools as a way to accentuate, retain, and surface that hard-won knowledge. Those accumulated insights are the primitives that become the foundation for any new capabilities you build.

Many of the companies I talk to still start with the same questions: what can we automate? Why am I not seeing productivity improvements since turning on Co-pilot? Our teams are using Claude Code but we're not shipping product any faster. Why? Where can we cut headcount?

I think we should start somewhere more fundamental.

What John Kay calls "distinctive capabilities" — the institutional knowledge, the domain judgment that walks out the door every evening and comes back the next morning — is what differentiates a company. For many companies, this is both their greatest asset and their greatest vulnerability.

First, identify the atomic primitives that power your business. These are the core, reusable capabilities required to deliver on a specific Job To Be Done. They are the fundamental building blocks of your competitive advantage. For a sales team, these primitives might be pricing, negotiation, and customer insight. For a marketing team, they might be creative judgment, audience empathy, and distribution. These are not just tasks; they are the distinct capabilities that make your company useful.

Once those primitives are clear, the real leverage comes from building an intelligence layer on top of them. This layer captures the judgment, edge cases, and philosophy your best people have developed over years. It encodes the "why" behind the "what." The intelligence layer doesn't just automate a task; it orchestrates your primitives to solve problems, often in real-time.

Take the pricing primitive. Instead of just giving a team faster spreadsheets, you build an intelligence layer that holds your best analyst's reasoning. Now the system can flag when a deal structure violates your core philosophy, suggest custom models for complex scenarios, and help a junior analyst perform at the level of a veteran. You aren't outsourcing the work to a generic model; you are scaling your own distinctive capabilities.

This is how you avoid Intelligence Debt. You're not outsourcing judgment to a generic model. You're using AI to amplify and scale the hard-won knowledge that already exists inside your company. The primitives are yours. The intelligence layer is yours. And that combination is very difficult for competitors to replicate.

A CPG marketing team came to us overwhelmed by tools (100+ tools!) and agencies (30+ agencies!) churning out generic assets. Instead of automating copy to cut costs, we rebuilt their tooling around the real primitives of their work: strategy, creative judgment, audience insight, and production capabilities. The quality of output improved, speed and volume increased, and most importantly, the team became positioned to create work that it hadn't been able to before — localized campaigns, limited-edition packaging runs, in-store activations, and brand partnerships.

There is a difference between shallow automation and fundamental restructuring, and I believe the delta between companies that invest in core capabilities now, versus those that may choose to buy off-the-shelf AI features or AI tools, will create winners and losers over the next decade. This is compounded by the nature of automation — you lose the people whose knowledge is what actually makes your business valuable. Especially in the arena of knowledge work where most companies are horizontally structured, their value chain is rented or bought. A useful question to ask, repeatedly, is "what makes my company valuable?" and then ask why until you get to the core driver of market strength.

The point is, that you can automate tasks and lose the institutional knowledge that made you distinctive. Or you can pour AI onto people as a generic augmenter and still remain trapped in the old org chart. Neither approach gives you resilience when markets shift.

The smarter path is to invest first in the atomic primitives that let your company perform a real job to be done for customers — the capabilities that make you strong, useful, and hard to replace. Then you build an intelligence layer on top of those primitives. The layer encodes the judgment, edge cases, and philosophy your best people have developed over years. It turns tacit knowledge into something composable and proactive.

Think back to that pricing team. Each sprint builds on the one before. Every edge case gets added in. Every interaction teaches the system something new about how the company thinks. Just as a new employee would learn by first-hand experience, so too the systems can learn from their first-hand experience, and then codify and share that knowledge in ways that humans cannot. After six months the system holds judgment the team didn't even have at the start. Crucially, this isn't a static piece of software; it is a collaborative loop where every human correction or edge case is fed back into the layer, ensuring the system evolves alongside the market.

There's a phrase I think about a lot: "the small steps of giant leaps." Where you are today comes from many small choices made over years. The daily decisions that build a strong future usually go unnoticed. There are often slow feedback cycles in business, and counter-intuitively businesses are often punished over the short term for doing what's right over the long term. There is seldom a short-term reward for doing the right thing.

So I don't think we can expect to see dramatic results in one quarter, but the value will accrue in the accumulated knowledge and the organizational muscle that is developing with the ability to keep creating new capabilities. A company that has done this four times is fundamentally different from one doing it for the first time. That muscle is the advantage, and no amount of tool-buying can replace it.

Packy McCormick recently wrote about what he calls "costless sacrifice," drawing on the Old Testament story of King David refusing to offer a burnt offering that cost him nothing. When the cost of producing something drops to near zero, the sacrifice disappears. And costless sacrifice is not sacrifice. It is hollow.

I see this everywhere. When you let technology think for you and skip the hard work of understanding your strengths, you end up automating whatever is in reach. The result is output with no judgment, volume without taste — the appearance of intelligence, but none of the substance.

This is how you pay down Intelligence Debt: you pay the cost of thinking up front.

Building capabilities used to take millions of dollars and years. Now it can be done in weeks. The thinking is the expensive part. Avoiding the allure of using AI to do the thing when the thing is important. It takes effort to understand your strengths. It means tough conversations about what actually makes you unique. It requires patience — letting something grow before you can prove its value on a dashboard. Taking leaps to build out a capability without a clean ROI, because you can't evaluate all the future possibilities that capability will enable.

For example, if you automate customer service right away, tickets get answered faster. But who really knows what your customers need? Who understands the patterns, frustrations, and unspoken expectations? Someone has to do that thinking before automation is worth anything. And the work never stops: you have to keep refining how you encode judgment in your systems, and keep going even when feedback is slow.

The companies that ultimately win with AI won't be the ones that move the fastest or adopt the most generic tools. They will be the teams that do the deeper, slower work of defining their core competencies and building atomic primitives around them. By encoding their own judgment into a proprietary intelligence layer, they build the organizational muscle required to be truly adaptable. In a world where every competitor can buy the same off-the-shelf speed, the only way to maintain a superior position is to own the "why" behind your operations. This work creates a compounding advantage and the infinite optionality needed to pivot, scale, and thrive, no matter how the market moves.

A few of the principles I think about often with transformation: No efficiency without capability. No automation without understanding. No compounding without consistency. No intelligence without context.

And most of all: no shortcuts to building something that lasts.