Waldek Mastykarz

I was wrong: there are bad prompts

Mon, 10 Aug 2026 19:47:00 GMT

For the longest time, I believed there was no such thing as a bad prompt. When you ask AI to do something and get an unexpected result, that result gives you feedback. You may have left an important constraint implicit or used a word such as simple that you and the model interpreted differently. Change the prompt and try again. Learn from what happened.

Working with AI forces us to make explicit countless assumptions about what good looks like. We normally carry those assumptions in our heads, where other people can infer them from context or ask questions. AI exposes them by following what we said rather than what we meant. Calling the prompt bad always seemed too easy because the failure revealed what the instruction left unclear.

I still believe that a bad result can teach you something useful about your instruction. When we call a prompt bad, we usually mean that it produces an undesired result. That definition however overlooks prompts that produce the desired result only some of the time. Those prompts are even worse because a successful run can make them look dependable.

Getting the right result once proves very little

Imagine that you write a prompt and get exactly what you wanted. You run it again, and the result is even better. Would you ship it? Run it a few more times first, because the next result may be mediocre while another completely misses the point.

What is the quality of that prompt? You have established that it can produce an excellent result. Yet, you have no evidence yet that it works reliably. If you need the instruction to work tomorrow and thousands of times after that, one excellent run tells you remarkably little.

Now compare it with another prompt that you also run five times. Every result is good, although none is as impressive as the best result from the first prompt. Which one would you choose for a product? I'd choose the second one because I can depend on it.

Judge the prompt across runs

We usually judge prompts along one dimension: did I get the result I wanted? An undesired result leads us to call the prompt bad, inspect why it failed and improve the instruction. A desired result gives us no reason to look further, which allows unreliable prompts to pass as good ones.

Prompt quality has another dimension: will I get the result I want again? That question matters because an occasional success can hide an underspecified instruction. You run it once and like what you see, so you conclude that you wrote a good prompt. All you have proved is that one possible execution happened to satisfy your intent.

A consistently wrong result is at least understandable. The model interprets your instruction in a particular way, so you can compare its interpretation with your intent and clarify the instruction. Occasional success is more dangerous because it gives you false confidence. To learn whether the prompt works, you need to run it again, and again.

Reliability is part of prompt quality

A good prompt reliably produces the desired outcome. Reliability belongs in the definition because prompts are instructions that we expect to reuse. An instruction that works only when the model makes several fortunate choices leaves too much of the outcome to chance.

This changes how you improve prompts. Imagine that five runs produce three good results, one mediocre result and one unusable result. After you clarify the instruction, all five results are good. The best new result may be less impressive than the best one you got before, yet the prompt has improved substantially because you can now depend on it.

For agents, that difference matters enormously. An agent that occasionally finds an ingenious solution but frequently gets lost may be less useful than one that consistently produces an adequate solution. Capability matters, and dependability determines how safely you can use that capability without watching every step.

Variation can still produce dependable outcomes

Language models are nondeterministic, and agentic tasks often have many legitimate solutions. If you ask an agent to build a website five times, you probably want five different websites. Reliable outputs can differ while each one still satisfies your intent.

Suppose repeated runs produce wildly different outcomes. You clarify the goal and spell out the constraints. You also define what success looks like. The solutions still differ, but now you would happily use each one. You preserved the model's freedom to choose a path while ruling out paths that lead somewhere you never wanted to go.

Some unpredictability comes from the model, while some comes from decisions we leave unspecified. The model fills those gaps differently on each run, which makes an underspecified prompt look creative when it is actually unreliable. Repeated execution exposes those hidden decisions and shows you where the instruction needs to be clearer.

Test the prompt, not the luckiest output

We spend a lot of time inspecting individual AI outputs. We write a prompt, run it, read the result, then adjust the instruction. That loop is useful while developing a prompt, but eventually we need to evaluate the kind of results the instruction produces across repeated runs.

You can start without sophisticated statistics. Run an important instruction several times and ask whether every result is acceptable. Look for failure patterns and clarify the choices that caused them. Then run the instruction again. The goal is dependable quality with room for legitimate variation.

A bad result may come from a good instruction, while a great result may come from luck. One great result shows that your prompt can work. Reliably getting good results shows that it does work. Turns out there are bad prompts after all.

Your agents should learn for the organization

Mon, 10 Aug 2026 11:00:00 GMT

Your agents learn while doing the work. They discover where the organization's guidance no longer matches reality. But when the conversation ends, most of that learning disappears with it. Agent memory may preserve some of it for you. What about the rest of the organization?

For agents to work effectively inside an organization, they need access to its knowledge. They need to know which APIs to use and which services already exist. They need current conventions and clear warnings about deprecated approaches. The quality of that knowledge shapes every decision they make. The challenge is that organizational knowledge constantly changes. Documentation describes what we knew when someone last updated it, while everyday work keeps producing new evidence. Agents are increasingly present at the exact moment we discover that the guidance no longer matches reality.

Work produces knowledge

Imagine an agent implementing a feature. It recommends API A, and the developer corrects it: We don't use API A for new applications anymore. Use API B. The developer solved the immediate problem, but the interaction also revealed that the organization's recorded knowledge may be incomplete, outdated, or difficult to retrieve. What happens to that learning? Today, it will often remain in the conversation. The developer's agent might remember it and avoid the mistake next time. But another developer will still face the same old recommendation tomorrow, make the same correction, and pay for the same learning again.

Agents also make discoveries on their own. An agent might try API A, encounter a constraint, and investigate alternatives until API B works. It completes the task, but its path exposed information that could help the next person. Without a way to capture that signal, the organization benefits from the result once and loses the reasoning that made it possible.

A discovery is a signal

Should every correction become organizational guidance? Of course not. API B may work only in one scenario. The agent may have misunderstood API A, while the human correction may reflect a team preference rather than an organizational decision. Older documentation may simply have been easier to find than the current guidance.

Human corrections and agent discoveries are signals. They tell you that something deserves a closer look. Treating each signal as truth would fill the knowledge base with contradictions and make future decisions harder. Ignoring them on the other hand, wastes evidence produced during real work. This gives us a useful distinction: capture potential learnings generously, then verify them before they influence everyone else. The cost of proposing a correction should be low. The standard for accepting it as current organizational knowledge, however, should be high.

Change knowledge like code

We already have a model for changing shared systems. When someone modifies software, they propose a change and another person reviews it before it reaches production. Organizational knowledge can follow the same pattern. An agent could recognize that a task produced a potential learning and propose the smallest useful change: New customer-facing applications should use API B rather than API A. A reviewer can compare that proposal with current guidance and check it with the team that owns the decision. Once verified, the update becomes available to other people and their agents.

The code change and the knowledge change need separate lifecycles. A pull request may be rejected while revealing a valid constraint that belongs in the organization's guidance. Code may also be merged without teaching the organization anything new. Keeping the artifacts separate lets each one follow the review process it needs.

Keep current knowledge coherent

Suppose current guidance says to use API A, while a task suggests API B. Adding another document that recommends B leaves future agents with two plausible answers. Which one should they follow? More information has made the decision worse.

A reviewer needs to resolve what the signal means. If B supersedes A, update the current guidance. If B applies to a specific scenario, qualify the recommendation. If the knowledge already says B, the organization may have a retrieval problem rather than a documentation problem. An unresolved conflict should remain visible as a conflict until the owner makes a decision.

The goal is a coherent representation of what the organization currently knows. Version control can preserve that API A was once recommended, without forcing that historical guidance to compete with today's decision. Decision records can explain why a particular application still uses A. Current knowledge answers a different question: what should we do now?

Preserve the boundary of the conversation

Agent conversations are valuable learning environments because they can be private. People can ask questions they would hesitate to ask publicly. They can explore half-formed ideas, be wrong, and change their minds without turning every thought into an organizational record. Closing the learning loop shouldn't publish the conversation. The agent can extract a small candidate change while leaving the exchange private. Reviewers may occasionally need more context to verify a proposal, which creates a real tension between learning and privacy. Still, an organization can capture the useful claim without treating the whole conversation as knowledge.

Access boundaries matter too. An agent acts on behalf of a person and should see only the organizational knowledge that person can access. In a large organization, two agents may legitimately work with different views because confidential projects and sensitive systems require it. Shared learning should improve knowledge within those boundaries, not collapse them.

Close the learning loop

We usually describe organizational knowledge as flowing in one direction. The organization teaches the agent, and the agent does the work. But agents now participate in the work through which that knowledge evolves. They encounter stale guidance, discover constraints, and receive corrections from people who understand the context. Every one of those moments is an opportunity for the organization to learn: capture the signal, verify what it means, and update the current guidance only when the evidence supports it. That's how agents can improve organizational knowledge without turning every observation into truth or every conversation into a public record. Your organization already pays for these discoveries. The question is whether the next person will have to pay for them again.

When organizations can build faster than they can agree

Sun, 09 Aug 2026 12:00:00 GMT

Organizations can now build software faster than they can agree on what to build. AI makes a feature that once took weeks possible in days. A product manager can create a prototype without waiting for an engineer, while a support team can automate a workflow on its own. Working software can emerge from almost anywhere in the organization. That sounds like an unambiguous win, until everyone starts building.

Implementation is no longer the filter

For decades, engineering capacity acted as a natural constraint. Ideas were abundant, while turning them into working software required scarce time and specialized skills. Most ideas never made it past a document or a conversation because implementation was expensive. That cost encouraged teams to prioritize and align before committing significant engineering effort.

AI changes that equation. When an experiment takes an afternoon, you no longer need a roadmap slot or a dedicated team to try it. That's wonderful because more people can test their ideas with real users instead of debating them in the abstract. It also means that an organization can produce far more working solutions than it can reasonably adopt. The bottleneck moves from building solutions to deciding which solutions become the organization's solutions.

AI changes the economics of alignment

Imagine two teams facing a similar problem. They can spend time understanding each other's work, negotiating priorities and agreeing on a shared direction. They can also build separate solutions. When implementation takes months, the cost of duplicate work gives both teams a strong reason to align. When each team can build its version in two days, another meeting suddenly looks expensive.

So, what will a team do when building is cheaper than agreeing? It will often build. That's a rational response to the incentives in front of it. Implementation costs have dropped dramatically, while alignment still requires context and trust. It still takes conversations between people with different goals.

This shift can make a well-run organization look fragmented. Teams move quickly and solve real problems for their immediate customers. Each decision makes sense locally. Yet the organization gradually accumulates several answers to the same question, each with its own assumptions and maintenance burden. Nobody chose the fragmentation: it simply emerged from a series of reasonable decisions.

Parallel exploration is healthy

Preventing duplicate work altogether would waste much of what AI makes possible. Parallel exploration helps teams challenge assumptions and discover approaches that a central plan might miss. Two teams solving the same problem may learn that they were solving different problems all along. They may also discover that one approach works better than the other. You want that learning before committing the whole organization to a standard.

The challenge appears after the experiments produce useful results. How do you recognize that several local solutions now represent one shared need? Who compares them, and who has the authority to choose a direction? How does the chosen solution absorb what the other experiments learned? Organizations need a way to answer these questions without turning every experiment into a governance project before it starts.

Exploration and convergence happen at different times. Early on, variety creates learning. Later, the same variety creates friction for everyone who needs a dependable answer. The skill is knowing when the value of another experiment falls below the cost of another option. Too early, and you standardize before you understand the problem. Too late, and every team has something it can no longer afford to abandon.

The audience eventually becomes the customer

Several overlapping solutions may coexist quietly while each team serves its own audience. The cost becomes visible when someone crosses those boundaries. Another team looks for a capability and finds three. A customer encounters different ways to solve what appears to be the same problem. An AI agent recommends whichever approach happens to be documented best, rather than the one the organization wants people to use.

Who pays for that inconsistency? At first, it's the people trying to navigate it. They compare options and ask around to learn which guidance still applies. Later, the organization pays through duplicated maintenance and support work, while customers encounter conflicting experiences and leaders see initiatives pulling in different directions. By the time the full cost becomes visible, several teams already depend on their own solutions.

Software makes this pattern easy to see, but the same economics apply throughout an organization. Marketing teams can create overlapping assistants. HR teams can automate onboarding in different ways, while consulting teams build competing accelerators for similar customer needs. These outcomes can come from capable teams responding quickly to local needs. And AI makes each response cheaper, so organizations accumulate them faster.

Convergence becomes a capability

As experimentation gets cheaper, convergence becomes more valuable. The more ideas you can explore, the more important it becomes to recognize which ones deserve broader adoption. That requires visibility into what teams are building and a credible way to compare the results. It also requires teams to let go of a working local solution when another option serves the organization better. That last part in particular is hard, because working software creates ownership remarkably quickly.

Perhaps one day we'll measure how well organizations do this. DevOps gave us deployment frequency and mean time to recovery because those measures exposed capabilities that mattered to delivering software. AI may give us another: time-to-convergence, the time it takes to recognize overlapping solutions and align around the one that best serves the organization and its customers.

A short time-to-convergence lets teams learn before divergence becomes expensive. Healthy organizations will keep room for local experiments and make the path from a successful experiment to a shared capability clear. They will know who can make that decision and what evidence matters. They will also show how the chosen solution earns adoption.

Can convergence keep pace with creation?

Large organizations have always balanced team autonomy with company-wide coherence. AI changes the economics of that balance. More ideas can become a working solution, and more teams can solve problems independently. Each local optimization becomes easier to create, and the ability to converge now needs to grow at the same rate.

Building faster is a remarkable opportunity. It gives organizations more chances to learn and more ways to serve their customers. But the value comes from turning the best experiments into capabilities that others can discover and reuse with confidence. The question for organizations shifts from how quickly they can create to something harder: can they agree just as fast?

Every team needs a knowledge interface

Sat, 08 Aug 2026 15:35:00 GMT

For years, organizations invested in intranets, enterprise search, wikis, metadata, taxonomies, knowledge graphs, none of this is new. So why does it suddenly feel urgent again? Because the audience changed.

Three eras of organizational knowledge

We started organizing knowledge using folders. If you wanted to know how something worked, you needed to know where it lived:

Engineering/
    APIs/
        Identity/

Then came search, and location mattered less because you could look for what you needed. Organizations quickly discovered though, that search quality depended on knowledge quality. This realization led to years of work on metadata, ownership, ranking, freshness, and deduplication.

Eventually, the pendulum swung back. Search engines became smarter, and knowledge management systems inferred some metadata from documents. People no longer had to fill in pages of properties for every document. But the underlying work never disappeared. Organizations still had to decide which document was authoritative, which one had been superseded, and which guidance should be retired.

Now we enter the third era: reasoning. At the first glance, the difference seems subtle, but it changes everything. Where search returned documents, agents return decisions.

Search helped people find information

Search engines gave you ten documents and let you decide. When two documents contradicted each other, an experienced engineer often noticed. They asked around, knew which team had changed direction, and ignored the outdated wiki. We became remarkably good at compensating for imperfect knowledge.

Agents work differently. An agent doesn't ask Alice which API the team actually recommends. It reasons over whatever the organization made available and adjusts its course accordingly. And finding the right information is only the beginning.

Discovery is only the beginning

Suppose you ask an agent to implement a new feature. It searches your organization's knowledge and discovers an existing service. Great, but now it has to decide:

Is this service still supported?
Is it the preferred implementation?
Is there a newer replacement?
Does it fit this scenario?
Should I reuse it or build something new?

Finding information gets the agent only so far. The quality of its decision depends on the quality of your organization's current decisions. A stale document produces more than a stale search result: it produces a stale recommendation. And the more autonomy we give agents, the more expensive those mistakes become.

The audience changed

Historically, most teams optimized their knowledge for themselves. Most questions traveled from person to person. When someone outside the team needed help, they asked and someone explained. The people who knew the answers acted as the interface. That allowed teams to get by with incomplete documentation.

Increasingly, as we use AI more and more, the first consumer of your team's knowledge will be an agent acting on someone's behalf. That means that from now on, every team publishes a knowledge interface.

Think of the interface as a reliable way for the rest of the organization to understand what your team offers and how to work with it. It should answer questions such as:

What do we own?
What guidance is current?
What should people use or stop using?
Which exceptions apply?
Who owns the decision?
Where should someone start?

Notice what's missing: meeting notes, team discussions/chats, drafts, internal debates. Like a software API, a knowledge interface hides the internal discussions and exposes your team's current decisions.

Publish decisions, not everything

Writing more documentation won't solve this. Instead, you need to make the right things easy to discover, understand, and trust.

A team maintaining a small internal service may wonder why it should spend time curating guidance for five engineers. After all, everyone who needs to know, knows already. But say tomorrow, another team may face the same problem. When neither they nor their agent discovers the existing service, they'll build another one, not only unnecessarily duplicating the work, but also complicating the decision tree for the rest of the organization.

AI raises the value of organizational knowledge

Knowledge management and the importance of its curation isn't new. For years we've been highlighting the benefits of it to organizations. Today, as AI agents become more prevalent, that urgency increases even further. The more work we automate, the more decisions agents make without asking humans for clarification. That raises the cost of outdated guidance and contradictory recommendations as well as the cost of work that nobody outside the team can discover.

The organizations that benefit most from AI will make their current decisions easy to discover, trust, and reuse. This is necessary, because even the smartest model can't compensate for guidance your teams never published.

In the age of agents, every team publishes an interface to its knowledge. How good is yours?

"Coding is solved" misses the point

Sat, 08 Aug 2026 10:24:00 GMT

Every few weeks, someone declares that coding is solved. If they mean turning a well-specified problem into runnable code, they're increasingly right. But was writing runnable code ever the whole problem?

Language models have solved software engineering to the extent that word processors solved journalism: they made producing the output easier. Ask one to build a REST API or implement a React component, and work that used to take hours now takes minutes. Remarkable. But organizations need an answer to a harder question. Can AI solve the problems they actually have?

Demo apps are startups in miniature

The easiest way to convince yourself that coding is solved is to build a demo app, like a to-do list, a weather app, or a personal dashboard. The requirements are clear, there are hardly any constraints, and you're free to invent the architecture as you go. You build on a blank canvas.

Now compare that to a mature organization. The task might sound like add a button, but before you write a line of code, you need to answer:

Which service owns this functionality?
Is there already an API we should use?
Why wasn't this implemented before?
What security requirements apply?
Which architectural patterns are acceptable?
Which team owns this area?
How will this affect downstream systems?
What business constraints are we optimizing for?

How many of these questions are about writing code? Very few. Most require you to understand the organization.

That's why the same model that feels magical on a weekend project can feel merely helpful inside an enterprise. While the programming problem is similar, the environment makes it harder.

We mistook the bottleneck for the discipline

For decades, writing code was one of the most expensive parts of software development, so we started equating software engineering with implementation. AI is making implementation dramatically cheaper and moving the bottleneck elsewhere.

Automation changes what becomes scarce. As implementation becomes abundant, our attention shifts to the next layer of abstraction. Software engineering has always involved several layers:

Each layer answers a different question:

Business objective: What problem are we trying to solve?
Product design: How should users experience the solution?
Solution design: How should the system realize that experience?
Implementation: How do we express that in code?

Today's models are becoming exceptionally good at the bottom layer. The rest of the stack is still there. We're finally noticing how much work sits above implementation.

Organizations run on context

One thing separates hobby projects from enterprise software: context. More specifically, organizational context.

An organization already knows its architecture and domain language. It has historical decisions, engineering conventions, ownership boundaries, security requirements, business priorities. Years of building software have also created countless assumptions that nobody wrote down.

This context influences every step. Strategy and regulation shape business decisions. Customer expectations and established UX patterns shape product decisions. Existing platforms and services constrain solution design. Coding standards, frameworks, and deployment pipelines guide implementation.

A startup creates this context as it grows. An enterprise inherits decades of it. Asking a language model to build a greenfield application is fundamentally different from asking it to extend a ten-year-old production system. The hard part is making good decisions within an existing landscape.

AI is climbing the abstraction ladder

Every generation of development tools automates the most concrete layer of software engineering. Compilers automated machine code. Higher-level languages automated low-level programming. Frameworks automated infrastructure. Now language models are automating implementation. Each breakthrough feels revolutionary until the next layer becomes the bottleneck. That's exactly what's happening today.

The better AI gets at producing code, the more our conversations move towards understanding the business problem and designing the right experience. We spend more time fitting solutions into existing systems and making sound decisions within organizational constraints.

Code was always the medium

AI can write code. The mistake is assuming that writing code was ever the whole job. Organizations pay engineers to solve business problems. Code is the medium, not the outcome. So, is coding solved? If you mean translating a clear specification into working software, we're getting close.

Software engineering covers the whole stack. Every generation of tools makes one layer cheaper and reveals the next. AI is revealing what software engineering was all along.

Inference efficiency is about protecting attention

Mon, 03 Aug 2026 15:05:00 GMT

Inference efficiency is about protecting attention. Every unnecessary step an AI agent takes, spends reasoning capacity on the tooling instead of the user's problem.

For decades, we've worked to make software easier for people to use. We designed intuitive APIs and polished CLIs. We wrote better documentation and meaningful error messages. Good developer experience reduced friction, so developers could spend more time solving their actual problem.

The cost of getting this wrong was real, but mostly hidden. A confusing API wasted an afternoon. A cryptic error sent someone searching through Stack Overflow. Contradictory documentation led to trial and error. Eventually though, the task got done and everyone moved on.

AI makes that cost tangible. Each unnecessary step consumes reasoning time and space in the context window. It also uses tool calls and API quota. Ultimately however, it costs money.

Cost is only a consequence though. The scarce resource is attention.

Attention is finite

Imagine asking an engineer to solve a difficult problem. But first, before they start, you ask them to read two pages of legal disclaimers. You then ask them to recall their last vacation and walk you step by step through what they've done, as detailed as possible. Finally, you let them work on the problem at hand.

They'll probably solve the problem eventually. But every minute spent on something else, is distracting them from the actual problem.

Language models work in a similar way. Unnecessary reasoning takes capacity away from the user's task. That's the essence of what I call inference efficiency:

Inference efficiency is the degree to which an AI's reasoning is spent on solving the user's problem rather than overcoming the tooling.

Why is it defined around reasoning rather than tokens or latency? Because those are the effects you can measure. Attention is the resource you're trying to preserve.

Good tooling requires less reasoning

Consider a CLI that prints a large ASCII banner before every command. A person will likely ignore it, but a language model still needs to process it.

Or consider an API that returns pages of prose where a structured JSON object would do. A person skims the response, but a language model has to find and interpret the relevant parts.

Contradictory documentation is worse. A person becomes frustrated. An agent explores multiple paths and retries different approaches. It spends more inference before it reaches the correct answer.

A vague error message has the same effect. The agent has to diagnose the tooling before it can continue with the user's task. None of these problems are new. But we can finally see their consequences.

Cost tells only half the story

Poor inference efficiency makes tasks more expensive. Agents use more tokens and make more tool calls. They need retries and take longer to finish. But what happens when all that extra output stays in the context window? Decorative output and duplicate information compete with the details the model needs to solve the problem. Failed attempts stay there too.

As useful context gets crowded out, the model can make more mistakes. Those mistakes lead to more retries, which consume even more context. In the end, poor inference efficiency compounds. Yes, uou can correct the model or provide better instructions. You can even restore missing context. But the inference already spent is gone.

Someone else pays the bill

The people creating these inefficiencies rarely bear their cost. A tool author adds decorative output. An SDK team ships an API that requires several round trips. A documentation writer publishes ambiguous guidance. Each choice looks harmless in isolation.

Everyone using those tools pays the cumulative inference cost. Maintainers already pay for low-quality pull requests they didn't write. Operations teams pay for inefficient infrastructure they didn't design. Organizations are now paying for unnecessary AI reasoning caused by tooling they don't control.

People naturally optimize for the costs they bear, and inference efficiency makes the displaced costs visible.

Optimize for attention

We've spent decades asking how quickly software runs and how much memory it consumes. We measure bandwidth and latency. AI adds another useful question: how much reasoning does this require?

The best tools for agents will do more than let them complete a task. They will keep the agent focused by returning concise, structured output. Their errors will explain what failed and what to do next. Their documentation will give one clear path through the task. Minimal viable output.

The less an AI has to think about your tooling, the more it can think about the user's problem. That's inference efficiency. Protect attention.

Making room for your thoughts while reading

Sun, 02 Aug 2026 15:55:40 GMT

I want to try something new with this blog. Most websites treat reading as consumption: you open a page, read it, and close the tab. But reading often starts a line of thinking that continues long after the page is gone.

What if the page could support that thinking without getting in the way? I've added marginalia, a place for your thoughts next to mine.

Stop when something catches your attention

When a sentence makes you pause, select it. An action appears next to the passage so that you can capture what crossed your mind before you continue reading.

You don't need to switch to another app or collect quotes in a separate document. The note starts where the thought did, attached to the words that prompted it.

Think in the margin

Write as much as you need. A note can capture a quick reaction or a question you want to revisit. It can also help you work out what you actually believe.

These notes are yours. They stay in your browser on your device, next to the essay where you wrote them. Nothing is uploaded or shared, and I can't see what you write.

Return to your train of thought

When you return to the essay in the same browser, your notes return too. The marked passages reconnect each thought with its original context, so you can pick up where you left off.

The essay now holds what I wrote alongside what the text made you notice and where your thinking took you.

Take your thinking with you

Browser storage is useful until you want to keep a durable copy. Print the essay to PDF and your notes appear alongside the passages they belong to.

The PDF gives you a copy you can archive or read away from the browser. You can also give it to an LLM and continue working through the essay with your own observations already included as context.

Marginalia is an experiment in making this blog support the process that happens around reading. Try it on an essay and follow a thought into the margin. When you're done, take it with you.

We're measuring AI productivity the wrong way

Thu, 30 Jul 2026 11:50:00 GMT

When we measure AI productivity, we tend to ask one question: how much time did this save me? That's the wrong way to look at it. The one that actually matters is how much time did this save everyone. It sounds like a nuance, but it changes the whole picture.

The hidden cost shift

For centuries, writing had a built-in quality filter. It was expensive. Writing an email, a report, or a proposal took time, and every extra paragraph cost you effort. So authors edited themselves. The cost of communicating was paid primarily by the person doing the writing.

Language models changed that overnight. Today I can turn a handful of bullet points into three polished pages in seconds. The cost of producing words went to near-0. But the cost of reading them didn't move at all. We took the burden off the author and quietly handed it to the audience. In fact it increased, because the more we produce, the more there is to consume.

The productivity illusion

Say I use AI to generate a design document. Before AI, I'd spend two hours writing it, and ten colleagues would spend ten minutes each reading it. Total human effort: three hours and forty minutes.

Now I spend ten minutes generating it. I don't bother editing, because it already looks good. The document comes out four times longer, so those same ten colleagues now spend twenty-five minutes each getting through it. My productivity went up twelvefold. The organization's productivity went down.

Measure only my time and AI looks like a miracle. Measure everyone's time and it made us slower.

We've seen this before

The same thing is happening in software engineering. Submitting a pull request used to represent real effort. Someone had spent hours understanding, thinking, implementing, testing, and polishing. Now AI can produce a convincing implementation in minutes, so opening the PR is cheap. Reviewing it isn't.

The maintainer still has to answer the questions AI can't. Is this actually correct? Does it fit our architecture? What edge cases did it miss? Did the author understand the change, or just accept the output? The friction just moved onto the reviewer's desk.

Or think about stand-ups

We've all seen the 500-word daily update that says almost nothing. It documents every command run, every file touched, every meeting attended, and still leaves the only questions anyone cares about unanswered. What changed? What's blocked? Is there anything I need to do? The update optimized for activity, not for understanding.

The Bond principle

There's a scene in Spectre where Bond orders a vodka martini at a health clinic bar. The bar doesn't serve alcohol, so the barman hands him a prolytic digestive enzyme shake instead. Bond takes one look at it: "Do me a favor, will you? Throw that down the toilet. Cut out the middleman."

Increasingly, that's what we do with AI. Someone feeds bullet points into a model to expand them into polished prose. The recipient pastes that prose into another model and asks it to summarize the whole thing back into bullet points. We built an expensive pipeline whose input and output are nearly identical. All we added is noise.

The wrong metric

This is the same mistake I keep seeing when people evaluate AI coding agents. They obsess over token usage, but by itself it's meaningless. An agent that burns ten times more tokens but finishes the work in one attempt can be far cheaper than a "token-efficient" one that needs constant retries and hand-holding. So the number that matters isn't resource consumption. What we should be looking at instead is the total cost of getting the task done.

Communication works the same way. Writing- and reading time is our token count: the resource everyone consumes. The completed task is the decision or the shared understanding the communication exists to produce. So the number that matters isn't how quickly I could write this, or even how long it takes to read. It's the total human time it takes everyone to reach the decision. Time to decision is our task cost.

The real bottleneck

AI didn't eliminate the cost of communication. It redistributed it. Generation is becoming free. Evaluation is becoming the expensive part: reading, reviewing, verifying, summarizing, choosing. Those are the scarce resources now, and that changes what our job actually is.

The best AI users are the ones who remove the most. Generating a wall of text is easy. The real skill, and the real courtesy, is leaving everyone else with as little to read as possible.

Every configuration option is a question we couldn't answer

Wed, 29 Jul 2026 11:30:00 GMT

Open your favorite AI coding agent and you're greeted with a surprising number of decisions. Which model should you use? Should reasoning be low, medium, or high? How much context should the agent load? Which MCP servers should be enabled, which skills should it have access to, and should it ask before running commands?

Most of us don't know the right answers. We pretend we do, but if we're honest, we usually pick whatever worked last time or leave the defaults untouched. The strange thing is that we don't consider this a problem. We accept it as the cost of using cutting-edge AI, while actually, it's a sign that the technology hasn't matured yet.

A brief history of hiding complexity

The first cars expected drivers to know far more than they do today. You adjusted the choke before starting the engine, you watched gauges that most drivers today have never heard of, and if something sounded wrong, you diagnosed it yourself.

Modern cars are dramatically more sophisticated. They manage fuel injection, traction control, anti-lock braking, adaptive cruise control, and increasingly parts of the driving itself. Yet despite all that added sophistication, the driver makes fewer decisions than ever before. The technology became more complex and the experience became simpler.

The same thing happened with cameras, computers, and smartphones. As products mature, implementation details disappear behind intent. You no longer tell the product how to do something. You tell it what you want to accomplish.

Simplicity isn't about removing features

People often confuse simplicity with minimalism. A modern car has vastly more capabilities than one from fifty years ago. An iPhone has incomparably more features than the first mobile phones. So if it's not about features, what is it about then? Responsibility.

The product has taken responsibility for decisions that users previously had to make themselves. That's what "it just works" really means. The problem stayed just as hard, but someone else got good enough at solving it for you.

Configuration is a form of uncertainty

Every implementation setting exposed to the user represents a decision that the product wasn't yet confident enough to make on the user's behalf.

Notice the qualifier: implementation. Not every choice belongs to the product. Whether I prefer dark mode or light mode is a personal preference. Whether I want notifications is a personal preference. Those choices should stay mine.

But asking me which reasoning level to use? Or which model, or which context window? Those are implementation details dressed up as choices. Most users aren't qualified to answer those questions, and they shouldn't have to.

Why we still need all these knobs

To be clear, today's AI products are being honest about what they don't know. An AI agent doesn't yet know whether I'm prototyping or fixing production, whether I care more about speed or quality, or whether this repository is mission critical. It doesn't know whether I prefer concise or comprehensive solutions, or whether I value lower cost over a marginally better result.

Lacking that understanding, it delegates those decisions to me. Configuration compensates for missing understanding.

Better defaults won't fix this

Many people assume the solution is better defaults. I don't think that's enough. A single default can never know my situation. The real evolution is contextual judgment.

Imagine asking:

Refactor issue #123.

Instead of immediately asking you which model to use or which reasoning mode to enable, the agent begins by understanding the problem. It reads the issue, explores the repository, discovers the relevant code, and identifies dependencies. If something important is ambiguous, it asks.

I assume backward compatibility matters because this modifies a public API. Is that correct?

Notice, that the agent isn't asking implementation questions. It's validating assumptions about your intent. Once it understands the task, it quietly configures itself. Maybe it picks a small model to classify the problem, a larger reasoning model for the architectural decisions, and a fast model to update documentation before a verifier checks the pull request. Those decisions shouldn't be mine. They're part of the agent's job.

Product judgment

This reveals something broader about product design. It's easy to remove options. Any product team can hide an advanced menu. What's difficult is removing choices without removing capability.

That requires something much harder than simplification. It requires judgment, and judgment comes from experience, empathy, telemetry, and thousands of interactions with real users. The best product teams don't remove settings because they dislike configuration. They remove settings because they've learned enough to make better decisions than their users would have made themselves. They've earned that confidence.

AI is following a familiar pattern

Back in the 1990s, computer magazines compared processor clock speeds, RAM, hard drive capacities, and cache sizes. For enthusiasts, these numbers mattered. For everyone else, they were meaningless.

Companies like Apple rarely led with specifications. They sold outcomes. Not "5GB of storage" but "1,000 songs in your pocket." Not "an ARM processor" but "the internet in your pocket." The underlying technology still mattered enormously. It simply wasn't the customer's problem anymore.

AI feels remarkably similar today. We compare models, context windows, benchmark scores, and tokens per second. Enthusiasts will always enjoy those discussions, just as car enthusiasts still tune engines and photographers still shoot in manual mode. But most people don't want to become experts in model routing. They want to build a website, fix a bug, or write a proposal. The technology shouldn't be the task.

A prediction

I don't think model selection is going away. Modern cars still have manual modes and professional cameras still expose every control, so AI products will keep offering advanced settings for people who want them. But they will increasingly fade into the background.

The products that win the next wave of AI adoption won't have the largest context windows or the most configurable interfaces. They'll be the ones that consistently make good decisions on behalf of their users. They'll have earned the right to make those decisions, one good call at a time.

So the next time you're tempted to add another dropdown, checkbox, or slider to your AI product, ask yourself: is this expressing the user's intent, or is it exposing my uncertainty?

The next literacy is externalizing judgment

Tue, 28 Jul 2026 13:54:00 GMT

When ChatGPT took off, everyone wanted to learn prompt engineering. People shared templates and debated personas, XML, markdown, and chain-of-thought. Entire businesses emerged around writing the perfect prompt.

We've been optimizing the wrong thing. The biggest productivity gains from LLMs come from learning to externalize judgment. Better prompts barely move the needle by comparison.

I'll know it when I see it

Imagine you hire a designer. You ask for a "premium" website. A week later they show you a design, and it's not premium enough.

"What do you mean by premium?" they ask. "It just doesn't feel premium," you say. Neither of you is wrong. The problem is that "premium" exists only in your head.

Your designer has one mental model. You have another. The project becomes a series of guesses until you eventually land somewhere you're both happy with. This is an old problem. It's been part of every creative profession for decades.

What's changed is that now everyone has access to an LLM. Instead of occasionally hiring a designer or a consultant, we commission bespoke work dozens of times a day. LLMs didn't invent the problem of communicating taste. They democratized it.

The mirror we didn't ask for

One of the most common complaints about LLMs is that they don't get it. Ask them to make something more engaging, more premium. Sometimes the result improves, sometimes it gets worse.

It's tempting to conclude that the model isn't smart enough. But after months of working with LLMs, I've come to a different conclusion. LLMs don't eliminate the need for clear thinking. They make it impossible to hide its absence.

Humans are remarkably good at filling in the gaps. A designer asks follow-up questions, an engineer infers intent. They compensate for our ambiguity. LLMs don't. They faithfully expose it.

That can be frustrating. It can also be one of the fastest feedback loops you'll ever get for improving your own thinking.

From intuition to judgment

Many of us operate on intuition. We know good work when we see it. We know when a blog post feels too long or when a UI feels cluttered. But knowing isn't the same as explaining.

The moment you try to tell an LLM what you want, you discover how much of your expertise is implicit. "Make it more engaging." How? "Make it more premium." Compared to what?

The LLM has plenty of taste. The problem is that your taste exists as intuition rather than language. The real skill is translating that intuition into explicit judgment.

Judgment outlasts the prompt

A great prompt can produce a great result. But prompts are artifacts. Tomorrow you'll face a different task and write a different prompt. Judgment transfers.

Say you've spent time understanding what makes an article compelling. You might land on principles like these:

Challenge conventional wisdom early.
Introduce concrete examples before abstractions.
Every section should teach something new.
Leave readers with one idea they can't stop thinking about.

Those principles don't belong to one prompt. They become part of how you think. Your prompts improve because your judgment improved. The prompt is just one expression of it.

Why this matters

The people who get the most out of LLMs treat every interaction as an experiment. Instead of asking why the model got it wrong, they ask what they failed to communicate.

Instead of saying "make it better," they ask themselves: I'll know it's better when... That shift changes everything. You're no longer iterating randomly. Instead, you're refining your own understanding of quality.

The next literacy

For years, computers struggled to produce. Now they produce endlessly. The bottleneck has moved from generating work to defining what good looks like.

That requires a skill most of us were never taught, because we could usually rely on other people to bridge the gap. Today, you're often both the customer and the expert. There's no one in the middle to translate vague aspirations into concrete decisions.

The LLM can't read your mind. So you're forced to do something uncomfortable: make your standards explicit. This is the next literacy: learning how to externalize judgment.

Once you can explain what good looks like, prompts become easier and evals become possible. Iteration becomes systematic. And AI starts scaling something far more valuable than output. It starts scaling your thinking.