Software architecture, human judgment, and AI's limits with Grady Booch

I talked with Grady Booch about software engineering, AI agents, architecture, and what remains durable when implementation gets cheaper.

Grady has been part of several major chapters in software history: Rational Software, the Booch method, UML, IBM Research, software architecture, and work that spans everything from defense systems to NASA-related research. That background made this conversation especially valuable because he is not looking at AI coding tools from the same slice of software I usually live in.

We did not agree on every framing. In particular, Grady pushed back hard on the idea that the software development industry is going to end. I appreciated that pushback. It helped clarify something important: when people talk about "software engineering," they are often talking about very different parts of a very large discipline.

Software engineering is more than code

One of Grady's most useful frames was the journey from imagination to executable artifacts.

Some of that journey belongs to computer science: the laws of physics, algorithms, complexity, and theoretical limits. Some of it belongs to architecture: the significant design decisions shaped by cost of change. Some of it belongs to organizations, economics, risk, legal constraints, and ethics.

That broader view matters because it changes the AI conversation. If by "implementation" we mean writing common application code in a well-understood domain, agents can already help a lot. If we mean understanding risk, architecture, organizational scale, operational constraints, and moral responsibility, the picture is very different.

Grady also offered a helpful three-axis model: risk, repetition, and complexity. A low-risk, common, well-understood task is a very different thing from a high-risk system where people can die if the software fails. Product engineers need to know where their work sits in that space before they decide how much process, review, architecture, and trust is appropriate.

LLMs are useful but untrusted

I also appreciated Grady's language around large language models. He called them "unreliable narrators." They can be extremely useful, but they are nondeterministic, they confabulate, and they are not grounded in truth.

That does not make them useless. Grady uses Claude regularly. But he uses it with the judgment of someone who can smell when the work is going wrong. He treats it like a junior, overly enthusiastic programmer: helpful, fast, and often wrong in ways that require experienced supervision.

That distinction feels important. The goal is not to reject the tools. The goal is to understand what kind of work they are good at, what kind of work still requires human judgment, and what guardrails are necessary when the stakes rise.

Curiosity builds better judgment

The part of the conversation I keep coming back to is Grady's emphasis on curiosity.

He encouraged engineers to read source code from systems completely foreign to their normal work. Go look at MacPaint. Go look at MediaWiki. Go look at the Linux kernel. Study systems that were created under different constraints by people solving different kinds of problems.

That is such a practical way to grow judgment. You cannot develop broad taste, architecture sense, or engineering wisdom only by repeating the same kind of work forever. You have to expose yourself to other systems, other histories, other trade-offs, and other ways of thinking.

The same is true for books. Grady's homework was to read outside your domain. Read Herbert Simon. Read John Gall. Read Marvin Minsky. The point is not to collect references. The point is to expand the context that informs your decisions.

That is what product engineering needs in the AI era: not less engineering, but broader judgment about what should be built, what should not be built, and what it will cost to make the thing last.

Guest

Grady Booch

Computing: The Human Experience