Smart Understanding

Setting the stage

In the early 1980s, after graduating from MIT, I worked at the MIT AI Lab on a project called the Programmer's Apprentice.

Looking back, although the details differ because we were using technology over 40 years younger than what's available today, the vision behind it wasn't that much different.

We programmed in a language called Lisp that continues in use today, but at the time was one of the most common languages used in AI research. Machines called Lisp Machines had hardware support for the language, and the Zmacs text editor of the time was very much like the Emacs text editor is today. The Programmer's Apprentice was written in Lisp.

To go with that, we wanted to have an AI-style assistant that could see the programs we were writing, understand them, and take abstract instructions in English about how to modify them, and would make those edits for us. Pretty much the same model as an agentic “AI” programmer might be today.

Under the covers

The broad architecture of the Knowledge-Based Editor (KBE), or KBEmacs as it later came to be called, was very much like the way Large Language Models (LLMs) work today. Commands (similar to today's “prompts”) were issued to the KBE via an input box, the program itself would be digested into an internal representation, some transformations would be made to the program while it was in the internal representation, and then the result would be coded back up in Lisp (or whatever other language might be selected). At the time, Ada and PL/I were other languages that might be used.

What interested me at the time, though it seems routine now, was the idea of interacting asynchronously with another entity. Nowadays, people take agentic interactions for granted and there's been lots of intervening work on how to manage it all. At the time, we had the ability to spin off background processes and we had locking tools, but the rest of the mailboxing and coordination had to be spelled out by hand.

It was fun, actually. Charting the unknown. Something with the new “AI” people are unlikely to do any more. Now you just ask that it be done. That's convenient, but also a little intellectually sad.

Programmers bow to units of packaging called libraries. An apt metaphor in fact, because I think they often represent the packaged death of thought in the same way that great philosophy texts can represent as much the death of thought as the birth of it. Grand thoughts once having been thought, need not be revisited. Any goal of—much less right to—the simple joy of discovery is outright poo-pooed.
 —Kent Pitman, Whither Original Thought

A routine demonstration

Periodically, people would show up to see what we were cooking up. We had a well-practiced script that we felt illustrated the main ideas we were working with.

I was particularly interested in the interactions with the people funding us. They would watch everything carefully. I would watch them with equal interest.

Unlike some for whom we did demonstrations, it seemed obvious that they felt their role was to be a little bit skeptical. My sense was that they would watch as our interactions with the apprentice, directing it in English-like language to put a program together, and think “This is just form filling.” Form filling would not have been interesting. You could get that for cheap. They wanted to be wowed.

So they would watch with apparent boredom as the demonstration proceeded. I watched their faces. I could hear their minds. “Yeah, yeah. Bunch of forms being filled. Yep. Yep. More form stuff. Probably a garden variety template thing.” Their bored silence spoke volumes.

A less-routine response

Then an unusual command is issued. “Document the program.” That doesn't result in either a regurgitation of what has been said or a transliteration of the program into text. The documentation speaks at a high level about the structure of what the program is doing.

The funder is now alert to something unusual. They do a double-take. They pause and say, “Oh. It's smart.”

I am pretty sure I saw this effect more than once in different people. It really stuck with me. And I pondered why always in that moment. And why “smart”?

I came to believe that “not smart” meant a single representation being used in the ordinary way. A form to fill out. A function to graph. A program to execute.

“Smart,” I inferred after some thought, meant that the entity in question had different ways to approach a problem and could shift representation in order to accommodate the need of the moment.

A long-held idea

It wasn't a scientific proof, just an intuitive hunch, but it's been decades now that I have carried this in my head, watching the world to see what people think intelligence is. I have not found any strong reason to think that this is a terrible way to understand what we mean by “smart.”

When we know how to do something, we think of that as a method, an algorithm, a tool. But when we don't know how to do something, we have to improvise, to build something out of spare parts. Those parts have to fit together, and often they were not designed to go together. Correspondences have to be found that will let them align, communicate, and function as a kind of ad hoc machine of the moment.

Intelligence, it seems to me, is not the machine. It's the ability to cobble it together on the fly. It's why we have trouble looking at the ability to answer standardized tests as a true Turing test. Once the questions are known, there is no cobbling together. There is just executing of a known pattern.

Surprise is essential to testing intelligence. And that's why these funders were pleased. The outcome caught them by surprise. They trusted us not to have just wired up the answer, and so what they saw were signs of a toolbox of different models being seamed together dynamically. To them, it seemed to me, and still does, it was the earmark of intelligence.

 

---

Author’s Notes:

If you got value from this post, please “Share” it.

This is memoir, not autobiography. I did my best to piece together old memories. Although there is a recent project of mine to make a public archive of some of the artifacts. The sources to the KBE are available now at github as part of my history project. Jeff Shrager then took what I had restored and created code that runs again, with the help of an LLM, as part of his Executable Archaeology project.

The image was generated with help from abacus.ai's ChatLLM (Claude Opus 4.8 and GPT Image 2.5), with light post-processing by me in Gimp to reduce the resulting image in size for faster web download.