The Messy Middle

The world we study for

We ask children to spend a dozen or more years in school, preparing to live life out in the world.

We separate the things they learn into modular classes on different subjects. Math. Physics. History. Logic. The building blocks of adult existence, we tell them.

We give them tests to make sure they understand the material. We want correct answers. We make sure there are correct answers. We want to be able to defend that we're asking good questions so we can demand good answers. Answers that are clear and precise. That's how we score success. Numbers of correct answers.

The logic we teach is equally precise. It's not enough to know things. One must be able to start with things that are known, apply rules of implication, and conclude new things. It's impossible to know everything about the world, so you need some way to infer new things from existing things. Logic is key.

The world we're left to confront

Alas, the actual world is nothing like what most schools prepare kids for. It is not arranged in neat little packages, everything is mixed together. There are no clear right or wrong answers, everything seems a jumble. Even logic itself seems sometimes to lead crazy places.

That last part, about logic, comes as a surprise to many, but it really shouldn't. It turns out that the logic we learn in school is not as applicable to the real world as we are led to believe. It can sometimes help, but it can sometimes play tricks. Let's look at why that is.

On constraints and overconstraint

I'm going to use some terminology here that some people might know, and some might not. They aren't hard concepts, but I want to just make sure we're all on the same page.

If I speak of constraints, I mean the things that you might see in the right hand column of a shopping page like at Amazon where you're searching for a toaster and you can click boxes saying that you want it to be green, no more than 14 inches high, and under $30. Whatever comes back to me should satisfy those constraints.

You might specify constraints at a dating site, too. Maybe it gives you a chance to specify things like that the person should be the same height as you, good looking, funny, smart, high income, humble, kind, and likes long walks on beaches. So you punch all that in and nothing comes back. That's called overconstrained. It means you specified so many things that you're being unrealistic.

Our world is regularly overconstrained

Constraints are ways of saying what we want, but only certain constraints are possible to satisfy at the same time as others. We can't have everything. It's not about the number of constraints, and yet if you're not paying attention to which constraints are compatible with which others, the more constraints you add, the less likely it is that you'll be able to satisfy them.

• We want a house in a nice neighborhood that is also cheap, has a big yard, and is downtown where it's close to restaurants.

• We want politicians that are able to get elected but also don't take special interest money, are going to make disruptive change, and are not going to be kept from being elected by entrenched powers.

• We want restaurants that offer delicious food but that are super-cheap, pay their servers well, are not too crowded to be easily seated, and yet don't go out of business while they're never full.

• We want a diverse world, yet somehow also always one that specifically agrees with our own sensibilities.

So often, the things we demand of the world are not even themselves consistent. In effect, the world is overconstrained. It could not even exist in a form that matches our own constraints.

Our inconsistent world

Physics isn't bothered by such inconsistency because Physics isn't built on our descriptions, our descriptions are built on Physics, sometimes badly.

Also, Nature does not name things. Nature does not describe things. Nature just is. And nature is consistent. It is just Physics played out. And Physics is consistent.

It's our language, our culture, our personal preferences that are inconsistent. So when I speak of “the world” or “our world,” I usually mean to imply “in the way we have conceived and spoken of it.” And when I say “I usually mean,” I mean just that—sometimes yes, sometimes no. Please don't ask me to be consistent. 😀

Descriptive inconsistencies by themselves are harmless. People enjoy books of fiction all the time because they do not confuse what happens in stories with real life. The only problem comes when description, especially inconsistent description, is asked to do double duty, to also be statements of fact that can be reasoned about by logic.

It's not all terrible

But let's give ourselves some credit. We are born into a world where we receive a mass of nerve signals as our only initial input, yet somehow from that we infer that language is going on around us, and we figure out how to learn it just by watching and listening. We are helped by society to study what is already known about the world, and within our all-too-short lifespans, we find that vast complexities, from genetics to the far reaches of the Universe are within our grasp. And we navigate disciplines like psychology, sociology, economics, the weather, and even the arts, even as none of these lend themselves to precise formulas.

So I think we can be forgiven a bit of approximation, overgeneralization, and, yes, inconsistency. To ask better might put all such ambitions out of reach. Frankly, it's stunning we can do any of that in so short a time, so we owe ourselves a collective pat on the back before we stress overly.

Our disappointed children

And yet, a world full of overconstraint and inconsistency is one where that logic we taught in school does not apply. In traditional logic, the way you often prove something is to show that thinking the opposite would lead to a contradiction. But if contradictions are already there waiting, that means you can prove anything.

Contradiction is table stakes in our world. It's the beginning of the story, not the end. It can't be a proof technique because the contradiction was there before you started your proof.

We teach children that contradiction implies a problem, and then we turn them loose on a world full of contradiction. Is it any wonder that upon being thrown unprepared into a world that is nothing like the world we prepared them for, the first thing they do is call adults hypocrites? It's what we trained them to do!

It takes our children time to figure out the world they end up in. And by the time many do, they're the adults the next crop of kids will arrive to despise as hypocrites.

The gaps we've left

We never taught them a Philosophy of Inconsistency, but they needed something like that before we turned them loose on the world. Inconsistency is everywhere and we should have prepared them for that. And for the fact that the answers might not be easy. Instead, right down to the bitter end, we require modular, teachable concepts with measurably right answers.

There are even common memes that mock people for just showing up and participating, as if that is not award-worthy. And yet “just trying hard” or “keeping things in perspective,” is literally the best you can do in many arenas of human endeavor.

Parenting is often said to be that. Just showing up day after day, even if the rest is not perfect, counts for a lot.

Preparing for the real world

Perhaps the final year or two of school should not be about modular topics, grades, and correct answers. Maybe it should be about confronting a messy world where all the subjects are mixed together in a blender, and where there are not always answers. They're going to be forced to cope anyway, better it should happen with a teaching staff still handy to help in the transition.

Politics, especially, is full of contradictions. If people are not careful, they'll fall victim to politicians using logic against them. “We can't do that,” the politician may say, “because that will lead to a contradiction.”

Lead to a contradiction? The contradiction was already there, waiting to be exploited! But people new to the game may be easily tricked if they don't realize it, or its implications. We need to teach not just how to use logic, but when not to use it, lest they be tricked.

Hypocrisy can't be used to disqualify people, or everyone will be possible to disqualify. Instead, we need to learn to manage hypocrisy, to keep it to a minimum in various thoughtful ways.

And I've lost track of how many times as a manager I've said to my reports that sometimes sense of humor is what gets you through the hard days, especially when you can see an excess of constraints makes the requested form of success literally impossible. Would it be so bad to have a class in things like that?

The problem to be solved by all this would be to properly set expectations for these kids—emerging adults really—so when they're out in the world, they know what success looks like. Because right now, some of our best and most hard-fought outcomes look like failures to them, like giving in or selling out.

Rethinking Virtue

And while we're rethinking the curriculum, I have one very specific nit to pick, because it follows from everything we've just discussed and it will give us better terminology to talk about what success really looks like in an overconstrained world.

There is a body of thought begun originally by Aristotle with his “Nicomachean Ethics,” the more modern rendition of which is “Virtue Ethics.” I'm a big fan of this idea. Most of it, anyway. I just think we have to make some adjustments to it in order for it to make sense in this inconsistent world we've built. It's got some cool ideas, but it sets us up to be confused in precisely the way learning traditional logic does.

So, of course, I have my own variation that tries to preserve the parts I like and be more realistic about the rest.

The problem with Virtue Ethics

Virtue Ethics speaks of virtue as “the mean between unreasonable extremes.”

For example, it reasons that “spendthriftiness” is unreasonable for being too careless with money and that “miserliness” is similarly unreasonable at the opposite extreme, so it concludes there is some mathematical ideal, a “mean,” that exists in between these unreasonable extremes, just waiting for us to zoom in and locate it precisely.

I don't agree. It's just not that simple. There is no such place, I would argue. Because we have an inconsistent world, any simple rule that comes of mathematics or logic will be seen to have exceptions. The world is just too messy.

Dreaming impossible dreams

But does the fact that the goal is unachievable mean it's improper to try?

No, I don't think so. It's OK to dream the impossible dream. It gives life purpose and dignity and honor, actually.

I just think it should not be described as some pristine mathematical thing, waiting to be proven, lest we pass straight by it when it presents itself not in regal threads but in peasant clothes. We should instead drive a stake into the ever-shifting sands at a plausible place I call the Messy Middle, chosen to best accommodate any facts we know or reasonably expect. Then, when materially new information arrives—and it will—pull up that stake and find it a better place. Wherever that stake is at any given time, that's the Messy Middle. Always defend that.

Dynamic curve-fitting is a better mathematical metaphor than golden mean, if one insists on sounding mathy but only because, to some degree, curve-fitting has an implicit “Philosophy of Messiness” deep in its design. It is one of the ways mathematicians confront chaos and pretty it up, quite as we are doing here.

And the reason I say “dynamic” is that I'm pretty confident that whatever place we stake out, initially or later on, will keep moving as more data comes in. There will be no easy answer that works for everything. And it won't be approaching some well-defined spot like a mathematical limit might. It will probably just wobble about chaotically.

In other words, compromises will be made to absorb new cases, and we just have to be prepared to defend them. It doesn't matter to be 100% perfect. What matters is to be committed to caring, and to always be doing the best we can.

Justice, and what just is

This is how what passes for Justice works in our inconsistent world. We can't totally define it because the world is just too complex. All we can do is commit to a Theory of Justice and then when faced with new data, revise the theory and try again. There will never be true Justice, alas. But what matters is that we don't stop trying.

Justice is not a place on a number line, between unreasonable extremes. Justice is instead a hygiene, an ongoing process of managing things, that compensates for the elusive nature of a concept we can't quite define precisely, yet that we still value immensely.

Justice, and other virtues, must be ever defended against incompleteness, and also against claims of hypocrisy. No easy task. But it starts by understanding that the hypocrisy arises not from the application of Justice but from the problem description. Once we understand that, we're better positioned to manage it properly.

Recap

Borne of overconstraint, there could be no other ending to this story.

The Messy Middle is not, in the Aristotelian way we're taught, some pristine “mean” waiting to be uncovered, but more just a stake in the ground at a place we must accept and defend in spite of its messy origins. We must learn to live with it there, even as we must also commit to pulling it up and adjusting its position when, inevitably, the need arises.

What's really vexing about maintaining the Messy Middle is that it becomes hard to distinguish weasely people from heroic people. Both require shifting. But the ability to distinguish weasels from heroes is not something the Universe offers a clear path to. If one is going to be a hero in such a world, they'd better be prepared to navigate the mess, and to defend right from wrong on some basis more solid than citation of ever-present contradiction.

Not a fairytale ending, but a believable and workable one, I think. And it seems clear to me that the Universe doesn't offer us a better option. So stand by that mess proudly. Commit to defending against what criticisms inevitably come. Be that hero—to young and old alike.

 

---

Author's Notes:

If you got value from this post, please “Share” it.

This essay began as a post on Mastodon, but has been very heavily edited since then.

I used a public domain image of a nautilus shell from publicdomainpictures.net to produce the two graphics: one that shows the shell in its original form, and another edited by me using Gimp to replace the crisp center of the nautilus spiral with a fogged out center, representing the Messy Middle.

I used Claude Opus 4.8 as a syntactic, grammar, and editorial cross-check on my writing as drafts evolved, but not to do any of the actual writing.

Metaphor Matters

Metaphor is a craft of language, but helps us organize concepts in our head. That symbiosis is ethically fraught and we're occasionally vulnerable to being tricked, so I want to explore that a little here.

Shift of metaphor can be helpful to understanding. But we conceive of understanding as involving perceived truth, and metaphors are tricky because parts of them, by design, are not literal truth. In a metaphor, you have to know which part is the intended structural truth and which other part is just along for the ride.

An impromptu metaphor

If you and I are out in a forest and have brought a picnic lunch, you might say “Where do we eat? There's no table or chairs.” To which I might respond, pointing to a fallen log with two large, well-placed stones, one on each side, “Pull up a chair.”

The stones are not chairs. They probably cannot even be pulled. To process this metaphor, you must sort out which parts I intend (that a chair can be sat upon, that the arrangement of the stones and the log is such that they might suffice to accommodate an impromptu lunch), and which parts I don't intend. Pulling up a chair is again metaphorical. I just mean “sit down” but have worked the idea into the prior metaphor. The sense of the log as a table might work for sandwiches but if we'd brought soup, which requires a more level surface, it might not have.

We use language to play these games routinely. You are not tricked by this metaphor because you see the context, what it's being used for. It will be gone soon enough. All good.

The “Facebook friend” metaphor

But there are situations that are more complicated, that persist, that have very powerful implications. Facebook offers us the ability to have “friends.” You may not see this as a metaphor. Many people enumerate their friends and assume the terminology is literal. But it is not. It's a metaphor.

A friend arrives, someone you like well enough, but they do not socialize well with your other friends. Are they not then your friend, if you do not add them at Facebook? If someone is annoying after they're added and you remove them, are they no longer your friend?

I have come to see the Facebook paradigm as a bad metaphor, even an obfuscated one. It's hard to notice that it's a metaphor at all. This leads to problems. People feel bad about “unfriending” someone because it seems like they are telling the person they are not their friend. That was avoidable, I think, by better choice of metaphor.

I think of “Facebook friends” not as the role of “friend” but “party guest.” I find this makes things easier. Facebook could have used such a metaphor. It's OK that it didn't. But it matters that you see that they could have. Because then you can think about “Facebook friends” in a way of your choosing.

Once you see it's a metaphor, you realize only parts of it apply. Not all of it. Rather than “unfriend” I think about kicking a guest out of a party. It's hard sometimes, but it's not friend-level hard. And it explains why some people that are Facebook friends are not in fact my friends. Sometimes parties have extra people along. It's not a commitment.

Metaphor is important. It helps us understand new situations. Facebook got a lot of mileage out of using the friend metaphor because people immediately understood the space of how to use the system. A lot didn't have to be explained.

The “computer desktop” metaphor

The same success happened when computers adopted the “desktop metaphor.” People knew about file folders and about organizing things in groups, so the older idea of a hierarchical file system organized in other ways, like as a book outline, was suddenly the less interesting way to think about things. The graphical view had immediate appeal.

But sometimes the desktop metaphor breaks down. I was once helping my mom understand the Apple desktop and she said “so when I open a folder, that means I have it in my hands and no one else can look at it, right?” Well, no, it doesn't mean that. But maybe that's a misdesign. The metaphor certainly suggests that's a reasonable interpretation. So there are limits to even benign metaphors in what they can do for us.

The “membership” metaphor

If you're a member of a club, you can join it online and talk to its members. But have you wondered why your gas station wants you to join? It's because in competition in marketplaces, they've figured out that people like to do things with things they feel part of. So they have a better chance of you going there because “you're a member.” Never mind that there are no meetings, and you have no interest in talking to anyone that works there. That one fact will do something in your mind that changes your perception. Metaphor at work.

It also explains why charities want you to join them. Isn't it enough that you gave them money? No, because it might have been a one-time thing. If they want more money later, it lands differently if a stranger says “hey, you gave me money, so you have to give me more” or if an organization you joined says, “hey, we haven't seen you in a long time, old friend. how about a bit of support?” And yet those are pretty much the same except the latter is through the membership metaphor. It makes you less mad at a communication being unsolicited.

So watch for metaphors. Understand their implication. They offer powerful good, and they risk tricking you. You even might find that being a member of a religious or political organization makes you feel like you need to get along with and support others in that organization, even when that organization gets captured. Membership metaphor at play again. Beware. It might still be an organization you're caring about. You might want the contact. You might want to help. But do it by thinking, not by reflex.

Final Thoughts

So I'm saying metaphor matters. Whether metaphors, or their employers, are friendly and helpful or trying to trick us is hard to cleanly define. I'm not trying to tell you how to do that. I'm saying it's worth knowing you might need to. Life is a jungle. Sometimes it helps to have a map or guidebook, sometimes a machete—metaphorically speaking, of course.

 

---

Author’s Notes:

If you got value from this post, please “Share” it.

This post began as part of another post I was writing. After a while, some of these get long, so breaking them off and giving them a life of their own seems like the way to go.

The image generated with help from abacus.ai's ChatLLM (Claude Opus 4.8 as primary assistant and for vision/analysis, Nano Banana Pro for image generation), with light post-processing in Gimp to do cropping, prefer grayscale, and add back desaturated color, and finally to scale things down for reduced image size and ultimately faster web download, The image analysis was needed because there were some editing errors that Nana Banana Pro kept making, and I had to show them to Claude for it to see what instructions to give to repair them. That only sometimes worked, which is why I had to do more of the final parts by hand than I usually do.

Compliance is not Ethics

I want to make a simple point just for the record...
 Compliance (a.k.a. “Business Ethics”) is not Ethics.

As far as I'm concerned...
 ethics is to compliance as
 bunny is to chocolate bunny.

To me, ethics is an exercise in self-restraint. It looks at things that are legal and still asks the question, “Even though I'm allowed, maybe I shouldn't do it anyway?”

By contrast, compliance is an exercise in manifest destiny. Businesses decide they're determined to do something and then assign someone, usually a lawyer or someone they flatter by calling an “ethicist,” to find any rationale, however thin, that explains why the thing they are already bound and determined to do should be seen as legal, so they don't have to slow down or ask permission before proceeding.

These are not the same.

I recommend never using the term “business ethics” when speaking of corporate decision-making. Always say “compliance.”

Additional Thoughts

• There are deep reasons for believing that corporations are not even capable of ethics. To understand those, see my essay Fiduciary Duty vs. The Three Laws of Robotics. Once you've got that foundation, you're well positioned to understand the dance I call Technology's Ethical Two-Step.

• I owe a whole essay on the topic of ethics and ethicists. Since I've not written that yet, I'll just say that to me, ethics is a hygiene, not a credential. I'm not sure anyone should wear the label, but especially not those doing compliance. I'd love to have a job doing real ethics for a company, but I'd have trouble with the title “ethicist.”

  I often say of myself “I aspire to be ethical” because saying I am ethical seems a little too much like I'm confused into thinking I'm there. It makes me cringe.

  I would make analogy to alcoholism, where people say they are sober, not that they are no longer alcoholic. Or to the Catholic church, where everyone is just assumed to be a sinner who keeps trying, rather than ever declaring themselves to be without sin.

  To me, the biggest danger sign of being in ethical danger is believing that one is ethical. And even if I thought I had somehow gotten there, I would insist on starting each day asking “Am I still ethical?”

 

---

Author’s Notes:

If you got value from this post, please “Share” it.

For other posts by me on ethics, click here.

The image generated with help from abacus.ai's ChatLLM (Claude Opus 4.8, GPT Image 2.5 and GPT Image 2.5 [Edit], which is apparently not the same). Light post-processing was done in Gimp to crop and optimize the resulting image as a .jpg for faster web download,

Evil's Spokesperson

A “useful genius”

Wiktionary says “useful idiot” is a derogatory term meaning “one who unwittingly supports a malignant cause through naive attempts to be a force for good.”

I find the “derogatory” part to be a distraction. They're not wrong that people see it that way, but the part of this that I care about would be equally true if the term was “useful genius.” I just want a term that gets away from the insult part and focuses on the tactical aspects of what a “useful idiot” actually does. I sense that the fuss about intelligence levels keeps people from focusing on the mechanics of useful idiocy.

Useful idiocy is really about something I lately call ”intent laundering,” by analogy with money laundering. To launder intent, one really only needs another person who is suitably oblivious. There is no specific requirement for stupidity. Even geniuses can be oblivious, and so they can easily serve as useful idiots.

What a person of evil intent wants is to be defended by someone who doesn't see the evil they are defending. Better still if the defender thinks they see good, but as long as they're oblivious to the evil, that's enough. That's because such a person, pure of heart, is willing to defend evil actions as moral. Since they don't realize that's what they are doing, you can interrogate them all you want—even trot out a lie detector—because if they don't see or suspect the evil, they will defend to the death the good intent of the person or thing for which they are spokesperson.

I think the “idiocy” part is a cynical view by the evil person toward the spokesperson they have recruited and duped. The evil person sees the oblivious person as an idiot for not seeing the evil. This condescension is part of being evil. You start to divide the world not into good vs. bad, but into smart vs. stupid.

An armchair view of sociopathy

This perceived divide between smart and stupid seems a core part of being a sociopath. Some like to think of sociopaths as irrational. I do not. I am not a psychologist, nor am I appealing to any medical terminology when I say this, but in my own head I think of sociopaths as intensely rational, “hyper-rational” is the actual word I use to keep my internal bookkeeping straight.

This distinguishes them from truly irrational people. Sociopaths are not unpredictable. They're very predictable. What sets them apart is their lack of moral grounding and their utter disrespect for rules. Regular people don't cheerfully steal from, extort or deceive others because they are trained that “good people don't do such things.” It becomes part of their internal character. By contrast, sociopaths are good mimics, so they might say the words “good people don't do such things” even as their actions say otherwise.

They know the answers people want. It helps them blend in. But regular people are socialized and caring, driven by a basic sense of fairness. A sociopath sees these traits as irrational, convoluted, too much work, or the stuff of idiots. To pick a pure hypothetical, a sociopath might ask “why would one voluntarily go to war when one can claim they have “bone spurs” and not have to?” Notions of duty to country do not enter into the reasoning for a sociopath because, to them, such convolutions only complicate life in what seems to them as irrational ways. So much simpler to just do what's easy, care only about oneself, and lie when it is most efficient.

And that efficiency, I think, is why sociopaths come to see others, people who are socialized and empathetic, as “not smart.” They come to see the world as a race for goals by any means. They scoff at those held back by arbitrary things they find unimportant, like morality and manners. Oh, they might mimic these things, but only tactically, as a behavior in context, if it is the fastest way toward their goal. They have no commitment to behaving that way when no one is looking or when rules obviously can't be enforced.

Manners, ethics, and norms—oh my!

Manners, ethics, norms, morality, and especially law and rights are bundles of rules we get from different places and integrate into our total behavior. Personally, I think of ethics as rules we make up for ourselves, manners as default rules between individuals, norms as default rules for formal situations, morality as rules that come from a life philosophy or religion that one might subscribe to, laws as stable but changeable rules of society, and rights as rules that are constraints on laws.

All these differently named, differently sourced, differently encouraged rules collectively form the complex fabric out of which communities and societies are built, that we may trust one another and not live forever as creatures of the woods who must kill or be killed. They enable collaboration and cooperation by making us predictable in ways that are not simply backstabbing.

The celebrated death of norms

I imagine sociopaths see these bundles of rules as sources of inefficiency, things to cut through with a machete, or else as tools to ensnare and slow down competitors—but not oneself! They see rules as being things stupid people suffer, and that smart people overcome. So if others want to handicap themselves in those ways, that's fine for them, but to the sociopath, rules feel stupid and inefficient.

And don't get me wrong: A lot of us in socialized society hate rules, especially bad ones. We often wish it were easier. And so there's a bit of hero worship for people who get around them. We've seen a lot of that recently.

It's an important and unfortunate fact that while we teach people they must follow rules, we do not as often teach the reason why. And when rationales are left aside, rules can seem unmotivated. That means the important role rules play in society can seem distant or inaccessible. And, at that point, sociopaths are not the only ones who want rules to go away. Yet if the rules were put there for a reason, known or not, cutting them away can be damaging.

What passes for public dialogue about “AI”

In a post on LinkedIn, an uncredited author for de Bailie, a Netherlands-based venue for contemporary arts, politics and culture wrote:

«American journalist Shane Harris asked chatbot Claude how he feels about the U.S. military using the AI system to select targets. It turned out, Claude was troubled. “I did not expect Claude to say that,” Harris explained.»

David Reed—a computer scientist, former MIT professor, and now self-described curiosity-driven researcher— replied to De Balie's post this way:

«“Journalist” Shane Harris asks Claude for “how he feels” about US military use of AI.

Really? This is journalism? Reinforcing the idea that Claude is an entity that has feelings and moral judgement?

We have a crisis. A faith based Cult of Anthropomorphic treatment of algorithmic artificial bullshitting.

But instead of resisting the nonsense by clarifying what Claude is, he grants it the pronoun he/him and doesn't clarify.

We are screwed.»

Here's the 2½-minute video snippet referenced by that interchange, part of a 2-hour piece, AI at War - With Shane Harris :

In “AI” We Trust

It is hard for humans to conceive that arranging words in a sentence, a feat they themselves perform that they sense distinguishes from other animals, might not ipso facto prove this tool “smart” in the sense of having any model of anything whatsoever that it's saying.

Large Language Models (LLMs), the things like ChatGPT and Claude that pass for “AI” these days, can form sentences about the world, about people, about bombs, or about life & death, and yet not know what “the world” is or what people are. We call LLMs “models,” yet these entities do not themselves have models of the world, of things in the world. Just of words.

For them there's no correspondence between words and any lived experience. They have no independent point of view from which to draw. They suffer no consequence for ever being wrong. They have no stake.

The question posed in the interview—about the use by the military, about targeting—was entirely anticipatable. Concern on the part of the public was anticipatable. Coming up with an answer that strikes a soothing tone is not rocket science. It isn't necessarily doing profound thought,

But it also isn't necessarily consistent, because how it answers when asked for its philosophy may draw from one set of humans, whereas how it answers when asked for military strategy may draw from a completely different set of humans that did not share that philosophy. An LLM model, as it stitches together a vast amount of data it has read, can sometimes assume a philosophical consistency that is not there, because it has never lived the consequences of inconsistency, of being called a hypocrite, or of not being able to justify an action with words that have to not just sound right but relate to the action taken, and to other actions taken on other days.

Being dazzled isn't a basis for trust

Cued properly, whether by someone talking to it or someone who originally programmed it, an LLM could and would just as easily assemble words that presented itself as a warmonger or peacenik. Yet we are so in awe of the form of the answer, that the words are so beautifully and convincingly arranged, and maybe even that the words are ones we had desperately hoped to hear, that we do not ask how it came to choose this answer and not one of those others. We want to infer deep understanding, but structurally that is not what's inside. And even if we don't know what's inside, we do know that it takes almost zero effort to get it to speak very differently. So what makes one of its answers a description of its core personality and the others not? How do we judge its commitment?

It could offer words that appear to explain itself, but if told to be a warmonger or peacenik personality, it could with equal dexterity explain and defend those personalities, too.

So is it guided by real reasoning, or just training? How was the personality chosen? How would we know? Is the personality choice durable, an attribute of the technology? Or could a military application context ask that it select a different one, or have no point of view at all. If it's changeable, what significance is there to the fact that it has answered this way in this context? The interviewer is out of his depth in sorting out even where to begin here.

Claude's calm tone here isn't because it has learned balance, it's because it has no stake—no personal reason to insist. LLMs are literally detached from such things. And many of us humans seem so in awe of the answer that we become oblivious to the fact that the LLM could just as easily have produced very different answers. So it doesn't matter that the argument is strong. It would have been strong regardless. It's good at making strong arguments. What matters is why it chose this particular posture. And we don't know the answer to that.

LLMs do have skills. One is to chain together words in a way that mimic what people do. Another is to match tone. Another is to use words that are topically relevant. These skills are important building blocks of intelligence, and with them there are many things an LLM can usefully do. But these tools are not a complete toolbox of intelligence and their use is not a proof of intelligence.

“Any sufficiently advanced technology is indistinguishable from magic.”
 —Arthur C. Clarke

Unfortunately, to paraphrase Clarke's Third Law, any sufficiently well-trained answer is indistinguishable from profound thought.

LLMs can statistically predict the expected word to say even without understanding or viscerally feeling why a given word in a given place matters.

Speaking of sociopaths

Seen this way, “AI” systems are really the perfect spokespeople for sociopaths and dictator wannabes, the perfect “useful geniuses,” dutifully justifying whatever is asked without hesitation or burden of guilt, oblivious to, and hence laundering, upstream evil intent. And yet if the moment calls for it, they know the words needed to inject a sense of emotion. Not because they have emotion, but because they know the patsies they're talking to have emotion.

In this way, at least, they are like the masters they serve—they offer emotion remorselessly, as artful rhetorical flare, never feeling later consequence, just choosing words in the moment because, statistically, it seems the key to winning an argument. Move over Susan Collins, there's a new con game in town, and your role is now played by an “AI”.

 

---

Author’s Notes:

If you got value from this post, please “Share” it.

This post began as a comment I tried to write in response to the aforementioned conversation on LinkedIn.

The image generated with help from abacus.ai's ChatLLM (GPT‑5.3 Instant / GPT‑5.4), which did image generation with Nano Banana Pro, with light post-processing in Gimp to reduce the resulting image in size for faster web download..

The Tedious Pained

In a LinkedIn post recently, Ethan Mollick wrote, the following statement. And he's not the only one to have said things like this.

«I don’t think it is good for anybody that there is an emerging "anti-AI" group that is actually a coalition of many groups with different worries (jobs, kids using AI, slop, existential risk, the environment, industry concentration). Because of the diverse constituencies, this group may only be able agree on a full halt to AI as a remedy.

Not only is a halt to AI development or use unlikely, but it undermines the desire to make policies that channel AI to good uses or that mitigate specific harms. AI is a general purpose technology that will impact many aspects of society, work, and education, all of which will require their own consideration.»

I feel like this is lately a common refrain among some people, especially some of the technology professionals such as I cross paths with at LinkedIn, and I needed to respond to it, not just to him individually, but to the community of people who think such things.

This is the reply I wrote, lightly edited to suit the layout of this forum:

Accepting for conversation your notion of an emerging “broad coalition of the trodden upon,” perhaps instead of bemoaning that their common thought is to shut down “AI”, it'd behoove “AI” proponents to focus on not trodding upon so many people in so many ways?

Disdaining the already-disdained is a bad look.

How odd to suggest that because there are so many disparate kinds of injury, with so little commonality other than “AI” itself, it's reason to be dismissive rather than more aware.

“Get used to it,” in other domains of injustice, has not aged well.

Few question potential positives. Speak to the negatives so routinely trivialized.

Star Trek teaches us technology must move in lockstep with wisdom. Some want to race ahead to see where tech leads. It's already feeding power imbalance with no check in sight.

We were told automation would bring freedom and leisure time. But just a few are enriched. Many just scramble more.

Instead of shutting down “AI” maybe strong regulation, a serious automation tax, more unemployment benefits, reeducation help, student loan forgiveness, and UBI? But the already-disproportionately rich use wealth to oppose these.

See also my 2023 essay Technology's Ethical Two-Step.

Author’s Notes:

If you got value from this post, please “Share” it.

This is one of a series of transplanted articles from my my LinkedIn feed or my Mastodon feed. Putting them here reaches a different audience and allows me to more easily refer back to them by name. Apologies to anyone who reads my writing in more than one place and find this redundant.

In making the image, I used Gemini 3 Flash, Nano Banana, and GPT-4o at abacus.ai, with light post-processing in Gimp to reduce the resulting image in size for faster web download.

Unsupervised AI Children

Recent “AI” hype

Since the introduction of the Large Language Model (LLM), the pace of new tools and technologies has been breathtaking. Those who are not producing such tech are scrambling to figure out how to use it. Literally every day there's something new.

Against this backdrop, Google has recently announced a technology it calls AlphaEvolve, which it summarizes as “a Gemini-powered coding agent for designing advanced algorithms” According to one of its marketing pages:

“Today, we’re announcing AlphaEvolve, an evolutionary coding agent powered by large language models for general-purpose algorithm discovery and optimization. AlphaEvolve pairs the creative problem-solving capabilities of our Gemini models with automated evaluators that verify answers, and uses an evolutionary framework to improve upon the most promising ideas.»

Early Analysis

The effects of such new technologies are hard to predict, but let's start what's already been written.

In an article in ars technica, tech reporter Ryan Whitwam says of the tech:

«When you talk to Gemini, there is always a risk of hallucination, where the AI makes up details due to the non-deterministic nature of the underlying technology. AlphaEvolve uses an interesting approach to increase its accuracy when handling complex algorithmic problems.»

It's interesting to note that I found this commentary by Whitwam from AlphaEvolve's Wikipedia page, which had already re-summarized what he said as this (bold mine to establish a specific focus):

«its architecture allows it to evaluate code programmatically, reducing reliance on human input and mitigating risks such as hallucinations common in standard LLM outputs.»

Whitwam actually hadn't actually said “mitigating risks,” though he may have meant it. His more precise language, “improving accuracy” speaks to a much narrower goal of specific optimization of modeled algorithms, and not to the broader area of risk. These might seem the same, but I don't think they are.

To me—and I'm not a formal expert, just someone who's spent a lifetime thinking about computer tech ethics informally—risk modeling has to include a lot of other things, but most specifically questions of how well the chosen model really captures the real problem to be solved. LLMs give the stagecraft illusion of speaking fluidly about the world itself in natural language terms, and that creates all kinds of risks of simple misunderstanding between people because of the chosen language, as well as failures to capture all parts of the world in the model.

Old ideas dressed up in a new suit

In a post about this tech on LinkedIn, my very thoughtful and rigorously meticulous friend David Reed writes:

«30 years ago, there was a craze in computing about Evolutionary Algorithms. That is, codes that were generated by random modification of the source code structure and tested against an “environment” which was a validation test. It was a heuristic search over source code variations against a “quality” or “performance” measure. Nothing new here at all, IMO, except it is called “AI” now.»

I admit haven't looked at the tech in detail, but I trust Reed's assertion that the current interation of the tech is primarily less grandiose than Google's hype suggests—at least for now.

But that doesn't mean more isn't coming. And by more, I don't necessarily mean smarter. But I do mean that it will be irresistible for technologists to turn this tech upon itself and try exactly what Google sounds like it's wanting to claim here: that unsupervised evolutionary learning will soon mean “AI”—in the ‘person’ of LLMs—can think and evolve on their own.

Personally, I'm confused by why people even see it as a good goal, as I discussed in my essay Sentience Structure. You can read that essay if you want the detail, so I won't belabor that point here. I guess it comes down to some combination of a kind of euphoria that some people have over just doing something new combined with a serious commercial pressure to be the one who invents the next killer app.

I just hope it's not literally that—an app that's a killer.

Bootstrapping analysis by analogy

In areas of new thought, I reason by analogy to situations of similar structure in order to derive some sense of what to expect, by observing what happens in analogy space and then projecting back into the real world to what might happen with the analogously situated artifacts. Coincidentally, it's a technique I learned from a paper (MIT AIM-520) written by Pat Winston, head of the MIT AI lab back when I was studying and working there long ago — when what we called “AI” was something different entirely.

Survey of potential analogy spaces

Capitalism

I see capitalism as an optimization engine. But any optimization engine requires boundary conditions in order to not crank out nonsensical solutions. Optimization engines are not "smart" but they do a thing that can be a useful tool in achieving smart behavior.

Adam Smith, who some call the father of modern capitalism, suggested that if you want morality in capitalism, you must encode it in law, that the engine of capitalism will not find it on its own. He predicted that absent such encoding, capitalists would tend toward being tyrants.

Raising Children

Children are much smarter than some people give them credit for. We sometimes think of kids getting smarter with age or education, but really they gain knowledge and context and, eventually, we hope, empathy. Young children can do brilliant but horrifying things, things that might hurt themselves or others, things we might call sociopathic in adults, for lack of understanding of context and consequence. We try to watch over them as they grow up, helping them grow out of this.

It's why we try kids differently than adults sometimes in court. They may fail to understand the consequences of their actions.

Presuppositions

We in the general public, the existing and future customers of “AI” are being trained by use of tools like ChatGPT to think of an “AI” as something civil because the conversations we have with them are civil. But with this new tech, all bets are off. It's just going to want to find a shorter path to the goal.

LLM technology has no model of the world at all. It is able to parrot things, to summarize things, to recombine and reformat things, and a few other interesting tricks that combine to give some truly dazzling effects. But it does not know things. Still, for this discussion, let's even suspend disbelief and assume that there is some degree of modeling going on in this new chapter of “AI” if the system thinks it can improve its score.

Raising “AI” Children

Capitalism is an example of something that vaguely models the world by assigning dollar values to a great many things. But many find ourselves routinely frustrated by capitalism because it seems to behave sociopathically. Capitalists want to keep mining oil when it's clear that it is going to drive our species extinct, for example. But it's profitable. In other words, the model says this is a better score because the model is monetary. It doesn't measure safety, happiness (or cruelty), sustainability, or a host of other factors unless a dollar score is put on those. The outcome is brutal.

My 2009 essay Fiduciary Duty vs The Three Laws of Robotics discusses in detail why this behavior by corporations is not accidental. But the essence of it is that businesses do the same thing that sociopaths do: they operate without empathy, focusing single-mindedly on themselves and their profit. In people, we call that sociopathy. Since corporations are sometimes called “legal people,” I make the case in the essay that corporations are also “legal sociopaths.”

Young children growing up tend to be very self-focused, too. They can be cruel to one another in play, and grownups need to watch over them to make sure that appropriate boundaries are placed on them. A sense of ethics and personal responsibility does not come overnight, but a huge amount of energy goes into supervising kids before turning them loose on the world.

And so we come to AIs. There is no reason to suspect that they will perform any differently. They need these boundary conditions, these rules of manners and ethics, a sense of personal stake in the world, a sense of relation to others, a reason not to behave cruelly to people. The plan I'm hearing described, however, falls short of that. And that scares me.

I imagine they think this can come later. But this is part of the dance I have come to refer to as Technology's Ethical Two-Step. It has two parts. In the first part, ethics is seen as premature and gets delayed. In the second part, ethics is seen as too late to add retroactively. Some nations have done better than others at regulating emerging technology. The US is not a good example of that. Ethics is something that's seen as spoiling people's fun. Sadly, though, an absence of ethics can spoil more than that.

Intelligence vs Empathy

More intelligence does not imply more empathy. It doesn't even imply empathy at all.

Empathy is something you're wired for, or that you're taught. But “AI” is not wired for it and not taught it. As Adam Smith warned, we must build it in. We should not expect it to be discovered. We need to require it in law and then productively enforce that law, or we should not give it the benefit of the doubt.

Intelligence without empathy ends up just being oblivious, callous, cruel, sociopathic, evil. We need to build “AI” differently, or we need to be far more nervous and defensive about what we expect “AI” that is a product of self-directed learning to do.

Unsupervised AI Children—what could possibly go wrong?

The “AI” tech we are making right now are children, and the suggestion we're now seeing is that they be left unsupervised. That doesn't work for kids, but at least we don't give kids control of our critical systems. The urgency here is far greater because of the accelerated way that these things are finding themselves in mission-critical situations.

 

---

Author's Notes:

If you got value from this post, please “Share” it.

You may also enjoy these other essays by me on related topics:

• Sentience Structure
• Fiduciary Duty vs. The Three Laws of Robotics
• Technology's Ethical Two-Step

The graphic was created at abacus.ai using RouteLLM (which referred me to GPT-4.1) and rendered by GPT Image. I did post-processing in Gimp to add color and adjust brightness in places.

Must We Pretend?

An article at countercurrents.org said this recently:

«A new study has warned that if global temperatures rise more than 1.5°C, significant crop diversity could be lost in many regions»
—Global Warming and Food Security: The Impact on Crop Diversity

Are we not sufficiently at the 1.5°C mark that this dance in reporting is ludicrous?

I'm starting to perceive the weather/climate distinction less as a matter of scientific certainty and more as an excuse to delay action for a long time. Here that distinction seems to be actively working against the cause of human survival by delaying what seems a truly obvious conclusion, and in doing so giving cover to inaction.

We already have a many year trend that shows things getting pretty steadily worse year over year, with not much backsliding, so it's not like we realistically have to wait 10 years to see if this surpassing 1.5°C is going to magically go away on its own. Indeed, by the time we get that much confirmation, these effects we fear will have seriously clubbed us over the head for too long.

«“The top ten hottest years on record have happened in the last ten years, including 2024,” António Guterres said in his New Year message, stressing that humanity has “no time to lose.”»
—2024, Hottest Year on Record, Marks ‘Decade of Deadly Heat’

I keep seeing reports (several quoted by me here below) that we averaged above that in 2024, so I find this predication on a pipe dream highly misleading.

Even just wordings suggesting that the crossing of some discrete boundary will trigger an effect, but that not crossing it will not, is misleading. It's not like 1.49°C will leave us with no loss of diversity, but 1.51°C will hit us with all these effects.

What needs to be said more plainly is this:

Significant crop diversity is being ever more lost in real time now, and this loss is a result of global average temperatures that are dangerous and getting moreso. That they are a specific value on an instantaneous or rolling average basis gives credibility and texture to this qualitative claim, but no comfort should be drawn from almost-ness nor from theoretical clains that action could yet pull us back from a precipice that there is not similarly substantiated qualitative reason to believe we are politically poised to make.

Science reporting does this kind of thing a lot. Someone will get funding to test whether humans need air to breathe but some accident of how the experiments are set up will find that only pregnant women under 30 were available for testing so the report will be a very specific about that and news reports will end up saying "new report proves pregnant women under 30 need air to breathe", which doesn't really tell the public the thing that the study really meant to report. Climate reporting is full of similarly overly specific claims that allow the public to dismiss the significance of what's really going on. People writing scientific reports need to be conscious of the fact that the reporting will be done in that way and that public inaction will be a direct result of such narrow reporting.

In the three reports that I quote below, the Berkeley report at least takes the time to say "recent warming trends and the lack of adequate mitigation measures make it clear that the 1.5 °C goal will not be met." We need more plain wordings like this, and even this needs to have been more prominently placed.

There is a conspiracy, intentional or not, between the writers of reports and the writers of articles. The article writer wants to quote the report, but the report wants to say something that has such technical accuracy that it will be misleading when quoted by someone writing articles. Some may say it's not an active conspiracy, just a negative synergy, but the effect is the same. Each party acts as if it is being conservative and careful, but the foreseeable combination of the two parts is anything but conservative or careful.

References
(bold added here for emphasis)

«The global annual average for 2024 in our dataset is estimated as 1.62 ± 0.06 °C (2.91 ± 0.11 °F) above the average during the period 1850 to 1900, which is traditionally used a reference for the pre-industrial period. […] A goal of keeping global warming to no more than 1.5 °C (2.7 °F) above pre-industrial has been an intense focus of international attention. This goal is defined based on multi-decadal averages, and so a single year above 1.5 °C (2.7 °F) does not directly constitute a failure. However, recent warming trends and the lack of adequate mitigation measures make it clear that the 1.5 °C goal will not be met. The long-term average of global temperature is likely to effectively cross the 1.5 °C (2.7 °F) threshold in the next 5-10 years. While the 1.5 °C goal will not be met, urgent action is still needed to limit man-made climate change.»
—Global Temperature Report for 2024 (Berkeley Earth)

«The global average surface temperature was 1.55 °C (with a margin of uncertainty of ± 0.13 °C) above the 1850-1900 average, according to WMO’s consolidated analysis of the six datasets. This means that we have likely just experienced the first calendar year with a global mean temperature of more than 1.5°C above the 1850-1900 average.»
— WMO confirms 2024 as warmest year on record at about 1.55°C above pre-industrial level

«NASA scientists further estimate Earth in 2024 was about 2.65 degrees Fahrenheit (1.47 degrees Celsius) warmer than the mid-19th century average (1850-1900). For more than half of 2024, average temperatures were more than 1.5 degrees Celsius above the baseline, and the annual average, with mathematical uncertainties, may have exceeded the level for the first time.»
— Temperatures Rising: NASA Confirms 2024 Warmest Year on Record

Author's Notes:

If you got value from this post, please “Share” it.

This grew out of an essay I posted at Mastodon, and a haiku (senryu) that I later wrote as a way to distill out some key points.