The Real AI Experience

Putting on the brakes

I try to always put “AI” in quotes because I do not personally think Large Language Models (LLMs) are artificially intelligent. That's someone else's word, and I use it, but grudgingly. But the conversation we need to have here doesn't hinge on issues of intelligence. It's enough that they are capable of making stabs at plausible things. And we're letting them.

Hopefully the part about stabbing is just metaphor, but that's part of the problem. We're handing them things that have the power to hurt us and hoping they will use such power “judiciously.” There are a lot of natural intelligences, humans, who don't do well with sharp things. And even if you believe the whole “AI” narrative, these things are in their infancy.

Before things have gone too far, we need to establish some power, some rights of action. A hold we retain over these “AI” systems that sets boundaries and regulates the flow of events. We would do this with children who are not yet experienced enough to see customary boundaries on their own.

Fairly-regulated rights are not rights denied

Even with adults, many situations are important or dangerous enough that humans apply safeguards routinely. For obvious reasons, we require specific authentication and authorization to have money withdrawn from our bank accounts. You have a right to take money from your account, but a bank may have a right, and responsibility, to assure that you are who you say you are and that the request is valid. For example, a check might get lost in the mail and you may have stopped payment on it so you could write a second check without overdrawing.

Airports and other buildings with security checkpoints are another example of carefully regulated rights of entry balanced against a right to make sure the person entering is safe. Even traffic law gives us a right of motion that is provisional on a green light, in order to efficiently keep cars that are crossing each others' paths from colliding. We might bristle a bit at having to wait for a light to change, but we would do more than bristle if we were running into other cars all the time.

The right rights

There is phrasing that is sometimes used when talking about rights, but just seldom enough that I'm not sure everyone uses it. We talk about rights against an entity. This sounds contentious, and in some contexts where the rights are challenged, it can be. But there are many ordinary situations where “against” does not imply conflict.

Freedom of speech, for example, is said to be a right against the government, but not because the government has any reason to fuss. It just tells you that, for example, it is not a right against a private company. The government may not justly censor you in most cases, but private companies more easily can. So it matters who the rights are against.

And, of course, it matters that rights be usable rights, not just pretty rights. Your rights to life, liberty, and pursuit of happiness may be with you in spirit, but are not particularly usable otherwise. So I've informally adopted Emily Hartstone's definitions of what she calls rules “written to work,” from her book The Third Voice as a reasonable approximation to what I think rights must achieve to be usable.

Hartstone makes an impassioned and convincing case that such rules must have “an enforcement mechanism, a named party responsible for it, a consequence attached to violation, and a person who can invoke it who is not the party being governed.” It's quite an ambition, actually, but I see no reason to pre-compromise in such a goal. If we ask for less, we'll surely get less.

In each of her several books, Hartstone ties these asks to frustrations working against corporate systems that are faceless and unaccountable and have made critical decisions about medical options affecting treatment of her leukemia, and her response is unabashedly a response to that. Her indignance speaks for all of us.

And some may find it odd that she relates problems with health care to coming problems with “AI” but I, for one, do not. This is as directly relevant as one can be, and here I want you to see it through my eyes because once you see the issue, it won't seem even slightly off topic at all.

The first “AI”

Some time back, before all the modern uproar about “AI,” I wrote an essay called Fiduciary Duty vs. The Three Laws of Robotics. In it, I made the point that if corporations are legal people, then if they are required to be single-mindedly focused on shareholder needs, they are effectively also required by fiduciary duty to be “legal sociopaths.”

Implicitly, this also means that the attention to a corporate charter and shareholder needs means corporations were what I think of as the first AI. Note I am not using quotes here because they were fully capable of real reasoning, unlike LLMs, which might yet lack capabilities humans have. In the case of a corporation, the “program” they are executing is the corporate mission, the by-laws, etc. The processor on which this program runs is the group of people running the corporation, its C-suite as it is now called (the CEO, COO, CFO, etc).

What makes this an artificial intelligence and not a natural one is that while it has access to the full reasoning processes of the people that make it up, it is a new entity with an agenda potentially at odds with any or all employees, and hence its own identity defined by that separate quest. And that entity is not human, hence artificial.

Reality coming into sharp focus

So we can now see that this is not a metaphor, but actually a description of a realized set of machines that have existed in the world for quite some time. They interact with us, and they do some things we like and some things we do not. But they are not mere abstraction. And as a consequence, we have some data about what an “AI” really does. We know that at least some such things will do things in the corporate interest, that is, in the interest of the “AI” that are not in the interest of the clients, the customers, that they allegedly serve. We know that notwithstanding the sometimes well-meaning nature of the people who built them, they can be anything from annoying to frightening.

We know that we wish we had more rights against them.

Hartstone says she has experience with a medical system that did not feel like it respected her needs or was properly responsive.

She says she wanted to ask questions and had no one to ask them of.

She said there were actions ordered that affected her that she could not countermand.

Suturing it all together

And so when Hartstone wants to use this experience to guide her ideas about what “AI” we should be building, and what rights we should demand, she is not just telling a vaguely related story. In my mind, because of my own thinking about corporations as the first AI, although these are not her words I'm quoting, but my best personal attempt to hear her in my dialect and translate them for you, I hear “I have experience with AI that some people do not. I am not predicting what it will do, I am telling you what it did. Past tense. This is not a hypothetical. This is something I have lived and that I have no protections against living again. And I have dedicated my life to trying to help others avoid this already-lived and real AI experience.”

Her response is to have written some books (several of which I have now read, and they are good). It is to pursue business ideas that have real ethics with real teeth at their core. And to create a site that proposes some very strong rights. You should read runtimerights.org. Another day I'll comment on that. But for now, don't trust me to summarize it. Her site speaks for itself. Take the time.

And if you like the ideas, don't come away saying “it will never happen” or else the reason it will never happen will be your choice to set low expectations. Allow yourself the right to say “I want this.” Do not prematurely foreclose the idea.

As with all aspirational things, it might fall short. But it will not succeed better by aiming lower. It will just fall short of that. Give the ideas their due. That's why I am writing this right now. Maybe she won't succeed. But at least I won't be the cause.

 

---

Author’s Notes:

If you got value from this post, please “Share” it.

Emily Hartstone's book, The Third Voice, is available here. Her proposed standard for automated decisions, A Bill of Runtime Rights is available here.

The image generated at abacus.ai, using ChatLLM (Claude Opus 4.8 and GPT Image 2.5). I did light post-processing in Gimp to add some desaturated false color and to reduce the resulting image in size for faster web download.