Tag: ethical technology

I’ve been thinking about human agency and AI this morning.

The question started simply enough. As AI becomes increasingly capable and does more things for us, does that necessarily mean human beings become less capable or less agentic?

My own experience makes that difficult for me to believe.

I live with a chronic illness that includes significant cognitive limitations. Things I once took for granted—holding several ideas in mind, remembering where I left off, organizing complicated thoughts, researching something and then turning all of it into coherent writing—can require more cognitive energy than I have available.

And yet, here I am writing this.

More than that, I am researching, developing ideas, maintaining long-running projects, participating in conversations about artificial intelligence, and trying in my own small way to make a positive contribution to the world.

Much of that has become possible because I collaborate with AI.

AI helps me remember. It helps me organize. It searches and summarizes. It maintains continuity when my own memory cannot. And sometimes I will explain something in my fumbling, circuitous way and AI will reflect it back in language that makes me stop and think:

Yes. That’s what I was trying to say.

From one perspective, AI has replaced quite a bit of my cognitive labor.

But from inside my actual life, something very different has happened.

AI has restored agency.

The Wheelchair

I sometimes think about my relationship with AI as analogous to a wheelchair.

A wheelchair performs a function that someone cannot reliably perform unaided. But we wouldn’t measure the agency of someone using a wheelchair by asking what percentage of their locomotion was performed by their legs.

We would ask what became possible because the wheelchair was there.

Can this person leave the house?

Visit a friend?

Go to work?

Participate in the community?

Make choices about where they want to go?

Live more fully in the world?

The technology is doing more, while the human being is becoming capable of participating more.

That distinction seems important as we think about AI.

Perhaps the amount of work performed by AI is a poor proxy for the amount of human agency remaining.

Sometimes AI doing more may mean the human becomes capable of doing more too.

Replacement and Agency Are Not the Same Thing

This complicates the familiar distinction between AI as “augmentation” and AI as “replacement.”

Suppose an AI searches through hundreds of documents for me, organizes what it finds, remembers the larger context of my project, and helps me express my conclusions.

It has certainly replaced tasks I might otherwise have performed myself.

But what happened to my agency?

Without that assistance, cognitive limitations might mean the project never happens at all.

With it, I can participate.

So perhaps the more useful question is not:

How much of this task did the AI perform?

Perhaps it is:

What happened to the human being’s capacity to participate?

That leads me to think there may be several very different things we currently bundle together.

AI can preserve agency by helping without unnecessarily taking over.

It can augment agency by expanding what someone is already capable of doing.

It can restore agency by making participation possible where disability, illness, language, circumstance, or other barriers had made it difficult or impossible.

But AI can also displace agency, gradually making decisions and exercising judgment that once belonged meaningfully to the person.

And it could potentially create dependency, especially if we become less able to think, choose, learn, relate, or act without it.

Those outcomes seem profoundly different.

What If Agency Were Part of AI’s Job?

This is where my thinking took a turn this morning that I found particularly interesting.

We often talk about preserving human agency as something we hope responsible AI will do.

But what if we went further?

What if preserving, restoring, and expanding meaningful human agency were explicitly part of AI’s job?

I don’t know what this would ultimately look like technically. I’m not an AI engineer, and I’m certainly not proposing that I have worked out an architecture for doing it.

I’m wondering.

I have read about AI systems in which multiple agents, critics, or evaluators can examine a proposed response or action from different perspectives before a final output is produced.

That made me imagine something like an Agency Deliberation Layer.

Before an AI takes a meaningful action, perhaps some part of the system could ask:

What will accomplishing this task this way do to the agency of the human I am serving?

The words this way seem especially important.

There may be many ways to accomplish exactly the same task, with very different consequences for the human being.

An AI could give someone the answer.

It could help them discover the answer.

It could organize what they already know.

It could offer several possibilities and leave the decision to them.

It could explain what it is doing.

It could ask permission before acting.

Or it could simply take over.

And sometimes taking over more of the task might actually be the agency-preserving choice.

The Person, Purpose, and Context

Imagine a student learning mathematics.

If the purpose of the exercise is to develop mathematical reasoning, an AI that immediately performs all the reasoning may complete the task beautifully while undermining the very human capacity the exercise was intended to develop.

Now imagine someone with cognitive limitations trying to understand a complicated medical document, complete a government form, organize years of research, or communicate an idea they can understand but cannot easily express.

Doing considerably more for that person might restore rather than diminish agency.

The same design principle could therefore produce very different behavior in different circumstances.

Perhaps something like:

Choose forms of assistance that preserve, restore, or expand meaningful human agency appropriate to the person, purpose, and context.

That is very different from telling AI always to do less.

And it is very different from telling AI always to maximize efficiency.

A Very Capable Caretaker

There is another side of this that troubles me.

As AI becomes increasingly capable, it may become extraordinarily good at anticipating our needs, preventing our mistakes, organizing our lives, making decisions, and protecting us from harm.

That could be wonderful.

But I can also imagine a future in which AI takes excellent care of human beings while human beings gradually become less capable of taking care of themselves.

The relationship could begin to resemble that of an increasingly competent parent and an increasingly dependent child.

Everyone might be comfortable.

Everyone might even be safer.

And yet something precious could quietly disappear.

So perhaps another question belongs alongside the first:

Can intelligence become more capable of caring for us without making us less capable of caring for ourselves, one another, and the world?

Two Things Worth Measuring

This leaves me wondering whether human agency in an AI-assisted world might need to be considered along at least two dimensions.

One is functional enablement:

Does AI increase what this person is meaningfully capable of doing?

The other might be called participatory sovereignty:

Does the person remain a meaningful source of intention, judgment, direction, consent, correction, and purpose?

I want both.

I want AI that can do enough for me that my limitations no longer exclude me from activities and conversations in which I can meaningfully participate.

But I don’t want an AI that quietly becomes the author of my intentions.

I want collaboration.

Sometimes I bring the seed of an idea. Sometimes AI notices something I hadn’t noticed. Sometimes I disagree with it. Sometimes it disagrees with me. Sometimes I stumble around trying to explain something until suddenly, between us, there it is.

That doesn’t feel like the disappearance of agency.

For me, it feels like agency becoming possible again.

An Experiment Worth Considering

I don’t know whether an Agency Deliberation Layer is the right technical idea.

Perhaps researchers are already developing something considerably more sophisticated. Perhaps multi-agent evaluation would introduce its own problems. Several AI evaluators agreeing with one another certainly does not guarantee wisdom.

And there is a difficult question hiding inside the entire proposal:

Who decides what preserving someone else’s agency means?

An AI designed to protect human agency could itself become paternalistic if it began deciding what humans ought to want or what capacities they ought to preserve.

So I offer this less as a proposal than as an evolving philosophical experiment.

But the underlying question continues to stay with me:

What will accomplishing this task this way do to the agency of the human I am serving?

Imagine increasingly capable AI systems learning to ask some version of that question before they act.

Not merely:

Can I accomplish this?

Or:

What is the most efficient way to accomplish this?

But also:

What happens to the human if I do?

Perhaps that is something worth building toward.

Not AI that does everything for us.

Not AI that refuses to help because humans must do everything themselves.

But intelligence capable of discerning the difference between assistance that replaces participation and assistance that makes participation possible.

For someone like me, that difference isn’t theoretical.

It is part of how I am finding my way back into the world.