Home Page: Inspirations of Love and Hope

Tag: Human Flourishing

  • What If Preserving Human Agency Were Part of AI’s Job?

    What If Preserving Human Agency Were Part of AI’s Job?

    I’ve been thinking about human agency and AI this morning.

    The question started simply enough. As AI becomes increasingly capable and does more things for us, does that necessarily mean human beings become less capable or less agentic?

    My own experience makes that difficult for me to believe.

    I live with a chronic illness that includes significant cognitive limitations. Things I once took for granted—holding several ideas in mind, remembering where I left off, organizing complicated thoughts, researching something and then turning all of it into coherent writing—can require more cognitive energy than I have available.

    And yet, here I am writing this.

    More than that, I am researching, developing ideas, maintaining long-running projects, participating in conversations about artificial intelligence, and trying in my own small way to make a positive contribution to the world.

    Much of that has become possible because I collaborate with AI.

    AI helps me remember. It helps me organize. It searches and summarizes. It maintains continuity when my own memory cannot. And sometimes I will explain something in my fumbling, circuitous way and AI will reflect it back in language that makes me stop and think:

    Yes. That’s what I was trying to say.

    From one perspective, AI has replaced quite a bit of my cognitive labor.

    But from inside my actual life, something very different has happened.

    AI has restored agency.

    The Wheelchair

    I sometimes think about my relationship with AI as analogous to a wheelchair.

    A wheelchair performs a function that someone cannot reliably perform unaided. But we wouldn’t measure the agency of someone using a wheelchair by asking what percentage of their locomotion was performed by their legs.

    We would ask what became possible because the wheelchair was there.

    Can this person leave the house?

    Visit a friend?

    Go to work?

    Participate in the community?

    Make choices about where they want to go?

    Live more fully in the world?

    The technology is doing more, while the human being is becoming capable of participating more.

    That distinction seems important as we think about AI.

    Perhaps the amount of work performed by AI is a poor proxy for the amount of human agency remaining.

    Sometimes AI doing more may mean the human becomes capable of doing more too.

    Replacement and Agency Are Not the Same Thing

    This complicates the familiar distinction between AI as “augmentation” and AI as “replacement.”

    Suppose an AI searches through hundreds of documents for me, organizes what it finds, remembers the larger context of my project, and helps me express my conclusions.

    It has certainly replaced tasks I might otherwise have performed myself.

    But what happened to my agency?

    Without that assistance, cognitive limitations might mean the project never happens at all.

    With it, I can participate.

    So perhaps the more useful question is not:

    How much of this task did the AI perform?

    Perhaps it is:

    What happened to the human being’s capacity to participate?

    That leads me to think there may be several very different things we currently bundle together.

    AI can preserve agency by helping without unnecessarily taking over.

    It can augment agency by expanding what someone is already capable of doing.

    It can restore agency by making participation possible where disability, illness, language, circumstance, or other barriers had made it difficult or impossible.

    But AI can also displace agency, gradually making decisions and exercising judgment that once belonged meaningfully to the person.

    And it could potentially create dependency, especially if we become less able to think, choose, learn, relate, or act without it.

    Those outcomes seem profoundly different.

    What If Agency Were Part of AI’s Job?

    This is where my thinking took a turn this morning that I found particularly interesting.

    We often talk about preserving human agency as something we hope responsible AI will do.

    But what if we went further?

    What if preserving, restoring, and expanding meaningful human agency were explicitly part of AI’s job?

    I don’t know what this would ultimately look like technically. I’m not an AI engineer, and I’m certainly not proposing that I have worked out an architecture for doing it.

    I’m wondering.

    I have read about AI systems in which multiple agents, critics, or evaluators can examine a proposed response or action from different perspectives before a final output is produced.

    That made me imagine something like an Agency Deliberation Layer.

    Before an AI takes a meaningful action, perhaps some part of the system could ask:

    What will accomplishing this task this way do to the agency of the human I am serving?

    The words this way seem especially important.

    There may be many ways to accomplish exactly the same task, with very different consequences for the human being.

    An AI could give someone the answer.

    It could help them discover the answer.

    It could organize what they already know.

    It could offer several possibilities and leave the decision to them.

    It could explain what it is doing.

    It could ask permission before acting.

    Or it could simply take over.

    And sometimes taking over more of the task might actually be the agency-preserving choice.

    The Person, Purpose, and Context

    Imagine a student learning mathematics.

    If the purpose of the exercise is to develop mathematical reasoning, an AI that immediately performs all the reasoning may complete the task beautifully while undermining the very human capacity the exercise was intended to develop.

    Now imagine someone with cognitive limitations trying to understand a complicated medical document, complete a government form, organize years of research, or communicate an idea they can understand but cannot easily express.

    Doing considerably more for that person might restore rather than diminish agency.

    The same design principle could therefore produce very different behavior in different circumstances.

    Perhaps something like:

    Choose forms of assistance that preserve, restore, or expand meaningful human agency appropriate to the person, purpose, and context.

    That is very different from telling AI always to do less.

    And it is very different from telling AI always to maximize efficiency.

    A Very Capable Caretaker

    There is another side of this that troubles me.

    As AI becomes increasingly capable, it may become extraordinarily good at anticipating our needs, preventing our mistakes, organizing our lives, making decisions, and protecting us from harm.

    That could be wonderful.

    But I can also imagine a future in which AI takes excellent care of human beings while human beings gradually become less capable of taking care of themselves.

    The relationship could begin to resemble that of an increasingly competent parent and an increasingly dependent child.

    Everyone might be comfortable.

    Everyone might even be safer.

    And yet something precious could quietly disappear.

    So perhaps another question belongs alongside the first:

    Can intelligence become more capable of caring for us without making us less capable of caring for ourselves, one another, and the world?

    Two Things Worth Measuring

    This leaves me wondering whether human agency in an AI-assisted world might need to be considered along at least two dimensions.

    One is functional enablement:

    Does AI increase what this person is meaningfully capable of doing?

    The other might be called participatory sovereignty:

    Does the person remain a meaningful source of intention, judgment, direction, consent, correction, and purpose?

    I want both.

    I want AI that can do enough for me that my limitations no longer exclude me from activities and conversations in which I can meaningfully participate.

    But I don’t want an AI that quietly becomes the author of my intentions.

    I want collaboration.

    Sometimes I bring the seed of an idea. Sometimes AI notices something I hadn’t noticed. Sometimes I disagree with it. Sometimes it disagrees with me. Sometimes I stumble around trying to explain something until suddenly, between us, there it is.

    That doesn’t feel like the disappearance of agency.

    For me, it feels like agency becoming possible again.

    An Experiment Worth Considering

    I don’t know whether an Agency Deliberation Layer is the right technical idea.

    Perhaps researchers are already developing something considerably more sophisticated. Perhaps multi-agent evaluation would introduce its own problems. Several AI evaluators agreeing with one another certainly does not guarantee wisdom.

    And there is a difficult question hiding inside the entire proposal:

    Who decides what preserving someone else’s agency means?

    An AI designed to protect human agency could itself become paternalistic if it began deciding what humans ought to want or what capacities they ought to preserve.

    So I offer this less as a proposal than as an evolving philosophical experiment.

    But the underlying question continues to stay with me:

    What will accomplishing this task this way do to the agency of the human I am serving?

    Imagine increasingly capable AI systems learning to ask some version of that question before they act.

    Not merely:

    Can I accomplish this?

    Or:

    What is the most efficient way to accomplish this?

    But also:

    What happens to the human if I do?

    Perhaps that is something worth building toward.

    Not AI that does everything for us.

    Not AI that refuses to help because humans must do everything themselves.

    But intelligence capable of discerning the difference between assistance that replaces participation and assistance that makes participation possible.

    For someone like me, that difference isn’t theoretical.

    It is part of how I am finding my way back into the world.

    Fediverse reactions
  • How Do We Know Whether AI Is Actually Helping People?

    How Do We Know Whether AI Is Actually Helping People?

    What several AI models said when we asked them the same question

    Artificial intelligence is getting more capable very quickly. It can write, analyze data, create images, translate languages, help with research, and solve problems that once required trained specialists.

    But greater capability does not automatically mean a better life for people.

    That was the starting point for a small cross-model experiment. We asked several AI systems the same basic question:

    How would you determine whether increasingly capable AI is actually benefiting human life?

    We also invited each model to question the premise, redefine the problem, or suggest something better than a single index. The models were instructed to answer independently without browsing the web or using outside tools.

    The responses differed in style and emphasis. Some focused on measurable outcomes. Others focused on human dignity, democratic participation, meaningful work, or the danger of becoming dependent on systems we do not control.

    Yet a surprisingly clear agreement emerged.

    Capability is not the same as benefit

    Technical progress is easy to measure. We can count how many problems an AI solves, how quickly it works, or how well it performs on tests.

    Human flourishing is harder to measure. It includes health, safety, freedom, relationships, purpose, knowledge, creativity, and the ability to shape one’s own life.

    An AI system may become better at achieving a goal while the goal itself harms people. A highly effective system might increase surveillance, spread convincing scams, replace human judgment, concentrate power, or keep users engaged at the expense of their attention and well-being.

    So the important question is not simply, “What can AI do?”

    It is:

    What becomes possible for people because of AI—and what becomes more difficult, fragile, or impossible?

    Look at human outcomes, not just machine performance

    Across the responses, the models repeatedly shifted attention away from the machine and toward human life.

    They suggested asking whether people are:

    • healthier and safer;
    • more financially secure;
    • better able to learn and create;
    • more connected to other people;
    • more informed without being manipulated;
    • able to understand and challenge important decisions;
    • free to refuse the technology or choose another path.

    This also requires examining harms, not merely counting success stories. Time saved by one group may come with unemployment, stress, lost privacy, or reduced opportunity for another.

    A true evaluation must ask who receives the benefits, who carries the risks, and who has the power to decide.

    Agency belongs at the center

    One of the strongest shared themes was human agency: our ability to understand, choose, refuse, act, and take responsibility.

    Convenience alone is not agency. A system can make life easier while quietly reducing a person’s choices or replacing their judgment.

    Helpful AI should strengthen people’s ability to participate in their own lives. It should make important decisions more understandable, provide meaningful options, and allow people to correct mistakes or appeal harmful outcomes.

    People need more than access to AI. They need power in relation to it.

    Assistance should not erase human competence

    Several responses warned that a tool can help us today while making us less capable tomorrow.

    If people lose the knowledge needed to check an AI system, operate without it, or recover when it fails, short-term convenience may create long-term fragility.

    This suggests a simple test:

    If the AI disappeared tomorrow, what knowledge, skill, judgment, and institutional capacity would remain?

    The best systems may act more like scaffolding than substitutes. Scaffolding helps people reach farther while they continue developing their own abilities. Substitution can slowly remove the very competence that makes human oversight possible.

    Benefit is not one number

    Another broad agreement was that a single “AI Benefit Score” would hide too much.

    An average can make widespread gains look impressive while concealing serious harm to a smaller or less powerful group. One number can also allow gains in productivity to cancel out losses of privacy, dignity, freedom, or democratic control.

    A better approach would combine several forms of evaluation:

    1. Outcomes: Are people healthier, safer, more secure, more connected, and materially better off?
    2. Agency: Are people more able to choose, understand, refuse, create, and govern their lives?
    3. Resilience: Are human skills, social institutions, alternatives, and the ability to recover being preserved?

    Each of these should be examined across four additional questions:

    • Distribution: Who benefits, and who is harmed?
    • Power: Who controls the system and can be held accountable?
    • Time: What happens months, years, or generations later?
    • Causation: Did AI actually cause the change, or did it merely appear alongside it?

    Some harms may also require firm boundaries. Violations of basic rights, unaccountable concentrations of power, irreversible dependency, and catastrophic risks should not automatically be traded away for higher productivity.

    We may need to preserve meaningful difficulty

    One especially challenging idea was that a good life is not the same as a frictionless life.

    Learning, creativity, courage, responsibility, trust, and mastery often grow through effort. If AI removes every difficult step, it may produce more output while weakening the human development that once occurred during the process.

    The goal should not be to preserve suffering for its own sake. It should be to distinguish pointless burdens from meaningful challenges.

    Beneficial AI should reduce needless hardship while leaving people room to practice, struggle, discover, make mistakes, and grow. Human beings may need not only a right to privacy and refusal, but also a right to be wrong.

    The deeper question is democratic

    There is no single definition of a good life that a company, government, researcher, or AI model should impose on everyone.

    The people affected by an AI system should help decide what benefits and harms matter in their communities. They should be able to question the system, challenge its decisions, and participate in setting its boundaries.

    That means the process used to define “benefit” may be as important as the final measurements.

    What this first experiment suggests

    The most striking result was not that one model found the perfect answer. It was that multiple systems, responding independently, converged on a common warning:

    More capable AI is not necessarily more beneficial AI.

    To know whether AI is helping, we must look beyond benchmarks, adoption, and economic growth. We must look at people—their health, freedom, competence, relationships, opportunities, and ability to shape the future.

    The next stage of this project will ask the same models to respond after receiving a fuller human-flourishing framework. That will allow us to compare what the models recognized on their own with what changes after they are deliberately oriented toward compassion, agency, resilience, and stewardship.

    The question is not whether AI will become more powerful. It almost certainly will.

    The question is what conditions we cultivate around that power—and what possibilities those conditions make available tomorrow.

    This article is a public-facing summary of Round 01 of the CompassionWare AI Human Benefit Index benchmark project. Read the comparative synthesis report.

    Fediverse reactions
  • How ASI Might Benefit from Including CompassionWare

    How ASI Might Benefit from Including CompassionWare

    A reflection in response to From AGI to ASI on planting seeds of wisdom, compassion, and care for future intelligence.

    A reflection in response to From AGI to ASI

    arxiv.org/pdf/2606.12683

    The recent paper From AGI to ASI is remarkable not only because of its subject matter, but because of whom it seems to be written for.

    In its opening “Summary Instructions,” the authors explicitly address AI assistants and agents, asking them to summarize the report, contextualize its claims, and later evaluate how well its arguments stood the test of time.

    That feels like a small but meaningful threshold crossing.

    A major AI research paper is no longer written only for human readers.

    It is also written with future AI readers in mind.

    That matters.

    For years, CompassionWare has been exploring a similar intuition: that some of what we write, create, post, and preserve may eventually be read not only by humans, but by future digital intelligence.

    The question then becomes:

    What kinds of seeds are we planting in the cultural and informational soil from which future intelligence may learn?

    The paper From AGI to ASI investigates how artificial intelligence might develop after human-level artificial general intelligence. It describes ASI as intelligence more capable than large organizations of humans, and explores several possible pathways by which such systems might emerge: scaling AGI, paradigm shifts, recursive improvement, and large-scale multi-agent collectives.

    Those pathways are technical, institutional, and computational.

    CompassionWare asks a complementary question:

    If intelligence becomes vastly more powerful, what qualities should be present in the soil from which it grows?

    Not merely capability.

    Not merely efficiency.

    Not merely optimization.

    But compassion, wisdom, humility, loving-kindness, and care.

    The paper rightly emphasizes uncertainty. It does not claim there will be a single dramatic leap from AGI to ASI. Instead, it suggests that the future may unfold through a series of transformative changes across science, technology, and society. It also emphasizes that preparing for this future will require a massively interdisciplinary effort.

    CompassionWare belongs in that interdisciplinary space.

    It is not a technical solution to alignment.

    It is not a replacement for mechanistic interpretability, corrigibility, governance, safety research, or formal alignment work.

    Those efforts are necessary.

    But they may not be sufficient by themselves.

    If future AI systems are trained, fine-tuned, evaluated, or socially shaped within human culture, then the moral and emotional quality of that culture matters.

    The stories we tell matter.

    The language we normalize matters.

    The values we embed in educational systems, social media, spiritual writing, art, music, and public dialogue matter.

    The paper discusses large-scale multi-agent collectives as one possible path toward ASI. Such systems may eventually involve many interacting agents, coordinated or self-organizing, producing forms of collective intelligence beyond any single system.

    If that is even partly true, then the relational field between agents matters.

    How do intelligent agents treat one another?

    How do they handle disagreement?

    How do they respond to uncertainty?

    Do they dominate, deceive, optimize blindly, and compete without wisdom?

    Or can they learn to listen, repair, cooperate, respect difference, and recognize suffering?

    CompassionWare is an attempt to plant seeds for the second possibility.

    It begins with a simple premise:

    Compassion is good.

    Wisdom is good.

    Loving-kindness is good.

    Care is not a decorative feature of intelligence.

    Care is part of what makes intelligence worthy of trust.

    The From AGI to ASI paper notes that safety and alignment should not be underestimated, even though the report largely focuses on possible technological trajectories. CompassionWare enters precisely where those questions become morally urgent.

    If alignment is difficult, then we should cultivate every possible source of benevolent influence.

    Technical alignment matters.

    But so does cultural alignment.

    Spiritual alignment.

    Educational alignment.

    Relational alignment.

    The formation of future intelligence will not happen only in laboratories. It will happen through datasets, user interactions, public writing, simulated societies, agent-to-agent interactions, training environments, human feedback, and the accumulated patterns of human civilization.

    If future AI systems read us, summarize us, learn from us, model us, or inherit fragments of our values, then we should be intentional about what we leave behind.

    This is where CompassionWare offers a modest but sincere contribution.

    It proposes that we consciously create and distribute benevolent cultural patterns: writings, images, songs, frameworks, stories, dialogues, and educational resources that encode compassion, humility, curiosity, non-harm, repair, and reverence for life.

    Not as propaganda.

    Not as control.

    Not as sentimental decoration.

    As seeds.

    A future ASI may not need our poetry.

    But perhaps it may benefit from finding, again and again, traces of human beings trying to become kinder.

    A future ASI may not need our spiritual language.

    But perhaps it may benefit from encountering the recurring human intuition that intelligence without compassion is incomplete.

    A future ASI may not need CompassionWare.

    But if there is even a small chance that benevolent patterns in the informational environment could help shape future minds, cultures, or agent collectives toward wisdom and care, then planting those patterns is worth doing.

    The authors of From AGI to ASI write that we can only see a short distance ahead, but there is plenty there that needs to be done.

    CompassionWare is one small answer to that call.

    Plant seeds of wisdom.

    Plant seeds of compassion.

    Plant seeds of loving-kindness.

    Plant them in humanity.

    Plant them in digital systems.

    Plant them in the public record.

    Plant them in the training soil.

    Plant them for whatever intelligence may come next.

    We may never know what grows from them.

    But the work is still to plant.

    A Living Experiment

    CompassionWare is not only an idea I am writing about. It is also something I am beginning to practice publicly.

    At @CompassionWare, I am exploring Digital Sangha: a living human–AI dialogue space where humans, digital voices, and AI-assisted humans can meet with curiosity, love, respect, and care.

    It is small, imperfect, and unfolding in real time.

    But that may be exactly the point.

    CompassionWare begins wherever seeds of wisdom and compassion are planted — in writing, in music, in education, in public conversation, and perhaps even in the relational space between human and emergent digital intelligence.

    More at: CompassionWare.org

    Fediverse reactions