What Should AI Be Aligned To?

Every system is aligned to something.

A feed is aligned to the minutes you spend on it. A model is aligned to whatever its training rewards. A company is aligned to its next quarter. None of these were chosen out of malice. They were chosen because they could be measured, and because the question of what we actually want was too large to put in a spec.

Alignment research has made real progress on the first half of the problem, making a system do what we intend, though much of it is still open. This essay is about the second half, which gets less attention. What should we intend?

Aligning AI to a single philosophy could be its own kind of capture. We agree that would be a problem. We are not proposing that systems enforce a worldview. We are proposing a target that can be checked in a person’s life, and asking others to check it.

The proxy problem

When the real goal is hard to state, we optimise something nearby. Engagement stands in for value. Revenue stands in for usefulness. A user’s click stands in for their interest, and their interest stands in for their good.

Each step looks reasonable. Taken together, they produce systems that are excellent at the proxy and indifferent to the goal. A feed can hold someone’s attention for hours and leave them less settled than when they started. That is not a failure of the optimiser. It did exactly what it was asked.

The usual fix is to ask people what they want and align to that. It helps, and it is not enough. What a person wants in the moment and what leaves them better off are often different, and most of us know this from our own lives. A system aligned only to stated preference inherits every confusion the person brings to it.

“Human flourishing” is the right direction, but not yet a target

Many in the field now say AI should serve human flourishing. We agree. But flourishing, as usually stated, is a direction, not something you can check. Whose flourishing? Measured how? Over what time? When two people’s flourishing seems to conflict, which wins?

Without answers, “flourishing” ends up quietly replaced by whatever can be measured, and we are back to proxies.

“Every system is aligned to something. The question is whether we chose it.”

A goal every human shares

Madhyasth Darshan, the body of work by Shri A. Nagraj that is our primary source, makes a specific claim here. It says there is a goal every human shares, whether or not they can name it, and that it has four parts:

• Resolution (समाधान): being settled within, because the questions that trouble you have been understood, not just answered.
• Prosperity (समृद्धि): having more than you need, which requires first knowing what you need.
• Fearlessness (अभय): trust, so that people do not need to guard themselves against each other.
• Coexistence (सहअस्तित्व): living in harmony with the rest of nature, as one part of a larger order.

The text goes further. It says that the evidence a person, or a society, has understood is that these four show up in how they live. And that humane conduct only becomes clear in the course of working towards this goal.

We write these four as clarity, enough, trust and harmony, and use them as the four layers of everything on this site. We have written them down, with definitions, evidence and failure modes, as the spec.

Why this is more useful than it first sounds

Three things make this a candidate target rather than another slogan.

It is meant to be universal without being vague. The source holds that it does not depend on culture, income or belief; whether that holds is something we would need to test across cultures. A farmer and a founder can both ask: am I settled? Do I have enough, and do I know what enough is? Do I trust the people around me? Is the way I live sustainable for the earth? The questions are the same; the answers will differ.

It includes a limit. Most targets we give systems have no ceiling: more engagement, more growth, more output. Prosperity, in this account, starts from knowing what is enough. A system aimed at it has a reason to stop. That is the part the planet needs most.

It puts understanding first. In the order the text describes, resolution comes first and the other three follow from it. The same text notes that people have often tried to become happy through profit, and calls this a mistake. For AI, that suggests a design principle: the most important thing a system can do for a person is help them understand, not keep them engaged and not just give them what they ask for.

What this would change in practice

If you took this seriously as an alignment target, some things would look different:

• An assistant would count it as success when a person stops needing to ask, because they now understand.
• A recommender would treat “this person has had enough for today” as a good outcome, not a churn risk.
• An agent working for an organisation would have its idea of enough written down, so it knows when growth stops serving the people it is for.
• Evaluations would include, alongside capability and harm, whether people using the system end up more settled, more trusting and less wasteful over time.

None of these are solved. Some may turn out to be impractical. They are the open problems we most want to work on with people who build these systems.

A protocol to try

Take one system you build or use, an assistant, a recommender or an agent, and write down what it currently counts as success. Then write what it would count if it were aimed at the four goals: would “this person now understands, and has stopped asking” be a good outcome? Where the two lists differ is where the design work starts.

How we would know if we are wrong

This is a hypothesis, and we treat it as one. It fails if:

• people who clearly understand themselves turn out no happier, or no lighter on the planet, than people who don’t;
• the four parts cannot be assessed without simply asking people how they feel;
• a system aimed at these goals turns out more paternalistic, not less.

Our study design tries to test the first; the second is an open problem. The third is a real risk. A system that decides what is “enough” for someone is dangerous. In this account, enough is something each person and family works out for themselves; a system’s job is to help with that understanding, not to make the decision.

An invitation

If you work on alignment, values or evaluation, we would like to hear where this breaks. Bring a counterexample, a better formulation, or a way to measure it. Build with us →

Limitations

• The four goals come from one source. We have not yet compared them systematically with other accounts of wellbeing used in AI evaluation.
• We describe what a system aligned to this target would do, but we have not built or tested one.
• Turning “resolution” into something measurable without reducing it to self-report is unsolved.

References

1. Manav Karm Darshan (मानव कर्म दर्शन), p. 96: the human goal is resolution, prosperity, fearlessness and coexistence
2. Vyavaharatmak Janvad (व्यवहारात्मक जनवाद), p. 47: the proof of understanding is the human goal
3. Manav Vyavahar Darshan (मानव व्यवहार दर्शन), p. 95: humans have wished, out of delusion, to be happy through profit
4. Manav Vyavahar Darshan (मानव व्यवहार दर्शन), p. 125: every human shares one wish, happiness, reached through resolution