AI & Design

Core distinction #

Extraction-optimised technology treats the user as an object to model from the outside, aiming to predict and influence behaviour such as clicks, time spent, or purchases.

Emergence-optimised technology treats the user as a reflexive aperture: someone whose understanding of their own preferences, motives, and possibilities can change through interaction. Its aim is to increase that self-understanding and the ability to act on it.

Predicting and influencing behaviour #

A product focused on prediction and conversion asks:

• What will the user click?
• What will keep them engaged?
• What will lead them to buy?
• What can we infer about them that they do not know about themselves?
• How can the system predict the user’s next action?

These goals may be effective, but often reduce users’ agency and reflection while increasing compulsive use and dependence.

Supporting understanding and choice #

A product focused on users’ understanding and choice asks:

• What is the user trying to understand?
• What are they not seeing about their own preferences, fears, motives, or possibilities?
• How can the system help the user understand their own reasons?
• How can the interaction help the user act independently?
• Can the user make and explain a choice after the interaction?

Design principle #

An AI assistant should help people understand their needs and make choices they can explain.

Product implications #

Recommendation

Extraction asks: What is the user most likely to consume?

Emergence asks: Which options help the user understand what they want and why?

E-commerce and travel

Extraction asks: Which product, listing, hotel, flight, or bundle maximizes conversion?

Emergence asks: What does the user want to buy or do, and which trade-offs matter to them?

Travel example: Before showing hotels or flights, an assistant could ask a traveller to choose between “quiet recovery with fewer logistics” and “dense exploration with more friction.” It would then explain which options fit that preference and what each involves giving up. The same approach can apply to other purchases: clarify the purpose, then explain the trade-offs. Its value would be tested by whether users understand their choices better.

Journaling

Extraction asks: How can we keep the user writing?

Emergence asks: Can the user recognise a recurring pattern in their thoughts or actions?

AI companionship

Extraction asks: How can the system become indispensable?

Emergence asks: How can the system support the user’s relationships, judgement, and daily life?

Practical evaluation #

Alongside engagement, measure:

• clarity after the interaction;
• ability to make and act on one’s own choices;
• reduced compulsive return;
• preference stability;
• quality of decision-making;
• self-reported self-understanding;
• willingness to leave the tool when the work is complete.

The companion study Coherence Without Contact sets out a pre-registered design and a rule for what would count against it. It measures 48-hour action integrity: whether a user follows through after the session on an action they chose for a goal of their own.

What would count as success #

By these definitions, a system is extraction-optimised when it becomes better at predicting users while users become less able to understand their own choices.

It is emergence-optimised when it helps users understand, choose, and act, even if they spend less time with the product.