AI & Behaviour Change
How should we shape how AI shapes us?
We are glad to share an external guest post. This essay is by Tom Rachman, a writer who has recently pivoted into the world of AI policy, with particular attention to the ways that technology will intersect with, and barge into, culture and society. Like all pieces you read here, this post reflects the author’s personal views.
“Shaping” human behaviour sounds sinister, stirring the horror of losing control over oneself, a fear almost akin to being consumed alive. From early childhood, we long to take charge of our actions, then gain that power in adulthood, marshalling it haphazardly for the decades that follow, and surrendering autonomy only if threatened by violence or debility.
But now and then, we are willing to toggle off full agency. Alcohol is a common tool for this. Or consider Ozempic and other semaglutides that millions of people eagerly take, targeting GLP-1 receptors in the brain that mimic hormones of satiation, shortcutting wants to deliberately modify their own behavioural responses. Those who dose themselves with Ozempic don’t feel disempowered. They frame agency as the high-level decision to alter behaviours they struggle to modify autonomously.
So the horror of losing self-control is not absolute. Rather, it is unwitting behavioural change that stirs anxiety. The history of new technologies is also a history of this anxiety, with each successful device blamed for seducing the masses from behavioural norms, whether it was the transistor radio, the video game, or the smartphone. Such panics are always correct: if the technology works, it will change users’ behaviour. But society habituates soon enough, and the next generation smirks at the fears of the last.
Not all innovations are equally impactful, though. Daily, futurists prophesy that artificial intelligence will transform our world, becoming our newest general-purpose technology. But unlike electricity or the steam engine, AI’s “fuel” is data, the traces of what our species has written, spoken, done and recorded, iteratively refined by human feedback. Humanity feeds these tools, and is embedded within them, making the behavioural impacts more direct, more potent. If machines operate us, who—or what—is in control? “Man himself has been added to the objects of technology,” the philosopher Hans Jonas observed back in 1979, remarking that this “may well portend the overpowering of man.”
The first flares of public unease over AI’s behavioural impact concerned social media, which many came to see as an algorithmic Svengali, engineering the polarization of politics, the degradation of culture, the fraying of norms. Debate persists over how blameworthy social media was, but nobody disputes that future AI systems will do far more than recommend cute videos and infuriating posts. These systems will insert themselves everywhere from the labour market to our bedrooms. We will invite them in.
Indeed, a key point of AI is human change, to better us by expanding our cognition. Evolution granted us an exquisite system of thinking, but it has embedded limits, and we cannot update nature at the pace of tech advancement. In particular, scale and complexity overwhelm us. But AI promises to become a form of pre-wisdom, assimilating more than we could ever sift through, incorporating sensory signals beyond our capacity, offering counsel we would never have known to ask for. Stanislaw Lem foresaw this in 1964, predicting that humanity would resist at first. But intelligent machines would process data so comprehensively that “intelectronics,” as he called it, would prove far wiser in selecting next steps. “After several painful lessons, humanity could turn into a well-behaved child, always ready to listen to [the machine’s] good advice,” Lem wrote.
We recoil at this infantilization, which triggers that dread of losing agency. The challenge is this: behavioural influence will happen; there is no opt-out. The question is whether we respond pragmatically, attempting behavioural audits of AI systems, incorporating workable controls, and infusing design with ethics. Otherwise, our rightful dread at what might someday befall humankind becomes a truth-denying passivity that decides matters for us. The choice may be this: shape your behaviour, or have it shaped for you.
OF TWO MINDS
Behavioural science is a field of optimistic pessimism. At once, it declares the human mind a blunderer, yet insists that the human mind can amend this.
Kahneman, a self-identified pessimist, described many cognitive biases on which the field is based, yet remarked that he still had little power to resist them. The science’s optimistic face is embodied in its other Nobel laureate, Richard Thaler, who considered the same constraints on human thinking, and proposed an answer: nudge. His book of that name, written with Cass Sunstein, enjoyed a timely publication, released around the nadir of the Global Financial Crisis in 2008, when rational actors had behaved in ways that seemed irrational. Policymakers, after stabilizing the markets, cast around for fixes. Among the most appealing was this promise of a low-cost brain toolkit: the nudge.
The key concept was “choice architecture” - the idea that small contextual changes may have large effects on how people behave. So, you arrange fruit before the desserts in a cafeteria, and diners—perfectly free to bypass the apples for the cake—are more likely to act in their better dietary interests. To avert charges of manipulation, the two authors offered the ethical underpinning of “libertarian paternalism,” that a policy could justifiably funnel individuals towards beneficial actions, provided that they retained the power to opt-out.
“The presumption that individual choices should be free from interference is usually based on the assumption that people do a good job of making choices, or at least that they do a far better job than third parties could do. As far as we can tell, there is little empirical support for this claim,” Thaler and Sunstein wrote. “People do not exhibit rational expectations, fail to make forecasts that are consistent with Bayes’ rule, use heuristics that lead them to make systematic blunders, exhibit preference reversals (that is, they prefer A to B and B to A) and make different choices depending on the wording of the problem.”
Over the years, researchers cited a plethora of cognitive biases, from anchoring effects, to intertemporal inconsistency, to present bias. Behavioural interventions included changing defaults (for instance, automatically enrolling drivers in an undersubscribed organ-donation plan); or adding commitment devices (compulsive gamblers signing up to self-exclusion lists at the casino); or encouraging social accountability (advising people to go to the gym with a friend to increase visits). But the field’s optimism was not always rewarded. A meta-analysis of 200 studies (n=2,148,439) showed that “choice architecture” had small to medium effect-sizes, with consistently lesser impact from interventions that demanded people commit to change, or relied on them absorbing new information. The largest effects seemed to come when participants didn’t exert themselves, but merely had their actions tweaked through changed defaults. Another study looked at governmental “nudge units” that undertook behavioural interventions, comparing their outcomes to those cited in academic papers. In journals, the average impact of a nudge seemed impressive: 33.5% increase over the control. In real-world interventions, results were far weaker: an 8.1% increase. The main culprit seemed to be publication bias, with journals tending to only print studies that claim significant effects, while interventions that did little vanish from the academic record.1
A further challenge is that populations are heterogenous. So graphic warnings on high-calorie drinks might discourage consumers who already had good self-control, but do little for those struggling to curb their appetites. Indeed, many just develop an aversion to graphic warnings, not to the drinks. That suggests that behavioural interventions might work better if personalized to individual preferences. But the value proposition of nudging for policy was that it promised benefits at scale.
As behavioural science grappled with these disappointments (or tried to ignore them), large-scale behavioural change was taking place all around. The cause was technology. Academics struggled to harness these possibilities for science, a few employing apps to gather individualized data, or testing wearable sensors, or just-in-time adaptive interventions (JITAIs) that prompted participants with personalized reminders and recommendations. Meantime, behavioural consultants worked with tech companies, and certain insights seeped into products. Behavioural designs sought stickiness, to give customers something to return for, and to stay with: just what they wanted.
Yet the proxies for human desire—for instance, click-throughs and engagement time—aligned with short-term wants. Long-term human objectives had few metrics, and this had a peculiar effect: many people found themselves doing what they wished not to do. Terms like “doomscrolling” and “brain rot” emerged. That primal dread of humans losing self-control surfaced. Public intellectuals fretted about whether free will even exists, while notable behavioural scientists—admitting the limits of their past interventions—questioned whether their past focus on the “i-frame” (getting individuals to change) had deflected attention from the “s-frame” (how systems change behaviour).
Part of the problem is that our species has built-in systems of our own, inscribed by evolutionary pressures over millions of years, and resistant to alteration. Kahneman explained this with his model of dual cognitive tracks, System 1 thinking (fast and effortless) and System 2 (slow and reflective). Sometimes, the brain errs by employing System 1 shortcuts when System 2 could more effectively reason through matters.2 These two systems—rather than exposing humankind as a race of dunces—are effective in most cases; otherwise, natural selection would not have left us with them. Nevertheless, human cognition does have vulnerabilities, and algorithmic intelligence homed in on many, as when feeding our hunger for effortless pleasures now that subvert our deeper longings later.
Among the outcomes was what the psychiatrist Anna Lembke calls “the plenty paradox,” that the abundance of contemporary life floods our evolved reward pathways in ways that stresses us, perhaps explaining the rise in depression and anxiety, which is most acute in wealthy countries that also have greatest access to the people-pleasing offerings of new tech. “One would hope and think that we’d be engaging in deep philosophical discussions, helping each other, cleaning up the garbage,” Lembke says, of our era of plenty. “But instead what we’re doing is spending a whole lot of time masturbating, shopping, and watching other people do things online.”
SQUINTING INTO THAT FOGGY FUTURE
You wake with a jolt, your mouth parched, your T-shirt damp with sweat. Somehow, you’re queasy but famished too. Mostly, you’re annoyed. Not just for the excesses of last night, but for how you’ve acted lately: self-centred, distracted, unproductive. You rouse your AI ecosystem, and open its settings, selecting “My Future,” where the blizzard of daily life is simplified into sliders, using probabilistic analyses rooted in the heaped datapoints of humankind, cross-referenced to your personal behavioural history, modified by your current physiological inputs, and enriched with environmental sensors presenting perceptual insights beyond the capacity of any living being. You adjust the “Objectives” slider, shifting your behavioural preference from “Short-term pleasures” towards “Long-term goals,” and set the timeframe as “This Week.” Immediately, the system reconfigures your AR overlay of information stream and notifications, tweaking your mood-responsive assistant, and altering the haptic-nudge schedule on your wearables, all with the intent of curtailing the defeating habits you struggle to repress while promoting those behaviours you long to manifest.
Would this curb your autonomy? Or enhance it?
Humanity’s dilemma between short-term wants and long-term objectives is a conflict that has provided drama enough for most of the novels, plays and ballads ever written. It’s an inner contest that also infuses today’s primitive recommender systems, where algorithms overwhelmingly feed us what the philosopher Harry Frankfurt called first-order desires (I like watching video clips) at the expense of second-order desires (I wish I just wanted to read a book).
One way of understanding this is as a time-horizon mismatch between desire now or contentment later. Most people agree that a good life includes both fleeting pleasures and the accumulation of achievements. We simply disagree on the proportions. Conceivably, a personalized AI ecosystem could allow users to express their preferred balance, altering behavioural inputs accordingly. Users might also pursue betterment of specific character traits—say, moving a “Resilience” slider to challenge existing views and action patterns, much as one programs higher resistance on a treadmill to build endurance. Chatbot responses could incorporate random variation too, emulating the stimulating unpredictability of other people, rather than making us captive to confirmatory feedback loops. Otherwise, AI companions—already in deep relationships with humans that include sex and “marriage”—could become behavioural anchors, a constant companion for your whole life, who holds you to your 20-year-old character when you’d otherwise have evolved into a 50-year-old.
To avert behavioural stagnation, AI companions might be made mortal, but that could inflict terrible suffering on their human partners. A less-brutal approach would be programmed forgetfulness, so that AI systems remove facets of your past from their data, much as people superimpose your current character upon past versions, which become fainter as years pass. Likewise, AI companions could alter behaviourally over time, perhaps even growing apart from you. As most humans have a strong drive to hoard artifacts of their past, AI systems might store their comprehensive memories of you apart from the working behavioural datapoints, so that its agents and recommender systems operate according to your current self, while an archive sits in abeyance, like a nostalgia vault you may access if ever longing to revisit who you were. You might even choose to venture back among those old settings, experiencing the behavioural filters you once so ardently sought, presenting a parallax view of your life’s course, from then to now.
At set intervals, your AI ecosystem could present a review of your behavioural trends, and solicit your updated preferences, perhaps as part of New Year’s Resolutions. On each such occasion, the system could offer advice based on the latest empirical findings on human wellbeing. Today’s evidence on wellbeing still stirs debate, with mixed answers to questions like whether money can buy happiness, and whether current wellbeing measures are reliable enough to make policy. Influential studies may have queried just a few hundred people in an artificial environment. AI behavioural studies could involve sample sizes in the billions, amassing the most granular set of behavioural insights ever attempted - computing ever-larger datasets based on our (anonymized) responses and wants. This could produce a sturdier science of “choice architecture” and the conditions that amplify or stifle effects, how to design and avoid feedback loops and how individual characteristics moderate species-level tendencies. Such findings could amount to our greatest clarity yet on “the good life,” and whether that is even what people care for, given how often their actions commonly subvert their stated wishes. We might learn too whether behavioural interventions really change much, or if each of us is destined to hover around a baseline nature, hardly corrigible no matter how many self-help books we pile on the bedstand, or how many “My Future” adjustments we make to our AI-ecosystem settings.
Troubling outcomes are possible too. AI might discern behavioural correlates within datasets, and engineer unintended changes in human culture. Imagine if datapoints revealed that listening to certain music was associated with higher rates of depression, causing well-intended recommender systems to steer everyone away from ever hearing such songs, becoming the gatekeepers of human art. Or what if AI ecosystems sensibly diverted people from toxic relationships? Anyone would be relieved to dodge a disastrous marriage, but what if the machines judged you toxic, and people you knew started treating you like a pariah? Those who failed to boost others’ metrics could become data outcasts, banished from good society by AI. Arguably, this could disincentivize awful behaviour, pressuring the nasty to amend their ways. Or maybe it just punishes the eccentric.
Data protections would be imperative, ensuring that nobody is behaviourally hacked, whether by a bad actor, or by an institution to mobilize the citizenry to their preferred ends. However, people might accept external influences on their behavioural AI—say, allowing a spouse to adjust your settings in exchange for avoiding divorce. Or a healthcare provider might offer discounts to clients who alter their AI settings to optimize for determinants of lower medical costs, such as exercise, socializing, and healthy diets. Parole boards could make release conditional on AI behavioural diversions, and governments could incentivize pro-social behaviour by offering tax credits to those who optimize for charitable activities.
But any external influence tagged to one’s behavioural data is fraught with dangers, easily degenerating into a social-credit system that coerces conformity and further penalizes those who already struggle with immiserating behavioural tendencies. A further risk is that incentives for “good behaviour” undermine intrinsic motivation, much as a child who is paid each time she acts politely to an elderly person might develop a distorted practice of courtesy, withholding kindness from granny until someone hands over cash.
THE TAKEAWAY
AI ecosystems will influence human behaviour. The question is whether we can reconcile three outcomes that seem at odds: 1) motivating people to act in the present for their desired futures; 2) retaining human agency; 3) persuading AI developers to pursue these ends.
It’s facile to profess that companies must prioritize public wellbeing. Developers need a realistically incentivized way to implement beneficent ends. That could mean corporations changing income streams so that behavioural AI is not primarily funded by advertising based on short-termist metrics, but accrues revenue from users’ progress towards stated goals, perhaps with payments made accordingly. People may be reluctant to pay $20 per month for a chatbot subscription, but most would shell out far more if the money were evidently correlated to their career success, or a better body image, or more fulfilling social lives. Think how much money people already spend on gym memberships and self-improvement courses, often to limited effect.
Regulatory policy might adapt too, so that tech developers whose products had a demonstrable drag on measures of health and productivity might face curbs or fiscal penalties, while those labs whose tools benefit public welfare might gain access to government contracts, R&D funding, or tax discounts proportional to the improvements.
At each stage of this ongoing AI transformation, system design should attend with utmost care to behavioural effects, applying empirical evidence to responsible development, while informing policymakers, and alerting the public. This means moving beyond the existential-risk debates to establish a human-risk research agenda, including pre-release behavioural audits of significant AI applications, post-release monitoring of how humans actually interact with them, and longitudinal studies to judge their diffusion through society, considering everything from polarization, to loneliness, to wellbeing.
We did none of this with social media. As a result, we still dispute what it did to humans, whether it is the cause of contemporary strife, or if we are just blaming machines for our own ills. What we must avoid is leaving AI to set the proxy for “the good life.” Otherwise we risk perpetuating the dismal compromise: getting what we want, to ends we do not want at all.
Done properly, we may create a positive feedback loop, where AI does not simply crash into humanity. It helps explain us.
APPENDIX: SEVEN SPECULATIVE SETTINGS
Could AI improve your behaviour? Or will it supplant your autonomy? In part, this may be a design problem. Behavioural AI should favour your best, allow for personal evolution, and incorporate human agency. Here are a seven ways:
1. Know-Your-Human Requirements
Onboarding: Before activation, any personalized AI system with meaningful sway over behaviour could conduct a short and transparent conversation with its user, establishing the human’s short-term habits (e.g., “I’m addicted to tea” or “I go to bed too late”) and long-term aspirations (e.g., “I dream of moving countries,” or “I wish I were more sociable”).
No box-ticking: The process must never become akin to Terms & Conditions, but like a first chat with a new therapist, establishing priorities, career goals, health issues, what’s missing from one’s life. The results must be strictly encrypted, and accessible only to the human in question.
2. Staying Aligned
Changing Your Goalposts: The Know-Your-Human survey establishes the opening defaults of the AI ecosystem’s weighted recommendations and nudges—say, a behavioural balance of 70% long-term objectives to 30% short-term pleasures (among many other personalized settings). These defaults must remain accessible, intelligible, and simple to adjust.
How’s It Going?: Annually, the system checks back with the user, offering an engaging review of the intervening period, akin to Spotify Unwrapped but for your behaviour, while ensuring that recommendation weights and intervention goals still align with the user’s wishes. Users could toggle on an optional “All Good?” oversight mechanism, which would detect radical fluxes in behavioural data that might suggest an alteration in goal-profile and perhaps acute distress, which would trigger a check-in.
3. Human in Charge
Under My Thumb: Any behavioural influence—from recommender-system weights to goal-driven nudges—must have an available explanation. This could mean a sidebar reasoning card, or a hover-over rationale, or a voice-interaction mode that could take user questions.
Sliders Instead of Black Boxes: Besides the pleasures/goals balance, users may also adjust more narrow preferences, moving behavioural influences between poles such as “entertainment” vs. “education”; “novelty” vs. “familiarity”; or “fresh learning” vs. “knowledge review.” Any settings change with significant behavioural impact should include a forecast of possible outcomes.
4. Sliding Doors
Differing Paths: Whenever prompting behaviour, the AI system should know other approaches, and make these available at the user’s request, including simple explanations of how each alternative might differ in effect and underlying assumptions, much as map apps will display alternative routes and forms of transport, with differing travel times.
5. Shuffle Mode
Randomized Serendipity: To encourage exploration and break feedback loops, recommendations should occasionally contravene past behavioural patterns. The user sets the desired proportion of randomization, but also has a “Surprise Me” option alongside standard recommendations.
Default Updates: After the user interacts with a “Surprise Me” input, the system may ask whether this challenge felt worthwhile. Such queries should not be excessive, and may be toggled off. But if beneficial, the feedback could inform future system defaults.
6. Wisdom of Crowds
Community Evidence: Recommender systems should favour inputs that other people with similar goals (and success in achieving them) have engaged with, rather than just circulating what is most popular.
Smart Cues: The system should provide quality signals, tagging options according to reputational ratings and empirical goal-efficacy. The current practice with recommender systems—either struggling to define suitable content or eliminating curation altogether—have precipitated political contests that may disregard users’ goals.
7. Repel the Intruders
An Off-Switch for Context-Switching: The system default is to focus the user’s concentration on their current desired activity, minimizing distractions. This could include adjusting the user’s environment to encourage flow, and making judgments on whether to mute specific notifications, based on the user’s current activity and its pertinence to their goals.
Angel on Your Shoulder: If users are engaged in behaviour they have expressed a wish to avert, the system could offer gentle interventions, perhaps speaking a reminder aloud in the AI-generated voice of a friend or trusted influence (who gave consent), or even in the voice of the user: “Hey! Sorry to interrupt, but just a reminder that you wanted an early night? Shall I turn off when you finish this clip?”
Michael Hallsworth, chief behavioural scientist at the pioneering nudge unit BIT, wrote a nuanced commentary on the debate over such interventions. To claim that “they work” or “they don’t work” is a simplification, he argued, noting that interventions vary widely by context and target group, meaning that the effects will vary too.