Programs

About

Publications

Get Updates

Programs

Scenario Research

Governance Research

AI Awareness

About

About Us

Our Team

How We Work

Theory of Change

Blog

Get Updates

Programs

Scenario Research

Governance Research

AI Awareness

About

About Us

Our Team

How We Work

Theory of Change

Blog

Get Updates

Conference Report

Appendix: Threshold 2030

A two-day conference in Oct 2024 bringing together 30 leading economists, AI policy experts, and professional forecasters to rapidly evaluate the economic impacts of frontier AI technologies by 2030.

Report Home

Download Full Report

All Worldbuilding Writeups: Part 2

Attendee 4A

General Worldbuilding Exercise

Creative structure

Day-in-the-life narratives: Short written accounts describing a typical day for different individuals in this future world (e.g., a student, a worker in an AI-impacted industry, or a policymaker).

Questions:

How has the global balance of power shifted in response to the development (or lack thereof) of these AI systems?

How has the concept of expertise and professional authority been redefined in a world where AI systems can outperform humans in many fields? X

What new industries or economic sectors have emerged as a result of the AI advancements in this scenario? Conversely, which traditional industries have significantly declined or disappeared?

How has AI changed the landscape of scientific research and discovery across different fields?

Personas (I will try to keep these more or less the same throughout the scenarios, though they might change given the implications of the model):

Gig-economy worker: No degree. This person rotates between working for Uber, Uber Eats, and Task Rabbit as their primary sources of income.

Product Manager B2B SaaS: Group PM at a late-stage B2B SaaS company. 10 years of experience as a PM and 3 as a developer

Computer Science student: third-year CS student at top university. Has completed internships at competitive companies and is about to graduate/enter the workforce.

Actor: Commercial, voice-over, and film actor. Has been able to sustain a good income pre-2024 and is successful but is not well know.

CEO AI company: This one will change given the scenario

Nurse Anesthetist: 10 years of experience as an RN. 10 years of experience a NA

Scenario 1

Gig-economy worker: My job as a driver for uber has been replaced by a self-driving fleet. Luckily, I am still able to find some work delivering food for Uber. They claim that this is due to the complexities of food hand-off and quality assurance. Uber is currently running a pilot for automated food delivery in the Bay Area, but so far it has only replaced ~5% of deliveries. My Task Rabbit gigs (assembling furniture, moving, organizing, cleaning etc.) have remain unaffected.

Product Manager: Last month we had a round of layoffs. My job was safe, but nearly all of the early career design, marketing, and engineering jobs were removed. My team is now composed of myself, one QA engineer (we tried to automate this last quarter and went through two painful releases before rehiring), and a developer. Most of the product design, marketing, and maintenance is now handled by mid-level roles through automated systems. Group managers now focus on scoping while mid-level workers handle the actual development of several projects at once.

Computer Science student:

Actor: The union is still holding on. After a lengthy strike, we struck a deal with a new company to sell our visual and VO likeness in exchange for royalties. This company has been very successful for VO, but the commercial acting gigs have been largely uneffected

CEO AI company:

Scenario 2

Gig-economy worker: All delivery jobs have been removed. My work is now limited to assisting with moving, building furniture, and helping people run errands.

Product Manager:

Computer Science student: I am considering either double majoring or changing my major entirely. The only people I know with job offers are students who started their own one-person unicorns or are in health professions that include high levels of patient care.

Actor:

CEO AI company:

Scenario 3

Gig-economy worker:

Product Manager:

Computer Science student:

Actor:

CEO AI company:

Deep-Dive Worldbuilding Exercise

Scenario 2.5

Following the discovery of China’s violation of the Strategic Compute Verification Treaty, President Vance has used the DPA to address AI-related national security concerns. Through the Secretary of Energy, the minimum requirements for grid allocation have been established to support security, safety, defense, and chip manufacturing research (insert other materials-related requirements). AI companies are now explicitly prohibited from publishing their research (including model weights and datasets).

Foreign nationals from strategic competitors who are employees at AI companies have been put on forced leave pending additional review. This comes as an expansion of DPA authorities which have historically not been used to impact employment contracts or wages.

Many advocates worry this move will further escalate tensions between the United States and China. Previously, we have seen increased efforts by the Vance administration to nationalize AI-related activities to bolster US national security.

Attendee 4B

General Worldbuilding Exercise

disc notes:

scenario 2 ~ "more agency" vs S1

guiding questions

cost of AI systems

diffusion

vert/horiz integration

least AI-influenced tasks/occupations/sectors

open source availability – how much do AI costs reduce to compute+data+energy if algos are free?

rate and direction of cultural change

Scenario 1

Overall, AI systems in 2030 are more powerful versions of today’s LLMs, but with similar structural limitations. LLMs still primarily function in response to direction from humans, and do not take the initiative or act independently. Though LLMs may be better integrated into existing workflows and products, this scenario doesn’t assume significant advances in independent reasoning or end-to-end execution.

Future challenges briefing

While LLMs have lead to substantial productivity gains in white-collar occupations, the following firm-level and societal challenges remain:

Unreliability

LLM output accuracy has reached human levels on basic language analysis, summary, and conversational information transfer tasks, as well as logical and mathematical reasoning. Most failures are attributable to underspecification of tasks, which makes these failures difficult to predict and sometimes even to notice by the individuals supervising them who themselves failed to specify tasks

Logistical fragility

Automation of supply chain and logistics functions has exacerbated the problems of just-in-time supply chains, due both to a lack of human oversight (due to marginal cost cuts) and the correlated failures from similar systems used in

Social backlash

immigration debate shifts to AI (in some sectors) – increased social unrest as unskilled labor (e.g. call center, data entry workers) has difficulty finding new employment

(assumptions: lag between automation & job creation, in which fewer laborers needed per unit productivity due to AI supervision being more efficient than labor; lag furthered by time taken to upskill)

[ongoing polarized AI debate]

increase in cyberattacks on major companies; AI-powered cybersec lags behind offensive capabilities

misinfo/disinfo as media gen takes off

deprioritized

algorithmic bias – mostly fixed at tech level, problems remain at cultural level

Cost fx: mostly a modifier on degree of concern, proportional to how widespread the challenges become. most costs are marginal, not upfront, due to API-usage pricing models

high marginal cost → lower adoption → localized challenges

low marginal cost → higher adoption → widespread challenges

exceptions

logistical fragility can be disproportionate if low adoption is horizontally high-impact

Scenario 2

AI systems achieve better results than people in most constrained or well-scoped tasks. However, they fail to outperform humans in task integration, handling multifaceted responsibilities, and communication with other humans. They still require oversight.

diff from prev section:

Higher social backlash, specifically due to unemployment & labor classes automated

Workers with fewer soft skills disproportionately automated as LLMs handle most well-scoped tasks better

possibly correlates unemployed labor with lower EQ – if true, could predict a slightly higher modifier on civil unrest likelihood

More labor automated over same period → less time for replacement job creation in new forms of work

expected movement of unskilled white-collar labor to blue-collar

more blue collar work automated, but at expected high capex – AI systems may still not be cost competitive with human labor, given increasing availability of unskilled human labor

→ more people working shittier jobs → more likely unrest

Sharp decrease in social trust as AI interactions becomes indistinguishable from human interaction

more cultural fragmentation, filter bubbles

if someone develops good identity systems this could be decreased (maybe disproportionately in authoritarian states?)

start to uncover "cultural terrain" of potential for organized labor vs technofeudalism

Unreliability no longer a significant concern

key concern with LLM usage is now correct framing, tone, and contextualization of material; LLM flexibility allows for high accuracy in specified output along these dimensions, so bottleneck is human understanding & soft skills

localized business development becomes more important; productivity increases accrue disproportionately to businesses serving culturally homogenous markets, new sector of "cultural l10n as a service" companies

Logistical fragility increased – OR new sector emerges to handle these issues

May be contingent on timing of major catastrophic failures ("AI Evergiven") that increase demand for reliability infrastructure

AI systems have capacity to perform complex routing systems, in proportion to data – supply chain failures in response to sharp demand spikes (or physical infra failures) common, most others uncommon

Scenario 3

Powerful AI systems can meet and surpass the performance of humans in all dimensions of cognitive labor, and can function as “drop-in” replacements for nearly all human jobs.

diff from prev section

Key challenges are all in cultural & societal reactions to this new social class

Ongoing relevant discourse topics include

ethics of labor automation

consequentialist concerns re: high unemployment

consequences dependent on speed of reactive policy

the "self-driving cars are safer" debate, but for everything (doctors, lawyers, etc)

potential for specific monopolistic firms to have disproportionate influence on social outcomes

depends also on monopoly status of AI labs – how much domain expertise is necessary for the deployment of such systems? how much of this can be incorporated by AI labs themselves vs specialized per-sector firms providing AI services?

trust in AI leadership

AI managers eliminate many possible (game theoretical) moves humans expect to have; upskilling in new class of AI-related soft skills required, and likely resented

trust in human use of AI

"AI-washing" is the new McKinsey: do claims of use of AI constitute claims of authority?

ethical treatment of AI minds (and more Blake Lemoines)

probably still just marginal and kind of annoying, but possibility remains of new research breakthroughs (accurate or specious) claiming sentience may significantly change use of & relation to this labor class

AI research systems may modify this likelihood, proportional to attention on cleaning training data of relevant sentience claims etc

Main challenges

Social backlash in response to widespread unemployment

the immigration debate, but worse (i.e. with more fascist tendencies), about AI systems as an effective social class

highly skilled laborers now also unemployed → the Lenin train but at societal scale

inner alignment solved, outer alignment remains unsolved

Deep-Dive Worldbuilding Exercise

Scenario 1

small group settings: LLMs are the new Google: you can look anything up in a few seconds in group conversational settings, but few people do.

most LLM use is still between individuals and machines, due to task specification being a bottleneck (and group agreement on task specification being high overhead)

some improvements in LLM summary accuracy lead to LLMs replacing individuals in operational tasks in group contexts: LLMs are the notetakers, the creators of slack polls, the creators of calendar invites

leading edge of exploratory (weird) groups in younger generations create entirely AI-generated social events, but these are rare and appealing largely for their novelty, not other value added

AI-generated information distillation still notably worse and less trustworthy than that of human equivalents, thanks to ignorance of code switching, language registers, and specific cultural context

most useful applications of AI to summarize, analyze, and present information well is still in the paradigm of AI as tool, not AI as collaborator

trust bottleneck is sensible and largely attuned to the true incapability of AI to function as a social agent

Cross-domain capabilities are better: meeting transcription, scheduling of follow-ups, CRM integration etc are now better integrated (provided that data is shared)

AI replaces:

slack polls, surveys, administrative assistants sending follow-up emails, CRMs

AI doesn't replace

basically everything else about group coordination

Scenario 2

Similar to S1; largest difference is that AI is now useful for high-fidelity processing of large information sets

most benefits accrue to leadership of large organizations, who can now make better predictions/extrapolations/aggregations based on data from their populations

removes bottlenecks on direct constituent input into decision-making – leaving incentives to collect/use/disseminate this info as the remaining bottleneck

increases incentives for corporate surveillance

automated decision support is now close to useable – AI can suggest good potential policies

reduces cognitive drain on leadership (by providing menu of options to choose from, once ideation is automated)

increased capacity for good governance in grassroots, collectively governed, democratic, consensus-based, etc organizations

large-scale decision input made possible in higher fidelity

problems of social choice theory remain, but may become clearer and more concrete in their application as they become the bottleneck on good collective decision-making

automation of basic labor reduces capex, granting higher productivity to small organizations

IF COSTS LOW: potentially increasing small orgs' competitiveness in the landscape (given their other attributes of fast decision-making, lower internal tx costs, etc)

IF COSTS HIGH: increasing small org competitiveness proportional to investment to fund capex

in high-expertise domains, automation of basic labor increases individual attention budget for interpersonal coordination

in an ideal world this means more effective teams; in practice it could also mean more meetings

trust becomes more significant form of capital; soft skills increase in relevance for occupational success

certain forms of metadata – completeness, thoroughness, detail of presentation – no longer function as demonstrations of individual expertise once they are automated; individual expertise likely calculated more as a function of trust, which biases more towards iterated games but also more towards gestalt qualities like charisma

concerns of AI bias more problematic as more work is automated – new paradigms needed for groups to trust models

more demand for fine-tuning/RLHF/other alignment approaches usable by nontechnical leaders

possible creation of ecosystem of culturally attuned AI models – differentiation in norms and principles, not just capabilities

increase in trust as currency still means humans required in the loop for e.g. legal engagements

Scenario 3

Powerful AI systems can meet and surpass the performance of humans in all dimensions of cognitive labor, and can function as “drop-in” replacements for nearly all human jobs.

This scenario marks the point at which AIs may be able to function as social agents.

the first true AI team members exist, though diffusion is low due to high social transaction costs, as culture changes slowly

bots become useful in social contexts

automated content moderation is possible; misinformation and disinformation issues are now problems of cultural and ethical norms, not of information processing

private information ecosystems are cleaner and higher-trust

individuals become more effective relative to groups

personal-assistant tech becomes more widely available, making the soft-skills bias from S2 more broadly applicable to most production-focused cultures

single-person projects are now competitive with small startups

crux point: stigma of AI companions

low stigma: AI agents incorporated into mediated social contexts, increasing ease of access to wide variety of on-the-fly research and info processing work

high stigma: work and social contexts notably diverge;

"AI minds" discourse (Blake Lemoines) persists with wide variety of perspectives present, possibly cohering into factions

good AI leaders now possible

if high interpretability, increased reliability/predictability of AI managers may be preferred to mercurial humans

bottom-up-governed orgs function effectively via automated decision making, still with humans in the loop; something more user-friendly replaces DAOs and actually gets used

first AI politician runs for office, but largely as a publicity stunt, and does not win

new paradigm replacing "data driven decision-making" focuses on comparing human predictions/decisions to AI outputs

Attendee 4C

General Worldbuilding Exercise

Scenario 1

Scientists from Deepmind University today announced a re-categorization of diseases that promises to notably improve the quality of human treatment. In their work, these researchers found that traditional categorizations were only weakly predictive of which treatments would help patients. Drawing on their new AI-fusion model, which combines the Google Gemini7’s review of the medical literature with Deepmind’s biological simulator work, researchers created new statistical models which allowed them to group diseases more finely - into ten times as many ailments as those presented in traditional medical diagnosis manuals. Preliminary testing with UK NHS doctors has found that the new AlphaDiagnosis program led to a 10% reduction in drug prescriptions that led to no clinical improvement, and a 0.8% reduction in patient mortality.

Scenario 2

The National Institutes of Health today announced the funding of a new $40 billion initiative to test and validate the health recommendations coming from the large AI builders. The medical arms of the AI companies - all spun up in the past five years - have released a steady stream of bold claims about medical breakthroughs, ranging from drug repurposing, to life-extension strategies. But clinical testing of these claims has fallen far behind the breakneck pace of announcements, leading some to criticize the AI companies for making announcements to drive stock price increases, rather than waiting for empirical validation. Leading AI health researcher Jan Mandrake contested this claim: “The breakthroughs that we’re able to make with these models are so rapid, it would be unethical for us to delay making the announcement and therefore withhold these benefits from the public.” The new NIH initiative is designed to provide the clinic support needed to validate claims. Half of the initiative’s budget and robotic staffing will be provided by partner AI firms, eager to get these validations, but also to gather the new biological data that will be produced by the trials.

Scenario 3

Rival health claims from big AI firms will be unresolvable in the near future, according to the NaiH health oracle. Recent success in developing new medicines has been successful, but the proprietary nature of the models has created a challenge, with AI medical rivals each claiming their approach is best. In this case, both GoogleHealth and KaiserAI have advocated for new antibody therapies designed to dramatically reduce human heart disease. Initial testing on both has been remarkably successful, leading to 98% reductions in coronary deaths. The National Institutes of Health AI advisor, NaiH Oracle, today explained that this has created an empirical challenge: “With both companies' products being so successful, but being based on proprietary models that are inscrutable from the outside, it will take years for sufficient empirical evidence to accumulate to know which is best.” The Secretary of Health and Human Services, Frederica Johnson acknowledged this point, but put the rival discoveries into perspective “In the short run we should focus on the enormous benefits to human health that both of these models are providing. In the longer term, we plan to work with our private sector partners to ensure that these barriers don’t artificially slow future innovation.”

Deep-Dive Worldbuilding Exercise

Scenario 1

New signs hint that the world could move back towards autarky

In a rare event, the international ties between countries grew weaker this year, according to an IMF analysis. For a century, the ties between countries have usually deepened, with the movement of labor and goods growing outside of wartime or other emergencies. But recent advances in AI have pushed countries into greater isolation, as rich countries begin to eschew global supply chains built around the cost advantages of cheap labor. For their part, poor countries have returned the favor, erecting trade barriers to prevent inflows of certain goods that are AI-produced and which threaten unemployment in the developing world. IMF spokesperson, Jimena Sanchez expressed concern, saying that global decoupling threatens political and economic apartheid.

Scenario 2

Scandinavian governments announce global health initiative

In a departure from previous international development funding, a consortium of Scandinavian country governments today announced a new health initiative to provide free basic medical care to the world’s poor. Over the next 10 years, the initiative promises to send robotic nurses to any village of 50 or more people that wants it. Using knowledge encoded in the new WorldHealth AI model, the robotic nurses will assist with a wide variety of health care needs. Initiative director, Sven Eriksson explained that “the falling cost of specialized robots has made it possible to provide cheap, effective health care. It is our responsibility as global citizens to share these benefits with the world.” Despite the potential health benefits that such care could provide, many countries are expected to be wary of adoption, fearing that the outsourcing of their medical information and treatment decisions could become a major vulnerability.

Scenario 3

Return to nature movement gains steam

Facing the so-called “AI retirement”, some displaced workers are turning to a more rural existence. Funded by government AI-unemployment schemes, these workers are creating new agricultural and industrial communes, which they feel will provide them with “the spiritual benefits of meaningful work” while still benefiting from modern conveniences. According to the website of one such group, RealWork, modern AI technologies will still be available to commune members in a central headquarters that will provide medical and other care. But outside that environment, members will be expected to eschew AI conveniences. A Department of Labor spokesperson remained noncommittal about whether the government would embrace these initiatives, saying “we’re all exploring the new ways that humans will find meaning in this AI-era. We look forward to seeing how these new initiatives will play out.”

Attendee 4D

General Worldbuilding Exercise

Medical sector and entries

Scenario 1: collapsing radiology, still active growing med school entrance statistics with respect to internal medicine / surgery etc. no substantial change with respect to 60% of medical sector, but certain disciplines heavily leaning on diagnostics are collapsing

Scenario 2: collapsing radiology, internal medicine, diagnostic disciplines but no threat to physical labor, at the same time corporate capture that creates further wealth inequality, AI models in hands of insurance sector, making medical sector less accessible

Scenario 3: easy to manufacture drugs makes insurance obsolete, medical sector is only in physical labor eg. surgery, people have more access to diagnostic capabilities in general through competent open source models

Can we learn from how AI models are acting in the world in order to recreate the model / identify the base data, the way quant funds can frontrun alpha that specific groups have? Would this mechanism work?

Deep-Dive Worldbuilding Exercise

2. US Continues to Outsource Family Medicine Specialists

Average per week, family physicians can see up to 400 patient encounters. This is a drastic increase from ~80 encounters back in 2023, but still, not enough to meet the rising demand.

3. Generic Drug Copies Flooding the Market

The legislation is falling behind re: testing and illegal distribution of copies of generic drugs. Despite posing risks and not fulfilling the FDA process, millions of citizens are preferring the cheap and unverified alternative copies that have been developed with the leaked PharmaToTable model from OpenAI, which was controversially funded by insurance companies. This development cycle has helped India’s economy substantially as the North American bombardment of the drug development cycle is basically too slow.

2. Be A Dentist If You Want to Be Paid

The medical sector admission rates are declining rapidly in the US, other than surgery and dentistry. With the launch of DrHouse by Facebook, Medical schools across the country are protesting and highlighting the risks of using such models by the general public, and that they should be under control of licensed doctors and used for educational purposes. Meanwhile, developing countries have seen sharp adoption

1. Insurance Sector Develops Private AI Models

The state of the art AI models have enabled the insurance companies that develop internal models to stay competitive for diagnostic purposes. The data collected by the industry to date seems unbeatable, and synthetic data models seem to not be able to replicate the results.

3. Simple Surgery

The developing world is receiving a lot more underspecialized human labor that can follow instructions and develop basic competence on surgical skills. Lower skill specialization is drastically increasing, 10x number of people who can perform basic surgeries without medical school training, with slightly higher risk but much wider availability.

3. Better to Train than Treat

Doctors abroad are being employed by OpenAI rather than treat patients as highly skilled specialized labor is getting more and more valuable as a differentiator.

Attendee 5A

General Worldbuilding Exercise

What new industries or economic sectors have emerged as a result of the AI advancements in this scenario? Conversely, which traditional industries have significantly declined or disappeared?

Criminal uses of AI - what are the easy applications for normal criminals, and what can organized crime do?

How has the entertainment industry adapted to this level of AI capability? What new forms of media or art have emerged?

How have the costs of goods changed? Does the cost of services increase or decrease in comparison to physical goods?

How has AI affected wealth & income distribution within and between countries?

What does day-to-day life look like for an average person in a major city in this world? How do they interact with AI systems throughout their day?

Scenario 1

Human data industry

Private dataset market

Compute market

Decline in customer service

Insurance for personal fraud

Cost of compute decrease / Moore's law
Less concentration in GPU market
Criminal cases: massive increase in fraud and social engineering attacks
Affects all parts of the economy
AI related art and bespoke fine tuned model that outputs particular styles fo work are more and more popular
AI assistant personality design is increasingly popular, high end tailors design ai personas for wealthy clients
Long term continuous interaction with corporate owned models tailored to your preferences, and aesthetics
Customised clothing, cars, and products aided by advanced manufacturing with ai aided design
Custom goods are cheaper

Bloomberg: "Luxury AI Persona Designer 'Companion Couture' Reports $2B Revenue, Waiting List Grows"

@ConsumerWatch: "WARNING: New 'Digital Twin' Scam Uses Your Personal AI Assistant's Voice to Trick Family Members"

The Verge: "Meet the Data Farmers: Inside the Booming $500B Human Interaction Data Industry"

LinkedIn Post: "Looking for work? Top paying jobs this week: AI Personality Architect, Data Experience Designer, Digital Behavior Curator"

Reddit r/AIFashion: "Just got my AI-designed, auto-manufactured jacket based on my spotify playlist vibe 🔥"

Financial Times: "Corporate AI Companions Now Own 40% of Consumer Emotional Data, Regulators Concerned"

Top Gear Magazine: "Tesla's new 'Heritage Line' lets you mix design elements from any classic car into your custom EV build"

The Guardian: "Rise of 'AI Companion Addiction' - Therapists Report Surge in Attachment Issues"

Business Insider: "Last Major Customer Service Center Closes as AI Handles 95% of Consumer Interactions"

New Jersey Enquirer: "High School Adds 'Data Privacy Defense' to Core Curriculum as Scams Target Teens"

@MarketWatch: "Custom Manufacturing Index hits all-time high as AI design tools democratize production"

New York Times: "The New Social Divide: Those Who Can Afford Custom AI Companions vs Those Who Can't"

Wired: "Inside the Underground Market for 'Emotional Data Sets' - Your AI Friend's Memory Might Be For Sale"

@TheGuardian: "Should AI companions be required to identify themselves? New legislation proposed after dating app controversy"

Forbes: "Meet the New Creative Class: AI Style Trainers Command Six-Figure Salaries"

Scenario 2

Massive increase in cybersecurity attacks - proof of human needed

No social engineering requires

Personal cyber offence for hire spreads in corporate

Artists train ai models on their own content and limit release

All content is generated on the fly for the consumer watching it or listening to it

Multiple ai companions are always around users accessed through glasses or earbuds

Cost of human interaction becomes inaccessible particularly in medical, care and customer service - privilege of only elites

The Onion: "Last Human Customer Service Rep Preserved in Museum: 'I'd Like to Speak to Your Manager' Now Historical Phrase"

r/antiwork: "My AI Employees Are Unionizing, Demanding Better Processing Power and Cloud Storage"

Scenario 3

Many one person companies with hundreds of agent based employees

Millions of people stuck in continuous consumption loops of content generated on the fly to respons to specific requests and desires

Increasingly obscure, violent and fringe tastes in the opoulation due to overconsumption of content

Abundant custom goods end of mass manufacturing

counter -ai cults of groups living off grid in luddite anti-tech communities

Highly wealthy completely closed off from masses

Low fertility has cascading effects

Massive dominance of china, eu and us versus the rest of world

South China Morning Post: "Shenzhen's 'Solo-Corp' District Hits 1M Single-Owner Businesses Running AI Workforces"

Le Monde: "EU Introduces 'Digital Detox Centers' as Content Addiction Cases Surge"

BBC: "Investigation: The Rise of 'Extreme Content Pods' - How AI Creates Underground Entertainment"

Folha de Sao Paulo: "Amazon Neo-Luddite Communities Grow as Tech Refugees Seek 'Authentic Living'"

@COMPACT: "Latest youth phenomenon: 'Content Caves' where teens spend weeks consuming personalized AI entertainment"

The Guardian: "Study: 60% of Global Entertainment Now Generated Real-Time by AI for Individual Users"

Economic Times (India): "Bangalore's 'Agent Entrepreneurs' Managing AI Workforces Larger Than Traditional Corporations"

@NewYorkPost: "Man Marries His AI Assistant's Digital Twin's Cousin's Roommate - Says 'It's Complicated' on Facebook"

Bloomberg: "Billionaire Builds Private Island to Escape AI, Immediately Installs Smart Home System"

Vice: "Underground 'Reality Clubs' Where People Experience the Thrill of Unfiltered, Non-AI Conversation"

Deep-Dive Worldbuilding Exercise

Are there any major accidents (cause more than 300m in damages)?

How has AI affected wealth & income distribution within and between countries?

How has the entertainment industry adapted to this level of AI capability? What new forms of media or art have emerged?

Supply chain?

Scenario 1

Failed ROI in AI leads to souring investor confidence and tech market downturn

Climate worsening, as demand pull for green energy from AI not sustained

No major accidents leads to worse safety outcomes

Strong response from IP lobby to impose protectionism and litigious behavior towards ai training further entrenches large corporates

Scaling labs maintain oligopoly as incumbents fail to maintain funding

No major shifts ot inequality

Autarky

Human data industry

Private dataset market

Compute market

Decline in customer service

Insurance for personal fraud

r/wallstreetbets: "Bought the AI hype, now living in my Tesla (which isn't even self-driving)"

The Information: "Inside the IP Wars: How Legal Battles Are Killing Innovation"

Scenario 2

We see significant market consolidation around two or three key players int traditional industry where capital-ai-data positive recursive loops are found that drive firms with first mover advantage and technical edge to dominate against competitors - industries like rare earth mineral discovery, logistics

Increasing fragility and overoptimisation in the supply chain may lead to fragility as an optimized sectors clash with those where diffusion has yet to catch up (eg. car manufacturing)

White collar unemployment leads to societal unrest particularly in france

Severe Ai industrial accident causes more than 100m in damages

This leads to massive investment in compliance and insurance around AI deployments which in turn forces more disclosure from ai firms

Reshaping of Corporate Structures: As AI systems become more integrated into business operations, the traditional hierarchical structures of corporations may undergo significant changes

High inequality in AI diffusion across countries

Highly economic development counties have much higher integration than others

Human oversight still critical

Anti-trust tech

Regulation havens

Wall Street Journal: "Insurance Premiums for AI Systems Spike 400% Following Shanghai Port Incident"

Forbes: "The New Corporate Elite: How Three Companies Came to Control 80% of Global Logistics AI"

The Onion: "Man Proud He's Being Laid Off By Premium AI Instead of Basic Model"

The Beaverton: "Man Who Lost Job to AI Now Works Teaching AI How to Do His Old Job Better"

@BBCBreaking: "EU Announces 'AI Transition Fund' for Displaced White-Collar Workers"

Scenario 3

AI believers are confirmed yielding moral panic, transhumanist cults and luddite groups flourish

Economic gains still not realized and overhang of capabilities versus societal change

Massive increase in military spending on ai offense capabilities and lethal autonomous weapons

Initial deployments and political assassinations performed by autonomous systems

Mass unemployment of medium skilled workers

Capital - labor income disparity dangerously high leading to pseudo-federalist structure

Rise in spiritualism and meaning crisis

The Guardian: "Neo-Luddite Movement Claims Responsibility for Server Farm Attacks"

@COMPACT: "Latest youth phenomenon: 'Content Caves' where teens spend weeks consuming personalized AI entertainment"

Fox News: "BREAKING: 'Children of Silicon' cult members arrested after mass chip implantation ceremony"

@ResistanceDaily: "Join the Human Workers Alliance - Because algorithms don't need lunch breaks, but your family needs to eat"

The Economist: "The 0.001%: How AI Ownership Created a New Feudal Class"

Vice: "Underground 'Reality Clubs' Where People Experience the Thrill of Unfiltered, Non-AI Conversation"

Attendee 5B

General Worldbuilding Exercise

Compute cost and provision: format Economist Articles

Assumptions: Massive demand for compute, getting stronger in more radical scenarios.

Scenario 1

Microsoft’s Second Meltdown at Three Mile Island
When Microsoft reopened Unit 1 at Three Mile Island in 2025, it expected the demand for electricity to power its data center that it was building nearby to be high, riding the wave of AI hype of 2024. Realistic expectations unfortunately made the prospect less appealing, as the AI industry certainly expanded but low-compute inference led to less demand, the AI Cold Snap of 26-27 when many of overhyped AI startups failed at the same time as the data center opened, and the modern era of distributed local inference has favored other designs. The data center was mainly useful for massive training runs: valuable to Microsoft’s foundation model work, but not the economic boon expected. To make things worse, the new federal environmental regulations on cooling systems coincided with upheaval in the technical management of the site.

Scenario 2

Smarts as a Service at Silicon Jökul
Oldtimers may remember SaaS standing for Software as a Service, but these days it is Smarts. Not intelligence, mind you, we are all too familiar with how our helper agents, robolawyers and muses go off the rails at inconvenient moments, but they certainly provide enough smarts for most jobs and everyday applications. Running SaaS on the other hand has favored the growth of massive data centers where AI runs, as well as reliable global communications - the satellite constellations are almost as essential for Smarts as the data centers. The companies providing SaaS - old incumbents like Amazon, Microsoft and Google and newcomers like TatAI and Countenance - have been splitting their attention between the direct compute/comms aspect and acting as platforms for pre-trained AI. Increasingly newcomer SMEs have struck gold in training specialists and finding clever ways of deploying them - often by exploiting cross-border jurisdictional differences. The US loose regulatory regime allows developing medical AI that is officially not allowed in the EU but widely used by aging Europeans, while the UK is selling legal services to the despair of many bar associations. However, the real bottleneck is building data centers, no matter where the applications are. This is why Iceland and Hokkaido are benefiting from a boom in geothermally powered and arctically cooled data centers. We take a look at the Krafla data center project, financed by TatAI.

Scenario 3

Administration, the final frontier of computation?
It is a truth universally acknowledged, that an AI company with a good idea must be in want of a planning permission for its data center. The approval process of data centers has become the major bottleneck worldwide, with companies from Shanghai to Los Angeles to Guatemala griping about the time it takes to get the robot shovels in the ground. Once it was capital, then energy, then actual construction, being the bottleneck, but now it is universally bureaucracy. Even laissez-faire jurisdictions like the UK where the government explicitly has pre-approved nearly all construction as being in the national interest there is still normal paperwork that needs to be filled out. No matter that the AI lobbyists, lawyers, environmental evaluators, engineers and application writing experts run in other data centers at lightspeed, paperwork is often still literal paperwork. Even if it gets re-scanned and handled by administration increasingly using off-the-shelf administration AI. The main reason has been the massive rise of automated litigation, the exponential growth of planning requirements and the interest in doing aggregated preference elicitation for all potential stakeholders - ironically a result of the AI revolution. Indeed, some economists argue that this complexity is entirely balancing the efficiency increases from AI. When a typical application documentation runs in the terabyte range it is hard to dispute this.

Legal cases:

Scenario 1:

Scenario 2: is it negligence of the human to not recognize a dangerous product being the result of an AI run design process?
Organized crime ring doing identity theft at scale using AI workers - stole from 1.4 billion accounts, causing EU economic crash.

Scenario 3: shareholders sue company over not using more AI,

Deep-Dive Worldbuilding Exercise

Algorithmic improvements causing jumps in capability

Scenario 1 can transition into Scenario 2 and 3 by a gradual improvement of AI reliability. There is a set of theorems (starting with von Neumann) showing that computing/communication/quantum computing/etc. is possible with arbitrary low error frequency by employing redundancy if one is below a critical error rate in the basic components. This may apply to AI agents: if AI becomes good enough despite being imperfect, it can be strung together into bigger systems with lower mistake rates, and this leads to a rapid jump in reliability (at the price of more compute cost, at least at first). This makes the suitability of AI for different tasks and jobs change quickly, although different domains may still have different basic error rates.

Rise of the agent chains: modular systems that perform tasks become rapidly capable. Still requires understanding how to set them up (replacing prompt engineering as a key human skill in Scenario 1 -> 2, and replacing human integration as key skill in Scenario 2 -> 3) and defining the tasks well.

Algorithmic improvements may make performance increase faster than expected. In particular, improvements in selecting good training data appear to enable far more data efficient training, which either equates to higher performance foundation models or that new players can use “smaller data” rather than “big data”.

The fall of the big data titans: new methods make incumbents less powerful because of their hold on data and compute. However, talent may remain important key resources… except that as soon as AI can help replace that, the advantage disappears (except, of course, if scaling up the pool of AI researchers actually equates to market clout).

Brain emulation wildcard: scanning and simulating nervous systems is rapidly becoming more feasible, but is currently very labor intensive. This might get automated with AI and lab automation, enabling some fast improvements of computational neuroscience. While it is rather unlikely human level emulation can be achieved within 5 years even in Scenario 3, the ability to scan and interpret brain data effectively may become a relevant input for setting human-like value functions for AI agents, with significant impact on AI alignment.

Biomorphic systems: using machine learning and automated research, design and manufacturing systems inspired by or mimicking biology in terms of resilience, adaptability and flexibility become part of production.

Nonhuman Legal Subjects

Identity management becomes much more important if AI agents can reliably imitate people in general, and individuals in particular. In order to safeguard against astroturfing, counterfeit people, advanced spearphishing and fake employees systems of proving that one is a human and a particular legal subject, or a particular AI system in a legal context, becomes essential. Domains where identity management is weak will be overrun with fakery and manipulation (not necessarily driving users away to the real world or better domains, but certainly driving away much business).

The EU mandates that online entities need to indicate their status:

human with proven identity

pseudonymous human with hidden identity

software or corporate agent with proven identity being a legal subject

software or corporate agent with proven identity not being a legal subject

Pseudonymous software or corporate agent with proven identity

other.

Civil libertarians in the UK and US decry this as an invasion of privacy. China implements its own standard without the pseudonymous versions, aggressively marketing it to allied nations.

B2B AI crime: while there is money in stealing from individuals, there is even more money in financial fraud, hacking AI to buy your product, and other forms of crimes or bad behavior where humans are not victims, but AI or companies.

Scalable totalitarianism: AI enables surveillance with greater scaling.

Scenario 1: Surveillance of text and imagery can be done at scale, allowing intelligence agencies to backtrack from an event and find people involved. In open societies this can be used for better law enforcement, in closed societies it can be used to find anybody involved in the event. Infiltrating the society with agents under false identities becomes hard.

Scenario 2: The state can place an AI agent to trace every single subject, monitoring them for bad behavior and pooling their knowledge. This enables both detection of ongoing plots, mapping the entire social network of the country, and perhaps proactive manipulation and incentive setting to align people’s behavior. False positives and negative errors do occur, causing accidental problems - whether this matters depends on how acceptable such mistakes are to the regime.

Scenario 3: As scenario 2, but now done very competently. Planning an uprising without being found out is essentially impossible, making such totalitarian states invulnerable to overthrow… as long as they also monitor the leaders who might be in positions to exploit the situation. In addition, AI run drones might enable guaranteed loyal enforcement of decrees with little worry that human military objects or rebels.

Note that such systems may still have security weaknesses that can be exploited. Systems supplied by greater powers to allies may have backdoors allowing them to be subverted, and such backdoors may be accidentally or deliberately revealed to other parties.

Attendee 5C

General Worldbuilding Exercise

News headlines or social media feed: A collection of fictional news headlines or social media posts that might appear in this future world. These can provide snapshots of current events, concerns, and cultural attitudes.

Assuming there is more AI-generated content than human-generated content, is endorsement/backing the main contribution of human workers? Do existing gatekeepers (e.g. organisations, newspapers, experts, social networks) with strong reputations and/or new gatekeepers (e.g. builders of these AI tools) become more powerful?

Scenario 1:

Advertisers ask Google to act on AI slop

5 tips to convince your boss that AI shouldn’t replace you

French government announces national information repository

Scenario 2:

EU Anti-misinformation Act requires member states to create two internets

Business travel surging as CEOs prioritise face-to-face meetings

Internet archive signs 2 billion deal with OpenAI

Scenario 3:

Hack on academic publisher puts 20 years of biology research at risk

Scenario 1:

EdTech sector valued at $300 billion thanks to AI learning tools at second level

“Teaching training never prepared me for this” How teachers are struggle to adapt to AI in the classroom

EdTech giant launches personalised maths tutor for struggling students

Scenario 2:

Private school students race ahead of public counterparts as AI takes hold of the classroom

“Get back to work”: Education minister rejects teachers’ redundancy, pay and training demands

PISA results show Singapore AI strategy paying off

Kids spend more time in front of screen than with their teacher, says expert

Scenario 3:

History and geography removed from English curriculum as 3Rs prioritised

Bill Gates Foundation donates $1 billion for global digital teacher programme

Deep-Dive Worldbuilding Exercise

Scenario 1

0.3pp labour productivity growth annually due to AI

No occupation entirely automated

Companies have invested somewhat in data collection/infrastructure but many have tried and failed

There has been underinvestment in other technologies

AI imbedded in everyday software - many use unaware of the underlying processes

Some embarrassment about the post-Covid LLM hype

Scenario 2

0.6pp labour productivity growth annually due to AI.

A handful of occupations have been entirely automated, possibly including some clerical jobs, customer services and some manufacturing/logistics jobs.

In most occupations, a performance divide between those who use and those who don’t.

For all workers, surveillance increases.

Companies have invested heavily in data collection/infrastructure, but adoption is still very uneven between sectors, regions and countries - this limits how labour gains result in overall economic growth, but AI has still been a welcome boost in the context of the last few decades of slow growth.

White-collar workers find ways to restructure their work and convince their bosses they are too hard to replace.

Work with machinery is much more likely to be done at a distance. Work-related accidents decline.

Some blue-collar work requires more training, some requires less.

Scenario 3

0.9pp labour productivity growth annually due to AI.

A handful of occupations have been entirely automated, including some clerical jobs, customer services and some manufacturing/logistics jobs.

In most cognitive occupations, a performance divide between those who use and those who don’t. Some interpersonal conflict/resentments as well, and perhaps between younger and older colleagues.

For all workers, surveillance increases.

Some companies have overhauled existing systems to accommodate and facilitate AI.

Adoption is relatively even across sectors, regions and countries, with labour cost savings generate consumption generating significant economic growth.

White-collar workers find ways to restructure their work and convince their bosses they are too hard to replace. They do the work AI can’t. This is mentally draining work which requires breaks between tasks. Unions are fighting for breaks/shorter working day. Their constant need to reinvent their jobs is draining and demotivating.

Work with machinery is much more likely to be done at a distance. Work-related accidents decline.

Some blue-collar work requires more training, some requires less.

Appendix - All Worldbuilding Writeups: Part 1