Two truths and a lie: there's more to the AI apocalypse than you think

A few days ago, all of our feeds were flooded with AI-generated pictures following the 80s trend. Unlike the Ghibli trend from a few years back, this time, the backlash against this was fast, furious, and on a bigger scale, as more people are now aware of the environmental cost and privacy risks.

While a pushback from anti-AI activists, environmentalists, and researchers is expected, the conversation has now shifted to include voices we didn't think we'd hear.

Panic in the machine

On 8 September, Jacob Coxon resigned from Anthropic warning that OpenAI and Anthropic were “racing straight to self improving superintelligence” and “gambling with our lives.” Anthropic alignment researcher Evan Hubinger estimated a greater than 10 percent chance that AI could kill all humans within a decade. Other researchers, including former Google DeepMind scientist Alex Turner, Anthropic's Samuel Marks and OpenAI researcher Jason Wolfe, also warned about the pace of development. And then the CEOs also joined the discussion. Anthropic CEO Dario Amodei called for a slowdown. OpenAI CEO Sam Altman agreed that the frontier should be paced. Elon Musk and Meta CEO Mark Zuckerberg also backed Amodei. The "Big 4" of AI - OpenAI, Anthropic, Google, and Meta all came out with the same statement.

At first glance, this sounds like a simple story about scientists becoming worried about their own technology. But there is another part of the story that is harder to ignore. The AI companies warning us about the future are also involved in one of the most expensive expansion races in the history of technology, which needs enormous amounts of money to keep that race going.

The economics behind the warnings

Training a frontier AI model is no longer simply a matter of hiring researchers and buying some servers. The companies need enormous computing capacity, data centres, chips, electricity and networking infrastructure. And every new generation of models requires more of it. OpenAI's own projections show just how extreme this has become. According to a recent presentation seen by the Financial Times, OpenAI expects to spend about $856 billion on computing power and infrastructure between 2026 and 2030. Its revenue is also expected to grow dramatically, from about $36 billion in 2026 to $350 billion in 2030.

But there is a problem. Even with that extraordinary revenue growth, OpenAI projects to spend more than the cash flows in. So, they are looking for more money. The company is reportedly in discussions with investors for funding. This is not a company preparing to slow down. It is a company preparing to spend on a scale that requires the financial markets to believe in its future. OpenAI itself describes how the flow is working. More compute produces more capable models; better models attract more users; more users generate more revenue; and that revenue can then be reinvested into more compute.

OpenAI is not alone. Anthropic has also been expanding its computing infrastructure at extraordinary speed. In February, Anthropic raised $30 billion at a $380 billion valuation, saying the money would support frontier research, products and infrastructure. In April, Anthropic announced an agreement with Amazon that would secure up to 5 gigawatts of new computing capacity and involve more than $100 billion in AWS technology commitments over ten years. Amazon also announced another $5 billion investment in Anthropic. Anthropic is also expanding its use of Google's specialised AI chips.

And the financing is not limited to equity. In June, Apollo, Blackstone and Broadcom announced about $35 billion in debt to support computing infrastructure tied to Anthropic's expansion.

So the pattern is clear. The AI companies are not preparing for a slowdown. They are preparing for a much bigger AI industry. And they need more cash flow to keep their expansion going.

This is where the existential-risk debate becomes interesting. The companies are telling the public: AI development should be slowed down. At the same time, they are telling investors: AI is going to become extremely big. If investors believe that frontier AI will become one of the most important technologies in history, they may be willing to finance enormous losses today in expectation of much larger revenues tomorrow.

And the companies need that belief.

Who gets to write the rules?

The AI industry is also increasingly asking governments to regulate powerful AI. That sounds reasonable. If a technology could cause enormous harm, governments should obviously have a role in managing those risks.

But there is another question: Who gets to decide what the rules look like?

The largest AI companies have billions of dollars in capital, enormous computing infrastructure, relationships with cloud providers and access to policymakers. If they influence what the policy will look like, it will benefit them over the small companies and let them control the industry. And they are lobbying governments over the very rules that will shape their industry.

At least in the US, they have already succeeded in doing that. They are getting unusually close access to those making the rules. Sam Altman, for example, has held private and high-level discussions with White House officials, lawmakers and national security officials about OpenAI’s technology and the future of AI. Senior executives from OpenAI, Palantir and Meta were commissioned as lieutenant colonels in the U.S. Army Reserve’s Executive Innovation Corps, The US government is also simultaneously trying to accelerate AI development, secure access to advanced chips and maintain technological advantages over China.

This is why some people inside the technology industry have questioned the push for AI regulation. Aidan Gomez, CEO of Cohere, described industry coordination on AI safety as “cartel behavior.” Because when major AI companies coordinate on safety rules, they could end up setting standards that make it harder for smaller competitors to compete.

Andrew Ng, founder of DeepLearning.AI, has similarly argued. “If these regulations on AI are passed, such regulations will make it so much harder for almost any company and country to access cutting-edge AI technology,” Andrew said. “Some companies that would rather not compete with open source are lobbying government under the guise of ‘safety’ to pass regulations to stifle it,” His words make sense because Reuters reported in July that US firms including OpenAI and Anthropic were pushing restrictions on Chinese models amid accusations involving model distillation and access to advanced chips.

Senate Democratic Leader Chuck Schumer also urged the Trump administration to work with members of Congress and industry leaders to implement guardrails and block China from accessing advanced US AI chips.

The argument is not that AI regulation is inherently bad. It is about who influences the regulation and who benefits.

The concerns around AI are not new. Researchers and critics have warned for years about job displacement, surveillance, warfare, misinformation, privacy and the environmental cost of running increasingly powerful systems.

Even now, some of the strongest criticism is focused on harms that are already happening, rather than a hypothetical future where AI destroys humanity. Computer scientist Timnit Gebru, one of AI's fiercest critics, described the machine god narrative as something “meant to distract us” from harms that are already here. Nvidia CEO Jensen Huang also rejected the statement. He said, “2030 is not going to be the end of the world. There is 0% chance that's going to be the end of the world” Huang suggested AI firms talking about a "slow down" could be seeking liability protections for harms their products may cause. "They're actually not asking for more laws. They're asking to be relieved of the laws we do have, and I think that that's a problem."

So, why now?

This is where the timing becomes difficult to ignore.

The companies warning about the dangers of increasingly powerful AI are also entering a period of extraordinary financial expansion. OpenAI has raised billions while projecting hundreds of billions in spending. Anthropic has secured massive investments and computing commitments. And Anthropic and OpenAI are both reportedly preparing for IPO. The more powerful and unavoidable AI appears, the more governments may be willing to fund it, regulate it, build infrastructure for it and treat its development as a national priority. And the more valuable that future appears to investors, the easier it becomes for companies to justify the enormous amounts of capital being poured into the industry.

And perhaps that’s why the question is worth asking: who benefits from the panic?