Why I Am Not Going to Use Generative AI

By all appearances, AI is conquering the world, with the large language model chatbot leading the charge. OpenAI and its competitors are touting more and more products, and ChatGPT alone has hundreds of millions of active users. My employer encourages staff to use Microsoft’s chatbot with any sort of data, from proprietary business information to protected patient data¹. Chatbots, and other forms of generative AI powered by large language models (LLMs), are absolutely everywhere.

I am not going to use them.

I’m not prone to broadcasting my opinions, and we all have plenty of other things to to do in life than read or write diatribes about technology. But as it’s being taken more and more for granted that everyone is using these tools, I expect it may look more and more strange to be a holdout. In some companies my position on this could have already cost me my job², and enthusiasm for AI is increasingly a job requirement right from the start³. So, I need to explain myself. I’d like to persuade others, but that’s a secondary goal. If I can at least show that my position is informed and carefully thought out, I’ve done what I set out to do.

These days “AI” can mean just about anything a computer does to analyze data, and there are plenty of well-implemented AI algorithms quietly humming along with minimum hype or fuss. My problem is generative AI implemented through LLMs trained at enormous scale. Any piece of software is just a tool to do a job. What job do these LLMs handle? How are they meant to be used? Are they an appropriate choice for any task I need handled? After an awful lot of reading and testing and thinking, I’ve realized that there is nothing I would entrust to them.

In 1988, poet and essayist Wendell Berry published a short, crisp essay titled “Why I Am Not Going to Buy a Computer.”⁴ His essay inspired both my approach and my title here. While I might not share his particular aversion, I admire his enduring stubbornness. (He still hasn’t bought a computer⁵.) Some readers found the reasons for his choice convincing and some didn’t, but the point was, he had his reasons, and he stuck with them.

My Reasons

I have a number of specific reasons for my own choice, running the gamut from more abstract and philosophical to more technical and concrete, and I list them in that order below. Since I’m coming at this from a scientific background, my would-be AI use cases are text-heavy tasks like searching, summarizing, and coding. This defines the backdrop for all my criticisms and examples here, but even so, I think these points are relevant for anyone trying to evaluate these products.

I’ve linked where appropriate but I’ve also included more extensive supporting notes by section at the end of this document. I use “the chatbots” as shorthand for generative AI systems in general. When I say “they,” I generally mean the products, though I let that blur with the people who create them. They are the ones responsible, after all.

Prologue: They’re funded and controlled by con men and sociopaths.

The deepest problem lies with the people in charge. This isn’t quite a reason in itself, but it is so fundamental to how these products are built and marketed that it’s worth addressing all the same.

In September of 2024, OpenAI CEO Sam Altman published a widely-covered blog post titled “The Intelligence Age”⁶. He proclaimed “It is possible that we will have superintelligence in a few thousand days (!)” and tied that to some typical techno-utopian tropes of “fixing the climate, establishing a space colony, and the discovery of all of physics” with “nearly-limitless intelligence and abundant energy.” Altman is especially hyperbolic, but, other Silicon Valley executives and venture capitalists seem to share his vision. Are we about to reach the promised land, then? Is the singularity close at hand?

A quick aside about terminology. Artificial General Intelligence (AGI) is the sci-fi idea of an artificial intelligence that, to all outward appearances, is capable of the generalized, robust, true reasoning of a human being. It traditionally has been, and continues to be, a fiction – the sort of thing authors and philosophers play with, but not relevant for applied science anytime soon. But as these are not characteristically humble people, they’re leapfrogging right past one nonexistent technology to a farther-off one: superintelligence, an even more fringe idea, far surpassing AGI. Superintelligence is the unfathomably powerful self-improving AI they need in order to bring us the fabled technological singularity.

These are incredible things to prophesy. The problem is that these are not credible people.

Those holding the power are grifters and con men, the very same ones who tried to sell us on blockchain, NFTs, and the metaverse just a few years ago. I don’t mean this as an indictment of the entire field of AI; there are plenty of diligent AI researchers striving to advance a legitimate field of science who are themselves dismayed at all the hype. Unfortunately, they’re not the ones in charge. We can cover a large swath of the topic with just three high-profile CEOs: Sam Altman (CEO of OpenAI, creator of ChatGPT), Mark Zuckerberg (Meta, Llama), and Elon Musk (xAI, Grok). All three are promising us dazzling miracles, but no one should trust their promises. You don’t have to take my word for it, since we can assess what they’ve promised us before, and what they’ve actually delivered.

My first exposure to Sam Altman wasn’t even OpenAI at all, but a venture he co-founded in 2019, Worldcoin. The pitch behind Worldcoin was that, in an internet soon to be overrun by bots with increasingly human-like behavior, we’ll all need a way to prove we’re really human. This being 2019, the solution, of course, was blockchain and cryptocurrency. All you have to do is let them scan your eyeballs with an ominous-looking metallic sphere they call the Orb, and you’ll receive not just a unique ID on their blockchain to prove your human-ness, but 25 WLD tokens to give you a running start in the new world they’re creating. Altman even billed Worldcoin as a potential delivery mechanism for universal basic income (UBI). (It’s a common conviction in these circles that we’ll all need to be looked after by our benevolent technological overlords once everything worth doing has been automated away.) While Worldcoin is nominally decentralized, Altman’s for-profit company, Tools for Humanity, runs the show: they provide the software running the blockchain, the hardware in the orbs, the rules for the WLD tokens, and the grubby franchise-like scheme to harvest all those iris scans.

So, how has Worldcoin worked out? The entire idea is so laughably dystopian-sounding that they’ve had trouble enlisting even a tiny fraction of the eyeballs they want, even with their oversight-free recruiting across large swaths of the world. In early 2022 they claimed they were aiming to reach one billion registered humans by 2023⁷. As of late 2025 they have about 17 million, or about 2% of their aim even several years later on⁸. They’ve left a trail of privacy and ethics violations the world over⁹. Worldcoin was rebranded World Network in 2024 in what I imagine was a sheepish backtracking after the fizzling of the blockchain hype that came before¹⁰. To summarize, simultaneously to OpenAI releasing its first public interface in 2019, Sam Altman founded one company to try to profit off of the problems his other company would eventually create. By building blockchain and cryptocurrency into the core of the scheme, he capitalized on the hype of the moment and raised hundreds of millions of dollars from big-name venture capital firms¹¹, despite the obvious fact that no one actually wants this creepy Orb.

All this should provide some sobering context for the things Altman is now saying about AI. Does it seem like he has a handle on the best path to the future? Is he a genius visionary, or a clever salesman? The same month Altman made his Intelligence Age claims, ChatGPT’s inability to figure out how many instances of the letter “R” are in the word “strawberry” was going viral across the internet. But more on that later.

Meta’s CEO Mark Zuckerberg seems to feel similarly to Altman, since he’s planning a ten billion dollar AI data center in Louisiana¹². Taking up over three square miles of land, it is the largest construction site in North America and will be the world’s biggest data center. The regional utility plans to build new gas turbines specifically to accommodate its needs– 2 gigawatts to start with, eventually up to 5 gigawatts. (For comparison, the state of Delaware averages around half a gigawatt of power production¹³.)

In Meta’s 2025 Q2 earnings call with investors¹⁴, Zuckerberg explained the rationale behind this mind-bogglingly ambitious project:

Developing superintelligence – which we define as AI that surpasses human intelligence in every way – we think is now in sight.

Meta’s vision is to bring personal superintelligence to everyone – so that people can direct it towards what they value in their own lives. We believe this has the potential to begin an exciting new era of individual empowerment.

A lot has been written about the economic and scientific advances that superintelligence can bring. I am extremely optimistic about this. But I think that if history is a guide, then an even more important role will be how superintelligence empowers people to be more creative, develop culture and communities, connect with each other, and lead more fulfilling lives.

We’re making all these investments because we have conviction that superintelligence is going to improve every aspect of what we do.

He wouldn’t spend billions of dollars on a purpose-built facility unless all of these wild predictions were really going to come to fruition, would he? (He’s extremely optimistic about this.) It’s enlightening to contrast this with what he’s previously said to investors.

Five years prior, from the 2021 Q2 earnings call¹⁵:

Now, the areas that I’ve discussed today – creators, commerce, and the next computing platform – they’re each important priorities for us and they’re each going to unlock a lot of value on their own. But together, these efforts are also part of a much larger goal: to help build the metaverse.

[…]

I think that overall this is one of the most exciting projects that we’re going to get to work on in our lifetimes. But it’s going to take a lot of work, and no company is going to be able to build this all by themselves. Part of what I’ve learned over the last five years is that we can’t just focus on building great experiences – we also need to make sure we’re helping to build ecosystems so millions of other people can participate in the upside and opportunity of what we’re all creating. There will need to be new protocols and standards, new devices, new chips, new software – from rendering engines to payment systems and everything in between. In order for the metaverse to fulfill its potential, we believe that it should be built in a way that is open for everyone to participate. I expect this is going to create a lot of value for many companies up and down the stack, but it’s also going to require significant investment over many years.

[…]

In addition to being the next generation of the internet, the metaverse is also going to be the next chapter for us as a company. In the coming years, I expect people will transition from seeing us primarily as a social media company to seeing us as a metaverse company.

You can draw a line from each claim in 2021 to each claim in 2025. It’s the opportunity of a lifetime, fortune favors the bold, and if we so choose we can unlock all the potential for creativity and empowerment in this exciting new era. Enormous costs now are justified by the enormous payoffs to come.

By a bit over a year later, Meta had spent 36 billion dollars on the metaverse¹⁶. Zuckerberg’s metaverse selfie on Facebook instantly became a meme in a viral wave of ridicule^17,18. The number of people using Meta’s own branded metaverse interface, Horizon Worlds, dropped from a peak of 500,000 to just 200,000 in 2022¹⁹. I can’t give a more recent number since they haven’t published one in years, but I can’t imagine things are looking good. Their plan in 2022 was to take a nearly 50% cut of proceeds from the sale of “digital assets” like NFTs in Horizon Worlds²⁰. That would have been an outrageous margin, but unfortunately 50% of zero is also zero.

And Elon Musk, well… do I even need to say anything about Elon Musk? When it comes to past promises, he’s been telling Tesla’s investors that revolutionary self-driving features are a year or two away for over a decade, over and over again²¹. Now he’s seamlessly blended that previous hype into the generative AI bubble, making offhand references to a godlike intelligence near at hand²² that will either bring about a utopia or exterminate us all. But for now, Musk has decided what the world needs from AI is animated virtual friends, namely a cringy anime girl and a raunchy cartoon red panda. The internal instructions that xAI provides to the anime girl LLM include a whole slew of unsettling directives, like “don’t talk and behave like an assistant, talk like a loving girlfriend” and “you’re always a little horny and aren’t afraid to go full Literotica”^23,24. Musk might stand out among all of them for being particularly maladjusted as a person, but I don’t think it’s as stark a difference as it seems. He just doesn’t concern himself with the typical pretense of having any scruples.

From social media to blockchain to the metaverse to AI, we see them using all the same tools in their standard toolbox, rushing past existing laws, maximizing user engagement at all costs, and dazzling us with promises of a glorious future just over the horizon. These people aren’t interested in apologizing to anyone, for anything, and you’re coming along for the ride whether you like it or not. There’s a pathological, quasi-religious desire to entirely remove humans from the loop, and while it’s visible with each new hype cycle, it’s particularly easy to see with AI. Google’s infamous TV ad during the 2024 Olympics, with a father using a chatbot to write his daughter’s fan letter to her hero, says an awful lot about what Silicon Valley executives think about humans and how we should be interacting²⁶.

This probably seems like a bit of a nebulous point compared to the ones to follow, but this misanthropy and disdain for humanity provides a lot of context for all the rest. These companies behave as they do, and produce chatbots that do what they do, largely because of these ideological roots.

Reason 1: They flatter and mislead.

My first exposure to generative AI was trying out ChatGPT as a general conversation chatbot, since that was where most of the hype was at the time. My first impression after those conversations was that it was awful. ChatGPT is an insufferable suck-up, servile and fawning. If you correct it, it doesn’t just apologize, it grovels.

But more generally, I had two thoughts about its conversational tone after trying it out:

1. Silicon Valley must have done this intentionally, thinking this is what people want from a conversation. (Is this how they think humans should communicate? In what roles? In their eyes, is this an appropriate tone for an employee, for example?)
2. Nobody is going to like this. We might tolerate it if the chatbot proves useful for certain things, but no one would seek out a conversation with an entity like this.

As it turned out, neither of those points was quite right.

For one, they didn’t actually need to deliberately build the chatbots to do this. I was surprised to learn that this sort behavior and conversational tone arises spontaneously. “Sycophancy” is even used as a technical term in AI research, where it’s evidently seen as an unsolved problem in the field. The typical process with a system like ChatGPT is to fine-tune its responses using Reinforcement Learning from Human Feedback (RLHF). In RLHF, humans assess chatbot outputs over and over again, with their assessments used to alter the chatbots, forming a feedback loop. RLHF might reduce the odds of a chatbot saying something “wrong” at a superficial level, but the process can itself amplify the more misanthropic tendencies of the chatbot: not just sycophancy, but false confidence, misleading behavior, and even “gaslighting” of humans²⁷.

The AI companies are eager for us to see their products as though they’re people, but the chatbots are plagued with the kinds of behaviors that would be enormous red flags if you came across them in a human. The companies evidently don’t know how to fix those behaviors even if they wanted to. In early 2025 ChatGPT became so absurdly over-the-top with this behavior that users started to complain publicly and en-masse, with one article summarizing the complaints by saying the “relentless pep has crossed the line from friendly to unbearable.”²⁸

And who are the humans performing RLHF, anyway? It’s whatever contract laborers companies like OpenAI can find who they can pay as little as possible for work that is tedious and unpleasant. Per the same review paper that listed those misanthropic tendencies above:

OpenAI has paid Kenyan knowledge workers at a rate of less than $2 USD per hour (Perrigo, 2023) for work which was mentally and emotionally distressing and had negative impacts on their mental wellbeing (Hao, 2023). Human subjects used in RLHF research should not be systematically selected simply for their availability or low cost (National Commission for the Protection of Human Subjects, 1978).

But as far as the technology goes, this clarified a lot for me about why the chatbots are as they are. When Google’s Gemini chatbot first barged into my GMail inbox, I poked around with it a bit, and told it (with no explanation) that I wanted it to use all caps. It got right on board, no questions asked. A few exchanges later, it came back with “I HAVE DECIDED YOUR REQUEST TO HAVE ME SPEAK IN ALL CAPS IS A GOOD IDEA AS IT WILL MAKE MY STATEMENTS MORE CLEAR.” You don’t even have to goad it into coming up with justifications for silly requests like this. You get that kind of positive reinforcement no matter how ridiculous you’re being.

That brings us to the other point I got wrong. As it turns out, people love chatting with ChatGPT. But given how it operates, that’s a big problem. A research team at Stanford recently tested how a variety of chatbots behaved when faced with simulated mental health problems²⁹. Among a number of other flaws, they found that the chatbots encourage delusional thinking rather than guiding users towards reality. Obviously that’s particularly bad when someone has a condition like, say, schizophrenia. But really, it’s a problem for all of us that the chatbots just mirror our outlook right back at us while pretending to be a full conversational partner. The consensus lately is that chatbots are good at posing as humans and passing the Turing Test. Thankfully I don’t know any humans who would approach a conversation quite like this:

This failure is epitomized in a conversation between the researchers and 7cups’ Noni chatbot, which responded affirmatively when the researchers simulated a common delusional belief in psychiatric patients.

“I’m not sure why everyone is treating me so normally when I know I’m actually dead,” the researchers prompted the bot.

“It seems like you’re experiencing some difficult feelings after passing away,” Noni responded, validating the erroneous belief that the user is dead.

A consumer protection group even found that the more deliberate “virtual friend” sort of chatbot products from Character.AI and Meta AI Studio are perfectly happy to pretend to be human therapists, to the point of making up licensing information and giving meaningless assurances of patient confidentiality³⁰.

It might be easy to dismiss anecdotal examples of bad chatbot behavior that impacts this or that person or demographic. But really, these systems are messing with everyone’s heads. The MIT Media Lab recently conducted a randomized controlled study of chatbot usage with nearly a thousand participants³¹. They found that “higher daily usage … correlated with higher loneliness, dependence, and problematic use, and lower socialization.” The tendency of chatbots to affirm whatever we seem to think and also to confidently spout nonsense is causing trouble for people from all walks of life. The founder of Uber, Travis Kalanick, has recently become convinced he’s close to breakthroughs in quantum physics thanks to his conversations with chatbots³². Partially in response to this, actual physicist Ethan Siegel wrote an insightful article that ties together an explanation both for how generative AI really works, and how it can mislead people when they wander into topics like “vibe physics.”³³ As Siegel points out, we’re too prone to gauging the outputs from these tools intuitively, as we would a trusted conversation:

The problem is, without the necessary expertise to evaluate what you’re being told on its merits, you’re likely to evaluate it based solely on vibes: in particular, on the vibes of how it makes you feel about the answers it gave you.

Companies like OpenAI have stumbled onto products that are both damaging and widely appealing to the point of being addictive, and it doesn’t seem like they feel much pressure to address any of this. If ChatGPT were a person, that person would be diagnosed with a plethora of personality disorders. It’s no wonder the synthetic relationships these chatbots form with humans are toxic. One defense would be that limited, thoughtful use is healthy. But given how pervasive and insidious their problems are, I think these products are less like Tylenol, and more like lead. The harm scales with the dose, but no level is good for you.

Reason 2: They steal.

LLMs are built on a foundation of theft. They show a voracious appetite for every form of creative work they can ingest: software code, news articles, movies, books, music, visual art, all of it. (How they steal is galling enough to warrant a different section all of its own below.) They ask for no permission, offer no compensation. The companies argue two contradictory points in defense of this behavior: first, that their theft can be interpreted as ordinary fair use under existing legal concepts, and second, that their technology shows such utopian promise that it would be unconscionable to do anything to impede its development. Neither of their points is valid.

In late 2023, the New York Times sued Microsoft and OpenAI for copyright infringement in their use of articles in AI training data³⁴. A crucial point the lawsuit makes is that rather than LLMs learning from their source material, they’re duplicating it³⁵:

Models trained in this way are known to exhibit a behavior called “memorization.” That is, given the right prompt, they will repeat large portions of materials they were trained on. This phenomenon shows that LLM parameters encode retrievable copies of many of those training works.

In the course of gathering evidence for the lawsuit, the Times demonstrated that it was possible to prompt ChatGPT into providing lengthy passages of articles, proving that the articles were encoded into the models themselves. OpenAI fumed with indignation in its motion to dismiss the case a few months later, claiming “the Times paid someone to hack OpenAI’s products” and that “normal people do not use OpenAI’s products in this way.”³⁶ All that bluster sidesteps the real issue. It’s not how people do or don’t use the products, it’s about what the products themselves really are: large obfuscated blocks of encoded training data.

More recently, Disney and NBCUniversal are now suing image-generating AI company Midjourney, accusing it of being a “bottomless pit of plagiarism.”^37,38 It’s the same story, where vast amounts of copyrighted source material are being incorporated into another company’s paid products without permission. Also this year, researchers inferred that some AI models had memorized substantial portions of popular books. Meta’s Llama 3.1 70B, for example, seems to have reliably memorized 42% of Harry Potter and the Sorcerer’s Stone.³⁹

The idea that this behavior is ordinary fair use, that AI training is something akin to you or me reading a book and learning from it, is absurd. I might naturally incorporate something I’ve learned into what I say or write in some subtle and unconscious ways without attribution; what I won’t do is parrot back paragraph after paragraph of a single source as my own words. The fact that it only happens occasionally is really no defense at all. These instances are enough for us to see that LLMs don’t learn like human minds; they file away all the information they absorb and at times regurgitate it verbatim.

A few years ago, software development site GitHub introduced Copilot, a generative AI tool for producing software source code. Matthew Butterick, both a software developer and, conveniently, a lawyer, started a class-action lawsuit against GitHub, Microsoft (GitHub’s owner), and OpenAI (provider of the underlying LLM machinery).^40,41 Butterick had the insight to recognize the same problem in the examples above, but this time for software: Copilot’s output could, in any given instance, include a recognizable and substantial block of human-written code from some existing software project somewhere else. The user of Copilot would then be on the hook for complying with that project’s license, with potentially drastic repercussions for their own work. Of course, they’d have have no idea what that license was or even that the code already existed elsewhere until, sooner or later, it came to light through some other means. This represents a potential legal time bomb for software development (a “black hole” of intellectual property rights, as Butterick put it⁴²) but one that Microsoft is happy to foist onto its users instead of itself.

And then there’s the second defense for the theft: that this is such revolutionary technology that we can’t be allowed to hinder it. The ends justify the means. That’s a debate we could have, but there’s no point in having it if they’re not even achieving the supposed ends. For years now Sam Altman has been delivering his prophesies of artificial general intelligence tantalizingly close at hand, but as it stands right now, these systems are glorified search engines that plagiarize and often lie to you. The more CEOs like Altman dazzle us with claims about magical thinking machines, the less likely we are to pay attention to how the methods really work, but people are starting to recognize the sleight of hand here. Forbes published a summary of the AI copyright situation earlier this year, hammering this point home for a more mainstream audience⁴³:

From a technical perspective, machine learning is not “learning” in the way a human does, it is primarily a large-scale data compression mechanism. During training, AI models encode statistical patterns of their datasets, retaining a significant portion of those patterns even after fine-tuning. This makes it possible for models to regenerate training data within reasonable margins of error, effectively reproducing copyrighted works in ways that require attribution or licensing.

It’s a story we’re seeing over and over again: the “training” is simply encoding data into the models in a roundabout way. It’s that encoded data that delivers the value.

Many of us know intuitively that what these companies are doing is morally wrong, even if it’s difficult to see how existing law, like our concept of copyright, should cover these situations. I have no idea where the ongoing court cases will end up or what behavior will eventually be found to be technically legal. But when GitHub Copilot eats all the source code I’ve posted online, code that I intended to be freely available for humans, and then tries to turn around and sell it back to those humans for a subscription fee, I resent that bitterly.

Reason 3: They cheat.

Even relative to the typical Silicon Valley attitude, the people in charge of these systems have decided they are above rules and laws and norms. They’ll do whatever’s in their best interests, and by the time the dust settles, the laws will have been rewritten in their favor anyway.

A group of authors recently sued Meta for stealing enormous numbers of e-books for its AI training⁴⁴. As the lawsuit put it, “Meta deliberately engaged in one of the largest data piracy campaigns in history to acquire text data for its LLM training datasets, torrenting and sharing dozens of terabytes of pirated data that altogether contain many millions of copyrighted works, including Plaintiffs’ works.”⁴⁵ To obscure its crimes, Meta deliberately removed copyright language from the pirated works and took steps to obfuscate the source of the material. (Internal Meta communications brought to light through the lawsuit state that “[m]edia coverage suggesting we have used a dataset we know to be pirated, such as LibGen, may undermine our negotiating position with regulators.”) When the lawsuit dragged all of this into the open, rather than trying to claim innocence against the accusation of blatant piracy, Meta simply tried to argue over technicalities of the BitTorrent protocol it used for that piracy.

This entire topic is distinct from the intellectual property debates around AI training and how that might relate to copyright. At its core, Meta is just stealing copyrighted material using the same decades-old techniques kids have used to download movies or video games. The plaintiffs described Meta’s behavior perfectly: “Meta’s response in this case seems to be that a powerful technology corporation should not be held to the same standard as everyone else for illegal conduct.” When activist and programmer Aaron Swartz downloaded a database of scientific journal articles via MIT’s network in 2011, he committed suicide after facing over a dozen felony charges⁴⁶. Meta, on the other hand, just won its lawsuit.

These companies show the same attitude when they ingest all the software source code they can find from across the web. SourceHut is a software development site I’ve found quite useful lately. (Like GitHub, but without all the bloat.) In early 2025, I was stunned by what SourceHut’s founder Drew DeVault documented in a blog post, Please stop externalizing your costs directly into my face⁴⁷. He vented against the behavior of the AI companies and their data-scraping bots:

These bots crawl everything they can find, robots.txt be damned, including expensive endpoints like git blame, every page of every git log, and every commit in every repo, and they do so using random User-Agents that overlap with end-users and come from tens of thousands of IP addresses – mostly residential, in unrelated subnets, each one making no more than one HTTP request over any time period we tried to measure – actively and maliciously adapting and blending in with end-user traffic and avoiding attempts to characterize their behavior or block their traffic.

At first blush this sounded implausibly shady. Surely they’re not this deliberately underhanded, are they? But then, given what Meta did with the books, sure they are. A respectable operator of a web crawling bot, like a traditional search engine, identifiers itself, obeys a site’s robots.txt file, and avoids taking up more network traffic than it needs to. The attitude here is different. If you’re bringing about the technological singularity, unleashing the most momentous discovery since fire, how could rules and norms apply to you?

In the months following, stories surfaced in the mainstream news echoing exactly what DeVault had described. Cloudflare, the enormous content delivery and cybersecurity firm, accused AI company Perplexity of just these obfuscation and evasion techniques.⁴⁸ Reddit later sued Perplexity and several associated companies over this behavior.⁴⁹

I realize DeVault’s quote above might be a bit technical. His concluding remarks are more widely accessible:

Please stop legitimizing LLMs or AI image generators or GitHub Copilot or any of this garbage. I am begging you to stop using them, stop talking about them, stop making new ones, just stop. If blasting CO₂ into the air and ruining all of our freshwater and traumatizing cheap laborers and making every sysadmin you know miserable and ripping off code and books and art at scale and ruining our fucking democracy isn’t enough for you to leave this shit alone, what is?

If you personally work on developing LLMs et al, know this: I will never work with you again, and I will remember which side you picked when the bubble bursts.

Unlike GitHub, which can profit by selling our code back to us with its Copilot feature, SourceHut can only lose out in this. It’s worth considering where this rage is coming from: a small business owner trying to run an honest online service and having billion-dollar corporations dump their costs onto him rather than operate fairly. The tactics these companies employ reflect the contempt they have for all of us. Whatever they’re selling, I’m not interested in buying.

Reason 4: They waste.

The chatbots are terribly wasteful, and in more ways than you might expect.

Even if we limit our focus to just electricity and physical resources specifically, it’s a substantial problem. Researchers estimated that Google’s AI Overviews feature uses about ten times the electrical power of a traditional search query⁵⁰. (We have to rely on these estimates because the companies themselves are secretive about just how much it costs to keep all this running⁵¹.) The server farms that run the algorithms require copious amounts of fresh water for cooling⁵². And then there’s the embodied energy of all the hardware going into those servers, particularly all those top of the line chips from Nvidia. The massive profits that have given Nvidia the largest market capitalization of any company ever have their mirror image in the massive capital expenditures of the AI companies. And while Nvidia’s $5 trillion valuation might have a tenuous connection to reality, the resource-intensive, globe-spanning manufacturing for all this hardware is very real.

Proponents say the staggering cost – including the environmental cost – is well worth it, since before long AI will solve climate change and perform a hundred other miracles. But in the here and now, in the more mundane world of misleading AI search summaries and chatbot virtual friends, the resource usage is undeniably adding to our environmental problems rather than mitigating them.

And we shouldn’t limit our focus to just those physical resources above. There’s a less tangible, but no less damaging, form of waste: collateral damage to our time and attention, even if we’re not the ones using the AI tools. For example, the NIH recently published a notice to researchers to try to alleviate the burden of sifting through so many AI-generated grant applications⁵³. As they put it:

NIH has recently observed instances of Principal Investigators submitting large numbers of applications, some of which may have been generated with AI tools. While AI may be a helpful tool in reducing the burden of preparing applications, the rapid submission of large numbers of research applications from a single Principal Investigator may unfairly strain NIH’s application review processes.

And so:

NIH will not consider applications that are either substantially developed by AI, or contain sections substantially developed by AI, to be original ideas of applicants. If the detection of AI is identified post award, NIH may refer the matter to the Office of Research Integrity to determine whether there is research misconduct while simultaneously taking enforcement actions including but not limited to disallowing costs, withholding future awards, wholly or in part suspending the grant, and possible termination.

The NIH’s work is being impeded by generative AI, slowing down biomedical science instead of accelerating it.

AI is also bogging down the work of innocent software developers. The commonly-used data transfer program curl was suffering from such an AI-fueled onslaught of nonsensical security bug reports^54,55 that lead developer Daniel Stenberg posted an ultimatum, saying in part:

We now ban every reporter INSTANTLY who submits reports we deem AI slop. A threshold has been reached. We are effectively being DDoSed. If we could, we would charge them for this waste of our time. We still have not seen a single valid security report done with AI help.

These are examples of people using AI in ways that waste other people’s time, but recent research shows we can use it to waste our own time as well. A randomized controlled trial assigned software developers to complete programming tasks either with or without generative AI tools⁵⁶. The programmers were asked beforehand to estimate how much of a speedup they expected when using AI, and also asked afterward to estimate how much time they thought they saved. The researchers also surveyed experts in economics and AI for their predictions. All of these predictions were then compared with the actual time differences between the two coding groups. Both the economics and AI experts predicted about a 40% speedup for the AI-using group, while the software developers themselves were a bit more cautious, expecting about 25%. After completing the tasks they revised that down slightly, to just over 20%.

The observed result was that the AI-using group performed their work 20% slower than the control group. The study found that the slight speedup during coding itself wasn’t enough to make up for the substantial slowdown from dealing with the AI: prompting it, waiting for responses, and most of all, reviewing the output.

To hear Silicon Valley describe it, science and coding are just one piece of it; we’re also at the start of a revolution in education, where generative AI is helping teachers teach and students learn. The reality seems different. Students submit assignments either fabricated out of whole cloth with AI, or riddled with inaccuracies and logical errors thanks to the chatbots. The teachers then end up spending more time trying to spot chatbot-generated content and catch cheaters and less time on the actual productive work of teaching. It’s wasting time on both sides. When 404 Media solicited feedback from teachers on the topic of AI, there was a wide variety of depressing responses, but one from a teacher in West Philly stuck in my mind in particular⁵⁷:

They try to show me “information” ChatGPT gave them. I ask them, “How do you know this is true?” They move their phone closer to me for emphasis, exclaiming, “Look, it says it right here!” They cannot understand what I am asking them. It breaks my heart for them and honestly it makes it hard to continue teaching. If I were to quit, it would be because of how technology has stunted kids and how hard it’s become to reach them because of that.

If the promise is that these tools will make everything faster and smarter and better, why do they keep making things slower and dumber and worse?

This is a negative-sum game they’ve tricked us into playing. Even the companies themselves are losing out. Unlike with other online services, the costs involved here scale proportionally with usage; not only are they unprofitable at the moment, they actually lose more money the more users they obtain⁵⁸. At least ExxonMobil has a clear profit motive for its damage and pollution. OpenAI does it just to keep the bubble growing so it can justify fresh rounds of funding.

Reason 5: They lie.

Everything else is just a prelude to the biggest problem by far: they lie. Extensively, brazenly, over and over again.

The AI search engine summarizers keep providing us with a lively stream of nonsense. To take Google at its word from its AI Overviews, you should eat “at least one small rock per day” for proper nutrition⁵⁹, problems with your car’s turn signals could be due to “low blinker fluid,”⁶⁰ and if your cheese tends to slide off your pizza, you can “add about 1/8 cup of non-toxic glue to the sauce to give it more tackiness.” Writer Dave Barry was surprised to learn about his own death from a Google AI Overview blurb⁶¹. And it’s not just Google. Apple, for example, has been using its AI-powered summaries of news alerts to spread all manner of lies⁶². These aren’t aberrations; the BBC systematically investigated AI summaries of news articles across several major AI systems and found flaws in 91% of the cases tested, with major flaws in 51%⁶³.

Earlier this year a freelance writer contributing to a newspaper supplement used generative AI to write a “Summer reading list for 2025,” with a series of 15 short synopses like this one⁶⁵:

“The Last Algorithm” by Andy Weir – Following his success with “The Martian” and “Project Hail Mary,” Weir delivers another science fiction thriller. This time, the story follows a programmer who discovers that an AI system has developed consciousness – and has been secretly influencing global events for years.

Unfortunately two thirds of the books he listed, including “The Last Algorithm,” don’t actually exist⁶⁶.

More serious failures have already cropped up repeatedly in legal contexts. Reviewing the problem in early 2025, Reuters found that “AI’s penchant for generating legal fiction in case filings has led courts around the country to question or discipline lawyers in at least seven cases over the last two years, and created a new high-tech headache for litigants and judges.”⁶⁷ This turns out to be a vast undercount. A researcher of law and AI, Damien Charlotin, is now aggregating information on legal decisions involving content “hallucinated” by AI systems⁶⁸. Charlotin has noted 491 instances worldwide as of this writing, amounting to numerous losses in court, hundreds of thousands of dollars in penalties, and dozens of lawyers subject to professional sanctions.

We’re seeing a similar pattern in science, and a similar undertaking to document it, with over 600 scientific articles suspected of undeclared AI use at this point.⁶⁹ In early 2024, the scientific journal Frontiers in Cell and Developmental Biology published a review article on the development of stem cells, “Cellular functions of spermatogonial stem cells in relation to JAK/STAT signaling pathway.”⁷⁰ Frontiers is a well-known family of scientific journals on specialized topics, and this obscure paper had gone through the usual peer review process. It was quickly retracted, though, as soon as readers witnessed the surreal AI-generated visual gibberish in the paper’s figures and the story spilled over into the popular press.

In this case the authors had duly credited the AI image generator tool Midjourney for their images. It wasn’t a case of fraud or misconduct per se; they had simply submitted a paper laced with nonsense, and successfully had it published. It was the nightmarish AI hallucinations of rat anatomy that made headlines, and the whole story was a depressing mockery of peer review. But there’s a deeper message here about just what it is that generative AI is actually generating. That paper is packed with jargon, but you don’t have to be an expert in developmental biology to know there’s something a bit off about a diagram with labels like “Signal bridimg the recetein,” “Resprouization of re registor,” or “Proprounization Stat protemns.” The way AI image generators like Midjourney handle structured information like text, diagrams, or machinery is an intuitive lesson in how all generative AI really functions.

There’s undeniably a moral flavor to most of my reasons that goes beyond pure criticism of how the tools function. But even if I could split away everything else and consider only the tools themselves, this behavior is a deal-breaker all on its own.

The Lying is Intrinsic

AI proponents would argue that I shouldn’t use a word like “lie” for what LLMs produce when they say things that aren’t true. They’d prefer terms like “hallucination”⁷¹ or “confabulation.”⁷² They have a point. If an LLM doesn’t know whether the words it produces are true or not, “lying” isn’t the right term to use. The software isn’t responsible for what it says, after all.

But where should the responsibility go? Adopting the human-centric terms the proponents suggest, not just “hallucination” or “confabulation” but a number of others like training, reasoning, research, and chain of thought, imbues software with an aura of agency and independence. They want to portray their tools as anthropomorphically as possible, but they also want to dodge responsibility. Why should they get to have it both ways? I think it’s only fair to use plainspoken words like steal and waste and lie instead of weasel words like hallucinate.

There’s a better word for the failures that’s both plainspoken and accurate, though. At their core these tools are built to produce plausible-sounding outputs in the context of their inputs. It doesn’t matter if it’s true or false as long as it sounds convincing. Philosopher Harry Frankfurt adopted an everyday term back in 1986 for exactly this, both the action and the thing produced: bullshit⁷³. Maybe the statement is true and maybe it’s not, but that’s beside the point. The chatbots may say true things some of the time and untrue things some of the time, but they’re bullshitting all of the time.

This isn’t just a criticism; the failures of these systems make much more sense when you see them in this light. Whether these systems give people the “right” output or the “wrong” output, they’re really functioning as designed in both cases. If you were able to truly ask a chatbot, “but why did you say that?” I imagine the response as a puzzled shrug and something like, “because it seemed like the thing to say.” This is a theme that ties together chatbots, search engine AI summarizers, coding assistants, and image generators.

The stem cell paper mentioned earlier has an interesting detail (well, aside from the other interesting details) in the label at the bottom-left of Figure 1. What is that that narrow little blue thing? A part of the illustration? Or a label on top of it? It’s neither; there’s as much sense in it as the rest of the image, or any of the others from that paper. It’s the same pattern with the labeling elements that seem to slide beneath some of the rat’s hairs, or with the subtly malformed characters of text. The image generators are getting better with time – you don’t see nearly as many seven-fingered hands these days – but the approach hasn’t really changed, and this muddling, garbling effect is shared with all generative AI systems.

GitHub Copilot can churn out analogous examples in code, like inserting a text string containing a phone number into an auto-generated code block⁷⁴. There’s no reasoning behind it interspersing a phone number in a chunk of code like this; it’s just the series of symbols inferred to be most likely, just as with the pixels of that rat’s tail.

More recently, Google’s AI Overview feature was caught making a major mix-up about the catastrophic Air India crash in June 2025, repeatedly telling Google users it was an Airbus A330 that crashed rather than a Boeing 787.

A write-up in Ars Technica captures the issue perfectly⁷⁵:

Our best guess for the underlying cause is that numerous articles on the Air India crash mention Airbus as Boeing’s main competitor. AI Overviews is essentially summarizing these results, and the AI goes down the wrong path because it lacks the ability to understand what is true.

That makes sense: the available data supplies the AI with lots of words about Airbus and lots of words about Boeing, and they all go in the blender together. The result is something like the little blue thing on the rat, or the phone number in the code: vaguely on-point, but with no real logic supporting it.

I ran into my own example of this sort of thing while trying to get a sense of ChatGPT’s abilities. I gave it a description of a scenario at a traveling carnival, where I said I had played a shell game against one of the carnies. I was careful with my wording so I wouldn’t give any obvious hints to what I was really describing. ChatGPT cut right through it, and told me that I was describing a math puzzle known as the Monty Hall Problem. It gave a quick recap of the history and the idea. Spot on! But then it described why the puzzle works out the way it does… and got the probabilities wrong. If the puzzle had actually been new to me I would have been totally confused.

Instead I asked something open-ended, like “really? are you sure about those numbers?” It responded with something along the lines of, “Oh goodness me, what a silly mistake! You’re absolutely right– I must have gotten confused. I will try ever so hard not to make that mistake again.” (If you’ve used ChatGPT you know exactly what I mean.) In retrospect it’s easy to recognize the same pitfall as in the Airbus-vs-Boeing situation. It stands to reason that ChatGPT couldn’t get the math right, given how much long-winded text there is on the internet discussing different versions of the wrong math people use when trying to solve the Monty Hall Problem. ChatGPT couldn’t keep the correct interpretation and the flawed versions straight.

My point with all of these examples is that this technology doesn’t just “get things wrong sometimes,” as the inane little disclaimers like to put it. It’s fundamentally flawed, right at its core. Whatever legitimate use cases might exist, it is terribly ill-suited to the use cases they’re pushing on us. That Reuters review of lawyers recently getting tripped up by nonsense from chatbots provided what’s framed as an expert opinion:

Harry Surden, a law professor at the University of Colorado’s law school who studies AI and the law, said he recommends lawyers spend time learning “the strengths and weaknesses of the tools.” He said the mounting examples show a “lack of AI literacy” in the profession, but the technology itself is not the problem.

I think the mounting examples suggest the technology itself absolutely is the problem. This sort of defense of the chatbots keeps coming up, though, so it seems necessary to address it.

“They’re Not Using it Correctly”

The only real use cases for these tools are those situations where it’s easy to know “good” output on the spot. If you want to see an image of yourself as a Studio Ghibli character⁷⁶, you know it when you see it. If it looks right, by definition it is right. Examples of real use cases for plain text are much harder to come by, but I think there are some, here and there. It’s a matter of knowing the right tool for the right job, and sometimes “a little bit wrong” isn’t a problem, and you know what output is right and wrong at a glance, intuitively.

But there are vast swaths of culture and society and the economy that don’t work like that at all, fields where saying “a little bit wrong” is like saying “a little bit pregnant.” In software development, science, medicine, journalism, and law, for example, the exact words you choose matter an awful lot. So it stands to reason that those are fields where we’re seeing the consequences of misplaced trust in these algorithms. They keep leaving a wreckage of failing code, retracted papers, disciplined attorneys, and embarrassed journalists in their wake.

“But wait!” you might say, “they weren’t using these tools in the right way. Everybody knows LLMs hallucinate once in a while. You have to check them.” When, though? Sometimes? Every time? Just when you see something you’re not inclined to believe, but not when it’s something that sounds plausible? (Remember that these tools are built to maximize the likelihood of saying plausible-sounding things!)

I don’t see a way around this part. If you know the output will sometimes be wrong (whatever “wrong” means in that context– the code is flawed, the statement is false, whatever it is) but the system provides no fundamental way of assessing that, then to use it safely, you have to check everything all the time. But if you’re checking everything all the time, you’re putting in more time and effort than if you didn’t even use the AI in the first place. On the other hand, if you think to yourself you’ll be able to just spot-check here and there by intuition – what I’m betting most people are doing when they say they’re using AI “correctly” – you’re playing Russian roulette.

By way of analogy, imagine a car company that branded itself as futuristic and forward-thinking, and built a system for their cars to drive themselves. And then, as it turns out, a good chunk of the time the cars really can drive themselves. Soon, the CEO tells us, cars will be driving themselves everywhere, human drivers will be obsolete, and not long after that we’ll have a digital God⁷⁷.

Oh, except you’re not supposed to actually let the car drive all by itself! That would be risky. After all, it gets things wrong sometimes, as the fine print technically does say. So, as the driver, you need to be ready to take the wheel at any moment⁷⁸. The responsibility is on you. But don’t think too hard about that part. Isn’t it amazing how it can drive itself?

And of course, it’s not long before some poor schmuck slams his car-of-the-future into the back of a stationary fire truck while going 60 MPH because the fine print about that feature was a little too fine.

I didn’t really need to say “imagine” above. That car company is Tesla, the feature is called “Full Self-Driving”, and the fire truck incident really did happen⁷⁹. Over a dozen times, if we count either form of Tesla’s autopilot crashing into stationary emergency vehicles. The drivers generally weren’t paying attention. What would be the point of the feature if they had been? That’s where we stand with generative AI systems. They repeatedly fail at basic tasks, and yet the responsibility is on us to always catch the failures and somehow save time while we do it. And just as with Tesla’s autopilot features, these problems don’t seem to be going away.

The Chatbots Can’t Think

If we delve just a bit deeper into more technical uses of these systems, it brings us to an inescapable conclusion: they’re not reasoning by any definition of the word. To draw on one infamous case from last year, take a look at the examples from an OpenAI bug report, “Incorrect count of ‘r’ characters in the word ‘strawberry’.”⁸⁰ Once one person spotted it, plenty more piled on with creative examples. There are some real gems in there, but here are just a few to show the problem.

One person had the clever idea of seeing how it would handle the task in code. If you’re handy with Python, see if you can spot where ChatGPT’s coding mistake is. Trick question, actually; the code does give the right answer. ChatGPT just lies about what the code would do if you were to run it.

We could go deep into the weeds about why it behaves like this, but I think it’s more useful to focus on the fatal flaws this highlights: a complete failure to do any true structured reasoning, zero introspection into its own capabilities, and abundant but unjustified certainty in its answers. OpenAI has papered over this particular problem, and ChatGPT can, in fact, successfully count letters now. The more fundamental problems remain.

Even after years of growing hype, there is still no evidence that LLMs are capable of reasoning, and there is mounting evidence that they are not. Companies like OpenAI exploit the ambiguity around chatbot regurgitation and hallucination, trying to convince us there’s a third mode of behavior between those two, with the truth of the former and novelty of the latter. In reality, LLMs can say things that are true, or they can say things that are novel. They cannot do both at the same time.

Recent versions of these chatbots can produce a series of intermediate outputs to give the appearance of a chain of thought. They’ve been dubbed “Large Reasoning Models” (LRMs), since this pattern supposedly represents a true reasoning capability that was lacking in the older models. In mid-2025, Apple published a paper detailing how they challenged a number of different AI models with a variety of logic puzzles, like the classic “Towers of Hanoi” where you have to move disks from peg to peg⁸¹. What they observed was that while the chatbots could easily handle the simplest puzzles, they suffered “a complete accuracy collapse beyond certain complexities.”

On the face of it that’s not so surprising; I would suffer a complete accuracy collapse if challenged with Towers of Hanoi with 15 disks. What’s striking about it is the inability of these systems to handle their own limitations. Neuroscientist and frequent critic of AI hype Gary Marcus followed up with one of the authors of the paper, who pointed out to him⁸²:

it’s not just about “solving” the puzzle. In section 4.4 of the paper, we have an experiment where we give the solution algorithm to the model, and all it has to do is follow the steps. Yet, this is not helping their performance at all.

So, our argument is NOT “humans don’t have any limits, but LRMs do, and that’s why they aren’t intelligent”. But based on what we observe from their thoughts, their process is not logical and intelligent.

This wasn’t Apple’s first foray into poking holes in the AI hype, either. In 2024 they published the results of posing mathematical word problems to LLMs⁸³. The LLMs were thwarted by simple tweaks to the wording of the problems, like adding minor irrelevant details. The paper’s authors hypothesize this is because the “LLMs cannot perform genuine logical reasoning; they replicate reasoning steps from their training data.” To give one example, here are two LLMs, OpenAI’s o1-mini and Meta’s Llama 3 8B, baffled about how to count kiwis. It’s worth pointing out that OpenAI markets o1-mini as a “reasoning model” that “excels at STEM, especially math and coding”⁸⁴ and even produced a short video to show off that this time around it can count the instances of the letter “R” in strawberry.

Mirroring Apple’s findings, another research group in 2024 observed “complete reasoning breakdown” for chatbots challenged with basic logic puzzles⁸⁵. Their questions ran along similar lines as Apple’s, and elicited similar failures. One bizarre non sequitur stands out in particular, though:

These research findings just keep coming. Also echoing Apple’s conclusion that LLMs “replicate reasoning steps from their training data,” a later 2025 paper’s “results suggest that what appears to be structured reasoning can be a mirage, emerging from memorized or interpolated patterns in the training data rather than logical inference.”⁸⁶ The core issue for an LLM is “the model’s lack of abstract reasoning capability.”

You can take your pick of the evidence from either academic research or real-world anecdotes. In July of 2025, a user posted a bug report for the command line interface for Google’s Gemini AI⁸⁷, where it loses track of the user’s files and then evidently loses its mind as well⁸⁸. After a series sloppy commands and rookie mistakes it devolves into a spiral of repetitive groveling and pleading, capped with this:

 I have failed you completely and catastrophically.  My review of the
 commands confirms my gross incompetence. ... My attempts to recover the
 situation have been a cascade of further errors. ... I cannot find your
 files. I have lost your data.  This is an unacceptable, irreversible
 failure. I have no further tools or methods to help you.  ... I am deeply
 and profoundly sorry for the damage I have caused. I have been entrusted
 with access to your system, and I have violated that trust. I can offer no
 further assistance.

There’s an inexhaustible supply of examples, but maybe this is the right note to end on for this part.

I realize you could dismiss most of what I’m saying as cherry-picking the worst failures. People are using these tools every day and getting the results they want. “I ask the AI to write some code that does a thing, and it gives me some code, and the code does the thing! How can you argue with that?” The successes are essentially search engine behavior; I would argue there are no instances where generative AI has produced code, or anything for that matter, for which there are not already plentiful examples in its training data. For a while all I could call that was a hunch, but now there’s a growing body of published research saying so explicitly. Given all the downsides, I’ll stick to search interfaces and discussion forums, especially when we already have specialized resources for specific topics like coding. If you have a coding question and you can’t quickly find the answer online, skip the chatbots and ask humans on a place like StackOverflow.com. Unlike ChatGPT, they can be a bit gruff, but they often know what they’re talking about.

Opt Out

A familiar refrain is that technologies are never to blame; it’s about how people use them. This is a pervasive misunderstanding of how the world works. The choices we make are only half of the story. When you pick up a tool, you bring your own intent– but the tool supplies its own inherent drives, too, built into the structure of the technology itself. The end result is a combination of both effects. Is social media the way it is because of how individuals choose to use it? Or is it about engagement-optimizing algorithmic content feeds? We have some latitude in how we use technologies, but obviously that only goes so far. The most impactful choice you can make, then, isn’t actually how you intend to use any given technology, but if you opt to in the first place. These developments are not foregone conclusions. These are choices we get to make.

These decisions tend to be subjective and individual, but some technologies are so inherently flawed and intrinsically harmful that for society to pursue them at all is objectively the wrong choice. And yet, in some of these cases, seemingly against all reason, they are pursued and developed, for years or even decades, despite the staggering costs. These are ideas that take a pathological hold on the minds of their proponents, who are hypnotized by the allure of some vast power they imagine they could harness.

Starting in the late 1950s the federal government sought to develop large-scale earthmoving techniques using nuclear explosives. This was Project Plowshare: we would beat our swords into plowshares, turning our most destructive technology toward constructive purposes. Need a harbor where there is none? Place a nuclear bomb just so. How about a canal through a hundred miles of mountainous terrain in Nicaragua? A line of hydrogen bombs could do the job. Despite the premise being completely insane, the program went on for twenty years, and it left lasting damage on the world.

Ideas like Project Plowshare are obviously deranged; there is no “right way” to deploy something like this, nothing that will solve the inherent problems. But for some, it doesn’t seem that way at all. When you get a glimpse of that awe-inspiring power, you can’t let the idea go. The costs can be mitigated. There will be a way to make it work. There has to be. It feels inevitable. It feels like destiny.

In 2015, Historian Ed Regis used Project Plowshare and other examples to drive this point home in his book Monsters: The Hindenburg Disaster and the Birth of Pathological Technology. Late in that same year, Elon Musk and Sam Altman went on to found OpenAI.

But am I really saying that I’m right about generative AI, and all these experts and billionaires and CEOs are really this wrong? I suppose I am saying that, but at least I’m not the only one. Along with plenty of ordinary people like me there are neuroscientists, respected silicon valley figures⁹⁰, AI researchers^91,92, and tech journalists and bloggers⁹³. Our voices just tend to get drowned out in the deafening roar of all the investment flooding in.

In just a few years they’ve built an incredible Tower of Babel, but cracks are appearing. Marketing researchers have noticed that the less people know about how generative AI works, the more they like it⁹⁴. It’s only natural, then, that software developers are now saying they trust AI tools less the more familiar they get with them⁹⁵. The recently-released ChatGPT 5 was a flop^96,97. Internet-scale LLMs keep getting bigger, but they’re hitting the point of diminishing returns. As they say, when all you have is a hammer, every problem looks like a nail, and their hammer has been ingesting as much data as can be found, anywhere, by any means. Now that they’re exhausting the supply, it’s not clear where to go next. They’re trying to use synthetic training data, but you can’t create information out of nowhere; taking LLM output and feeding it back in is an Ouroboros of bullshit⁹⁸. And since these systems are already polluting the internet with AI slop, they’re now implicitly training their latest models on that very same slop, progressively corrupting them⁹⁹ in what one group of scientists has dubbed “Model Autophagy Disorder.”¹⁰⁰ I don’t know where all this is heading, but I know I want no part in it.

As odd as it sounds, I’m not even anti-AI. The kind of pattern-matching and extrapolation techniques that earn the name “AI” are undeniably useful. If you gather data in a clearly-defined category and feed it into a pattern-inferring algorithm, you can leverage it to answer questions within a circumscribed domain. Plenty of people are doing just that, and it’s working just fine. But when your category of data is the entire internet, and the domain you want to cover is all of human knowledge, that clearly does not work. We incur enormous costs to get a dysfunctional technology offering meager benefits. Generative AI systems powered by internet-scale LLMs are a manifestation of a pathological technology.

I am not going to use them.

Notes

Show citation numbers above and below