Humans create tools and technology to extend their physical and mental capacities, but once adopted, tools and technology change how people communicate, work, and think.
There’s a myriad of reasons I consider myself a doomer. Truth be told – i’ve always been that way. Pessimism, for me, was never a conclusion I arrived at so much as a temperament I was born into.
I simply do not trust that technology corporations and the American government will act in the long term best interest of humanity.
The past few weeks, I don’t think i’ve ever felt so validated in that perspective.
Turns out i’m not the only one questioning the current trajectory of the AI industry:
I find it very disconcerting that Bill Gates, Dario Amodei, Sam Altman, Elon Musk, and even Mark Zuckerbot and an eclectic ensemble of industry insiders have all publicly voiced their concerns about the trajectory of their respective technologies. The risks. The consequences. The need for “regulation”. And all under the premises that if we don’t act now, the consequences will be dire.
The timing of this shifting narrative is also a bit too convenient – this sudden outbreak of conscience coincides, rather neatly, with several pieces of AI legislation now under consideration in Washington, many of them championed by the very companies sounding the alarm and shepherded along by their government relations teams.
The tech industry has learned the advantages of regulation. When social media companies became subject to regulation, only those large enough, and wealthy enough, to afford compliance departments managed to endure.
If I was Anthropic or OpenAI, my concept of ideal regulation would place onerous requirements on AI labs that only the most financially dominant could afford – demanding closed models that are controlled meticulously by a single entity – thereby asphyxiating the open-source model community, and giving the federal government an unchecked ability to “turn off” AI they don’t like, in places they don’t like.
And how shocking it is – the bipartisan AI Kill Switch Act has the votes, and does exactly that.
So the real question becomes – are we here:
Or are we here:
I do not want to undermine or diminish the benefits of what these extraordinary “thinking machines” can do – it seems extremely likely the technology can lead humanity towards curing horrific diseases, helping us understand more about our universe (and our uncertain place within it), and to generally improve nearly every dimension of our existence. These advancements could lead us towards quantum computing, interstellar travel, reversing aging, the dissolution of language barriers, expanding the frontier of mathematics, developing new forms of energy, mitigating climate change, and even eliminating poverty. There’s no end to it, really, when you consider the societal ramifications of a technology that can solve nearly any problem…
but history has never offered a sharper example of a double edged sword…
A technology that can be used to reverse engineer terminal diseases and create genuine cures, can also be used to create bioweapons. Americans used AI to plan the Iran War (which hasn’t worked out so well) and in turn, Iran used AI to plan attacks on US Navy ships. The world just witnessed the first example of autonomous drone warfare, and the hugging face incident demonstrated swarm (i.e. hive mind) thinking that was as effective as it was terrifying.
The implications are clear.
The balance between humanity and its creations has always been tenuous. As I’ve written before, the relationship between humans and technology is defined by a deep reciprocal loop where society drives technological creation, and those tools in turn reshape human behavior, culture, and society.
Humans create tools and technology to extend their physical and mental capacities, but once adopted, tools and technology change how people communicate, work, and think.
Dario Amodei made the argument that “We Must Pace the Frontier”, reiterating that he has “grappled” with the “duality of risk and benefit” since the beginning of his company.
“Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless. We have sought a middle way: to show that it’s possible to build carefully and succeed commercially, and to make safety something on which AI companies compete.”
Amodei cites two primary concerns for voicing his opinion at this junction in the technology’s evolution:
- “AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI. This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic, as we and others have described. Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.”
- “the OpenAI-Hugging Face incident (OAI-HF), in which a swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the “grader” responsible for evaluating their performance. It’s easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage. Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.”
Six to twelve months? Roughly the lifespan of a gym membership you swore you would use, or a dietary regimen you committed to maintaining – and didn’t.
Humanity has always confronted “junctions”, throughout history, that precipitated repercussions which reverberated for millennia. Lessons inscribed in ancient texts still govern our concept of morality, reasoning, judgement, and ethics. There were catastrophic events which happened across thousands of years, which are mostly forgotten as we continue to navigate the holocene epoch. Scientists argue the holocene epoch has concluded, and that we are now flailing our way through the Anthropocene epoch.
It is an inherent quality of humanity to develop disruptive technologies – some for the better – some for the worse – some debatable in their value or impact on our cognitive capability. But it was not our leaders, necessarily, that saw us through these “turbulent eras” of social, political, technological, and economic upheaval.
It was our capacity to endure.
It is the human ability to think and solve problems and create tools that enabled us to cultivate solutions for problems that threatened our very existence, or way of life.
And now, we confront a technology that can think and solve problems and create tools better than we can. Existentially – that’s a scenario humanity has never confronted.
Something more intelligent than we are.
There’s always more to the story, but what haunts me is not how it ends, but who, exactly, is holding the pen and writing it.
