We are Drunk
During the time in which humanity is experiencing the greatest acceleration of technological development in its history, we are doing so within a state of collective inebriation.
The views expressed here are my own, and do not necessarily reflect official positions of Helena.
We are drunk. During the time in which humanity is experiencing the greatest acceleration of technological development in its history, we are doing so within a state of collective inebriation.
Drunkenness does not guarantee catastrophe. But it increases its chances. It reduces our ability to think logically about the longer term in service of procuring our next endorphin hit, again and again, until addiction is formed. As addicts, we then lose control. Our focus turns away from an objective optimization of what is most in service of humanity, and instead, toward the next marginal improvement of the machine that produces us our supply. And so long as the machine pumps out hits of ever-increasing economic and strategic power, we’ll justify whatever resources and excuses to keep feeding it.
Today’s zeitgeist tells us there are two camps on artificial intelligence — the techno-optimists and the doomers. The framing of both feels tragically wrong to me. I fear their juxtaposition is a distraction.
The chief technology problem facing civilization is the developmental rate of artificial intelligence relative to the ability of humans to maximize its benefits and minimize its risks. We are simply out of balance — feeding one side exponentially more than the other.
Like so many technologies before it, artificial intelligence is dual use. We’re getting drunk off of its very real benefits and rushing to generate more. All the while, less relative capital, willpower and time is going toward a sober analysis of the technology’s risks and tipping points.
This state of play shouldn’t be a surprise to us. The economic and political world-system we have built over thousands of years incentives tools that generate asymmetric power, compounding economic value, and increased efficiency. A tool that converts units of energy into units of god-like intelligence is precisely the final product such a system is designed to produce.
When we have been at our best, humanity has acted soberly in this regard. We have realized that acting solely to optimize for efficiency, capital and power can often be an act of self-harm. Chlorofluorocarbons were the most capital efficient and effective optimization function for air conditioners, refrigerators, foaming agents, and aerosol propellants. But, thankfully, we weren’t willing to keep using them when they burned a giant hole through our planet’s ozone layer. Unfortunately, our ability to think in such collective interest does not seem to scale relative to the power of the tools we invent. Civilization did not exactly handle the splitting of the atom, the most powerful dual-use technology developed prior to artificial intelligence, as elegantly.
I submit that history unfolds from here in one of two directions: further drunkenness or newfound sobriety. It may feel more narratively satisfying to speak of each as a binary: drunkenness leads to catastrophe and sobriety negates it. But I think the reality is more complex and intertwined.
In a drunken future in which humanity continues the development and deployment of artificial intelligence as fast as possible, thus failing to invest enough resources into the requisite sober guardrails our society deserves, possibilities still likely exist in which we “get away with it.” Luck may be on our side; perhaps such significant benefits roll off the conveyor belt first, and we leverage those intelligence benefits to recursively mitigate future catastrophes before they happen. More likely, as with nuclear power and weaponry, we may narrowly make it through the chasm, but only after significant, dystopian episodes of suffering that nonetheless doesn’t stop the forward march of humanity.
Likewise, even a sober future — what we might today consider a successful global mobilization of cooperative guardrails — may not mitigate existential risk. By definition, we are dealing with a technology that will soon outthink us in every generalized capacity of intelligence. As Donald Rumsfeld famously coined, we are playing with a set of “unknown unknowns” that are definitionally incalculable. A third category of futures also exists, in which the psychological effect artificial general intelligence has on humanity so fundamentally changes us that we evolve away from our current system of decision making and change what we value.
While the possible futures ahead are unpredictable, how we choose to walk into them does not have to be. A drunk driver may be able to get me home safely. But I would prefer not to find out. In the face of existential repercussions, we must begin by acknowledging of our collective drunkenness in order to do something about it.
How We Got Here: Curiosity as Humanity’s Gateway Drug
March 18, 1923 — the New York Times publishes a story interviewing the legendary mountaineer George Mallory. The reporter asks why, after repeated attempts to reach an impossible peak that had not once been summited, Mallory planned to go yet again. “Because it is there.”
There is something about this quote that stirs the soul. It reminds us that there are a set of unknowns out there unbounded in their potential. That we don’t need to conform to the societal norms of the present to justify seeking them. And that humanity, at least amongst the humble species of our home planet, is special in its ability to problem-solve its way through the improbable and actually pull it off.
Kennedy was filled by that spirit, and he used it to successfully justify an exponentially more challenging task. He invoked Mallory in his legendary 1962 speech “We choose to go to the Moon:”
“Well, space is there and we’re going to climb it. And the moon and the planets are there. And new hopes for knowledge and peace are there. And therefore, as we set sail, we ask God’s blessing on the most hazardous and dangerous and greatest adventure on which man has ever embarked.”
Because it is there is an intoxicating philosophy because it is a valid one. It works. Installed into our human operating system is the remarkable trait of curiosity. Tightly braided with it is an ability to innovate, and this pairing comprises the greatest differentiator of our species. Again and again we have leveraged it in the production of technology that has, net-net, improved the basic tenants of the human experience and widened our aperture to do more.
This is where Pinker’s controversial view of development is right. The speed at which we can speak to one another went from the human sprint to the horse to the train to the speed of light. Lightning went from Zeus’ mythological plaything to harnessable, and thus came a virtual universe. Alexander the Great conquered the entire known world, then keeled over and died on the way back from a disease he nor 60 billion humans to date could overcome. Now curing it is rudimentary. Yet these are achievements infinitesimally less than what our science, and our Mallorian curiosity, tell us is to come.
The dirty secret is that curiosity is dual use. And like most dirty secrets, you realize the really bad part only after becoming addicted to the good part.
Thus far, the development of technology has disproportionally benefitted us more than it has harmed us. We are a species of mimicry and short term thinking. If it aided our ancestors, can help us, and can benefit our children, our incentive is to continue. More resources are set aside to catalyze a larger pool of humans to participate in technological development. The returns, financial, military and beyond, exceed the holistic cost again and again. Technology then itself seeps into the system of incentivizing more innovation. Modern finance, from fractional currency to the Bloomberg terminal, reaches its own exponential development curve, which in turn spreads more resources even more efficiently to the pockets of humanity capable of serving our desired optimization functions.
The result is a flywheel that spins progressively faster. Spinning off from this system are technological breakthroughs increasingly more powerful and complex. And this is the critical part. Those breakthroughs are then combined with one another and thrown back into the flywheel. Each time one creates something new, the building blocks available to use are themselves ever-more complex technologies.
Johann Rudolf Glauber produces hydrogen chloride for the first time in 1625. Joseph Priestley figures out ammonia in 1773. In 1913 Fritz Haber takes those component parts and develops the ammonia-synthesis process that ushered in an era of modern agriculture capable of feeding a planet of 8 billion people. But that same underlying technology also unleashed an era of chemical warfare that killed millions across both World Wars and enabled some of the worst atrocities of the holocaust.
Ascanio Sobrero first discovered nitroglycerin in 1847. It has been used for over a century to successfully treat chronic heart failure. It’s also quite explosive. 17 years later, a guy named Emil Oskar Nobel learned that the hard way when his family’s factory producing it blew up, killing him and several others. While mourning Emil’s death, his brother Alfred Nobel realized that mixing nitroglycerin with a specific type of rock — diatomite — made it much more stable, enabling mass production and transport of an explosive material. Three years later he patented the result, called it dynamite, and hoped it could help create the roads, tunnels, canals and railways required to navigate our planet. He was right, but it was used for a few other things as well. And now we give the Nobel Peace Prize to those who contribute solutions to a world fundamentally reshaped by weapons of war.
Each of these technologies yielded positive advancements and quite negative and destructive externalities. It is not possible to separate one from the other — the good and the bad are intertwined results of same underlying discovery. Yet neither were existential.
The saying “what doesn’t kill you makes you stronger” applies well here. Humanity was able to take stock of the destructive use cases of these technologies and react by creating governance systems to reduce those cases. We got stronger because we didn’t die. Explosives and toxic gas did not scale so fast than institutions were unable to create guardrails for their use before our species destroyed itself with them. Chemical weapons bans and arms control policy resulted, and while incredibly flawed, those measures allowed us to live another day. Humanity’s R&D playbook remained one of invent and unleash. From there, we analyze the real world effects the invention produced and rely on our institutions to minimize the negative parts and magnify the positive. Then we repeat.
This was the relationship between humanity and technology from the Stone Age until 1942. Our flywheel developing increasingly complex technologies kept spinning, and so we spun it faster. Addicted to and reliant upon the benefits, we justified the nasty byproducts. Then nasty turned to species-threatening. Dynamite didn’t destroy the world, but thermonuclear bombs could.
The Tipping Point
It is hard to blame us for getting drunk on a system that produces asymmetrically more collective benefit than risk. Especially when the risks it produces are not themselves existential. And especially when the system has worked this way for centuries.
As Nassim Taleb famously writes in The Black Swan, a turkey on a farm creates a written record of its daily life. Our turkey is an empiricist, and notes that humans continue to feed and care for it for 100 straight days. It therefore begins developing confidence that its human overlords are “friendly providers” with every meal. After 999 days of this, statistical significance and an absence of any countervailing evidence leads the turkey to conclude it is a certainty it will be fed tomorrow. Tomorrow ends up being Thanksgiving, and the turkey’s head is chopped off.
Absence of evidence does not mean evidence of absence. It was easy for the turkey to fall into an illusion of stability because no past data pointed to a black-swan future. I contend that humanity has it even worse than the turkey: over thousands of years, we keep getting fed more each day than before. From Imhotep right up to Oppenheimer, the benefits of scientific and technological development compounded without one instance of annihilation. The music never stopped, so we keep playing it louder and louder.
The Manhattan Project marked the moment our species crossed that rubicon, by developing a self-terminating tool. On July 16, 1945, we exited an era of technology we could unleash without fear it would eliminate us. We entered an era of technology of negative externalities that overwhelm us with scale, speed and complexity.
Reading Oppenheimer and Taleb as a teenager shook me to my core because it made me realize that Thanksgiving is a real possibility. Perhaps it did for you as well. As one gets old enough to understand geopolitical game theory, the nightmare gets worse; the greatly increased odds of us using self-destructive technology on each other is due to a system of perverse incentives we built, then broke. The most advanced intergovernmental institutions humans had come up with to avert catastrophe and optimize for cooperation were themselves defeated by a nuclear arms race.
In response to the nuclear weapon, the species ran the same playbook as it always does, just at unprecedented scale. We reacted. To accommodate our newfound ability to split the atom, new scaffolding to our existing geopolitical structure was built. Mutually Assured Destruction was relative management of the problem, not a proactive solution. The United Nations Security Council, International Atomic Energy Agency, and other institutions were custom-built to balance the incentives and realities of sovereign power in such a manner that tomorrow could be preserved.
Now, 80+ years later, artificial intelligence is generating risks today that are far more complex — and potentially as existential — as the nuclear bomb. But due to our drunkenness, we haven’t symmetrically updated our institutions to safeguard against them. Mutually assured destruction is still our de-facto prevention system, but it won’t work on AI-designed biological pathogens.
It isn’t inherently bad that we can create technology capable of destroying us. It just requires a level of civilizational development that requires an “operating system” change in how we behave. But we haven’t updated our operating system.
The Manhattan project was the tipping point in which humanity leveled up to a stage where it created technology that could destroy civilization. That isn’t inherently bad. But it does require that we would need to, likewise, create best-in-class systems to makes sure that only the good parts of its dual-use nature are used.
Now, we are building dozens of Manhattan project scale technological developments all at once. A possible future awaits us where AI engineered biology solves cancer and ends disease, but we may not get there if we allow the same technology to produce bioweapons.
Our Odds
An advanced extraterrestrial civilization from thousands of light years away observes Earth. What is their view of us? I think they point, laugh, and start making bets.
They reminisce about their home planet’s history: when they first split the atom, when they first reached fusion power, when they first produced general intelligence in a box. Their Manhattan Project didn’t lead to a bomb; their governance structures prevented weaponization from the start and distributed safe power to their populace. Their OpenAI and Anthropic worked hand in hand with an equally brilliant and prepared public sector to develop intelligence only at the rate in which it optimized social good, carefully pausing to patch tripwires as they approached.
The laughing comes when they observe what motivates us — how we choose to allocate our resources and focus. Accumulation of wealth and power over our tiny dominions as the chief optimization function? Arms races, multi-polar traps and games of duplicity and deception as a result? All to satiate our individual egos and promote our individual stories after we die?
Our challenges must genuinely be amusing to their audience because they remain so childish relative to how mature our technology continues to develop. Doubly funny must be their observation that we are self-aware of all of this, yet seemingly helpless due to our drunken addiction to the very system we created. As E.O. Wilson famously said: “The real problem of humanity is the following: we have Paleolithic emotions, medieval institutions, and god-like technology.”
The bets then roll in. To reach such a point of technological development in the first place, this civilization had to make it through “the great filter” — the concept that most civilizations are wiped out before they can encounter each other by exogenous factors like natural disasters or failures of their own, like thermonuclear war. I’d like to think we beat the odds on some of their earlier bets. Perhaps during the Cuban Missile Crisis, their extraterrestrial Polymarket on Earth’s extinction took the over. But the trend-line is clear: their prediction is that we likely won’t make it through too many further such trials. Unless we sober up.
Can I just say that I absolutely loved this piece!
I have been actively looking for creators that tie in history to modern day advancements or just about any facet of our world. Literally subscribed because I want more of these! KEEP GOING!
Enjoyed this immensely. I’m working on something similar (more philosophical in nature) but concerning the very idea of ‘Humanity’ and how we might salvage our understanding of the natural world through deeper introspection and critical thinking.