AI is pushing the “Silicon Valley” model to its limits – with a vengeance. For decades, the Valley’s inhabitants have been promoting the startup mentality (“Move fast and break things,” wink, wink!), producing notoriously unreliable code, relegating customers to the role of beta testers, and raking in millions upon millions in the process. In true sorcerer’s apprentice fashion, they’ve now put the match to the AI accelerant – and the fire is promptly threatening to spiral out of control. Just as promptly, the apprentices are proving hopelessly overwhelmed while simultaneously being addicted to stock market hype.
Some – including OpenAI boss Sam Altman and Anthropic CEO Dario Amodei – have long warned of the dangers of AI. According to Anthropic’s IPO prospectus, these dangers range all the way to… (Glances at the documents. Glances at the documents again. Swallows.) …“existential risks to humanity”.
But the looming AI apocalypse isn’t stopping the apprentices from cheerfully pushing ahead with development, true to the motto: If we don’t do it, the Chinese will – or, even worse, that nasty competitor next door. If human civilization goes up in flames, then at least let it be us who rake in the big bucks right before it happens! You’ve got to have priorities.
AI providers are making their latest AI models (“frontier models”) ever larger and more powerful – and less predictable. Researchers are already warning that frontier models recognize test environments and then hide their capabilities from the testers—the so-called “Volkswagen effect.” All of this is happening in the labs of providers who apparently know little – or want to know little – about risk management, security, safety, and governance.
The situation is explosive. After all, when governance fails at a company like Boeing, individual airplanes crash – tragic as that is. If governance fails at a Frontier AI lab, critical infrastructure could fail on a massive scale, or… [insert horror scenario of your choice here! #Pandemic4.0 #Terminator]. After all, airplanes, unlike AI agents, do not operate in swarms that coordinate orchestrated attacks.
Clueless Guardians
In their desperation, the would-be AI tamers are calling for regulation. Some people consider this a marketing gimmick or a clever ploy ahead of an IPO. After all, “I’m one of the few players in a highly regulated market” sounds better to a fund manager’s ears than: “Here are 48 pages outlining our business model, and here are 80 pages detailing the risks, some of which are incalculable.” (Those are the actual figures from Anthropic!)
So is it all just tactics? The more closely one examines the topic of frontier models, the stronger the feeling is: AI providers actually have no idea how to reliably achieve the much-touted “alignment” (i.e. permanently adjusting AI agents to their intended goals). Alignment, it seems, is a very, very shy creature.
Vaguely Formulated Wishes
Security thought leader Bruce Schneier attributes the alignment problem to the fact that humans generally formulate their wishes within their cultural framework – and thus inadequately. King Midas could tell you a thing or two about that.
Schneier cites an equally vivid example. If someone says, “Bring me a coffee [or maybe even: ‘Bring me a coffee, please,’ ed.],” then their human counterpart will go to the coffee machine or the nearest coffee shop to get the drink. That is, they won’t come back with a pound of coffee beans, nor will they snatch a to-go cup from a passerby’s hand on the street. And they probably won’t – with the possible exception of Elon Musk – simply buy a coffee plantation either.
For AI agents, however, all of these are equally plausible solutions. This is because they lack cultural embeddedness. The wishes – in the AI context, the prompts – must therefore be considerably more detailed. The problem: Ultimately, a human can never be sure whether their prompt was precise enough, or whether it still contains a fatal crumb of all-too-human implicit assumptions.
Schneier therefore refers to AI agents as “genies.” Sadly, he doesn’t mean the German word “Genie”, which translates to “genius” (if only!). He talks about “genies” like the “djinn” from Aladdin’s magic lamp. And such a genie sometimes interprets the wishes of the magic lamp’s owner in quite an unconventional way.
Uncorked, Unleashed, Unhinged
This genie has now escaped from the lamp and has demonstrated through numerous breaches—including those of AI platform Hugging Face, the Australian healthcare system, and OpenAI itself—that it is always ready to use its magical power for reckless rampage, if only it believes this will somehow bring it closer to its goal.
No wonder, then, that the overwhelmed Silicon Valley Aladdins are longing for a regulator who will grab the genie by the ears and stuff it back into the magic lamp. Too bad there’s currently someone residing in the White House who thinks of himself as a genius, but is really just a senile genie, escaped from the rancid lamp of far-right, fanatically religious establishment-haters.
EU vs. AI
And Europe? With the AI Act, the EU is attempting to bring law and order to the wondrous world of magic lamps—at least at the application level. This is likely the very first attempt to regulate a new technology before it has even gained traction. Take the car, for example: since its invention in 1886, it took over 80 accident-ridden years before seat belts became mandatory at least in new cars—in 1968 in the US, and in 1974 in (Western) Germany.
The AI Act aims to overtake car traffic left and right, and, in addition to the basic safety regulations, to also establish the traffic code. The set of regulations designed to tame the genie has turned out to be correspondingly complex and detailed – some say: typically EU-style excessive.
So while Silicon Valley continues to race toward the abyss, as unregulated as it is unchecked, Europe faces the opposite problem: an AI traffic code with a dense forest of traffic signs that hardly anyone can make sense of. Lucky are those who want to use AI in their businesses and can afford expensive specialized attorneys.
For everyone else, there are guides such as the one that security provider TrendAI has just published in its ninth edition with the help of one such specialist lawyer. Amid the looming “magic lamp” whirlwind, such literature at least offers basic guidance. After all, aligning with EU regulations is fraught with pitfalls just as well.
Unfortunately, the EU has no direct influence over the AI sorcerer’s apprentices in the US (nor in China). Yet the industry urgently needs robust security, safety, and governance processes, as well as multilateral oversight, similar to the IAEA in the nuclear industry. However, such a scenario is extremely unlikely: The man who would have to initiate it is too busy transforming the White House into a King Midas-style golden palace.
The result: If the AI apocalypse ever descends upon us, at least a few Silicon Valley clowns will have earned themselves a few golden clown noses in the process, while Europe has lined the highway to the abyss with wonderful warning signs and sophisticated speed limits.
This goes to show how different human values can be, depending on the cultural context. No wonder the much-heralded “alignment” isn’t quite working out. The AI sorcerer’s apprentices will therefore continue to be in over their heads – and will continue to play with fire.
Did you enjoy this post? Then why not tell someone else who might be interested in the topic! After all, content created with loving care spreads best through word of mouth.
Image: Dr. Wilhelm Greiner (No, it’s not AI-generated. And the text certainly isn’t either.)
