In the filing for Anthropic’s IPO set for this week, they warned of existential risks to humanity. But maybe AI isn’t our only cause of significant risk.
Greg Bufithis discusses a significant concern in his article here that has nothing to do with AI. As he notes here, referencing technology analyst Dustin Patrick:
“Four systems that hold modern life together broke in 48 hours. Not one of them was attacked.
- On Friday the 13th, the FAA stopped every departing flight in the United States. It was only the second nationwide ground stop since 9/11. No hackers. Old switches and routers in a network nobody had replaced simply got overloaded and gave up.
- The day before, Microsoft 365 went down across 74 countries and stayed down for over 24 hours. Email, Teams, Copilot. Hospitals, schools, police departments, banks. The cause was a network configuration change.
- Friday morning, Claude went dark worldwide.
- Then Workday went down on payday, locking workers at some of the country’s biggest retailers out of their pay and their hours.”
None of them were apparently caused by AI either.
Dustin also noted this:
“For thirty years, we built everything to be efficient. Fewer people. Fewer backups. One cloud, one vendor, one login, one genius at the top.
Every layer of slack we removed showed up as profit. Every one of those layers looked like waste right up until the day it was the only thing holding.
We did not build a fragile world on purpose. We built a world that runs perfectly as long as nothing goes wrong, and called that progress.
Now hardly anyone alive fully understands the systems we depend on. The people who built them retired. The people who maintain them were cut. The ones who remain are told to move faster.
So the world does not collapse. It flickers.
A ground stop here. An outage there. A founder gone. A power plant dark. Each one fixed in hours, each one explained away and forgotten by morning.
But every flicker teaches the same lesson, and almost nobody is learning it.
Civilization is not the buildings, the planes, or the code. It is the quiet, invisible work of keeping things from breaking, and we have spent a generation treating that work as a cost to cut.
The question is not whether the lights will go out.
The question is whether anyone will still remember how to turn them back on.”
And, as Greg notes:
“Every complex system is filled with skeletons of ugly compromises and shortcuts. When the builders leave, the systems can keep running, but if things change, it’s very hard to find and exorcise those skeletons.”
And that’s even before cyber hacks, which Greg notes “will get far, far worse”.
Greg references the recent PHP flaws story, noting this:
“Security researchers used AI to find them, in code written in 2012. Within 45 minutes of their official disclosure, the internet security company Cloudflare was seeing tens of thousands of break-in attempts. Within two hours, hundreds of thousands. Within 24 hours, millions. Then the security firm Bitsight found that nearly 60 percent of exposed PHP servers run versions so old that no official fix exists for them; a week after the fix came out, only about a third of the servers it tracked had installed the patch. Bad software is like asbestos. Asbestos was cheap and no one understood the health consequences, so it went everywhere. Bad software is now everywhere too, and we are starting to see how toxic it can be.”
This all struck a nerve with me because I have felt that it seems as though systems are becoming more fragile, too complex, much more vulnerable to things going wrong. Most of the “things going wrong” have been minor (brief outages, functionality problems, etc.), but I’m concerned that events last week illustrate how they can and will get worse.
Stocks go up because companies are always looking to build the next bigger, better thing. Fixing existing issues doesn’t move the needle financially, so it usually takes a back seat to other work.
Maybe AI isn’t our only cause of significant risk? Maybe we’ve created plenty of risk on our own. As Greg states:
“The fragility is real. The people uncovering it responsibly today will soon be joined by others with the same tools and no scruples. And there is no more time.
We got away with writing sloppy software for so long that we never thought there would be a price. The penance has arrived, and it is not cheap. We can pay now, on our schedule, or later, on theirs.”
Ruh-roh.
So, what do you think? Are you worried about growing issues with existing infrastructure? Please share any comments you might have or if you’d like to know more about a particular topic.
Image created using ChatGPT, using the term “robot standing next to a person looking at a computer workstation with a ‘All Systems Are Down’ message and holding a sign saying ‘It’s not my fault!’”.
Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.
Discover more from eDiscovery Today by Doug Austin
Subscribe to get the latest posts sent to your email.
