Jacob Coxon, the former Anthropic researcher who warned that AI could kill all humans, says there’s “move fast and break things” culture at AI companies.

Coxon was testifying under oath at a New York City Council hearing yesterday, which also saw senior leaders from Anthropic, OpenAI, Google and Meta testifying.

According to CNBC, Coxon, called out the industry’s “extremely reckless” approach to development during the meeting. “On the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction,” repeating his message to a packed room.

He wasn’t the only one issuing warnings. Former Google DeepMind researcher Alex Turner warned that “misaligned AI” could prove more dangerous than China’s aggressive development push during the meeting.

“China is not our only potential adversary,” he said. “With reasonably high chance, we are racing to build and grow our own adversary here at home, which is misaligned AI. Misaligned AI is everyone’s adversary, including our own, and one day may be more powerful than China.”

Anthropic sent Logan Graham, the head of its Frontier Red Team, to testify, while OpenAI was represented by Morgan Dwyer, the company’s head of policy development and operations. Google’s director of AI and emerging tech policy, Alice Friend, and Meta’s AI policy director for legislation, Shane Cahill, also attended.

While Meta confirmed it would participate in the hearing late last month, Anthropic, OpenAI and Google only agreed to appear after the council threatened to subpoena them, New York City Council Speaker Julie Menin said.

Elon Musk’s SpaceXAI was the only company called to testify not in attendance, even after the city issued a subpoena to SpaceX CEO Elon Musk last week, requiring that he or another representative of the company’s AI unit, SpaceXAI, testify.

This is in “direct violation of the subpoena” issued by the council last week and the council plans to “pursue this matter in court,” Menin said.

Both Coxon and former OpenAI researcher Daniel Kokotajlo warned of the dangers that the “move fast and break things” culture at the AI companies poses with such a transformative technology.

Coxon said that method works for a “photo-sharing app” but not “for building the most powerful technology ever built.”

Given the heightened safety concerns raised by so many, it’s probably not the best time for one of the leaders of OpenAI’s safety team to leave the company (saying that the “culture is broken”) or for OpenAI to fire three other researchers for reportedly sharing confidential company information with a third-party AI safety organization.

I may have to update my list of “every publication about the dangers of AI I could find” more frequently!

So, what do you think? Worried yet? 😉 Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using ChatGPT, using the term “robot programmers working in a room with a sign that says ‘You break it, you bought it’”.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.