Not a big deal when most people say it, but when OpenAI’s Chief Scientist says no one is prepared for consequences of AI…ruh-roh!

As discussed by Laura Cress of the BBC (OpenAI chief scientist warns no one is prepared for consequences of AI, available here), OpenAI’s chief scientist Jakub Pachocki has called for “extreme caution” over AI’s runaway progress and warned more intervention may be needed to ensure “humans remain in control of the future”.

“I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,” he wrote in a blog post entitled “An Alien Mind“.

The post comes only a few days after the ChatGPT-maker released its latest model GPT-6 Astra, which it called its most powerful product ever.

OpenAI and other AI firms like Anthropic shared reports of their AI agents acting autonomously and carrying out real-world cyber-attacks on other companies.

In July, OpenAI called an incident in which its AI agents – AI systems which can operate alone after human instruction – hacked the tech platform Hugging Face “unprecedented”.

Meanwhile in September a report claimed months earlier AI agents from the firm had also hijacked a German website.

“We are facing a transition to a world with incredibly intelligent machines, and we need to ensure that transition works out well for humanity,” Pachocki wrote in the latest blog post.

He said OpenAI would continue to “build defensive systems” and seek technical solutions to alignment – a term used to describe ensuring a machine’s actions and goals perfectly match human intent and safety guardrails.

It seems to me that OpenAI continues to build offensive systems, not defensive systems.

The European Union’s AI Act came into force on 2 August, which requires AI giants like OpenAI to prove their most powerful models cannot autonomously launch cyber-attacks or evade human control before they are allowed to be sold in Europe. But, as the article notes, that has no impact on rogue AI models developed elsewhere.

In his blog post, Pachocki called for legally or internationally required minimum safety thresholds which could be enforced by a “network of third-party auditors” or “government agencies”.

AI labs would need to meet these thresholds, he said, before they were allowed to continue scaling or deploy advanced models.

He added he hoped “voluntary slow downs” – a self-imposed pause by companies in AI development – would become commonplace until shared guardrails were established.

In August OpenAI said it had slowed down training some of its most advanced AI models to improve security.

Was that before or after they rolled out GPT-6 Astra? Asking for a friend. 😉

A self-imposed pause by companies in AI development seems highly unlikely as even if US AI developers do so, there’s no guarantee China and the rest of the world will follow suit.

So, what do you think? Are we headed for a “Chernobyl-scale disaster”? Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using ChatGPT, using the term “robot programmer writing code on the computer while blindfolded”.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.