More than 20 prominent artificial intelligence researchers released a new research paper warning about extreme risks from automated artificial intelligence research. The group includes Geoffrey Hinton, Yoshua Bengio, and OpenAI research lead Jakub Pachocki. They caution that self-improving artificial intelligence could soon trigger an intelligence explosion. The authors write that much uncertainty remains, but artificial intelligence research and development automation might soon cause this event.
Artificial intelligence writes most lab code
The study states that artificial intelligence systems already write most of the code at the companies building them. These systems could automate the entire artificial intelligence research and development pipeline within years. Progress that normally takes years could then happen in months. The authors warn that society might not keep up with this accelerated pace.
Control could slip away from humans
Control over superhuman artificial intelligence could slip away if automation outpaces society. Power balances between nations, companies, and governments could also erode. The researchers urge policymakers to gain far more visibility into how artificial intelligence research is being automated. This publication joins a growing list of warnings from the scientific community.
Mathematicians and employees raise alarms
Forty-two leading mathematicians recently called for more attention to existential artificial intelligence risks. Several artificial intelligence lab employees have voiced fears that artificial intelligence could destroy humanity. Jakub Pachocki stated that no lab has solved alignment well enough to continue responsibly scaling at maximum speed for much longer. Some Anthropic employees are reportedly scouting safe havens in case artificial intelligence goes sideways.
Policymakers must review visibility into automated artificial intelligence research.
