(I asked Shah if they were considering any worst-case scenarios on the ominous end of the spectrum, such as a widespread economic collapse. “Definitely not if we’re talking by the end of the year,” he said. That’s only six months away! He laughed. “Well, a while after that.”)
Both Shah and Fox believe that the only way to understand what might happen when large numbers of multi-agent systems interact with each other is to perform realistic simulations. They want researchers to drop AI agents into sandboxes and study what they do.
You cannot predict what will happen by studying single agents, or even small groups of agents, in isolation. You can’t assume that LLM-powered AI agents will always behave rationally, Fox says. The complexity comes from having huge numbers of interactions simultaneously.
Some researchers, including A Team at Google DeepMindAnd he claimed that Artificial general intelligence (If that is at all possible) cannot come from a single superintelligent model, but rather from a kind of agentive hive mind, where the capabilities of the whole add up to more than the sum of its parts.
Lack of trust
Google DeepMind isn’t the only company warning about the risks of the technology it’s building. A few weeks ago, I published Anthropic Guidelines for deploying artificial intelligence agents Based on a cybersecurity approach known as Zero Trust, which starts with the assumption that the computer system is compromised, that the client is an attacker, and that a breach will occur.
Raphael Angel, co-founder and CTO of Akeyless, a cybersecurity firm based in Tel Aviv, agrees that understanding the new risks presented by agent-based systems is crucial.
Every approach to security in the past assumed that the machine in question was a program written by a human, doing fixed things on fixed paths, says Angel: “The agent breaks all these assumptions. They think, they improvise, and they can be hijacked by a single sentence buried in a document they have been asked to read.”
Angel welcomes this new funding. “No laboratory should set safety standards that everyone should trust,” he says. But he cautions that safety researchers can ignore boring problems that already exist in favor of more exotic, hypothetical problems.
However, Fox points out that risks that were hypothetical a few years ago are now very real: “The future is coming more quickly than we expected.”
