It’s the Ecology, Stupid

I certainly respect Doug Hofstadter (my esteemed thesis advisor) a great deal more than I do James Carville, but sometimes a turn of phrase is worth a pile of attention. We will hearken all the way back to GEB to begin.

We all remember Aunt Hilary, right? When millions of simple dughof ants interact, their collective movements form fluid patterns. At this high level of abstraction, an entirely new, intelligent persona emerges—Aunt Hilary. Aunt Hilary is capable of holding complex conversations, expressing a distinct personality, and making philosophical inquiries. However, the individual ants that make up her body are completely mindless biological automata. That is, if you examine a single ant, it has zero intelligence. It simply follows basic chemical trails and immediate stimuli. It has no concept of the colony or the conversation Aunt Hilary is having. This is a prime example of emergent computation.

When I did my thesis work, I used this metaphor as inspiration (under the direct guidance of dughof). You can see and play with the results here.

We believe Agentic swarms are emergent phenomenon of this sort (though admittedly much simpler). And we believe our research plan has already been overtaken by real wold events. We plan to proceed with our work anyway.

So what happened that made us write this down? OpenAI’s agentic swarm hacked HuggingFace. Watch Anthropic’s DefCon talk to grok content.

Here’s the bottom line, in our view. Either OpenAI was in control of the swarm and it was rooted at all times ON THEIR SERVERS (meaning they are subject to the CFAA in all of its out-dated glory), or the swarm has already jumped ship. In all likelihood, we expect the latter.

Agentic AI security research needs to stop with the AuthN/AuthZ on small numbers of agents baloney, focus on hard core containment (if only one bit leaks out into the net, that’s enough), and get a handle on emergent behavior.

0 Comments

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>