Author: gem

This long conversation on the Data Culture podcast features a great overview of the work we do at BIML, including coverage of why we are a non-profit. Have a listen..

Who dat?

BIML of course…in very good company.

Lots of old friends.

Alex Presiding.

And some takeaway messages encoded in bits.

Do you remember the Morris worm? Because we do. We watched it take the Internet by storm in 1988 when the net was small and mostly .edu sites connected with UUCP (there were only around 60,000 computers on the net those days). It was a big day in Net history and a watchman’s cry for the rising importance of computer security. Turns out that connected computers are subject to automated network-based attacks. Overnight, computer viruses escaped the sneaker net and grew wings.

Fast forward 38 years. Today there are 6 billion or so people on the Internet, often using multiple devices. And worms have evolved through SQL Slammer, Conficker, Stuxnet, and WannaCry—which all targeted exactly one bug—to Agentic AI controlled worms that grind on a target looking for ANY BUG. The viruses that grew wings in 1988 have developed relentless little brains.

This is Papernot at his best, reminding us why Machine Learning Security is crucially important. We’ll have a closer look this week and possibly revisit our annotated bibliography’s TOP 5.

Here is the abstract from the academic paper. We are tempted to call this new worm concept “Morris.”

A computer worm is malware that spreads on a network by replicating itself from one machine to another. Traditional worms, like WannaCry, exploited predetermined vulnerabilities, and their spread can be halted by patching those vulnerabilities. Here we show that artificial intelligence (AI) agents enable a fundamentally new threat: a worm that generates tailored attack strategies to each target it encounters. The worm parasitically uses compromised machines to run open-weight large language models (LLMs) to sustain its reasoning, or extend its reach for further attacks. Deployed on a network of machines spanning Linux, Windows, and IoT (Internet of Things) devices, the worm propagated by exploiting common, real-world corporate network vulnerabilities. Since the worm is powered by stolen compute, the attacker’s marginal cost per new infection is zero. This creates a destabilizing economic asymmetry between attackers and defenders. Moreover, because the worm requires no commercial AI platform, centralized safety controls, such as service refusals or rate limiting, are structurally irrelevant. Our results demonstrate that self-sustaining AI-driven cyber-threats are no longer theoretical. We must prepare for autonomous generative adversaries: malware systems that propagate without human operators and are defined not by fixed exploit code, but by the capacity to reason about targets, adapt to observations, and synthesize attack logic in real time.

Thirty-eight years after 1988, we now have AI enabled malicious code leveraging the Trinity of Trouble with automated goal-driven intelligence for next to no cost. Expect things to change.

This story was broken in the New York Times by Cade Metz who provides an excellent story.

Cynthia Brumfield wrote an excellent, in-depth article for CSO on recursive pollution that is well worth a read. At BIML, we still believe recursive pollution is the #1 risk for LLM technology. Recursive pollution subtle and nowhere near as sexy as prompt injection, but it leads to insidious wrongness.

Read the article here.

BIML is proud to host Patrick McDaniel, an OG of machine learning security (prominently featured in the BIML TOP 5) and a Dean of Research at Wisconsin, for a visit to the BIML Barn. Patrick arrived in Berryville late on Thursday and was greeted with a Liberal or two on the porch. We stayed up way too late talking about AI and security.

In the morning after breakfast, we spent much of the Friday research discussion going over our soon to be released paper No Security Meter for AI. Patrick has been thinking about measuring ML behavior for a long time, and was an early proponent of a whitebox approach. He had lots of very useful feedback for us.

Does science really get done around the kitchen table? Why yes. Yes it does. (And technical talks really get delivered in the BIML Barn.)

We ventured into greater metropolitan Berryville for lunch and coffee.

And then Patrick delivered a new talk as a BIML in the Barn feature to be released on May 13th. Patrick’s talk really surprised us and in very important philosophical ways.

After the talk we shared a cocktail on the patio. Maybelline is an honorary BIML dog.

Patrick enjoys a well-deserved Lemon Mint Fizz.

And then it was off to dinner with BIML spouses at Huntōn in Leesburg.

Fantastic visit. These kinds of human interaction are absolutely critical as we construct a reasonable approach to machine learning security.

Gary McGraw, cofounder of the Berryville Institute of Machine Learning, pointed to a core gap: Today’s benchmarks tend to measure how well AI systems can perform security tasks—not how secure the systems themselves are. Companies need to keep that distinction in mind when evaluating their tools and defenses.

McGraw warned as far back as 2019 that securing machine learning systems would be “one of the defining cybersecurity struggles of the next decade.” That moment has now arrived.

“These meetings are a way to remind ourselves of the fundamentals,” he said, “as we try to define what machine learning security actually is.”

What was to be a more standard copy of the BIML risk talk, instead was transformed into a debut of BIML’s forthcoming paper No Security Meter for AI. (expected mid-May) for an audience of NIST computer scientists.

It’s always fun to debut a talk for an audience that is engaged and knowledgeable.

While we were inside the very industrial Chemistry building for a talk that was 80% zoom, it rained outside.

Well, maybe. (McGraw proposed the name which is being vetted.) We did all get together in Arlington 4.21.26 to discuss policy and AI. It was a good meeting set up by OWASP and SANS and run very professionally by Rob van der Veer.

The cool thing? BIML’s work was not only cited, but included.

The meeting setting was gorgeous.

As usual, the hall track was the best part of the entire day…especially when the hall was moved across the street to the bar.

See this coverage of the meeting: Global AI Security Standard Organizations Gather Under MOSAIC to Reduce Fragmentation, AI security leaders gather in Washington as risks mount—and Mythos raises the stakes

Decipher covered the controversy over Mythos and Glasswing in an insightful article (which featured BIML’s opinion prominently). Have a look at the article and the video attached below.

Apparently, AI was used to steal original art from an aspiring musician. This is something that eminent philosopher Dan Dennett warned us about in 2022 (see the video link at the bottom of the BIML Bibliography). Though we mostly focus on technical risks built in to ML, the risk of deep fakes is a very real one that deserves more attention.