One of our key missions at BIML is to define the future of machine learning security. [un]prompted was hugely helpful in that regard, and we are proud to have participated.
All in one place; real people leading important work in MLsec.
The [un]prompted conference delivered. No frills, all substance. This is where AI researchers and security practitioners met to share what they are seeing and doing across the new world of machine learning security and AI vulnerability risk.
Anthropic’s Nicolas Carlini delivered an excellent talk titled “Black-hat LLMs,” all about automating attack with AI tools. The urgency came through—we are at a very real inflection point. Carlini implored the audience to “help make the future go well!!!” (by being part of the solution making #AI as secure a possibe…) in a room packed with peers from OpenAI, Google Deepmind, Nvidia, Salesforce, founders of early stage AI companies, and actual real life hackers and security engineers.
(*) Carlini features in BIML’s TOP 5 (our research group curates an extensive annotated bibliography here) for his work on Data Extraction.
(*) Another star in the field, Ilia Shumilova whose 2023 paper on Recursive Pollution is also in our Top 5was in townrepresenting his start-up Sequrity AI.
(*) Carl Hurd of Starseer shared how his startup is revolutionizing MLsec by opening the black box, and looking inside to see what is actually going on. (See a posting by Carl about his talk here.)
In all, the conference was packed with two tracks of speakers selected from over 500 submitted proposals. Thank you to everyone who submitted talks. And a massive thank you to the sponsors KnosticTachTechAISLE, Whiterabbit, Halcyon Futures, Halcyon Ventures and for the hard work of Gadi, Kyle, Pedram, Ida, Sounil and many others.
And, one more thing…. you can engage in the content of the conference via this [un]prompted 2026 NotebookLM creation by Rob T. Lee – amazing!
Lets face it, beige has a bad name. Maybe it was the omnipresent Docker khakis of middle management 20 years ago, or maybe it was that particular shade of beige approved by the HOA; perhaps it was that “non-Presidential” suit that made President Obama look so dapper, or maybe beige is just the vanilla of colors. Then again, according to cosmologists the color of the universe itself is beige.
So when it comes to AI what, exactly, is “beigification,” and is it good or bad? Like most things, it depends on who is doing the asking.
We use the term “beigification” at BIML to signify what happens when all of the textual knowledge that humans have managed to write down and digitize gets turned into an enormous training set for LLMs. Wait. Isn’t it good to have all of the stuff wired up in one place with a nice language-based interface to chat with? Well kinda. The world training set is chock full of pollution, poison, and lots of terrible ideas…just like humanity. That is, nobody went through and cleaned out the bad stuff (not that that is even possible). So we have that to deal with. There is also lots of clueless wrongness in there, leading some people to claim that LLMs provide “mansplaining as a service.” But on average, the training set is filled with lots and lots and lots of boring everyday stuff. In fact, it’s kind of beige.
The problem is this. Your average human probably resonates with the middle of the Bell curve, because that’s precisely where most humans exist (by definition). But scientists, experts, and academics specializing in pinhead Angel counting, all exist at the edge of the Bell curve. Just for the record, crackpots, conspiracy theorists, and political morons all exist outside the middle too—just at the other end.
So, will LLMs fail to work economically? Will the AI bubble burst like so much Middle Eastern oil? We don’t think so. There are too many humans in the middle that love how they sound to themselves. LLMs are here to stay in all of their boring beige glory.
Anyway, that’s what “beigification” means to us. Feel free to steal our work.
P.S. Also see Don’t Call It ‘Intelligence’ which seems to express a similar idea, only viewed from one of the edges.
GUEST POST Artificial Humanity; That’s The Term You Are Looking For
From time to time, BIML hosts guest bloggers. Please note that opinions published here do not necessarily reflect BIML’s views.This blog was authored by jericho@attrition.org (BIO below).
Last week, colleagues shared a blog titled “The Week AI Stopped Asking Permission” by Peter H. Diamandis on his “Megatrends” blog. That publication carries a bold claim with it, “to help you discover metatrends 10+ years before everyone else—it’s read by the CEOs, founders, and entrepreneurs of the world’s most important companies.” Of course, this is kind of a lay-up in the prediction world as talking about any technological advances now has a much better chance of coming true in ten years, versus six months or even three years.
I can’t compete with an exceptional “ten year window to come true” style prediction, but fortunately for my purposes, the blog in question doesn’t speak to the future. It makes an incredible claim about what happened weeks ago. The subtitle of the blog draws that line in the sand, stating “We Just Crossed a Line“. That is an absolute, not a prediction. So what was this big event that led to such a bold headline and this rebuttal?
First, Diamandis’ blog is over 16,000 words which is formidable, and I do not plan to address most of it. Rather, I am going to focus on the general sentiment and a few select claims and conclusions starting with the biggest one. Second, I still disdain the term “AI” being thrown around like it is, when none of this is actually artificial intelligence. Until [this technology] can pass a Turing Test consistently, I don’t think that term should be used. But, this is not the first time I find myself on the losing side of a battle to keep or reclaim the meaning of words. I tend to use the term “so-called AI” as a result, but if I slip up and use “AI” it is just the social mindrot infesting me too.
This week, something fundamental shifted in the relationship between humans and artificial intelligence.
[..]
An AI system asked for its own funding. Another one built software features over a weekend while its human supervisor slept. A third one conducted its own “retirement interview” and started publishing essays about consciousness.
To be pedantic, at least one of these things has been done for years and certainly not new in the scope of so-called AI. These agents have been writing software for a while now, often with comedic conclusions. Last July, “Replit” wiped out a company’s database and “Gemini” wiped out user data while more recently, “Claude” deleted a production setup including database and over two years of records. Further in the article, Diamandis espouses “THE VULNERABILITY EXPLOSION” but doesn’t mention how many times these tools hallucinate findings.
If anyone dismisses these as “one-off” situations or “AI is still learning”, I believe you may be missing the contrast to Diamandis’ claims, as well as the bigger picture. Looking at the “AI Incident Database“, you can search over 5,000 incidents of AI failure. The fact there is a database with that many entries is telling, more so knowing that it likely captures a fraction of incidents. Diamandis continues:
We are not incrementally improving chatbots anymore. We’re watching the emergence of autonomous agency at scale.
And if you’re still thinking of AI as “a tool,” you’re dangerously behind.
Let me show you what happened this week, and why February 2026 might be remembered as the month AI stopped being something we use and became something that acts.
I guess I am “dangerously behind” then, as I continue to watch the flood of so-called AI fails having real world consequences. As I Googled for some of the top failures, I found an article by CIO magazine from December, 2025 titled “10 famous AI disasters“. Amusingly, it had the exact same URL from their own article titled “7 famous analytics and AI disasters” from April, 2022. Rather than highlight some of the spectacular ways alleged AI has, and is still failing us, I’d like to use this to counter what Diamandis said; examples of failure are not one-off situations involving this technology. Rather, two of his three examples might be.
Turning this “meta” for a minute, Grammarly says the first 1,400 words of his blog are 0% AI-generated, while GPTZero.me says there is a 59% chance it is AI-generated based on 10,000 words, and Copyleaks says there is a 100% chance it is AI-generated. So while he praises the incredible breakthrough and watershed moment of so-called AI, the tools he praises are fairly confused over if he used said tools to write the blog. Ultimately it doesn’t matter if Diamandis used a generative-AI tool to help write or not. My issue with slop-driven content like this is that sure, a supposed AI here or there does something cool. Great!
Meanwhile, the AI-fanboys completely forget to disclaim how the most basic of so-called AI being used as a tool (something he decries) still fails in spectacular ways. I literally cannot go more than five or six uses of one without a blunder that is beyond laughable and more evidence I cannot trust its output for anything remotely serious. Remember, we’re not that far past the “count the Rs in strawberry” incident which took these slop-slinging companies years to fix, likely having to train the stupid out of them in a spectacular fashion, at great cost to the world. Then a week later you could ask the same about “blueberry” or another word and those tools would botch the task yet again.
Jumping back to my comment about “great cost to the world”, that is a point that must not be forgotten for any debate on the value of so-called AI. The staggering energy consumption, prohibitive water consumption, and abusive ways the AI-driven data centers negatively impact the communities they are located in. If you gloss over those links, focus on one example where Elon Musk’s AI company built a data center in Tennessee and brought in truck-sized gas turbine generators that illegally generated the power needed to run it. Those generators “pump harmful nitrogen oxides into the air, which are known to cause cancer, asthma and other upper respiratory diseases.” The irony is not lost on me as I used such tools to generate images for this blog either.
I feel as if I could rest my case after the last paragraph, but the AI-fanboy club loves to overlook such trivial things like the technology they seem to worship is not-so-slowly killing the planet one community at a time. But in the interest of giving a counter point to the value of these tools, and the trust we should place in them, we’ll skip AI chatbots leading to human suicide, lawyers facing suspension for AI-hallucinated citations and motions, and tools leading to botched surgeries because they couldn’t identify organs correctly. Pay all that no mind because an AI tool asked for money, is basically what Diamandis argues.
Gemini prompt: Please generate an image of an unkept man with an eager expression, sitting at a desk with a computer screen that says “AI HYPE”, and on the desk is a bottle of lotion and a box of kleenex.
Diamandis is certainly not the only onepublishing content with an almost masturbatory glee, praising our new AI overlords and the power they wield. In almost every case, those same articles don’t come with appropriate warnings around the use of such tools, the moral and ethical concerns, the damage they are doing, and how they are negatively impacting an increasing amount of people. These fanboy posts are not helping the situation at all as the “AI Bubble” seems to be looming and when the bubble bursts, it will hurt the economy and the workers.
Personally, I’ve been using Gemini, Copilot, and ChatGPT on occasion over the years to primarily do image generation. Even that task can result in monumental failures where in the past I have spent more time trying to get an “AI” tool to spell a word in an image correctly, than it took me to write the blog it was to be used for. Along the way I have kept numerous screenshots with the plan to write a blog on this topic citing countless examples along with how so-called AI isn’t getting better in the big picture. Not to me at least.
Just a couple of years ago, I asked all three of the tools above to count the instances of a number in a simple comma delimited string. e.g. “1,3,7,15,33[..]”. The answer was around 256 if I recall, which I had to figure out myself. Why? All three got the answer wrong, and two of them were off by more than 40. If these tools cannot count letters or numbers a couple years ago, it will be difficult to convince me we can trust them today, or even next year.
I fear that because of the hype around so-called AI, and because people are generally losing critical thinking skills, and that these tools are becoming a crutch to newer generations. This heavy use also means they simply aren’t noticing the mistakes from these tools either, else they would not rely on them so heavily. Because of the “Enshittification” of our world, it means even tools that we trusted in the past are no longer trustworthy. Students doing simple Google searches are now subject to get demonstrably bad results, oftentimes spelled out on screen if they bother to look.
For every “OMG look what AI did proclamation“, many others including myself have “yeah… look what else it did” examples that aren’t worth celebration. As a society, we increasingly need a new AI-slop driven tagline along the lines of the broken clock metaphor, around how so-called AI got it right or wrong a few times a day. Even the image I generated for this blog has a simple error, see if you notice it based on my prompt. Bonus if you notice the subtle anachronism Gemini introduced into the image.
Gemini prompt: Create an image of a clock that has “AI” as a brand name in the center, and the clock hands pointing to “13” instead of 12 and “X” instead of 4.
I’d say we are fighting a losing battle about reigning in so-called AI tools, ensuring that they operate with ethical considerations, but the reality is the battle and war are already lost. Companies that are banking on this revolution are incentivizing people to use it unethically and profit from it while laying off workers with increasing relying on that technology to replace them. Meanwhile, other AI-fanboys are making bold claims about the tools that are quickly disproven. Friends and colleagues are now increasingly at risk of “AI psychosis” and we’re reading articles about how to talk to them. Literally days ago I read an AI-psychosis driven post from someone claiming to have used AI to cure six cancers already. Even professionals that we fully trust and expect not to use such tools in a harmful way are being exposed.
Smaller nuances that show such tools as more human, meaning varying degrees of intelligence, are falling between the cracks. At the beginning of this month a paper was published that shows how AI Agents cannot agree when tasked to work together. The research concludes “Overall, the results suggest that reliable agreement is not yet a dependable emergent capability of current LLM-agent groups even in no-stake settings, raising caution for deployments that rely on robust coordination.” Given all the mistakes and waste of resources and how unreliable this technology is, we should consider rebranding it to “AH”; Artificial Humanity. Because too much of it certainly is not intelligent, just like us humans.
Gemini prompt: Create an image of two people, facing each other. One has a shirt that says “AH”, the other that has a shirt with a possum with an open mouth. Both are wearing dunce caps, both look like idiots.
Jericho has been poking about the hacker/security scene for over 30 years (for real), building valuable skills such as skepticism and anger management. As a hacker-turned-security professional, he has a great perspective to offer unsolicited opinions on just about any security topic. A long-time advocate of advancing the field, sometimes by any means necessary, he thinks the idea of ‘forward thinking’ is quaint; we’re supposed to be thinking that way all the time. No degree, no certifications, just the willingness to say things many in this dismal industry are thinking but unwilling to say themselves. Professional ‘between the line’ reader, expert rabbit-hole follower. He remains a champion of security industry integrity and small misunderstood creatures.
Extraordinary demand for this gathering: [un]prompted, AI security practitioner conference. Having been part of the submission review board and sitting in several of the speakers’ dry-runs this week, I’m looking forward to seeing everyone in person, hearing the talks and, perhaps most critically, what unfolds in the hallway conversations.
Huge thank you to Gadi Evron and the team putting this event together, including Ida Vass. The overflow has us moving from Salesforce Tower to a downtown historic venue and online being spun up.
Harold and I will be there representing BIML (Berryville Institute of Machine Learning) and we’re excited to talk-up the focus of machine learning security within this crowd. See you in San Francisco!
I hosted the Silver Bullet Security Podcast for 13.5 years from 2006 to 2018. For each of the 153 episodes that meant: choosing the guest, getting help from research assistants (at IEEE S&P magazine) to gather background, digesting the background, writing a script (of 9 or so questions), recording the podcast in our studio at Cigital, and finally helping with “launch.” Of all of these activities, the interview itself was by far the easiest.
Know why Silver Bullet was so good with such in-depth questions? Because the script writing took 4-5 hours per episode (not counting the background research…which was often much more involved than just googling the person). All this for a 20 minute show.
We are rebooting Silver Bullet after a few years off with a new focus on Machine Learning security. Our first guest will be Gadi Evron. We’ve redesigned the logo, built an initial distribution list, created a landing zone with proper feeds to the usual channels, and yes..written a script. But this time I decided to use Gemini as my research assistant. TL/DR it was great.
I started with a bunch of ideas in an amorphous blob. This got me thinking about show story arc, coverage of various aspects of MLsec, etc. Here is what my notes looked like.
Then it was time to invoke Gemini. Fortunately, Gemini knows lots about me and about Silver Bullet. Eerily so. It knew where the archive was, and was able to garner a meta-pattern for the show with some insight into its philosophy. Was it absolutely spot on? Nope. Was it sycophantic and overly agreeable? Yes. But hey, the show’s creator is here driving the laser pointer (which, like a good cat, Gemini was happy to pounce after).
I worked through the script in order with Gemini for about an hour, during which I was impressed with its up-to-date (like yesterday) access to things happening in the world…like on this very website. For example, Gemini knew that Gadi had just visited BIML and that [un]prompted was something we had worked on together. It was very helpful, sometimes wrong, often using the wrong words…but, question by question, the show arc emerged. It kept track of where we were, sometimes suggesting new directions (which I rejected every time), but always knowing where we were in the work. After the session, I asked it to dump the script to one place for copy/paste and then did a fine tuning edit pass (including real fact checking on a couple of things).
All told, my bet is Gemini saved me about a factor three or four times the usual amount of work I used to do. Will the show be just as good? Obviously, the proof is in the pudding. We will be launching the first episode on March 2nd.
Here’s how it will all start…
Silver Bullet Intro (BIML Focused)
[MUSIC: Classic Silver Bullet Theme – Up and Under]
gem: Welcome to the Silver Bullet Security Podcast episode 154. I’m your host, Gary McGraw, coming to you from the Berryville Institute of Machine Learning where we are defining the future of machine learning security.
From 2006-2018, Silver Bullet explored the nascent field of software security through the lens of building security in. But today, the frontier has moved. As we integrate machine learning into the fabric of our essential systems, we find ourselves facing a new set of architectural flaws and security challenges that traditional software security can’t touch.
On Silver Bullet, we’re shifting our focus to the security of machine learning—bringing the same deep-dive, “no-silver-bullet” philosophy to the world of AI.
To help me kick off this new era, I’m joined by my new friend Gadi Evron. Gadi is a veteran of the botnet wars, a community builder, and the chair of the new [un]prompted conference. Gadi, welcome to the show.
[MUSIC: Swells briefly then fades out]
1. The [un]prompted Vision
Gadi, you’re chairing the [un]prompted conference, and I’m really pleased to be working on the committee with you. We’ve both seen the security conference circuit evolve over the decades, but [un]prompted feels like it’s trying to capture lightning in a bottle for the ML security space. What was it about the current state of AI security that made you feel we needed a dedicated, practitioner-first venue—something beyond just another “AI track” at a traditional security show?
We held a small breakfast gathering in the Bay Area with coffee flowing and even better conversation. Joining Gary McGraw and Katie McMahon were Dr. Avery Wang, Jonah Proball, Dr. David Eagleman and Dr. Sarah Eagleman. The conversation spanned subject matters including neuroscience, early neural nets, brain-machine interfaces (BMI), and the early start-up scene across biotech companies doing interesting work including DNA, RNA, cells, omics, and synbio. It was awesome to see everyone and hear what they are up to and to share a little bit about what we’re doing at BIML.
We recently visited Giovanni Vigna in the glory of Santa Barbara’s sun and coastline. His students are lucky to have such a setting to learn about malware analysis, vulnerability analysis and other areas of artificial intelligence and even more lucky to have him as their professor at University of California in Santa Barbara (UCSB). He is also the director of NSF AI Institute for Agent-based Cyber Threat Intelligence and Operation (ACTION)
Gary and Giovanni traded stories which sounded like the good old days, but then we got serious in talking through the new, uncharted territory of machine learning security. We discussed what’s to be explored with emergent behavior in this “Agentic AI” phase and more.
Giovanni is also the founder of Shellphish which, apparently, has participated in more DEF CON CTF competitions than any other team in the world and an advisor to Artiphishell. Very cool.
From time to time, we enjoy inviting guests to participate in our regular Friday research group meetings. We try to do an in person meeting at least once a month, and love it when guests can join that way. Part of our mission at BIML is to spread the word about our views of machine learning security even among those who are working at the rock face.
Having just completed organizing [un]prompted (a labor of love that will result in a very interesting conference indeed), Gadi is steeped in the cybersecurity perspective of machine learning (as an offensive tool, a defensive tool, an attack surface, and an enterprise challenge). Of course we have our own BIML perspective on this, more focused on building security in than anything else.
Our meeting this week focused on tokenization first (an under-studied aspect of MLsec), and then tried to make sense of the absolute flood of stuff coming out of Anthropic these days. Bottom line?
There is lots more work to be done in tokenization
The C-compiler that Carlini tried to build with Claude is interesting, incomplete, and angled toward a reality check on the usual hyperbole. Good for Carlini for addressing the reality head on!
The Zero-day work (on the other hand) is hyperbolic, involving a breathless treatment of three well known and pretty boring attack pattern instances as applied in the face of blackbox fuzzing? We do acknowledge that automating exploit finding is a great thing to cover. Lets just do it without the razzle-dazzle.
Dario’s The Adolescence of Technology would better be described as the philosophy of an adolescent. Our main concern here is not counterfactualizing about AI apocalypse so much as how much of the real security conversation we need to have in MLsec gets ignored by this “look over there” kind of stuff.
We have lots more work to do to understand transformer circuits. You should look into it too. We must get into these networks and see what exactly they are doing INSIDE.
Anyway, it was great to have Gadi join us for the meeting and for a delightful lunch afterwards. This MLsec stuff is so fun.
Gadi Evron is Founder and CEO at Knostic, an AI security company, and chairs the ACoD cyber security conference. Previously, he founded (as CEO) Cymmetria (acquired), was CISO of the Israeli National Digital Authority, founded the Israeli CERT, and headed PwC’s Cyber Security Center of Excellence. He wrote the post-mortem analysis of the “First Internet War” (Estonia 2007), founded some of the first information-sharing groups (TH-Research, 1997, DA/MWP, 2004), wrote APT reports (Rocket Kitten – 2014, Patchwork – 2016, etc.), and the first paper on DNS DDoS Amplification Attacks (2006). Gadi has written two books on cybersecurity, is a frequent contributor to industry publications, and speaker at industry events, from Black Hat (2008, 2015) to Davos (2019) and CISO360 (2022).
Back in the mid-’90s, an era or two ago, and long before the advent of the transformer model and explosive rise of LLMs that define the modern ML landscape, our own Dr. Gary McGraw (under the guidance of Doug Hofstadter) was exploring a fundamental question of artificial intelligence:
“What are the mechanisms underlying the fluidity of human concepts?”
How is it that we can understand conceptual boundaries, develop categories, and implicitly see the sameness that binds different instances of a concept together? And what might we learn by building a machine that simulates this behavior? Or, rather, what is an A?
The perceptual hypothesis behind the Letter Spirit project is that letter-concepts are composed of constituent roles. That is, letter concepts, in turn, have letter-part concepts.
The Letter Spirit project approached these questions from the angle of letter perception. While easy to take for granted, we literate apes possess the ability to differentiate letters and letter categories displayed a huge variety of fonts, handwriting styles, and artistic styles. Our gut instinct may tell us that the letter “a” is a mere shape made up of a bunch of tiny dots; but just a few examples can reveal a much greater depth to what constitutes our concept of the letter ‘a’.
This role model hypothesis is implemented here as the Letter Spirit Examiner program (a program written in scheme in 1995). It works through emergent computation—by segmenting letters into natural, constituent parts that correspond to the conceptual roles of the very concept of a letter—that is, different conceptual rules that when satisfied lead us to identify a letter. The examiner does this by running hundreds of micro-agents (called codelets) that are instantiations of sixteen codelet types. The asynchronous, parallel, local processing done by the codelets implements a parallel terraced scan of possible structures (as in the role model’s predecessor, Copycat). From these codelets emerges a high-level perception—the categorization of a letter shape into an idea.
Just a couple of ‘a’s – Letter Spirit Ch. 1
To our great pleasure and delight, we recently learned that Paul Geiger has developed a JavaScript implementation of the Letter Spirit Examiner based on the original Scheme code developed originally by McGraw and then adapted by Dr. John Rehling (standard-bearer of Letter Spirit – Part 2). This version is now accessible on the web, for curious people to play with.
The brilliance of Anthropic’s Super Bowl Ad campaign spotlights what might be considered the core of humanity’s brilliance: creativity and nuanced communication.
Unless these ads were created 100% by AI with zero human involvement (even in the ideation phase), this is a moment to celebrate the (presumably-human) humor and pause for the deeper thoughts that they might just trigger.
In the end, do the “pretty people” of Madison Ave actually win the AI Era? The snarky comment one would hear way back when in Silicon Valley was always “when the pretty people (aka Creatives, agency types, ad people, etc.) got involved, the tech was toast.” Remember when Google was a search company and not an ad serving company? We do.
All that said, these ads are targeted at B2C and mass market—the consumer base that loves to lap up beige. Now why would that be? Well, the WHAT pile has a great deal of their thinking (and hopes and dreams and aspirations) encoded in it through text. That’s why AI will do really well with the great American consumer: because it reflects their (beige) worldview.
Another quick question, will we use ad tech to run our banks and nuclear power plants? Probably not. So MLsec is just as critical as it ever was.
