Too Dangerous to Release (Again): Software Security and AI

Have you heard? The mythos model from Anthropic is so dangerously good at finding software vulnerabilities that its release must be initially limited to companies participating in the Glasswing software security project! {Oh my. Also lions and tigers and bears!}

Does that sound like a marketing ploy to you? Because it does to most expert bug finders that I know best. In fact, the software exploit community (some of whom make a very good living selling bugs to the very companies that produced them…LOL) is pretty evenly split on this issue. So what is a grownup to think?

Those of who have been around the block a few times in AI-land remember way back when Chat-GPT2 was too dangerous to release too (because it could generate fake news even faster than a political PR flak). That garnered some press and helped with the launch for sure. Well, it’s happening again…just look at the tech headlines! Go, Anthropic, go!

Fortunately, there is some balanced coverage out there adopting a thoughtful approach (thanks, Cade). Here’s what we think:

  1. We still have a very real software security problem, so ANYTHING that helps people find AND FIX bugs in code is good. Everyone who is serious about software vulnerability has been using Agentic AI to do this better. You should too. Want to get started using AI to find bugs? Hold your nose (because LinkedIn) and check out this link. But please also figure out how to FIX the bugs you find. And don’t expect to be paid for slop.
  2. LLMs really are good at helping find easy vulnerabilities, but expert mode requires human experience and expertise. Will you become Halvar Flake by strapping on mythos? No, you will not.
  3. Building exploits that really work is much harder than just finding bugs. In fact, I wrote a whole book about this in 2004, 22 years ago, and it is still true. Patching is also harder than finding vulnerabilities. Hopefully AI will help with both of these software security activities.
  4. AI tools are all helpful in different ways. Use them all. Use the ones that are already released. (We hear tell that a well prompted Opus-4.6 (82%) does nearly as well as Mythos (84%) on CRSBench…which calls into question just what the hell these benchmarks measure—a topic we have been thinking about a bunch.)

As a last thought, we’re going to appeal to the four I’s that excellent human designers are familiar with: Intuition, Insight, and Inspiration (the fourth one is the “self” kind of I). AI is great and we love it. We are really going to need lots more software architects, information architects, designers, actual building architects, and humans who know what they are doing. If you know what you’re doing, you’ll be fine. If you are simply a bullshitter, you’re toast.

0 Comments

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>