gem
This is the Silver Bullet Security Podcast with BIML. I’m your host, Gary McGraw, CEO of the Berryville Institute of Machine Learning and author of Software Security. This podcast series is sponsored by BIML, a nonprofit science and technology organization whose research focuses on machine learning security. For more, see berryvilleiml.com/podcast.

And I’m pleased to have today with me, Greg Kurtzer. Hi, Greg.

GREGORY
Hello. It’s great to be here.

gem
Greg Kertzer is the founder and CEO of CIQ and president of the Rocky Enterprise Software Foundation, a high-performance computing pioneer with over 25 years of systems software experience. Kertzer is best known as the original creator of CentOS, Rocky Linux, Werewolf and Singularity, now AppTainer under the Linux Foundation, which transformed containerized environments across scientific and supercomputing infrastructure.

A relentless advocate for open governance, reproducible computing, and supply chain transparency, he co-founded the Open Enterprise Linux Association, Open ELA, and leads OpenWaldo to bring auditable data provenance and open source rigor to modern AI ecosystems. Kurtzer holds a Bachelor of Science in Biochemistry from Holy Names University.

Hi, Greg.

GREGORY
Hi.

gem
Greg, you’ve spent your career building transparent, high-performance open infrastructure because you know that systems you can’t inspect are systems you can’t trust. Yet today, the entire tech industry is racing to hand over core operational logic to uninspected, uninspectable black box models. Why are engineers making the exact same foundational mistakes with machine learning that we fought to fix in early operating systems, and how do we stop the industry from sleepwalking into a complete loss of architectural control?

GREGORY
That is a great question. I’ve kind of been describing it as there’s two populations that exist. There’s the open source group who historically have been building and managing and securing a lot of the infrastructure that runs the internet, that runs enterprises, that runs personal devices. And they’ve been directly impacted by how open source has made, and not to sound cliche, made the world a better place. Through that transparency and through that community and collaborative effort on standards and everything else that open source brings to the table, they’ve lived to see it, they’ve been part of that, and it has made their lives better.

On the other hand, on the other side of the population, we have a number of people that have been doing research in AI, and the research and work that they’ve been doing in AI has been incredibly and massively just amazing. It’s been impactful. The innovations that they’ve been able to drive have been fantastic. And a lot of the tooling around AI is in fact open source. But when we go to create these models, we are using various bits of data, where the data side of the coin has never been quite so clear when it comes to open source.

gem
That’s a very kind way of putting it.

GREGORY
So, to build these AI models, you have to have a tremendous amount of data. A huge amount of data. And you’re training on this data and you’re organizing this data in such a way that the end result is going to provide logical outcomes, and so people are kind of confusing a little bit where the IP is and where the innovation is.

And so, again, on one hand we have, you know, the open source people who are like, you know, everything should be open source. If you’re going to call something open source, the source should be open. Kind of in the name. And the source for an AI is the training data. It’s the tooling. It’s the build process. It’s how you’re composing that model. But I think a lot of people are actually really confused on this one because…

gem
Well, I was going to, you know, really address that. Here’s what I think. Let’s just cut through the marketing around open source AI. All right, so a lot of vendors publish pre-trained weights, and they claim they’ve delivered open source. But from a security perspective, an open weight model is basically a compiled binary with no source code and no build instructions. So you can’t inspect how it was trained. You can’t verify if toxic or malicious data was baked into the substrate. You can’t patch it or recompile a model if a fundamental flaw emerges later.

So why is releasing static weights without full training pipeline provenance, including raw corpora, filtering scripts, and RLHF parameters, kind of security washing? It’s sort of an illusion. And why does this missing history make real risk analysis impossible for down-fundal engineers? So I want to address the technical aspects of this more than the social aspects.

GREGORY
That’s a great point. So when you’re training a model, the data that you’re using to train that model is what is going to generate the weights and the probability and statistics of what the next tokens that it’s going to complete are going to be. And if you are training an AI model on, let’s just say, the Terminator series, don’t be surprised when it believes that, you know, machines are great and it should exterminate all people because we are in their way. And I’m being somewhat…

gem
Skynet is good.

GREGORY
I’m being sarcastic about this, but at the same token, it really kind of articulates the point. And if you have enough data on an entire fictitious world, and you train the AI model on that, it will confidently describe and respond to questions as if, you know, it is built and based on that fictitious world. So the training…

gem
Absolutely right. I mean, you know, that artificial data idea sort of forgets to tell you that part. So I think you end up with like a Potemkin world. So you’re going through, you got your data that you’ve made. You know, it’s fake artificial data, and you eat it and you build a model of your artificial data, and you have a little pretend world.

GREGORY
You can definitely do that. And the people who are actually, you know, claiming that models are now conscious all of a sudden are exactly like, there’s certain arguments you can bring to the table. Although, as you know, with many religions, it doesn’t matter what facts you bring to the table.

gem
I love it.

GREGORY
They’re going to believe it. And they’re going to believe AI is conscious. It’s going to do what it’s going to do. But here’s where this is really important, and it’s the point you were making, which is… If you train the model on bad data, let’s say for example you’re building a coding model, and that coding model has tons of code examples, all with similar backdoors in the code, and you train your model based on all of this data. Now, I’m going to say it’s a hard thing to do because data is, you need so much of it to train a model. But if you take those code snippets, those code examples, which are trained with malicious code inside of it, don’t be surprised when the AI starts injecting some of that malicious code into your own projects that you’re using it for coding. And if we don’t have the capability of looking and auditing, there’s no way to go from the weights backwards to figure out what was the data it was trained on. You lose provenance.

gem
That’s, I’m totally with you. Do you know Dave Evans’ work on Trojan stuff from the machine learning security pile?

GREGORY
I need to go back and research that some more.

gem
It’s a good one. I think you would like that paper.

So let’s talk about data pollution. We’re rapidly approaching a tipping point where the open internet is flooded with synthetic model generated text and code and media and everything else. And as new models scrape this synthetic noise to train future generations, we run straight into what we call epimal recursive pollution, which is a feedback loop where statistical artifacts and bias and poison get exponentially amplified in every training cycle. And it eventually leads to model collapse, which is what some people call this phenomenon.

If our computational substrate is being continually contaminated by the unvetted outputs of previous machines, how do initiatives like OpenWaldo actually wall off clean training corpora? And how do we prevent future machine learning systems from consuming their own digital waste?

GREGORY
I love that. So OpenWaldo, and I’ll give a quick kind of background on OpenWaldo. OpenWaldo exists really to be kind of the open source infrastructure, tooling, and data around AI. And the first thing that I started with for that was the training corpus. And there’s a lot of challenges with building a very scalable training corpus, as I kept saying, like the amount of data necessary to build a big model is massive. And so to do this, you need a ridiculously large amount of data and you need to be able to host that. And my goal of it was to be able to host that in a way that is compatible with the open source kind of way that we do things. Over the last 30, 40 years, we’ve created a ton of really great innovations, capabilities, methods to manage provenance, et cetera, accountability, all within open source. And how do we bring that to something that is such a large training corpus?

So I leveraged some of my background in, you know, how do you manage an open source Linux distribution that has tons of binary artifacts, tarballs and other things, where you can’t just throw everything in Git. So the way we do it is we use look aside caches. And that look aside cache is indexed in Git, but that index is pointing elsewhere to where those look asides exist. So what I created was an index that is all in Git and the appropriate tooling to manage that index, so people can come in and add their own material into this corpus or review or audit the material in the corpus, where the index and the metadata for this all exist in the index. So the index is in Git. So it makes it really nice for somebody to actually go and submit a PR to add more data into the corpus. And the look is…

gem
I like it. I mean, as an author, I feel like I want to give you my books.

GREGORY
Love that. And so to the question now directly: how do we kind of ensure that we’re not bringing in a bunch of AI slop? And while we can never completely say we’re not going to bring in AI slop, what we can say is at least it’s auditable, and we can build a situation where the quality of the data that you’re ingesting will get you better results within the AI.

So in marketing, people have been talking about SEO for decades, search engine optimization. It’s how a particular website or a particular page or landing page will end up ranking reasonably when you do a particular search. It’s kind of the same premise that we need to solve in AI. Companies are like, how do I ensure that my company, my products, my services, and my company itself is actually getting ranked in AI responses? And so the way everybody’s approaching it is they’re basically just flooding their own websites and their own documents and whatnot that’s public, and they’re just flooding it with content hoping some AI search engine or crawler is going to see it and then add it to the training data. It’s actually a worse case even than SEO, and this goes back into something we already touched on a little bit, which is the training data itself. Like, what’s the license of data that’s on a website? I mean, we all kind of know at this point, if you’ve been in open source for any amount of time, if there’s no license, it’s covered under copyright. And copyright law is pretty well understood. The question is, is this fair use? And that question is still open for debate. There’s been a few suits on this, lawsuits.

gem
The suits are ongoing. I mean, you know, Anthropic apparently owes me some cash, but I haven’t seen anything yet.

GREGORY
I know a few authors that have actually gotten eight or $9,000 checks in the mail from that $1.5 billion settlement. So.

gem
What? I haven’t gotten mine yet. Why not? What’s going on? That sucks. I’m going to have to go, you know, talk to Addison Wesley, those buggers. They probably took it.

GREGORY
So… The question is going to come at some point, what is this data trained on? If the data is trained on copyleft instead of copyright, you know, so if it’s GNU licensed software or GNU licensed data, I mean, those licenses are viral. If a company is writing code based on a model that was trained on copyleft GPL GNU code, like, what does that mean for the code that they’re creating? Is that a derivative? Is it not a derivative? Like, there’s so many…

gem
These are tricky things.

GREGORY
Too many open questions right now. And I know a lot of companies that are, they’re on the AI bandwagon, but they’re like, we’re not going into this until some of these questions have been answered.

gem
Well, they better not hold their breath because it’s going to be a minute.

GREGORY
And when and if it gets answered, it may be a catastrophic backlash on how we’ve been doing all of this. So inside of the OpenWaldo index, for every corpus of data that somebody brings in, you have to associate a license to it. And the contributor has to commit to that git repository for the index what that license is, and use the developer certificate of origin, the DCO, to properly take accountability for what it is that they’re…

gem
That is so cool. I really like this. That’s really nice. I want to switch gears a little bit.

GREGORY
Yeah, please.

gem
If that’s cool with you. So I want to talk about high performance computing a little bit. So you built Singularity for that. And you did it because standard container tools kind of failed to handle security correctly because they were giving processes too much ambient power. Today, we’re seeing the exact same mistake play out with AI agents, right? So developers are spinning up models and giving them direct command line access and file system privileges and execution tools without any real operational boundaries. When you have a non-deterministic AI agent executing actions inside enterprise systems, why is trusting standard web container wrappers a massive security gamble? And how should we be thinking about privilege and isolation in an agent-driven world?

GREGORY
Well, the first thought that comes up, is… it goes back to the… there are these two populations of people. And…

gem
Back to the people.

GREGORY
Back to the people. On the open source side, it’s not just the open source people. That side also contains a lot of the infrastructure people, the people that have been managing large infrastructures for decades. It also includes the people who’ve been managing security for those infrastructures for decades.

And I can’t say this for a definitive fact, but I can tell you anecdotally, I know a lot of security experts that are fantastic at what they do. I would trust them with my infrastructure, heck, I trust them with the keys to my house. And they have been, and this is not just like one or two, this is groups of these capable individuals, who have tried to get jobs within a lot of the AI labs and a lot of companies that are focusing on AI first, and they’re being discounted due to either, and it’s silly reasons, but I’ve heard, cultural misalignment, expertise misalignment.

gem
I think, you know, it reminds me of software security, which is a field that I helped to found a long time ago. And the same problem existed then. And we had to invent a new category of people called software security people, who were developers first and security people second. We didn’t take guys who were good at firewall management and try to turn them into software security people. You could in principle do that, but it takes a lot longer than taking a software guy and shoving some security into his brain or her brain.

And so I think we’re in some sense in a very similar situation, and I feel, in my own work in machine learning security, like we are at the very, very beginning. It just reminds me of when software security was like, gosh, why is all the software broken? Why are we just putting a thing in front of it? That’s not a solution. And I think we’re going to make rapid progress, but it’s still early days there.

GREGORY
So I definitely have a point on that I’d like to make. But I also want to really just kind of push forth the notion of the initial question, which was, we’re in a situation where we’re giving a black box, whether that black box is an open weight model that we’ve downloaded from China or it is an API that we’re using from one of the major frontier labs. We don’t know what it’s going to respond with, and that’s kind of the point of AI. And there’s a lot of variables in the mix, and we’re giving it tooling for it to basically do lots of different things without thinking about the security access controls and sandboxing, reasonable sandboxing.

So going back to my point on security engineers and security, you know, software people, like, we are going to need, and this will also fall into my next point, we are going to need to be escalating up the people who understand how to do proper sandboxing, proper air gapping, and proper controls and instrumentation to make sure that the tooling is bound. It is being controlled, and…

gem
Totally agree with you. And we’re going to have to stop saying the agent decided to do X, because agents don’t have intentionality. The last time I checked, it was somebody ran that program and it did a bad thing. And whoever pressed the button is probably who should be in trouble.

GREGORY
Oh, the AI did it should never be a legal defense.

gem
Very, very concisely put and perfect.

GREGORY
So real quick, there’s a second point I wanted to make on this because it’s hugely, hugely important. We’ve been sitting on decades of human-created code. Jokingly, we can call it human slop, as we call AI slop. But we’ve been sitting on decades of human-created and human-curated code, and the wonderful thing about humans is, well, we miss things and we sometimes make mistakes. We’ve got decades of bugs sitting in our software infrastructure right now that nobody has seen. Nobody has found. And we are starting to see an increasingly prevalent situation where all of a sudden, it’s like, the AI is exploiting this. The AI is exploiting this. The AI is exploiting that. And…

gem
And it’s just like, no, we turned on the lights and the roaches are running around.

GREGORY
Exactly. I love that analogy. And it’s going to get worse. We are going to accelerate finding the bugs that exist currently in our human-created code, and that’s going to increase. If you look at the Linux kernel, there’s a graph that Greg Kroah-Hartman posted not too long ago which shows the amount of CVEs that they’ve been doing for a number of different releases, and like, it hockey sticks.

gem
Yes, it’s a huge amount of work. Just huge.

GREGORY
It’s going up and it’s only going to get worse. And it’s not just the kernel, it’s the entire user space. PHP was just afflicted with this over the last week.

gem
I mean, it’s your own fault for using PHP. What can you say? I rolled eight servers. Guilty. Guilty as charged.

GREGORY
I am a huge fan of Rasmus. He did wonderful work. So I’m not going to say anything negative about PHP, but I can’t argue as well that it’s not just PHP. It’s the way people also code in PHP. It’s too easy of a programming language. So people who shouldn’t be coding are coding and then putting it on the web.

gem
You know, I was once on a panel at Usenix Technical with Larry Wall, and Larry said, the great thing about Perl is there’s six ways to do everything. And I said, Larry, I have a different alternative perspective from security. The really problematic aspect of Perl is that there are six ways to do everything, and you only thought of three ways to block the six ways. So you’re screwed. And Larry didn’t talk to me anymore after that.

GREGORY
But… Perl was my second programming language of choice after C. And I was talking with a Python developer at Berkeley Lab a while ago, and he said, you know what Perl looks like to me? If you press the shift button on your keyboard and drag your elbow up and down the number row of your keyboard, you will end up with proper obfuscated Perl.

gem
I love it.

GREGORY
And it’s like, yeah, okay, I get the point.

gem
All right, so let’s hop back on track. Right now the dominant approach in the market is to take that uninspectable black box and wrap it in some sort of secondary classifier models or prompt filters or external guardrails to block malicious behavior. But in traditional software, we learned decades ago that input sanitization and perimeter filters never really fix vulnerabilities. They just create some sort of cat and mouse game, and the attacker is eventually going to win, or somebody is going to come and turn the lights on and they’re going to go, oh my God, look at all the bugs.

So in machine learning security, why is treating architectural flaws, like, you know, let’s say, mixing control and data together in the same channel, as treating that as simple input-output filtering problems, a dangerous delusion? And from an open source infrastructure perspective, what does it actually mean to build an auditable system that’s secure by design from the bottom up, and, you know, not just an open version of the broken design depth we have, seemed to converge on, but something better?

GREGORY
So, in reverse order. Building the infrastructure, the architecture, it again comes down to you have to know what’s going in. So if the source is open, and you’re building a model based on things that are auditable, we can easily say, you know, in this corpus of books and movie text, we’re going to leave out the Matrix and Terminator and other sci-fi examples in which it…

gem
Bad robot behavior is right out.

GREGORY
Yeah, it’s probably a good idea to leave out, you know, how to kill humans off in mass extinction, like… or how to use us as batteries. It’s probably a good idea just to leave that one out. And again, I’m being sarcastic about this to make the point. But there’s a lot of information in training corpuses that we probably could just leave out. It’s like training a child.

gem
But, you know, then it becomes a political question of what’s right and what’s not right, and what’s allowed and what’s not allowed. And that’s, you know, as you can tell, humans are really bad at the political stuff, especially lately. Like, particularly heinously bad at that.

GREGORY
So I agree completely. And this is why it’s so important to have the open corpus of data, because somebody who wishes to build a model that, for example, has no medical information… We just had a situation where some poor girl passed, because the AI was telling her that the medication that she needed was in fact optional, and she passed. Like, this is horrible. This is really bad.

But I think this is your point too. Do we go and censor? And to me, the answer is no. I do not believe in censorship. I believe that open access to data and intelligence guided by wisdom is important. And with that being said, there are going to be models that you’re going to purposely want to leave things out of. We’ve already talked about copyrighted material. We talked about copyleft material. And I’ll give an example, as a coding model. I personally would feel more comfortable, if I’m developing commercial code on that, not to include copyleft, because I don’t want to have that argument or that legal result blow up in my face later.

gem
Right. You don’t want it to be legal… but legally binding it.

GREGORY
Yeah, so I would be proactive on that one. So it’s, but… the fact that we can have lots of people building models that are optimized for particular tasks means you can choose what out of the corpus you’re putting in. Not everything has to be a world model.

gem
Totally with you. So I want to shoot a little bit at a different angle. A core thread running through your entire career is reproducible computing, the principle that if an execution state or a scientific run can’t be deterministically recreated, it can’t be trusted. So today, model builders hide behind the idea that non-deterministic output is some sort of proof of, quote, creativity, or, quote, intelligence, but stochastic behavior isn’t creative. It’s just pseudo-random statistical sampling across a high-dimensional vector space. So from a machine learning security standpoint, when an agent or inference pipeline produces different execution paths every time you run it, how do security engineers conduct forensic analysis or verify invariant enforcement or prove that a system didn’t quietly break its security boundaries?

GREGORY
So proper instrumentation on the agents, I think, is fantastic. I think it’s needed. But to go back to your earlier point, it seems we’re building infrastructure around an architecture that is… I don’t want to call it flawed, because it’s by design to do this, but… it is a, and I will kind of do what you said not to do, which is, it is a non-deterministic system. I get the aspect of the LLM not understanding, not knowing where the LLM is going to go at a high level. Specifics about where the LLM is going to go is 100% though dictated by its training. And it’s going to…

gem
So, and there are ways to build infrastructure. Like this is what I was talking about with Christoph, that you can think of as kind of like the skeleton. And the organs and the flesh and all the stuff around the skeleton, we can let that be stochastic and slightly non-deterministic. But the skeleton itself, maybe that should have some, you know, proof-carrying formality with it. And that might be an answer. I know at Google they’re doing some of that stuff, at least in science land, of course. Google’s so big, they’re probably doing the opposite as well. So, you know, maybe that’s an answer.

GREGORY
So… I hate it when people talk about comparing LLMs to, like, humans and the human brain. I actually really have a, like, a knee jerk, but I’m going to do it.

gem
You know, you have to know I have a PhD in cognitive science too, so it drives me absolutely up the wall. So here we go. Let’s see if we hit the kill switch.

GREGORY
So. But how I’m going to use it is not comparing it and using it as justification for one to the other. What I’m going to say, using the metaphor of the skeletal system and whatnot, is I would actually say that the LLM is probably most logically associated with the brain, which means the agent is most logically associated with the body, which means, because the body is the, the har… excuse me, the agent would be the harness plus the LLM, right? So the body’s the encompassing unit of everything.

And then there’s laws. And then there’s society. And then there’s how are you supposed to act, and how are you as a human supposed to be within society? And if I work for a company and I do something illegal on behalf of that company, who is responsible? The company is going to be the first line of defense.

gem
If your dog bites somebody, whose dog is it again? I mean, you know.

GREGORY
So, responsibility is so incredibly important here. And I think that is the undercurrent of so much of what we’re talking about, is we’re seeing these issues happening. We’re seeing exploits happening with very little accountability at this point because, to the earlier point, the AI did it. And so that is not a legal justification for why somebody or a company should not be held accountable. We have to be holding organizations accountable, and this goes now directly into your point of… companies need the appropriate tooling. They need the appropriate sandboxing, instrumentation, to see what’s happening. And they need to be held accountable in such a way that they build this infrastructure. And this infrastructure now becomes extremely valuable for anybody who is using AI as part of their core infrastructure.

gem
Alright, another thread. You helped form Open ELA because leaving core enterprise Linux infrastructure under sole control of one vendor created unacceptable systemic risk. So today in AI, Western enterprises are caught between two structural extremes. One is relying on closed proprietary cloud monopolies, or deploying open weights models, and I’m putting a quote around there, built predominantly by Chinese tech giants like Alibaba and DeepSeek. So from a machine learning security standpoint, if the entire global economy builds its software stack on top of a few closed API gateways on one side, and uninspectable open weight binaries from strategic adversaries on the other, what happens to resilience, like, and how does open, fully auditable infrastructure, like say the kind that you’re working on, rescue us from that dual trap?

GREGORY
So there’s a common thread between both of those scenarios that people need to choose through. And it is the lack of visibility, lack of transparency, lack of understanding. And we’re kind of stuck between two situations, and actually one of them, in my opinion, is more dangerous than the other. When somebody is, let’s make the comparison back to open source again, open source infrastructure. If a company wishes to go download a kernel from kernel.org, glibc from GNU, and gcc from GNU, and start compiling their own operating system from scratch, or their own tooling, or their own applications, and running that themselves, they are taking on the onus of responsibility and accountability themselves. If something breaks, it is on them. They went and downloaded an open, freely available thing and they went and built it themselves.

The difference between open source and open weight models is they can actually look at the source. Most people don’t, to be clear, but they could. And you could audit it to see, is there any malicious code in here? Or are other people auditing it and you’re trusting the community, the crowdsource audited source code, which is what most people end up doing. But you take on that onus, but you have a level of trust in what it is you’re running. When you’re leveraging a SaaS interface for an AI model from a company, well, yes, there are free accounts, but you’re agreeing to a EULA when you get that free account. But more importantly, companies don’t use free accounts. They are going to build a commercial relationship, which now means you have a commercial throat to grab. And you have some account of now responsibility and accountability through that commercial relationship. When we’re going back to open weight models, when a company goes and downloads an open weight model that has no company behind it, no accountability behind it, and it is a black box, it is literally the worst of all situations.

gem
It reminds me of just getting software globs in the 80s. I remember just going and getting globs, and then we were like, damn, that’s a bad idea. I wonder what’s in that glob.

GREGORY
And there’s one aspect that even makes it worse, at least for American companies. There’s not even a throat to hold in this country. Like, all of our laws don’t apply. So it literally is the worst and most dangerous situation that you can come up with. I’m going to go download a free open weight model. I’m going to go run it on my core infrastructure. I’ve got no idea what’s inside of it. It’s a black box. I don’t know what it’s going to do. I can’t possibly ever blame anyone else because they’re not even in this country. So I can’t even sue them if they really screw it up. And I’m going to give it access not only to my corporate network but my commercial data, and I’m going to base my future on it.

gem
Anything wrong here? Oh, nothing to see here. Move on. Move on.

GREGORY
Yeah. And it’s actually also funny, because they take these binary blobs of data, which are the weights, and then they went and open sourced them under an open source license. Right. So you can go download these open weights under an MIT or BSD or Apache 2 license. And it’s like, wait a second, that’s an open source license. So everybody says, oh, it’s open source. But it’s not.

gem
I know that. I’m so pleased to do this episode because I felt that way the whole time. And it was like, thank God somebody is doing the right thing. So thank you for your work on that front.

Whenever corporate interests attempt to lock down systems or enclose the comments, the open source community steps up to rebuild open alternatives.

But as automated code generation accelerates and agentic tools begin opening pull requests and maintaining repositories all by themselves, we risk confusing automated synthetic churn with actual software engineering. So automated tools can pattern match across existing code, but they lack human insight and inspiration and intuition and all of those things that make a real maintainer incredibly valuable, what we call the “three I’s” at BIML. As machine learning systems take over more of the build and maintain cycle, how do we preserve human stewardship at the heart of foundational software, ensuring our infrastructure reflects genuine architectural wisdom rather than just automated noise?

GREGORY
So I just released a research paper, and this research paper describes a process that I used to build an operating system kernel from scratch. In Rust. From first line of code to a usable operating system that is Linux compatible. So I can go and spin up Rocky Linux, Alpine, Debian, whatever user space you want, on top of a non-Linux kernel. And I wrote, again, the whole thing from scratch. And to say I wrote it is a misnomer.

I came up with a method. And I’m sure I’m not the only person to do this, but I think I might be the first person to actually write it down and publish it, where I call it Human-Directed Agentic Engineering: HDAE. And it describes a process in which the human is in charge of various roles and the AI is responsible for various roles. And with this process, I was able to create, in 11 days, an entire operating system kernel that is an alternative to the Linux kernel.

gem
I think the only reason that worked is because… you are you.

GREGORY
Yes. So this goes directly to…

gem
I mean, you could probably do it with a Swiss Army knife also.

GREGORY
This is the human directed part. It has to be. So I have this hate relationship with the term “vibe coding.”

gem
Oh, me too.

GREGORY
Because, to me, vibe coding is somebody who knows nothing about the discipline, nothing about the code, and goes and asks an AI to go solve a problem, build an app, or do something. And the AI goes and does it, and there’s no oversight. None. And that to me is vibe coding. That to me might end up with a proof of concept. It might end up with an MVP. But you are not going to get the same quality of output than if you had an expert driving it. And so, humans that are experts driving agentic engineering, you can get 10 to 100x efficiency, capability, code output, and even quality, if you do it right. And I outlined how I did it in this research paper that I’ve published.

gem
Super. We will provide a link to that when we put the episode out.

GREGORY
Very cool. But I was also, just as an example, the process introduces the notion of gates, where you define a particular milestone in your plan and architecture, and you have the AI sit there and iterate on this gate until that gate is solved.

gem
Passes. Yeah, of course.

GREGORY
And it would literally run all night. I’d wake up in the morning and be like, oh, cool, it’s actually solved these three gates and it’s onto the fourth one. And then I come in and I’ll be like, okay, let’s do a checkpoint here. I need to validate where we are. I need to see where it’s at. And sometimes I’ll say after each gate, I want to actually look, if it’s a big gate. And then I’ll go and validate, make sure it works.

But this resulted in, in total, it was about 180 thousand lines of code. And I did this whole thing in 11 days. There is no way I’m reviewing 100-plus thousand lines of code. So at some point, how we think about engineering, how we think about development has to shift. And we already talked about AI turning on the light and all of the bugs running away. But we can see those bugs now and we’re bringing them to light. And that’s going to happen faster and faster. Humans alone will not be able to keep up, and I’ll just put that out there. We will not be able to keep up. Even the Linux kernel at this point is leveraging AI to solve a lot of the bugs that AI is finding.

We have to be smart about how we’re going to move into the future now. And Human-Directed Agentic Engineering, which is the concept that I wrote up in this article, I think is a really good possible way of doing it. And I’m sure there’s many more. Bye. This is a solved problem. I don’t want to say solved, sorry. There are multiple solutions to solving this problem. And we just need to have not only expert engineers in the room, but expert engineers who are willing to use AI.

gem
I totally agree with you. And the human part is not going to disappear. On the contrary, it is essential to the success of the project.

GREGORY
And the human role is going to change. It’s going to evolve. And right now we are right in the midst of evolution of technology and disruptive technology and disruptive innovation in general. It is always going to cause evolution, and evolution, it’s a very painful process. It’s not a kind process, right? It causes individuals to, like, die before having, you know, offspring.

gem
You need a t-shirt that says, “Don’t be culled.”

GREGORY
So we are in the process of evolution right now. We all have to change how we are thinking about our jobs, our work, how we operate, how we think. I wrote a simple post just recently that said, “Knowledge is no longer the moat, wisdom is the moat.” Like, you need to be smart. We are inundated with information and knowledge at this point. Everything is out there. If you look hard enough, you will find an explanation for every conspiracy theory and every silly idea you’ve ever had, and you will get justified in your own mind. Now it’s about having the wisdom to know, given all the information is available, what are you going to listen to? And…

gem
Oh boy, are we in trouble as a bunch of monkeys.

So, let’s wrap this up. We’re hurtling towards some sort of stark fork in the road, I think. On one path we got, we surrender our digital future to some closed proprietary tech monopolies and black box models, where security is kind of an illusion and users are certainly captured tenants. On the other path, we build sovereign, open, and rigorously auditable foundations, though doing so means fighting entrenched commercial interests in an ocean of unvetted synthetic noise every single day.

So as we look at the next five years–which is a long damn time–this isn’t a cooperative ecosystem dance, I don’t think. It’s an architectural war for control of compute. What is your battle cry for engineers and open source builders who refused to bend the knee to the black box?

GREGORY
The first thing that I’m going to say is, between open source and the commercial industry, the commercial industry always seems to kind of drive a lot of the innovation. Not always, the wrong word. Often seems to drive a lot of the innovation, so we see things come out of commercial industry first and then open source kind of fills the gap.

I mentioned also earlier that there’s two populations. My call to action is I am really trying to pull those two populations together. Right now there’s too much separation between them. The AI world right now needs open source, it needs experts who have been doing this for ages to come in and be–pardon the expression–the adults in the room on how to do…

gem
Well, wield the wisdom, let’s just say.

GREGORY
That’s a yes. Yes. And the open source side needs to get a heck of a lot more open and curious about AI. We’ve got open source project leaders, and there was actually just a ban in… not that I’m a Reddit follower or user, but on the Reddit open source subreddit, that if you use AI, you are not allowed to be part of this community, or opposed to this community, or something along those lines. And it’s like…

gem
It’s going to be like one guy and his dog pretty soon.

GREGORY
I mean… It will be, I think, but for now it’s not. And the amount of people in open source that are making an active stance against progress is going to end up biting them and all of their users in the foot. Or shooting them in the foot.

gem
So we’ve got to bring them together.

GREGORY
We’ve got to bring these two communities together. And right now the overlap, OpenWaldo sits perfectly in that overlap. It is a fantastic place for people on the open source side who want to get more involved with AI, understand how AI infrastructure works, want to know how to train. We have a composed ladder where every rung on this ladder is a different training level, both for training a model as well as training a person.

gem
Yeah, so… Get it and do it. That’s fantastic.

GREGORY
Anyway, bringing those two communities together is my personal goal and call to action.

gem
Love it. Thanks so much for participating.

GREGORY
It’s been a pleasure. Thank you.

gem
This has been the Silver Bullet Security Podcast with BIML. Silver Bullet is sponsored by the Berryville Institute of Machine Learning, a nonprofit science and technology organization whose research focuses on machine learning security. You can find a permanent archive of all our episodes dating back to 2006. GaryMcGraw.com/technology/silver-bullet-podcast.

Show links, notes, and an online discussion can be found on the Silver Bullet webpage at berryvilleiml.com/podcast. This is Gary McGraw.