Attacks, Risks, Security Engineering and ML
I gave a talk this week at a meeting hosted by Microsoft and Mitre called the 6th Security Data Science Colloquium. It was an interesting bunch (about 150 people) including the usual suspects: Microsoft, Google, Facebook, a bunch of startups and universities, and of course BIML.
I decided to rant about nomenclature, with a focus on RISKS versus ATTACKS as a central tenet of how to approach ML security. Heck, even the term “Adversarial AI” gets it wrong in all the ways. For the record, we call the field we are in “Machine Learning Security.”
Here is one of the slides in my deck. You can get the whole deck here.
In our view at BIML, every attack has a one or more risks behind it, but every risk in the BIML-78 does not have an associated attack. For us, it is obvious that we should work on controlling risks NOT stopping attacks one at a time.