The risk analysis of the generic ML system above uses a set of nine “components” to help categorize and explain risks found in various logical pieces.  Components can be either processes or collections. Just as understanding a system is easier when a system is divided up into pieces, controlling security risk is easier when the pieces themselves are each secured separately.  Another way of thinking about this is to compare old fashioned “monolithic” software design to “micro-services” design.  In general, both understanding and securing a monolith is much harder than securing a set of services (of course things get tricky when services interact in time, but we’ll ignore that for now).  In the end we want to eradicate the monolith and use compartmentalization as our friend.

      Lets imagine one security principle and see how compartmentalization can help us think it through.  Part of the challenge of applying the principle of least privilege in practice (described above) has to do with component size and scope.  When building blocks are logically separated and structured, applying the principle of least privilege to each component is much more straightforward than it would be otherwise.  Smaller components should by and large require less privilege than complete systems.  Does this component involve pre-processed training data that will directly impact system learning?  Hmm, better secure those data!

     The basic idea behind compartmentalization is to minimize the amount of damage that can be done to a system by breaking up the system into a number of units and isolating processes or data that carry security privilege. This same principle explains why submarines are built with many different chambers, each separately sealed. If a breach in the hull causes one chamber to fill with water, the other chambers are not affected. The rest of the ship can keep its integrity, and people can survive by making their way to parts of the submarine that are not flooded.

      The challenge with security and compartmentalization comes when it is time to consider the system as a whole.  As we’ve seen in our generic ML system here, data flows between components, and sometimes those data are security sensitive.  When implementing an ML system, considering component risks is a good start, but don’t forget to think through the risks of the system as a whole.  Harkening back to the principle of least privilege, don’t forget to apply the same sort of thinking to the system as a whole after you have completed working on the components.

Read the rest of the principles here.