BIML founder Gary McGraw delivered the last talk of the semester for the Center for Applied Cybersecurity Research (CACR) speakers series at Indiana University. You can watch the talk on YouTube.
If your organization is interested in having a presentation by BIML, please contact us today.
Some nice coverage in the security press for our work at BIML. Thanks to Rob Lemos!
As our MLsec work makes abundantly clear, data play a huge role in security of an ML system. Our estimation is that somewhere around 60% of all security risk in ML can be directly associated with data. And data are biased in ways that lead to serious social justice problems including racism, sexism, classism, and xenophobia. We’ve read a few ML bias papers (see the BIML Anotated Bibliography for our commentary). Turns out that social justice in ML is a thorny and difficult subject.
We were joined this week by Martiza Johnson, a Computer Scientist and the inaugural director of a new center for data science, AI, and society at the University of San Diego. Maritza assigned us some homework (reading Chapter One and Chapter Four of Data Feminism, this blog entry, and watching Coded Bias), and then led us in a very interesting and far ranging conversation on bias in ML.
We recorded our conversation with Maritza which you can listen to. A video of our conversation is below.
We were very fortunate to have Melanie Mitchell, author of Artificial Intelligence: A Guide for Thinking Humans (and famous programmer of Copycat), join us for our regular BIML meeting.
We discussed Melanie’s new paper Abstraction and Analogy-Making in Artificial Intelligence. We talked about analogy, perception, symbols, emergent computation, machine learning, and DNNs.
A recorded version of our conversation is available, as is a video version.
We hope you enjoy what you see here. This is what BIML meetings are like.
An important part of BIML’s mission as an institute is to spread the word about our understanding of machine learning security risk throughout the world. We recently decided to take on three college and high school interns to provide a bridge to academia and to inculcate young minds early in the intricacies of machine learning security. We introduce them here in a series of blog entries.
We are very pleased to introduce Aishwarya Seth who is a BIML University Scholar.
Aishwarya is a graduate student at North Carolina State University in Raleigh, North Carolina. An ardent fan of crime thrillers since early childhood, she has always been passionate about security. When Aishwarya was introduced to Java programming in high school, her interest in security took a turn towards computer security.
The rise of Machine Learning coincides directly with Aishwarya’s study of security and cryptography, the confluence of which fascinate her. After earning her undergraduate degree in Computer Science, Aishwarya worked as a team member of the Clari5 AI/ML team where she focused on reducing the number of false positives detected for potentially fraudulent transactions online.
Apart from pondering different ways to secure the world, Aishwarya likes to read novels, scribble, travel, and explore.
As BIML University Scholar, Aishwarya will:
BERRYVILLE — When thinking about Clarke County, farms and rolling hills generally come to mind, not sophisticated gadgets or high-tech wizardry.
In fact, many parts of the county still lack high-speed internet service.
But hidden away in the countryside is a small group of researchers trying to find ways to make technology safer so hackers cannot breach vital — or even secret — information.
The Berryville Institute of Machine Learning (BIML) was established in 2019 to address security issues associated with machine learning (ML) and artificial intelligence (AI). Recently, the institute received a $150,000 grant from the Open Philanthropy foundation to help further its work.
BIML, a think tank, was founded by software security expert Gary McGraw plus Richie Bonett, a computer scientist from Berryville; Harold Figueroa, director of Machine Intelligence Research and Applications Lab at Ntrepid, a Herndon-based cybersecurity firm, and Victor Shepardson, an artist and research engineer at Ntrepid.
Artificial intelligence is brainpower demonstrated by emotionless machines, in contrast to that of humans and animals which involves consciousness and, in certain instances, sensitivity.
Machine learning, on the other hand, involves developing computer programs that help machines access data and use it for their own benefit. The intent is to help computer systems develop the ability to automatically learn and improve their functions from experience without being specially programmed along that line.
“Usually, computers are programmed with a bunch of rules telling them what to do,” McGraw said. “Machine learning involves enabling machines to recognize certain inputs and outputs so they can do certain tasks themselves.”
An example of such a machine, he mentioned, is Alexa, a device developed by Amazon that uses speech recognition abilities in performing tasks.
“When you’re talking to Alexa, you’re interacting with a machine learning system,” McGraw noted.
Automatic banking machines are another example of the technology, he pointed out. So are some types of video games.
Technology is ever-evolving. And, “when technologies catch on fast, people forget to secure them properly,” McGraw said.
That can lead to trouble.
“A bad person may intentionally trick a system into doing the wrong thing” for personal gain or harm, said McGraw. “What we’re trying to do at BIML is to make it harder for bad people to misuse systems.”
Each computer system is unique, “so they learn in unique ways,” he said. As a result, unique solutions must be created to prevent potential problems with them.
BIML’s research and recommendations are placed into the “creative common” so people have free access to them, McGraw said.
According to its website, BIML has become well-known within ML circles for its pioneering research document, “Architectural Risk Analysis of Machine Learning Systems: Toward More Secure Machine Learning.”
McGraw said the Open Philanthropy grant will be used for various purposes, including research, recruiting interns and making presentations on cybersecurity issues at colleges and universities nationwide.
The institute already has recruited its first High School Scholar: Nikil Shyamsunder, a sophomore at Handley High School in Winchester. He will be involved in curating the “BIML Annotated Biography,” a resource for ML security workers providing an overview of research in that field, including a “Top 5 Papers” section.
As part of his internship, Shyamsunder will receive a $500 college scholarship.
BIML is based in the Berryville area largely because McGraw lives there — much of its work is based at his home — and Bonett is from there.
“It doesn’t really matter where this type of work is done,” McGraw said. “You don’t have to be physically present somewhere with people to get the work done. The majority of the work is done over the internet,” consulting with researchers and AI and ML practitioners.
As technology evolves, “it’s hard to anticipate” what BIML will be doing in the future, he said. But the machine learning field is growing, so demand for services that the institute provides is increasing, he asserted. Therefore, he expects the institute to be around for many years to come.
More information about the institute is online at berryvilleiml.com.
— Contact Mickey Powell at mpowell@winchesterstar.com
We are very pleased to introduce Trinity Stroud who is a BIML University Scholar.
Trinity is a senior at the University of South Alabama in Mobile, Alabama. She has been programming since middle school, where she cut her teeth on the ROBOTC language. Later, in high school, she learned to code in Python and Java. In college she became passionately interested in the area of computer security.
Trinity participates in her university’s DayZero Cyber Competition Team and represents the USA School of Computing as a Student Government Association Senator. She enjoys participating in security competitions such as National Cyber League and Cyber FastTrack, the latter during which she was named a national finalist and awarded a full scholarship for the Undergraduate Certificate Program in Applied Cybersecurity with the SANS Technology Institute.
In her free time, Trinity reads science fiction novels written by such authors as Robert A. Heinlein, Orson Scott Card, and Anne McCaffrey.
As BIML University Scholar, Trinity will:
We are very pleased to introduce Nikil Shyamsunder who is the first BIML High School Scholar.
Nikil is a sophomore at John Handley High School in Winchester, VA. He has been programming for most of his (short) life and has become keenly interested in Machine Learning.
Nikil organizes and teaches coding camps and also offers private coding classes to his peers. He enjoys participating in a philosophy-based style of debate called Lincoln-Douglass. He is currently competing in the International Public Policy Forum on the pros and cons of AI. He and his team recently reached the Top 16 and continue to compete. Nikil is fascinated by linguistics and has advanced through the Spelling Bee to Nationals twice.
In his free time, Nikil plays the violin, an Indian tonal percussion instrument called Mridangam, and enjoys producing music.
As BIML High School Scholar, Nikil will help to curate the BIML annotated bibliography. This bibliography has become an important resource for researchers working in the field of Machine Learning security as it provides an opinionated overview of work in the field, including a top 5 papers section.For his efforts on behalf of BIML, Nikil will receive a scholarship of $500 to put towards expenses at the college of his choice.
Berryville Institute of Machine Learning (BIML) Gets $150,000 Open Philanthropy Grant. Funding will advance ethical AI research
Online PR News – 27-January-2021 – BERRYVILLE, VA – The Berryville Institute of Machine Learning (BIML), a research think tank dedicated to safe, secure and ethical development of AI technologies, announced today that it is the recipient of a $150,000 grant from Open Philanthropy.
BIML, which is already well known in ML circles for its pioneering document, “Architectural Risk Analysis of Machine Learning Systems: Toward More Secure Machine Learning,” will use the Open Philanthropy grant to further its scientific research on Machine Learning risk and get the word out more widely through talks, tutorials, and publications.“In what is by now an all too familiar pattern our embrace of advanced ML technology is outpacing an understanding of the security risks its use drags along with it. AI and ML automation continues to accelerate at an alarming pace. At BIML we’re dedicated to exposing and elucidating security risk in ML systems. We are pleased as punch that Open Philanthropy is pouring accelerant on our spark.”
“In a future where machine learning shapes the trajectory of humanity, we’ll need to see substantially more attention on thoroughly analyzing ML systems from a security and safety standpoint,” said Catherine Olsson, Senior Program Associate for Potential Risks from Advanced Artificial Intelligence at Open Philanthropy. “We are excited to see that BIML is taking a holistic, security-engineering inspired view, that considers both accidental risk and intentional misuse risk. We hope this funding will support the growth of a strong community of ML security practitioners at the intersection of real-world systems and basic research.”
Early work on ML security focuses on specific failures, including systems that learn to be sexist, racist and xenophobic, and systems that can be manipulated by attackers. The BIML ML Security Risk Framework details the top 10 security risks in ML systems today. It is designed for use by developers, engineers, designers and others who are creating applications and services that use ML technologies, and can be practically applied in the early design and development phases of any ML project.
“In what is by now an all too familiar pattern, our embrace of advanced ML technology is outpacing an understanding of the security risks its use drags along with it. AI and ML automation continues to accelerate at an alarming pace,” said Dr. Gary McGraw, co-founder of BIML and world renowned software security pioneer. “At BIML, we’re dedicated to exposing and elucidating security risk in ML systems. We are pleased as punch that Open Philanthropy is pouring accelerant on our spark.”
About BIML
The Berryville Institute of Machine Learning was created in 2019 to address security issues with ML and AI. The organization was founded by Gary McGraw, author, long-time security expert and CTO of Cigital (acquired by Synopsys); Harold Figueroa, director of Machine Intelligence Research and Applications (MIRA) Lab at Ntrepid; Victor Shepardson, an artist and research engineer at Ntrepid; and Richie Bonett, a systems engineer at Verisign. BIML is headquartered in Berryville, Virginia. For more information, visit http://berryvilleiml.com/.
About Open Philanthropy
Open Philanthropy identifies outstanding giving opportunities, makes grants, follows the results, and publishes its findings. Its mission is to give as effectively as it can and share the findings openly so that anyone can build on them.
