McAfee Advanced Threat Research (ATR) has a specific goal: identify and illuminate a broad spectrum of threats in today’s complex landscape. With model hacking, the study of how adversaries could target and evade artificial intelligence, McAffe has the incredible opportunity to influence the awareness, understanding and development of more secure technologies before they are implemented in a way that has real value to the adversary.
M
cAffe focused efforts on the broadly deployed MobilEye camera system, today utilized across over 40 million vehicles, including Tesla models that implement Hardware Pack 1.
McAffee started by replicating industry papers on methods of attacking machine learning image classifier systems used in autonomous vehicles, with a focus on causing misclassifications of traffic signs. They were able to reproduce and significantly expand upon previous research focused on stop signs, including both targeted attacks, which aim for a specific misclassification, as well as untargeted attacks, which don’t prescribe what an image is misclassified as, just that it is misclassified. Ultimately, they were successful in creating extremely efficient digital attacks which could cause misclassifications of a highly robust classifier, built to determine with high precision and accuracy what it is looking at, approaching 100% confidence.
They created create physical stickers, that model the same type of perturbations, or digital changes to the original photo, which trigger weaknesses in the classifier and cause it to misclassify the target image.
This set of stickers has been specifically created with the right combination of color, size and location on the target sign to cause a robust webcam-based image classifier to think it is looking at an “Added Lane” sign instead of a stop sign.
They developed attacks that were resistant to change in angle, lighting and even reflectivity, knowing this would emulate real-world conditions. While these weren’t perfect, the results were relatively consistent in getting the MobilEye camera to think it was looking at a different speed limit sign than it was. The next step in testing was to reduce the number of stickers to determine at which point they failed to cause a misclassification. At the beginning the HUD continued to misclassify the speed limit sign. They continued reducing stickers from 4 adversarial stickers in the only locations possible to confuse the webcam, all the way down to a single piece of black electrical tape, approximately 2 inches long, and extending the middle of the 3 on the traffic sign.
The tiny piece of sticker was all it took to make the MobilEye camera’s top prediction for the sign to be 85 mph.
“In order to drive success in this key industry and shift the perception that machine learning systems are secure, we need to accelerate discussions and awareness of the problems and steer the direction and development of next-generation technologies. Puns intended.” concluded McAfee’s Steve Povoly.