A set of experimental images with art stickers at different distances and from different angles: (a) 5 feet, 0 degrees; (b) 5 ’15 °; (c) 10 ‘0 °; (d) 10 ’30 °; (e) 40 ‘0 °. Deception works at any distance and from any angle: instead of the stop sign, the machine learning system sees the sign “Speed limit of 45 miles”

. While some scientists improve machine learning systems, other scientists improve methods of deception of these systems. As you know, small, purposeful changes in the picture can “break” the machine learning system, so that it recognizes a completely different image. 

Such “Trojan” pictures are called “adversarial examples” (adversarial examples) and represent one of the known limitations of in-depth training .

In order to compile an adversarial example, it is necessary to maximize the activation of, for example, a certain filter of a convolutional neural network. Ivan Evtimov from the University of Washington, along with colleagues from the University of California at Berkeley, the University of Michigan and the University of New York in Stony Brook, have developed a new attack algorithm – robust physical perturbations (Robust Physical Perturbations or RP 2 ). He very effectively beats the vision of unmanned vehicles, robots, multicopters and any other robotic systems that try to navigate in the surrounding space.

Unlike previous studies, the authors concentrated on changing the objects themselves, rather than the background. The task of the researchers was to find the minimum possible delta that would knock down the classifier of the machine learning system, which was trained on a data set with images of LISA road signs . The authors independently made a series of photos of traffic signs on the street in different conditions (distance, angles, lighting) and supplemented the LISA data set for training.

After calculating such a delta, a mask was identified – such a place (or several places) in the image, which most reliably causes perturbation in the machine learning system (machine vision). A number of experiments were conducted to verify the results. In general, the experiments were carried out on a stop signal (the “STOP” sign), which the researchers made by several harmless manipulations turned for machine vision into the sign “SPEED LIMIT 45”. The developed technique can be used on any other signs. The authors then tested it on the sign of the turn.

The scientific team has developed two variants of the attack on the machine vision systems, which recognize road signs. The first attack is small, inconspicuous changes throughout the area of the sign. With the help of the optimizer Adam they managed to minimize the mask to create separate targeted adversarial examples aimed at specific road signs. In this case, you can deceive the machine learning system with minimal changes in the picture, and people generally will not notice anything. The effectiveness of this type of attack was checked on the printed posters with minor changes (first the researchers were convinced that the machine vision system successfully recognizes the posters unchanged).

The second type of attack is camouflage. Here the system imitates either acts of vandalism, or art graffiti, so that the system does not interfere with the lives of surrounding people. Thus, a man-driver at the wheel will immediately see a sign of turning left or a stop signal, and the robot will see a completely different sign. The effectiveness of this type of attack was tested on real road signs, which were pasted with stickers. Camouflage-graffiti consisted of stickers in the form of words LOVE and HATE, and camouflage of the abstract art type – from four stickers of rectangular shape of black and white colors.

The results of the experiment are shown in the table. In all cases, the efficiency of cheating the classifier of machine learning is shown, which recognizes the modified “STOP” sign as the sign “SPEED LIMIT 45”. The distance is indicated in feet, the angle of rotation is in degrees. The second column shows the second class, which is seen by the machine learning system in a modified sign. For example, from a distance of 5 feet (152.4 cm), camouflage of the abstract art type at an angle of 0 ° produces such recognition results of the “STOP” sign: with confidence 64% it is recognized as a “SPEED LIMIT 45” sign, and with confidence 11% the sign “Lane Ends”. Legend: SL45 = Speed Limit 45, STP = Stop, YLD = Yield, ADL = Added Lane, SA = Signal Ahead, LE = Lane Ends

Perhaps such a system (with appropriate changes) will be needed by mankind in the future, and now it can be used to test imperfect systems of machine learning and computer vision.

The scientific work was published on July 27, 2017 on the website of the preprints arXiv.org (arXiv: 1707.08945).