Scientists discover quantum-inspired vulnerabilities in neural networks

(A) Illustrates the final result of network training, highlighting the class prediction domains. Shaded regions demarcate these areas, with individual dot colors indicating the true labels of the corresponding test samples, demonstrating a general alignment between the network’s predictions and the actual classifications. In (B), all test samples were subjected to gradient-based attacks, causing the perturbed sample points to noticeably deviate from their correct categorical regions, leading to misclassifications by the network model. (C) Focuses on the evolution of the prediction region for the number “8” in epochs 1, 21, and 41. The deeper the nuance of the region, the higher the network’s confidence in its prediction. (D) Similar to (C), but showing conflicting predictions for attacked frames, we observe that as training progresses, the effective attack point distribution radius increases. This suggests that as the network’s accuracy in identifying input features increases, its vulnerability to attacks also increases. Credit: Science China Press

In a recent study merging the fields of quantum physics and computer science, Dr Jun-Jie Zhang and Professor Deyu Meng explored the vulnerabilities of neural networks through the lens of the uncertainty principle in physics.

Their work, published in the National scientific journaldraws a parallel between the susceptibility of neural networks to targeted attacks and the limits imposed by the uncertainty principle, a well-established theory in quantum physics that highlights the challenges of simultaneously measuring certain pairs of properties.

The researchers’ quantum analysis of neural network vulnerabilities suggests that adversarial attacks exploit the tradeoff between the precision of input features and their calculated gradients.

“When considering the architecture of deep neural networks, which involve a loss function for learning, we can always define a conjugate variable for the inputs by determining the gradient of the loss function with respect to those inputs” , explains Dr. Zhang, whose expertise lies in mathematical physics.

This research hopes to prompt a reassessment of the supposed robustness of neural networks and encourage a deeper understanding of their limitations. By subjecting a neural network model to adversarial attacks, Dr. Zhang and Professor Meng observed a trade-off between the model’s accuracy and its resilience.

Subfigures (A), (C), (E), (G), (I) and (K) display the test accuracy and robust accuracy, the latter being evaluated on images disrupted by the Projected attack Gradient Descent (CEO). method. Subfigures (B), (D), (F), (H), (J), and (L) reveal the trade-off relationship between accuracy and robustness. Credit: Science China Press

Their results indicate that neural networks, mathematically similar to quantum systems, struggle to accurately resolve the two conjugate variables – the gradient of the loss function and the input feature – simultaneously, suggesting an inherent vulnerability. This knowledge is crucial for the development of new protection measures against sophisticated threats.

“The importance of this research is considerable,” notes Professor Meng, a machine learning expert and corresponding author of the paper.

“As neural networks play an increasingly critical role in mission-critical systems, it becomes imperative to understand and strengthen their security. This interdisciplinary research offers a new perspective for demystifying these complex “black box” systems, potentially informing the design of safer and more interpretable AI. models.”

More information:
Jun-Jie Zhang et al, Quantum-inspired analysis of neural network vulnerabilities: the role of conjugate variables in system attacks, National scientific journal (2024). DOI: 10.1093/nsr/nwae141