West Wake News

Researchers at North Carolina State University have developed a new technique to address the issue of spurious correlations in artificial intelligence (AI) models. These correlations, often learned during training, can lead AI systems to make decisions based on irrelevant or misleading data. The novel approach involves identifying and removing a small subset of training data that contributes to these spurious correlations.

Jung-Eun Kim, an assistant professor of computer science at NC State and corresponding author of the study, explained the significance of this development: “This technique is novel in that it can be used even when you have no idea what spurious correlations the AI is relying on.” Kim further noted that the method is useful both for those who already know which features are problematic and for those facing unexplained performance issues.

Spurious correlations typically arise from simplicity bias during AI training. For instance, an AI model trained to recognize dogs might incorrectly use collars as a key identifier if many images in its dataset feature collared dogs. This could result in misidentifying cats with collars as dogs. Conventional methods require practitioners to identify these misleading features manually and adjust their datasets accordingly.

However, Kim highlighted a limitation of existing techniques: “Our goal with this work was to develop a technique that allows us to sever spurious correlations even when we know nothing about those spurious features.”

The new method involves pruning difficult-to-understand samples from the training data. According to Kim, these samples are likely sources of noise and ambiguity that lead models astray: “By eliminating a small sliver of the training data that is difficult to understand, you are also eliminating the hard data samples that contain spurious features.”

The researchers demonstrated improved performance using this technique compared to previous approaches where spurious features were identifiable. Their findings will be presented at the International Conference on Learning Representations (ICLR) in Singapore from April 24-28.

The study's first author is Varun Mulchandani, a Ph.D. student at NC State. The paper titled "Severing Spurious Correlations with Data Pruning" highlights how deep neural networks often rely on spurious correlations present in their training data and proposes this innovative solution without needing domain knowledge or human intervention.