Source: Journal of Advances in Modeling Earth Systems (JAMES)
Neural networks are everywhere in modern science, providing insights into complex topics such as facial recognition, cancer research, and risk management, among others. The machine learning technique uses networks of computational nodes working together to find patterns in massive data sets and make predictions on the basis of those data.
Neural networks may be useful in identifying meaningful relationships in the increasingly large and high-quality data sets available in the geosciences. But their application in those fields has been limited so far by the fact that the internal reasoning the networks use to make decisions and predictions is not always apparent.
In a new study, Toms et al. apply two new methods for interpreting neural networks: backward optimization and layerwise relevance propagation (LRP). Both methods help researchers identify which inputs are most influential in a neural network’s decision-making process, which could make the technique more applicable to the geosciences, where understanding a network’s reasoning may be critical for validating its predictions.
The team first applied both methods to a simple task: identifying whether a specific sea surface temperature (SST) pattern was indicative of a positive or negative phase of the well-studied El Niño–Southern Oscillation (ENSO). They trained the neural networks on SST data from 1880 to 1990 and tested it using data from 1990 to 2017. The network identified the ENSO phase accurately 100% of the time.
Next, the team applied the methods to a more complex task: predicting how sea surface temperature anomalies will impact seasonal temperatures. They trained the network on data from 1950 to 2000, tested it on data from 2000 to 2018, and found that the neural network approaches were more accurate than a traditional, regression-based approach.
Machine learning studies are becoming more common in the geosciences, but this one was the first to apply an LRP technique to the field. The authors show that the machine learning techniques can provide valid predictions, confirming that their output matches our understanding of the physical processes driving Earth systems and setting the stage for future studies that could use these techniques to identify as yet unknown relationships hiding in geoscience data. (Journal of Advances in Modeling Earth Systems (JAMES), https://doi.org/10.1029/2019MS002002, 2020)
—Kate Wheeling (@katewheeling), Science Writer
Citation:
Wheeling, K. (2020), Interpreting neural networks’ reasoning, Eos, 101, https://doi.org/10.1029/2020EO148517. Published on 02 September 2020.
Text © 2020. AGU. CC BY-NC-ND 3.0
Except where otherwise noted, images are subject to copyright. Any reuse without express permission from the copyright owner is prohibited.