Experimental design of the calibration and testing experiments performed by Shen et al.
Experimental design of the calibration and testing experiments performed by Shen et al. The authors used U.S. catchments with 35 years of discharge data. The top left corner (a) shows the case where a model is assumed to be built in 1990. This provided 10 years to calibrate and validate the model. The green years were then used to calibrate the model, with the pink years to validate it. After the calibration and testing have passed a quality test, the model was subsequently used to predict for a period it has never seen (the orange years) and evaluated on prediction accuracy. As can be seen a total of 50 different split sample schemes were evaluated, depending on the year the model was assumed to be built. The experiments using all the available data (the lowest row in each panel) almost always yielded the best results, for both models, for all model building years and for all catchments. Credit: Shen et al. [2022], Figure 1
Editors’ Highlights are summaries of recent papers by AGU’s journal editors.
Source: Water Resources Research

An important part of a hydrologist’s training is the art and science of building parsimonious and robust hydrological models. An established methodology in robustness testing is calibrating a model with one part of the data and then validating it on the remaining part. This so-called “split sample approach” is a widely used and accepted procedure.

Shen et al. [2022], however, suggest that this practice is likely flawed and should be discarded. Their impressive empirical study, using two hydrological models in 463 United States catchments and 50 different data splitting schemes, convincingly shows that all data should be used for model development and calibration before the model is implemented for decision making. Although this conclusion needs further testing with other model types, it is bound to shake hydrologic practice.

Citation: Shen, H., Tolson, B. A., & Mai, J. (2022). Time to update the split-sample approach in hydrological model calibration. Water Resources Research, 58, e2021WR031523. https://doi.org/10.1029/2021WR031523

—Marc F. P. Bierkens, Editor, Water Resources Research

Text © 2022. The authors. CC BY-NC-ND 3.0
Except where otherwise noted, images are subject to copyright. Any reuse without express permission from the copyright owner is prohibited.