‘Leaving one out’ cross-validation methodology
ANNEX IISupplementary provisions
ANNEX II ‘Leaving one out’ cross-validation methodology The ‘leaving one out’ cross-validation strategy is a methodology used for the evaluation of modelling applications that incorporate the integrated use of modelling and measurements. It consists of the following steps: (a) select a sampling point to be left out in the data fusion or data assimilation step (i.e. Step (b)). Data fusion in the context of this decision refers to combining observational datasets and models to obtain the best estimate of a certain variable. Data assimilation in the context of this decision refers to coupling observations with dynamic numerical models to optimise model states and forecasts; (b) derive a fused or assimilated concentration field with the remaining sampling points; (c) extract a time series at the location of the sampling point that is left out in the fusion or assimilation process (i.e. the station selected in Step (a)); (d) calculate a modelling quality indicator value based on the independent measurement values and the fused or assimilated result of Step (c); (e) repeat the process from Step (a) with another sampling point until all points have been left out in one iteration of this methodology each; (f) evaluate the modelling quality objective with the set of independently derived modelling quality indicators at all sampling point locations.