Editing IPCC:AR6/WGI/Chapter-11 (section)

=== 11.4.3 Model Evaluation ===

<div id="h2-31-siblings" class="h2-siblings"></div>

Evaluating climate model competence in simulating heavy precipitation extremes is challenging due to a number of factors, including the lack of reliable observations and the spatial scale mismatch between simulated andobserved data ( [[#Avila--2015|Avila et al., 2015]] ; [[#Alexander--2019|Alexander et al., 2019]] ). Simulated precipitation represents areal means, but station-based observations are conducted at point locations and are often sparse. The areal-reduction factor, the ratio between pointwise station estimates of extreme precipitation and extremes of the areal mean, can be as large as 130% at CMIP6 resolutions (about 100 km) ( [[#Gervais--2014|Gervais et al., 2014]] ). Hence, the order in which gridded station based extreme values are constructed (i.e., if the extreme values are extracted at the station first and then gridded, or if the daily station values are gridded and then the extreme values are extracted) represents different spatial scales of extreme precipitation and needs to be taken into account in model evaluation (Wehner et al. 2020). This aspect has been considered in some studies. Reanalysis products are used in place of station observations for their spatial completeness as well as spatial-scale comparability( [[#Sillmann--2013a|Sillmann et al., 2013a]] ; [[#Kim--2020|Kim et al., 2020]] ; [[#Li--2021|Li et al., 2021]] ). However, reanalyses share similar parametrizations to the models themselves, reducing the objectivity of the comparison.

Different generations of CMIP models have improved over time, though quite modestly ( [[#Flato--2013|Flato et al., 2013]] ; [[#Watterson--2014|Watterson et al., 2014]] ). Improvements in the representation of the magnitude of the Expert Team on Climate Change Detection and Indices (ETCCDI) in CMIP5 over CMIP3( [[#Sillmann--2013a|Sillmann et al., 2013a]] ; [[#Chen--2015a|Chen and Sun, 2015a]] ) have been attributed to higher resolution, as higher-resolution models represent smaller areas at individual grid boxes. Additionally, the spatial distribution of extreme rainfall simulated by high-resolution models is generally more comparable to observations ( [[#Sillmann--2013b|Sillmann et al., 2013b]] ; [[#Kusunoki--2017|Kusunoki, 2017]] , 2018b; [[#Scher--2017|Scher et al., 2017]] ) as these models tend to produce more realistic storms compared to coarser models ( [[#11.7.2|Section 11.7.2]] ). Higher horizontal resolution alone improves simulation of extreme precipitation in some models ( [[#Wehner--2014|Wehner et al., 2014]] ; [[#Kusunoki--2017|Kusunoki, 2017]] , 2018b), but this is insufficient in other models ( [[#Bador--2020|Bador et al., 2020]] ) as parametrization also plays a significant role (M. [[#Wu--2020|]] [[#Wu--2020|Wu et al., 2020]] ). A simple comparison of climatology may not fully reflect the improvements of the new models that have more comprehensive process formulations ( [[#Di%20Luca--2015|Di Luca et al., 2015]] ). [[#Dittus--2016|Dittus et al. (2016)]] found that many of the eight CMIP5 models they evaluated reproduced the observed increase in the difference between areas experiencing an extreme high (90%) and an extreme low (10%) proportion of the annual total precipitation from heavy precipitation (R95p/PRCPTOT) for Northern Hemisphere regions. Additionally, CMIP5 models reproduced the relation between changes in extreme and non-extreme precipitation: an increase in extreme precipitation is at the cost of a decrease in non-extreme precipitation ( [[#Thackeray--2018|Thackeray et al., 2018]] ), a characteristic found in the observational record ( [[#Gu--2018|Gu and Adler, 2018]] ).

The CMIP6 models perform reasonably well in capturing large-scale features of precipitation extremes, including intense precipitation extremes in the intertropical convergence zone (ITCZ), and weak precipitation extremes in dry areas in the tropical regions ( [[#Li--2021|Li et al., 2021]] ) but a double-ITCZ bias over the equatorial central and eastern Pacific that appeared in CMIP5 models remains ( [[IPCC:Wg1:Chapter:Chapter-3#3.3.2.3|Section 3.3.2.3]] ). There are also regional biases in the magnitude of precipitation extremes ( [[#Kim--2020|Kim et al., 2020]] ). The models also have difficulties in reproducing detailed regional patterns of extreme precipitation, such as over the north-east USA ( [[#Agel--2020|Agel and Barlow, 2020]] ), though they performed better for summer extremes over the USA ( [[#Akinsanola--2020|Akinsanola et al., 2020]] ). The comparison between climatologies in the observations and in model simulations shows that the CMIP6 and CMIP5 models that have similar horizontal resolutions also have similar model evaluation scores, and their error patterns are highly correlated ( [[#Wehner--2020|Wehner et al., 2020]] ). In general, extreme precipitation in CMIP6 models tends to be somewhat larger than in CMIP5 models ( [[#Li--2021|Li et al., 2021]] ), reflecting smaller spatial scales of extreme precipitation represented by slightly higher-resolution models ( [[#Gervais--2014|Gervais et al., 2014]] ). This is confirmed by [[#Kim--2020|Kim et al. (2020)]] , who showed that Rx1day and Rx5day simulated by CMIP6 models tend to be closer to point estimates of HadEX3 data ( [[#Dunn--2020|Dunn et al., 2020]] ) than those simulated by CMIP5. Figure 11.14 shows the multi-model ensemble bias in mean Rx1day over the period 1979–2014 from 21 available CMIP6 models when compared with observations and reanalyses. Measured by global land root-mean-square error, the model performance is generally consistent across different observed/reanalysis data products for the extreme precipitation metric (Figure 11.14). The magnitude of extreme area mean precipitation simulated by the CMIP6 models is consistently smaller than the point estimates of HadEX3, but the model values are more comparable to those of areal-mean values (Figure 11.14) of the ERA5 reanalysis or REGEN ( [[#Contractor--2020b|Contractor et al., 2020b]] ). Taylor-plot-based performance metrics reveal strong similarities in the patterns of extreme precipitation errors over land regions between CMIP5 and CMIP6 ( [[#Srivastava--2020|Srivastava et al., 2020]] ; [[#Wehner--2020|Wehner et al., 2020]] ) and between annual mean precipitation errors and Rx1day errors for both generations of models ( [[#Wehner--2020|Wehner et al., 2020]] ).

<div id="_idContainer057" class="Basic-Text-Frame"></div>

[[File:aee292997bbe519c407ddaba59181c17 IPCC_AR6_WGI_Figure_11_14.png]]

'''Figure 11.14 |''' '''Multi-model mean bias in annual maximum daily precipitation (Rx1day, %) for the period 1979–2014.''' Calculated as the difference between the Coupled Model Intercomparison Project Phase 6 (CMIP6) multi-model mean and the average of available observational or reanalysis products including ''(a)'' ERA5, ''(b)'' HadEX3, and ''(c)'' REGEN. Bias is expressed as the percent error relative to the long-term mean of the respective observational data products. Brown indicates that models are too dry, while green indicates that they are too wet. Areas without sufficient observational data are shown in grey. Adapted from [[#Wehner--2020|Wehner et al. (2020)]] under the terms of the Creative Commons Attribution licence. Further details on data sources and processing are available in the chapter data table (Table 11.SM.9).

In general, there is ''high confidence'' that historical simulations by CMIP5 and CMIP6 models of similar horizontal resolutions are interchangeable in their performance in simulating the observed climatology of extreme precipitation ''.''

Studies using regional climate models (RCMs), for example, CORDEX ( [[#Giorgi--2009|Giorgi et al., 2009]] ) over Africa ( [[#Dosio--2015|Dosio et al., 2015]] ; [[#Klutse--2016|Klutse et al., 2016]] ; [[#Pinto--2016|Pinto et al., 2016]] ; [[#Gibba--2019|Gibba et al., 2019]] ), Australia, East Asia ( [[#Park--2016|Park et al., 2016]] ), Europe ( [[#Prein--2016a|Prein et al., 2016a]] ; [[#Fantini--2018|Fantini et al., 2018]] ), and parts of North America ( [[#Diaconescu--2018|Diaconescu et al., 2018]] ) suggest that extreme rainfall events are better captured in RCMs compared to their host GCMs due to their ability to address regional characteristics, for example, topography and coastlines. However, CORDEX simulations do not show good skill over South Asia for heavy precipitation, and do not add value with respect to their GCM source of boundary conditions ( [[#Mishra--2014b|Mishra et al., 2014b]] ; S. [[#Singh--2017|]] [[#Singh--2017|Singh et al., 2017]] ). The evaluation of models in simulating regional processes is discussed in detail in [[IPCC:Wg1:Chapter:Chapter-10#10.3.3.4|Section 10.3.3.4]] . The high-resolution simulation of mid-latitude winter extreme precipitation over land is of similar magnitude to point observations. Simulation of summer extreme precipitation has a large bias when compared with observations at the same spatial scale. Simulated extreme precipitation in the tropics also appears to be too large, indicating possible deficiencies in the parametrization of cumulus convection at this resolution. Indeed, precipitation distributions at both daily and sub-daily time scales are much improved with a convection-permitting model ( [[#Belušić--2020|Belušić et al., 2020]] ) over Western Africa ( [[#Berthou--2019b|Berthou et al., 2019b]] ), East Africa ( [[#Finney--2019|Finney et al., 2019]] ), North America and Canada ( [[#Cannon--2019|Cannon and Innocenti, 2019]] ; [[#Innocenti--2019|Innocenti et al., 2019]] ) and over Belgium in Europe ( [[#Vanden%20Broucke--2019|Vanden Broucke et al., 2019]] ).

In summary, there is ''high confidence'' in the ability of models to capture the large-scale spatial distribution of precipitation extremes over land. The magnitude and frequency of extreme precipitation simulated by CMIP6 models are similar to those simulated by CMIP5 models ( ''hig'' ''h confidence'' ).

<div id="11.4.4" class="h2-container"></div>

<span id="detection-and-attribution-event-attribution-1"></span>