Editing IPCC:AR6/WGI/Chapter-3 (section)

=== 3.8.2 Multivariate Model Evaluation ===

<div id="h2-29-siblings" class="h2-siblings"></div>

Similar to the assessment of multivariate attribution of climate change in the previous section, this section covers the performance of the models across different variables (Sections 3.8.2.1) and different classes of models ( [[#3.8.2.2|Section 3.8.2.2]] ). Here the focus is on a system-wide assessment using integrative measures of model performance that characterize model performance using multiple diagnostic fields derived from multi-model ensembles.

<div id="3.8.2.1" class="h3-container"></div>

<span id="integrative-measures-of-model-performance"></span>
==== 3.8.2.1 Integrative Measures of Model Performance ====

<div id="h3-26-siblings" class="h3-siblings"></div>

The purpose of this section is to use multivariate analyses to address how well models simulate present-day and historical climate. For every diagnostic field considered, model performance is compared to one or multiple observational references, and the quality of the simulation is expressed as a single number, for example a correlation coefficient or a root mean square difference versus the observational reference. By simultaneously assessing different performance indices, model improvements can be quantified, similarities in behaviour between different models become apparent, and dependencies between various indices become evident ( [[#Gleckler--2008|Gleckler et al., 2008]] ; [[#Waugh--2008|Waugh and Eyring, 2008]] ).

AR5 found significant differences between models in the simulation of mean climate in the CMIP5 ensemble when measured against meteorological reanalyses and observations ( [[#Flato--2013|Flato et al., 2013]] ), see also [[#Stouffer--2017|Stouffer et al. (2017)]] . The AR5 determined that for the diagnostic fields analysed, the models usually compared similarly against two different reference datasets, suggesting that model errors were generally larger than observational uncertainties or other differences between the observational references. In agreement with previous assessments, the CMIP5 multi-model mean generally performed better than individual models ( [[#Annan--2011|Annan and Hargreaves, 2011]] ; [[#Rougier--2016|Rougier, 2016]] ). The AR5 considered 13 atmospheric fields in its assessment for the instrumental period but did not assess multi-variate model performance in other climate domains (e.g., ocean, land, and sea ice). The AR5 found only modest improvement regarding the simulation of climate for two periods of the Earth’s history (the Last Glacial Maximum and the mid-Holocene) between CMIP5 and previous paleoclimate simulations. Similarly, for the modern period only modest, incremental progress was found between CMIP3 and CMIP5 regarding the simulation of precipitation and radiation. The representation of clouds also showed improvement, but remained a key challenge in climate modelling ( [[#Flato--2013|Flato et al., 2013]] ).

The type of multi-variate analysis of models presented in AR5 remains critical to building confidence for example in projections of climate change. It is expanded here to the previous-generation CMIP3 and present-generation CMIP6 models and also to more variables and more climate domains, covering land and ocean as well as sea ice. The multi-variate evaluation of these three generations of models is performed relative to the observational datasets listed in Annex I, Table AI.1. For many of these datasets, a rigorous characterization of the observational uncertainty is not available, see discussion in Chapter 2. Here, as much as possible, multiple independent observational datasets are used. Disagreements among them would cause differences in model scoring, indicating that observational uncertainties may be substantial compared to model errors. Conversely, similar scores against different observational datasets would suggest model biases may be larger than the observational uncertainty.

An analysis of a basket of 16 atmospheric variables (Figure 3.42a) assessed across CMIP3, CMIP5, and CMIP6 models but excluding high-resolution models participating in HighResMIP, reveals the progress made between these three generations of models ( [[#Bock--2020|Bock et al., 2020]] ). Progress is evidenced by the increasing prevalence of blue colours (indicating a performance better than the median) for the more recent model versions. Additionally, a few CMIP6 models outperform the best-performing CMIP5 models. Progress is evident across all 16 variables. As noted in AR5, the models typically score similarly against both observational reference datasets, indicating that indeed uncertainties in these reference datasets are smaller than model biases. Several models and model families perform better compared to observational references than the median, across a majority of the climate variables assessed, and conversely some other models or model families compare more poorly against these reference datasets. Such a good correspondence across a range of diagnostic fields probing different aspects of climate enhances confidence that the improved performances reflect progress in the physical realism of these simulations. An alternative explanation, that progress is due to a cancellation of errors achieved by model tuning, appears improbable given the large number of diagnostic fields involved here. However, several instances of poor model performance (red colours in Figure 3.42) still exist in the CMIP6 ensemble. Family relationships (i.e. various degrees of shared formulations; [[#Knutti--2013|Knutti et al., 2013]] ) between the models are apparent, for example, the GISS, GFDL, CESM, CNRM, and HadGEM/UKESM1/ACCESS families score similarly across all atmospheric variables, both for the CMIP5 and CMIP6 generations. In the cases of CESM2/CESM2(WACCM), CNRM-CM6-1/CNRM-ESM2-1, NorCPM1/NorESM2-LM, and HadGEM3-GC31-LL/UKESM1-0-LL, the high-complexity model scores as well or better than its lower-complexity counterpart, indicating that increasing complexity by adding Earth system features, which by removing constraints could be expected to degrade a model’s performance, does not necessarily do so. Several high climate-sensitivity models (Section 7.5; [[#Meehl--2020|Meehl et al., 2020]] ), in particular CanESM5, CESM2, CESM2-WACCM, HadGEM3-GC31-LL, and UKESM1-0-LL, score well against the benchmarks. In accordance with AR5 and earlier assessments, the multi-model mean, with some notable exceptions, is better than any individual model ( [[#Annan--2011|Annan and Hargreaves, 2011]] ; [[#Rougier--2016|Rougier, 2016]] ).

<div id="_idContainer094" class="Basic-Text-Frame"></div>

[[File:a39746d3d4c62039c839f67687ac547f IPCC_AR6_WGI_Figure_3_42.png]]

Figure 3.42 | '''Relative space–time root-mean-square deviation (RMSD) calculated from the climatological seasonal cycle of the CMIP simulations (198''' '''0–''' '''1999) compared to observational datasets. (a)''' CMIP3, CMIP5, and CMIP6 for 16 atmospheric variables '''(b)''' CMIP5 and CMIP6 for 10 land variables and four ocean/sea-ice variables. A relative performance measure is displayed, with blue shading indicating better and red shading indicating worse performance than the median of all model results. A diagonal split of a grid square shows the relative error with respect to the reference data set (lower right triangle) and an additional data set (upper left triangle). Reference/additional datasets are from top to bottom in (a): ERA5/NCEP, GPCP-SG/GHCN, CERES-EBAF, CERES-EBAF, CERES-EBAF, CERES-EBAF, JRA-55/ERA5, ESACCI-SST/HadISST, ERA5/NCEP, ERA5/NCEP, ERA5/NCEP, ERA5/NCEP, ERA5/NCEP, ERA5/NCEP, AIRS/ERA5, ERA5/NCEP and in (b): CERES-EBAF, CERES-EBAF, CERES-EBAF, CERES-EBAF, LandFlux-EVAL, Landschuetzer2016/ JMA-TRANSCOM; MTE/FLUXCOM, LAI3g, JMA-TRANSCOM, ESACCI-SOILMOISTURE, HadISST/ATSR, HadISST, HadISST, ERA-Interim. White boxes are used when data are not available for a given model and variable. Figure is updated and expanded from [[#Bock--2020|Bock et al. (2020)]] , their Figure 5 CC BY 4.0 [https://dx.doi.org/10.1002/rog.20022 https://creativecommons.org/licenses/by/4.0/] . Further details on data sources and processing are available in the chapter data table (Table 3.SM.1).

Regarding model performance for the ocean and the cryosphere (Figure 3.42b), it is apparent that for many models there are substantial differences between the scores for Arctic and Antarctic sea ice concentration. This might suggest that it is not sea ice physics directly that is driving such differences in performance but rather other influences, such as differences in geography, the role of large ice shelves (which are absent in the Arctic), or large-scale ocean dynamics. As for atmospheric variables, progress is evident also across all four ocean and ten land variables from CMIP5 to CMIP6.

In summary, CMIP6 models perform generally better for a basket of variables covering mean historical climate across the atmosphere, ocean, and land domains than previous-generation and older models ( ''high confidence'' ). Earth System models characterized by additional biogeochemical feedbacks often perform at least as well as related more constrained, lower-complexity models lacking these feedbacks ( ''medium confidence'' ). In many cases, the models score similarly against both observational references, indicating that model errors are usually larger than observational uncertainties ( ''high confidence'' ). Moreover, synthesizing across Sections 3.3–3.7, we assess that the CMIP6 multi-model mean captures most aspects of observed climate change well ( ''high confidence'' ).

Using centred pattern correlations (quantifying pattern similarity on a scale of –1 to 1, with 1 expressing perfect similarity and 0 no relationship) for selected fields, AR5 documented improvements between CMIP3 and CMIP5 in surface air temperature, outgoing longwave radiation, and precipitation (Figure 9.6 of [[#Flato--2013|Flato et al., 2013]] ). Little further progress between CMIP3 and CMIP5 was found for fields that were already quite well simulated in CMIP3 (such as surface air temperature and outgoing longwave radiation). For precipitation, the spread reduced because the worst-performing models improved. The shortwave cloud radiative effect remained relatively poorly simulated with significant inter-model spread (e.g., [[#Calisto--2014|Calisto et al., 2014]] ). This comparison of centred pattern correlations is designed to help determine the quality of simulation of different diagnostics relative to each other, and also to examine progress between generations of models. Figure 3.43 shows the centred pattern correlations for 16 variables for CMIP3, CMIP5 and CMIP6 models. In the ensemble averages, CMIP6 performs better than CMIP5 and CMIP3 for near-surface temperature, precipitation, mean sea-level pressure, and many other variables. For the variables shown, the uncertainties in observational datasets, in particular for precipitation and northward wind at 850 hPa, remain substantial relative to mean model errors (see grey dots in Figure 3.43).

<div id="_idContainer096" class="•-2-columns"></div>

[[File:010a3587f1a6573bfacb264de3bea8fd IPCC_AR6_WGI_Figure_3_43.png]] Figure 3.43 | '''Centred pattern correlations between models and observations for the annual mean climatology over the period 1980–1999.''' Results are shown for individual CMIP3 (green), CMIP5 (blue) and CMIP6 (red) models (one ensemble member from each model is used) as short lines, along with the corresponding multi-model ensemble averages (long lines). Correlations are shown between the models and the primary reference observational data set (from left to right: ERA5, GPCP-SG, CERES-EBAF, CERES-EBAF, CERES-EBAF, CERES-EBAF, JRA-55, ESACCI-SST, ERA5, ERA5, ERA5, ERA5, ERA5, ERA5, AIRS, ERA5). In addition, the correlation between the primary reference and additional observational datasets (from left to right: NCEP, GHCN, -, -, -, -, ERA5, HadISST, NCEP, NCEP, NCEP, NCEP, NCEP, NCEP, ERA5, NCEP) are shown (solid grey circles) if available. To ensure a fair comparison across a range of model resolutions, the pattern correlations are computed after regridding all datasets to a resolution of 4° in longitude and 5° in latitude. Figure is updated and expanded from [[#Bock--2020|Bock et al. (2020)]] , their Figure 7 CC BY 4.0 [https://dx.doi.org/10.1038/nature19082 https://creativecommons.org/licenses/by/4.0/] . Further details on data sources and processing are available in the chapter data table (Table 3.SM.1).

In addition to the multivariate assessments of simulations of the recent historical period, simulations of selected periods of the Earth’s more distant history can be used to benchmark climate models by exposing them to climate forcings that are radically different from the present and recent past ( [[#Harrison--2015|Harrison et al., 2015]] , 2016; [[#Kageyama--2018|Kageyama et al., 2018]] ; [[#Tierney--2020a|Tierney et al., 2020a]] ). These time periods provide an out-of-sample test of models because they are not in general used in the process of model development. They encompass a range of climate drivers, such as volcanic and solar forcing for the Last Millennium, orbital forcing for the mid-Holocene and Last Interglacial, and changes in greenhouse gases and ice sheets for the LGM, mid-Pliocene Warm Period, and early Eocene (Sections 2.2 and 2.3). These drivers led to climate changes, including in surface temperature ( [[IPCC:Wg1:Chapter:Chapter-2#2.3.1.1|Section 2.3.1.1]] ) and the hydrological cycle ( [[IPCC:Wg1:Chapter:Chapter-2#2.3.1.3.1|Section 2.3.1.3.1]] ), which are described by paleoclimate proxies that have been synthesized to support evaluations of models on a global and regional scale. However, the more sparse, indirect, and regionally incomplete climate information available from paleo-archives motivates a different form of the multivariate analysis of simulations covering these periods versus the equivalent for the historical period, as described below.

AR5 found that reconstructions and simulations of past climates both show similar responses in terms of large-scale patterns of climate change, such as polar amplification ( [[#Flato--2013|Flato et al., 2013]] ; [[#Masson-Delmotte--2013|Masson-Delmotte et al., 2013]] ). However, for several regional signals (e.g., the north–south temperature gradient in Europe and regional precipitation changes), the magnitude of change seen in the proxies relative to the pre-industrial period was underestimated by the models. When benchmarking CMIP5/PMIP3 models against reconstructions of the mid-Holocene and LGM, AR5 found only a slight improvement compared with earlier model versions across a range of variables. For the Last Interglacial, it was noted that the magnitude of observed annual mean warming in the Northern Hemisphere was only reached in summer in the models. For the mid-Pliocene Warm Period, it was noted that both proxies and models showed a polar amplification of temperature compared with the pre-industrial period, but a formal model evaluation was not carried out.

Since AR5, new simulation protocols have been developed in PMIP4 ( [[#Kageyama--2018|Kageyama et al., 2018]] ), which are further described for the mid-Holocene and the Last Interglacial by [[#Otto-Bliesner--2017|Otto-Bliesner et al. (2017)]] , for the LGM by [[#Kageyama--2017|Kageyama et al. (2017)]] , for the Pliocene by [[#Haywood--2016|Haywood et al. (2016)]] , and for the early Eocene by [[#Lunt--2017|Lunt et al. (2017)]] . These have resulted in new model simulations for these time periods ( [[#Brierley--2020|Brierley et al., 2020]] ; [[#Haywood--2020|Haywood et al., 2020]] ; [[#Kageyama--2021a|Kageyama et al., 2021a]] ; [[#Lunt--2021|Lunt et al., 2021]] ; [[#Otto-Bliesner--2021|Otto-Bliesner et al., 2021]] ). These time periods span an assessed temperature range of 20°C ( [[IPCC:Wg1:Chapter:Chapter-2#2.3.1.1|Section 2.3.1.1]] ), and for all periods the PMIP4 multi-model ensemble mean is within 0.5°C of the assessed range of GSAT (Figure 3.44a). Those time periods for which the multi-model ensemble mean is outside the assessed range of GSAT, the mid-Holocene and the Last Interglacial, are primarily forced by orbital changes not greenhouse gas forcing, and as a result the forcing as well as the assessed and modelled response are relatively close to zero in the global annual mean. During these periods, climate change therefore is a consequence of more poorly understood Earth System feedbacks acting on the response to orbital differences versus the present-day, affecting the seasonality of insolation.

<div id="_idContainer098" class="•-2-columns"></div>

[[File:a8dd352c0ca568ef2392a9996863ccd2 IPCC_AR6_WGI_Figure_3_44.png]]

Figure 3.44 | '''Multivariate synopsis of paleoclimate model results compared to observational references.''' Data-model comparisons for (a) GSAT anomalies for five PMIP4 periods and for regional features for the '''(b)''' mid-Holocene and '''(c)''' LGM periods, for PMIP3 and PMIP4 models. The results from CMIP6 models are shown as coloured dots. In (a) the light orange shading shows the ''very likely'' assessed ranges presented in [[IPCC:Wg1:Chapter:Chapter-2#2.3.1.1|Section 2.3.1.1]] . In (b) and (c), the regions and variables are defined as follows: North America (20°N–50°N, 140°W–60°W), Western Europe (35°N–70°N, 10°W–30°E) and West Africa (0°–30°N, 10°W–30°E); mean temperature of the coldest month (MTCO; °C), mean temperature of the warmest month (MTWA; °C), mean annual precipitation (MAP; mm yr <sup>–1</sup> ). In (b) and (c) the ranges shown for the reconstructions ( [[#Bartlein--2011|Bartlein et al. (2011)]] for mid-Holocene and [[#Cleator--2020|Cleator et al. (2020)]] for LGM) are based on the standard error given at each site: the average and associated standard deviation over each area is obtained by computing 1000 times the average of randomly drawn values from the Gaussian distributions defined at each site by the reconstruction mean and standard error; the light orange colour shows the ±1 standard deviation of these 1000 estimates. The dots on (b) and (c) show the average of the model output for grid points for which there are reconstructions. The ranges for the model results are based on an ensemble of 1000 averages over 50 years randomly picked in the model output time series for each region and each variable: the mean ± one standard deviation is plotted for each model. Figure is adapted from [[#Brierley--2020|Brierley et al. (2020)]] , their Figure S3 for the mid-Holocene; and from [[#Kageyama--2021b|Kageyama et al. (2021b)]] , their Figure 12 for the LGM. Further details on data sources and processing are available in the chapter data table (Table 3.SM.1).

Polar amplification in the LGM, mid-Pliocene Warm Period, and Early Eocene Climatic Optimum (EECO) simulations is assessed in Section 7.4.4.1.2. Here we focus on the mid-Holocene and the LGM, which have been a part of AMIP or CMIP through several assessment cycles, and as such serve as a reference to quantify regional model-data agreement from one IPCC assessment to another. We compare the results from 15 CMIP6 models using the PMIP4 protocol (CMIP6-PMIP4), with non-CMIP6 models using the PMIP4 protocol, with PMIP3 models, and with regional temperature and precipitation changes from proxies for the mid-Holocene (Figure 3.44b). For six out of seven variables shown, the CMIP6 multi-model mean captures the correct sign of the change. For five out of seven of them the CMIP6 ensemble mean is within the reconstructed range. For the other two variables (changes in the mean temperature of the warmest month over North America and in the mean annual precipitation over West Africa) nearly all PMIP4 and PMIP3 models are outside the reconstructed range. CMIP6 models show regional patterns of temperature changes similar to the PMIP3 ensemble ( [[#Brierley--2020|Brierley et al., 2020]] ), but the slight mid-Holocene cooling in PMIP4 compared with PMIP3, probably associated with lower imposed mid-Holocene carbon dioxide concentrations ( [[#Otto-Bliesner--2017|Otto-Bliesner et al., 2017]] ), improves the regional model performance for summer and winter temperatures (Figure 3.44b). However, this cooling also results in a CMIP6 mid-Holocene GSAT that lies further from the assessed range (Figure 3.44a). All models show an expansion of the monsoon areas from the pre-industrial to the mid-Holocene simulations in the Northern Hemisphere, but this expansion in some cases is only large enough to cancel out the bias in the pre-industrial control simulations ( [[#3.3.3.2|Section 3.3.3.2]] ; [[#Brierley--2020|Brierley et al., 2020]] ). There is a slight improvement in representing the northward expansion of the West African monsoon region in PMIP4 compared with PMIP3 (Figures 3.11 and 3.44b).

Fourteen simulations of the LGM climate have been produced following the CMIP6-PMIP4 protocol using 11 models, five of which are from the latest CMIP6 generation. The multi-model-mean global cooling simulated by these models is close to that simulated by the CMIP5-PMIP3 ensemble, but the range of results is larger. The increase in the range is largely due to the inclusion of CESM2 which simulates a much larger cooling than the other PMIP4 models (Figure 3.44a). This is consistent with its larger climate sensitivity (see also ( [[#3.3.1.1|Section 3.3.1.1]] ; [[#Zhu--2021|Zhu et al., 2021]] ). The other models on average also simulate slightly larger cooling in PMIP4 versus PMIP3 ( [[#Kageyama--2017|Kageyama et al., 2017]] , 2021a). The PMIP4 multi-model mean is within the range of reconstructed regional averages for four out of seven regional variables; this is unchanged from PMIP3 but for different variables (Figure 3.44c). For all fields, the results of many individual models are outside the reconstructed range. For two variables out of seven (changes in the mean temperature of the warmest month and mean annual precipitation over Western Europe) no model is within the range of the reconstructions. This analysis is strengthened compared with the equivalent analysis in AR5 because it is based on larger and improved reconstructions ( [[#Cleator--2020|Cleator et al., 2020]] ). Most CMIP6-PMIP4 models simulate a slightly stronger AMOC in the LGM, but no strong deepening of the AMOC ( [[#Kageyama--2021a|Kageyama et al., 2021a]] ), while most other PMIP4 models simulate a strengthening and strong deepening of the AMOC, as was the case for the PMIP3 models ( [[#Muglia--2015|Muglia and Schmittner, 2015]] ; [[#Sherriff-Tadano--2018|Sherriff-Tadano et al., 2018]] ). Only one model (CESM1.2) shows a shoaling of LGM AMOC which is consistent with reconstructions ( [[#Marzocchi--2017|Marzocchi and Jansen, 2017]] ; [[#Sherriff-Tadano--2018|Sherriff-Tadano et al., 2018]] ).

17 PMIP4 models completed Last Interglacial simulations ( [[#Otto-Bliesner--2021|Otto-Bliesner et al., 2021]] ). The comparison to reconstructions is generally good, except for some discrepancies, such as for upwelling systems in the South East Atlantic or discrepancies which may result from local melting of remnant ice sheets absent in the Last Interglacial simulation protocol. All models simulate a decrease in Arctic sea ice in summer, commensurate with increased summer insolation, while some models even simulate a large or complete loss ( [[#Guarino--2020|Guarino et al., 2020]] ; [[#Kageyama--2021b|Kageyama et al., 2021b]] ). Sea ice reconstructions for the central Arctic are, however, too uncertain to evaluate this behaviour. The Last Interglacial simulations indicate a clear relationship between simulated sea ice loss and model responses to increased greenhouse gas forcing ( [[#Kageyama--2021b|Kageyama et al., 2021b]] ; [[#Otto-Bliesner--2021|Otto-Bliesner et al., 2021]] ).

Overall, the PMIP multi-model means agree very well (within 0.5°C of the assessed range) with GSAT reconstructed from proxies across multiple time periods, spanning a range from 6°C colder than pre-industrial (Last Glacial Maximum) to 14°C warmer than pre-industrial (Early Eocene Climate Optimum) ( ''high confidence'' ). During the orbitally-forced mid-Holocene, the CMIP6 multi-model mean captures the sign of the regional changes in temperature and precipitation in most regions assessed, and there have been some regional improvements compared to AR5 ( ''medium confidence'' ). The limited number of CMIP6 simulations of the LGM hinders model evaluation of the multi-model mean, but for both LGM and mid-Holocene, models tend to underestimate the magnitude of large changes ( ''high confidence'' ). Some long-standing model-data discrepancies, such as a dry bias in North Africa in the mid-Holocene, have not improved in CMIP6 compared with PMIP3 ( ''high confidence'' ).

<div id="3.8.2.2" class="h3-container"></div>

<span id="process-representation-in-different-classes-of-models"></span>
==== 3.8.2.2 Process Representation in Different Classes of Models ====

<div id="h3-27-siblings" class="h3-siblings"></div>

Based on new scientific insights and newly available observations, many improvements have been made to models from CMIP5 to CMIP6, including changes in the representation of physics of the atmosphere, ocean, sea ice, and land surface. In many cases, changes in the detailed representation of cloud and aerosol processes have been implemented. The new generation of CMIP6 climate models also features increases in spatial resolution, as well as inclusion of additional Earth system processes and new components (see further details in [[IPCC:Wg1:Chapter:Chapter-1#1.5.3.1|Section 1.5.3.1]] and in Tables AII.5 and AII.6). Such changes to model physics and resolution are often designed to improve the fitness-for-purpose of a model such as for projecting regional aspects of climate (Section 10.3) or to more fully represent feedbacks to make the models more fit for long-term climate projections affected for example by carbon cycle feedbacks (see also ( [[IPCC:Wg1:Chapter:Chapter-1#1.5.3.1|Section 1.5.3.1]] ).

Factors affecting model performance include resolution, the type of dynamical core (spectral, finite difference or finite volume), physics parameters and parameterisations, model structure, for example, many of the coupled HighResMIP models ( [[#Haarsma--2016|Haarsma et al., 2016]] ) use the NEMO ocean model, affecting model diversity, and the range and degree of process realism (e.g., for aerosols, atmospheric chemistry and other Earth System components). This section particularly explores the influence of model resolution and of complexity on model performance (see also Section 8.5.1).

A key advance in CMIP6 compared to CMIP5 is the presence of high-resolution models that have participated in HighResMIP. Resolution alone can significantly affect a model’s performance, with some effects propagating to the global scale. Recent studies have shown that enhancing the horizontal resolution of models is seen to significantly affect aspects of large-scale circulation as well as improve the simulation of small-scale processes and extremes when compared to CMIP3 and CMIP5 models (see also Section 11.4.3; [[#Haarsma--2016|Haarsma et al., 2016]] ), with some models approaching 10 km resolution in the atmosphere ( [[#Kodama--2021|Kodama et al., 2021]] ) or ocean ( [[#Caldwell--2019|Caldwell et al., 2019]] ; [[#Gutjahr--2019|Gutjahr et al., 2019]] ; [[#Roberts--2019|Roberts et al., 2019]] ; [[#Chang--2020|Chang et al., 2020]] ; [[#Semmler--2020|Semmler et al., 2020]] ).

As discussed in Section [[#_idTextAnchor000|3.3]] , CMIP6 models reproduce observed large-scale mean surface temperature patterns as well as their CMIP5 predecessors, but biases in surface temperature in the mean of HighResMIP models are smaller than those in the mean of the corresponding standard resolution CMIP6 configurations of the same models ( [[#3.3.1.1|Section 3.3.1.1]] and Figure 3.3). The extent and causes of improvements due to increased horizontal resolutions in the atmosphere and ocean domains depend on the model ( [[#Kuhlbrodt--2018|Kuhlbrodt et al., 2018]] ; [[#Roberts--2018|Roberts et al., 2018]] , 2019; [[#Sidorenko--2019|Sidorenko et al., 2019]] ), although they typically involve better process representation (for example of ocean currents and atmospheric storms) which can lead to reduced biases in top of atmosphere radiation and cloudiness. Precipitation has likewise improved in CMIP6 versus CMIP5 models, but biases remain. The high resolution (&lt;25 km) class of models participating in HighResMIP compares regionally better against observations than the standard resolution CMIP6 models (of order 100 km, Figure 3.13; [[#3.3.2|Section 3.3.2]] ), partly because of an improved representation of orographic (mountain-induced) precipitation which constitutes a major fraction of precipitation on land, but other processes also play an important role ( [[#Vannière--2019|Vannière et al., 2019]] ). However, there are also large parts of the tropical ocean where precipitation in high-resolution models is not improved compared to standard resolution CMIP6 models ( [[#Vannière--2019|Vannière et al., 2019]] ).

Additionally, the representation of surface and deeper ocean mean temperature is improved in models with higher horizontal resolution (Sections 3.5.1.1 and 3.5.1.2) with systematic improvements in coupled tropical Atlantic sea surface temperature and precipitation biases at higher resolutions ( [[#Roberts--2019|Roberts et al., 2019]] , single model; [[#Vannière--2019|Vannière et al., 2019]] , multi-model), the North Atlantic cold bias ( [[#Bock--2020|Bock et al., 2020]] , multi-model; [[#Roberts--2018|Roberts et al., 2018]] , 2019; [[#Caldwell--2019|Caldwell et al., 2019]] ; all single models) as well as deep-ocean biases ( [[#Small--2014|Small et al., 2014]] ; [[#Griffies--2015|Griffies et al., 2015]] ; [[#Caldwell--2019|Caldwell et al., 2019]] ; [[#Gutjahr--2019|Gutjahr et al., 2019]] ; [[#Roberts--2019|Roberts et al., 2019]] ; [[#Chang--2020|Chang et al., 2020]] , all single model studies). Atlantic ocean transports (heat and volume) are also generally improved compared to observations ( [[#Grist--2018|Grist et al., 2018]] ; [[#Caldwell--2019|Caldwell et al., 2019]] ; [[#Docquier--2019|Docquier et al., 2019]] ; [[#Roberts--2019|Roberts et al., 2019]] , 2020c; [[#Chang--2020|Chang et al., 2020]] ), as well as some aspects of air-sea interactions (P. [[#Wu--2019|Wu et al., 2019]] , single model; [[#Bellucci--2021|Bellucci et al., 2021]] , multi-model). However, warm-biased sea surface temperatures in the Southern Ocean are worse in comparison to standard resolution CMIP6 models ( [[#Bock--2020|Bock et al., 2020]] ). The AR5 noted problems with the simulation of clouds in this region which were later attributed to a lack of supercooled liquid clouds ( [[#Bodas-Salcedo--2016|Bodas-Salcedo et al., 2016]] ). Mesoscale ocean processes are critical to maintaining the Southern Ocean stratification and response to wind forcing ( [[#Marshall--2003|Marshall and Radko, 2003]] ; [[#Hallberg--2006|Hallberg and Gnanadesikan, 2006]] ), and their explicit representation requires even higher ocean resolution ( [[#Hallberg--2013|Hallberg, 2013]] ). Similarly, atmospheric convection remains unresolved even in the highest-resolution climate models participating in HighResMIP. However, there is also evidence of improvements in the frequency, distribution and interannual variability of tropical cyclones in HighResMIP ( [[#Roberts--2020a|Roberts et al., 2020a]] , b), particularly in the Northern Hemisphere (see further discussion in Section 11.7.1.3), and their interaction with the ocean ( [[#Scoccimarro--2017|Scoccimarro et al., 2017]] , single model), as well as the global moisture budget ( [[#Vannière--2019|Vannière et al., 2019]] ). At higher resolution the track density of tropical cyclones is increased practically everywhere where tropical cyclones occur. Simulation of some climate extremes is shown to be improved at higher resolution including explosively developing extra-tropical cyclones ( [[#Vries--2019|Vries et al., 2019]] ; [[#Jiaxiang--2020|Jiaxiang et al., 2020]] ), blocking ( [[#3.3.3.3|Section 3.3.3.3]] ; [[#Fabiano--2020|Fabiano et al., 2020]] ; [[#Schiemann--2020|Schiemann et al., 2020]] ) and European extreme precipitation due to a better representation of the North Atlantic storm track ( [[#van%20Haren--2015|van Haren et al., 2015]] ) and orographic boundary conditions ( [[#Schiemann--2018|Schiemann et al., 2018]] ).

In CMIP6 a number of Earth system models have increased the realism by which key biogeochemical aspects of the coupled Earth system are represented, affecting, for example, the carbon and nitrogen cycles, aerosols, and atmospheric chemistry (e.g., [[#Cao--2018|Cao et al., 2018]] ; [[#Gettelman--2019|Gettelman et al., 2019]] ; [[#Lin--2019|Lin et al., 2019]] ; [[#Mauritsen--2019|Mauritsen et al., 2019]] ; [[#Séférian--2019|Séférian et al., 2019]] ; [[#Sellar--2019|Sellar et al., 2019]] ; [[#Sidorenko--2019|Sidorenko et al., 2019]] ; [[#Swart--2019|Swart et al., 2019]] ; [[#Dunne--2020|Dunne et al., 2020]] ; [[#Seland--2020|Seland et al., 2020]] ; [[#Wu--2020|Wu et al., 2020]] ; [[#Ziehn--2020|Ziehn et al., 2020]] ). In addition to increased process realism, the level of coupling between the physical climate and biogeochemical components of the Earth system has also been enhanced in some models ( [[#Mulcahy--2020|Mulcahy et al., 2020]] ) as well as across different biogeochemical components (see Section 5.4 for a discussion and Table 5.4 for an overview). For example, the nitrogen cycle is now simulated in several ESMs ( [[#Zaehle--2015|Zaehle et al., 2015]] ; [[#Davies-Barnard--2020|Davies-Barnard et al., 2020]] ). This advance accounts for the fertilization effect nitrogen availability has on vegetation and carbon uptake, reducing uncertainties in the simulations of the carbon uptake responses to physical climate change ( [[#3.6.1|Section 3.6.1]] ) and to CO <sub>2</sub> increases ( [[#Arora--2020|Arora et al., 2020]] ), thus improving confidence in the simulated airborne fraction of CO <sub>2</sub> emissions ( [[#Jones--2020|Jones and Friedlingstein, 2020]] ) and better constraining remaining carbon budgets (Section 5.5). Such advances also allow investigation of land-based climate change mitigation options (e.g., through changes in land management and associated terrestrial carbon uptake ( [[#Mahowald--2017|Mahowald et al., 2017]] ; [[#Pongratz--2018|Pongratz et al., 2018]] )) or interactions between different facets of the managed Earth system, such as interactions between mitigation efforts targeting climate warming and air quality ( [[#West--2013|West et al., 2013]] ). A number of developments also explicitly target improved simulation of the past.

Further such ESM developments include: (i) Apart from the nitrogen cycle, extending terrestrial carbon cycle models to simulate interactions between the carbon cycle and other nutrient cycles, such as phosphorus, that are known to play an important role in limiting future plant uptake of CO <sub>2</sub> ( [[#Zaehle--2015|Zaehle et al., 2015]] ). (ii) Introducing explicit coupling between interactive atmospheric chemistry and aerosol schemes ( [[#Gettelman--2019|Gettelman et al., 2019]] ; [[#Sellar--2019|Sellar et al., 2019]] ), which has been shown to affect estimates of historical aerosol radiative forcing ( [[#Karset--2018|Karset et al., 2018]] ). Furthermore, interactive treatment of atmospheric chemistry in a full ESM supports investigation of interactions between climate and air quality mitigation efforts, such as in AerChemMIP ( [[#Collins--2017|Collins et al., 2017]] ), as well as interactions between stratospheric ozone recovery and global warming ( [[#Morgenstern--2018|Morgenstern et al., 2018]] ). (iii) Coupling between components of Earth system models has been extended to increase their utility for studying future interactions across the full Earth system, such as between ocean biogeochemistry and cloud-aerosol processes ( [[#Mulcahy--2020|Mulcahy et al., 2020]] ), and vegetation and impacts on dust production ( [[#Kok--2018|Kok et al., 2018]] ), production of secondary organic aerosols (SOA, [[#Zhao--2017|Zhao et al., 2017]] ) and Equilibrium Climate Sensitivity (ECS), whereby enhanced CO <sub>2</sub> fertilization of land vegetation causes changes in regional surface albedo ( [[#Andrews--2019|Andrews et al., 2019]] ). Increased coupling between physical climate and biogeochemical processes in a single ESM, along with an increased number of interactively represented processes, such as permafrost thaw, vegetation, wildfires and continental ice sheets increases our ability to investigate the potential for abrupt and interactive changes in the Earth system (see Sections 4.7.3 and 5.4.9, and Box 5.1). Table 5.4 provides an overview of recent advances in representing the carbon cycle in ESMs.

In summary, both high-resolution and high-complexity models have been evaluated as part of CMIP6. In comparison with standard resolution CMIP6 models, higher resolution probed under the HighResMIP activity ( [[#Haarsma--2016|Haarsma et al., 2016]] ) improves aspects of the simulation of climate (particularly concerning sea surface temperature) but discrepancies remain and there are some regions, such as parts of the Southern Ocean, where currently attainable resolution produces inferior performance ( ''high confidence'' ). Such model behaviour can indicate deficiencies in model physics that are not simply associated with resolution. In several cases, high-complexity ESMs that include additional interactions between Earth system components and thus have potential for additional associated model errors nevertheless perform as well as their low-complexity counterparts, illustrating that interactively simulating these Earth System components as part of the climate system is now well established.

<div id="acknowledgements" class="h1-container"></div>