Editing IPCC:AR6/WGII/Chapter-17 (section)

=== 17.5.2 Adaptation Monitoring, Evaluation &amp; Learning ===

<div id="h2-14-siblings" class="h2-siblings"></div>

<div id="17.5.2.1" class="h3-container"></div>

<span id="purpose-of-monitoring-and-evaluation"></span>
==== 17.5.2.1 Purpose of Monitoring and Evaluation ====

<div id="h3-28-siblings" class="h3-siblings"></div>

Adaptation responses have been observed in every region and across a wide variety of sectors ( [[IPCC:Wg2:Chapter:Chapter-16#16.3|Section 16.3]] ), but little evidence exists of their outcomes in terms of climate risk reduction ( ''high confidence'' ) ( [[IPCC:Wg2:Chapter:Chapter-1#1.4.3|Section 1.4.3]] ; [[#Ford--2016|Ford and Berrang-Ford, 2016]] ; [[#Tompkins--2018|Tompkins et al., 2018]] ; [[#Berrang-Ford--2021|Berrang-Ford et al., 2021]] ; [[#Eriksen--2021|Eriksen et al., 2021]] ; [[#UNEP--2021a|UNEP, 2021a]] ). To advance on that, the Paris Agreement is encouraging countries to engage in ‘Monitoring and evaluating and learning from adaptation plans, policies, programmes and actions’ ( [[#UN--2015|UN, 2015]] , Article 7.9d). Monitoring and evaluation (M&amp;E) is the systematic process of collecting, analysing and using information to assess the progress of adaptation and evaluate its effects—for example, risk reduction outcomes, co-benefits and trade-offs—mostly during and after implementation (AR6 Glossary, Annex II). Distinctions between monitoring and evaluation typically view monitoring as a continuous process of tracking implementation and informing management to allow for corrective action including in situations of deep uncertainty (see Cross-Chapter Box DEEP in this Chapter), while evaluation is described as a more comprehensive assessment of achievements, unintended effects and lessons learned carried out at certain point in time ( [[#OECD--2002|OECD, 2002]] ). M&amp;E is an important part of the adaptation process (Figure 1.9). It can help to generate information on adaptation success or maladaptive outcomes.

M&amp;E of adaptation is undertaken for different purposes, including: (1) understanding whether responses have achieved their intended objectives and contributed to a reduction in climate risks and vulnerability or to an increase of adaptive capacity and resilience, (2) informing ongoing implementation and future responses, and (3) providing upward and downward accountability (Preston et al., 2009; [[#UNFCCC--2010a|UNFCCC, 2010a]] ; [[#Pringle--2011|Pringle, 2011]] ; [[#Spearman--2011|Spearman and McGray, 2011]] ). M&amp;E is also commonly linked to learning ( [[#17.5.2.7|Section 17.5.2.7]] ). By continuously monitoring implementation, for example, to assess whether adaptation is on track or needs to be accelerated, M&amp;E can aid decision-making under uncertainty. Adaptation M&amp;E is distinct from tracking financial flows related to adaptation since financial accounting does not provide information on implementation and outcomes ( [[#17.5.2.5|Section 17.5.2.5]] ; [[#Adaptation%20Partnership--2012|Adaptation Partnership, 2012]] ; [[#World%20Bank%20Independent%20Evaluation%20Group--2012|World Bank Independent Evaluation Group, 2012]] ).

<div id="17.5.2.2" class="h3-container"></div>

<span id="adaptation-me-approaches"></span>
==== 17.5.2.2 Adaptation M&amp;E Approaches ====

<div id="h3-29-siblings" class="h3-siblings"></div>

Adaptation M&amp;E can be conducted for various purposes and in a wide variety of different contexts ranging from the local to the global level ( [[#McKenzie%20Hedger--2008|McKenzie Hedger et al., 2008]] ; [[#UNFCCC--2010a|UNFCCC, 2010a]] ; [[#Spearman--2011|Spearman and McGray, 2011]] ). The context and specific purpose of M&amp;E determine what information needs to be generated, and together with the available resources also determine the suitability of particular approaches and methods ( [[#Leiter--2016|Leiter, 2016]] ; [[#Leiter--2017|Leiter, 2017]] ). Several frameworks and approaches have been proposed for M&amp;E of adaptation and climate resilience ( [[#Bours--2014d|Bours et al., 2014d]] ; [[#Schipper--2015|Schipper and Langston, 2015]] ; [[#Adaptation%20Committee--2016|Adaptation Committee, 2016]] ; [[#ODI--2016|ODI, 2016]] ; [[#Cai--2018|Cai et al., 2018]] ; [[#Gregorowski--2018|Gregorowski et al., 2018]] ), including sector-specific ones for agriculture ( [[#FAO--2017|FAO, 2017]] ; [[#FAO--2019a|FAO, 2019a]] ; [[#FAO--2019b|FAO, 2019b]] ), health ( [[#Ebi--2018|Ebi et al., 2018]] ), ecosystem-based adaptation ( [[#Donatti--2018|Donatti et al., 2018]] ; [[#Donatti--2020|Donatti et al., 2020]] ; [[#GIZ--2020|GIZ, 2020]] ) and cities ( [[IPCC:Wg2:Chapter:Chapter-6#6.4.6|Section 6.4.6]] ).

Adaptation M&amp;E generally seeks to answer whether implementation is taking place and what effects it has (Figure 17.12). Accordingly, M&amp;E can focus on the processes, activities and outputs or on their outcomes and ultimate impacts ( [[#Harley--2008|Harley et al., 2008]] ; [[#Pringle--2011|Pringle, 2011]] ; [[#Ford--2013|Ford et al., 2013]] ). Most of the available guidance for the development of adaptation M&amp;E systems is aimed at the household, local or project level ( [[#Pringle--2011|Pringle, 2011]] ; [[#Villanueva--2012|Villanueva, 2012]] ; [[#Olivier--2013|Olivier et al., 2013]] ; [[#CARE--2014|CARE, 2014]] ; [[#BRACED--2015|BRACED, 2015]] ; [[#Leiter--2016|Leiter, 2016]] ; [[#Jones--2019b|Jones, 2019b]] ) with only limited guidance for national or cross-sectoral M&amp;E systems ( [[#Price-Kelly--2015|Price-Kelly et al., 2015]] ) or frameworks that are applicable at different scales ( [[#Brooks--2014|Brooks et al., 2014]] ). The available guidebooks take users through a series of steps which are synthesised in Figure 17.12.

<div id="_idContainer054" class="Figure"></div>

[[File:79840ac5ec788348e69a606a4c7948a1 IPCC_AR6_WGII_Figure_17_012.png]]

'''Figure 17.12 |''' '''Adaptation M&amp;E and learning as part of the adaptation process (based on Hammill et al''' '''.''' ''', 2014a; [[#Price-Kelly--2015|Price-Kelly et al., 2015]] ; [[#Leiter--2016|Leiter, 2016]] ).''' This figure shows the main steps involved in developing an adaptation M&amp;E system where the context informs the purpose of M&amp;E, which in turn determines the information needs. To achieve the M&amp;E purposes, the chosen approach and data sources need to be able to generate the needed information, which needs to be communicated in a suitable way to the target audiences.

The majority of adaptation M&amp;E efforts have so far focused on processes and outputs rather than on achieved outcomes such as climate risks, vulnerability, well-being or development ( [[#Droesch--2008|Droesch et al., 2008]] ; [[#GIZ%20and%20Adelphi--2017|GIZ and Adelphi, 2017]] ; UNDP [[#Cambodia--2014|Cambodia, 2014]] ; [[#Fawcett--2017|Fawcett et al., 2017]] ) ( ''high confidenc'' e) or use a combination thereof ( [[#Brooks--2011|Brooks et al., 2011]] ; [[#Brooks--2014|Brooks et al., 2014]] ). Newly emerging approaches include perception-based measurements and the use of data collected via mobile phones ( [[#Jones--2018|Jones et al., 2018]] ; [[#Jones--2019a|Jones, 2019a]] ), which can be collected frequently ( [[#Clare--2017a|Clare et al., 2017a]] ; [[#Knippenberg--2019|Knippenberg et al., 2019]] ; [[#Jones--2020|Jones and Ballon, 2020]] ). Such advances call into question the common reliance on ‘objective’ indicators defined from an external perspective. Instead, they suggest that multiple complementary approaches combined with higher-frequency data collection produce a more elaborate picture of the effects of adaptation and resilience responses ( [[#Jones--2019|Jones and d’Errico, 2019]] ; [[#Knippenberg--2019|Knippenberg et al., 2019]] ; [[#Singh--2019|Singh et al., 2019]] ; [[#Jones--2019a|Jones, 2019a]] ; see Cross-Chapter Box PROGRESS in this Chapter) ( ''medium confidence'' ).

Central to designing, monitoring and evaluating adaptation responses is outlining how activities are expected to lead to intended objectives, for example, via a theory of change ( [[#Bours--2014c|Bours et al., 2014c]] ; Oberlack and al., 2019). Theories of change or similar change models provide a basis to decide what to measure, but more attention needs to be paid to how theories of change are constructed and who is involved ( [[#Mason--2007|Mason and Barnes, 2007]] ; [[#Forsyth--2018|Forsyth, 2018]] ). Participatory approaches can support understanding how climate risks affect the respective population, how these risks interact with social and cultural processes, and how responses could most effectively address climate risks ( [[#Conway--2019|Conway et al., 2019]] ). Inclusive M&amp;E systems can facilitate ownership and enhance the meaningfulness and usability of the generated information ( [[#CARE--2014|CARE, 2014]] ; [[#Faulkner--2015|Faulkner et al., 2015]] ). Meaningfulness is not associated with a particular approach or method but depends on whether the chosen M&amp;E design fits the M&amp;E purpose and the information needs of the intended audience ( [[#Fisher--2015|Fisher et al., 2015]] ; [[#Leiter--2017|Leiter, 2017]] ). Effective communication of M&amp;E findings and feedback into decision-making processes is essential to achieve the respective M&amp;E purpose and facilitate learning ( [[#17.5.2.7|Section 17.5.2.7]] ).

<div id="17.5.2.3" class="h3-container"></div>

<span id="adaptation-indicators-and-indices"></span>
==== 17.5.2.3 Adaptation Indicators and Indices ====

<div id="h3-30-siblings" class="h3-siblings"></div>

A set of all-purpose and globally applicable standard indicators that could comprehensively measure adaptation does not exist ( ''high confidence'' ) ( [[#IPCC--2014|IPCC, 2014]] ; [[#Leiter--2018|Leiter and Pringle, 2018]] ). A wide variety of indicators have been used to assess adaptation and its results ( [[#CARE--2010|CARE, 2010]] ; [[#Harvey--2011|Harvey et al., 2011]] ; [[#Lamhauge--2013|Lamhauge et al., 2013]] ; [[#Brooks--2014|Brooks et al., 2014]] ; [[#Hammill--2014b|Hammill et al., 2014b]] ; [[#Mäkinen--2018|Mäkinen et al., 2018]] ; [[#HM%20Government--2019|HM Government, 2019]] ). Literature has also noted unrealistic expectations of what indicators can accomplish. For instance, decisions involving competing political interests would not be adequately informed through simple indicators; and learning requires knowledge of how and why change has happened, something that indicators often do not capture ( [[#Hinkel--2011|Hinkel, 2011]] ; [[#Bours--2014b|Bours et al., 2014b]] ). Indicators can also become misguided incentives and might steer attention away from what matters ( [[#Leiter--2018|Leiter and Pringle, 2018]] ; [[#Hallegatte--2019|Hallegatte and Engle, 2019]] ; [[#Klonschinski--2021|Klonschinski, 2021]] ). Surveys, scorecards, interviews and focus groups are alternative methods of gaining insights on adaptation progress ( [[#Brooks--2014|Brooks et al., 2014]] ; [[#Porter--2015|Porter et al., 2015]] ; [[#Das--2019|Das, 2019]] ; [[#McNamara--2020|McNamara et al., 2020]] ).

The difficulties of assessing adaptation and an emphasis on short-term results have contributed to the common practice of relying on easily quantifiable indicators rather than assessing actual changes, that is, outcomes and impacts ( [[#World%20Bank%20Independent%20Evaluation%20Group--2012|World Bank Independent Evaluation Group, 2012]] ; [[#Fisher--2015|Fisher et al., 2015]] ). In fact, indicators used by international climate funds largely measure outputs which provide little evidence of the actual effectiveness of adaptation, that is, its outcomes and impacts ( [[#GCF%20Independent%20Evaluation%20Unit--2018|GCF Independent Evaluation Unit, 2018]] ; [[#Leiter--2019|Leiter et al., 2019]] ; [[#Pauw--2020|Pauw et al., 2020]] ).

Indices, the combination of multiple indicators into a single score, are common products of risk and vulnerability assessments to compare countries or other entities, often in the form of rankings or maps ( [[#Preston--2011|Preston et al., 2011]] ; [[#Reckien--2018|Reckien, 2018]] ; de Sherbinin and et al., 2019). They can indicate changes in vulnerability over time within their respective conceptualisation of vulnerability or risk. The construction of indices, including indicator selection, their weighting, normalisation and data sources, has a profound impact on their scores ( [[#Reckien--2018|Reckien, 2018]] ). Research has consistently found large discrepancies between country vulnerability rankings ( [[#Brooks--2005|Brooks et al., 2005]] ; [[#Eriksen--2007|Eriksen and Kelly, 2007]] ; [[#Leiter--2017b|Leiter et al., 2017b]] ; [[#Visser--2020|Visser et al., 2020]] ). Reviews of vulnerability and resilience indices identified ‘substantial conceptual, methodological and empirical weaknesses’ ( [[#Füssel--2010|Füssel, 2010]] : 8) and a widespread lack of validation ( [[#Cai--2018|Cai et al., 2018]] ). Using countries as a unit of analysis also masks significant sub-national variation ( [[#Otto--2015|Otto et al., 2015]] ; [[#Mohammadpour--2019|Mohammadpour et al., 2019]] ). Individual indices therefore ‘fail to convene a robust guidance for policy makers’ ( [[#Muccione--2017|Muccione et al., 2017]] : 4) and should not present the sole basis for policy decisions ( [[#Brooks--2005|Brooks et al., 2005]] ; [[#Leiter--2018|Leiter and Pringle, 2018]] ). Due to their limitations ( [[#Singh--2017|Singh et al., 2017]] ), the OECD suggests that indices are primarily used for ‘initiating discussion and stimulating public interest’ ( [[#OECD--2008|OECD, 2008]] : 13).

<div id="17.5.2.4" class="h3-container"></div>

<span id="empirical-evidence-of-national-adaptation-me-systems"></span>
==== 17.5.2.4 Empirical Evidence of National Adaptation M&amp;E Systems ====

<div id="h3-31-siblings" class="h3-siblings"></div>

Tracking the implementation of national adaptation plans is essential for understanding their effectiveness, that is, the progress made in addressing climate risks, and can support assessing the success of adaptation and the risk of maladaptation. Over 60 countries have developed or started developing national adaptation M&amp;E systems, although less than half are yet reporting on implementation ( [[#Leiter--2021b|Leiter, 2021b]] ; Table 17.8). Country-specific adaptation M&amp;E systems vary considerably regarding their legal mandate, purpose, content, involved actors and types of reporting ( [[#Hammill--2014a|Hammill et al., 2014a]] ; EEA, 2015; [[#Leiter--2015|Leiter, 2015]] ; [[#Leiter--2017a|Leiter et al., 2017a]] ; [[#EEA--2020|EEA, 2020]] ). In most cases, they focus primarily on monitoring implementation rather than assessing outcomes, although some are linked to national climate risk or vulnerability assessments (e.g., in Germany and the UK) ( [[#EEA--2018|EEA, 2018]] ). At least 15 countries have published evaluations of national adaptation plans which help inform the development of successive adaptation plans or strategies (Table 17.8). Nevertheless, there is only limited empirical evidence of the ability of M&amp;E systems to facilitate action or increase the level of ambition of revised policies. More research is needed to determine the quality of national adaptation M&amp;E systems and how well they support the policy cycle.

Under the Paris Agreement, countries are encouraged to provide information on adaptation, including its adequacy and effectiveness ( [[#Möhner--2017|Möhner et al., 2017]] ; [[#Adaptation%20Committee--2021|Adaptation Committee, 2021]] ). National adaptation M&amp;E systems can inform both national as well as international reporting and contribute to the Global Stocktake (see Cross-Chapter Box PROGRESS in this Chapter; [[#Craft--2015|Craft and Fisher, 2015]] ; [[#Leiter--2017a|Leiter et al., 2017a]] ). Guidance for and examples of national adaptation progress assessments are provided by [[#Price-Kelly--2015|Price-Kelly et al. (2015)]] , [[#Brooks--2014|Brooks et al. (2014)]] , [[#Brooks--2019|Brooks et al. (2019)]] , EEA (2015), [[#GIZ--2017|GIZ (2017)]] , [[#Karani--2018|Karani (2018)]] and [[#van%20Rüth--2018|van Rüth and Schönthaler (2018)]] . Global assessments of adaptation progress have so far often focused on adaptation planning and, to a lesser extent, implementation, while evidence of the collective effect of adaptation globally remains limited ( ''high confidence'' ) ( [[#UNEP--2021a|UNEP, 2021a]] ; Cross-Chapter Box PROGRESS in this Chapter).

'''Table 17.8 |''' Countries in different stages of developing or operating a national adaptation M&amp;E system as of 1 August 2021 (Source: [[#Leiter--2021b|Leiter, 2021b]] ). Countries can appear twice if they have published both a progress report and an evaluation.

{| class="wikitable"
|-
! rowspan="2"|
! colspan="3"| National adaptation M&amp;E system

|-
! Stage

! Definition

! Country

|-
| rowspan="2"| Under development

| Early stage

| Tangible steps have been undertaken to develop a national adaptation M&amp;E system, for example a stocktake of relevant existing data sources and engagement with stakeholders on the objectives of the M&amp;E system

| Benin, Cook Islands, Jordan, Paraguay, Sri Lanka, Uganda

|-
| Advanced stage

| Details of the adaptation M&amp;E system have been developed, including, for instance, institutional arrangements, indicators and data sources, but it has not yet been applied

| Albania, Bulgaria, Cameroon, Canada, Colombia, Ethiopia, Fiji, Grenada, Indonesia, Moldova, Morocco, Mozambique, Nauru, Peru, Rwanda, Senegal, St. Lucia, St. Vincent and the Grenadines, Suriname, Thailand, Togo, Tonga, Turkey, Vietnam

|-
| rowspan="2"| In operation

| Adaptation progress report published

| A progress report on the implementation of the national adaptation plan or strategy has been published

| Austria, Belgium (Flanders), Brazil, Burkina Faso, Cambodia, Chile, Cyprus, France, Germany, Japan, Kenya, Kiribati, Lithuania, Mexico, the Netherlands (Delta Programme), Norway, Portugal, Slovakia, Spain, South Africa, South Korea, Switzerland, UK

|-
| Evaluation published

| An evaluation of the implementation of the national adaptation plan or strategy has been undertaken and published

| Belgium, Cambodia, Chile, Czech Republic, Finland, France, Germany, Ireland, Mexico, the Netherlands, Philippines, South Korea, Spain, Switzerland, UK

|}

<div id="17.5.2.5" class="h3-container"></div>

<span id="challenges-of-assessing-adaptation"></span>
==== 17.5.2.5 Challenges of Assessing Adaptation ====

<div id="h3-32-siblings" class="h3-siblings"></div>

To date, literature has largely focused on aspects prior to implementation such as assessments of climate vulnerability and risks or appraisals of adaptation options ( [[#Sietsma--2021|Sietsma et al., 2021]] ; Cross-Chapter Box Adaptation). To understand adaptation progress, the assessment of implemented adaptation actions and their outcomes requires more attention ( ''very high confidence'' ) (Cross-Chapter Box PROGRESS in this Chapter).

Outcomes on risk reduction are typically expressed in ways that are specific to the respective sector or context (e.g., as agricultural yields, health benefits or reduced water stress) highlighting that ‘adaptation has no common reference metrics in the same way that tonnes of GHGs or radiative forcing values are for mitigation’ ( [[#IPCC--2014|IPCC, 2014]] : 856). Assessments of adaptation progress therefore need to specify what they are measuring and how they are measuring it. The way adaptation is conceptualised, for example as a continuum between successful adaptation and maladaptation ( [[#17.1.1|Section 17.1.1]] ), and the way adaptation is framed, for example as a technical challenge or a political process ( [[#Juhola--2011|Juhola et al., 2011]] ; [[#Bassett--2013|Bassett and Fogelman, 2013]] ; [[#Eriksen--2015|Eriksen et al., 2015]] ), shape the understanding of progress and its subsequent measurement ( [[#Singh--2021|Singh et al., 2021]] ).

Furthermore, people can be differently affected even in the same location owing to, among others, differential vulnerability among the population ( [[#Reckien--2019|Reckien and Petkova, 2019]] ; [[#Thomas--2019|Thomas et al., 2019]] ). Different views and values can also affect what it means to adapt ( [[#Few--2021|Few et al., 2021]] ). Assessments of adaptation progress therefore need to be transparent and reflective about how they define and measure adaptation and account for culturally and geographic contingent concepts of what it means to adapt in light of the global diversity of livelihoods and concepts.

The lack of knowledge on adaptation progress is associated with further measurement challenges, including that avoided impacts are difficult to measure and that risk levels change over time, meaning what is effective today may not be effective in the future ( [[#Brooks--2011|Brooks et al., 2011]] ; [[#Pringle--2011|Pringle, 2011]] ; [[#Spearman--2011|Spearman and McGray, 2011]] ; [[#Villanueva--2012|Villanueva, 2012]] ; [[#Bours--2014a|Bours et al., 2014a]] ). Moreover, adaptation is embedded in complex political and social realities where power and politics shape outcomes and where simplistic views of how adaptation would take place may be ill-conceived ( [[#Nightingale--2017|Nightingale, 2017]] ; [[#Mikulewicz--2018|Mikulewicz, 2018]] ; [[#Mikulewicz--2020|Mikulewicz, 2020]] ). In practice, this means that theories of change of adaptation projects may miss important causes of risks and could subsequently lead to inaccurate assessments ( [[#Forsyth--2018|Forsyth, 2018]] ). Measuring adaptation is therefore a matter of understanding drivers of vulnerability and risk and of designing responses and M&amp;E systems accordingly ( [[#UNFCCC--2019a|UNFCCC, 2019a]] , section V).

The importance of context and the dependence on viewpoints make comparative assessments of adaptation across nations, regions or responses challenging. Comparison requires a consistent conceptualisation of adaptation, comparable units of analysis and access to relevant data sets ( [[#Ford--2015|Ford et al., 2015]] ; [[#Ford--2016|Ford and Berrang-Ford, 2016]] ). Comparative adaptation policy assessments to date often lack clarity in concepts and explanatory variables ( [[#Dupuis--2013|Dupuis and Biesbroek, 2013]] ; Biesbroek R, 2018a). The trade-off between standardisation and context specificity also complicates attempts to aggregate adaptation progress across scales to the national or global level ( [[#Leiter--2018|Leiter and Pringle, 2018]] ; Cross-Chapter Box PROGRESS in this Chapter).

<div id="17.5.2.6" class="h3-container"></div>

<span id="tracking-adaptation-finance"></span>
==== 17.5.2.6 Tracking Adaptation Finance ====

<div id="h3-33-siblings" class="h3-siblings"></div>

Adaptation finance tracking is capturing the financial flows associated with adaptation. It can indicate how much is being spent on adaptation, where funds are going to and whether spending matches allocated budgets. Thus, adaptation finance tracking can provide useful information for decision-making, but it does not provide information on the achievements resulting from the invested funds. Accordingly, it can complement, but not substitute, M&amp;E of actions and outcomes. Adaptation finance tracking can be applied domestically ( [[#Guzmán--2017|Guzmán et al., 2017]] ; [[#Guzmán--2018|Guzmán et al., 2018]] ) as well as internationally, for instance by developed countries to report on the goal to mobilise USD 100 billion yr −1 by 2020 in climate finance ( [[#UNFCCC%20SCF--2018|UNFCCC SCF, 2018]] ). Data on adaptation finance can be used alongside information on planning and implementation to assess adaptation progress ( [[#UNEP--2021a|UNEP, 2021a]] ).

Tracking adaptation finance requires defining what counts as adaptation. Different definitions can lead to large variations in the estimated amount of adaptation finance ( [[#Donner--2016|Donner et al., 2016]] ; [[#Hall--2017|Hall, 2017]] ). A further challenge is how to account for adaptation that is mainstreamed, that is, where adaptation-specific investments form only part of a larger programme or budget line, or where actions contribute to adaptation without being labelled as adaptation. These challenges limit the direct comparability between adaptation and mitigation finance ( [[#UNFCCC--2019a|UNFCCC, 2019a]] ). In fact, tracking adaptation finance differs from tracking mitigation finance since activities cannot be ''a priori'' assumed to constitute adaptation but instead have to be assessed for their linkage to climate risks in a particular context (MDBs &amp; IDFC, 2018). Methods for adaptation finance tracking continue to be further developed aiming at better comparability and completeness ( [[#Richmond--2019|Richmond and Hallmeyer, 2019]] ; [[#Richmond--2021|Richmond et al., 2021]] ).

Various methods are used to track adaptation finance, which makes comparisons between adaptation finance figures challenging ( [[#UNFCCC%20SCF--2018|UNFCCC SCF, 2018]] ; [[#Weikmans--2019|Weikmans and Roberts, 2019]] ). For example, multi-lateral development banks use a different methodology than countries do under the OECD Development Assistance Committee (DAC) (Box 17.4; [[#MDBs--2019|MDBs, 2019]] ). One of the differences concerns the treatment of partially adaptation-relevant projects, namely whether only parts or the full amount of a given project volume are counted as adaptation finance (see, e.g., [[#MDBs--2019|MDBs, 2019]] ). Under the OECD DAC methodology, countries often use a fixed percentage (e.g., 50% of the total project value), whereas the MDB methodology attempts for a project-specific estimation of the adaptation-relevant proportion (MDBs &amp; IDFC, 2018). Another aspect is whether tracking distinguishes between financial instruments, such as grants or loans. Different accounting rules can lead to large differences in reported amounts of adaptation finance and to a lack of comparability between providers ( [[#Weikmans--2019|Weikmans and Roberts, 2019]] ). Studies identified an over-reporting (i.e., counting non-adaptation-related finance) by a factor of two to three, which suggests the need for a more consistent and transparent accounting system ( [[#Weikmans--2017|Weikmans et al., 2017]] ; [[#CARE--2021|CARE, 2021]] ).

Good coverage of adaptation finance data exists around international public finance flows, predominantly official development assistance flows from OECD DAC members and from multi-lateral development banks. Less data exist around domestic public finance and private finance flows to adaptation activities, but data sources continue to be further expanded, for example through climate change expenditure tagging and city-level data ( [[#Weikmans--2017|Weikmans et al., 2017]] ; [[#UNFCCC%20SCF--2018|UNFCCC SCF, 2018]] ; [[#Richmond--2021|Richmond et al., 2021]] ). Recent estimates of adaptation finance are provided in [[#UNFCCC%20SCF--2018|UNFCCC SCF (2018)]] , [[#Macquarie--2020|Macquarie et al. (2020)]] and Cross-Chapter Box FAR in this Chapter.

<div id="17.5.2.7" class="h3-container"></div>

<span id="evaluation-and-learning"></span>
==== 17.5.2.7 Evaluation and Learning ====

<div id="h3-34-siblings" class="h3-siblings"></div>

Most adaptation M&amp;E frameworks and tools proposed to date refer to monitoring rather than evaluation ( ''high confidence'' ) ( [[#Adaptation%20Committee--2016|Adaptation Committee, 2016]] ). Evaluations are envisioned to go beyond monitoring by examining how and why results have been achieved and what could be improved ( [[#Brousselle--2018|Brousselle and Buregeya, 2018]] ; [[#Vähämäki--2019|Vähämäki and Verger, 2019]] ). Evaluations of adaptation outcomes are still rare, particularly quantitative impact evaluations ( [[#Weldegebriel--2013|Weldegebriel and Prowse, 2013]] ; [[#Das--2019|Das, 2019]] ; [[#Béné--2020|Béné et al., 2020]] ). Impact evaluations of adaptation need to address several methodological as well as practical challenges ( [[#Dinshaw--2014|Dinshaw et al., 2014]] ; [[#Fisher--2015|Fisher et al., 2015]] ; [[#Béné--2017|Béné et al., 2017]] ; [[#Puri--2020|Puri et al., 2020]] ). Different types of evaluations are appropriate for different evaluation questions ( [[#Silvestrini--2015|Silvestrini et al., 2015]] ). Evaluations of the available evidence of effective adaptation, in particular topics or sectors, have emerged more recently, for instance on mainstreaming ( [[#Runhaar--2018|Runhaar et al., 2018]] ) and agricultural climate services ( [[#Vaughan--2019a|Vaughan et al., 2019a]] ). Impact evaluations of capacity building measures are important because capacity building is assumed to lead to adaptation, but its actual effects are seldom examined ( [[#Mortreux--2017|Mortreux and Barnett, 2017]] ; Alpizar F and Meiselman, 2019). If well designed and utilised for learning, evaluations can play an important role in improving adaptation responses ( [[#Hildén--2011|Hildén, 2011]] ).

Learning requires information about how and why change occurred and what experiences have been made ( [[#Feinstein--2012|Feinstein, 2012]] ). M&amp;E is frequently associated with learning, but it is rarely made explicit how learning is supposed to take place ( [[#Armitage--2008|Armitage et al., 2008]] ; [[#Baird--2015|Baird et al., 2015]] ; [[#Borras--2015|Borras and Hølund, 2015]] ). The design of adaptation M&amp;E systems can support learning by gathering relevant information and disseminating it in a way that is accessible and effectively linked to decision-making processes ( [[#Spearman--2011|Spearman and McGray, 2011]] ; [[#Villanueva--2012|Villanueva, 2012]] ; [[#Fisher--2015|Fisher et al., 2015]] ). Options include institutionalised feedback mechanisms, peer learning and knowledge sharing events, a learning culture and ways to gather in-depth insights beyond indicators (ibid; [[#Oswald--2010|Oswald and Taylor, 2010]] ). Since AR5, adaptation programmes and funds such as the BRACED programme, the Adaptation Fund, the Climate Investment Funds and the Green Climate Fund have created knowledge-sharing units and provide resources to support learning activities ( [[#BRACED--2015|BRACED, 2015]] ; [[#Roehrer--2015|Roehrer and Kouadio, 2015]] ; [[#Adaptation%20Fund--2016|Adaptation Fund, 2016]] ; [[#Leavy--2018|Leavy et al., 2018]] ; [[#CIF--2020|CIF, 2020]] ; [[#Puri--2020|Puri et al., 2020]] ), but there is little information about their longer-term effectiveness.

<div id="cross-chapter-box-progress" class="h2-container box-container"></div>

'''Cross-Chapter Box PROGRESS | Approaches and Challenges to Assess Adaptation Progress at the Global Level'''
<div id="h2-24-siblings" class="h2-siblings"></div>

Authors: Matthias Garschagen (Germany), Timo Leiter (Germany/UK), Robbert Biesbroek (the Netherlands), Alexandre K. Magnan (France), Diana Reckien (the Netherlands/Germany), Mark New (South Africa), Lea Berrang-Ford (UK/Canada), So Min Cheong (Republic of Korea), Lisa Schipper (Sweden/USA), Robert Lempert (USA).

This Cross-Chapter Box responds to a growing demand for assessing global climate change adaptation progress, which currently faces the challenge of lacking consensus on how adaptation progress at this level can be tracked ( ''high confidence'' ). The box therefore assesses the rationale and methodological approaches for understanding adaptation progress globally across sectors and regions. It discusses strengths and weaknesses of existing approaches and sources of information, with a view towards informing the first Global Stocktake of the Paris Agreement in 2023.

'''Rationale for assessing adaptation progress at the global level'''
Global assessments of adaptation are expected to help answer key questions of climate policy ( [[#Ford--2015|Ford et al., 2015]] ; [[#UNEP--2017|UNEP, 2017]] ; [[#Adaptation%20Committee--2021|Adaptation Committee, 2021]] ) ( ''limited evidence'' , ''high agreement'' ), including: Do the observed, collective investments in adaptation lead humanity to being better able to avoid or reduce the negative consequences from climate change? Where is progress being made, and what gaps remain in the global adaptation response to climate risks?

While more than 170 countries have policies that address adaptation ( [[#Nachmany--2019b|Nachmany et al., 2019b]] ; [[#17.4.2|Section 17.4.2]] ), very few have operational frameworks to track and evaluate implementation and results ( [[#Leiter--2021a|Leiter, 2021a]] ; [[#17.5.2.4|Section 17.5.2.4]] ). In Europe, for example, most countries have adopted a national adaptation plan or strategy, but only few are tracking whether ambitions are realised ( [[#EEA--2020|EEA, 2020]] ; [[IPCC:Wg2:Chapter:Chapter-13#13.11.2|Section 13.11.2]] ). Moreover, climate risks are interconnected across scales, regions and sectors ( [[#Eakin--2009|Eakin et al., 2009]] ; [[#Challinor--2017|Challinor et al., 2017]] ; Cross-Chapter Box INTERREG in Chapter 16; [[#Hedlund--2018|Hedlund et al., 2018]] ) ( ''high confidence'' ), complicating causal attribution. National assessments of progress usually do not assess private sector and non-governmental adaptation and barely account for climate risks that transcend across borders, for example through supply chains or shared ecosystems ( [[#EEA--2018|EEA, 2018]] ; [[#Benzie--2019|Benzie and Persson, 2019]] ). In addition, adaptation action in one place or time can potentially lead to negative effects elsewhere (externalities) ( [[#Magnan--2016|Magnan and Ribera, 2016]] ; [[#Atteridge--2018|Atteridge and Remling, 2018]] ; 17.5.1). Hence, determining the collective adequacy and effectiveness (see Figure 1.7 in Chapter 1) of adaptation responses is different from simple aggregates of national and sub-national information ( [[#UNEP--2017|UNEP, 2017]] ).

Assessing global progress on adaptation is therefore of high relevance to the scientific community, policymakers and other actors. Global assessments serve different information needs than local assessments, and their meaningfulness depends on the chosen approaches and their limitations. Aggregated global assessments of adaptation progress are therefore not meant to substitute place-specific ones but to complement them to enhance the knowledge base on adaptation beyond actions by or within individual countries. The Paris Agreement stipulates a Global Stocktake to be undertaken every 5 years to assess the collective progress towards its long-term goals, including on adaptation ( [[#UNFCCC--2015|UNFCCC, 2015]] , Article 14). Yet very few scientific studies have addressed the adaptation-specific aspects of the Global Stocktake ( [[#Craft--2018|Craft and Fisher, 2018]] ; [[#Tompkins--2018|Tompkins et al., 2018]] ), and there are different views and options on how assessing global progress could take place ( ''high confidence'' ).

'''Considerations in designing global adaptation assessments'''
A number of key considerations for the design of global adaptation assessment approaches are discussed in the literature ( [[#Ford--2016|Ford and Berrang-Ford, 2016]] ; [[#Berrang-Ford--2017|Berrang-Ford et al., 2017]] ). Some of these involve trade-offs, such as global applicability versus context specificity, for which there is no simple solution. Design considerations directly depend on the objectives of global adaptation assessments, which can differ between actors and can include, for example, providing transparency, enabling accountability, understanding effectiveness or guiding policy development ( [[#17.5.2.1|Section 17.5.2.1]] ). The underlying objectives determine the suitability of approaches and the data requirements.

<div id="_idContainer056" class="Box_Header-continued"></div>

Cross-Chapter Box PROGRESS

Comparability

Global assessments may have the objective to compare adaptation over time and across sectors and regions ( [[#Ford--2015|Ford et al., 2015]] ). Such comparison requires a consistent definition of concepts ( [[#Hall--2017|Hall, 2017]] ; [[#Berrang-Ford--2019|Berrang-Ford et al., 2019]] ) and the identification of variables that are both generic enough to be applicable from one context to another and specific enough to illustrate national circumstances. To date, finding such balance has proven to be challenging ( [[#Dupuis--2013|Dupuis and Biesbroek, 2013]] ). The context dependence of adaptation outcomes poses limits for meaningful comparisons. Even people exposed to the same climate hazard may be differentially affected due to varying levels of vulnerability and resilience ( [[#Jones--2018|Jones et al., 2018]] ; [[#Thomas--2019|Thomas et al., 2019]] ), meaning that perceptions on adaptation outcomes can also differ ( [[#Jones--2019|Jones and d’Errico, 2019]] ).

Aggregation

The aggregation of data from local or regional to global scales can take different forms ranging from qualitative synthesis to quantitative aggregation, which may involve condensing a diverse set of variables into a single score ( [[#Leiter--2015|Leiter, 2015]] ; [[#17.5.2.3|Section 17.5.2.3]] ). In contrast to climate change mitigation, adaptation does not have a global reference metric against which adaptation levels could be assessed to identify progress or gaps. Experience from the Global Environment Facility, for example, has shown that mechanical aggregation based on standardised indicators fails to capture what makes the greatest difference on the ground ( [[#Chen--2014|Chen and Uitto, 2014]] ).

''Results: Input, process, output or outcome''

Adaptation progress at any spatial scale can in principle be assessed in terms of input (e.g., resources spent), process (i.e., the way adaptation is organised), output (i.e., adaptation capacities and actions) and outcomes (i.e., actual changes induced) ( [[#17.5.2.2|Section 17.5.2.2]] ). Due to the challenges inherent in measuring adaptation outcomes (Sections 16.3, 17.5.1 and 17.5.2.5), most global assessments to date have focused on outputs, such as whether countries have adopted adaptation plans ( [[#Berrang-Ford--2021|Berrang-Ford et al., 2021]] ; [[#UNEP--2021a|UNEP, 2021a]] ) ( ''high confidence'' ). Understanding the effectiveness of adaptation responses globally requires a way to conceptualise and capture outcomes, for example in terms of effective climate risk reduction, while avoiding simplifications that mask maladaptation at the global level, such as where climate risks are shifted to other countries, sectors or population groups (Cross-Chapter Box INTERREG in Chapter 16, [[#17.5.1|Section 17.5.1]] ).

Data

Global assessments typically require global availability of consistent data, be they quantitative or qualitative, which has proven to be a constraining factor for attempts to assess global adaptation ( ''high confidence'' ). For example, many countries face difficulties in reporting adequately on progress in implementing the Sendai Framework and risk-related SDGs ( [[#UNDRR--2019|UNDRR, 2019]] : vi). The availability of data also influences which variables can be eventually selected in an assessment. This limitation can affect the ability to meet the initial objectives and lead to biases in the framing and interpretation of assessment outcomes. For some variables, an alternative to relying on nationally provided data can be to develop new global data sets ( [[#Magnan--2019|Magnan and Chalastani, 2019]] ) or utilise data from Earth Observation ( [[#Andries--2018|Andries et al., 2018]] ). Adaptation is hence faced with a dilemma between globally available yet generic data and regionally or locally more detailed yet patchy data ( ''high confidence'' ).

'''Assessment of existing approaches to assess adaptation progress at the global level'''
Only few global assessments of adaptation progress across sectors have been undertaken to date ( ''high confidence'' ). They focus, for example, on whether countries have progressed their adaptation policies and actions over time ( [[#Lesnikowski--2015|Lesnikowski et al., 2015]] ; [[#Nachmany--2019b|Nachmany et al., 2019b]] ), the extent of implemented adaptation globally ( [[#Leiter--2021a|Leiter, 2021a]] ; [[#Leiter--2021b|Leiter, 2021b]] ), and the type and actors of responses ( [[#Berrang-Ford--2021|Berrang-Ford et al., 2021]] ), evidence for reduced vulnerability to climate-related hazards ( [[#Formetta--2019|Formetta and Feyen, 2019]] ; [[#UNDRR--2019|UNDRR, 2019]] ) or adaptation planning in cities across the globe ( [[#Araos--2016a|Araos et al., 2016a]] ; [[#Reckien--2018a|Reckien et al., 2018a]] ; [[#Olazabal--2019a|Olazabal et al., 2019a]] ). Each of these assessments draws on different approaches and data, and all have particular potential but also limitations (Table Cross-Chapter Box PROGRESS.1) ( ''high confidence'' ). The application of differing approaches shows that there is no single ‘best’ approach or data source to assess global progress on adaptation ( ''high confidence'' ). Existing global assessments have provided valuable insights into the extent and types of responses and their level of planning and implementation ( [[IPCC:Wg2:Chapter:Chapter-16#16.3.2.4|Section 16.3.2.4]] ). However, they do not provide comprehensive and robust answers so far on whether climate risk and vulnerability have been reduced ( [[#Berrang-Ford--2021|Berrang-Ford et al., 2021]] ) ( ''high confidence'' ). As a result, combining different approaches and integrating data on climate risk levels, policy measures, implemented actions and their effects on climate risk reduction is currently regarded as the most robust approach ( [[#Berrang-Ford--2019|Berrang-Ford et al., 2019]] ) ( ''medium evidence'' , ''high agreement'' ).

<div id="_idContainer057" class="Box_Header-continued"></div>

Cross-Chapter Box PROGRESS

'''Table Cross-Chapter Box PROGRESS.1 |''' Key approaches and data sources used for global adaptation assessments.

{| class="wikitable"
|-
! Approach/data source

! Potential added value

! Limitations

|-
| Systematic assessment of adaptation responses reported in academic literature (e.g., systematic reviews, evidence synthesis, meta-analysis, large- ''n'' comparative studies)

Examples:

Berrang-Ford, 2011, Global Adaptation Mapping Initiative, [[#Berrang-Ford--2021|Berrang-Ford et al. (2021)]]

| Provides an indication of the status, trends and gaps in adaptation responses

| Not a representative sample; biased towards responses published in scientific literature; excludes grey literature; some topics and regions not well covered; challenges in terms of comparability and aggregation; inconsistency in definitions and use of concepts; English language bias

|-
| Self-reported progress documents by countries (e.g., National Communications, Biennial Transparency Reports or domestic progress and evaluation)

Examples:

[[#Gagnon-Lebrun--2007|Gagnon-Lebrun and Agrawala (2007)]] ; [[#Lesnikowski--2015|Lesnikowski et al. (2015)]] ; [[#Lesnikowski--2016|Lesnikowski et al. (2016)]] ; [[#Leiter--2021a|Leiter (2021a)]]

| Context-specific information; official government documents enable assessments of national progress

| May only be available every few years; content is sensitive to political and policy changes; possible bias towards positive examples; challenges in terms of comparability and aggregation; inconsistency in definitions and use of concepts

|-
| Self-reported information from the private sector (e.g., information on actions taken in response to climate risks within the context of climate-related financial disclosure or in company reports).

Examples:

[[#Committee%20on%20Climate%20Change--2017|Committee on Climate Change (2017)]] ; [[#Street--2019|Street and Jude (2019)]] ; [[#UNFCCC--2021|UNFCCC (2021)]] , responses reported under Climate-related Financial Disclosure

| Provides an indication of the status, trends and gaps in adaptation responses by the private sector; complements information published in the scientific literature; could enable better understanding of supply chain risks

| Sample biased towards larger companies; challenges in terms of comparability and aggregation; potential inconsistencies in definitions and use of concepts

|-
| Project documents and evaluations (e.g., from climate funds or implementing organisations)

Examples:

[[#Leiter--2021b|Leiter (2021b)]] ; [[#Eriksen--2021|Eriksen et al. (2021)]]

| Detailed information on context, intended or achieved results and activities

| Actual implementation can differ from what was proposed; fragmented picture of local/regional actions; results may be challenging to aggregate; challenges in terms of comparability and aggregation; inconsistency in definitions and use of concepts

|-
| Existing global data sets of mostly quantitative indicators

Examples:

United Nations ( [[#UN--2016a|UN, 2016a]] ; [[#UN--2016b|UN, 2016b]] ; [[#UN--2019|UN, 2019]] ; [[#UNDRR--2019|UNDRR, 2019]] )

| Comparable information based on globally defined indicators

| Global data availability constrains indicator choice; reporting burden for new indicators; trade-off between global applicability and national circumstances; usefulness and meaningfulness of global indicators is contested ( [[#Leiter--2018|Leiter and Pringle, 2018]] ; [[#Lyytimäki--2020|Lyytimäki et al., 2020]] ; [[#Pauw--2020|Pauw et al., 2020]] ).

|-
| Tracking financial flows

Examples:

[[#CPI--2019|CPI (2019)]] , OECD (2018a), [[#MDBs--2019|MDBs (2019)]]

| Comparable data on financial flows directed at adaptation; standardised methodologies (e.g., OECD RIO markers; climate finance tracking method of multi-lateral development banks; [[#17.5.2.6|Section 17.5.2.6]] ; Cross-Chapter Box FINANCE in this Chapter)

| No information about implementation of measures and their adaptation effect (Eriksen et al, 2021), i.e., it tracks inputs, not outputs or outcomes; inconsistency in what gets counted as adaptation finance ( [[#Donner--2016|Donner et al., 2016]] ; [[#Doshi--2020|Doshi and Garschagen, 2020]] ); evidence of over-reporting ( [[#Michaelowa--2011|Michaelowa and Michaelowa, 2011]] ; [[#Weikmans--2017|Weikmans et al., 2017]] )

|}

'''Conclusion—Combining approaches for assessing adaptation progress at the global level'''
Understanding to what extent the world is on track to adapt to climate change impacts and risks globally is a pressing question in scientific and policy communities, especially in light of the Global Stocktake under the Paris Agreement. Important considerations for a robust assessment framework (e.g., consistency), as well as the associated scientific challenges (e.g., aggregation, externalities, breadth versus depth of data) and the role of underlying objectives (e.g., on the contested issue of comparability) are increasingly understood ( ''high confidence'' ). There is also a growing and diverse body of information on adaptation progress, although most assessments of global progress undertaken to date focus on processes and outputs (e.g., policies and plans) rather than outcomes (i.e., risk reduction). A variety of approaches and data sources are employed, such as systematic reviews of observed adaptation, formal communications by Parties to the UNFCCC, and project documents to international funding agencies. Novel approaches, including big data tools (Ford et al., 2016; Biesbroek et al., 2020), are also being explored but still have to prove their practical value. Each approach and source of information can contribute additional knowledge, but also demonstrates limitations, so that there is no single ‘best’ approach ( ''high confidence'' ). Yet, to date, the international community has not sufficiently explored the relative strengths and weaknesses of different approaches and their applicability and, therefore, their potential synergies in complementing each other. Triangulated assessments have only rarely been applied ( ''high confidence'' ) due to multiple conceptual and methodological challenges, despite their potential for increasing the robustness of knowledge. One overarching conclusion of this Cross-Chapter Box therefore is that the combination of different approaches will provide a more comprehensive picture of global adaptation progress than is currently available from individual approaches ( ''limited evidence'' , ''high agreement'' ).

<div id="_idContainer058" class="Box_Header-continued"></div>

Cross-Chapter Box PROGRESS

<div id="box-17.4" class="h2-container box-container"></div>

'''Box 17.4 | The Rio Markers Methodology to Track Climate Finance'''
<div id="h2-25-siblings" class="h2-siblings"></div>

The OECD Development Assistance Committee (DAC) introduced a methodology to track the amount of bilateral official development assistance (ODA) that is targeting climate change mitigation and/or adaptation. It distinguishes whether activities have adaptation as a ‘principal’ objective (score ‘2’), as a ‘significant’ objective (score ‘1’) or as not targeting it (score ‘0’) ( [[#OECD--2016|OECD, 2016]] ). The associated project value is counted in full, in part, or not counted as adaptation finance, respectively. Countries count the volume of partial adaptation projects (score ‘1’) to a different extent, which limits comparability and can lead to over-reporting ( [[#OECD--2019|OECD, 2019]] ). The first data on this ‘adaptation marker’ became available in 2012 for the financial flows of 2010. It forms the basis for developed countries’ reporting to the UNFCCC Secretariat on their financial commitments towards developing countries ( [[#Weikmans--2019|Weikmans and Roberts, 2019]] ).

While a guidebook with requirements for adaptation as a principle or significant objective has been developed ( [[#OECD--2016|OECD, 2016]] ), several studies have shown that OECD DAC donors tend to overestimate the number of activities in their portfolio that genuinely have adaptation objectives ( [[#Michaelowa--2011|Michaelowa and Michaelowa, 2011]] ; [[#Weikmans--2017|Weikmans et al., 2017]] ; [[#CARE--2021|CARE, 2021]] ). Hence, the amount of adaptation finance from public sources may be lower than reported. The use of just three categories leads to a broad range of the extent of adaptation being concentrated in the middle category (‘significant objective’). Accordingly, the category ‘principle objective adaptation’ provides a more robust predictor of the relevance of an activity to adaptation ( [[#Donner--2016|Donner et al., 2016]] ).

<div id="17.6" class="h1-container"></div>

<span id="managing-and-adapting-to-climate-risks-for-climate-resilient-development"></span>