Editing IPCC:AR6/WGII/Chapter-17 (section)

==== 17.5.2.7 Evaluation and Learning ====

<div id="h3-34-siblings" class="h3-siblings"></div>

Most adaptation M&amp;E frameworks and tools proposed to date refer to monitoring rather than evaluation ( ''high confidence'' ) ( [[#Adaptation%20Committee--2016|Adaptation Committee, 2016]] ). Evaluations are envisioned to go beyond monitoring by examining how and why results have been achieved and what could be improved ( [[#Brousselle--2018|Brousselle and Buregeya, 2018]] ; [[#Vähämäki--2019|Vähämäki and Verger, 2019]] ). Evaluations of adaptation outcomes are still rare, particularly quantitative impact evaluations ( [[#Weldegebriel--2013|Weldegebriel and Prowse, 2013]] ; [[#Das--2019|Das, 2019]] ; [[#Béné--2020|Béné et al., 2020]] ). Impact evaluations of adaptation need to address several methodological as well as practical challenges ( [[#Dinshaw--2014|Dinshaw et al., 2014]] ; [[#Fisher--2015|Fisher et al., 2015]] ; [[#Béné--2017|Béné et al., 2017]] ; [[#Puri--2020|Puri et al., 2020]] ). Different types of evaluations are appropriate for different evaluation questions ( [[#Silvestrini--2015|Silvestrini et al., 2015]] ). Evaluations of the available evidence of effective adaptation, in particular topics or sectors, have emerged more recently, for instance on mainstreaming ( [[#Runhaar--2018|Runhaar et al., 2018]] ) and agricultural climate services ( [[#Vaughan--2019a|Vaughan et al., 2019a]] ). Impact evaluations of capacity building measures are important because capacity building is assumed to lead to adaptation, but its actual effects are seldom examined ( [[#Mortreux--2017|Mortreux and Barnett, 2017]] ; Alpizar F and Meiselman, 2019). If well designed and utilised for learning, evaluations can play an important role in improving adaptation responses ( [[#Hildén--2011|Hildén, 2011]] ).

Learning requires information about how and why change occurred and what experiences have been made ( [[#Feinstein--2012|Feinstein, 2012]] ). M&amp;E is frequently associated with learning, but it is rarely made explicit how learning is supposed to take place ( [[#Armitage--2008|Armitage et al., 2008]] ; [[#Baird--2015|Baird et al., 2015]] ; [[#Borras--2015|Borras and Hølund, 2015]] ). The design of adaptation M&amp;E systems can support learning by gathering relevant information and disseminating it in a way that is accessible and effectively linked to decision-making processes ( [[#Spearman--2011|Spearman and McGray, 2011]] ; [[#Villanueva--2012|Villanueva, 2012]] ; [[#Fisher--2015|Fisher et al., 2015]] ). Options include institutionalised feedback mechanisms, peer learning and knowledge sharing events, a learning culture and ways to gather in-depth insights beyond indicators (ibid; [[#Oswald--2010|Oswald and Taylor, 2010]] ). Since AR5, adaptation programmes and funds such as the BRACED programme, the Adaptation Fund, the Climate Investment Funds and the Green Climate Fund have created knowledge-sharing units and provide resources to support learning activities ( [[#BRACED--2015|BRACED, 2015]] ; [[#Roehrer--2015|Roehrer and Kouadio, 2015]] ; [[#Adaptation%20Fund--2016|Adaptation Fund, 2016]] ; [[#Leavy--2018|Leavy et al., 2018]] ; [[#CIF--2020|CIF, 2020]] ; [[#Puri--2020|Puri et al., 2020]] ), but there is little information about their longer-term effectiveness.

<div id="cross-chapter-box-progress" class="h2-container box-container"></div>

'''Cross-Chapter Box PROGRESS | Approaches and Challenges to Assess Adaptation Progress at the Global Level'''
<div id="h2-24-siblings" class="h2-siblings"></div>

Authors: Matthias Garschagen (Germany), Timo Leiter (Germany/UK), Robbert Biesbroek (the Netherlands), Alexandre K. Magnan (France), Diana Reckien (the Netherlands/Germany), Mark New (South Africa), Lea Berrang-Ford (UK/Canada), So Min Cheong (Republic of Korea), Lisa Schipper (Sweden/USA), Robert Lempert (USA).

This Cross-Chapter Box responds to a growing demand for assessing global climate change adaptation progress, which currently faces the challenge of lacking consensus on how adaptation progress at this level can be tracked ( ''high confidence'' ). The box therefore assesses the rationale and methodological approaches for understanding adaptation progress globally across sectors and regions. It discusses strengths and weaknesses of existing approaches and sources of information, with a view towards informing the first Global Stocktake of the Paris Agreement in 2023.

'''Rationale for assessing adaptation progress at the global level'''
Global assessments of adaptation are expected to help answer key questions of climate policy ( [[#Ford--2015|Ford et al., 2015]] ; [[#UNEP--2017|UNEP, 2017]] ; [[#Adaptation%20Committee--2021|Adaptation Committee, 2021]] ) ( ''limited evidence'' , ''high agreement'' ), including: Do the observed, collective investments in adaptation lead humanity to being better able to avoid or reduce the negative consequences from climate change? Where is progress being made, and what gaps remain in the global adaptation response to climate risks?

While more than 170 countries have policies that address adaptation ( [[#Nachmany--2019b|Nachmany et al., 2019b]] ; [[#17.4.2|Section 17.4.2]] ), very few have operational frameworks to track and evaluate implementation and results ( [[#Leiter--2021a|Leiter, 2021a]] ; [[#17.5.2.4|Section 17.5.2.4]] ). In Europe, for example, most countries have adopted a national adaptation plan or strategy, but only few are tracking whether ambitions are realised ( [[#EEA--2020|EEA, 2020]] ; [[IPCC:Wg2:Chapter:Chapter-13#13.11.2|Section 13.11.2]] ). Moreover, climate risks are interconnected across scales, regions and sectors ( [[#Eakin--2009|Eakin et al., 2009]] ; [[#Challinor--2017|Challinor et al., 2017]] ; Cross-Chapter Box INTERREG in Chapter 16; [[#Hedlund--2018|Hedlund et al., 2018]] ) ( ''high confidence'' ), complicating causal attribution. National assessments of progress usually do not assess private sector and non-governmental adaptation and barely account for climate risks that transcend across borders, for example through supply chains or shared ecosystems ( [[#EEA--2018|EEA, 2018]] ; [[#Benzie--2019|Benzie and Persson, 2019]] ). In addition, adaptation action in one place or time can potentially lead to negative effects elsewhere (externalities) ( [[#Magnan--2016|Magnan and Ribera, 2016]] ; [[#Atteridge--2018|Atteridge and Remling, 2018]] ; 17.5.1). Hence, determining the collective adequacy and effectiveness (see Figure 1.7 in Chapter 1) of adaptation responses is different from simple aggregates of national and sub-national information ( [[#UNEP--2017|UNEP, 2017]] ).

Assessing global progress on adaptation is therefore of high relevance to the scientific community, policymakers and other actors. Global assessments serve different information needs than local assessments, and their meaningfulness depends on the chosen approaches and their limitations. Aggregated global assessments of adaptation progress are therefore not meant to substitute place-specific ones but to complement them to enhance the knowledge base on adaptation beyond actions by or within individual countries. The Paris Agreement stipulates a Global Stocktake to be undertaken every 5 years to assess the collective progress towards its long-term goals, including on adaptation ( [[#UNFCCC--2015|UNFCCC, 2015]] , Article 14). Yet very few scientific studies have addressed the adaptation-specific aspects of the Global Stocktake ( [[#Craft--2018|Craft and Fisher, 2018]] ; [[#Tompkins--2018|Tompkins et al., 2018]] ), and there are different views and options on how assessing global progress could take place ( ''high confidence'' ).

'''Considerations in designing global adaptation assessments'''
A number of key considerations for the design of global adaptation assessment approaches are discussed in the literature ( [[#Ford--2016|Ford and Berrang-Ford, 2016]] ; [[#Berrang-Ford--2017|Berrang-Ford et al., 2017]] ). Some of these involve trade-offs, such as global applicability versus context specificity, for which there is no simple solution. Design considerations directly depend on the objectives of global adaptation assessments, which can differ between actors and can include, for example, providing transparency, enabling accountability, understanding effectiveness or guiding policy development ( [[#17.5.2.1|Section 17.5.2.1]] ). The underlying objectives determine the suitability of approaches and the data requirements.

<div id="_idContainer056" class="Box_Header-continued"></div>

Cross-Chapter Box PROGRESS

Comparability

Global assessments may have the objective to compare adaptation over time and across sectors and regions ( [[#Ford--2015|Ford et al., 2015]] ). Such comparison requires a consistent definition of concepts ( [[#Hall--2017|Hall, 2017]] ; [[#Berrang-Ford--2019|Berrang-Ford et al., 2019]] ) and the identification of variables that are both generic enough to be applicable from one context to another and specific enough to illustrate national circumstances. To date, finding such balance has proven to be challenging ( [[#Dupuis--2013|Dupuis and Biesbroek, 2013]] ). The context dependence of adaptation outcomes poses limits for meaningful comparisons. Even people exposed to the same climate hazard may be differentially affected due to varying levels of vulnerability and resilience ( [[#Jones--2018|Jones et al., 2018]] ; [[#Thomas--2019|Thomas et al., 2019]] ), meaning that perceptions on adaptation outcomes can also differ ( [[#Jones--2019|Jones and d’Errico, 2019]] ).

Aggregation

The aggregation of data from local or regional to global scales can take different forms ranging from qualitative synthesis to quantitative aggregation, which may involve condensing a diverse set of variables into a single score ( [[#Leiter--2015|Leiter, 2015]] ; [[#17.5.2.3|Section 17.5.2.3]] ). In contrast to climate change mitigation, adaptation does not have a global reference metric against which adaptation levels could be assessed to identify progress or gaps. Experience from the Global Environment Facility, for example, has shown that mechanical aggregation based on standardised indicators fails to capture what makes the greatest difference on the ground ( [[#Chen--2014|Chen and Uitto, 2014]] ).

''Results: Input, process, output or outcome''

Adaptation progress at any spatial scale can in principle be assessed in terms of input (e.g., resources spent), process (i.e., the way adaptation is organised), output (i.e., adaptation capacities and actions) and outcomes (i.e., actual changes induced) ( [[#17.5.2.2|Section 17.5.2.2]] ). Due to the challenges inherent in measuring adaptation outcomes (Sections 16.3, 17.5.1 and 17.5.2.5), most global assessments to date have focused on outputs, such as whether countries have adopted adaptation plans ( [[#Berrang-Ford--2021|Berrang-Ford et al., 2021]] ; [[#UNEP--2021a|UNEP, 2021a]] ) ( ''high confidence'' ). Understanding the effectiveness of adaptation responses globally requires a way to conceptualise and capture outcomes, for example in terms of effective climate risk reduction, while avoiding simplifications that mask maladaptation at the global level, such as where climate risks are shifted to other countries, sectors or population groups (Cross-Chapter Box INTERREG in Chapter 16, [[#17.5.1|Section 17.5.1]] ).

Data

Global assessments typically require global availability of consistent data, be they quantitative or qualitative, which has proven to be a constraining factor for attempts to assess global adaptation ( ''high confidence'' ). For example, many countries face difficulties in reporting adequately on progress in implementing the Sendai Framework and risk-related SDGs ( [[#UNDRR--2019|UNDRR, 2019]] : vi). The availability of data also influences which variables can be eventually selected in an assessment. This limitation can affect the ability to meet the initial objectives and lead to biases in the framing and interpretation of assessment outcomes. For some variables, an alternative to relying on nationally provided data can be to develop new global data sets ( [[#Magnan--2019|Magnan and Chalastani, 2019]] ) or utilise data from Earth Observation ( [[#Andries--2018|Andries et al., 2018]] ). Adaptation is hence faced with a dilemma between globally available yet generic data and regionally or locally more detailed yet patchy data ( ''high confidence'' ).

'''Assessment of existing approaches to assess adaptation progress at the global level'''
Only few global assessments of adaptation progress across sectors have been undertaken to date ( ''high confidence'' ). They focus, for example, on whether countries have progressed their adaptation policies and actions over time ( [[#Lesnikowski--2015|Lesnikowski et al., 2015]] ; [[#Nachmany--2019b|Nachmany et al., 2019b]] ), the extent of implemented adaptation globally ( [[#Leiter--2021a|Leiter, 2021a]] ; [[#Leiter--2021b|Leiter, 2021b]] ), and the type and actors of responses ( [[#Berrang-Ford--2021|Berrang-Ford et al., 2021]] ), evidence for reduced vulnerability to climate-related hazards ( [[#Formetta--2019|Formetta and Feyen, 2019]] ; [[#UNDRR--2019|UNDRR, 2019]] ) or adaptation planning in cities across the globe ( [[#Araos--2016a|Araos et al., 2016a]] ; [[#Reckien--2018a|Reckien et al., 2018a]] ; [[#Olazabal--2019a|Olazabal et al., 2019a]] ). Each of these assessments draws on different approaches and data, and all have particular potential but also limitations (Table Cross-Chapter Box PROGRESS.1) ( ''high confidence'' ). The application of differing approaches shows that there is no single ‘best’ approach or data source to assess global progress on adaptation ( ''high confidence'' ). Existing global assessments have provided valuable insights into the extent and types of responses and their level of planning and implementation ( [[IPCC:Wg2:Chapter:Chapter-16#16.3.2.4|Section 16.3.2.4]] ). However, they do not provide comprehensive and robust answers so far on whether climate risk and vulnerability have been reduced ( [[#Berrang-Ford--2021|Berrang-Ford et al., 2021]] ) ( ''high confidence'' ). As a result, combining different approaches and integrating data on climate risk levels, policy measures, implemented actions and their effects on climate risk reduction is currently regarded as the most robust approach ( [[#Berrang-Ford--2019|Berrang-Ford et al., 2019]] ) ( ''medium evidence'' , ''high agreement'' ).

<div id="_idContainer057" class="Box_Header-continued"></div>

Cross-Chapter Box PROGRESS

'''Table Cross-Chapter Box PROGRESS.1 |''' Key approaches and data sources used for global adaptation assessments.

{| class="wikitable"
|-
! Approach/data source

! Potential added value

! Limitations

|-
| Systematic assessment of adaptation responses reported in academic literature (e.g., systematic reviews, evidence synthesis, meta-analysis, large- ''n'' comparative studies)

Examples:

Berrang-Ford, 2011, Global Adaptation Mapping Initiative, [[#Berrang-Ford--2021|Berrang-Ford et al. (2021)]]

| Provides an indication of the status, trends and gaps in adaptation responses

| Not a representative sample; biased towards responses published in scientific literature; excludes grey literature; some topics and regions not well covered; challenges in terms of comparability and aggregation; inconsistency in definitions and use of concepts; English language bias

|-
| Self-reported progress documents by countries (e.g., National Communications, Biennial Transparency Reports or domestic progress and evaluation)

Examples:

[[#Gagnon-Lebrun--2007|Gagnon-Lebrun and Agrawala (2007)]] ; [[#Lesnikowski--2015|Lesnikowski et al. (2015)]] ; [[#Lesnikowski--2016|Lesnikowski et al. (2016)]] ; [[#Leiter--2021a|Leiter (2021a)]]

| Context-specific information; official government documents enable assessments of national progress

| May only be available every few years; content is sensitive to political and policy changes; possible bias towards positive examples; challenges in terms of comparability and aggregation; inconsistency in definitions and use of concepts

|-
| Self-reported information from the private sector (e.g., information on actions taken in response to climate risks within the context of climate-related financial disclosure or in company reports).

Examples:

[[#Committee%20on%20Climate%20Change--2017|Committee on Climate Change (2017)]] ; [[#Street--2019|Street and Jude (2019)]] ; [[#UNFCCC--2021|UNFCCC (2021)]] , responses reported under Climate-related Financial Disclosure

| Provides an indication of the status, trends and gaps in adaptation responses by the private sector; complements information published in the scientific literature; could enable better understanding of supply chain risks

| Sample biased towards larger companies; challenges in terms of comparability and aggregation; potential inconsistencies in definitions and use of concepts

|-
| Project documents and evaluations (e.g., from climate funds or implementing organisations)

Examples:

[[#Leiter--2021b|Leiter (2021b)]] ; [[#Eriksen--2021|Eriksen et al. (2021)]]

| Detailed information on context, intended or achieved results and activities

| Actual implementation can differ from what was proposed; fragmented picture of local/regional actions; results may be challenging to aggregate; challenges in terms of comparability and aggregation; inconsistency in definitions and use of concepts

|-
| Existing global data sets of mostly quantitative indicators

Examples:

United Nations ( [[#UN--2016a|UN, 2016a]] ; [[#UN--2016b|UN, 2016b]] ; [[#UN--2019|UN, 2019]] ; [[#UNDRR--2019|UNDRR, 2019]] )

| Comparable information based on globally defined indicators

| Global data availability constrains indicator choice; reporting burden for new indicators; trade-off between global applicability and national circumstances; usefulness and meaningfulness of global indicators is contested ( [[#Leiter--2018|Leiter and Pringle, 2018]] ; [[#Lyytimäki--2020|Lyytimäki et al., 2020]] ; [[#Pauw--2020|Pauw et al., 2020]] ).

|-
| Tracking financial flows

Examples:

[[#CPI--2019|CPI (2019)]] , OECD (2018a), [[#MDBs--2019|MDBs (2019)]]

| Comparable data on financial flows directed at adaptation; standardised methodologies (e.g., OECD RIO markers; climate finance tracking method of multi-lateral development banks; [[#17.5.2.6|Section 17.5.2.6]] ; Cross-Chapter Box FINANCE in this Chapter)

| No information about implementation of measures and their adaptation effect (Eriksen et al, 2021), i.e., it tracks inputs, not outputs or outcomes; inconsistency in what gets counted as adaptation finance ( [[#Donner--2016|Donner et al., 2016]] ; [[#Doshi--2020|Doshi and Garschagen, 2020]] ); evidence of over-reporting ( [[#Michaelowa--2011|Michaelowa and Michaelowa, 2011]] ; [[#Weikmans--2017|Weikmans et al., 2017]] )

|}

'''Conclusion—Combining approaches for assessing adaptation progress at the global level'''
Understanding to what extent the world is on track to adapt to climate change impacts and risks globally is a pressing question in scientific and policy communities, especially in light of the Global Stocktake under the Paris Agreement. Important considerations for a robust assessment framework (e.g., consistency), as well as the associated scientific challenges (e.g., aggregation, externalities, breadth versus depth of data) and the role of underlying objectives (e.g., on the contested issue of comparability) are increasingly understood ( ''high confidence'' ). There is also a growing and diverse body of information on adaptation progress, although most assessments of global progress undertaken to date focus on processes and outputs (e.g., policies and plans) rather than outcomes (i.e., risk reduction). A variety of approaches and data sources are employed, such as systematic reviews of observed adaptation, formal communications by Parties to the UNFCCC, and project documents to international funding agencies. Novel approaches, including big data tools (Ford et al., 2016; Biesbroek et al., 2020), are also being explored but still have to prove their practical value. Each approach and source of information can contribute additional knowledge, but also demonstrates limitations, so that there is no single ‘best’ approach ( ''high confidence'' ). Yet, to date, the international community has not sufficiently explored the relative strengths and weaknesses of different approaches and their applicability and, therefore, their potential synergies in complementing each other. Triangulated assessments have only rarely been applied ( ''high confidence'' ) due to multiple conceptual and methodological challenges, despite their potential for increasing the robustness of knowledge. One overarching conclusion of this Cross-Chapter Box therefore is that the combination of different approaches will provide a more comprehensive picture of global adaptation progress than is currently available from individual approaches ( ''limited evidence'' , ''high agreement'' ).

<div id="_idContainer058" class="Box_Header-continued"></div>

Cross-Chapter Box PROGRESS

<div id="box-17.4" class="h2-container box-container"></div>

'''Box 17.4 | The Rio Markers Methodology to Track Climate Finance'''
<div id="h2-25-siblings" class="h2-siblings"></div>

The OECD Development Assistance Committee (DAC) introduced a methodology to track the amount of bilateral official development assistance (ODA) that is targeting climate change mitigation and/or adaptation. It distinguishes whether activities have adaptation as a ‘principal’ objective (score ‘2’), as a ‘significant’ objective (score ‘1’) or as not targeting it (score ‘0’) ( [[#OECD--2016|OECD, 2016]] ). The associated project value is counted in full, in part, or not counted as adaptation finance, respectively. Countries count the volume of partial adaptation projects (score ‘1’) to a different extent, which limits comparability and can lead to over-reporting ( [[#OECD--2019|OECD, 2019]] ). The first data on this ‘adaptation marker’ became available in 2012 for the financial flows of 2010. It forms the basis for developed countries’ reporting to the UNFCCC Secretariat on their financial commitments towards developing countries ( [[#Weikmans--2019|Weikmans and Roberts, 2019]] ).

While a guidebook with requirements for adaptation as a principle or significant objective has been developed ( [[#OECD--2016|OECD, 2016]] ), several studies have shown that OECD DAC donors tend to overestimate the number of activities in their portfolio that genuinely have adaptation objectives ( [[#Michaelowa--2011|Michaelowa and Michaelowa, 2011]] ; [[#Weikmans--2017|Weikmans et al., 2017]] ; [[#CARE--2021|CARE, 2021]] ). Hence, the amount of adaptation finance from public sources may be lower than reported. The use of just three categories leads to a broad range of the extent of adaptation being concentrated in the middle category (‘significant objective’). Accordingly, the category ‘principle objective adaptation’ provides a more robust predictor of the relevance of an activity to adaptation ( [[#Donner--2016|Donner et al., 2016]] ).

<div id="17.6" class="h1-container"></div>

<span id="managing-and-adapting-to-climate-risks-for-climate-resilient-development"></span>