Editing IPCC:AR6/WGII/Chapter-17 (section)

=== 17.3.1 Decision-Analytic Methods and Approaches ===

<div id="h2-6-siblings" class="h2-siblings"></div>

Different classes of decision-analytic methods have been variously presented in IPCC reports since AR4 but without a summary assessment of their capacity to deal with different contexts of the decision maker. ‘Communities-of-practice’ are developing tool boxes to support analysing and making of decisions generally ( [[#French--2020|French, 2020]] ). These communities of decision analysts can act like broad-based statisticians to advise on matching methods to the climate risk and its context, before individual decision specialists are consulted. Some scientific literature is presenting guides for choosing different methods, tools and approaches ( [[#Shi--2019|Shi et al., 2019]] ). This sub-subsection provides a summary guide for policy analysts and decision makers to help identify the classes of decision-analytic methods that may be suitable for their context for managing climate risks. It focuses on decision-analytic methods, noting that decision-support tools will underpin many of these methods by organising information ( [[#Bourne--2016|Bourne et al., 2016]] ; [[#Papathanasiou--2016|Papathanasiou et al., 2016]] ; [[#Ceccato--2018|Ceccato et al., 2018]] ; [[#Haße--2018|Haße and Kind, 2018]] ) or support modelling ( [[#Papathanasiou--2016|Papathanasiou et al., 2016]] ; [[#Kwakkel--2017|Kwakkel, 2017]] ; [[#Gardiner--2018|Gardiner et al., 2018]] ), sometimes with a particular decision-analytic process in mind ( [[#Hadka--2015|Hadka et al., 2015]] ; [[#Torresan--2016|Torresan et al., 2016]] ; [[#Tonmoy--2018|Tonmoy et al., 2018]] ).

<div id="17.3.1.1" class="h3-container"></div>

<span id="factors-to-consider-in-selecting-methods-to-facilitate-decision-making"></span>
==== 17.3.1.1 Factors to Consider in Selecting Methods to Facilitate Decision-Making ====

<div id="h3-17-siblings" class="h3-siblings"></div>

The choice of methods and approaches to decision-making for climate risks (next section) will depend on (i) the cognitive needs of the deliberations, otherwise considered to be the phase in developing a decision, (ii) the types of models and modelling available to facilitate the deliberations, (iii) the degree of uncertainty surrounding the choices and (iv) the context of a choice ( ''high confidence'' ) ( [[#Richards--2013|Richards et al., 2013]] ; [[#Jones--2014|Jones et al., 2014]] ; [[#Shi--2019|Shi et al., 2019]] ; [[#French--2021|French, 2021]] ).

<div id="17.3.1.1.1" class="h4-container"></div>

<span id="cognitive-phases-of-decision-making"></span>
===== 17.3.1.1.1 Cognitive phases of decision-making =====

<div id="h4-2-siblings" class="h4-siblings"></div>

The decision process often involves overlapping and iterative development of the components leading towards a decision, resulting in the blurring of stages but involving different phases of cognitive activity (Figure 17.7; [[#Holtzman--1989|Holtzman, 1989]] ; [[#French--2015|French, 2015]] ; [[#French--2020|French, 2020]] ). Framing the problem ( [[#Orlove--2020|Orlove et al., 2020]] ), by modelling its relationships with the human and natural systems and eliciting objectives, values and scope of the problem from stakeholders, is a precursor to analyses of options but may be returned to whenever a phase of ‘ ''sense-making and modelling'' ’ is required ( ''high confidence'' ) ( [[#Ackermann--2012|Ackermann, 2012]] ; [[#Keeney--2012|Keeney, 2012]] ; [[#Slotte--2014|Slotte and Hämäläinen, 2014]] ; [[#Abbas--2015|Abbas and Howard, 2015]] ; [[#Marttunen--2017|Marttunen et al., 2017]] ; [[#Korhonen--2020|Korhonen and Wallenius, 2020]] ; [[#French--2021|French, 2021]] ).

<div id="_idContainer025" class="Figure"></div>

[[File:34622b1f05ef57da670dfdbd919a03b1 IPCC_AR6_WGII_Figure_17_007.png]]

'''Figure 17.7 |''' '''Relationships between different processes of decision-making to manage climate-related risks in the real world, noting that, when appropriate, some aspects may only require experience to be re-used.''' (1) Formulation of risks of concern and accompanying policies and objectives for managing those risks, forming prescriptive models for the decision maker. (2) Knowledge, understanding and observations of the real world are used to assess past and current impacts and future risks using descriptive models, based on the perspectives and prescriptive models arising from (1). If not well formulated from other experience, processes in (1) and (2) interact to make sense of the world and what needs to be done. In iterative management, (1) and (2) also form the basis for monitoring, reviewing and evaluating effectiveness of adaptations. (3) Use of decision-support and decision-analytic tools to appraise costs and benefits of different options for ameliorating future risks. The double-headed arrow indicates where two-way interactions occur between different activities (likely to be iterative, feedback and nonlinear processes); modelling and assessments are repeated and revised in tandem with the planning and evaluation of options, based on interactions with the policymakers and stakeholders. (4) The decision maker, which may be a group of people, interacts with the evaluation of options (two-way interaction) and interprets the efficacy of the options and the implications for the real world, ultimately choosing one or more actions to satisfy the policy objectives to manage the risks. ( '''5''' ) Implementation of the actions in the real world, which may be once-only actions or instigation of a feedback management system that enables ongoing adjustments to meet objectives.

The cognitive phase of ‘ ''analysing and exploring'' ’ uses models and existing data and/or knowledge services as available to explore the relevance/efficacy of adaptations to ameliorate risk or to meet other adaptation objectives, as well as possible flow-on effects of those actions ( [[#17.3.1|Section 17.3.1.4]] ). Sensitivity and robustness analyses can be useful if conditions are favourable to supplement the decision analysis, setting bounds on some of the residual uncertainty ( ''high confidence'' ) ( [[#Borgonovo--2016|Borgonovo and Plischke, 2016]] ; [[#Ferretti--2016|Ferretti et al., 2016]] ). Validation of models and verification of data ( [[#Tittensor--2018|Tittensor et al., 2018]] ) are becoming highlighted as important steps in this phase or in the sense-making phase, particularly in their capacity to understand and test decision makers and stakeholders’ perceptions ( ''medium confidence'' ). Randomisation methods, Bayesian methods, interval methods, multi-criteria decision analysis (MCDA), decision-making under deep uncertainty (DMDU) and economic and financial approaches (e.g., Real Options Analysis) are tools of choice in this phase ( ''high confidence'' ) (Table 17.4) ( [[#Abbas--2015|Abbas and Howard, 2015]] ; [[#Bendoly--2016|Bendoly and Clark, 2016]] ; [[#Borgonovo--2016|Borgonovo and Plischke, 2016]] ; [[#Iooss--2017|Iooss and Saltelli, 2017]] ; [[#Korhonen--2020|Korhonen and Wallenius, 2020]] ; [[#Saltelli--2020|Saltelli et al., 2020]] ). Decision-support tools in the provision of data and/or modelling methods are regularly used in this and the sense-making phase ( ''high confidence'' ) ( [[#17.3.1.2|Section 17.3.1.2]] ).

The phase of interpreting the analyses to make decisions on climate adaptation followed by implementation is the least described in the literature (Figure 17.8). Decision process management tools and methods for communicating choices, outcomes and implementation are expected to be used to provide support in this phase, particularly for understanding whether the advice is fit for purpose, and the efficacy of choices are clear ( ''low confidence'' ) ( [[#Spetzler--2016|Spetzler et al., 2016]] ).

<div id="_idContainer028" class="Figure"></div>

[[File:0264e4c2abe136a1320fb72de84b23e4 IPCC_AR6_WGII_Figure_17_008.png]]

'''Figure 17.8 |''' '''Decision-analytic tools used across different geo-political scales and how they contributed to decision outcomes.''' Points comprise the type of decision-making body (C = Community; G = Government; B = Business/Industry; F = Finance; N = NGO; A = All categories) coupled with the reference number in square brackets, which correspond to numbered references in the case studies of Table 17.4. Colours of the points correspond to the class of decision-analytic tool: Bayesian (red), DMDU (decision-making under deep uncertainty) (brown), decision process management (dark blue), economic and financial methods (purple), interval methods (light blue), MCDA—full ranking (light green) or partial ranking (dark green), soft elicitation (Black).

'''Table 17.4 |''' Characteristics of the main approaches to decision analysis with respect to their ''Cynefin'' context, the manner in which they can be used to address different uncertainties, where they may be used in different cognitive phases of the decision-making process, the resources required and some case studies for further exploring how they might be used. Numbers in square brackets after references in case studies refer to the references plotted in Figure 17.8.

{| class="wikitable"
|-
| colspan="10"| '''A: Bayesian methods''' ( [[#Keeney--1993|Keeney and Raiffa, 1993]] ; [[#Smith--2010|Smith, 2010]] ; [[#Gelman--2013|Gelman et al., 2013]] ; [[#Reilly--2013|Reilly and Clemen, 2013]] ; [[#Abbas--2015|Abbas and Howard, 2015]] ; [[#Sperotto--2017|Sperotto et al., 2017]] ; [[#Marchau--2019|Marchau et al., 2019]] )

A structured approach to assembling information around the consequences of choices, either by modelling, by analysis of multiple scenarios or by structuring deliberation; underpinned by a theoretical base, coherent assumptions and powerful computational methods; can use both observational data and expert knowledge, weighting them appropriately; same approaches as in artificial intelligence algorithms. Biases (information, stakeholders, decision makers) can be made explicit. Traditionally, Bayesian methods computationally identify an ‘optimal’ decision, based on maximising the expected utility across a number of specified requirements, represented as functions.

|-
| colspan="10"| '''Examples''' include the general application of decision network models ( [[#Richards--2013|Richards et al., 2013]] ; [[#Sperotto--2017|Sperotto et al., 2017]] ); the use of decision network analyses based on elicitation to choose adaptations to coastal management in a lagoonal area in Italy ( [[#Catenacci--2013|Catenacci and Giupponi, 2013]] ) and coastal community in UK ( [[#Jäger--2018|Jäger et al., 2018]] ); combination of economic models and decision models to assess research and development priorities ( [[#Baker--2011|Baker and Solak, 2011]] ); combining outputs from models, observations and opinions in a decision framework for assessing climate impacts on water nutrient loads in Italy ( [[#Sperotto--2019|Sperotto et al., 2019]] ) and a general review for water resource management ( [[#Phan--2019|Phan et al., 2019]] ); combining results from different dynamic models to assess human mortality from ozone in the USA ( [[#Alexeeff--2016|Alexeeff et al., 2016]] ); assessing adaptive capacity of surf lifesaving in Australia ( [[#Richards--2016|Richards et al., 2016]] ); and assessing urban flood risks in Denmark (Åström et al., 2014).

|-
| colspan="6"| '''''Cognitive phase'''''

| rowspan="2" colspan="2"| '''''Resources required'''''

| rowspan="2" colspan="2"| '''''Case studies'''''

|-
| colspan="2"| ''Sense-making and modelling''

| colspan="2"| ''Analysing and exploring''

| colspan="2"| ''Interpreting and implementing''

|-
| colspan="2"| Construction of hierarchical models, belief nets ( [[#Sperotto--2017|Sperotto et al., 2017]] ; [[#Phan--2019|Phan et al., 2019]] ), decision trees ( [[#Keeney--1993|Keeney and Raiffa, 1993]] ) and influence diagrams ( [[#Keeney--1993|Keeney and Raiffa, 1993]] ; [[#Reilly--2013|Reilly and Clemen, 2013]] ) supplemented by many soft elicitation techniques helps build models for quantitative analysis ( [[#Gelman--2003|Gelman, 2003]] ; [[#Bendoly--2016|Bendoly and Clark, 2016]] ).

| colspan="2"| Bayesian updating and expected utility analysis supplemented by robustness and sensitivity analyses ( [[#Rios%20Insua--1999|Rios Insua, 1999]] ; [[#Rios%20Insua--2000|Rios Insua and Ruggeri, 2000]] ; [[#French--2009|French et al., 2009]] ; [[#Smith--2010|Smith, 2010]] ; [[#Reilly--2013|Reilly and Clemen, 2013]] ; [[#Abbas--2015|Abbas and Howard, 2015]] ).

| colspan="2"| Use of graphical models (decision trees, belief nets and influence diagrams) and sensitivity plots can help make transparent and explain reasoning for strategy to stakeholders and implementers ( [[#Bendoly--2016|Bendoly and Clark, 2016]] ) and provide for auditable building of consensus.

| colspan="2"| Bayesian decision-analytic models can be applied with increasing complexity and sophistication to any given problem. Coherence between different levels of sophistication can be maintained. Thus, the resources can be tailored to the time and support available for the analysis. The most sophisticated analyses are computationally demanding.

| colspan="2"| Alexeeff et al. 2016) [1], Åström et al. (2014) [2], [[#Baker--2011|Baker and Solak (2011)]] [3], [[#Catenacci--2013|Catenacci and Giupponi (2013)]] [4], [[#Jäger--2018|Jäger et al. (2018)]] [5], [[#Phan--2019|Phan et al. (2019)]] [6], [[#Richards--2013|Richards et al. (2013)]] [7], Richards et al. (2016) [8], [[#Sperotto--2017|Sperotto et al. (2017)]] [9], [[#Sperotto--2019|Sperotto et al. (2019)]] [10]

|-
| colspan="3"| '''''Uncertainties'''''

| colspan="7"| '''''Cynefin context'''''

|-
| ''Stochastic, epistemic, analytical''

''(descriptive modelling)''

| colspan="2"| ''Ambiguity''

''value''

''(prescriptive modelling)''

| colspan="2"| ''Known''

| colspan="2"| ''Knowable''

| colspan="2"| ''Complex''

| ''Chaotic''

|-
| All can be modelled probabilistically, perhaps supplemented by sensitivity analysis ( [[#Rios%20Insua--1999|Rios Insua, 1999]] ; [[#Rios%20Insua--2000|Rios Insua and Ruggeri, 2000]] ; [[#Iooss--2017|Iooss and Saltelli, 2017]] ). Deep uncertainties can be investigated via scenarios ( [[#French--2020|French, 2020]] ).

| colspan="2"| Uncertainties resolved or reduced by discussion, then values modelled by multi-attribute values and utilities ( [[#Keeney--1992|Keeney, 1992]] ; [[#Keeney--1993|Keeney and Raiffa, 1993]] ; [[#Gregory--2012|Gregory et al., 2012]] ). Residual uncertainties explored via sensitivity analysis.

| colspan="2"| Any stochastic uncertainties modelled probabilistically; otherwise, deterministic modelling with sensitivity analysis. Value functions tend to be used more than utility functions ( [[#Keeney--1993|Keeney and Raiffa, 1993]] ; [[#Goodwin--2014|Goodwin and Wright, 2014]] ).

| colspan="2"| Epistemic uncertainties updated via Bayesian statistics/machine learning, then remaining stochastic uncertainties modelled probabilistically. Full Bayesian decision modelling possible ( [[#French--2009|French et al., 2009]] ; [[#Smith--2010|Smith, 2010]] ; [[#Abbas--2015|Abbas and Howard, 2015]] ).

| colspan="2"| More exploratory analysis ( [[#Gelman--2003|Gelman, 2003]] ) to understand behaviours with less complex Bayesian modelling support by sensitivity and robustness studies ( [[#Rios%20Insua--1999|Rios Insua, 1999]] ; [[#French--2003|French, 2003]] ). Scenario-focused decision analysis to cope with deep uncertainties ( [[#French--2020|French, 2020]] ). Careful deliberations to construct values and utilities. ( [[#Keeney--1993|Keeney and Raiffa, 1993]] ; [[#Gregory--2012|Gregory et al., 2012]] ).

| Formal modelling impossible. Much exploratory work to identify potential causes and effects. Little if any complex analysis.

|-
|
|-
| colspan="10"| '''B: Decision-making under deep uncertainty (DMDU)''' ( [[#Hallegatte--2012|Hallegatte et al., 2012]] ; [[#Weaver--2013|Weaver et al., 2013]] ; [[#Marchau--2019|Marchau et al., 2019]] ; [[#Workman--2021|Workman et al., 2021]] )

Deep uncertainty relates to circumstances in which data are too sparse, experts are in too much disagreement or time is too short to model the uncertainty. As such, DMDU methods are focused on working in the ''Cynefin'' Complex Space context. Approaches emphasise robustness (‘no regrets’ options) and the use of scenarios, and often link well with scenario-focused robust Bayesian studies (Cross-Chapter Box DEEP in this Chapter). DMDU studies draw in many other approaches to decision analysis, using them to identify robust rather than optimal strategies, as in robust decision-making (RDM). DMDU analyses can help decision makers to think contingently and build a more wide-ranging recognition of the risks. They often integrate with other classes of tools.

|-
| colspan="10"| '''Examples''' include RDM for hydro-power design using down-scaled climate data in Sub-Saharan Africa ( [[#Taner--2017|Taner et al., 2017]] ), RDM for water management in California, USA ( [[#Lempert--2010|Lempert and Groves, 2010]] ), the Colorado River, USA, and for international climate investment strategies ( [[#Groves--2019|Groves et al., 2019]] ), use of decision scaling ( [[#Brown--2019|Brown et al., 2019]] ), comparison of RDM and Info-gap methods ( [[#Hall--2012|Hall et al., 2012]] ) and review of using climate modelling in RDM ( [[#Weaver--2013|Weaver et al., 2013]] ).

|-
| colspan="6"| '''''Cognitive phase'''''

| rowspan="2" colspan="2"| '''''Resources required'''''

| rowspan="2" colspan="2"| '''''Case studies'''''

|-
| colspan="2"| ''Sense-making and modelling''

| colspan="2"| ''Analysing and exploring''

| colspan="2"| ''Interpreting and implementing''

|-
| colspan="2"| Some of the simpler DMDU tools complement soft elicitation tools and can help to identify relevant scenarios and help formulate problems.

| colspan="2"| Many Bayesian or MCDA tools can be used here but with DMDU’s additional emphasis on robustness and the exploration of several/many scenarios.

| colspan="2"| DMDU with its emphasis on robustness encourages contingency planning in implementation with careful monitoring to identify emerging risks.

| colspan="2"| Some of the simpler models do not require substantial resources, but the application of parallel sophisticated analyses in several scenarios can be computationally demanding. Also, the emphasis on discussion of robustness can be demanding on the time of problem-owners, experts and stakeholders.

| colspan="2"| [[#Brown--2019|Brown et al. (2019)]] [11], [[#Groves--2019|Groves et al. (2019)]] [12], [[#Hall--2012|Hall et al. (2012)]] [13], [[#Lempert--2010|Lempert and Groves (2010)]] , [14], Taner et al. (2017) [15], [[#Weaver--2013|Weaver et al. (2013)]] [16]

|-
| colspan="3"| '''''Uncertainties'''''

| colspan="7"| '''''Cynefin context'''''

|-
| ''Stochastic, epistemic, analytical''

''(descriptive modelling)''

| colspan="2"| ''Ambiguity''

''value''

''(prescriptive modelling)''

| colspan="2"| ''Known''

| colspan="2"| ''Knowable''

| colspan="2"| ''Complex''

| ''Chaotic''

|-
| Methods are designed for deep epistemic uncertainties. Some can deal with stochastic uncertainties. Analytical uncertainties seldom accounted for.

| colspan="2"| Some DMDU methods draw on MCDA methods and thus consider ambiguity and value uncertainties. In any case, DMDU methods support wide deliberation with stakeholders.

| colspan="2"| Deep uncertainty is absent, but the principles and processes of decision-making may be used.

| colspan="2"| Deep uncertainty is absent, but the principles of decision-making may be used.

| colspan="2"| The complex and chaotic spaces are home to deep uncertainties. DMDU tools and more particularly processes are relevant here. The emphasis on robustness is very relevant. The tools themselves are relatively simply structured but are effective at stimulating discussion.

| Deep uncertainties are rife in the chaotic contexts. DMDU emphases on robustness and possible scenarios can stimulate creative discussions of ill-understood issues.

|-
|
|-
| colspan="10"| '''C: Decision process management''' ( [[#Raz--2001|Raz and Micheal, 2001]] ; [[#Dalkir--2005|Dalkir, 2005]] ; Burstein and W. Holsapple, 2008; [[#Jashapara--2011|Jashapara, 2011]] ; [[#Bonczek--2014|Bonczek et al., 2014]] ; [[#Sauter--2014|Sauter, 2014]] ; [[#Holsapple--2019|Holsapple et al., 2019]] )

A range of tools and techniques to help manage the decision-making process and support risk management and the implementation of the chosen strategy. Some tools organise data and analyses, often being built on a geographic information system, known as decision support tools. Others manage processes, organising workflows. Some have inevitably expanded in function to support decision-making itself, even though their primary focus might be on, say, implementation and monitoring risks. Such tools are closely related to knowledge management systems; knowledge management processes and decision process management differ more in terminology than in substance.

|-
| colspan="10"| '''Examples''' include tools for agriculture ( [[#Biehl--2017|Biehl et al., 2017]] ), evaluating and comparing CMIP climate models ( [[#Parding--2020|Parding et al., 2020]] ), development of action cycles ( [[#Park--2012|Park et al., 2012]] ) and decision support systems across a range of sectors and decision-group applications ( [[#Papathanasiou--2016|Papathanasiou et al., 2016]] ).

|-
| colspan="6"| '''''Cognitive phase'''''

| rowspan="2" colspan="2"| '''''Resources required'''''

| rowspan="2" colspan="2"| '''''Case studies'''''

|-
| colspan="2"| ''Sense-making and modelling''

| colspan="2"| ''Analysing and exploring''

| colspan="2"| ''Interpreting and implementing''

|-
| colspan="2"| Process, project, knowledge elicitation and risk management tools help identify how to structure decision-making processes. Decision process tools can capture details for implementation and document process for audit trail.

| colspan="2"| Tools help structure decision-making processes and ensure timely involvement of problem owners, stakeholders, and experts. Knowledge management tools can capture details for implementation and document process for audit trail.

| colspan="2"| Project management tools plan implementation and risk management tools identify what to monitor during implementation. Knowledge management tools maintain audit trail and track reasoning for choices made during implementation.

| colspan="2"| Decision process management tools can reduce resources needed in the decision-making process. However, this assumes that the tools are already installed on local information systems and that the analysis team is experienced in using them. Otherwise, resource is needed to understand and train in the use of the tools.

| colspan="2"| [[#Biehl--2017|Biehl et al. (2017)]] [17], Papathanasiou et al. (2016) [18], [[#Parding--2020|Parding et al. (2020)]] [19], [[#Park--2012|Park et al. (2012)]] [20]

|-
| colspan="3"| '''''Uncertainties'''''

| colspan="7"| '''''Cynefin context'''''

|-
| ''Stochastic, epistemic, analytical''

''(descriptive modelling)''

| colspan="2"| ''Ambiguity''

''value''

''(prescriptive modelling)''

| colspan="2"| ''Known''

| colspan="2"| ''Knowable''

| colspan="2"| ''Complex''

| ''Chaotic''

|-
| Not designed to address uncertainties involved in the decision itself, but may handle project risks in the decision process, especially implementation.

| colspan="2"| Not usually addressed, since ambiguities and value uncertainties will be addressed in the decision-making itself, but may use those values in risk management of implementation.

| colspan="2"| Simple project management tools may be sufficient here.

| colspan="2"| Project management and risk management tools apply easily here.

| colspan="2"| Project management and risk management tools may be used, but attention needs to be paid to risks that are complex in nature with little knowledge of precise relationships between cause and effects.

| Project management and risk management tools may be used, but attention needs to be paid to risks that are complex in nature with little knowledge of precise relationships between cause and effects.

|-
|
|-
| colspan="10"| '''D: Economic and financial methods''' ( [[#Howell--2001|Howell et al., 2001]] ; [[#Pearce--2006|Pearce et al., 2006]] ; [[#Boardman--2017|Boardman et al., 2017]] ; [[#Atkinson--2018a|Atkinson et al., 2018a]] ; [[#Hurlbert--2019|Hurlbert et al., 2019]] )

Stem from economic theory and accounting practices: for example, cost–benefit analysis, which seeks to price out all aspects of the consequence of a strategy, portfolio analysis, or real options theory, which seeks to value financial investments allowing for their risks and the contingent buying and selling. Such methods are perceived as objective when dealing with tangibles, but are more controversial in their valuing of intangibles. Since these methods model uncertainties with probabilities and then work with expectations, they share much in common with Bayesian methods. However, many applications of cost–benefit analysis omit any detailed treatment of uncertainty.

|-
| colspan="10"| '''Examples''' examine the economic costs and benefits of adaptation pathways for storm water infrastructure in Singapore ( [[#Manocha--2017|Manocha and Babovic, 2017]] ), and a coastal mega city, Los Angeles in the USA ( [[#de%20Ruig--2019|de Ruig et al., 2019]] )

|-
| colspan="6"| '''''Cognitive phase'''''

| rowspan="2" colspan="2"| '''''Resources required'''''

| rowspan="2" colspan="2"| '''''Case studies'''''

|-
| colspan="2"| ''Sense-making and modelling''

| colspan="2"| ''Analysing and exploring''

| colspan="2"| ''Interpreting and implementing''

|-
| colspan="2"| In themselves, these methods do not support sense-making and modelling, though discussions of how to value impacts, both tangible and intangible can be catalytic in understanding the issues.

| colspan="2"| These tools focus mainly on analysis and evaluating the costs and benefits of various options. They are not designed to be used interactively so are more often deployed and communicated via reports than interactive workshops.

| colspan="2"| Since community-based adaptation (CBA) methods do not emphasise the analysis of uncertainties and risks, they are less suited for use in developing and communicating an implementation plan. Real options with their emphasis on contingency are much more suited ( [[#Fischhoff--2015|Fischhoff, 2015]] ).

| colspan="2"| Cost–benefit analysis for complex projects is a major undertaking, with much data collection needed to value outcomes. Real options also require data on risks and uncertainties. Both may have high computational needs.

| colspan="2"| [[#de%20Ruig--2019|de Ruig et al. (2019)]] [21], [[#Manocha--2017|Manocha and Babovic (2017)]] [22]

|-
| colspan="3"| '''''Uncertainties'''''

| colspan="7"| '''''Cynefin context'''''

|-
| ''Stochastic, epistemic, analytical''

''(descriptive modelling)''

| colspan="2"| ''Ambiguity''

''value''

''(prescriptive modelling)''

| colspan="2"| ''Known''

| colspan="2"| ''Knowable''

| colspan="2"| ''Complex''

| ''Chaotic''

|-
| Cost–benefit methods usually deal with uncertainty via expectations with little attention to probability distributions; real options methods tend to treat uncertainty in much more sophisticated ways. Both methods, when applied fully have many points of contact with Bayesian methods ( [[#Neely--2001|Neely and de Neufville, 2001]] ; [[#Bedford--2005|Bedford et al., 2005]] )

| colspan="2"| These methods reduce all value and preference information to financial equivalents. The key issue is to find a market in which all outcomes may be valued financially. Modern CBA methods use much more subtle techniques for this than those applied in the last century ( [[#Bedford--2005|Bedford et al., 2005]] ; [[#Saarikoski--2016|Saarikoski et al., 2016]] ).

| colspan="2"| Although CBA and many financial methods work in theory, the complexity makes them seldom worth the effort.

| colspan="2"| The methods may be applied to evaluate complex projects, but CBA tends to ‘average out’ rather than analyse uncertainty.

| colspan="2"| The recognition of the need to treat deep uncertainties using real options has been investigated ( [[#Hallegatte--2012|Hallegatte et al., 2012]] ; [[#Buurman--2016|Buurman and Babovic, 2016]] ).

| Formal modelling impossible. Much exploratory work to identify potential causes and effects. Little if any complex analysis.

|-
| colspan="10"| '''E: Interval methods''' ( [[#Shafer--1976|Shafer, 1976]] ; [[#Pedrycz--2011|Pedrycz et al., 2011]] )

Because of concerns that the statistical accuracy of some data is unknown, and that decision makers and experts cannot make numerical judgements accurately, analyses have been suggested which work with ranges of values in categories (intervals) as their inputs. While avoiding accuracy issues, weakening the arithmetic may result in other foundational assumptions not being met, including some basic principles of rationality. Different types of uncertainty can often be confused, and the analyses can contradict basic probability theory. Interval models of semantics and imprecision can be useful in exploring ambiguity and value uncertainty, though modelling rather than resolving such uncertainties does not necessary help in decision-making. Some interval methods can be thought of more as sensitivity techniques applied to other decision-analytic approaches. Typical approaches here relate to the fuzzy or possibility theory, and evidential reasoning.

|-
| colspan="10"| '''Examples''' include using fuzzy methods to assess climate adaptations in ports in China ( [[#Yang--2018|Yang et al., 2018]] ), water supply vulnerability in South Korea ( [[#Kim--2013|Kim and Chung, 2013]] ) and resilience of the Nile River Delta ( [[#Batisha--2015|Batisha, 2015]] ); and evidential reasoning in an environmental impact assessment for flood mitigation in Manila Philippines ( [[#Gilbuena--2013|Gilbuena et al., 2013]] ).

|-
| colspan="6"| '''''Cognitive phase'''''

| rowspan="2" colspan="2"| '''''Resources required'''''

| rowspan="2" colspan="2"| '''''Case studies'''''

|-
| colspan="2"| ''Sense-making and modelling''

| colspan="2"| ''Analysing and exploring''

| colspan="2"| ''Interpreting and implementing''

|-
| colspan="2"| The emphasis on modelling ambiguity may help structure a model initially, but the lack of structures to model and explore complex interdependencies may inhibit the ability to build a valid representation of the issues.

| colspan="2"| If there are substantial data available, then even the simplest of these methods can produce useful results. But with small quantities of data, their data analysis may be too inefficient. Evidential reasoning MCDA can be insightful on the preference side.

| colspan="2"| The emphasis on linguistic uncertainty may in some cases mask some of the issues ( [[#French--1995|French, 1995]] ).

| colspan="2"| Many methods are rather simple in application and require only moderate resources, but they may face issues in scaling up to major complex problems.

| colspan="2"| [[#Batisha--2015|Batisha (2015)]] [23], [[#Gilbuena--2013|Gilbuena et al. (2013)]] [24], [[#Kim--2013|Kim and Chung (2013)]] [25], [[#Yang--2018|Yang et al. (2018)]] [26]

|-
| colspan="3"| '''''Uncertainties'''''

| colspan="7"| '''''Cynefin context'''''

|-
| ''Stochastic, epistemic, analytical''

''(descriptive modelling)''

| colspan="2"| ''Ambiguity''

''value''

''(prescriptive modelling)''

| colspan="2"| ''Known''

| colspan="2"| ''Knowable''

| colspan="2"| ''Complex''

| ''Chaotic''

|-
| There are issues of operational definition of quantities in some methodologies. Some simpler interval methods have no concept of conditionality so cannot model learning effectively, but there are some very sophisticated theories of evidence that can. Interval methods can also provide sensitivity analyses for Bayesian and MCDA methods ( [[#Shafer--1976|Shafer, 1976]] ; [[#Rios%20Insua--1990|Rios Insua, 1990]] ).

| colspan="2"| Some methods can be simplistic, with quantities not being operationally defined. The evidential reasoning approach to MCDA allows exploration of the relative weights on different criteria or between levels in criteria ( [[#Xu--2012|Xu, 2012]] ; [[#Zhang--2017|Zhang et al., 2017]] ).

| colspan="2"| Methods can be applied here without major issue, possibly because the simple, repetitive nature of the problem allows access to much data and the possibility of tuning the methods to the application.

| colspan="2"| Since the methods often capture rather than explore and resolve ambiguity and value uncertainties, they can hide issues. Also, the lack, in some cases, of operational definitions may mean that some quantification is dubious. Evidential reasoning methods can help analyse conflicting objectives ( [[#French--1995|French, 1995]] ; [[#Xu--2012|Xu, 2012]] ).

| colspan="2"| The recognition of the need to treat deep uncertainties using real options has been investigated ( [[#Hallegatte--2012|Hallegatte et al., 2012]] ; [[#Buurman--2016|Buurman and Babovic, 2016]] ).

| The ability to deal with ambiguity may be helpful in poorly understood situations, but the emphasis on capturing ambiguity may ultimately slow the building of understanding.

|-
|
| colspan="2"|
|
|-
| colspan="10"| '''F: Multi-criteria decision analysis (MCDA): Full ranking and optimal seeking''' ( [[#Bell--2001|Bell et al., 2001]] ; [[#Belton--2002|Belton and Stewart, 2002]] ; [[#Bouyssou--2006|Bouyssou et al., 2006]] ; [[#Zopounidis--2010|Zopounidis and Pardalos, 2010]] ; [[#Tzeng--2011|Tzeng and Huang, 2011]] ; [[#Velasquez--2013|Velasquez and Hester, 2013]] ; [[#Kumar--2017|Kumar et al., 2017]] )

Covers many approaches: indeed, Bayesian, DMDU and interval methods are sometimes considered MCDA. Some MCDAs seek an optimal or best strategy; others form partial rankings, eliminating weak strategies but not discriminating fully between the better ones. Many MCDA methods eschew dealing with uncertainties and focus on modelling and exploring conflicting objectives and balancing these. MCDA techniques are especially useful in working with senior decision makers in setting policy and broad objectives, and in processes of stakeholder engagement.

|-
| colspan="10"| '''Examples''' include ranking adaptation and mitigation priorities at a national level in the Netherlands ( [[#de%20Bruin--2009|de Bruin et al., 2009]] ), Lithuania ( [[#Streimikiene--2013|Streimikiene and Balezentis, 2013]] ) and Bangladesh ( [[#Haque--2016|Haque, 2016]] ), in the forestry sector in Nicaragua ( [[#Guillén%20Bolaños--2018|Guillén Bolaños et al., 2018]] ) and in emissions trading in the European Union ( [[#Konidari--2007|Konidari and Mavrakis, 2007]] ).

|-
| colspan="6"| '''''Cognitive phase'''''

| rowspan="2" colspan="2"| '''''Resources required'''''

| rowspan="2" colspan="2"| '''''Case studies'''''

|-
| colspan="2"| ''Sense-making and modelling''

| colspan="2"| ''Analysing and exploring''

| colspan="2"| ''Interpreting and implementing''

|-
| colspan="2"| There is growing experience in combining soft elicitation with tools to formulate problems ( [[#Marttunen--2017|Marttunen et al., 2017]] ). Many MCDA tools naturally encourage discussion and deliberation on developing appropriate value structures. However, exploration and formulation of stochastic and epistemological uncertainties is less developed ( [[#Durbach--2020a|Durbach and Stewart, 2020a]] ).

| colspan="2"| Emphasis is usually on analysing and exploring, resolving conflicting objectives. MCDA methods come into their own at this stage of the process. Sensitivity tools and intuitive graphical displays exist for many of the methods ( [[#Gunawan--2005|Gunawan and Azarm, 2005]] ; [[#Boardman--2017|Boardman et al., 2017]] ).

| colspan="2"| Use of graphical models and sensitivity plots can help explain reasoning for strategy to stakeholders and implementers ( [[#Bendoly--2016|Bendoly and Clark, 2016]] ).

| colspan="2"| The more exploratory methods can be quite light in terms of computational resource, but require interactions with decision makers and stakeholders in workshops. Methods with use complex stochastic mathematical programming can be computationally demanding and require substantial data.

| colspan="2"| ( [[#de%20Bruin--2009|de Bruin et al., 2009]] ) [27], ( [[#Guillén%20Bolaños--2018|Guillén Bolaños et al., 2018]] ) [28], ( [[#Haque--2016|Haque, 2016]] ) [29], ( [[#Konidari--2007|Konidari and Mavrakis, 2007]] ) [30], ( [[#Streimikiene--2013|Streimikiene and Balezentis, 2013]] ) [31]

|-
| colspan="3"| '''''Uncertainties'''''

| colspan="7"| '''''Cynefin context'''''

|-
| ''Stochastic, epistemic, analytical''

''(descriptive modelling)''

| colspan="2"| ''Ambiguity''

''value''

''(prescriptive modelling)''

| colspan="2"| ''Known''

| colspan="2"| Knowable

| colspan="2"| Complex

| Chaotic

|-
| These methods tend to focus on balancing and resolving conflicting objectives and include little or no analysis of stochastic and epistemic uncertainties. Interactive methods that use complex objective functions do need to consider convergence criteria for analytic uncertainties.

| colspan="2"| Many methods here use multi-attribute value functions and focus on using weights to explore different emphases on conflicting objectives. One very popular method is analytic hierarchy processing (AHP) ( [[#Saaty--1980|Saaty, 1980]] ) though this has issues in scaling up to evaluate more than a handful of policies.

| colspan="2"| Usually in the known context, the objective function is well understood; but in cases where it is not, interactive multi-objective programming can offer a way forward (Klamroth et al., 2018).

| colspan="2"| If the objective function is not well understood, then these methods can be useful and can be extended to stochastic programming, but epistemic uncertainties are not really addressed ( [[#Gutjahr--2016|Gutjahr and Pichler, 2016]] ).

| colspan="2"| Methods can explore conflicting objectives, but seldom are able to address deep epistemic uncertainties, unless combined with scenarios ( [[#Stewart--2013|Stewart et al., 2013]] ; [[#Marchau--2019|Marchau et al., 2019]] ; [[#Durbach--2020a|Durbach and Stewart, 2020a]] ).

| Formal modelling impossible. Much exploratory work to identify potential causes and effects. Little if any complex analysis.

|-
|
|-
| colspan="10"| '''G: Multi-criteria decision analysis (MCDA): Partial ranking''' ( [[#Roy--1996|Roy, 1996]] ; [[#Bell--2001|Bell et al., 2001]] ; [[#Belton--2002|Belton and Stewart, 2002]] ; [[#Bouyssou--2006|Bouyssou et al., 2006]] ; [[#Behzadian--2010|Behzadian et al., 2010]] ; [[#Zopounidis--2010|Zopounidis and Pardalos, 2010]] ; [[#Tzeng--2011|Tzeng and Huang, 2011]] ; Bouyssou and others, 2012; De Smet and Lidouh, 2012; [[#Velasquez--2013|Velasquez and Hester, 2013]] ; [[#Figueira--2016|Figueira et al., 2016]] ; [[#Govindan--2016|Govindan and Jepsen, 2016]] )

|-
| colspan="10"| '''Examples''' include developing criteria for assessing climate protection strategies and applying these to retrofitting a school to manage climate risks in Germany ( [[#Markl-Hummel--2014|Markl-Hummel and Geldermann, 2014]] ); evaluating outranking approaches for managing heat stress in a large city in Australia ( [[#El-Zein--2015|El-Zein and Tonmoy, 2015]] ); using MCDA to manage the interactions of climate change with tourism in Greece (Michailidou et al., 2016); and identifying priorities to manage droughts and floods in agriculture in Bangladesh ( [[#Xenarios--2015|Xenarios and Polatidis, 2015]] ).

|-
| colspan="6"| '''''Cognitive phase'''''

| rowspan="2" colspan="2"| '''''Resources Required'''''

| rowspan="2" colspan="2"| '''''Case Studies'''''

|-
| colspan="2"| ''Sense-making and modelling''

| colspan="2"| ''Analysing and exploring''

| colspan="2"| ''Interpreting and implementing''

|-
| colspan="2"| Graphical representations of partial orders are useful in model formulation, and the emphasis on exploring what can be said objectively about dominance relations can build a kernel of consensus between decision makers and stakeholders.

| colspan="2"| ELECTRE and PROMETHEE implementations of outranking approaches have many tools for exploring partial relations and analysing agreements and the reasoning behind these.

| colspan="2"| The analysis of dominance can provide a sound footing for building risk registers to aid implementation. Understanding the kernel of consensus can also aid communication.

| colspan="2"| If an outranking algorithm is essentially combinatorial in its approach, then for complex problems there may be computational problems. Some of the methods may require less interaction with decision-makers and stakeholders if they can deduce many partial relations from objective data.

| colspan="2"| ( [[#El-Zein--2015|El-Zein and Tonmoy, 2015]] ) (Markl- [32], Hummel and Geldermann, 2014) [33], (Michailidou et al., 2016) [34], ( [[#Xenarios--2015|Xenarios and Polatidis, 2015]] ) [35]

|-
| colspan="4"| '''''Uncertainties'''''

| colspan="6"| '''''Cynefin context'''''

|-
| colspan="2"| ''Stochastic, epistemic, analytical''

''(descriptive modelling)''

| colspan="2"| ''Ambiguity''

''value''

''(prescriptive modelling)''

| ''Known''

| colspan="2"| Knowable

| colspan="2"| Complex

| Chaotic

|-
| colspan="2"| Modelling of all forms of uncertainty including epistemic uncertainty is not the primary objective of these methods. Stochastic uncertainty may be included as probability distributions, but there is no formalism for learning to address epistemic uncertainties ( [[#Hyde--2003|Hyde et al., 2003]] ; [[#Behzadian--2010|Behzadian et al., 2010]] ; [[#Gervásio--2012|Gervásio and Simões da Silva, 2012]] ).

| colspan="2"| Partial ranking or outranking methods seek, first of all, to identify dominance between options and preference relations that can be agreed somewhat objectively. Thus, first they eliminate suboptimal alternatives before seeking a fuller ranking. Ambiguity and value uncertainty may also be quantified ( [[#Behzadian--2010|Behzadian et al., 2010]] ; [[#Figueira--2016|Figueira et al., 2016]] ; [[#Govindan--2016|Govindan and Jepsen, 2016]] ).

| Usually in the known context, the objective function is well understood; but when it is not, outranking methods can identify a partial ranking without needing too many interactions with problem-owners.

| colspan="2"| Since epistemic uncertainties are not fully addressed, these methods can only help in relation to conflicting objectives, but robustness to uncertainties will need addressing ( [[#Hyde--2003|Hyde et al., 2003]] ).

| colspan="2"| Outranking methods may be combined with scenarios to explore and analyse decisions under deep uncertainty ( [[#Hyde--2003|Hyde et al., 2003]] ; [[#Durbach--2014|Durbach, 2014]] ).

| Formal modelling impossible. Much exploratory work to identify potential causes and effects. Little if any complex analysis.

|-
|
|-
| colspan="10"| '''H: Soft elicitation''' ( [[#Rosenhead--2001|Rosenhead and Mingers, 2001]] ; [[#Shaw--2006|Shaw et al., 2006]] ; [[#Shaw--2007|Shaw et al., 2007]] ; [[#Ackermann--2012|Ackermann, 2012]] ; [[#Bendoly--2016|Bendoly and Clark, 2016]] )

Also known as problem structuring, it is the process of asking problem owners, experts and stakeholders for the knowledge, perceptions, beliefs, uncertainties and values that a model needs to embody before being populated with numbers. Methods here help in problem formulation, structuring understanding: for example, cognitive maps, soft operational research diagrams, soft systems, prompts such as PESTLE and other qualitative tools ( [[#Prober--2017|Prober et al., 2017]] ; [[#Symstad--2017|Symstad et al., 2017]] ). The output of soft elicitation can lead to the building of sophisticated quantitative models ( [[#Symstad--2017|Symstad et al., 2017]] ) and can also structure communications and deliberations with stakeholders. Exploratory data analysis and visual analytics are also relevant. Soft elicitation has enormous advantages in setting the frame for communication between all parties ( [[#Prober--2017|Prober et al., 2017]] ); there are many cases in which the clarity brought by framing the issues well has obviated the need for formal quantitative analysis.

|-
| colspan="10"| '''Examples''' include Adaptation Pathway planning and elicitation on managing a national park in the USA ( [[#Symstad--2017|Symstad et al., 2017]] ), poverty alleviation in a province in Indonesia ( [[#Butler--2016|Butler et al., 2016]] ), woodland landscapes in Australia ( [[#Prober--2017|Prober et al., 2017]] ) and general considerations for contested adaptations ( [[#Bosomworth--2017|Bosomworth et al., 2017]] ).

|-
| colspan="6"| '''''Cognitive phase'''''

| rowspan="2" colspan="2"| '''''Resources required'''''

| rowspan="2" colspan="2"| '''''Case Studies'''''

|-
| colspan="2"| ''Sense-making and modelling''

| colspan="2"| ''Analysing and exploring''

| colspan="2"| ''Interpreting and implementing''

|-
| colspan="2"| Soft elicitation tools provide much support to sense-making, formulating problems and identifying relevant issues to be addressed ( [[#Shaw--2006|Shaw et al., 2006]] ; [[#Shaw--2007|Shaw et al., 2007]] ; [[#Ackermann--2012|Ackermann, 2012]] ).

| colspan="2"| Soft elicitation is not relevant to quantitative analysis and evaluation per se, but can support the exploration of residuals to understand the quality of the models and detect further factors to be addressed.

| colspan="2"| The results of soft elicitation provide the dimensions for communication by identifying the issues that are important to stakeholders and building understanding in those implementing the policies.

| colspan="2"| Physical resources requirements are relatively slight: sometimes post-its and a white board can be sufficient, though modern visual analytics can require substantial computing resource. However, the demands on the time of problem owners, stakeholders and experts can be significant.

| colspan="2"| ( [[#Bosomworth--2017|Bosomworth et al., 2017]] ) [36], ( [[#Butler--2016|Butler et al., 2016]] ) [37], ( [[#Prober--2017|Prober et al., 2017]] ) [38], ( [[#Symstad--2017|Symstad et al., 2017]] ) [39]

|-
| colspan="4"| '''''Uncertainties'''''

| colspan="6"| '''''Cynefin context'''''

|-
| colspan="2"| ''Stochastic, epistemic, analytical''

''(descriptive modelling)''

| colspan="2"| ''Ambiguity''

''value''

''(prescriptive modelling)''

| ''Known''

| colspan="2"| Knowable

| colspan="2"| Complex

| Chaotic

|-
| colspan="2"| Soft elicitation tools are available to elicit problem-owners’ and experts’ perceptions of these uncertainties and, more particularly, dependences and independences between them. Exploratory data analysis is also relevant ( [[#Steed--2013|Steed et al., 2013]] ; [[#Bendoly--2016|Bendoly and Clark, 2016]] ).

| colspan="2"| There are tools to catalyse deliberations and help problem-owners and stakeholders clarify their meanings and contextualise their values to the specific issues being considered ( [[#Keeney--1992|Keeney, 1992]] ).

| Usually, problems falling into known contexts are well understood and there is little need to elicit or structure models to perform analyses.

| colspan="2"| Problems falling into knowable space are usually well structured and problem owners’ values are also well understood. However, there may be a need to explore error structures in preparation to estimate parameters in the models ( [[#Gelman--2003|Gelman, 2003]] ; [[#Steed--2013|Steed et al., 2013]] ; [[#Fekete--2016|Fekete and Primet, 2016]] ).

| colspan="2"| Many soft elicitation tools were developed for complex contexts: 'wicked' problems with deep uncertainties: e.g., soft systems, cognitive maps and similar tools to elicit perceptions of relationships between entities and problem owners' and stakeholder's values ( [[#Keeney--1992|Keeney, 1992]] ; [[#Rosenhead--2001|Rosenhead and Mingers, 2001]] ).

| Soft elicitation tools and processes can be used to catalyse creative thinking about poorly understood contexts.

|}

<div id="17.3.1.1.2" class="h4-container"></div>

<span id="types-and-capacity-of-models-to-support-decision-making"></span>
===== 17.3.1.1.2 Types and capacity of models to support decision-making =====

<div id="h4-3-siblings" class="h4-siblings"></div>

‘Descriptive models’ of socio-biophysical systems and their responses to different drivers ( [[#Argyris--2017|Argyris and French, 2017]] ; [[#French--2018|French and Argyris, 2018]] ; [[#Saltelli--2020|Saltelli et al., 2020]] ) and ‘prescriptive models’, which capture the beliefs, values and objectives of decision makers and stakeholders ( [[#Parnell--2013|Parnell et al., 2013]] ; [[#Keisler--2014|Keisler et al., 2014]] ; [[#French--2018|French and Argyris, 2018]] ), provide the foundations of sense making ( ''high confidence'' ) and thereby influencing the options and choices available in the phase of analysis and exploration ( ''medium confidence'' ) ( [[#Gorddard--2016|Gorddard et al., 2016]] ).

Socio-biophysical models may be qualitative network models, statistical models or dynamic mathematical models ( [[#Melbourne-Thomas--2017|Melbourne-Thomas et al., 2017]] ). Qualitative network modelling can help assess the nature and consequences of the interactions, as well as facilitate understanding of possible structures to be used in dynamic models for assessing long-term adaptation options ( [[#Reckien--2013|Reckien et al., 2013]] ; [[#Reckien--2014|Reckien, 2014]] ; [[#Reckien--2014|Reckien and Luedeke, 2014]] ; [[#Symstad--2017|Symstad et al., 2017]] ). These approaches help articulate the direct and indirect effects of fixed, long-term engineering or structural adaptations. Dynamic stochastic modelling ( [[#Fulton--2014|Fulton and Link, 2014]] ; [[#Ianelli--2016|Ianelli et al., 2016]] ) has been used to assess short- to medium-term interactions of more dynamic and variable sectors, such as those with annual adjustments and management of water, agriculture, land and marine uses ( [[#Holsman--2019|Holsman et al., 2019]] ; [[#Hollowed--2020|Hollowed et al., 2020]] ; [[#Bahri--2021|Bahri et al., 2021]] ). On a longer time frame, scenarios are used to test long-term interactions but often with less variability and chance ( [[#Giupponi--2013|Giupponi et al., 2013]] ; [[#Adam--2014|Adam et al., 2014]] ; [[#Rosenzweig--2017|Rosenzweig et al., 2017]] ).

Many sensitivity analyses based on scenarios, including procedures to randomise across model uncertainty, relate to descriptive dynamic mathematical models with the user of the models characterised as an objective observer ( [[#Borgonovo--2016|Borgonovo and Plischke, 2016]] ; [[#Ferretti--2016|Ferretti et al., 2016]] ; [[#Symstad--2017|Symstad et al., 2017]] ; [[#French--2020|French, 2020]] ). Bayesian approaches enable these descriptive analyses to take account of the subjective choices in model construction and implementation ( [[#Abbas--2015|Abbas and Howard, 2015]] ; [[#Sperotto--2017|Sperotto et al., 2017]] ; [[#Jäger--2018|Jäger et al., 2018]] ; [[#Sperotto--2019|Sperotto et al., 2019]] ; [[#French--2020|French, 2020]] ). Organising descriptive analyses and deciding on a suitable option across a diversity of opinions among stakeholders use prescriptive processes, which can be supported with prescriptive modelling tools ( [[#Williamson--2012|Williamson and Goldstein, 2012]] ; [[#Gelman--2013|Gelman et al., 2013]] ; [[#Abbas--2015|Abbas and Howard, 2015]] ; [[#Dias--2018|Dias et al., 2018]] ; [[#Phan--2019|Phan et al., 2019]] ; [[#Hanea--2021|Hanea et al., 2021]] ). These approaches are subjective, in that they are constrained or directed by the particular views and emphases of the decision-making group ( [[#Gorddard--2016|Gorddard et al., 2016]] ). Not all tools are appropriate for all these activities.

Decision makers will be better able to choose decision-analytic methods when they have an understanding of the types, scale and breadth of uncertainties around the climate risk ( ''high confidence'' ) ( [[#Symstad--2017|Symstad et al., 2017]] ). The ''Cynefin'' framework ( [[#Snowden--2002|Snowden, 2002]] ; [[#French--2013|French, 2013]] ) is a policy-driven framework that broadly categorises the decision context of uncertainty within which decision makers and policy analysts may find themselves ( ''medium confidence'' ) ( [[#Hurlbert--2019|Hurlbert et al., 2019]] ; [[#Helmrich--2020|Helmrich and Chester, 2020]] ). As ''Cynefin'' has helped frame previous IPCC presentations on contexts of uncertainty ( [[#Hurlbert--2019|Hurlbert et al., 2019]] ) and has a community of practice to consult on its use ( [[#French--2020|French, 2020]] ), it is used here, also because it considers the uncertainty in knowledge around cause and effect in general terms, rather than specifically focusing on uncertainty in formal models. [[#Helmrich--2020|Helmrich and Chester (2020)]] show how ''Cynefin'' can be used to frame climate adaptation decision-making in the infrastructure sector.

The ''Cynefin'' contexts relate to how well the system is understood for knowing precisely the outcomes of actions that may be taken, ranging from known, knowable and complex to chaotic. If a context is known or knowable, then it will be possible to build sophisticated models and make sound predictions. If the context is complex and chaotic the outcomes of actions will be less predictable, no matter how complex the models may be, although more complex dynamic models may be useful to test ‘what if’ scenarios in these cases ( [[#Marchau--2019|Marchau et al., 2019]] ). Under complex and chaotic circumstances an ensemble of models and approaches may be needed to help categorise a satisfactory ‘solution space’ across the broad knowledge of relationships and dependencies, but will need to have iterative processes to update and refine adaptations as knowledge improves ( [[#Marchau--2019|Marchau et al., 2019]] ).

<div id="17.3.1.1.3" class="h4-container"></div>

<span id="uncertainty-and-attitudes-to-risk"></span>
===== 17.3.1.1.3 Uncertainty and attitudes to risk =====

<div id="h4-4-siblings" class="h4-siblings"></div>

Uncertainty does not just relate to what might happen given climate drivers or adaptations, but also to how much one values potential consequences ( [[#Butler--2016|Butler et al., 2016]] ; [[#Beven--2018a|Beven et al., 2018a]] ; Cross-Chapter Boc DEEP; [[#Beven--2018b|Beven et al., 2018b]] ; [[#French--2020|French, 2020]] ) ( ''high confidence'' ); the balance between how particular decision analyses address uncertainties relating to the external world (descriptive models) and those relating to the values driving the decision-making (prescriptive models) is important ( [[#Butler--2016|Butler et al., 2016]] ). Some analyses partially ignore uncertainties relating to the former in order to focus on conflicts in the values held by different stakeholders and help structure debate ( [[#Korhonen--2020|Korhonen and Wallenius, 2020]] ; [[#French--2020|French, 2020]] ), while others build very sophisticated models of the external world to predict potential consequences, but in doing so lose transparency and risk becoming untrustworthy black boxes to many stakeholders ( ''low confidence'' ) ( [[#Peterson--2020|Peterson and Thompson, 2020]] ).

Much of the readily available literature on how uncertainties affect decision-making relates to the uncertainty in the biophysical models, with a recognition that the choice of tools will be influenced by the types of uncertainty to be addressed ( [[#Le%20Cozannet--2017|Le Cozannet et al., 2017]] ; [[#Symstad--2017|Symstad et al., 2017]] ; [[#Beven--2018a|Beven et al., 2018a]] ; [[#Beven--2018b|Beven et al., 2018b]] ; [[#Durbach--2020b|Durbach and Stewart, 2020b]] ; [[#French--2020|French, 2020]] ). While terminology varies among disciplines, three types of uncertainty are important in understanding assessments of the future from descriptive models: epistemic (uncertainty in model construction relating to the lack of knowledge about the system being represented), analytic (the degree to which a model fits observations, and its accuracy) and stochastic (the natural variability or randomness in the system). The probability of an event arising in the future is determined from all three uncertainties, noting that stochastic uncertainty is a property of the system rather than a limitation of research ( [[#Le%20Cozannet--2017|Le Cozannet et al., 2017]] ; [[#Beven--2018a|Beven et al., 2018a]] ; [[#Beven--2018b|Beven et al., 2018b]] ).

Uncertainty in what constitutes a risk of concern is increasingly identified as important to consider when managing risk (Chapter 16; [[#Butler--2016|Butler et al., 2016]] ; [[#Prober--2017|Prober et al., 2017]] ; [[#French--2020|French et al., 2020]] ; [[#Reis--2020|Reis and Shortridge, 2020]] ). The uncertainty here arises from what is an acceptable risk. Acceptability relates to the value or importance of the consequence, which may include moral and ethical uncertainties ( [[#Prober--2017|Prober et al., 2017]] ), as well as how ambiguous the understanding of the consequence may be between different groups ( [[#Beven--2018a|Beven et al., 2018a]] ; [[#Beven--2018b|Beven et al., 2018b]] ). The development of strategies to ameliorate risk will benefit from considering these two uncertainties in specifying the risk to be managed ( [[#Prober--2017|Prober et al., 2017]] ; [[#French--2020|French et al., 2020]] ) because they can help set boundaries on a required likelihood of success, rather than simply casting stakeholders or decision makers as risk averse or risk tolerant, and can help identify and accept pathways of success ( [[#Gregory--2012|Gregory et al., 2012]] ). This can be important when decisions need to be made well in advance of the actions needing to take effect, such as for many climate risks (Chapter 1; Chapter 16; [[#17.2|Section 17.2.3]] ; Cross-Chapter Box DEEP in this Chapter).

Elicitation methods help reduce these uncertainties ( ''high confidence'' ) ( [[#Butler--2016|Butler et al., 2016]] ; [[#Prober--2017|Prober et al., 2017]] ; [[#Symstad--2017|Symstad et al., 2017]] ; [[#Beven--2018b|Beven et al., 2018b]] ). In addition, informal decision processes can assist in developing consensus in approaches and outcomes ( [[#Orlove--2020|Orlove et al., 2020]] ).

<div id="17.3.1.2" class="h3-container"></div>

<span id="decision-analytic-methods-used-in-decision-making-and-climate-risk-management"></span>
==== 17.3.1.2 Decision-Analytic Methods Used in Decision-Making and Climate Risk Management ====

<div id="h3-18-siblings" class="h3-siblings"></div>

Entities making decisions (countries, regions, organisations and individuals) select methods that best suit them in their context ( [[#Fünfgeld--2018|Fünfgeld et al., 2018]] ; [[#Shi--2019|Shi et al., 2019]] ; [[#French--2020|French, 2020]] ) ( ''high confidence'' ).

Classes of tools ( [[#Watkiss--2013|Watkiss and Hunt, 2013]] ; [[#French--2020|French, 2020]] ) include Bayesian methods, interval methods, decision-making under deep uncertainty (DMDU; see Cross-Chapter Box DEEP in this Chapter), cost–benefit analyses, multi-criteria decision analysis, elicitation and general decision support tools (Table 17.4). A summary guide for policy analysts and decision makers is presented in Table 17.4 to help identify the classes of decision-analytic methods that may be suitable for their context for managing climate risks. The table summarises how well the methods address the ''Cynefin'' context, the phase of decision-making, the types of uncertainties that exist through the decision-making process and the resources required. As terminology may vary between disciplines and research groups, suitable references to better explain the methods within the class are provided. Also, there may be overlap between the classes as individual methods are often paired with other methods to address specific requirements and approaches ( [[#Buurman--2016|Buurman and Babovic, 2016]] ; [[#Haasnoot--2019|Haasnoot et al., 2019]] ). In that respect, these methods are referred to in the next section discussing advances in the different approaches to managing climate risks.

Case studies in Table 17.4 describe the utility of classes of decision-analytic tools to facilitate decisions about climate adaptations (SM 17.2). These case studies are presented in Figure 17.8 according to the type of decision-making body and mapped according to their contribution to a decision outcome relative to the geopolitical scale of the actions being assessed. The effectiveness of these methods and tools in Table 17.4 in the context of climate change adaptation (Box 17.1) has yet to be evaluated.

Many published studies on the utility of decision-analytic methods in managing climate risks are theoretical, and therefore it is difficult to find studies on the value of analytic methods for underpinning final decisions on climate risk adaptation. Bayesian, Deep Uncertainty and elicitation methods and tools to support decision-making were the most easily located classes of methods to be used in different contexts (Figure 17.7), while the other classes were more oriented towards government processes. This result highlights a key gap at present in the need to have real-world experiences published and mapped for their utility for different tasks, thereby creating a resource for policymakers to identify suitable tools, such as in emerging communities-of-practice of decision practitioners ( [[#Watkiss--2013|Watkiss and Hunt, 2013]] ; [[#Street--2019|Street et al., 2019]] ; [[#French--2020|French, 2020]] ).

<div id="17.3.1.3" class="h3-container"></div>

<span id="approaches-to-support-decision-making"></span>
==== 17.3.1.3 Approaches to Support Decision-Making ====

<div id="h3-19-siblings" class="h3-siblings"></div>

The common approaches presented here are not undertaken in isolation and are often combined throughout, or applied at different stages of, a decision process, as illustrated in Figure 17.7.

<div id="17.3.1.3.1" class="h4-container"></div>

<span id="role-of-informal-processes"></span>
===== 17.3.1.3.1 Role of informal processes =====

<div id="h4-5-siblings" class="h4-siblings"></div>

Informal decision-making pervades decision-making in all contexts ( ''high confidence'' ) ( [[#Orlove--2020|Orlove et al., 2020]] ); decisions relating to climate change are affected not only by rational processes but also by many informal, often behavioural responses to the situation, some of which may not require formal processes. Informal processes were officially studied in only a few of the publications contributing to Figure 17.8, but all of the studies have hints to informal decision-making that pervades all levels of governance. Although there are not many concrete studies, citing roles of study participants can lead to a perception of a disconnect between the process and the outcome that resulted (see [[#17.5.1|Section 17.5.1]] for enablers of success).

Generally, while governance requirements may define the processes of formal deliberations and decision-making, informal deliberations will carry on in parallel, supported by social media, and these informal deliberations may be used to affect the outcome of the formal processes. Stakeholders may feel excluded from the formal deliberations either by governance structures or because they do not agree with their representatives. Conflicting value systems may cause some stakeholders to feel side-lined, particularly if some of the key decision makers are perceived holding different personal views and interests or to have engaged in political horse-trading, which connect independent decisions. There may be emotional responses, driven by poor comprehension of risk and probabilistic information, and potential for group biases or insularity of participants ( [[#Engler--2019|Engler et al., 2019]] ). Well-designed decision processes recognise the informal and seek to gain information from it without introducing bias ( ''medium confidence'' ) ( [[#French--2018|French and Argyris, 2018]] ).

<div id="17.3.1.3.2" class="h4-container"></div>

<span id="stakeholder-engagement"></span>
===== 17.3.1.3.2 Stakeholder engagement =====

<div id="h4-6-siblings" class="h4-siblings"></div>

Stakeholder engagement has become increasingly part of climate-relevant decision processes ( [[#Orlove--2020|Orlove et al., 2020]] ). The degree of stakeholder engagement ranges from instructive and consultative to cooperative, which are equivalent to information exchange, influence and partners in decision-making ( [[#Sen--2000|Sen, 2000]] ; Cattino and Reckien, in press). Since the AR5, climate change adaptation and resilience literature has seen an increase in participatory approaches that deepen engagement and overcome challenges, as well as making some assessments of their effectiveness (Newton [[#Mann--2017|Mann et al., 2017]] ; [[#Wamsler--2017|Wamsler, 2017]] ; [[#Esteve--2018|Esteve et al., 2018]] ), including structured interactions among different types of stakeholders and the use of place-based boundary organisations to strengthen the interactions and heighten the awareness of the institutional context. A higher degree of public participation can lead to more transformational adaptation as well as to higher ambition for local mitigation ( ''medium confidence'' ) ( [[#17.4.4.2|Section 17.4.4.2]] ; Cattino and Reckien, in press). Challenges to stakeholder participation are access to state-of-the-art science, capacity to recognise and respond to non-reliable or false climate science information, and the removal of cognitive and other biases ( ''high confidence'' ) ( [[#Gorddard--2016|Gorddard et al., 2016]] ; [[#Engler--2019|Engler et al., 2019]] ; [[#Fulton--2021|Fulton, 2021]] ).

Participatory and elicitation approaches, where the concerns and involvement of a broader range of interest groups and stakeholders are taken into account, can improve the effectiveness of decision-making ( ''medium confidence'' ) ( [[#Gregory--2012|Gregory et al., 2012]] ; [[#Cvitanovic--2019|Cvitanovic et al., 2019]] ). Participatory planning includes a variety of co-generative strategies and approaches (e.g., qualitative scenario or adaptation pathway development) through which goals and objectives, knowledge and strategy implementation and evaluation can be decided collaboratively between practitioners, policymaking, local interests and groups, and scientists ( [[#Butler--2016|Butler et al., 2016]] ; [[#Prober--2017|Prober et al., 2017]] ; [[#Symstad--2017|Symstad et al., 2017]] ). Specifically, for climate change adaptation, these decision-making strategies can incorporate expert, Indigenous and local knowledge ( ''high confidence'' ) (Cross-Chapter Box INDIG; [[#Gustafson--2016|Gustafson et al., 2016]] ). The challenge will be to bring together these different actors, as stakeholders tend to act within rather than among systems and procedures, and it is important that platforms are developed to integrate data effectively ( [[#Rizzo--2020|Rizzo et al., 2020]] ). Furthermore, reflexive and iterative risk management may further ensure acceptance by participating groups.

Bayesian methods are increasingly used in advancing approaches for decision-making and support in climate adaptation ( [[#Sperotto--2017|Sperotto et al., 2017]] ), by being able to include stakeholder and decision-maker perceptions and biases ( [[#Dias--2018|Dias et al., 2018]] ; [[#Engler--2019|Engler et al., 2019]] ; [[#Phan--2019|Phan et al., 2019]] ; [[#Fulton--2021|Fulton, 2021]] ) in a transparent modelling environment, thereby facilitating consensus and impartiality ( ''medium confidence'' ) ( [[#Catenacci--2013|Catenacci and Giupponi, 2013]] ; [[#Gelman--2017|Gelman and Hennig, 2017]] ). Increasing computational efficiency means that these methods can enable different approaches to be addressed and different descriptive and prescriptive models to be included within a single probabilistic environment, which also can be updated in iterative processes ( ''high confidence'' ) (Table 17.4; [[#Sperotto--2017|Sperotto et al., 2017]] ; [[#Phan--2019|Phan et al., 2019]] ).

<div id="17.3.1.3.3" class="h4-container"></div>

<span id="scenario-analyses"></span>
===== 17.3.1.3.3 Scenario analyses =====

<div id="h4-7-siblings" class="h4-siblings"></div>

Scenarios are described in SR1.5 ( [[#IPCC--2018a|IPCC, 2018a]] ) and SRCCL ( [[#IPCC--2019b|IPCC, 2019b]] ) as a description of how the future may develop based on a coherent and internally consistent set of assumptions about key driving forces (e.g., rate of technological change, prices) and relationships. Scenarios are neither predictions nor forecasts but are used to provide narratives and trajectories equipped with alternate outcomes. SR1.5 and the SRCCL describe a range of scenarios methods and how scenarios are used to guide risk management decision-making. Scenario analysis includes a range of potential future conditions from low-end and mid-range to high-end projections. Scenarios can also include a temporal component, that is, short term, medium term and long term, as defined in the SROCC ( [[#IPCC--2019c|IPCC, 2019c]] ).

Scenarios and pathways, combined with elicitation methods, are becoming widely used to assess adaptation and resilience strategies ( ''high confidence'' ) ( [[#Butler--2016|Butler et al., 2016]] ; [[#Prober--2017|Prober et al., 2017]] ; [[#Symstad--2017|Symstad et al., 2017]] ; [[#Lawrence--2019|Lawrence et al., 2019]] ; [[#Phan--2019|Phan et al., 2019]] ; [[#Sperotto--2019|Sperotto et al., 2019]] ; [[#Haasnoot--2020a|Haasnoot et al., 2020a]] ). They can support the consideration of a wide range of alternative possible futures ( [[#Catenacci--2013|Catenacci and Giupponi, 2013]] ; [[#Jäger--2018|Jäger et al., 2018]] ), enabling identification of potential path dependencies caused by adaptation options ( ''high confidence'' ) ( [[#Pretorius--2017|Pretorius, 2017]] ; [[#Haasnoot--2020a|Haasnoot et al., 2020a]] ). They can also increase the willingness of stakeholders to consider costly actions, by placing them within broader sequences of action ( ''limited evidence'' ) ( [[#Barnett--2014|Barnett et al., 2014]] ). The development, consideration and understanding of scenarios can be enhanced by using visualisation tools to better display storylines, enabling the discussion of alternative futures by participants in decision-making processes ( ''limited evidence'' ) ( [[#Winters--2016|Winters et al., 2016]] ).

<div id="17.3.1.3.4" class="h4-container"></div>

<span id="evaluating-trade-offs-robust-decision-making-and-deep-uncertainty"></span>
===== 17.3.1.3.4 Evaluating trade-offs, robust decision-making and deep uncertainty =====

<div id="h4-8-siblings" class="h4-siblings"></div>

Trade-offs are pervasive in decision-making for climate change adaptation, including between adaptation and mitigation, economic/social and environmental cost including distributional/equity considerations, affordability and risk reduction, short- and long-term consequences, and spatial variations ( [[#Borgomeo--2016|Borgomeo et al., 2016]] ; [[#Hudson--2016|Hudson et al., 2016]] ; [[#Gil--2018|Gil et al., 2018]] ; [[#Landauer--2019|Landauer et al., 2019]] ).

Trade-offs are often directly compared in cost–benefit analyses which require rigorous estimation of the monetised costs and benefits, where monetisation is feasible and values uncontested (such as for infrastructure) ( ''high confidence'' ) ( [[#de%20Ruig--2019|de Ruig et al., 2019]] ; Table 17.4). Other tools can be employed, such as cost-effectiveness analysis and multi-criteria analysis in order to draw stakeholders into the process ( [[#Posner--2004|Posner, 2004]] ; [[#Matheny--2007|Matheny, 2007]] ; [[#Mechler--2016|Mechler and Schinko, 2016]] ). Stakeholder participation in measuring costs and benefits and in the modelling can aid the process ( [[#Doukas--2020|Doukas and Nikas, 2020]] ).

Logic trees include a range of decision protocols and multi-criteria rules, either based on quantitative or qualitative categories ( [[#Roncoli--2016|Roncoli et al., 2016]] ), often termed multi-criteria analyses. The concept of the logic tree has been increasingly applied in climate risk decision-making contexts ( [[#Nikas--2018|Nikas et al., 2018]] ).

Since the AR5, robust decision-making methods are increasingly used to account for deep uncertainty in many climate-related risks ( ''high confidence'' ) ( [[#Marchau--2019|Marchau et al., 2019]] ; Table 17.4), particularly when decisions need to be made well in advance of when the adaptations need to be implemented (Cross-Chapter Box.5 in SROCC Chapter 1; Cross-Chapter Box DEEP in this Chapter). Reducing risk and building resilience under the context of these types of wicked problems require asking ‘what if’ questions about the future, remaining flexible in the face of uncertainty and seeking out policies that provide good outcomes no matter what the future climate might bring ( ''high confidence'' ) ( [[#17.6|Section 17.6]] ; e.g., [[#Larson--2015|Larson et al., 2015]] ; [[#Bhave--2016|Bhave et al., 2016]] ; [[#Bhave--2018|Bhave et al., 2018]] ). In these cases, trade-offs can be assessed and options can be prioritised through iterative decision-making processes, such as multi-criteria decision-making, robust decision-making and dynamic adaptation pathway planning ( ''high confidence'' ) (Table 17.4; [[#Kwakkel--2014|Kwakkel et al., 2014]] ; [[#Kwakkel--2016|Kwakkel et al., 2016]] ; [[#Shortridge--2016|Shortridge et al., 2016]] ; [[#Lawrence--2017|Lawrence and Haasnoot, 2017]] ; [[#Haasnoot--2019|Haasnoot et al., 2019]] ; [[#Lempert--2019|Lempert, 2019]] ; [[#Roelich--2019|Roelich and Giesekam, 2019]] ; [[#Haasnoot--2020a|Haasnoot et al., 2020a]] ). They can address limitations of data-intensive robust decision-making in developing countries ( [[#Daron--2015|Daron, 2015]] ), use proxy data to enable the use of robust decisions in data-scarce contexts ( [[#Shortridge--2016|Shortridge and Guikema, 2016]] ; [[#Ahmad--2019|Ahmad et al., 2019]] ), incorporate multiple-objectives into robust decision-making ( [[#Singh--2015|Singh et al., 2015]] ), and supplement pathway development with real options analysis ( [[#Buurman--2016|Buurman and Babovic, 2016]] ; [[#Smet--2017|Smet, 2017]] ; [[#Haasnoot--2019|Haasnoot et al., 2019]] ; [[#Lawrence--2019|Lawrence et al., 2019]] ). Often, there are close synergies between the application of these methods and using scenario analyses ( [[#Workman--2021|Workman et al., 2021]] ).

<div id="17.3.1.3.5" class="h4-container"></div>

<span id="adaptive-feedback-management"></span>
===== 17.3.1.3.5 Adaptive feedback management =====

<div id="h4-9-siblings" class="h4-siblings"></div>

Iterative decision-making requires that the implementation of adaptations be reviewed to determine whether the adaptation effectively achieved the objectives, and whether adjustments or additional actions were required ( [[#17.5|Section 17.5]] ). Adaptive feedback management is an approach to managing dynamic climate risks by designing a field monitoring programme to provide data to an assessment procedure which in turn advises on what adjustments need to be made to a ‘control action’, all of which are part of the adaptation to be implemented ( [[#Hurlbert--2019|Hurlbert et al., 2019]] ; Figure 17.7). Adaptive feedback management is more able to account for the dynamic nature of risk and the future emergence of unforeseen risks because of the active design of how to adjust the management approach ( [[#Dickey-Collas--2014|Dickey-Collas, 2014]] ).

Adaptive feedback management is important for managing climate risks that fall within the ''Cynefin'' context of chaos, relying on observations and indicators to learn about the system and to trigger actions ( ''medium confidence'' ) ( [[#Helmrich--2020|Helmrich and Chester, 2020]] ). It has been a valued approach for managing wildfish fisheries in many oceans ( ''high confidence'' ) ( [[#Fulton--2019|Fulton et al., 2019]] ; [[#Hollowed--2020|Hollowed et al., 2020]] ; [[#Bahri--2021|Bahri et al., 2021]] ) and is important for responding to the challenges of climate change ( ''high confidence'' ) ( [[#Holsman--2019|Holsman et al., 2019]] ; [[#Hollowed--2020|Hollowed et al., 2020]] ; [[#Bahri--2021|Bahri et al., 2021]] ).

While the benefits of investment in data and assessments can outweigh the costs of implementation ( ''low confidence'' ) ( [[#Fulton--2019|Fulton et al., 2019]] ), the implementation may take time when resources are limited, particularly in developing nations, where low-cost approaches will be needed for deciding on pathways for adaptation ( [[#Bhave--2016|Bhave et al., 2016]] ; [[#Shortridge--2016|Shortridge et al., 2016]] ).

Iterative decision-making and adaptive feedback management meet when the feedback management procedure is reviewed in total for its effectiveness in one of the review and adjustment iterations. At present, a common approach for assessing different adaptation options and their interaction is using, for example, scenarios in dynamic models ( [[#Adam--2014|Adam et al., 2014]] ; [[#Girard--2015|Girard et al., 2015]] ). An emerging field in adapting fisheries to climate change is to embed the decision-making system in the scenario models in order to assess the capability of feedback management (decision-making, monitoring and capacity for adjustment of the options over time) to achieve satisfactory trade-offs among the objectives of the different stakeholders ( ''medium confidence'' ) ( [[#Melbourne-Thomas--2017|Melbourne-Thomas et al., 2017]] ; [[#Holsman--2019|Holsman et al., 2019]] ; [[#Hollowed--2020|Hollowed et al., 2020]] ). This method can enable prospective evaluation of future whole-of-management scenarios described in this chapter.

<div id="17.3.2" class="h2-container"></div>

<span id="integration-across-portfolios-of-adaptation-responses"></span>