Department Environmental Social Sciences

Publications

ValueDecisions, a web app to support decisions with conflicting objectives, multiple stakeholders, and uncertainty

Complex environmental and public policy decisions profit from structured procedures such as multi-criteria decision analysis (MCDA). To support such decisions, the new open source application ValueDecisions provides advanced analysis and visualization with no programming expected from users. Based on multi-attribute value theory (MAVT), it offers analysis for decisions with conflicting and interacting objectives, multiple stakeholders, and uncertain consequences of options. Programmed in R, the shiny web framework makes it accessible via a graphical user interface in the browser. We exemplify using ValueDecisions for a wastewater infrastructure planning case in the Paris region. We surveyed preferences of 655 citizens and conducted sensitivity analysis of preference parameters. The best management options were robust across a range of preference profiles and assumptions. To evaluate the app, we developed a novel usability test based on the ISO standard for software quality and surveyed students using ValueDecisions for case studies.
Haag, F.; Aubert, A. H.; Lienert, J. (2022) ValueDecisions, a web app to support decisions with conflicting objectives, multiple stakeholders, and uncertainty, Environmental Modelling and Software, 150, 105361 (19 pp.), doi:10.1016/j.envsoft.2022.105361, Institutional Repository

Recommendations for online elicitation of swing weights from citizens in environmental decision-making

There is a growing demand for public participation in environmental decision-making. However, it is unclear how a large number of citizens can best engage in such complex public policy decision processes. This need from the civil society challenges the OR community to develop online decision-making tools. This article reports on a feasibility assessment of swing weight elicitation, implemented online, for real-world decisions about future wastewater infrastructure. Eliciting weights with the swing method is common in MAVT/MAUT, but not online. A total of 298 affected citizens from the Paris region answered the online swing weight elicitation survey. Another 357 citizens directly rated objectives. Three aspects of learning in the context of MCDA were considered: did participants learn facts about the wastewater topic? Did they comply with the swing elicitation process, i.e. follow the instructions? Did participants learn about their preferences? Factual learning was limited. Process compliance was really low (12%), leading to a number of recommendations for improving the interface for online swing weight elicitation. The collected preferences differed statistically significantly between the compliant and non-compliant participants, and also between the non-compliant and direct rating respondents. This emphasised the effect of the elicitation method on preference construction. Moreover, more participants experienced a strengthening of pre-existing opinions than a change in opinion, and most reported being uncertain about their answers. This calls for better understanding process learning and preference construction. We discuss our developed procedure for online swing weight elicitation, recommend ways to improve swing online surveys, and suggest interesting future research lines that would allow empirically verifying our propositions.
Aubert, A. H.; Esculier, F.; Lienert, J. (2020) Recommendations for online elicitation of swing weights from citizens in environmental decision-making, Operations Research Perspectives, 7, 100156 (13 pp.), doi:10.1016/j.orp.2020.100156, Institutional Repository

Gamified online survey to elicit citizens' preferences and enhance learning for environmental decisions

Multi-Criteria Decision Analysis (MCDA) requires a critical step, namely to elicit individual preferences. On the basis of learning theories, we formalize preference construction as learning about facts and values, and as a process; we also conceptualize an online preference elicitation survey that offers learning loops to increase factual learning and support preference construction. Another originality is gamification. Game elements (a narrative and non-player characters as motivational affordance) keep respondents engaged in the demanding task of weight elicitation. Our tool enables broad public participation in MCDA, allowing reliable online preference elicitation. The survey concept was tested with 107 students and a control treatment. Quantitative and qualitative data indicate that the concept works. Participants’ factual knowledge increased. The survey helped students to learn about their own preferences concerning the importance of objectives. The practical implication is that weighting can be reliably elicited by online surveys. Participants reported a positive experience; further ways to improve it are thoroughly discussed.
Aubert, A. H.; Lienert, J. (2019) Gamified online survey to elicit citizens' preferences and enhance learning for environmental decisions, Environmental Modelling and Software, 111, 1-12, doi:10.1016/j.envsoft.2018.09.013, Institutional Repository

A review of water-related serious games to specify use in environmental Multi-Criteria Decision Analysis

Serious games and gamification are nowadays pervasive. They are used to communicate about science and sometimes to involve citizens in science (e.g. citizen science). Concurrently, environmental decision analysis is challenged by the high cognitive load of the decision-making process and the possible biases threatening the rationality assumptions. Difficult decision-making processes can result in incomplete preference construction, and are generally limited to few participants. We reviewed 43 serious games and gamified applications related to water. We covered the broad diversity of serious games, which could be explained by the still unsettled terminology in the research area of gamification and serious gaming. We discuss how existing games could benefit early steps of Multi-Criteria Decision Analysis (MCDA), including problem structuring, stakeholder analysis, defining objectives, and exploring alternatives. We argue that no existing game allows for preference elicitation; one of the most challenging steps of MCDA. We propose many research opportunities for behavioral operational research.
Aubert, A. H.; Bauer, R.; Lienert, J. (2018) A review of water-related serious games to specify use in environmental Multi-Criteria Decision Analysis, Environmental Modelling and Software, 105, 64-78, doi:10.1016/j.envsoft.2018.03.023, Institutional Repository

Serious Games as an element of Multi-Criteria Decision Analysis

Structured methods are needed to support environmental policy decisions, including in the water sector. Multi-criteria decision analysis – or MCDA – is one of them, aiming at finding consensus between stakeholders with differing opinions. Traditionally, stakeholders are asked during interviews about their preferences concerning the outcomes of a set of decision options. Particularly, they have to give valuations by assigning weights to different objectives which are important in this decision. This is a tricky job as overall consequences of a weight are hard to understand, as people readily run into systematic biases during preference formulation processes, and as the interviewer might influence the interviewee when informing about such choices.

The project aims at introducing serious games, i.e. games including a non-entertaining purpose, for this phase when stakeholders state their preferences in the decision making process.

The study consists of two parts:

  • Can serious games as an element of the MCDA process facilitate the elicitation of preferences, and can visualization of results as part of the game be used as a consistency check?
  • Can serious games in MCDA enable involvement of a larger public, both numerically and in terms of diversity? Would that be beneficial for reaching a consensus decision?

A literature review on serious games used in environmental management will allow to identify game characteristics that would be needed to serve our purpose: the formulation of preferences and valuation processes in decision making and the involvement of a larger public.

Based on the review, a game – or set of games – will be tested in applied case studies, allowing evaluation of the game characteristics with regards to our aims. This will be done by comparing outcomes of different approaches (with games VS without games). The players behavior will also be a focus, aiming to observe if there is a preference typology. Pros and cons of implementing games in MCDA should be clearly stated to consider future adaptation of MCDA.

Team

PD Dr. Judit Lienert Group Leader, Cluster: DA (Decision Analysis) Tel. +41 58 765 5574 Send Mail

Contact

Information

Project start: October 2015

Project duration: 2 years

Project type: Eawag postdoc position