Multi-Criteria Decision Analysis to Support the Remediation of Polluted Soils: A Review of Case Studies

Abstract

For the successful remediation of polluted sites, priority setting among the possible remediation technologies is of major importance. The related decisions are typically conditioned by a limited set of alternative remediation techniques and multiple, often contradicting criteria. These characteristics make the decision problem suitable for applying a formalised discrete multi-criteria decision analysis (MCDA). This paper reports on the outcome of a systematic review of articles published between 1995 and 2020 encompassing 43 MCDA applications to support the selection of the remediation technique for polluted soils. It focuses on the comparison between implementations of the MCDA methodology. The review identifies four gaps where progress can be made to mobilise the full strength of the MCDA methodology to support the remediation of polluted soils: (i) early stakeholder engagement, (ii) inclusion of social criteria, (iii) an informed choice of the weighting and aggregation method, (iv) and sensitivity analysis.

1. Introduction

Pollution is one of the main threats affecting soils and the ecosystem services they provide. Soil pollution can be local or diffuse and originate from various natural and artificial sources, such as industrial processes, mining, waste treatment, agriculture, extraction and processing of fossil fuels, accidents leading to spills of chemical or radioactive pollutants, transport, or military actions. As described in [1], soil pollution causes ecosystem impairments, risks to human health, and negative social and economic impacts. A detailed description of the most common organic and inorganic (trace elements and radionuclides) contaminants is provided in [1], together with their impacts on humans and ecosystems. According to the European Environment Agency, only in Europe, there are more than 2.8 million potentially contaminated sites, and only 8.3% of the registered sites had been remediated as of 2016 [2]. Globally, anthropogenic pollution of soils has become a widespread problem that continues to grow [3].

In 2017, the United Nations Environmental Assembly (UNEA-3) adopted a resolution calling for accelerated action and cooperation to address soil pollution [4]. This initiative and numerous other regional and national endeavours reflect the increased concern regarding polluted soils worldwide. In many cases (e.g., heavy metals or long-lived radioactive contaminants), soil pollution is persistent and requires intensive and active management to be reduced in a reasonable period. Historically, soil remediation projects have focused on the single objective of reducing contaminant concentrations below regulatory standards in the shortest possible time [5]. Therefore, in most cases, the direct costs, time, and achievable risk reduction are the only criteria involved in the choice of the remediation approach [6]. However, this may not be the most cost-effective or socially acceptable solution. In recent years, there has been a paradigm shift towards more comprehensive approaches that consider a wider range of environmental, economic, and social impacts [5,7]. This reflects the growing attention toward green and sustainable remediation [8,9,10,11]. As defined by the International Standard ISO 18504:2017 [12], sustainable remediation is “the elimination and/or control of unacceptable risks in a safe and timely manner, whilst optimising the environmental, social and economic value of the work”.

To optimise decisions on remediation actions, a broad range of potentially conflicting economic, technical, environmental, and social aspects must be considered simultaneously. As a result, inherent trade-offs must be considered to ensure socially responsible solutions and the sustainability of the remediation outcome [13].

In line with sustainable development principles, the process of balancing social, environmental, and economic impacts also has to account for the plurality of values and perspectives of relevant stakeholders, ensuring that the residual risk is acceptable, creating opportunities for effective stakeholder participation, and ensuring transparency in decision making [14].

Researchers and practitioners increasingly highlight the need for more intensive interactions with stakeholders in environmental remediation decision-making; they argue that environmental decisions are scientific and political [15]. One-way provision of information to stakeholders and potentially affected people, commonly referred to as the decide–announce–defend strategy, is not only questionable from an ethical point of view but has also been proven ineffective and inadequate to address the uncertainty and controversy surrounding contamination issues [16]. Today, a more collaborative process with various stakeholders is advocated, as this not only enhances “the legitimacy of decisions and reduce[s] the level of conflict” [17] but also increases the quality of decisions in various ways, not in the least by improving the projects’ outcome [15,18]. Balasubramaniam et al. [19] and others advocated the involvement of a diverse set of actors ranging from civil society to regulators, decision-makers, implementers, and scientific and technical experts.

However, when various stakeholders with conflicting beliefs, interests, perceptions, and concerns are involved, the remediation project becomes more challenging to define and manage [20]. To tackle these challenges, it is recommended to use a formal decision analysis framework to allow for a structured, transparent, and sound decision process. Furthermore, if the decision-makers are challenged to convert expert assessments, stakeholder perceptions, and the inevitable degree of uncertainty into a robust commitment to remediation, such reproducible methodologies become a requirement.

Traditional decision support tools, such as cost–benefit analysis (CBA) or cost-effectiveness analysis (CEA), offer structured approaches to decision-making but cannot account for the multiple dimensions and stakeholders that should inform decisions on contaminated site management. CBA is rooted in the field of economics and aims to find the alternative that will achieve the greatest overall societal welfare. Consequently, it involves monetising all socio-economic costs and benefits attributed to every alternative [21]. Within CEA, the cost is measured per unit of effect, which enables a comparison of alternatives with respect to the costs of achieving the same outcome. However, criteria with different dimensions, such as social and environmental impacts, cannot be easily condensed into simple monetary expressions as CBA and CEA require [22]. Moreover, having economic efficiency as the guiding principle in CBA and, to a large extent CEA, stakeholders’ preferences are only included indirectly, expressed as consumers’ preferences on the market [23]. Principles such as equity or sustainability are, therefore, not compatible with these tools.

To overcome these limitations, multi-criteria decision analysis (MCDA) has been proposed as a suitable framework for environmental decision-making. MCDA supports a structured and inclusive decision process, being able to balance a plurality of preferences and socio-technical dimensions without the need to bring these to a common scale (e.g., monetary) [22,24]. In addition, it supports the direct inclusion of stakeholders’ preferences.

Initially developed in the field of operations research, MCDA is an umbrella term for a variety of decision-aid methods that support the decision-maker by describing and evaluating decision options with respect to multiple (qualitative or quantitative) criteria, representing several points of view. Each method applies specific procedures to involve stakeholders, elicit and evaluate inputs, aggregate information, and formalise recommendations. These different approaches have distinct information requirements and computational complexity that depend heavily on the assumptions about the decision-making process, the level of stakeholder engagement, the time pressure, and the data requirements [25,26,27]. If MCDA is used in an inclusive manner, as shown by [28], it can offer a scientifically sound as well as inclusive decision framework for the management of contaminated sites, addressing the previously mentioned challenges [21,22,23].

Although many variants of the MCDA framework exist, it could be argued that most approaches involve three main stages: (1) Description of the decision context and structuring of the problem; (2) Analysis of the alternatives with respect to a number of criteria; and (3) Ranking alternatives, selecting a subset, or selecting one preferred alternative among the candidates (Figure 1). The first stage aims to establish a shared understanding of the decision problem among the stakeholders. The problem is structured by identifying all possible alternative solutions and related evaluation criteria. The subsequent analysis stage includes assessing preferences among criteria (e.g., by weighting the criteria) and comparing alternatives based on aggregating preferences and scorings into an overall performance for each alternative. This second stage may encompass an analysis of the sensitivity and robustness of the outcome to the set model parameters and the related criteria scores. In the final selection stage of the MCDA, information from the previous stages is combined to formulate recommendations, preferably leading to an actual commitment to action [29].

In MCDA, various terms are used by different authors to refer to similar concepts. In this paper, we adhere to the terminology of [30], whereby the set of possible remediation options is a finite number of explicitly stated alternatives, while the aspects used to evaluate the alternatives are referred to as attributes. Here, a criterion is an attribute together with a direction of preference (e.g., to be minimised or maximised). Therefore, the term MCDA will be used in the following sections to represent the previously defined framework.

The application of MCDA frameworks has been the subject of reviews in many different fields, such as nature conservation [29], mining and processing of mined minerals [31], sustainable development [32], and management of contaminated sites [24]. More recently, reviews of MCDA applications to environmental problems were carried out by [26], and a follow-up study was conducted by [33]. These showed an increased interest in using multi-criteria decision-aiding approaches for the remediation of contaminated soils. While these review papers contain valuable information on the decision context and the MCDA frameworks, they lack a more detailed description of the studied cases. Additionally, Huysegoms et al. [34] provided an extensive review of the available decision support tools and their characteristics to support sustainable remediation of contaminated sites. Although the latter review addresses the potential use of these tools, it does not directly describe whether and how these tools were implemented to support environmental remediation projects.

This paper reports on a systematic review of literature showcasing the role of MCDA as a framework to support the remediation and management of polluted soils. In contrast to previous studies, it aims to provide an analysis of how and where MCDA was used in practice to address the remediation of polluted soils and report on its specific implementation using a generalised reporting structure. The analysis is structured along the main considerations in MCDA, as depicted in Figure 1. Section 2 describes the protocol that was applied to include or exclude case studies for this review. Section 3 shows the results of the literature review, and Section 4 and Section 5 provide a discussion and conclusions about our findings.

2. Review Methodology

To ensure the objectivity and reproducibility of the review, a systematic approach in line with [31] was used to select case studies. It encompasses the application of a search string to three literature databases and three sequential exclusion steps, as shown in Figure 2 and described in the following paragraphs. The search was conducted for peer-reviewed literature between 1985 and 2020.

The search string was composed of two sets of search terms, with elements in each of the two sets linked by the OR operator; the first included keywords capturing terms linked to decision-making and MCDA, resulting in the following set of search terms: “decision support” OR “decision approach” OR “decision analysis” OR “multi-criteria decision” OR “multi-objective decision” OR “decision tool” OR “evaluation tool$” OR “evaluation strategies” OR mcd$ OR decision*aiding OR ranking OR “decision making” OR “decision framework” OR TOPSIS OR ELECTRE OR PROMETHEE OR AHP OR ANP. The second contained keywords specifically addressing the remediation of polluted soil, resulting in the following set of search terms: “soil remediation” OR “land remediation” OR “site remediation” OR “environmental remediation” OR “contaminated site$” OR “contaminated soil$” OR “contaminated land” OR “remedial measures”. The final search string consisted of the two sets combined by the AND operator.

To determine the performance of the search string, a set of target papers was created based on the authors’ best available initial knowledge. The set consists of papers judged as particularly relevant from the first explorative search phase. The validity of the search string was determined based on the share of papers from the target paper list that were returned. Several measures were implemented to ensure consistency and minimise bias during the paper screening phase of the review. Multiple independent reviewers were involved in screening the papers. Prior to beginning the actual screening process, a training session was conducted where reviewers practised applying the inclusion and exclusion criteria on a pilot set of papers. This training helped to align the understanding and application of these criteria across all reviewers, promoting consistency in the selection process.

The three electronic academic databases that were searched based on title and keywords were Web of Science Core Collection (http://www.webofknowledge.com, accessed on 17 September 2020), Science Direct (http://www.sciencedirect.com/, accessed on 17 September 2020), and Scopus (http://www.scopus.com/, accessed on 17 September 2020). Combining the search results from all three, the complete set of preselected target papers was retrieved. Inclusion and exclusion are important in any systematic literature review. Figure 2 shows the hierarchy of the three steps performed. After completing this three-step selection procedure, out of 835 initial papers, 56 independent case studies were retained. This review focuses on scientific peer-reviewed publications in English. Papers addressing the same polluted site with multiple sets of weights and/or different MCDA techniques were considered as one single independent case study.

3. Results

A total of 56 case studies were identified to support the remediation of polluted soils, of which 43 cases were specifically about the choice among remedial techniques addressing soil pollution on a local scale. The other 13 cases were about the priority setting among different polluted sites, targeting the regional scale. This paper includes 43 case studies addressing decisions regarding remedial techniques. The 43 cases were assessed against the six steps of the generalised MCDA framework displayed in Figure 1. While not fully exhaustive, this set of 43 case studies was considered sufficiently large to gain valuable insights for supporting and optimising the use of MCDA in soil remediation. The complete list of 43 selected case studies will be made available by the authors upon request.

3.1. Decision Context

3.1.1. Type of Decision Problem

The first stage in the MCDA methodology consists of framing and structuring the decision problem. In the scope of environmental remediation, according to [35] and later elaborated by [25], MCDA is fit to tackle five types of decision problems: (i) the choice of the best (set of) alternatives, (ii) the ranking of alternatives, (iii) the classification of alternatives, (iv) the establishment of a portfolio of alternatives satisfying certain constraints, and (v) the description of alternatives. For each case study, we therefore clarified which type of recommendations are provided to support decisions in managing polluted soils. All papers, except for two, focused on ranking the decision alternatives. Solving this type of problem has the advantage that it can easily help to address the choice problem (select the best alternative) in a well-advised manner. The problem type addressed in the two remaining case studies [36,37] was the portfolio problem. For a portfolio of remedial actions, i.e., the optimal sequence/combination of remediation actions in time, different aspects of the remedial technology can become more significant. Due to future data acquisition, progressive insights into the contamination could change/reverse the direction of remediation management in the future. Therefore, the use of irreversible techniques can become less favoured when dealing with portfolios compared to single actions.

3.1.2. Inclusion of Stakeholders in the Evaluation of the Problem Framework

The case studies were analysed to understand which stakeholders were involved from what stage onwards to contribute to determining the decision context and defining the decision criteria and alternatives. Stakeholders can be defined as all parties being affected by, having an interest in, or being involved in the remediation decision-making or the remediation work [17]. Despite the potential advantages of involving a broad set of stakeholders from the beginning of the decision-making process, it was observed that the early involvement of non-technical stakeholders in the reported remediation projects was limited. Only in 6 of the 43 examined cases was a broad range of stakeholders involved from the beginning [38,39,40,41,42,43]. However, the engagement of stakeholders from an early stage of the project could help to determine the site’s future use, specify the remediation goals, the feasible remediation alternatives, and the evaluation criteria and their importance [18]. For instance, in [38], three stakeholder groups were engaged: residents, local interest groups, and non-resident sediment experts. Stakeholders are consulted using various approaches, such as focus groups, surveys, citizen panels, or workshops.

3.2. Evaluation Criteria

For the MCDA approach to evaluate alternatives with respect to a broad scope of aspects, a diverse set of evaluation criteria should be determined. These evaluation criteria are the fundamental building blocks for describing, choosing among, ranking, or sorting the decision alternatives. Each criterion should be clear, measurable, and relevant to the underlying goal, while the criteria should be minimal, complete, non-decomposable, non-redundant, and independent [44]. Criteria should be determined by all actors involved from the beginning in the decision process (stakeholders, experts) and are to be based on specific field knowledge, literature, or experience. For our review, evaluation criteria were grouped into five general classes, allowing for the analysis of their prevalence in the case studies (

Table 1. Relative occurrence of criteria over the five criteria classes for all case studies (n = 43).

General Criteria Class	Description	Relative Occurrence of the Class [%]
Economic	The costs and benefits that can be monetised from the alternatives.	98%
Human health risk	The human health risk or effect on human health of the alternative.	74%
Social	The social impact that the alternative has on the surrounding communities.	56%
Technical	The technical needs and impacts of the alternative.	84%
Environmental	The effect of the alternative on the local and regional environment in physical, chemical, and/or biological terms.	72%

). The human health criteria were separated from the social criteria to stress the difference between direct health effects from exposure to a contaminant and other social aspects affecting the community. Furthermore, a more in-depth assessment of the identified criteria was conducted for each general class. The class counter was increased when at least one criterion from that class was included in the case study. The exact criteria used to evaluate alternatives can differ between projects, as they are very case-specific depending on the contamination type, social context, and available data. The review showed that the number of criteria used in the case studies varied from 2 to 16, with a median of 6. In 37 cases, criteria from at least three different classes were considered; in 28 cases, criteria from four different classes were present.

Table 1 shows that each of the five criteria classes is present in more than half of the case studies, albeit to various extents. Economic criteria are nearly always present, followed by technical, human health, and environmental criteria. In contrast, social criteria were observed the least. Most case studies did not justify why the used criteria were selected. Nine of them reported that criteria selection was inspired by a sustainable remediation framework [42,45] or by national or regional regulations or guidance [46,47]. The framework most referred to in the case studies was the Sustainable Remediation Forum (SuRF) [48,49]. Other frameworks considering positive and negative environmental, economic, and social effects were the USEPA Green Remediation Programme [50], the Network for Industrially Contaminated Land in Europe [8], the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA), and the five year US superfund review [51]. To analyse the distribution of criteria within the five classes, the relative occurrence of criteria with respect to the total number of cases in the general class was assessed.

3.2.1. Economic Criteria

From

Table 2. Relative occurrence of specific economic criteria to evaluate the alternative remedial technologies compared to the total number of cases implementing economic criteria (n = 42).

Economic Criteria	Description	Example Indicator	Relative Occurrence of the Criterion [%]
Direct cost of alternative	The cost of complete implementation of the remediation technique from site investigation up until waste management. [Minimise]	Remedial cost [$/m2]	95%
Direct benefits of alternative	The potential economic benefits directly attributed to the use of this technique. [Maximise]	Improved agricultural yield after contaminant removal [ton/h]	19%
Indirect costs of alternative	The indirect cost or the potential negative influence caused by the remediation alternative. The public opinion of the site after remediation can have indirect financial effects on other sectors (e.g., tourism). [Minimise]	Decreased provision of other ecosystem services [$]	7%
Indirect benefits of alternative	The indirect benefit or the potential positive influence caused by the remediation alternative. The public opinion of the site after remediation can have indirect financial effects on other sectors (e.g., tourism). [Maximise]	Increased property value of the surroundings after remediation [$]	7%

, it is clear that the direct cost of the remedial technology is the dominant economic criterion used for evaluating alternative technologies. Moreover, direct cost and benefit criteria are generally used more often than their indirect counterparts. Interestingly, the review found one case study where a complete CBA framework was used as an economic criterion in the MCDA [52]. The CBA was used to quantify the overall economic benefit of the considered alternatives.

3.2.2. Social Criteria

The most frequently used social criteria are local acceptance (n = 17) and local impact considerations (n = 11) (

Table 3. Relative occurrence of specific social criteria to evaluate alternative remedial technologies compared to the total number of cases implementing social criteria (n = 24).

Social Criteria	Description	Example Indicator	Relative Occurrence of the Criterion [%]
Local acceptance	How acceptable is the alternative for the local communities? [Maximise]	Rated by community members [Qualitative scale]	71%
Local impact	What is the local impact of the remediation alternative on the neighbouring communities? [Minimise]	Nuisance from noise (dB)	46%
Local participation	What are the effects on the local community regarding local job opportunities, resources, and other local activities? [Maximise]	Number of people possibly employed from the local community [n]	8%
Cultural heritage	What impact does the remediation alternative have on cultural heritage, both on-site and off-site of the project, due to destruction, preservation, or restoration? Does the alternative impact the cultural practices of the local communities in any way? [Minimise]	Rated by community members [Qualitative scale]	8%
Equity	Does the remediation alternative provide fair chances (equity) for all community members? Are there effects on vulnerable groups in the society? [Maximise]	Rated by community members [Qualitative scale]	8%

). Social criteria scores for each alternative were predominantly determined by means of pairwise comparison of the alternatives. In other cases, the alternatives were assigned a rating by experts or stakeholder representatives using a scale ranging, for instance, from “unacceptable” to “acceptable”. The importance of local acceptance was stated by [53], who argued that if the remedial activity is not compatible with current management practices, for example, in agriculture, the community will be reluctant to support the new practices. It has been shown that local acceptance can be improved by a collaborative stakeholder engagement process, in which the local community gains knowledge about the remediation techniques and has the possibility to suggest adaptations. The regular use of ‘impact on the local community’ is not surprising since many remedial activities make use of heavy machinery and require large-scale storage of polluted soil, which can have large impacts on the everyday life of the surrounding communities (e.g., noise, dust, traffic). Additionally, many case studies dealt with sites situated close to or even in the urban environment, thus enlarging the impact of remedial activities on the population. The ‘local participation’ criterion is used less frequently in case studies and does not take into account stakeholder engagement but rather the explicit demand to the surrounding communities for labour and materials [54].

3.2.3. Technical Criteria

Technology performance (n = 18), feasibility (n = 16), and waste management (n = 16) are the most used technical criteria (

Table 4. Relative occurrence of specific technical criteria to evaluate alternative remedial technologies compared to the cases implementing technical criteria (n = 36).

Technical Criteria	Description	Example Indicator	Relative Occurrence of the Criterion [%]
Technology performance	The performance (contamination level reduction) of the remediation technique. [Maximise]	Reduction efficiency [%]	50%
Waste management	Does the remediation technology produce waste? [Minimise]	Amount of waste produced [tons/ha]	44%
Feasibility	The feasibility of the remediation technique for this particular site. This criterion can be quantified by the number of successful used cases of the technique, and the success rate of previous implementations. [Maximise]	Number of demonstration projects using the selected technology for the addressed pollutants with success [/]	44%
Time	Time to reach an acceptable contamination level for the site. [Minimise]	Time to reach acceptable contaminant level [year]	42%
Contaminant mobility	Does the remedial action optimally retain the contamination? [Minimise]	Number of pathways to receptors [/]	17%
Area	The size of the contaminated area or region affected by the remedial action. [Minimise]	Size of the region affected by the remedial action [ha]	8%
Future use	What is the effect of the remediation technique on the future use or aesthetic of the site? [Maximise]	Does the alternative increase or decrease the future uses of the site? [Qualitative scale]	6%

). The management of waste is a very important aspect and should not be overlooked since ineffective waste management would only relocate the problem instead of solving it. The time criterion may be related to contamination reduction efficiency, as some case studies use the time to reach a reasonable contaminant concentration as a measure of the technological efficiency of the alternatives.

3.2.4. Human Health Effects

Long-term exposure of the population is considered in more case studies than short-term exposure for determining the impact of a remedial technology (

Table 5. Relative occurrence of specific human health effects criteria to evaluate the alternative remedial technologies compared to the total number of cases implementing human health criteria (n = 32).

Human Health Effects Criteria	Description	Example Indicator	Relative Occurrence of the Criterion [%]
Long-term risk	The effects on human health and safety of the surrounding communities due to exposure to the remediation actions and residual contamination on the site. [Cost]	Toxicity indicators: Soil Carcinogenic Risk Index	97%
Short-term risk	The effects on human health and safety of the workers and users of the site due to exposure and spreading resulting from the remediation technique. [Cost]	Collective dose to workers and population [man Sv]	34%

). Long-term exposure is related to the accumulation/decrease in contaminants over time and, therefore, better reflects the sustainability of the remediation actions. Both the short and long-term risks related to specific contaminants are, in many cases, determined by transport and receptor sensitivity models. For many pollutants, the short and long-term health risk thresholds are strictly regulated. For example, the REC methodology (Risk reduction, Environmental merit, and Costs) does not explicitly consider the exposure of workers as an evaluation criterion because there are already very stringent legal limits on working conditions [7].

3.2.5. Environmental Criteria

The eco-toxicity of the contaminant is an important criterion (n = 22) (

Table 6. Relative occurrence of specific environmental criteria to evaluate the alternative remedial technologies compared to the total number of cases implementing environmental criteria (n = 31).

Environmental Criteria	Description	Example Indicator	Relative Occurrence of the Criterion [%]
Exposure of living organisms	The effect of the remediation on the exposure to the contaminant of living organisms and the residual contamination of the site after remediation (the ecological toxicity and risk to fauna and flora). [Maximise]	Bio-concentration Index [/]	71%
Primary non-living exposure	The effect of the remediation on the exposure to the contaminant of the abiotic environment and the residual contamination of the site after remediation. [Minimise]	Toxicity Index [/]	39%
The use of non-renewable energy sources and production of non-recyclable waste	Unsustainable use of resources and production of non-recyclable waste during the remediation process. [Minimise]	Use of non-renewable energy sources such as diesel [L]	35%
Secondary exposure	The effect of the remediation on the contaminant pathways to the air. [Minimise]	Release of the contaminant through volatilisation or dust formation [Qualitative scales]	26%
Contaminant concentration	The effect of the contaminant concentration on the remedial technology selection. Could the remedial technology deal with concentrations found on the site, or is residual contamination expected? [Minimise]	Contaminant concentration [mg/l]	13%
Land use	Are the remedial technologies suitable for the different specific land use types? [Maximise]	Suitable land use types [/]	3%

). A smaller number of cases (n = 12) have taken the exposure of living organisms (e.g., earthworms) into account. The use of non-renewable energy sources, carbon emissions, and production of non-recyclable waste is used to evaluate the sustainability of the remedial technology. In a few cases only, the contaminant concentration (n = 4) and land use type (n = 1) of the site have been used to assess the feasibility and impact of the remediation.

3.3. Criteria Weights

Throughout the case studies, different types of actors were involved in the determination of the criteria weights. In nearly half of the case studies (n = 20), weights were determined by experts, and in 15 case studies, groups of stakeholders were consulted. The remaining eight case studies used hypothetical weights designed to explore different views (e.g., the economic, green, conservative view, and an equal-weights view) and assessed their impact on the resulting recommendations. Alvarez-Guerra et al. [39] explained the necessity for hypothetical scenarios (like equal weights) because information on the weight distribution is often lacking. However, it should be noted that the assumption of equal weights is not compatible with the use of aggregation techniques (e.g., weighted sum), where weights have the meaning of trade-offs. Overall, seven different algorithms for eliciting criteria preference information were reported, with pairwise comparison (n = 19) being the most popular. All cases in which a form of direct rating method was applied are categorised as value judgment (n = 8). In these cases, different rating scales were applied, for example, a Likert scale (1–9) or an ordinal linguistic scale (good–bad) [55]. More complex weighting methods (e.g., SWING [5], SMART [54], and DEMATEL [56]) were applied when the determination of weights was performed by experts (n = 20). Another method used by experts was trade-off analysis, which uses a ratio between the scores of quantitative criteria to determine their weights [57]. Finally, in the best–worst case method, stakeholders must state their preference for the most important criterion) over the others and for all criteria over the least important criterion. From these preferences, the weights are determined mathematically.

3.4. Criteria Aggregation Methods

In the aggregation step, all the partial criterion assessments are converted into an overall assessment for each alternative. In general, no alternative is characterised by the optimal value for all criteria at the same time (a dominating solution). Consequently, a compromise solution that finds a balance between all the criteria must be found. According to [27], techniques for the assessment of alternatives can be divided into two types: (1) the synthesising preference relational system or outranking approach and (2) the synthesising criterion approach.

The latter approach is based on the assumption that the preferences of the decision-maker are consistent with an overall synthesising criterion, v. For each alternative, a, all the individual scores $g_1\left(a\right), \dots , g_n\left(a\right)$ for the corresponding n evaluation criteria are aggregated into one all-encompassing value (v). Subsequently, a comparison between two alternatives is then determined by their aggregated values $v\left(a\right)=V\left[g_1\left(a\right), \dots , g_n\left(a\right)\right]$ [27]. An important effect of using a synthesising criterion approach is the compensatory nature of the technique, as it allows high scores for specific criteria to compensate for lower scores for other criteria. Therefore, the problem becomes a trade-off problem, as stated by [58]. In comparison, the synthesising preference relational system does not assess each potential alternative separately from the others but successively compares it to the others [27]. The underlying assumption of these methods, among which the most known ones are ELECTRE and PROMETHEE, is that the decision-makers’ preference structure can be represented by an outranking relation [59]. For the outranking methods, the meaning of weights is the relative importance of each criterion. The values assigned to these weights depend upon the specific project context and on the judgements and belief structures of the actors. Depending on the number of actors, their background, and the consensus level, different weighting methods can be used.

In some of the 43 case studies, multiple aggregation techniques were used. Hence, we report on the 47 cases (and not on the 43) in this section. In 40 (85%) of the cases, the synthesising approach was used. Seven cases (15%) were addressed with the synthesising preference relational system or outranking approach.

The most commonly used technique for synthesising criteria (n = 11) is the weighted linear combination (WLC) of weights and criteria scores. WLC is a simple approach similar to the multi-attribute value theory (MAVT), but instead of a stakeholder-based value function, a simpler normalisation function is used to make the criteria commensurate. The multi-attribute value theory (MAVT) and the multi-attribute utility theory (MAUT) were applied in nine cases. MAVT is a special case of MAUT, which assumes that the decision-maker is relatively “ risk neutral” or that the attributes are known with certainty [60]. Value theory is based on the hypothesis that for every decision-maker in a specific decision context, a preference structure or value function exists to assess the alternatives on a criterion. Further, the aggregated score of all criteria for an alternative is determined, where a higher value represents a better alternative [58]. A linear aggregation function is most commonly used, although other functions with different mathematical properties are also used (e.g., multiplicative aggregation [61] or a Choquet integral [62,63,64]. The Choquet integral aggregation allows the inclusion of expert judgements for the elicitation of synergistic and antagonistic effects between the criteria [62].

In seven study cases, the analytical hierarchy process (AHP) was used, and in three, the analytical network process (ANP) framework. ANP is an elaboration of AHP where the hierarchical structure is transformed into a network that represents the interconnections and feedback loops among criteria [65].

Ideal point aggregation methods were applied in 5 out of the 47 cases, specifically the technique for order of preference by similarity to ideal solution (TOPSIS) [66,67,68] (n = 4) and compromise programming (CP) [53] (n = 1). Ideal point aggregations compare the alternatives in terms of their distance to an optimal solution in the normalised n-dimensional criterion space. The most observed distance metric was the Euclidean distance, but a variety of distance measures have been proposed. TOPSIS not only considers the distance to the ideal point (best alternative) but also uses a closeness metric that takes into account the distance to the anti-ideal point (worst alternative). The best–worst multi-criteria decision-making method (BWM MCDM) and the stochastic multi-criteria acceptability analysis (SMAA-2) are aggregation methods that inherently take uncertainty into account.

Within the group of cases in which an outranking aggregation method was applied, PROMETHEE II was the most frequently found method, with four cases out of seven. The interval-based methodologies (n = 2) use a procedure similar to PROMETHEE II with the calculation of leaving (or positive) and entering (or negative) flows. In only 1 out of the 47 cases, use was made of ELECTRE III, and in 1 case, a TOPSIS–PROMETHEE combination was used to determine the ranking. The two cases tackling the portfolio problem used a dynamic programming approach [36,37].

3.5. Sensitivity Analysis

Finally, uncertainty follows from imperfect knowledge of the world and is inherently present in decision-making. The uncertainty in the decision framework arises from different sources, including scarcity or uncertainty in the base data, measurement errors, conflicting information, uncertainty associated with the criteria scoring or the MCDA model parameters, and ambiguity in stakeholders’ opinions about criterion importance. Many of these sources of uncertainty are inherently present in all scientific investigations. However, specifically in MCDA, determining the weights is often considered the most subjective or uncertain step [69]. A sensitivity analysis can be performed to gain insights into the impact of these uncertainties and investigate the stability of the outcome of the MCDA. A stable decision alternative implies that a similar rank is maintained when minor perturbations are introduced to the data and/or model parameters.

In 23 of the 43 studied cases, a sensitivity analysis of the outcome from changes in the MCDA parameters was reported. In all these cases, the effect of uncertainty on the weights was studied through a stochastic sensitivity analysis. The most frequently used technique (n = 13) was the Monte Carlo simulation approach (MCS). In MCS, the weight of each criterion is represented by a probability distribution function (PDF) rather than by a unique value. The function is defined by both the range and the likelihood of the weight. Then, a random sampling technique selects a weight from each probability density function, with the enforcement that the sum of weights equals one. This procedure is repeated several times. For each combination of sampled weights, the aggregation technique is applied, and the output ranks become a rank probability. In some cases, the approach is further linked with the computation of a rank correlation measure, such as the Spearman rank correlation, to determine the robustness [46]. Techniques such as the Latin hypercube sampling are similar but use specific random sampling approaches [57]. The second most used technique (n = 4) was a combination of procedures that assess the utilities for decision options with the related probabilities in the data. In two of the cases, uncertainty was already taken into account in the aggregation process, i.e., by stochastic multi-criteria acceptability analysis for group decision-making (SMAA-2) [39,70] and by the best–worst method (BWM) MCDA [71]. SMAA-2 uses the central weight vector analysis to assess the sensitivity of the alternative preference order to changes in the weights. BWM is simpler: it determines the best and worst solutions.

Analysis of uncertainty can also be integrated into the weighting process and criteria evaluation step. A common approach for integrating uncertainty in the MCDA is based on fuzzy set theory [72]. Fuzzy logic and fuzzy set theory were introduced by [73] in 1965 and have been extensively used in ambiguity and uncertainty modelling in the context of decision-making. The different views and ambiguities in stakeholder opinions or between stakeholders can be handled using linguistic scales and fuzzy approaches [74]. Weight determination using linguistic scales and fuzzy set theory is deemed more similar to human thinking than numerical scaling. Therefore, it is less cognitively demanding for stakeholders to represent the importance of criteria on an intuitive linguistic scale [75,76]. Linguistic scales are also argued to be less prone to errors and more robust, leading to fuzzy logic being the preferred method when the information sources are qualitative or uncertain [44]. In 5 out of the 43 cases, fuzzy numbers instead of fixed/crisp values were used in the weighting procedure to represent uncertainty [77]. Fuzzy numbers can also be used as the basis for a probability distribution function in a Monte Carlo simulation [40].

4. Discussion

The purpose of this study was to review and synthesise findings from published cases in which use was made of MCDA to support the selection of remedial technologies for polluted soils. The review provided evidence that MCDA is a useful and flexible framework. It can accommodate several types of (i) contexts, (ii) criteria, (iii) alternatives, (iv) weighing, and (v) aggregation techniques. However, the review identified four major gaps: (i) a lack of early stakeholder engagement, (ii) a lack of inclusion of relevant social criteria, (iii) a mismatch between weighting and aggregation methodologies, and (iv) a lack of sensitivity analysis. As introduced in Section 1, these elements are key both to good methodological practice in MCDA application as well as to the framework of sustainable remediation. The four gaps will be discussed in depth in the following paragraphs.

4.1. Early Stakeholder Engagement

Stakeholder involvement in projects targeting the remediation of contaminated soils has been recognised by both the scientific community and governmental organisations as beneficial to the decision process as well as the project outcome [15,78]. The engagement of stakeholders creates a form of ‘buy-in’ into the project and is imperative for the eventual implementation of the MCDA outcome by these stakeholders. If consulted early in the decision-making process, stakeholders’ specific knowledge and insights can be mobilised to find innovative solutions aligned with the community’s needs, increase the credibility of the decision process, and enhance the acceptability of the project outcome [15,40,79]. In addition, including the local community in the decision process is also justified from an ethical point-of-view since the decisions will affect their lives [80]. However, from our review, it becomes clear that there is still a gap between the literature describing the advantage of decision-making with strong stakeholder involvement and the adoption in practice in contaminated soil management. This finding confirms the earlier observation by [40]. Further, the proportion of cases involving stakeholders that we found (14%) is in line with the 31% reported by [81]. In the latter study, particularly the views of the local communities were lacking. To this end, other application domains within the field of environmental management provide valuable examples of participatory methodologies that could inspire researchers and practitioners in environmental remediation [82]. A suitable framework is social multi-criteria evaluation [83], which provides a structured process for the inclusion of a plurality of perspectives in multi-criteria analysis and advocates for collaboration between technical and social sciences.

Institutional reports on stakeholder engagement [78,84] and previous studies on the application of MCDA (e.g., [22]) point to multiple reasons for the low engagement of stakeholders in MCDA, a major one being the reluctance to share the decision power. The reports agree that no “one-size-fits-all” methodology exists; therefore, for each specific remediation project, an appropriate methodology needs to be chosen or designed. Inevitably, this makes engagement with stakeholders more challenging and potentially more time-consuming. Furthermore, not all actors have the specific technical knowledge to participate effectively in the stakeholder consultation process, while other actors may not be as knowledgeable about the local social–cultural context. Hence, good knowledge-sharing mechanisms need to be set up. The increased complexity due to stakeholder engagement is a major reason why MCDA is often limited to technocratic support of decision-making, missing out on the sustainability of the decision process outcome. In addition, in the field of health care [85], finding consensus between multiple possible contrasting views was not explained.

4.2. Lack of Social Aspects within the Set of Criteria

On average, each of the 43 MCDA case studies considered six criteria, and more than half of them used criteria from four (economic, human health-related, technical, and environmental) out of five (four previous ones + social) criteria classes. It can, therefore, be stated that decision-makers use MCDA to address the remediation problem in a more holistic way, as compared to CBA and CEA. In general, technical, environmental, and economic criteria were the most frequently used. This can be explained by the fact that they are relatively easy to score in quantitative terms for the considered alternative technologies and are most related to the more common CBA and CEA approaches for decision-making. In addition, this review highlights a case where the CBA framework was incorporated into MCDA to quantify the economic attributes of the considered alternatives [52]. Therefore, MCDA can be seen as an extension of other decision frameworks to include difficult-to-monetise criteria. However, within the case studies, an underrepresentation of relevant criteria like “indirect cost”, “indirect benefit”, and “future use” show that even within the economic and technical classes, important aspects of soil remediation are not always taken into account.

Within the reviewed case studies, low adoption of social criteria was observed, confirming the findings of Cappuyns [13] and Hammond et al. [81]. Still, in several of the case studies where social criteria were included, it was stated that the MCDA methodology is convenient for incorporating difficult-to-quantify criteria such as those related to social impacts. While MCDA offers the ability to incorporate social criteria, it is not straightforward to operationalise them. However, it is important to stress that if not all dimensions of the remediation project are taken into account, the decisions will be sub-optimal [80]. As stated by [86], a sustainable remediation project is one that represents the best solution considering environmental, social, and economic factors, as agreed by the relevant stakeholders.

4.3. Informed Choice of Weighting and Aggregation Methodologies

In the studied cases, the most frequently applied techniques to determine the criterion weights were the pairwise comparison and the value judgment methods. Each of these techniques has its own strengths and weaknesses. Nevertheless, it was observed that in the majority of the cases, the weighting was done by experts and that the variation in weighting methods was larger when expert groups were involved. This preference for experts may be due to the specific knowledge and skills required by some of the weight determination methodologies, which are lacking or would cause a significant cognitive overburden to non-experts. However, even with weights assessed by experts, variations in weights may have large impacts. Li et al. [87] developed a modified AHP with an expert competence classification to reduce the impact of outliers regarding the weights. Although the inclusion of various stakeholder groups in determining priorities is important, a number of the studies reviewed by [81] argued for the application of solely professional judgment in the determination of weights for the MCDA.

It is important to note that the meaning of criteria weights differs between the two main aggregation approaches. While the concept of relative importance is valid for the outranking approaches, for the synthesising criterion approach, weights should be viewed as trade-off values. The combination of weighting and aggregation methods should be matched carefully to prevent inconsistencies in the meaning of the used weights [25,27]. A difference in meaning could potentially influence the weight given by the stakeholders to specific criteria. Our review identified several cases with such a mismatch, where value judgement techniques were used to assess the weight of a criterion, but these values were used as trade-off values in a compensatory aggregation rule.

Abundant use of the pairwise comparison method for assessing the weights was noted, possibly due to the long-standing tradition of AHP in MCDA since its introduction by Saaty in 1980 [88], as well as to the intuitive nature of expressing relative importance by direct pairwise comparison. Nevertheless, it is important to note that this method can lead to a high number of comparisons to be made, resulting in a cognitive burden for the decision-maker [89]. On the other hand, the frequent use of value judgement methods as a weighting procedure can be attributed to their ease of implementation, although the decision analyst must be careful to avoid inconsistent ratings [25]. Moreover, value judgement requires the decision-maker to be able to express, either directly or indirectly, the strengths of his or her preferences in a numeric way, which is not a natural cognitive task [27]. Methods like SWING or SMART allow working around this shortcoming; however, a low adoption rate of these techniques was observed in the case studies analysed.

The frequent use of synthesising criterion approaches can be explained by their simpler mathematical implementation and the relatively easier way to communicate results. Nevertheless, the compensatory behaviour of such methods can be undesirable for applications where social or environmental aspects are considered. For example, a remediation technology that has a negative impact on the environment and the community can still be ranked high because of a very favourable economic impact. In the case of environmental remediation, Sorvari et al. [90] highlighted the risk of a trade-off between risk reduction and costs in remediation. However, the aim of remediation is to eliminate or significantly lower the risk of contamination [91]. Consequently, sustainability principles may not be fulfilled by allowing for these kinds of compensation [83]. Therefore, it can be beneficial to only rank alternatives that meet a given threshold for one or more key criteria or groups of criteria. As an example, Rosen et al. [52] evaluate the remediation alternatives with respect to both “strong and weak sustainability” [92], as strong sustainability does not allow for the substitution of natural capital with human capital.

4.4. Sensitivity Analysis

While efforts to reduce uncertainty are important, recognising inherent sources of uncertainty and applying sensitivity analysis to assess the uncertainties’ impacts on the decision-making process outcome is essential. Two primary sources of uncertainty occur in MCDA. First is the measurement uncertainty, which relates to the methods used to score the criteria for the alternatives. Second, there is uncertainty regarding the MCDA model parameters. In particular, the weights assigned to the criteria are characterised by several uncertainties resulting from individual bias, lack of shared problem understanding, or value pluralism. The importance of dealing with uncertainties in decision-making has been the topic of many papers and projects. Even though it is considered an important step in MCDA, uncertainty was considered in only half of the reviewed case studies. This converges with the findings of [81] that, on average, half of the tools they reviewed included some sort of uncertainty analysis. While sensitivity analysis can challenge the decision-makers’ interpretation of results, it is essential to account for the uncertainty and communicate its impacts on MCDA results to increase transparency, provide a complete assessment, and allow for post-MCDA negotiations.

4.5. Limitations

The major limitation of this review is that it considered publications in international peer-reviewed journals for the period 1995–2020 only. It can be expected that a considerable number of valuable MCDA case studies are reported in institutional and non-governmental reports and grey scientific literature. Expansion of the literature search base and extension of the covered period to the early phase of the digital revolution in earth and environmental sciences (1980–1994) will undoubtedly further substantiate the findings of this review. The implementation (for example, consensus-building steps) and post-implementation phases of the selected alternative are not covered by any of the reviewed cases. More information on the extent to which the MCDA recommendations were effectively implemented and the effective economic, human health-related, social, technical, and environmental impacts is obviously of high interest. The review highlighted which criteria were used within the case studies. However, it did not attempt to improve the implementation of MCDA in supporting soil remediation or suggest the optimal criteria. Finally, relevant papers (e.g., [93,94]) published after the completion of the database coding were not included.

5. Conclusions

The review of 43 cases confirms the value of the MCDA framework to provide holistic support for selecting a remediation approach for polluted soils. It also identified four opportunities to strengthen the practical applications of MCDA for soil remediation: better inclusion of social indicators, early and continuous stakeholder involvement, attention to potential methodological pitfalls, and the inclusion of sensitivity analysis. The review highlighted that economic and technical criteria were predominantly present in the MCDA applications. However, within these classes, a more holistic view should be adapted to consider all facets, for example, indirect cost. In addition, social criteria should be included to a greater extent to move towards a fully holistic decision process, taking full benefit of the capacity of the MCDA framework to integrate difficult-to-quantify indicators, especially from the social and environmental domains. In several cases, the importance of engaging the stakeholders from an early phase was recognised to create a form of ‘buy-in’, which would increase credibility and enhance the acceptability of the MCDA outcome. Nevertheless, a significant number of case studies do not put this principle of early involvement of stakeholders into practice. With respect to criteria weights, a variety of techniques exists, each of them with their strengths and limitations. A careful selection and combination of weighting and aggregation methods is needed to prevent a mismatch between the meaning of weights and the logic of the subsequent technique for aggregating the weighted criteria scores. Currently, the most frequently used techniques for aggregation of criterion scores are of a synthesising nature, whereby the possible compensation of social and environmental criteria by economic criteria should be carefully monitored and, where possible, prevented. Finally, the case studies should pay more attention to uncertainty and sensitivity analysis, as this allows the impact of uncertainty to be assessed and improves transparency of the decision.