Dam Decision Support Tool
Welcome!
This free and open source tool was developed to aid people like you in considering tradeoffs associated with dams. It can help support your consideration of possible decision alternatives (e.g. keep and maintain dam,
improve hydropower generation, improve fish passage) for hydropower dams, and could potentially be tailored toward other types of dam decisions. The tool generates a data-driven recommendation for Federal Energy Regulatory Commission
(FERC) licensed hydropower dams in Maine's Penobscot River, based on your preferences. This tool is based on the Weighted Sum approach to Multi-Criteria Decision Analysis (MCDA) to compare decision-maker preferences for decision criteria
(e.g., annuitized cost, CO2 equivalent emissions reductions, sea-run fish habitat area, etc.) for hydropower dams with FERC license expiration dates in the next 10 years. The tool gathers user inputs and calculates a ranked set of decision
alternatives for each dam. Then, the tool shows the top-ranked set of decision alternatives for multiple dams. The purpose of the combined output is to encourage you to consider dams on the river as a system, in addition to one at a time.
NOTE: The results from this decision support tool are not official to any FERC licensing process and do not in any way represent the ruling of FERC.
More Information:
The Penobscot River is home to many dams and supports valuable ecosystem services. The watershed inclues pristine natural lakes, clean water sources, and significant biodiversity, including several sea-run fish species (e.g. Atlantic salmon, American eel, Blueback herring, and Alewife). Dams also provide important services: reservoirs for drinking water and recreation, flood protection, and generation of reliable, on-demand renewable hydropower, critical to reducing fossil-fuel emissions that contribute to climate change and poor human health. However, all dams need regular maintenance and older dams may need more extensive repairs as they age. Dams may interrupt flows and prevent sea-run fish passage, contributing to large population declines. They may also contribute to poor water quality downstream, increased predation, and climate change (decaying plant matter in reservoirs release methane into the atmosphere). Dams have long threatened indigenous cultural traditions, while at the same time helping to shape post-industrial town or city identities over the last two centuries.Click HERE for more information about the tradeoffs involved in dam decision making
View Existing FERC Dams Map
Please consider the following dams on the Penobscot River. These non-federally owned dams are coming up for FERC relicensing within the next 10 years. These are the dams you will focus on for the rest of the activity. Note: although the Penobscot Mills Project dams are licensed together under a single FERC license, we separate them here for consistency. Hover over the dams on the map for more information on each site.Click for more information about dam decision alternatives
Click for more information about decision criteria
Click to download Dam Data.
You may wish to refer to the resource links above and the watershed map below throughout the activity.
West Enfield Dam (FERC No. P-2600)
Please consider and rate your preference for the decision criteria listed below for West Enfield Dam. Download Dam Factsheet or Open in new tab. Move the slider bar for each decision criterion you care about to a position that represents the relative amount of preference you have for that decision criterion compared to others in the list. Once you have made your selections, click UPDATE at the bottom of the page when you are done moving the slider bars to mark this tab Complete. Click to view West Enfield Data.Warning: decision criteria ratings must sum to 100! The tracking indicator will help you keep track of the sum. Be aware that decision criteria are directly compensating (i.e., if the sum of all ratings is 100, then increasing the rating on one criterion requires another criterion rating to decrease to keep the sum equal to 100).
For ratings, 0 = not at all important and 100 = extremely important.
Sea-Run Fish Habitat Area
River Recreation
Reservoir Storage
Annuitized Project Costs
Breach Damage Potential
Number of Properties
Annual Electricity Generation
Annual CO2 Emissions Reductions
Indigenous Cultural Traditions and Lifeways
Industrial Historical Importance
Industrial historical importance is a unitless rating to convey the importance of preserving or restoring the industrial history of the site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Town/City Identity
Town/city identity is a unitless rating to convey the importance of preserving the existing identity of the existing town/city identity for residents living along the river.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Aesthetic Value
Aesthetic value is a rating to convey the importance of improving or preserving the aesthetics (e.g, appearance, scenic value, smell, sound) at a dam site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Medway Dam (FERC No. P-2666)
Please consider and rate your preference for the decision criteria listed below for Medway Dam. Download Dam Factsheet or Open in new tab.Move the slider bar for each decision criterion you care about to a position that represents the relative amount of preference you have for that decision criterion compared to others in the list. Once you have made your selections, click UPDATE at the bottom of the page when you are done moving the slider bars to mark this tab Complete. Click to view Medway Data.
Warning: decision criteria ratings must sum to 100! The tracking indicator will help you keep track of the sum. Be aware that decision criteria are directly compensating (i.e., if the sum of all ratings is 100, then increasing the rating on one criterion requires another criterion rating to decrease to keep the sum equal to 100).
For ratings, 0 = not at all important and 100 = extremely important.
Sea-Run Fish Habitat Area
River Recreation
Reservoir Storage
Annuitized Project Costs
Breach Damage Potential
Number of Properties
Annual Electricity Generation
Annual CO2 Emissions Reductions
Indigenous Cultural Traditions and Lifeways
Industrial Historical Importance
Industrial historical importance is a unitless rating to convey the importance of preserving or restoring the industrial history of the site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Town/City Identity
Town/city identity is a unitless rating to convey the importance of preserving the existing identity of the existing town/city identity for residents living along the river.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Aesthetic Value
Aesthetic value is a rating to convey the importance of improving or preserving the aesthetics (e.g, appearance, scenic value, smell, sound) at a dam site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.East Millinocket Dam (FERC No. P-2458)
Please consider and rate your preference for the decision criteria listed below for East Millinocket Dam. Download Dam Factsheet or Open in new tab.Move the slider bar for each decision criterion you care about to a position that represents the relative amount of preference you have for that decision criterion compared to others in the list. Once you have made your selections, click UPDATE at the bottom of the page when you are done moving the slider bars to mark this tab Complete. Click to view Penobscot Mills Data (PDF has information on all 5 dams, scroll to view).
Warning: decision criteria ratings must sum to 100! The tracking indicator will help you keep track of the sum. Be aware that decision criteria are directly compensating (i.e., if the sum of all ratings is 100, then increasing the rating on one criterion requires another criterion rating to decrease to keep the sum equal to 100).
For ratings, 0 = not at all important and 100 = extremely important.
Sea-Run Fish Habitat Area
River Recreation
Reservoir Storage
Annuitized Project Costs
Breach Damage Potential
Number of Properties
Annual Electricity Generation
Annual CO2 Emissions Reductions
Indigenous Cultural Traditions and Lifeways
Industrial Historical Importance
Industrial historical importance is a unitless rating to convey the importance of preserving or restoring the industrial history of the site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Town/City Identity
Town/city identity is a unitless rating to convey the importance of preserving the existing identity of the existing town/city identity for residents living along the river.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Aesthetic Value
Aesthetic value is a rating to convey the importance of improving or preserving the aesthetics (e.g, appearance, scenic value, smell, sound) at a dam site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Dolby Dam (FERC No. P-2458)
Please consider and rate your preference for the decision criteria listed below for Dolby Dam. Download Dam Factsheet or Open in new tabMove the slider bar for each decision criterion you care about to a position that represents the relative amount of preference you have for that decision criterion compared to others in the list. Once you have made your selections, click UPDATE at the bottom of the page when you are done moving the slider bars to mark this tab Complete. Click to view Penobscot Mills Data (PDF has information on all 5 dams, scroll to view).
Warning: decision criteria ratings must sum to 100! The tracking indicator will help you keep track of the sum. Be aware that decision criteria are directly compensating (i.e., if the sum of all ratings is 100, then increasing the rating on one criterion requires another criterion rating to decrease to keep the sum equal to 100).
For ratings, 0 = not at all important and 100 = extremely important.
Sea-Run Fish Habitat Area
River Recreation
Reservoir Storage
Annuitized Project Costs
Breach Damage Potential
Number of Properties
Annual Electricity Generation
Annual CO2 Emissions Reductions
Indigenous Cultural Traditions and Lifeways
Industrial Historical Importance
Industrial historical importance is a unitless rating to convey the importance of preserving or restoring the industrial history of the site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Town/City Identity
Town/city identity is a unitless rating to convey the importance of preserving the existing identity of the existing town/city identity for residents living along the river.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Aesthetic Value
Aesthetic value is a rating to convey the importance of improving or preserving the aesthetics (e.g, appearance, scenic value, smell, sound) at a dam site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.North Twin Dam (FERC No. P-2458)
Please consider and rate your preference for the decision criteria listed below for North Twin Dam. Download Dam Factsheet or Open in new tab.Move the slider bar for each decision criterion you care about to a position that represents the relative amount of preference you have for that decision criterion compared to others in the list. Once you have made your selections, click UPDATE at the bottom of the page when you are done moving the slider bars to mark this tab Complete. Click to view Penobscot Mills Data (PDF has information on all 5 dams, scroll to view)
Warning: decision criteria ratings must sum to 100! The tracking indicator will help you keep track of the sum. Be aware that decision criteria are directly compensating (i.e., if the sum of all ratings is 100, then increasing the rating on one criterion requires another criterion rating to decrease to keep the sum equal to 100).
For ratings, 0 = not at all important and 100 = extremely important.
Sea-Run Fish Habitat Area
River Recreation
Reservoir Storage
Annuitized Project Costs
Breach Damage Potential
Number of Properties
Annual Electricity Generation
Annual CO2 Emissions Reductions
Indigenous Cultural Traditions and Lifeways
Industrial Historical Importance
Industrial historical importance is a unitless rating to convey the importance of preserving or restoring the industrial history of the site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Town/City Identity
Town/city identity is a unitless rating to convey the importance of preserving the existing identity of the existing town/city identity for residents living along the river.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Aesthetic Value
Aesthetic value is a rating to convey the importance of improving or preserving the aesthetics (e.g, appearance, scenic value, smell, sound) at a dam site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Millinocket/Quakish Dam (FERC No. P-2458)
Please consider and rate your preference for the decision criteria listed below for Millinocket/Quakish Dam. Download Dam Factsheet or Open in new tab.Move the slider bar for each decision criterion you care about to a position that represents the relative amount of preference you have for that decision criterion compared to others in the list. Once you have made your selections, click UPDATE at the bottom of the page when you are done moving the slider bars to mark this tab Complete. Click to view Penobscot Mills Data (PDF has information on all 5 dams, scroll to view).
Warning: decision criteria ratings must sum to 100! The tracking indicator will help you keep track of the sum. Be aware that decision criteria are directly compensating (i.e., if the sum of all ratings is 100, then increasing the rating on one criterion requires another criterion rating to decrease to keep the sum equal to 100).
For ratings, 0 = not at all important and 100 = extremely important.
Sea-Run Fish Habitat Area
River Recreation
Reservoir Storage
Annuitized Project Costs
Breach Damage Potential
Number of Properties
Annual Electricity Generation
Annual CO2 Emissions Reductions
Indigenous Cultural Traditions and Lifeways
Industrial Historical Importance
Industrial historical importance is a unitless rating to convey the importance of preserving or restoring the industrial history of the site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Town/City Identity
Town/city identity is a unitless rating to convey the importance of preserving the existing identity of the existing town/city identity for residents living along the river.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Aesthetic Value
Aesthetic value is a rating to convey the importance of improving or preserving the aesthetics (e.g, appearance, scenic value, smell, sound) at a dam site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Millinocket Lake Dam (FERC No. P-2458)
Please consider and rate your preference for the decision criteria listed below for Millinocket Lake Dam. Download Dam Factsheet or Open in new tab.Move the slider bar for each decision criterion you care about to a position that represents the relative amount of preference you have for that decision criterion compared to others in the list. Once you have made your selections, click UPDATE at the bottom of the page when you are done moving the slider bars to mark this tab Complete. Click to view Penobscot Mills Data (PDF has information on all 5 dams, scroll to view).
Warning: decision criteria ratings must sum to 100! The tracking indicator will help you keep track of the sum. Be aware that decision criteria are directly compensating (i.e., if the sum of all ratings is 100, then increasing the rating on one criterion requires another criterion rating to decrease to keep the sum equal to 100).
For ratings, 0 = not at all important and 100 = extremely important.
Sea-Run Fish Habitat Area
River Recreation
Reservoir Storage
Annuitized Project Costs
Breach Damage Potential
Number of Properties
Annual Electricity Generation
Annual CO2 Emissions Reductions
Indigenous Cultural Traditions and Lifeways
Industrial Historical Importance
Industrial historical importance is a unitless rating to convey the importance of preserving or restoring the industrial history of the site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Town/City Identity
Town/city identity is a unitless rating to convey the importance of preserving the existing identity of the existing town/city identity for residents living along the river.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Aesthetic Value
Aesthetic value is a rating to convey the importance of improving or preserving the aesthetics (e.g, appearance, scenic value, smell, sound) at a dam site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Ripogenus Dam (FERC No. P-2572)
Please consider and rate your preference for the decision criteria listed below for Ripogenus Dam. Download Dam Factsheet or Open in new tab.Move the slider bar for each decision criterion you care about to a position that represents the relative amount of preference you have for that decision criterion compared to others in the list. Once you have made your selections, click UPDATE at the bottom of the page when you are done moving the slider bars to mark this tab Complete. Click to view Ripogenus Data.
Warning: decision criteria ratings must sum to 100! The tracking indicator will help you keep track of the sum. Be aware that decision criteria are directly compensating (i.e., if the sum of all ratings is 100, then increasing the rating on one criterion requires another criterion rating to decrease to keep the sum equal to 100).
For ratings, 0 = not at all important and 100 = extremely important.
Sea-Run Fish Habitat Area
River Recreation
Reservoir Storage
Annuitized Project Costs
Breach Damage Potential
Number of Properties
Annual Electricity Generation
Annual CO2 Emissions Reductions
Indigenous Cultural Traditions and Lifeways
Industrial Historical Importance
Industrial historical importance is a unitless rating to convey the importance of preserving or restoring the industrial history of the site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Town/City Identity
Town/city identity is a unitless rating to convey the importance of preserving the existing identity of the existing town/city identity for residents living along the river.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Aesthetic Value
Aesthetic value is a rating to convey the importance of improving or preserving the aesthetics (e.g, appearance, scenic value, smell, sound) at a dam site.
Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for this criterion.Multi-Dam Results
Click GENERATE to get MCDA results graphs.
Overview
Based on the preference values you entered for each dam in Step 3 and the data we collected/generated previously about what will happen to each decision criterion if each of the five decision alternatives is adopted, we have aggregated the top decision alternatives for the entire set of dams. In the figures below, you can compare the numeric score for each dam across decision criteria and decision alternatives. Step 5 (individual dams) results provide more detail about each individual dam. The outcome presented here is a recommendation designed to support the consideration of multiple dams, but the aggregated result shown here may not be optimal for coordinated, multi-dam decision making as described in Roy et al. (2018). This result is intended to support brainstorming about what single-dam decision making means for the river. This recommendation is not representative of any federal agency prescription or license ruling from FERC.
Questions for consideration: Do these results match your expectations? If not, why? If you feel discomfort at the result, you can return to the dam tabs and re-evaluate your decision criteria ratings. Remember to press "Update" under each tab. Then, return to this page and click GENERATE once more to see how your results change (note: you may want to download your results from this session, first).Do these results make sense, given the tradeoffs you made in balancing the set of decision criteria for each dam? Recall that the decision criteria are fully compensating, meaning that as the preference value for one decision criterion increases, the value for another decision criterion must necessarily decrease. The idea here is to emphasize tradeoffs between decision criteria.
Figure 1. Comparison of Final MCDA Score for each Decision Alternative at each Dam
Download GraphResults Interpretation for Figure 1. This graph shows the final MCDA score for each decision alternative at each dam. The taller the bar, the more preferred the decision alternative is. Use this graph to quickly see which decision alternative is the most highly ranked for each dam and to see how close the "next-best" decision alternative was to the top-ranked decision alternative. Small differences between scores for decision alternatives may signal a need to carefully consider multiple top-ranked alternatives for a specific dam for your final decision.
Figure 2. Contribution of Decision Criteria to Final MCDA Score for Top-Ranked Dam Decision Alternatives
Download GraphResults Interpretation for Figure 2. This graph shows the final MCDA score for the top-ranked decision alternative for each dam, broken down by the relative contribution of each decision criterion to the final MCDA score. Use this graph to give you an idea of how decision criteria scores (normalized data values x user-defined preference values) contribute to the final decision alternative selection for each dam. Consider this graph kind of a "zoom-in" on the tallest bar for each dam in Figure 2.
Figure 3. Comparison of User-Defined Preferences for each Dam
Download GraphResults Interpretation for Figure 3. This graph shows the results of the Step 3 Preference Elicitation. The numbers presented on the graph correspond to the position of the slider bars you moved in Step 3. Use this graph to see how your preferences for each decision criterion vary across dam sites. For example, did you mark the same preference value for River Recreation for Ripogenus as for West Enfield? Are your fish habitat preferences consistent across all dams? This graph is a good reality check to help you make sure you entered the right preferences in Step 3 for you. If you did not move the slider for a specific decision criterion for any dam, it will not show up in this graph. For example, if you left the slider for River Recreation at zero for all dams, River Recreation will not appear in the graph or legend on this page. The numbers in this graph do not include any researched data values for decision criteria (i.e., the data we compiled and/or calculated ahead of time and presented in the dam data for each dam). The graph only shows your own stated preference values from Step 3.
Download Preferences
It is a good idea to download your preferences for your records if you plan to use the Dam Decision Support Tool again.Download Preference Inputs (Step 3)
Save Preferences
Saving your preferences will load them automatically when you visit again. If you are using group mode, saving will add your preferences to the group average. Be aware: saving again will overwrite the old save.Results: West Enfield Dam
Now that you have seen the combined, top-ranked dam decision alternatives, we will drill down to explore the MCDA results for each individual dam. Remember that these results have been estimated using site-specific data values.Figure 1. Comparison of Final MCDA Scores for Each Decision Alternative
Download GraphResults Interpretation for Figure 1: This graph shows the final MCDA score for each decision alternative for this specific dam, based on the MCDA calculation that includes the preferences you entered in Step 3 and the decision criteria data we collected/generated through our research. The taller the bar, the more preferred the decision alternative us under the preferences you supplied. Use this graph for a quick comparison between decision alternatives for a single dam.
Figure 2. Contribution of Decision Criteria to Final MCDA Score for Dam Decision Alternatives
Download GraphResults Interpretation for Figure 2: This graph displays a zoomed-in version of Figure 1, with the final MCDA score bars for each decision alternative divided up by the contribution of each decision criterion to the final MCDA score. Similar to Figure 1, this graph includes your preference information and the researcher-defined data for each decision criterion. Use this graph to drill down and see which decision criteria are making up the largest portion of the final score for each decision alternative and whether you agree that is the way it should be. Remember, however, that these data take into account not only your preferences but also the research data. So, even if you gave a decision criterion a low rating in Step 3, it could still make up a large portion of this graph (unless you marked it as zero) because the data values may be larger relative to the full set of data values for that decision criteria for this dam. For example, if you assigned a 0.1 preference value to sea-run fish habitat area, you may be surprised to see a large segment for sea-run fish habitat in the Remove Dam decision alternative bar in this graph, but that could be because removing the dam increases fish habitat the most out of all of the decision alternatives for this dam site.
Data Inputs to these Results
The results presented in Figures 1 and 2 were calculated by multiplying your preference inputs from Step 3 by the normalized researcher-defined decision criteria data. This section displays these component data sets, reminding you in table and graph form of the preferences you entered in Step 3 for each decision criterion and showing you the researcher-defined decision criteria data sets (raw and normalized) that lead to the final calculation.Figure 3. User-Defined Preference Scores for West Enfield Dam
Download GraphResults Interpretation for Figure 3: This graph shows you the preferences you entered in Step 3 for each decision criterion. The scores are pulled directly from your slider bar settings under the West Enfield Dam tab and are not changed in any way. Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for these social decision criteria aside from Indigenous Cultural Traditions and Lifeways. If you wish to go back and change your settings, please do so before continuing. Remember to click GENERATE under Step 5. Multi-Dam Results.
Table 1. User-Defined Preference Scores for West Enfield Dam
Download TableThis table just shows the same thing as Figure 3 but in table form. If you would like to see all decision criteria preferences values at once, please select Show 25 entries from the drop-down menu above the table. Use the search bar to filter the table to a specific decision alternative (e.g. Keep and Maintain Dam).
Table 2. Data Values for West Enfield Dam
Download Table Download Dam DataResults Interpretation for Table 2. This table displays the raw data values we collected and/or calculated/generated through our research for each decision criterion and alternative. You may remember seeing these data when you clicked on the link for the data for this dam during the preference elicitation in Step 3. We include the raw data values again here to help make the MCDA calculations more transparent, so you can clearly see what goes into the final calculation that produces Figures 1 and 2 above. In addition, you can use this table to sort decision alternatives in ascending or descending order in each column by clicking on the arrow next to the column header. Note: fish survival values shown here are discrete, but in reality, the values are network-dependent and would be impacted by upstream or downstream changes. They are presented here as the average of a range of possible values for this dam.
Indigenous cultural traditions and lifeways data come from a survey of Penobscot Nation citizens (N=2), supported by informal conversations with Penobscot Nation citizens and representatives.
Table 3. Normalized Data Values for West Enfield Dam
Download TableResults Interpretation for Table 3. This table shows the data values from Table 2, normalized to be a score between 0 and 1 to make them comparable across different units. Normalization was performed using a min/max procedure: each raw data value was subtracted from the ideal value in the set (e.g., the maximum fish habitat area, for example) and divided by the difference between the maximum and minimum values in the set. The highest normalized values for most decision criteria, then, equal 1, and the lowest values equal 0. For decision criteria where lower values are more preferable (e.g. annuitized project cost, breach damage potential, number of properties impacted), the highest values equal 0, and the lowest values equal 1. This allows us to indicate that, for example, high costs are less desirable than low costs. The normalized data values in this table are multiplied by the preference weights displayed in Figure 3 and Table 1 to calculate the weighted scores in Table 4.
Normalization is a mathematical procedure that sets data values relative to their actual range, where the least desirable value (e.g., highest cost, lowest greenhouse gas emission reductions) in the existing data range is set equal to 0 and the most desirable value (e.g., lowest cost, highest greenhouse gas emissions reductions) in the existing data range is set equal to 1. In this way, normalization allows us to compare apples and oranges as if they were apples and apples. The normalization (minimax, maximin) procedure for the Dam Decision Support Tool bounds data within a range specific to the individual dam, not to the entire set of dams (i.e., the normalization operation makes the criteria data comparable for decision alternatives at an individual dam). Just as it does not consider data values for decision criteria at other dams, normalization does not consider possible improvements to the existing data. The highest and lowest values for the decision criterion for the specific dam site at set equal to zero or one, depending on what is considered desirable for the criterion (e.g., for cost, the lowest value is most desirable). Future studies could update the normalization procedure to work not only within the range for a specific dam but also across the entire set of dams.
Table 4. Weighted Scores for Individual Decision Criteria and Alternatives for West Enfield Dam
Download TableResults Interpretation for Table 4. This table shows the result of multiplying the preference scores from Table 1 (and Figure 3) by the normalized decision criteria data values displayed in Table 3. If you add together all numbers in one row in this table you will get the final MCDA score for that decision alternative, the same results that are presented in Figures 1-2 above.
Results: Medway Dam
Now that you have seen the combined, top-ranked dam decision alternatives, we will drill down to explore the MCDA results for each individual dam. Remember that these results have been estimated using site-specific data values.Figure 1. Comparison of Final MCDA Scores for Each Decision Alternative
Download GraphResults Interpretation for Figure 1: This graph shows the final MCDA score for each decision alternative for this specific dam, based on the MCDA calculation that includes the preferences you entered in Step 3 and the decision criteria data we collected/generated through our research. The taller the bar, the more preferred the decision alternative us under the preferences you supplied. Use this graph for a quick comparison between decision alternatives for a single dam.
Figure 2. Contribution of Decision Criteria to Final MCDA Score for Dam Decision Alternatives
Download GraphResults Interpretation for Figure 2: This graph displays a zoomed-in version of Figure 1, with the final MCDA score bars for each decision alternative divided up by the contribution of each decision criterion to the final MCDA score. Similar to Figure 1, this graph includes your preference information and the researcher-defined data for each decision criterion. Use this graph to drill down and see which decision criteria are making up the largest portion of the final score for each decision alternative and whether you agree that is the way it should be. Remember, however, that these data take into account not only your preferences but also the research data. So, even if you gave a decision criterion a low rating in Step 3, it could still make up a large portion of this graph (unless you marked it as zero) because the data values may be larger relative to the full set of data values for that decision criteria for this dam. For example, if you assigned a 0.1 preference value to sea-run fish habitat area, you may be surprised to see a large segment for sea-run fish habitat in the Remove Dam decision alternative bar in this graph, but that could be because removing the dam increases fish habitat the most out of all of the decision alternatives for this dam site.
Data Inputs to these Results
The results presented in Figures 1 and 2 were calculated by multiplying your preference inputs from Step 3 by the normalized researcher-defined decision criteria data. This section displays these component data sets, reminding you in table and graph form of the preferences you entered in Step 3 for each decision criterion and showing you the researcher-defined decision criteria data sets (raw and normalized) that lead to the final calculation.Figure 3. User-Defined Preference Scores for Medway Dam
Download GraphResults Interpretation for Figure 3: This graph shows you the preferences you entered in Step 3 for each decision criterion. The scores are pulled directly from your slider bar settings under the West Enfield Dam tab and are not changed in any way. Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for these social decision criteria aside from Indigenous Cultural Traditions and Lifeways. If you wish to go back and change your settings, please do so before continuing. Remember to click GENERATE under Step 5. Multi-Dam Results.
Table 1. User-Defined Preference Scores for Medway Dam
Download MedwayThis table just shows the same thing as Figure 3 but in table form. If you would like to see all decision criteria preferences values at once, please select Show 25 entries from the drop-down menu above the table. Use the search bar to filter the table to a specific decision alternative (e.g. Keep and Maintain Dam).
Table 2. Data Values for Medway Dam
Download Table Download Dam DataResults Interpretation for Table 2. This table displays the raw data values we collected and/or calculated/generated through our research for each decision criterion and alternative. You may remember seeing these data when you clicked on the link for the data for this dam during the preference elicitation in Step 3. We include the raw data values again here to help make the MCDA calculations more transparent, so you can clearly see what goes into the final calculation that produces Figures 1 and 2 above. In addition, you can use this table to sort decision alternatives in ascending or descending order in each column by clicking on the arrow next to the column header . Note: fish survival values shown here are discrete, but in reality, the values are network-dependent and would be impacted by upstream or downstream changes. They are presented here as the average of a range of possible values for this dam, depending on what happens at other dams. This interaction between decisions at other dams and these decision criteria are modeled in the multi-objective optimization that leads to the final Map Recommendation.
Indigenous cultural traditions and lifeways data come from a survey of Penobscot Nation citizens (N=2), supported by informal conversations with Penobscot Nation citizens and representatives.
Table 3. Normalized Data Values for Medway Dam
Download TableResults Interpretation for Table 3. This table shows the data values from Table 2, normalized to be a score between 0 and 1 to make them comparable across different units. Normalization was performed using a min/max procedure: each raw data value was subtracted from the ideal value in the set (e.g., the maximum fish habitat area, for example) and divided by the difference between the maximum and minimum values in the set. The highest normalized values for most decision criteria, then, equal 1, and the lowest values equal 0. For decision criteria where lower values are more preferable (e.g. annuitized project cost, breach damage potential, number of properties impacted), the highest values equal 0, and the lowest values equal 1. This allows us to indicate that, for example, high costs are less desirable than low costs. The normalized data values in this table are multiplied by the preference weights displayed in Figure 3 and Table 1 to calculate the weighted scores in Table 4.
Normalization is a mathematical procedure that sets data values relative to their actual range, where the least desirable value (e.g., highest cost, lowest greenhouse gas emission reductions) in the existing data range is set equal to 0 and the most desirable value (e.g., lowest cost, highest greenhouse gas emissions reductions) in the existing data range is set equal to 1. In this way, normalization allows us to compare apples and oranges as if they were apples and apples. The normalization (minimax, maximin) procedure for the Dam Decision Support Tool bounds data within a range specific to the individual dam, not to the entire set of dams (i.e., the normalization operation makes the criteria data comparable for decision alternatives at an individual dam). Just as it does not consider data values for decision criteria at other dams, normalization does not consider possible improvements to the existing data. The highest and lowest values for the decision criterion for the specific dam site at set equal to zero or one, depending on what is considered desirable for the criterion (e.g., for cost, the lowest value is most desirable). Future studies could update the normalization procedure to work not only within the range for a specific dam but also across the entire set of dams.
Table 4. Weighted Scores for Individual Decision Criteria and Alternatives for Medway Dam
Download TableResults Interpretation for Table 4. This table shows the result of multiplying the preference scores from Table 1 (and Figure 3) by the normalized decision criteria data values displayed in Table 3. If you add together all numbers in one row in this table you will get the final MCDA score for that decision alternative, the same results that are presented in Figures 1-2 above.
Results: East Millinocket Dam
Now that you have seen the combined, top-ranked dam decision alternatives, we will drill down to explore the MCDA results for each individual dam. Remember that these results have been estimated using site-specific data values.Figure 1. Comparison of Final MCDA Scores for Each Decision Alternative
Download GraphResults Interpretation for Figure 1: This graph shows the final MCDA score for each decision alternative for this specific dam, based on the MCDA calculation that includes the preferences you entered in Step 3 and the decision criteria data we collected/generated through our research. The taller the bar, the more preferred the decision alternative us under the preferences you supplied. Use this graph for a quick comparison between decision alternatives for a single dam.
Figure 2. Contribution of Decision Criteria to Final MCDA Score for Dam Decision Alternatives
Download GraphResults Interpretation for Figure 2: This graph displays a zoomed-in version of Figure 1, with the final MCDA score bars for each decision alternative divided up by the contribution of each decision criterion to the final MCDA score. Similar to Figure 1, this graph includes your preference information and the researcher-defined data for each decision criterion. Use this graph to drill down and see which decision criteria are making up the largest portion of the final score for each decision alternative and whether you agree that is the way it should be. Remember, however, that these data take into account not only your preferences but also the research data. So, even if you gave a decision criterion a low rating in Step 3, it could still make up a large portion of this graph (unless you marked it as zero) because the data values may be larger relative to the full set of data values for that decision criteria for this dam. For example, if you assigned a 0.1 preference value to sea-run fish habitat area, you may be surprised to see a large segment for sea-run fish habitat in the Remove Dam decision alternative bar in this graph, but that could be because removing the dam increases fish habitat the most out of all of the decision alternatives for this dam site.
Data Inputs to these Results
The results presented in Figures 1 and 2 were calculated by multiplying your preference inputs from Step 3 by the normalized researcher-defined decision criteria data. This section displays these component data sets, reminding you in table and graph form of the preferences you entered in Step 3 for each decision criterion and showing you the researcher-defined decision criteria data sets (raw and normalized) that lead to the final calculation.Figure 3. User-Defined Preference Scores for East Millinocket Dam
Download GraphResults Interpretation for Figure 3: This graph shows you the preferences you entered in Step 3 for each decision criterion. The scores are pulled directly from your slider bar settings under the West Enfield Dam tab and are not changed in any way. Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for these social decision criteria aside from Indigenous Cultural Traditions and Lifeways.If you wish to go back and change your settings, please do so before continuing. Remember to click GENERATE under Step 5. Multi-Dam Results.
Table 1. User-Defined Preference Scores for East Millinocket Dam
Download East MillinocketThis table just shows the same thing as Figure 3 but in table form. If you would like to see all decision criteria preferences values at once, please select Show 25 entries from the drop-down menu above the table. Use the search bar to filter the table to a specific decision alternative (e.g. Keep and Maintain Dam).
Table 2. Data Values for East Millinocket Dam
Download Table Download Dam DataResults Interpretation for Table 2. This table displays the raw data values we collected and/or calculated/generated through our research for each decision criterion and alternative. You may remember seeing these data when you clicked on the link for the data for this dam during the preference elicitation in Step 3. We include the raw data values again here to help make the MCDA calculations more transparent, so you can clearly see what goes into the final calculation that produces Figures 1 and 2 above. In addition, you can use this table to sort decision alternatives in ascending or descending order in each column by clicking on the arrow next to the column header . Note: fish survival values shown here are discrete, but in reality, the values are network-dependent and would be impacted by upstream or downstream changes. They are presented here as the average of a range of possible values for this dam, depending on what happens at other dams. This interaction between decisions at other dams and these decision criteria are modeled in the multi-objective optimization that leads to the final Map Recommendation.
Indigenous cultural traditions and lifeways data come from a survey of Penobscot Nation citizens (N=2), supported by informal conversations with Penobscot Nation citizens and representatives.
Table 3. Normalized Data Values for East Millinocket Dam
Download TableResults Interpretation for Table 3. This table shows the data values from Table 2, normalized to be a score between 0 and 1 to make them comparable across different units. Normalization was performed using a min/max procedure: each raw data value was subtracted from the ideal value in the set (e.g., the maximum fish habitat area, for example) and divided by the difference between the maximum and minimum values in the set. The highest normalized values for most decision criteria, then, equal 1, and the lowest values equal 0. For decision criteria where lower values are more preferable (e.g. annuitized project cost, breach damage potential, number of properties impacted), the highest values equal 0, and the lowest values equal 1. This allows us to indicate that, for example, high costs are less desirable than low costs. The normalized data values in this table are multiplied by the preference weights displayed in Figure 3 and Table 1 to calculate the weighted scores in Table 4.
Normalization is a mathematical procedure that sets data values relative to their actual range, where the least desirable value (e.g., highest cost, lowest greenhouse gas emission reductions) in the existing data range is set equal to 0 and the most desirable value (e.g., lowest cost, highest greenhouse gas emissions reductions) in the existing data range is set equal to 1. In this way, normalization allows us to compare apples and oranges as if they were apples and apples. The normalization (minimax, maximin) procedure for the Dam Decision Support Tool bounds data within a range specific to the individual dam, not to the entire set of dams (i.e., the normalization operation makes the criteria data comparable for decision alternatives at an individual dam). Just as it does not consider data values for decision criteria at other dams, normalization does not consider possible improvements to the existing data. The highest and lowest values for the decision criterion for the specific dam site at set equal to zero or one, depending on what is considered desirable for the criterion (e.g., for cost, the lowest value is most desirable). Future studies could update the normalization procedure to work not only within the range for a specific dam but also across the entire set of dams.
Table 4. Weighted Scores for Individual Decision Criteria and Alternatives for East Millinocket Dam
Download TableResults Interpretation for Table 4. This table shows the result of multiplying the preference scores from Table 1 (and Figure 3) by the normalized decision criteria data values displayed in Table 3. If you add together all numbers in one row in this table you will get the final MCDA score for that decision alternative, the same results that are presented in Figures 1-2 above.
Results: Dolby Dam
Now that you have seen the combined, top-ranked dam decision alternatives, we will drill down to explore the MCDA results for each individual dam. Remember that these results have been estimated using site-specific data values.Figure 4. Comparison of Final MCDA Scores for Each Decision Alternative
Download GraphResults Interpretation for Figure 1: This graph shows the final MCDA score for each decision alternative for this specific dam, based on the MCDA calculation that includes the preferences you entered in Step 3 and the decision criteria data we collected/generated through our research. The taller the bar, the more preferred the decision alternative us under the preferences you supplied. Use this graph for a quick comparison between decision alternatives for a single dam.
Figure 2. Contribution of Decision Criteria to Final MCDA Score for Dam Decision Alternatives
Download GraphResults Interpretation for Figure 2: This graph displays a zoomed-in version of Figure 1, with the final MCDA score bars for each decision alternative divided up by the contribution of each decision criterion to the final MCDA score. Similar to Figure 1, this graph includes your preference information and the researcher-defined data for each decision criterion. Use this graph to drill down and see which decision criteria are making up the largest portion of the final score for each decision alternative and whether you agree that is the way it should be. Remember, however, that these data take into account not only your preferences but also the research data. So, even if you gave a decision criterion a low rating in Step 3, it could still make up a large portion of this graph (unless you marked it as zero) because the data values may be larger relative to the full set of data values for that decision criteria for this dam. For example, if you assigned a 0.1 preference value to sea-run fish habitat area, you may be surprised to see a large segment for sea-run fish habitat in the Remove Dam decision alternative bar in this graph, but that could be because removing the dam increases fish habitat the most out of all of the decision alternatives for this dam site.
Data Inputs to these Results
The results presented in Figures 1 and 2 were calculated by multiplying your preference inputs from Step 3 by the normalized researcher-defined decision criteria data. This section displays these component data sets, reminding you in table and graph form of the preferences you entered in Step 3 for each decision criterion and showing you the researcher-defined decision criteria data sets (raw and normalized) that lead to the final calculation.Figure 3. User-Defined Preference Scores for Dolby Dam
Download GraphResults Interpretation for Figure 3: This graph shows you the preferences you entered in Step 3 for each decision criterion. The scores are pulled directly from your slider bar settings under the West Enfield Dam tab and are not changed in any way. Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for these social decision criteria aside from Indigenous Cultural Traditions and Lifeways.If you wish to go back and change your settings, please do so before continuing. Remember to click GENERATE under Step 5. Multi-Dam Results.
Table 1. User-Defined Preference Scores for Dolby Dam
Download DolbyThis table just shows the same thing as Figure 3 but in table form. If you would like to see all decision criteria preferences values at once, please select Show 25 entries from the drop-down menu above the table. Use the search bar to filter the table to a specific decision alternative (e.g. Keep and Maintain Dam).
Table 2. Data Values for Dolby Dam
Download Table Download Dam DataResults Interpretation for Table 2. This table displays the raw data values we collected and/or calculated/generated through our research for each decision criterion and alternative. You may remember seeing these data when you clicked on the link for the data for this dam during the preference elicitation in Step 3. We include the raw data values again here to help make the MCDA calculations more transparent, so you can clearly see what goes into the final calculation that produces Figures 1 and 2 above. In addition, you can use this table to sort decision alternatives in ascending or descending order in each column by clicking on the arrow next to the column header . Note: fish survival values shown here are discrete, but in reality, the values are network-dependent and would be impacted by upstream or downstream changes. They are presented here as the average of a range of possible values for this dam, depending on what happens at other dams. This interaction between decisions at other dams and these decision criteria are modeled in the multi-objective optimization that leads to the final Map Recommendation.
Indigenous cultural traditions and lifeways data come from a survey of Penobscot Nation citizens (N=2), supported by informal conversations with Penobscot Nation citizens and representatives.
Table 3. Normalized Data Values for Dolby Dam
Download TableResults Interpretation for Table 3. This table shows the data values from Table 2, normalized to be a score between 0 and 1 to make them comparable across different units. Normalization was performed using a min/max procedure: each raw data value was subtracted from the ideal value in the set (e.g., the maximum fish habitat area, for example) and divided by the difference between the maximum and minimum values in the set. The highest normalized values for most decision criteria, then, equal 1, and the lowest values equal 0. For decision criteria where lower values are more preferable (e.g. annuitized project cost, breach damage potential, number of properties impacted), the highest values equal 0, and the lowest values equal 1. This allows us to indicate that, for example, high costs are less desirable than low costs. The normalized data values in this table are multiplied by the preference weights displayed in Figure 3 and Table 1 to calculate the weighted scores in Table 4.
Normalization is a mathematical procedure that sets data values relative to their actual range, where the least desirable value (e.g., highest cost, lowest greenhouse gas emission reductions) in the existing data range is set equal to 0 and the most desirable value (e.g., lowest cost, highest greenhouse gas emissions reductions) in the existing data range is set equal to 1. In this way, normalization allows us to compare apples and oranges as if they were apples and apples. The normalization (minimax, maximin) procedure for the Dam Decision Support Tool bounds data within a range specific to the individual dam, not to the entire set of dams (i.e., the normalization operation makes the criteria data comparable for decision alternatives at an individual dam). Just as it does not consider data values for decision criteria at other dams, normalization does not consider possible improvements to the existing data. The highest and lowest values for the decision criterion for the specific dam site at set equal to zero or one, depending on what is considered desirable for the criterion (e.g., for cost, the lowest value is most desirable). Future studies could update the normalization procedure to work not only within the range for a specific dam but also across the entire set of dams.
Table 4. Weighted Scores for Individual Decision Criteria and Alternatives for Dolby Dam
Download TableResults Interpretation for Table 4. This table shows the result of multiplying the preference scores from Table 1 (and Figure 3) by the normalized decision criteria data values displayed in Table 3. If you add together all numbers in one row in this table you will get the final MCDA score for that decision alternative, the same results that are presented in Figures 1-2 above.
Results: North Twin Dam
Now that you have seen the combined, top-ranked dam decision alternatives, we will drill down to explore the MCDA results for each individual dam. Remember that these results have been estimated using site-specific data values.Figure 1. Comparison of Final MCDA Scores for Each Decision Alternative
Download GraphResults Interpretation for Figure 1: This graph shows the final MCDA score for each decision alternative for this specific dam, based on the MCDA calculation that includes the preferences you entered in Step 3 and the decision criteria data we collected/generated through our research. The taller the bar, the more preferred the decision alternative us under the preferences you supplied. Use this graph for a quick comparison between decision alternatives for a single dam.
Figure 2. Contribution of Decision Criteria to Final MCDA Score for Dam Decision Alternatives
Download GraphResults Interpretation for Figure 2: This graph displays a zoomed-in version of Figure 1, with the final MCDA score bars for each decision alternative divided up by the contribution of each decision criterion to the final MCDA score. Similar to Figure 1, this graph includes your preference information and the researcher-defined data for each decision criterion. Use this graph to drill down and see which decision criteria are making up the largest portion of the final score for each decision alternative and whether you agree that is the way it should be. Remember, however, that these data take into account not only your preferences but also the research data. So, even if you gave a decision criterion a low rating in Step 3, it could still make up a large portion of this graph (unless you marked it as zero) because the data values may be larger relative to the full set of data values for that decision criteria for this dam. For example, if you assigned a 0.1 preference value to sea-run fish habitat area, you may be surprised to see a large segment for sea-run fish habitat in the Remove Dam decision alternative bar in this graph, but that could be because removing the dam increases fish habitat the most out of all of the decision alternatives for this dam site.
Data Inputs to these Results
The results presented in Figures 1 and 2 were calculated by multiplying your preference inputs from Step 3 by the normalized researcher-defined decision criteria data. This section displays these component data sets, reminding you in table and graph form of the preferences you entered in Step 3 for each decision criterion and showing you the researcher-defined decision criteria data sets (raw and normalized) that lead to the final calculation.Figure 3. User-Defined Preference Scores for North Twin Dam
Download GraphResults Interpretation for Figure 3: This graph shows you the preferences you entered in Step 3 for each decision criterion. The scores are pulled directly from your slider bar settings under the West Enfield Dam tab and are not changed in any way. Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for these social decision criteria aside from Indigenous Cultural Traditions and Lifeways.If you wish to go back and change your settings, please do so before continuing. Remember to click GENERATE under Step 5. Multi-Dam Results.
Table 1. User-Defined Preference Scores for North Twin Dam
Download North TwinThis table just shows the same thing as Figure 3 but in table form. If you would like to see all decision criteria preferences values at once, please select Show 25 entries from the drop-down menu above the table. Use the search bar to filter the table to a specific decision alternative (e.g. Keep and Maintain Dam).
Table 2. Data Values for North Twin Dam
Download Table Download Dam DataResults Interpretation for Table 2. This table displays the raw data values we collected and/or calculated/generated through our research for each decision criterion and alternative. You may remember seeing these data when you clicked on the link for the data for this dam during the preference elicitation in Step 3. We include the raw data values again here to help make the MCDA calculations more transparent, so you can clearly see what goes into the final calculation that produces Figures 1 and 2 above. In addition, you can use this table to sort decision alternatives in ascending or descending order in each column by clicking on the arrow next to the column header . Note: fish survival values shown here are discrete, but in reality, the values are network-dependent and would be impacted by upstream or downstream changes. They are presented here as the average of a range of possible values for this dam, depending on what happens at other dams. This interaction between decisions at other dams and these decision criteria are modeled in the multi-objective optimization that leads to the final Map Recommendation.
Indigenous cultural traditions and lifeways data come from a survey of Penobscot Nation citizens (N=2), supported by informal conversations with Penobscot Nation citizens and representatives.
Table 3. Normalized Data Values for North Twin Dam
Download TableResults Interpretation for Table 3. This table shows the data values from Table 2, normalized to be a score between 0 and 1 to make them comparable across different units. Normalization was performed using a min/max procedure: each raw data value was subtracted from the ideal value in the set (e.g., the maximum fish habitat area, for example) and divided by the difference between the maximum and minimum values in the set. The highest normalized values for most decision criteria, then, equal 1, and the lowest values equal 0. For decision criteria where lower values are more preferable (e.g. annuitized project cost, breach damage potential, number of properties impacted), the highest values equal 0, and the lowest values equal 1. This allows us to indicate that, for example, high costs are less desirable than low costs. The normalized data values in this table are multiplied by the preference weights displayed in Figure 3 and Table 1 to calculate the weighted scores in Table 4.
Normalization is a mathematical procedure that sets data values relative to their actual range, where the least desirable value (e.g., highest cost, lowest greenhouse gas emission reductions) in the existing data range is set equal to 0 and the most desirable value (e.g., lowest cost, highest greenhouse gas emissions reductions) in the existing data range is set equal to 1. In this way, normalization allows us to compare apples and oranges as if they were apples and apples. The normalization (minimax, maximin) procedure for the Dam Decision Support Tool bounds data within a range specific to the individual dam, not to the entire set of dams (i.e., the normalization operation makes the criteria data comparable for decision alternatives at an individual dam). Just as it does not consider data values for decision criteria at other dams, normalization does not consider possible improvements to the existing data. The highest and lowest values for the decision criterion for the specific dam site at set equal to zero or one, depending on what is considered desirable for the criterion (e.g., for cost, the lowest value is most desirable). Future studies could update the normalization procedure to work not only within the range for a specific dam but also across the entire set of dams.
Table 4. Weighted Scores for Individual Decision Criteria and Alternatives for North Twin Dam
Download TableResults Interpretation for Table 4. This table shows the result of multiplying the preference scores from Table 1 (and Figure 3) by the normalized decision criteria data values displayed in Table 3. If you add together all numbers in one row in this table you will get the final MCDA score for that decision alternative, the same results that are presented in Figures 1-2 above.
Results: Millinocket/Quakish Dam
Now that you have seen the combined, top-ranked dam decision alternatives, we will drill down to explore the MCDA results for each individual dam. Remember that these results have been estimated using site-specific data values.Figure 1. Comparison of Final MCDA Scores for Each Decision Alternative
Download GraphResults Interpretation for Figure 1: This graph shows the final MCDA score for each decision alternative for this specific dam, based on the MCDA calculation that includes the preferences you entered in Step 3 and the decision criteria data we collected/generated through our research. The taller the bar, the more preferred the decision alternative us under the preferences you supplied. Use this graph for a quick comparison between decision alternatives for a single dam.
Figure 2. Contribution of Decision Criteria to Final MCDA Score for Dam Decision Alternatives
Download GraphResults Interpretation for Figure 2: This graph displays a zoomed-in version of Figure 1, with the final MCDA score bars for each decision alternative divided up by the contribution of each decision criterion to the final MCDA score. Similar to Figure 1, this graph includes your preference information and the researcher-defined data for each decision criterion. Use this graph to drill down and see which decision criteria are making up the largest portion of the final score for each decision alternative and whether you agree that is the way it should be. Remember, however, that these data take into account not only your preferences but also the research data. So, even if you gave a decision criterion a low rating in Step 3, it could still make up a large portion of this graph (unless you marked it as zero) because the data values may be larger relative to the full set of data values for that decision criteria for this dam. For example, if you assigned a 0.1 preference value to sea-run fish habitat area, you may be surprised to see a large segment for sea-run fish habitat in the Remove Dam decision alternative bar in this graph, but that could be because removing the dam increases fish habitat the most out of all of the decision alternatives for this dam site.
Data Inputs to these Results
The results presented in Figures 1 and 2 were calculated by multiplying your preference inputs from Step 3 by the normalized researcher-defined decision criteria data. This section displays these component data sets, reminding you in table and graph form of the preferences you entered in Step 3 for each decision criterion and showing you the researcher-defined decision criteria data sets (raw and normalized) that lead to the final calculation.Figure 3. User-Defined Preference Scores for Millinocket/Quakish Dam
Download GraphResults Interpretation for Figure 3: This graph shows you the preferences you entered in Step 3 for each decision criterion. The scores are pulled directly from your slider bar settings under the West Enfield Dam tab and are not changed in any way. Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for these social decision criteria aside from Indigenous Cultural Traditions and Lifeways.If you wish to go back and change your settings, please do so before continuing. Remember to click GENERATE under Step 5. Multi-Dam Results.
Table 1. User-Defined Preference Scores for Millinocket/Quakish Dam
Download MillinocketThis table just shows the same thing as Figure 3 but in table form. If you would like to see all decision criteria preferences values at once, please select Show 25 entries from the drop-down menu above the table. Use the search bar to filter the table to a specific decision alternative (e.g. Keep and Maintain Dam).
Table 2. Data Values for Millinocket/Quakish Dam
Download Table Download Dam DataResults Interpretation for Table 2. This table displays the raw data values we collected and/or calculated/generated through our research for each decision criterion and alternative. You may remember seeing these data when you clicked on the link for the data for this dam during the preference elicitation in Step 3. We include the raw data values again here to help make the MCDA calculations more transparent, so you can clearly see what goes into the final calculation that produces Figures 1 and 2 above. In addition, you can use this table to sort decision alternatives in ascending or descending order in each column by clicking on the arrow next to the column header . Note: fish survival values shown here are discrete, but in reality, the values are network-dependent and would be impacted by upstream or downstream changes. They are presented here as the average of a range of possible values for this dam, depending on what happens at other dams. This interaction between decisions at other dams and these decision criteria are modeled in the multi-objective optimization that leads to the final Map Recommendation.
Indigenous cultural traditions and lifeways data come from a survey of Penobscot Nation citizens (N=2), supported by informal conversations with Penobscot Nation citizens and representatives.
Table 3. Normalized Data Values for Millinocket/Quakish Dam
Download TableResults Interpretation for Table 3. This table shows the data values from Table 2, normalized to be a score between 0 and 1 to make them comparable across different units. Normalization was performed using a min/max procedure: each raw data value was subtracted from the ideal value in the set (e.g., the maximum fish habitat area, for example) and divided by the difference between the maximum and minimum values in the set. The highest normalized values for most decision criteria, then, equal 1, and the lowest values equal 0. For decision criteria where lower values are more preferable (e.g. annuitized project cost, breach damage potential, number of properties impacted), the highest values equal 0, and the lowest values equal 1. This allows us to indicate that, for example, high costs are less desirable than low costs. The normalized data values in this table are multiplied by the preference weights displayed in Figure 3 and Table 1 to calculate the weighted scores in Table 4.
Normalization is a mathematical procedure that sets data values relative to their actual range, where the least desirable value (e.g., highest cost, lowest greenhouse gas emission reductions) in the existing data range is set equal to 0 and the most desirable value (e.g., lowest cost, highest greenhouse gas emissions reductions) in the existing data range is set equal to 1. In this way, normalization allows us to compare apples and oranges as if they were apples and apples. The normalization (minimax, maximin) procedure for the Dam Decision Support Tool bounds data within a range specific to the individual dam, not to the entire set of dams (i.e., the normalization operation makes the criteria data comparable for decision alternatives at an individual dam). Just as it does not consider data values for decision criteria at other dams, normalization does not consider possible improvements to the existing data. The highest and lowest values for the decision criterion for the specific dam site at set equal to zero or one, depending on what is considered desirable for the criterion (e.g., for cost, the lowest value is most desirable). Future studies could update the normalization procedure to work not only within the range for a specific dam but also across the entire set of dams.
Table 4. Weighted Scores for Individual Decision Criteria and Alternatives for Millinocket/Quakish Dam
Download TableResults Interpretation for Table 4. This table shows the result of multiplying the preference scores from Table 1 (and Figure 3) by the normalized decision criteria data values displayed in Table 3. If you add together all numbers in one row in this table you will get the final MCDA score for that decision alternative, the same results that are presented in Figures 1-2 above.
Results: Millinocket Lake Dam
Now that you have seen the combined, top-ranked dam decision alternatives, we will drill down to explore the MCDA results for each individual dam. Remember that these results have been estimated using site-specific data values.Figure 1. Comparison of Final MCDA Scores for Each Decision Alternative
Download GraphResults Interpretation for Figure 1: This graph shows the final MCDA score for each decision alternative for this specific dam, based on the MCDA calculation that includes the preferences you entered in Step 3 and the decision criteria data we collected/generated through our research. The taller the bar, the more preferred the decision alternative us under the preferences you supplied. Use this graph for a quick comparison between decision alternatives for a single dam.
Figure 2. Contribution of Decision Criteria to Final MCDA Score for Dam Decision Alternatives
Download GraphResults Interpretation for Figure 2: This graph displays a zoomed-in version of Figure 1, with the final MCDA score bars for each decision alternative divided up by the contribution of each decision criterion to the final MCDA score. Similar to Figure 1, this graph includes your preference information and the researcher-defined data for each decision criterion. Use this graph to drill down and see which decision criteria are making up the largest portion of the final score for each decision alternative and whether you agree that is the way it should be. Remember, however, that these data take into account not only your preferences but also the research data. So, even if you gave a decision criterion a low rating in Step 3, it could still make up a large portion of this graph (unless you marked it as zero) because the data values may be larger relative to the full set of data values for that decision criteria for this dam. For example, if you assigned a 0.1 preference value to sea-run fish habitat area, you may be surprised to see a large segment for sea-run fish habitat in the Remove Dam decision alternative bar in this graph, but that could be because removing the dam increases fish habitat the most out of all of the decision alternatives for this dam site.
Data Inputs to these Results
The results presented in Figures 1 and 2 were calculated by multiplying your preference inputs from Step 3 by the normalized researcher-defined decision criteria data. This section displays these component data sets, reminding you in table and graph form of the preferences you entered in Step 3 for each decision criterion and showing you the researcher-defined decision criteria data sets (raw and normalized) that lead to the final calculation.Figure 3. User-Defined Preference Scores for Millinocket Lake Dam
Download GraphResults Interpretation for Figure 3: This graph shows you the preferences you entered in Step 3 for each decision criterion. The scores are pulled directly from your slider bar settings under the West Enfield Dam tab and are not changed in any way. Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for these social decision criteria aside from Indigenous Cultural Traditions and Lifeways.If you wish to go back and change your settings, please do so before continuing. Remember to click GENERATE under Step 5. Multi-Dam Results.
Table 1. User-Defined Preference Scores for Millinocket Lake Dam
Download Millinocket LakeThis table just shows the same thing as Figure 3 but in table form. If you would like to see all decision criteria preferences values at once, please select Show 25 entries from the drop-down menu above the table. Use the search bar to filter the table to a specific decision alternative (e.g. Keep and Maintain Dam).
Table 2. Data Values for Millinocket Lake Dam
Download Table Download Dam DataResults Interpretation for Table 2. This table displays the raw data values we collected and/or calculated/generated through our research for each decision criterion and alternative. You may remember seeing these data when you clicked on the link for the data for this dam during the preference elicitation in Step 3. We include the raw data values again here to help make the MCDA calculations more transparent, so you can clearly see what goes into the final calculation that produces Figures 1 and 2 above. In addition, you can use this table to sort decision alternatives in ascending or descending order in each column by clicking on the arrow next to the column header . Note: fish survival values shown here are discrete, but in reality, the values are network-dependent and would be impacted by upstream or downstream changes. They are presented here as the average of a range of possible values for this dam, depending on what happens at other dams. This interaction between decisions at other dams and these decision criteria are modeled in the multi-objective optimization that leads to the final Map Recommendation.
Indigenous cultural traditions and lifeways data come from a survey of Penobscot Nation citizens (N=2), supported by informal conversations with Penobscot Nation citizens and representatives.
Table 3. Normalized Data Values for Millinocket Lake Dam
Download TableResults Interpretation for Table 3. This table shows the data values from Table 2, normalized to be a score between 0 and 1 to make them comparable across different units. Normalization was performed using a min/max procedure: each raw data value was subtracted from the ideal value in the set (e.g., the maximum fish habitat area, for example) and divided by the difference between the maximum and minimum values in the set. The highest normalized values for most decision criteria, then, equal 1, and the lowest values equal 0. For decision criteria where lower values are more preferable (e.g. annuitized project cost, breach damage potential, number of properties impacted), the highest values equal 0, and the lowest values equal 1. This allows us to indicate that, for example, high costs are less desirable than low costs. The normalized data values in this table are multiplied by the preference weights displayed in Figure 3 and Table 1 to calculate the weighted scores in Table 4.
Normalization is a mathematical procedure that sets data values relative to their actual range, where the least desirable value (e.g., highest cost, lowest greenhouse gas emission reductions) in the existing data range is set equal to 0 and the most desirable value (e.g., lowest cost, highest greenhouse gas emissions reductions) in the existing data range is set equal to 1. In this way, normalization allows us to compare apples and oranges as if they were apples and apples. The normalization (minimax, maximin) procedure for the Dam Decision Support Tool bounds data within a range specific to the individual dam, not to the entire set of dams (i.e., the normalization operation makes the criteria data comparable for decision alternatives at an individual dam). Just as it does not consider data values for decision criteria at other dams, normalization does not consider possible improvements to the existing data. The highest and lowest values for the decision criterion for the specific dam site at set equal to zero or one, depending on what is considered desirable for the criterion (e.g., for cost, the lowest value is most desirable). Future studies could update the normalization procedure to work not only within the range for a specific dam but also across the entire set of dams.
Table 4. Weighted Scores for Individual Decision Criteria and Alternatives for Millinocket Lake Dam
Download TableResults Interpretation for Table 4. This table shows the result of multiplying the preference scores from Table 1 (and Figure 3) by the normalized decision criteria data values displayed in Table 3. If you add together all numbers in one row in this table you will get the final MCDA score for that decision alternative, the same results that are presented in Figures 1-2 above.
Results: Ripogenus Dam
Now that you have seen the combined, top-ranked dam decision alternatives, we will drill down to explore the MCDA results for each individual dam. Remember that these results have been estimated using site-specific data values.Figure 1. Comparison of Final MCDA Scores for Each Decision Alternative
Download GraphResults Interpretation for Figure 1: This graph shows the final MCDA score for each decision alternative for this specific dam, based on the MCDA calculation that includes the preferences you entered in Step 3 and the decision criteria data we collected/generated through our research. The taller the bar, the more preferred the decision alternative us under the preferences you supplied. Use this graph for a quick comparison between decision alternatives for a single dam.
Figure 2. Contribution of Decision Criteria to Final MCDA Score for Dam Decision Alternatives
Download GraphResults Interpretation for Figure 2: This graph displays a zoomed-in version of Figure 1, with the final MCDA score bars for each decision alternative divided up by the contribution of each decision criterion to the final MCDA score. Similar to Figure 1, this graph includes your preference information and the researcher-defined data for each decision criterion. Use this graph to drill down and see which decision criteria are making up the largest portion of the final score for each decision alternative and whether you agree that is the way it should be. Remember, however, that these data take into account not only your preferences but also the research data. So, even if you gave a decision criterion a low rating in Step 3, it could still make up a large portion of this graph (unless you marked it as zero) because the data values may be larger relative to the full set of data values for that decision criteria for this dam. For example, if you assigned a 0.1 preference value to sea-run fish habitat area, you may be surprised to see a large segment for sea-run fish habitat in the Remove Dam decision alternative bar in this graph, but that could be because removing the dam increases fish habitat the most out of all of the decision alternatives for this dam site.
Data Inputs to these Results
The results presented in Figures 1 and 2 were calculated by multiplying your preference inputs from Step 3 by the normalized researcher-defined decision criteria data. This section displays these component data sets, reminding you in table and graph form of the preferences you entered in Step 3 for each decision criterion and showing you the researcher-defined decision criteria data sets (raw and normalized) that lead to the final calculation.Figure 3. User-Defined Preference Scores for Ripogenus Dam
Download GraphResults Interpretation for Figure 3: This graph shows you the preferences you entered in Step 3 for each decision criterion. The scores are pulled directly from your slider bar settings under the West Enfield Dam tab and are not changed in any way. Note: Users may slide bars to indicate level of preference, but as of May 2020 we do not have data for these social decision criteria aside from Indigenous Cultural Traditions and Lifeways.If you wish to go back and change your settings, please do so before continuing. Remember to click GENERATE under Step 5. Multi-Dam Results.
Table 1. User-Defined Preference Scores for Ripogenus Dam
Download RipogenusThis table just shows the same thing as Figure 3 but in table form. If you would like to see all decision criteria preferences values at once, please select Show 25 entries from the drop-down menu above the table. Use the search bar to filter the table to a specific decision alternative (e.g. Keep and Maintain Dam).
Table 2. Data Values for Ripogenus Dam
Download Table Download Dam DataResults Interpretation for Table 2. This table displays the raw data values we collected and/or calculated/generated through our research for each decision criterion and alternative. You may remember seeing these data when you clicked on the link for the data for this dam during the preference elicitation in Step 3. We include the raw data values again here to help make the MCDA calculations more transparent, so you can clearly see what goes into the final calculation that produces Figures 1 and 2 above. In addition, you can use this table to sort decision alternatives in ascending or descending order in each column by clicking on the arrow next to the column header . Note: fish survival values shown here are discrete, but in reality, the values are network-dependent and would be impacted by upstream or downstream changes. They are presented here as the average of a range of possible values for this dam, depending on what happens at other dams. This interaction between decisions at other dams and these decision criteria are modeled in the multi-objective optimization that leads to the final Map Recommendation.
Indigenous cultural traditions and lifeways data come from a survey of Penobscot Nation citizens (N=2), supported by informal conversations with Penobscot Nation citizens and representatives.
Table 3. Normalized Data Values for Ripogenus Dam
Download TableResults Interpretation for Table 3. This table shows the data values from Table 2, normalized to be a score between 0 and 1 to make them comparable across different units. Normalization was performed using a min/max procedure: each raw data value was subtracted from the ideal value in the set (e.g., the maximum fish habitat area, for example) and divided by the difference between the maximum and minimum values in the set. The highest normalized values for most decision criteria, then, equal 1, and the lowest values equal 0. For decision criteria where lower values are more preferable (e.g. annuitized project cost, breach damage potential, number of properties impacted), the highest values equal 0, and the lowest values equal 1. This allows us to indicate that, for example, high costs are less desirable than low costs. The normalized data values in this table are multiplied by the preference weights displayed in Figure 3 and Table 1 to calculate the weighted scores in Table 4.
Normalization is a mathematical procedure that sets data values relative to their actual range, where the least desirable value (e.g., highest cost, lowest greenhouse gas emission reductions) in the existing data range is set equal to 0 and the most desirable value (e.g., lowest cost, highest greenhouse gas emissions reductions) in the existing data range is set equal to 1. In this way, normalization allows us to compare apples and oranges as if they were apples and apples. The normalization (minimax, maximin) procedure for the Dam Decision Support Tool bounds data within a range specific to the individual dam, not to the entire set of dams (i.e., the normalization operation makes the criteria data comparable for decision alternatives at an individual dam). Just as it does not consider data values for decision criteria at other dams, normalization does not consider possible improvements to the existing data. The highest and lowest values for the decision criterion for the specific dam site at set equal to zero or one, depending on what is considered desirable for the criterion (e.g., for cost, the lowest value is most desirable). Future studies could update the normalization procedure to work not only within the range for a specific dam but also across the entire set of dams.
Table 4. Weighted Scores for Individual Decision Criteria and Alternatives for Ripogenus Dam
Download TableResults Interpretation for Table 4. This table shows the result of multiplying the preference scores from Table 1 (and Figure 3) by the normalized decision criteria data values displayed in Table 3. If you add together all numbers in one row in this table you will get the final MCDA score for that decision alternative, the same results that are presented in Figures 1-2 above.
Limitations
This Dam Decision Support Tool is not in any way associated with FERC requirements for the relicensing process. While the Tool can be used to understand the impact of preferences on decision making, it reflects only the user's preference-driven outcomes. Other people will have different preferences, which may result in different outcomes. Using the Tool to simulate others' preferences is one way of exploring the impact that different preferences have on the outcome.Note: all the decision criteria mentioned herein are defined specifically in relation to hydropower dams. This list is non-exhaustive; it does not include all decision criteria that may be important to all people. The criteria identified in this list are representative of current issues important to dam decision making as expressed by stakeholders in interview. A more extensive discussion of decision criteria-specific limitations and ideas for future studies is included in the list of decision criteria.
Existing state and federal laws also limit the usefulness of these results/outcomes. For example: (1) Penobscot Nation subsistence fishing law, (2) Endangered Species Act (often used as a regulatory 'hammer' in dam decisions by enforcing federal agencies), (3) Clean Water Act (401 water quality certification), (4) Federal Power Act, (5) municipal ordinances (a town can explicitly specify that no new dams may be constructed or that no hydropower operations may be installed within town boundaries) all have direct legal bearing on the outcome of a FERC relicensing process. If brought to bear in the decision-making process, such laws and regulations may exclude decision alternatives from the 'problem set' altogether. Future work could involve possible regulatory 'scenarios' under which particular 'problem sets' of decision alternatives are relevant.
Developers
Dr. Emma Fox - Lead Developer (Graduate Research Assistant, University of Maine Ecology and Environmental Science Program) for the Dam Decision Support Tool. Designed user interface and initial server functionality. Adjusted WSM function for new dam decision application and advised model-related changes. Designed and wrote app text, and designed accompanying multi-dam decision example fact sheets, designed and wrote text for Dam Toolbox.Dr. Sharon J. W. Klein - Development Advisor (Associate Professor, University of Maine School of Economics). Helped develop and advise concept for the Dam Decision Support Tool, advised user-friendliness enhancements to the Dam Decision Support Tool and user interface/features, refined criteria definitions, revised app text.
Dr. Samuel G. Roy - Feature Developer (Postdoctoral Researcher, Senator George J. Mitchell Center for Sustainability Solutions, University of Maine) for the Dam Decision Support Tool. Created original Matlab scripts for multi-rank calculation, fitness functions for network-dependent criteria, and all maps.
William Winslow - Developer (Software Engineer, GeoSpatial Science Center(GSSC), University of New Hampshire). Deployment (Docker, Apache), server code reorganization, debugging/bug fixes, misc. feature implementations for user interface and app function.
Acknowledgments
We would like to thank Garrett Raymond for his technical consultation on R Shiny design.This tool was developed by researchers in the Future of Dams project. Support for the Future of Dams project is provided by the National Science Foundation's Research Infrastructure Improvement NSF #IIA-1539071, USDA National Institute of Food and Agriculture, Hatch project 0230040, and Department of the Interior, U.S. Geological Survey Grant No. G16AP00057 through the Senator George J. Mitchell Center at the University of Maine.
The Data Discovery Center at the University of New Hampshire is the host for this Dam Decision Support Tool. https://ddc.unh.edu
Citations (in alphabetical order)
B. Blachly and E. Uchida, Estimating the Marginal Cost of Dam Removal, in Environmental and Natural Resource Economics Working Papers, 2017, vol. Paper 2.C. Song, K. H. Gardner, S. J. W. Klein, S. P. Souza, and W. Mo, Cradle-to-grave greenhouse gas emissions from dams in the United States of America, Renew. Sustain. Energy Rev., vol. 90, pp. 945-956, Jul. 2018, doi: 10.1016/j.rser.2018.04.014.
D. G. Hall, K. S. Reeves, J. Brizzee, R. D. Lee, G. R. Carroll, and G. L. Sommers, Feasibility Assessment of the Water Energy Resources of the United States for New Low Power and Small Hydro Classes of Hydroelectric Plants, Idaho National Laboratory, DOE-ID-11263, Jan. 2006. Accessed: Jun. 06, 2016. [Online]. Available: http://www1.eere.energy.gov/water/pdfs/doewater-11263.pdf.
FERC: Documents & Filing - FERC Online - eLibrary. https://www.ferc.gov/docs-filing/elibrary.asp (accessed Mar. 13, 2020).
Form EIA-923 detailed data with previous form data (EIA-906/920). https://www.eia.gov/electricity/data/eia923/ (accessed Jan. 16, 2020).
P. W. O'Connor, S. T. DeNeale, D. R. Chalise, E. Centurion, and A. Maloof, Hydropower Baseline Cost Modeling, Version 2, Oak Ridge National Laboratory (ORNL), ORNL/TM-2015/471, Sep. 2015. Accessed: Apr. 20, 2016. [Online]. Available: http://info.ornl.gov/sites/publications/files/Pub58666.pdf.
Roy, S.G., Uchida, E., de Souza, S.P., Blachly, B., Fox, E., Gardner, K., Gold, A.J., Jansujwicz, J., Klein, S., McGreavy, B., Mo, W., Smith, S.M.C., Vogler, E., Wilson, K., Zydlewski, J., & Hart, D. (2018). A multiscale approach to balance trade-offs among dam infrastructure, river restoration, and cost. Proceedings of the National Academy of Sciences, 201807437. doi:10.1073/pnas.1807437115.
S. J. Klein and S. Whalley, Comparing the sustainability of US electricity options through multi-criteria decision analysis, Energy Policy, vol. 79, pp. 127-149, 2015.
W. Short, D. J. Packey, and T. Holt, A manual for the economic evaluation of energy efficiency and renewable energy technologies, NREL/TP--462-5173, 35391, Mar. 1995. doi: 10.2172/35391.
Software and Programs Used (in alphabetical order)
Alex Couture-Beil (2018). rjson: JSON for R. R package version 0.2.20. https://CRAN.R-project.org/package=rjson .Alex Gaynor, Carlton Gibson (2020) django-filter, version 2.3.0. https://django-filter.readthedocs.io/en/stable/ .
C. Sievert (2020). Interactive Web-Based Data Visualization with R, plotly, and shiny. Chapman and Hall/CRC Florida.
Dean Attali (2020). shinyjs: Easily Improve the User Experience of Your Shiny Apps in Seconds. R package version 1.1. https://CRAN.R-project.org/package=shinyjs .
Django (2020). Django version 2.2. Retrieved from https://djangoproject.com .
Erich Neuwirth (2014). RColorBrewer: ColorBrewer Palettes. R package version 1.1-2. https://CRAN.R-project.org/package=RColorBrewer .
Federico Di Gregorio, Daniele Varrazzo, The Psycopg Team (2020). psycopg2. https://www.psycopg.org/docs/index.html .
Hadley Wickham, Romain François, Lionel Henry and Kirill Müller (2018). dplyr: A Grammar of Data Manipulation. R package version 0.7.6. https://CRAN.R-project.org/package=dplyr .
Hadley Wickham (2009). ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. https://cran.r-project.org/web/packages/ggplot2/index.html .
Joe Cheng, Bhaskar Karambelkar and Yihui Xie (2019). leaflet: Create Interactive Web Maps with the JavaScript Leaflet Library. R package version 2.0.3. https://CRAN.R-project.org/package=leaflet .
Matt Dowle and Arun Srinivasan (2019). data.table: Extension of data.frame. R package version 1.12.8.https://CRAN.R-project.org/package=data.table .
Python Software Foundation (2020). Python Language Reference, version 3.7. https://docs.python.org/3/reference/ .
The PostgreSQL Global Development Group (2020). PostgreSQL database, version 9.6. https://www.postgresql.org/docs/9.6/reference.html .
Tom Christie, Encode OSS Ltd. (2020). Django REST Framework, version 3.11. https://www.django-rest-framework.org/ .
R Core Team (2017). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/ .
RStudio Team (2015). RStudio: Integrated Development for R. RStudio, Inc., Boston, MA URL http://www.rstudio.com/ .
Simon Urbanek and Jeffrey Horner (2020). Cairo: R Graphics Device using Cairo Graphics Library for Creating High-Quality Bitmap (PNG, JPEG, TIFF), Vector (PDF, SVG, PostScript) and Display (X11 and Win32) Output.R package version 1.5-12. https://CRAN.R-project.org/package=Cairo .
Tony Plate and Richard Heiberger (2016). abind: Combine Multidimensional Arrays. R package version 1.4-5. https://CRAN.R-project.org/package=abind .
Winston Chang, Joe Cheng, JJ Allaire, Yihui Xie and Jonathan McPherson (2020). shiny: Web Application Framework for R. R package version 1.4.0.2. https://CRAN.R-project.org/package=shiny .
Yihui Xie, Joe Cheng and Xianying Tan (2020). DT: A Wrapper of the JavaScript Library DataTables. R package version 0.13. https://CRAN.R-project.org/package=DT .