Abstract
Purpose: This study introduces managerial techniques applied for the first time in the high-level strategic public policy decision-making process in Slovakia with an aim to assess the strategic decision-making of groups of experts in a methodologically supported environment. It compares groups of internal analysts and external specialists and should demonstrate the extent to which these two groups are able to process problems analytically and suppress intuition.
Methodology/Approach: Multi-criteria decision methods are used when deciding on complex problems. One of the most popular and most frequently used is the Analytic Hierarchy Process. Application of this method enables measurement of preference consistency, and its relationship with cognitive reflection.
Findings: Consistency of judgement was very similar in both groups. The prioritisation of measures resulted in a similar set of priorities determined by both groups. The assumed relationship of consistency and cognitive reflection score and/or overconfidence was not detected, and decision makers proved to be well calibrated.
Research Limitation/Implication: The main limitation of our research was the small sample size of decision makers, which complied with the requirements of the decision method, but was not sufficient to confirm the statistical validity.
Originality/Value of paper: The introduction of the multi-criteria decision method into decision-making for public policy strategies combines practical policy exercises with scientific research on high-stakes decisions and enables to carry out participatory decision-making process with relevant stakeholders.
Full text article
References
Bačová, V., 2010. Rozhodovanie a usudzovanie. Pohľady psychológie a ekonómie I. Bratislava: Ústav experimentálnej psychológie SAV.
Baláž, V., 2009. Riziko a neistota. Úvod do behaviorálnej ekonómie a financií. Bratislava: VEDA.
Baláž, V., Dokupilová, D. and Filčák, R., 2021. Participatory multi-criteria methods for adaptation to climate change. Mitigation and Adaptation Strategies for Global Change, 26(4), pp.1-22. DOI: 10.1007/s11027-021-09955-4.
Bazerman, M.H. and Moore, D.A., 2013. Judgment in Managerial Decision Making. Wiley.
Bernasconi, M., Choirat, C. and Seri, R., 2014. Empirical properties of group preference aggregation methods employed in AHP: Theory and evidence. European Journal of Operational Research, 232(3), pp.584-592. DOI: 10.1016/j.ejor.2013.06.014.
Berrittella, M., Certa, A., Enea, M. and Zito, P., 2011. An Analytic Hierarchy Process for the Evaluation of Transport Policies to Reduce Climate Change Impacts. [FEEM Working Paper No. 12.2007] SSRN Electronic Journal. Available at SSRN: < https://ssrn.com/abstract=962379 > DOI: 10.2139/ssrn.962379.
Bhushan, N. and Rai, K., 2007. Strategic Decision Making: Applying the Analytic Hierarchy Process. Springer Science & Business Media.
Burgman, M.A., McBride, M., Ashton, R., Speirs-Bridge, A., Flander, L., Wintle, B., Fidler, F., Rumpff, L. and Twardy, Ch., 2011. Expert Status and Performance. PLoS ONE, 6(7), e22998. DOI: 10.1371/journal.pone.0022998.
Čavojová, V. and Hanák, R., 2014. Assessing fluid rationality and its relation to cognitive styles. In: T. Ahram, W. Karwowski and Marek, T., eds. Proceedings of the 5th International Conference on Applied Human Factors and Ergonomics. Krakow: AHFE. pp.142-153.
Chu, P. and Kuand-Han, L. J., 2002. Note on consistency ratio. Mathematical and Computer Modelling, 35, pp.1077-1080.
Danesh, D., Ryan, M. and Abbasi, A., 2017. A Systematic Comparison of Multi Criteria Decision Making Methods for the Improvement of Project Portfolio Management in Complex Organisations. International Journal of Management and Decision Making, 16(1), pp.280-320.
Dawes, R., 1979. The Robust Beauty of Improper Linear Models in Decision Making. American Psychologist, 34, pp.571-82.
Dokupilová, D., Baláž, V., Kurincová Čavojová, V., Ballová Mikušková, E. and Gombitová, D., 2020. Identifying major policy challenges and policy interventions via expert methods application of the Delphi and AHP methods in preparation of the Partnership Agreement for the Slovak Republic in period 2021-2027. Národohospodářský obzor, 20(3), pp.361-377. DOI: 10.2478/revecp-2020-0017.
Dunning, D., Johnson, K., Ehrlinger, J. and Kruger, J., 2003. Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12(3), pp.83-87. DOI: 10.1111/1467-8721.01235.
Emrouznejad, A. and Marra, M., 2017. The state-of-the-art development of AHP (1979–2017): A literature review with a social network analysis. International Journal of Production Research, 55(22), pp.6653-6675.
Franek, J. and Kresta, A., 2014. Judgment scales and consistency measure in AHP. Precedia Economics and Finance, 12, pp.164-173.
Frederick, S., 2005. Cognitive Reflection and Decision Making. Journal of Economic Perspectives, 19(4), pp.25-42. DOI: 10.1257/089533005775196732.
Gilmour, P. and Corner, J.L., 1998. The Role of the Expert’s Decision Making Skills in Management. 33rd Annual Operational Research Society of New Zealand, pp.195-204.
Hutton, R.J.B. and Klein, G., 1990. Expert decision making. Systems Engineering, 2(1), pp.32-45.
Ishizaka A. and Labib A., 2011. Review of the main developments in the analytic hierarchy process. Expert Systems with Applications, 38(11), pp.14336-14345.
Juanchich, M., Sirota, M. and Bonnefon, J. F., 2020. Anxiety-induced miscalculations, more than differential inhibition of intuition, explain the gender gap in cognitive reflection. Journal of Behavioural Decision Making, 33, pp.427-443. DOI: 10.1002/bdm.2165.
Kahneman, D., 2011. Thinking Fast and Slow. New York: Farrar, Straus and Giroux.
Kruger, J. and Dunning, D., 1999. Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence led to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), pp.1121-1134.
Larrick, R.P. and Feiler, D.C., 2015. Expertise in decision making. In: G. Keren, G. Wu, eds. 2015. The Wiley Blackwell Handbook of Judgment and Decision Making. John Wiley & Sons. pp.696-721.
Lockton, D., 2012, Cognitive biases, heuristics and decision-making in design for behaviour change. [Working Paper] SSRN Electronic Journal. Available at SSRN: < https://ssrn.com/abstract=2124557 >. DOI: 10.2139/ssrn.2124557.
Mu, E. and Pereyra-Rojas, M., 2017. AHP Models with Sub-criteria. In: E. Mu and M. Pereyra-Rojas, eds., 2017. Practical Decision Making. Cham: Springer. pp.45-56.
Noori, M., 2016. Cognitive reflection as a predictor of susceptibility to behavioural anomalies. Judgment and Decision Making, 11(1), pp.114-120.
OECD, 2017. Policy Advisory Systems: Supporting Good Governance and Sound Public Decision Making. Paris: OECD Public Governance Reviews, OECD Publishing. DOI: 10.1787/9789264283664-en.
Oppenheimer, D.M. and Thomson, K.S., 2016. Investigation of an alternate form of the Cognitive Reflection Test. Judgment and Decision making, 11(1), p.99.
Ossadnik, W., Schinke, S. and Kaspar, R.H., 2016. Group Aggregation Techniques for Analytic Hierarchy Process and Analytic Network Process: A Comparative Analysis. Group Decision and Negotiation, 25(2), pp.421-457. DOI: 10.1007/s10726-015-9448-4.
Pennycook, G., Ross, R. M., Koehler, D. J. and Fugelsang, J. A., 2017. Dunning–Kruger effects in reasoning: Theoretical implications of the failure to recognize incompetence. Psychonomic Bulletin and Review, 24(6), pp.1774-1784.
Peregrin, S. and Karahuta, M., 2014. Skupinový analytický hierarchický proces a jeho využitie v manažmente spoločnosti/Group analytic hierarchy process and its usage in management of company. Mladá Veda, 2(3), p.60.
Primi, C., Morsanyi, K., Chiesi, F., Donati, M.A. and Hamilton, J., 2016. The development and testing of a new version of the cognitive reflection test applying item response theory (IRT). Journal of Behavioral Decision Making, 29(5), pp.453-469.
Saaty, T., 2008. Decision making with the analytic hierarchy process. International Journal of Services Sciences, 1(1), pp.83-98.
Saaty, T.L. and Özdemir, M.S., 2014. How Many Judges Should There Be in a Group? Annals of Data Science, 1(3-4), pp.359-368.
Saaty, T.L. and Peniwati, K., 2013. Group decision making: drawing out and reconciling differences. RWS publications.
Shanteau, J. and Stewart, R., 1992. Why Study Expert Decision Making? Some Historical Perspectives and Comments. Organisational Behaviour and Human Decision Processes, 53, pp.95-106.
Shanteau, J., Weiss, D., Thomas, R.P. and Pounds, J.C., 2002. Performance based assessment of expertise: How to decide if someone is an expert or not. European Journal of Operational Research, 136(2), pp.253-263.
Sičáková-Beblavá, E., 2015. Behaviorálne základy verejnej politiky. Bratislava: Univerzita Komenského v Bratislave.
Sinayev, A. and Peters, E., 2015. Cognitive reflection vs. calculation in decision making. Frontiers in Psychology, 6, p.532. DOI: 10.3389/fpsyg.2015.00532.
Sirota, M. and Juanchich, M., 2018. Effect of response format on cognitive reflection: Validating a two- and four-option multiple choice question version of the Cognitive Reflection Test. Behaviour Research Methods, 50, pp.2511-2522.
Sirota, M., Dewberry, Ch., Janchich, M., Valuš, L. and Marshall, A.C., 2021. Measuring cognitive reflection without maths: Development and validation of the verbal cognitive reflection test. Journal of Behavioural Decision Making, 34, pp.322-343. DOI: 10.1002/bdm.2213.
Sleboda, P. and Sokolowska, J., 2017. Measurements of Rationality: Individual Differences in Information Processing, the Transitivity of Preferences and Decision Strategies. Frontiers in Psychology, 8, p.1844. DOI: 10.3389/fpsyg.2017.0184
Šoltés, V. and Gavurová, B., 2014. The functionality comparison of the health care systems by the analytical hierarchy process method. Ekonomie a Management, 17(3), pp.1-18.
Subramanian, N. and Ramanathan, R., 2012. A review of applications of Analytic Hierarchy Process in operations management. International Journal of Production Economics, 138, pp.215-241. DOI: 10.1016/j.ijpe.2012.03.036
Tetlock, P.E., 2005. Expert Political Judgment: How Good Is It? How Can We Know? Princeton University Press.
Toplak, M.E., West, R.F. and Stanovich, K.E., 2011. The Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasks. Memory & Cognition, 39(7), pp.1275-1289. DOI: 10.3758/s13421-011-0104-1.
Toplak, M.E., West, R.F. and Stanovich, K.E., 2014. Assessing miserly information processing: An expansion of the Cognitive Reflection Test. Thinking & Reasoning, 20(2), pp.147-168. DOI: 10.1080/13546783.2013.844729.
Tversky, A., 1969. Intransitivity of preferences, Psychological Review, 76, pp.31-48.
Vargas, R.V., 2010. Using the analytic hierarchy process (AHP) to select and prioritize projects in a portfolio. In: Project Management Institute (PMI®), Global Congress 2010-North America. 12 October 2010. Washington, DC. Newtown Square, PA: Project Management Institute. Available at: < https://www.pmi.org/learning/library/analytic-hierarchy-process-prioritize-projects-6608 > [10 December 2021].
Velasquez, M. and Hester, P.T., 2013. An Analysis of Multi-Criteria Decision-Making Methods. International Journal of Operations Research, 10(2), pp.55-66.
Vidal, L.M., Marle, F. and Bocquet, J.C., 2011. Using a Delphi process and the Analytic Hierarchy Process (AHP) to evaluate the complexity of projects. Expert Systems with Applications, 38(5), pp.5388-5405.
Wagenaar, W.A. and Keren, G.B., 1985. Calibration of probability assessments by professional blackjack dealers, statistical experts, and lay people. Organizational Behaviour and Human Decision Processes, 36(3), pp.406-416.
Wallenius, J., Dyer, J.S., Fishburn, P.C., Steuer, R.E., Zionts, S. and Deb, K., 2008. Multiple criteria decision-making, multi-attribute utility theory: Recent accomplishments and what lies ahead. Management Science, 54(7), pp.1336-1349.
Weinstein, B.D., 1993. What is an expert? Theoretical Medicine and Bioethics, 14(1), pp.57-73.
Welsh, M.B., Burns, N.R. and Delfabbro, P.H., 2013. The Cognitive Reflection Test: How much more than numerical ability? In: M. Knauff, M. Pauen, N. Sebanz and I. Wachsmuth, ed., 35th Annual Conference of the Cognitive Science Society. Berlin, Germany, July 31-August 3, 2013. Austin, TX: Cognitive Science Society. pp.1587-1592.
Wu, Z. and Tu, J., 2020. Managing transitivity and consistency of preferences in AHP group decision making based on minimum modifications. Information Fusion, 67, pp.125-135. DOI: 10.1016/j.inffus.2020.10.012.
Zyoud, S.H. and Fuchs-Hanusch, D., 2017. A bibliometric-based survey on AHP and TOPSIS techniques. Expert Systems with Applications, 78(C), pp.158-181. DOI: 10.1016/j.eswa.2017.02.016.
Authors
Copyright (c) 2022 Dagmar Gombitová, Dušana Dokupilová
This work is licensed under a Creative Commons Attribution 4.0 International License.
This is an open access journal which means that all content is freely available without charge to the user or his/her institution. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles in this journal without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open access. This journal is licensed under a Creative Commons Attribution 4.0 License - http://creativecommons.org/licenses/by/4.0.
Authors who publish with the Quality Innovation Prosperity agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.