The Delphi Study in Media Literacy (MEDLIT) Erasmus+ Project: Lessons Learnt

Rosya Izyanie Shamshudeen, Nik Maheran Nik Muhammad, Sabariah Mohamed Salleh, Mokhtarrudin Ahmad, Wan Anita Wan Abas

Abstract


The procedure and practice in conducting a study on media literacy education intervention need to be clarified to guide quality research. In recent years, only limited Delphi studies have been conducted in the Malaysian context. Despite recognizing the influence of socio-cultural settings on the conduct of any research, empirical literature on the best and most suitable means to conduct Delphi studies within the Malaysian socio-cultural context has not grown in proportion to the evolution of Delphi method application in local social-science studies. This article establishes methodological considerations in employing the Delphi method by investigating issues relating to data source, instrument, data collection procedures and management in the implementation in designing and conducting a successful Delphi study in the Malaysian socio-cultural context. The reflection of Delphi method in identifying media literacy knowledge that is deemed to be important for inclusion in an introductory media literacy Massive Open Online Course (MOOC) will be explored. This study demonstrates a novel and inclusive strategy that might be used in Southeast Asian nations. Our study suggested appropriate strategies in order to develop and carry out more feasible Delphi studies. New researchers should take into consideration the numerous issues and difficulties identified in this study. The main contribution of this study is a set of dependable and useful guidelines that may be used to design and carry out future Delphi studies that include the study's reasoning, justification of its advantages, the official logo of the institutions the researchers are affiliated with, and authorization from the funding organisations, which may add weightage to the study's significance.

 

Keywords: Delphi method, Massive Open Online Course (MOOC), media literacy, media competency, curriculum development.

 

https://doi.org/10.17576/JKMJC-2023-3902-10


Full Text:

PDF

References


Akins, R. B., Tolson, H. & Cole, B. R. (2005). Stability of response characteristics of a Delphi panel: Application of bootstrap data expansion. BMC Medical Research Methodology, 5, 37. https://doi.org/10.1186/1471-2288-5-37

Avella, J. R. (2016). Delphi panels: Research design, procedures, advantages and challenges. International Journal of Doctoral Studies, 11, 305-321.

Baboo, S. B. (2001). Media literacy in Malaysia: Making connection with critical awareness. In Mary Kalantzis and Ambiagapathy Pandian (Eds.), Literacy matters: Issues for new media (pp. 161-172). Penang: Universiti Sains Malaysia.

Bakeer, A., & Wynn, M. (2020). Towards the successful adoption of MOOCs in Libyan higher education: A case study of the University of Misurata. Journal of Pure and Applied Sciences, 19(4), 20-26.

Batool, S. H., Rehman, A. U., & Sulehri, I. (2022). The current situation of information literacy education and curriculum design in Pakistan: A discovery using Delphi method. Library Hi Tech, 40(6), 1705-1720. https://doi.org/10.1108/LHT-02-2021-0056

Beaumont, D. G. (2003). The interaction between general practitioners and occupational health professionals in relation to rehabilitation for work: A Delphi study. Occupational Medicine, 53(4), 249-253. https://doi.org/10.1093/occmed/kqg066

Boulkedid, R., Abdoul, H., Loustau, M., Sibony O., & Sibony, C. (2011). Using and reporting the Delphi method for selecting healthcare quality indicators: A systematic review. PLoS One, 6(6), e20476. https://doi.org/fnpbc5

Buck A. J., Gross M., Hakim S., & Weinblatt J. (1993) Using the Delphi process to analyse social policy implementation: A post hoc case from vocational rehabilitation. Policy Sciences, 26, 271-288. https://doi.org/dv6jkn

Bulger, M., & Davison, P. (2018). The promises, challenges, and futures of media literacy. Journal of Media Literacy Education, 10(1), 1-21. https://doi.org/ggh7zs

Brown, S. (2015). Transaction circles with digital texts as a foundation for democratic practices. Democracy and Education, 23(2), 1-12. https://democracyeducationjournal.org/home/vol23/iss2/4/

Baumgartner, H., & Steenkamp, J. B. E. M. (2001). Response styles in marketing research: A cross-national investigation. Journal of Marketing Research, 38(2), 143–156. https://doi.org/10.1509/jmkr.38.2.143.18840

Carlsson, U. (Ed.). (2019). Media and information literacy: Field of knowledge, concepts and history. In, Understanding media and information literacy (MIL) in the digital age: A question of democracy (pp. 37-56). Department of Journalism, Media and Communication (JMG).

Cheng, S., & King, J. A. (2017). Exploring organizational evaluation capacity and evaluation capacity building: A Delphi study of Taiwanese elementary and junior high schools. American Journal of Evaluation, 38(4), 521-539. https://doi.org/gcmcp2

Cho, H., Cannon, J., Lopez, R., & Li, W. (2022). Social media literacy: A conceptual framework. New Media & Society. https://doi.org/gpdcpz

Chyung, S. Y., Roberts, K., Swanson, I., & Hankinson, A. (2017). Evidence-based survey design: The use of a midpoint on the likert scale. Performance Improvement, 56(10), 15-23. https://doi.org/10.1002/pfi.21727

Davidson, P. L. (2013). The Delphi technique in doctoral research: Considerations and rationale. Review of Higher Education & Self-Learning, 6(22), 53-65.

De Abreu, B. S., Mihailidis, P., Lee, A.Y. L., Melki, J., & McDougall, J. (Eds.). (2017). International handbook of media literacy education (1st ed.). Routledge. https://doi.org/kf7g

Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning. Glenview, IL: Scott, Foresman, and Co.

De Villiers, M. R., De Villiers, P. J., & Kent, A. P. (2005). The Delphi technique in health sciences education research. Medical Teacher, 27(7), 639-643. https://doi.org/dbvztg

Dufva, T., & Dufva, M. (2019). Grasping the future of the digital society. Futures, 107, 17-28.

Dutta, S. (2018). Critical media literacy in a mediated democracy. Journal of Mass Communication & Journalism, 8(4), 1000377. https://doi.org/kfwz

Egfjord, K. F. H., & Sund, K. J. (2020). A modified Delphi method to elicit and compare perceptions of industry trends. MethodsX, 7, 1-9. https://doi.org/gp5bjr

Erstad, O. (2010). Media literacy and education. The past, present and future. In S. Kotilainen, S. & Arnolds-Granlund, S. (Eds.), Media literacy education: Nordic perspectives (pp. 15-27). Nordicom.

Festl, R. (2021). Social media literacy & adolescent social online behavior in Germany. Journal of Children and Media, 15(2), 249-271. https://doi.org/gjf6r3

Fini, A. (2009). The technological dimension of a massive open online course: The case of the CCK08 course tools. The International Review of Research in Open and Distributed Learning, 10(5), 1-26. https://doi.org/10.19173/irrodl.v10i5.643

Fink-Hafner, D., Dagen, T., Doušak, M., Novak, M., & Hafner-Fink, M. (2019). Delphi method: Strengths and weaknesses. Metodoloski Zvezki, 16(2), 1-9.

Fong, S. F., Ch’ng, P. E., & Por, F. P. (2013). Development of ICT competency standard using the Delphi technique. Procedia - Social and Behavioral Sciences, 103, 299–314. https://doi.org/10.1016/j.sbspro.2013.10.338

Gajendran, T., & Brewer, G. (2007). Integration of information and communication technology: Influence of the cultural environment. Engineering, Construction and Architectural Management, 14(6), 532-549. https://doi.org/d2tpfk

García-Jiménez, L., Rodrigo-Alsina, M., & Pineda, A. (2017). The social construction of intercultural communication: A Delphi study. Journal of Intercultural Studies, 38(2), 228-244. https://doi.org/10.1080/07256868.2017.1289903

Gordon, C. S., Rodgers, R. F., Slater, A. E., McLean, S. A., Jarman, H. K., & Paxton, S. J. (2020). A cluster randomized controlled trial of the SoMe social media literacy body image and wellbeing program for adolescent boys and girls: Study protocol. Body Image, 33, 27-37. https://doi.org/10.1016/j.bodyim.2020.02.003

Green, R. A. (2014). The Delphi technique in educational research. Sage Open, 4(2), 1-8. https://doi.org/10.1177/2158244014529773

Hirschhorn, F. (2019). Reflections on the application of the Delphi method: Lessons from a case in public transport research. International Journal of Social Research Methodology, 22 (3), 309-322. https://doi.org/gn4pbm

Harzing, A. W. (2006). Response styles in cross-national survey research: A 26-country study. International Journal of Cross-Cultural Management, 6(2), 243-266. https://doi.org/10.1177/1470595806066332

Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32(4), 1008-1015. https://doi.org/fkp685

Hasson, F., & Keeney, S. (2011). Enhancing rigour in the Delphi technique research. Technological Forecasting and Social Change, 78(9), 1695-1704. https://doi.org/10.1016/j.techfore.2011.04.005

Haworth, S., Montgomery, P., & Schaub, J. (2023). A Delphi study to develop items for a new tool for measuring child neglect for use by multi-agency practitioners in the UK. Social Sciences, 12(4), 239. https://doi.org/10.3390/socsci12040239

Hobbs, R. (2011). Keynote empowering learners with digital and media literacy. Knowledge Quest, 39(5), 13-17.

Hsu, C-C., & Sandford, B. A. (2007). The Delphi technique: Making sense of consensus. Practical Assessment, Research & Evaluation, 12(10), 1-7. https://doi.org/ghfcpw

Igbinovia, M. O., Okuonghae, O., & Adebayo, J. O. (2021). Information literacy competence in curtailing fake news about the COVID-19 pandemic among undergraduates in Nigeria. Reference Services Review, 49(1), 3-18. https://doi.org/gh4dtt

Johnson, T., Kulesa, P., Cho, Y. I., & Shavitt, S. (2005). The relation between culture and response styles: Evidence from 19 countries. Journal of Cross-Cultural Psychology, 36(2), 264-277. https://doi.org/10.1177/0022022104272905

Joshi, A., Kale, S., Chandel, S., & Pal, D. K. (2015). Likert scale: Explored and explained. British Journal of Applied Science & Technology, 7(4), 396-403. https://doi.org/dgbz

Ju, B., & Jin, T. (2013). Incorporating nonparametric statistics into Delphi studies in library and information science. Information Research: An International Electronic Journal, 18(3), 1-11.

Kamerer, D. (2013). Media literacy. Communication Research Trends, 32(1), 4-25.

Karupiah, P. (2015). Have beauty ideals evolved? Reading of beauty ideals in Tamil movies by Malaysian Indian youths. Sociological Inquiry, 85(2), 239-261.

Keller, J., & Heiko, A. (2014). The influence of information and communication technology (ICT) on future foresight processes—Results from a Delphi survey. Technological Forecasting and Social Change, 85, 81-92. https://doi.org/gdxrws

Lange, T., Kopkow, C., Lützner, J., Günther, K. P., Gravius, S., Scharf, H. P., Stöve, J., Wagner, R., & Schmitt, J. (2020). Comparison of different rating scales for the use in Delphi studies: Different scales lead to different consensus and show different test-retest reliability. BMC Medical Research Methodology, 20, 28. https://doi.org/gh6gjf

Lee, J. W., Jones, P. S., Mineyama, Y., & Zhang, X. E. (2002). Cultural differences in responses to a Likert scale. Research in Nursing & Health, 25, 295–306. https://doi.org/dmnkd7

Li, X., & Liu, Q. (2020). Social media use, eHealth literacy, disease knowledge, and preventive behaviors in the COVID-19 pandemic: Cross-sectional study on Chinese netizens. Journal of Medical Internet Research, 22(10), e19684. https://doi.org/10.2196/19684

Lincoln, Y. S., & Guba, E. G. (1985), Naturalistic inquiry. Sage Publications.

Ling, L. Y., Ali, M. N., & Mustaffa, N. (2019). Youth media literacy in interpreting the ideology of the film documentary the Malayan emergency (2010). Jurnal Komunikasi: Malaysian Journal of Communication, 35(2), 195-210.

Linke, A., & Zerfass, A. (2012). Future trends in social media use for strategic organisation communication: Results of a Delphi study. Public Communication Review, 2(2), 17-29. https://doi.org/10.5130/pcr.v2i2.2736

Linstone, H. A., & Turoff, M. (1975) The Delphi method techniques and applications. Boston: Addison-Wesley.

Livingstone, S. (2008). Engaging with media - A matter of literacy? Communication, Culture and Critique, 1(1), 51-62. https://doi.org/10.1111/j.1753-9137.2007.00006.x

Livingstone, S. (2004). What is media literacy? Intermedia, 32(3), 18-20.

Marfil-Carmona, R., & Chacón, P. (2017). Arts education and media literacy in the primary education teaching degree of the University of Granada. Procedia-Social and Behavioral Sciences, 237, 1166-1172.

Martens, H. (2010). Evaluating media literacy education: Concepts, theories and future directions. The Journal of Media Literacy Education, 2(1), 1-22.

McLean, S. A., Wertheim, E. H., Masters, J., & Paxton, S. J. (2017). A pilot evaluation of a social media literacy intervention to reduce risk factors for eating disorders. International Journal of Eating Disorders, 50(7), 847-851. https://doi.org/10.1002/eat.22708

McCormack, S., Till, K., Wenlock, J., Whitehead, S., Stokes, K. A., Bitcon, M., ... & Jones, B. (2022). Contributors to negative biopsychosocial health or performance outcomes in rugby players (CoNBO): A systematic review and Delphi study protocol. BMJ Open Sport & Exercise Medicine, 8(4), e001440. https://doi.org/kf7z

Md Azalanshah Md Syed, & Mohd Nazri Ibrahim. (2018). Literasi media, cabaran penyiaran sosial dan kelangsungan industri televisyen di Malaysia. Forum Komunikasi, 13(1), 15-39. https://forumkomunikasi.uitm.edu.my/index.php/fkvol13-1-2018

Moore, C. M. (1987). Group techniques for idea building. Thousand Oaks: Sage Publications.

Neo, R. (2021). A cudgel of repression: Analysing state instrumentalisation of the ‘fake news’ label in Southeast Asia. Journalism, 23(9), 1-20. https://doi.org/gh4dqd

Prasad, N. V., & Balraj, S. (2013). Developing media literacy practice among secondary school students in Malaysia: Case studies of video making on environmental issues. Media Watch, 4(1), 95-105. https://doi.org/10.1177/0976091120130109

Paré, G., Cameron, A.F., Poba-Nzaou, P. & Templier, M. (2013). A systematic assessment of rigor in information systems ranking-type Delphi studies. Information & Management, 50(5), 207-217. http://dx.doi.org/10.1016/j.im.2013.03.003

Popov, V., Jiang, Y., & So, H. J. (2020). Shared lessons in mobile learning among K-12 education, higher education and industry: An international Delphi study. Educational Technology Research and Development, 68(3), 1149-1180. https://doi.org/ghpfcn

Potter, W. J. (2019). Media literacy. (9th ed.). Thousand Oaks: Sage Publications.

Raaijmakers, Q., van Hoof, A., Hart, H., Verbgost, T., & Volleberge, W. (2000). Adolescences’ mid-point responses on Likert-type scale items: Neutral or missing values. International Journal of Public Opinion Research, 12(2), 208-216.

Robertson, S., Kremer, P., Aisbett, B., Tran, J., & Cerin, E. (2017). Consensus on measurement properties and feasibility of performance tests for the exercise and sport sciences: A Delphi study. Sports Medicine-Open, 3, 2. https://doi.org/ggkgd9

Rossouw, A., Hacker, M., & de Vries, M. J. (2011). Concepts and contexts in engineering and technology education: An international and interdisciplinary Delphi study. International Journal of Technology and Design Education, 21, 409-424.

Rowe, G., & Wright, G. (1999). The Delphi technique as a forecasting tool: Issues and analysis. International Journal of Forecasting, 15(4), 353-375.

Rüetschi, U., & Olarte Salazar, C. M. (2019). An e-Delphi study generates expert consensus on the trends in future continuing medical education engagement by resident, practicing, and expert surgeons. Medical Teacher, 42(4), 444-450. https://doi.org/kfw2

Sabariah Mohamed Salleh. (2020). Media and information literacy education in Malaysia. In M. Kajimoto, M. Ito, & M. K. Lim (Eds.), Media and information literacy education in Asia: Exploration of policies and practices in Japan, Thailand, Indonesia, Malaysia, and the Philippines (pp. 56-64). Bangkok: UNESCO Bangkok. https://unesdoc.unesco.org/ark:/48223/pf0000374575

Sabariah Mohamed Salleh, Rosya Izyanie Shamshudeen, Wan Anita Wan Abas, & Ezhar Tamam. (2019). Determining media use competencies in media literacy curriculum design for the digital society: A modified 2-wave Delphi method. SEARCH Journal of Media and Communication Research, 11(1), 17-36.

Sanz, M., Carrillo de Albornoz, A., Martín, C., Needleman, I., & Tonetti, M. S. (2023). Multi-stakeholder contribution to the identification of a core outcome set and measurements in implant dentistry (ID-COSM initiative) using the Delphi methodology. Journal of Clinical Periodontology, 50(S25), 107–121.

Sasaki, F. (2016). Online Political Efficacy (OPE) as a reliable survey measure of political empowerment when using the internet. Policy & Internet, 8(2), 197-214.

Shariff, N. (2015). Utilizing the Delphi survey approach: A Review. J Nurs Care, 4(3), 246-251. https://ecommons.aku.edu/cgi/viewcontent.cgi?article=1037&context=eastafrica_fhs_sonam

Shelton, K., Haynes, C. A., & Creghan, K. A. (2018). Fundamentals of Delphi research methodology. In V. Wang, & T. Reio Jr. (Ed.), Handbook of research on innovative techniques, trends, and analysis for optimized research methods (pp. 233-257). IGI Global. http://doi.org/10.4018/978-1-5225-5164-5.ch015

Shinta, A. D., Mohamad Salleh, M. A., & Ali, M. N. S. (2019). Analysis of the moderating effect of media literacy on cervical cancer preventive behaviours. Jurnal Komunikasi: Malaysian Journal of Communication, 35(1), 156-170. https://doi.org/kfw3

Skinner, R., Nelson, R. R. & Chin, W. W., & Land, L. (2015). The Delphi method research strategy in studies of information systems. Communications of the Association for Information Systems, 37, 31-63. http://doi.org/10.17705/1CAIS.03702

Skulmoski, G. J., Hartman, F. T., & Krahn, J. (2007). The Delphi method for graduate research. Journal of Information Technology Education, 6, 1-17.

Sourani, A., & Sohail, M. (2015). The Delphi method: Review and use in construction management research. International Journal of Construction Education and Research, 11(1), 54-76. https://doi.org/10.1080/15578771.2014.917132

Spivey, B. E. (1971). A technique to determine curriculum content. Journal of Medical Education, 46, 269-271.

Swart, J. (2021). Tactics of news literacy: How young people access, evaluate, and engage with news on social media. New Media & Society, 25(3), 505-521. https://doi.org/gjvwdz

Syam, H. M., & Nurrahmi, F. (2020). “I don’t know if it is fake or real news” How little Indonesian university students understand social media literacy. Jurnal Komunikasi: Malaysian Journal of Communication, 36(2), 92-105. https://doi.org/gg6cc9

Toma, C., & Picioreanu, I. (2016). The Delphi technique: Methodological considerations and the need for reporting guidelines in Medical Journals. International Journal of Public Health Research, 4, 47-59.

Thach, E. C., & Murphy, K. L. (1995). Competencies for distance education professionals. Educational Technology Research and Development, 43(1), 57-79.

Tsang, K. K. (2012). The use of midpoint on Likert scale: The implication for educational research. Hong Kong Teachers' Centre Journal, 11, 121-130.

Vergouw, D., Heymans, M. W., de Vet, H. C., Van Der Windt, D. A., & Van Der Horst, H. E. (2011). Prediction of persistent shoulder pain in general practice: Comparing clinical consensus from a Delphi procedure with a statistical scoring system. BMC Family Practice, 12, 63. https://doi.org/10.1186/1471-2296-12-63

Viswanathan, R. (2012). Teaching and learning through MOOC. Frontiers of Language and Teaching, 3(1), 32-40.

Wallner, C., Alpen, S., Adolf, M., Zita, M., B. (2017, December 14-16). Fostering Media Literacy Competencies for Navigating Digital Media Cultures: Findings of a Comparative Study in Southeast Asia. Paper Presentation at ICA Asian Regional Conference 2017. Communications Research in the Digital Age, Mumbai, India.

Watson, T. (2008). Public relations research priorities: A Delphi study. Journal of Communication Management, 12(2), 104-123. https://doi.org/cpt9pk

Weidman J. E., Miller K. R., Christofferson, J. P., & Newitt, J. S. (2011). Best practices for dealing with price volatility in commercial construction. International Journal of Construction Education and Research, 7(4), 276–293. https://doi.org/d2ztv2

Williams, P. L., & Webb, C. (1994). The Delphi technique: A methodological discussion. Journal of Advanced Nursing, 19(1), 180–186. https://doi.org/cswj99

Wu, M. J., Huang, C. Y., Kao, Y. S., Lue, Y. F., & Chen, L. C. (2018). Developing a professional performance evaluation system for pre-service automobile repair vocational high school teachers in Taiwan. Sustainability, 10(10), 3537. https://doi.org/gfm4wt

Wu, J., & Wang, Y. M. (2011). Unpacking new media literacy. Journal of Systemics, Cybernetics and Informatics, 9(2), 84-88.

Zalma, A. R., & Safiah, M. Y., & Ajau, D., & Anuar, M. I. K. (2013). Reliability and validity of television food advertising questionnaire in Malaysia. Health Promotion International, 30(3), 523-530. https://doi.org/10.1093/heapro/dat072

Zawacki-Richter, O. (2009). Research areas in distance education: A Delphi study. International Review of Research in Open and Distributed Learning, 10(3), 1-17. https://doi.org/10.19173/irrodl.v10i3.674


Refbacks

  • There are currently no refbacks.


e-ISSN: 2289-1528