Creating a Competency Domain Index: Enhancing assessment for online programming and electronic design for lower secondary school

Jing Rui Tang; Wan Nurlisa Wan Ahmad; Fardila Mohd Zaihidee; Jian Hong Me

doi:10.21834/e-bpj.v10iSI24.6374

Authors

Jing Rui Tang Faculty of Technical and Vocational, Universiti Pendidikan Sultan Idris, 35900 Tanjong Malim, Perak, Malaysia
Wan Nurlisa Wan Ahmad College of Electronics and Information Engineering, Hebei University, Via 54 East Rd 180, 071002 Baoding City, China.
Fardila Mohd Zaihidee Faculty of Technical and Vocational, Universiti Pendidikan Sultan Idris, 35900 Tanjong Malim, Perak, Malaysia
Jian Hong Me Faculty of Technical and Vocational, Universiti Pendidikan Sultan Idris, 35900 Tanjong Malim, Perak, Malaysia; College of Electronics and Information Engineering, Hebei University, Via 54 East Rd 180, 071002 Baoding City, China.

DOI:

https://doi.org/10.21834/e-bpj.v10iSI24.6374

Keywords:

Assessment, Competency, Education, Programming

Abstract

This study develops a Competency Domain Index (CDI) to enhance the assessment of online programming and electronic design for lower secondary schools in Malaysia. Conducted in two phases, the first identifies relevant competencies using the Nominal Group Technique (NGT), while the second develops the CDI with expert reviews. Results show over 70% expert consensus across all domains. The proposed CDI employs a spider web chart to visualise competencies, which are categorised according to Bloom’s taxonomy of cognitive levels, using normalised scores. Future work will focus on validating the CDI in schools to ensure its effectiveness and applicability.

References

Abdullah, M. M., & Islam, R. (2011). Nominal group technique and its applications in managing quality in higher education. Journal of Commercial & Social Sciences, 5(1), References 81-99.

Ashby-King, D. T., Iannacone, J. I., Ledford, V. A., Farzad-Phillips, A., Salzano, M., & Anderson, L. B. (2022). Expanding and constraining critical communication pedagogy in the introductory communication course: A critique of assessment rubrics. Communication Teacher, 36(3), 187-203. DOI: https://doi.org/10.1080/17404622.2021.1975789

Atherton, C. R. (2016). Group techniques for program planning: A guide to nominal group and Delphi processes. Group & Organization Studies, 1(2), 256.

Behforouz, B. (2022). Online assessment and the features in language education context: A brief review. Journal of Language and Linguistic Studies, 18(1), 564-576.

Buyrukoglu, S., Batmaz, F., & Lock, R. (2019). Improving marking efficiency for longer programming solutions based on a semi-automated assessment approach. Computer Applications in Engineering Education, 27(3), 733-743. DOI: https://doi.org/10.1002/cae.22094

Delbecq, A. L., & Van de Ven, A. H. (1971). A group process model for problem identification and program planning. Journal of Applied Behavioral Science, 7(4), 466-492. DOI: https://doi.org/10.1177/002188637100700404

Finstad, I., Knutstad, U., Havnes, A., & Sagbakken, M. (2022). The paradox of an expected level: The assessment of nursing students during clinical practice–A qualitative study. Nurse Education in Practice, 61, 103332. DOI: https://doi.org/10.1016/j.nepr.2022.103332

Harvey, N., & Holmes, C.A. (2012). Nominal group technique: An effective method for obtaining group consensus. International Journal of Nursing Practice, 18(2), 188-194. DOI: https://doi.org/10.1111/j.1440-172X.2012.02017.x

Kamarudin, H. A., Hanapi, Z., Rus, R. C., & Kiong, T. T. (2024). Nominal group technique: Application towards the construction of a visualization skills model technical communication graphics subject. Perdana: International Journal of Academic Research, 20(1), 17-28.

Kementerian Pendidikan Malaysia (KPM). (2017). DSKP Reka Bentuk dan Teknologi Tingkatan 2. Kurikulum Standard Sekolah Menengah.

Kilag, O. K., Andrin, G., Abellanosa, C., Villaver, M. Jr., Uy, F., & Sasan, J. M. (2024). MATATAG curriculum rollout: Understanding challenges for effective implementation. International Multidisciplinary Journal of Research for Innovation, Sustainability, and Excellence (IMJRISE), 1(5), 172-177.

Kwon, M., & Ko, I. (2024). Secondary mathematics teachers’ experiences of using ChatGPT to design probability and statistics assessment items. In ERME Topic Conference on Feedback & Assessment in Mathematics Education 2024.

Mahmood, A. S., Zain, A., & Zakaria, A. F. (2023). Nominal group technique application towards design of nutrition education module to prevent stunting problems among preschool children. International Journal of Academic Research in Progressive Education and Development, 12(1). DOI: https://doi.org/10.6007/IJARPED/v12-i1/16695

Maki, P. L. (2023). Assessing for learning: Building a sustainable commitment across the institution. Routledge. DOI: https://doi.org/10.4324/9781003443056

Marshall, R. (2008). Working with vagueness: Competence and vagueness in UK educational assessment. English Teaching, 7(1), 91.

Mohamed Salleh, F. H., Dewi, D. A., Liyana, N. A., & Md Nasir, N. R. (2021). A model for teaching and learning programming subjects in public secondary schools of Malaysia. In Advances in Visual Informatics: 7th International Visual Informatics Conference, IVIC 2021, Kajang, Malaysia, November 23-25, 2021, Proceedings 7 (pp. 362-373). Springer International Publishing. DOI: https://doi.org/10.1007/978-3-030-90235-3_32

Murugan, N., & Abdullah, A. S. (2024). Hubungan kreativiti guru terhadap pencapaian pelajar berdasarkan perspektif guru dalam amali subjek reka bentuk dan teknologi. ANP Journal of Social Science and Humanities, 5(2), 23-29. DOI: https://doi.org/10.53797/anp.jssh.v5i2.4.2024

Quinn, D. M. (2020). Experimental evidence on teachers’ racial bias in student evaluation: The role of grading scales. Educational Evaluation and Policy Analysis, 42(3), 375-392. DOI: https://doi.org/10.3102/0162373720932188

Qureshi, B. (2023). Exploring the use of Chatgpt as a tool for learning and assessment in undergraduate computer science curriculum: Opportunities and challenges. arXiv preprint arXiv:2304.11214.

Rini, T. A., & Cahyanto, B. (2020). Supporting elementary students creative writing skill with assessment as learning. In 2nd Early Childhood and Primary Childhood Education (ECPE 2020) (pp. 51-57). Atlantis Press. DOI: https://doi.org/10.2991/assehr.k.201112.010

Sahaat, Z., & Mohd Nasri, N. (2020). Cabaran pelaksanaan mata pelajaran reka bentuk dan teknologi sekolah menengah. Jurnal Pendidikan Malaysia, 45(1), 51-59.

Salleh, F. H. M., Drus, S. M., Gunasekaran, S. S., & Dewi, D. A. (2021). Evaluating the effectiveness of course assessments method in determining student’s competency level in programming. In Journal of Physics: Conference Series (Vol. 1878, No. 1, p. 012067). IOP Publishing. DOI: https://doi.org/10.1088/1742-6596/1878/1/012067

Schellekens, L. H., Bok, H. G., de Jong, L. H., van der Schaaf, M. F., Kremer, W. D., & van der Vleuten, C. P. (2021). A scoping review on the notions of Assessment as Learning (AaL), Assessment for Learning (AfL), and Assessment of Learning (AoL). Studies in Educational Evaluation, 71, 101094. DOI: https://doi.org/10.1016/j.stueduc.2021.101094

Shegaram, P. C., Naidu, N. B. M., Ibrahim, N. N., Ahmad, N. S., & Ganisan, H. (2024). An augmented reality-based interactive learning in moral education: advancing moral reasoning among secondary students: the nominal group technique approach. Journal of Contemporary Social Science and Education Studies (JOCSSES) E-ISSN-2785-8774, 4(2), 167-176.

Thornhill-Miller, B., Camarda, A., Mercier, M., Burkhardt, J. M., Morisseau, T., Bourgeois-Bougrine, S., & Lubart, T. (2023). Creativity, critical thinking, communication, and collaboration: Assessment, certification, and promotion of 21st-century skills for the future of work and education. Journal of Intelligence, 11(3), 54. DOI: https://doi.org/10.3390/jintelligence11030054

Ven, A. Van de & Delbeco, A. L. (1971). Nominal versus interacting group processes for committee decision-making effectiveness. The Academy of Management Journal, 14(2), 203-212. DOI: https://doi.org/10.2307/255307

Verheyen, S., & Storms, G. (2018). Education as a source of vagueness in criteria and degree. The semantics of gradability, vagueness, and scale structure: Experimental perspectives, 149-167. DOI: https://doi.org/10.1007/978-3-319-77791-7_6

Yusoff, S. M., Razak, R. A., Chin, H. L., & Marzaini, A. F. (2023). Exploring teachers’ conceptions of assessment: A quantitative study in a secondary school setting. Journal of Research, Policy & Practice of Teachers and Teacher Education, 13(2), 59-72 DOI: https://doi.org/10.37134/jrpptte.vol13.2.4.2023