Developing a terminology agreement in vocational education research: content validation of standard keyword indexes

  • Alzet Rama 
    Universitas Negeri Padang, Faculty of Engineering, West Sumatra, Indonesia
    Indonesia
  • Nizwardi Jalinus
    Universitas Negeri Padang, Faculty of Engineering, West Sumatra, Indonesia
    Indonesia
  • Muhammad Anwar
    Universitas Negeri Padang, Faculty of Engineering, West Sumatra, Indonesia
    Indonesia
  • Ifdil Ifdil
    Center for Educational Neuroscience, Trauma, and Human Behavior, Universitas Negeri Padang
    Indonesia
  • Wiki Lofandri
    Universitas Negeri Padang, Faculty of Vocational, West Sumatra, Indonesia
    Indonesia

Abstract

The study conducted an expert-based content validation to develop and validate a standardized keyword index for vocational education research. A lack of standard terminology in vocational education leads to problems in literature discovery, conducting systematic reviews, and international collaboration. Five content experts (three professors and two doctorate holders) with a minimum of 10 years of experience in vocational education using purposive sampling strategy, evaluated 79 keywords that were extracted from a systematic literature review of recent publications (2014-2025). The experts assessed each keyword for relevance on a 4-point Likert scale. The study used the Content Validity Index (CVI) method for the data analysis at both the item level (I-CVI) and the scale level (S-CVI). The result showed that the content validity was outstanding with the S-CVI/Ave of 0.916 which was higher than the recommended limit of 0.90. The 79 keywords were able to achieve I-CVI scores of at least 0.80, with 46 of the keywords (58.2%) getting perfect agreement (I-CVI = 1.00) and 33 of the keywords (41.8%) I-CVI = 0.80. The validated keywords were categorized into eight thematic categories: Core VET Systems & Models, Pedagogical Approaches, Technical & Occupational Skills, 21st Century & Soft Skills, Industry 4.0 & Emerging Technologies, Policy Standards & Quality Assurance, Labor Market & Economic Development, and Sustainability & Contemporary Issues. This standardized index remedies the problem of terminological inconsistency in vocational education research and thus leads to more efficient literature retrieval, knowledge synthesis, and international research collaboration. The study offers a clear and repeatable framework for keyword standardization that can be used to guide future updates and changes in different educational contexts.

Keywords
  • Vocational education, Content validity index, Keyword standardization, Expert panel, Research methodology, TVET
Lisensing Information
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

References
  1. Billett, S. (2020). Learning in the circumstances of work: The didactics of practice. Formazione & Insegnamento, 18(1), 9-21. https://doi.org/10.7346/-fei-XVIII-01-20_01
  2. Billett, S., Choy, S., Dymock, D., & Smith, R. (2020). Encouraging and supporting the participation and learning of older workers. In Supporting Older Workers (pp. 3-18). Springer. https://doi.org/10.1007/978-3-030-33813-7_1
  3. Brolpito, A. (2018). Digital skills and competence, and digital and online learning. European Training Foundation. https://www.etf.europa.eu/sites/default/files/2018-10/DSC%20and%20DOL_0.pdf
  4. Cedefop. (2020). Vocational education and training in Europe, 1995-2035: Scenarios for European vocational education and training in the 21st century. Publications Office of the European Union. https://doi.org/10.2801/794471
  5. Choy, S., Billett, S., & Kelly, A. (2018). Supporting students' integration of vocational knowledge. In S. McGrath, M. Mulder, J. Papier, & R. Suart (Eds.), Handbook of vocational education and training (pp. 1-17). Springer. https://doi.org/10.1007/978-3-319-49789-1_99-1
  6. Davis, L. L. (1992). Instrument review: Getting the most from a panel of experts. Applied Nursing Research, 5(4), 194-197. https://doi.org/10.1016/S0897-1897(05)80008-4
  7. Ferns, S., & Moore, K. (2022). Work integrated learning as a platform for career development and employability. In The Handbook of Work Integrated Learning (pp. 449-464). Routledge.
  8. Frey, C. B., & Osborne, M. A. (2017). The future of employment: How susceptible are jobs to computerisation? Technological Forecasting and Social Change, 114, 254-280. https://doi.org/10.1016/j.techfore.2016.08.019
  9. Gessler, M., & Siemer, C. (2020). Conceptualising VET from a lifelong learning perspective: Governance, sectors and fields of learning. In M. Pilz & J. Li (Eds.), Comparative vocational education research: Enduring challenges and new ways forward (pp. 29-51). Springer. https://doi.org/10.1007/978-3-658-29924-8_3
  10. Gessler, M., Fuchs, M., & Pilz, M. (2019). Conceptualising competence-based vocational and professional education: Theoretical and empirical dimensions and their interrelation from a cross-national perspective. In M. Gessler, M. Fuchs, & M. Pilz (Eds.), Concepts in vocational and professional education and training (pp. 1-28). Springer. https://doi.org/10.1007/978-3-658-24261-3_1
  11. Gordon, M., & Gibbs, T. (2014). STORIES statement: Publication standards for healthcare education evidence synthesis. BMC Medicine, 12(1), 143. https://doi.org/10.1186/s12916-014-0143-0
  12. Hallinger, P., & Kovačević, J. (2021). Science mapping the knowledge base in educational leadership and management: A longitudinal bibliometric analysis, 1960 to 2018. Educational Management Administration & Leadership, 49(1), 5-30. https://doi.org/10.1177/1741143219859002
  13. Haynes, S. N., Richard, D. C., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238-247. https://doi.org/10.1037/1040-3590.7.3.238
  14. International Labour Organization. (2022). International Standard Classification of Occupations: ISCO-08. ILO. https://www.ilo.org/public/english/bureau/stat/isco/isco08/
  15. Jossberger, H., Brand-Gruwel, S., van de Wiel, M. W., & Boshuizen, H. P. (2018). Learning in workplace simulations in vocational education: A student perspective. Vocations and Learning, 11(2), 179-204. https://doi.org/10.1007/s12186-017-9186-7
  16. Klotz, V. K., Billett, S., & Winther, E. (2014). Promoting workforce excellence: Formation and relevance of vocational identity for vocational educational training. Empirical Research in Vocational Education and Training, 6(6), 1-20. https://doi.org/10.1186/s40461-014-0006-0
  17. Li, J., & Pilz, M. (2022). International transfer of vocational education and training: A literature review. Journal of Vocational Education & Training, 74(2), 185-218. https://doi.org/10.1080/13636820.2020.1847566
  18. Loogma, K., Tafel-Viia, K., & Ümarik, M. (2019). Conceptualising educational changes in the context of increasing accountability: The case of Estonian VET. Journal of Vocational Education & Training, 71(1), 82-105. https://doi.org/10.1080/13636820.2018.1535517
  19. Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 382-386.
  20. McGrath, S., Mulder, M., Papier, J., & Suart, R. (2020). Handbook of vocational education and training: Developments in the changing world of work. Springer. https://doi.org/10.1007/978-3-319-94532-3
  21. Moed, H. F. (2017). Applied evaluative informetrics. Springer. https://doi.org/10.1007/978-3-319-60522-7
  22. Mourtzis, D., Angelopoulos, J., & Panopoulos, N. (2021). A literature review of the challenges and opportunities of the transition from Industry 4.0 to Society 5.0. Energies, 14(22), 6276. https://doi.org/10.3390/en14226276
  23. Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Blackwell Publishing. https://doi.org/10.1002/9780470754887
  24. Pilz, M., Uma, G., & Venkatram, R. (2020). Skills development in the informal sector in India: The case of street food vendors. International Review of Education, 66(4), 557-583. https://doi.org/10.1007/s11159-020-09857-4
  25. Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what's being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489-497. https://doi.org/10.1002/nur.20147
  26. Reed, D. A., Beckman, T. J., Wright, S. M., Levine, R. B., Kern, D. E., & Cook, D. A. (2021). Predictive validity evidence for medical education research study quality instrument scores: Quality of submissions to JGIM's Medical Education Special Issue. Journal of General Internal Medicine, 36(7), 2035-2041. https://doi.org/10.1007/s11606-021-06682-7
  27. Romero-Rodríguez, J. M., Ramírez-Montoya, M. S., Buenestado-Fernández, M., & Lara-Lara, F. (2020). Use of electroencephalography (EEG) to study the influence of audiovisual content on information recall: A systematic review. Sustainability, 12(21), 9231. https://doi.org/10.3390/su12219231
  28. Rubio, D. M., Berg-Weger, M., Tebb, S. S., Lee, E. S., & Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27(2), 94-104. https://doi.org/10.1093/swr/27.2.94
  29. Siekmann, G., & Fowler, C. (2017). Identifying work skill requirements for vocational education and training. International Journal of Training Research, 15(3), 180-197. https://doi.org/10.1080/14480220.2017.1387030
  30. UNESCO. (2021). Reimagining our futures together: A new social contract for education. UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000379707
  31. UNESCO-UNEVOC. (2019). UNEVOC TVETipedia glossary. UNESCO-UNEVOC International Centre for Technical and Vocational Education and Training. https://unevoc.unesco.org/home/TVETipedia+Glossary/lang=en
  32. Wheelahan, L., & Moodie, G. (2022). Improving the status and prestige of VET. Journal of Vocational Education & Training, 74(3), 357-376. https://doi.org/10.1080/13636820.2020.1862843
  33. Yusoff, M. S. B. (2019). ABC of content validation and content validity index calculation. Education in Medicine Journal, 11(2), 49-54. https://doi.org/10.21315/eimj2019.11.2.6
  34. Zhao, S., & Ko, J. (2018). Development of vocational education research in China: A quantitative content analysis of three major journals. ECNU Review of Education, 1(3), 128-142. https://doi.org/10.30926/ecnuroe2018010307
  35. Zlatkin-Troitschanskaia, O., Pant, H. A., & Coates, H. (2016). Assessing student learning outcomes in higher education: Challenges and international perspectives. Assessment & Evaluation in Higher Education, 41(5), 655-661. https://doi.org/10.1080/02602938.2016.1169501

Downloads

PDF

Statistic

Abstract Views
: 24 times
PDF
: 16 times