Psychometric characteristics of a pisa-based scientific literacy assessment instrument for ecology: a rasch model analysis

  • Erik Nanda Putra Putra 
    Universitas Negeri Padang
    Indonesia
  • Lufri Lufri
    Universitas Negeri Padang
    Indonesia
  • Syafri Anwar
    Universitas Negeri Padang
    Indonesia
  • Yuni Ahda
    Univrsitas Negeri Padang
    Indonesia

Abstract

Scientific literacy is an essential competency for the 21st century, yet Indonesian students continue to demonstrate relatively low levels of achievement. This study aimed to analyze the psychometric characteristics of a PISA-based scientific literacy assessment instrument on ecology during the field-testing phase using the Rasch model. The study involved 40 students who completed a 45-item assessment. Data were analyzed using Winsteps based on three Rasch indicators: person–item targeting, item fit, and separation index. The results showed appropriate targeting, with a 0.45-logit difference between the mean student ability and mean item difficulty. A total of 43 of the 45 items (95.6%) met the Rasch model fit criteria. The person separation index of 1.71 and item separation index of 2.18 classified students' abilities and item difficulties into three distinct strata. Overall, the findings indicate that the instrument demonstrates adequate psychometric characteristics for measuring scientific literacy in ecology. However, the relatively modest person separation suggests that additional items targeting lower ability levels may further improve the instrument's measurement precision.

Keywords
  • Assessment instrument
  • Ecology
  • Pisa
  • Rasch model
  • Scientific literacy
Lisensing Information
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

References
  1. Alti, R. P., Lufri, L., Helendra, H., & Yogica, R. (2021). Pengembangan instrumen asesmen berbasis literasi sains tentang materi keanekaragaman hayati kelas X. Journal for Lesson and Learning Studies, 4(1). https://doi.org/10.23887/jlls.v4i1.34270
  2. Arabbani, F. K., Mulyani, S., Mahardiani, L., & Ariani, S. R. D. (2019). Analysis the quality of instrument for measuring chemical literacy abilities of high school student using Rasch model. AIP Conference Proceedings, 2194, 020007. https://doi.org/10.1063/1.5139739
  3. Ayu, G. N., Putri, C. A., Riyanto, A. R., & Koto, I. (2025). The scientific literacy competence of students in Indonesia and Mexico based on PISA 2022: An international comparative study. TOFEDU: The Future of Education Journal, 4(5), 1033–1038. https://doi.org/10.61445/tofedu.v4i5.525
  4. Ayub, M. R. S. S. N., Istiyono, E., Munadi, S., Permadi, C., Pattiserlihun, A., & Sudjito, D. N. (2020). Analisa penilaian soal fisika menggunakan model Rasch dengan program R. Jurnal Sains dan Edukasi Sains, 3(2), 46–52. https://doi.org/10.24246/juses.v3i2p46-52
  5. Bashooir, K., & Supahar. (2018). Validitas dan reliabilitas instrumen asesmen kinerja literasi sains pelajaran fisika berbasis STEM. Jurnal Penelitian dan Evaluasi Pendidikan, 22(2), 219–230. https://doi.org/10.21831/pep.v22i2.19590
  6. Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). Routledge.
  7. Bybee, R., McCrae, B., & Laurie, R. (2009). PISA 2006: An assessment of scientific literacy. Journal of Research in Science Teaching, 46(8), 865–883. https://doi.org/10.1002/tea.20333
  8. Darman, D. R., Suhandi, A., Kaniawati, I., Samsudin, A., & Wibowo, F. C. (2024). Development and validation of scientific inquiry literacy instrument (SILI) using Rasch measurement model. Education Sciences, 14(3), 322. https://doi.org/10.3390/educsci14030322
  9. Familia, M. N., Haerani, R. P. R., & Suhartini, E. (2026). Development of scientific literacy skills test instruments in elementary school: Analysis of the Rasch model. Integrated Science Education Journal, 7(2), 255–264. https://doi.org/10.37251/isej.v7i2.2383
  10. Fisher, W. P. (2007). Rating scale instrument quality criteria. Rasch Measurement Transactions, 21(1), 1095.
  11. Hasnawati, Syazali, M., & Putra, G. P. (2023). Pengembangan asesmen literasi sains berbasis PISA untuk siswa sekolah dasar. BADA'A: Jurnal Ilmiah Pendidikan Dasar, 5(2), 240–250. https://doi.org/10.37216/badaa.v5i1.1213
  12. Ibrahim, N. L. (2025). Analisis literatur penggunaan model Rasch untuk validasi instrumen asesmen pendidikan: Tren, temuan, dan implikasi. Lencana: Jurnal Inovasi Ilmu Pendidikan, 3(2), 46–60. https://doi.org/10.55606/lencana.v3i2.5012
  13. Khery, Y., Sarjan, M., Ahzan, S., & Efendi, I. (2022). Konseptualisasi literasi sains mengacu pada kerangka sains PISA sejak tahun 2000. Educatoria: Jurnal Ilmiah Ilmu Pendidikan, 2(4), 200–231. https://doi.org/10.36312/ejiip.v2i4.117
  14. Krishnan, S., & Idris, N. (2018). Using partial credit model to improve the quality of an instrument. International Journal of Evaluation and Research in Education, 7(4), 313–321. https://doi.org/10.11591/ijere.v7i4.15146
  15. Lie, S., & Linnakylä, P. (2004). Nordic PISA 2000 in a sociocultural perspective. Scandinavian Journal of Educational Research, 48(3), 227–230. https://doi.org/10.1080/00313830410001695709
  16. Linacre, J. M. (1994). Sample size and item calibration stability. Rasch Measurement Transactions, 7(4), 328.
  17. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. https://doi.org/10.1007/BF02296272
  18. Mellyzar, M., Andriani, R., Lukman, I. R., Muttakin, M., Wati, E., Pasaribu, A. I., & Fadli, M. R. (2024). Digital assessment tools as a media for assessing high school science learning. Jurnal Penelitian Pendidikan IPA, 10(3), 1362–1374. https://doi.org/10.29303/jppipa.v10i3.6416
  19. Mulyana, V., & Desnita, D. (2023). Empirical validity and reliability of the scientific literacy assessment instrument based on the tornado physics enrichment book. Jurnal Penelitian Pendidikan IPA, 9(5), 3961–3967. https://doi.org/10.29303/jppipa.v9i5.3290
  20. Neumann, I., Neumann, K., & Nehm, R. (2010). Evaluating instrument quality in science education: Rasch-based analyses of a nature of science test. International Journal of Science Education, 33(10), 1373–1405. https://doi.org/10.1080/09500693.2010.511297
  21. Ngandoh, S. T., Riandi, Rahmat, A., & Muslim. (2025). Rasch model analysis of item quality for measuring conceptual understanding of the human body system. Jurnal Inovasi Pendidikan dan Sains, 6(1), 89–95.
  22. Permatasari, A. I., Maftukh, F. D., As'ari, & Muda, R. C. (2026). Insights from a teaching assistance program in Thailand: A scientific literacy analysis of a lower secondary science textbook. Science Education and Application Journal, 8(1), 76–87. https://doi.org/10.30736/seaj.v8i1.1346
  23. Putra, E. N. (2019). Analisis kemampuan literasi sains biologi siswa SMP se-Kecamatan Koto XI Tarusan Kabupaten Pesisir Selatan (Bachelor's thesis). STKIP PGRI Sumatera Barat. https://repo.upgrisba.ac.id/id/eprint/6964/
  24. Schmaltz, R. M., Jansen, E., & Wenckowski, N. (2017). Redefining critical thinking: Teaching students to think like scientists. Frontiers in Psychology, 8, 459. https://doi.org/10.3389/fpsyg.2017.00459
  25. Sumintono, B., & Widhiarso, W. (2014). Aplikasi model Rasch untuk penelitian ilmu-ilmu sosial (Edisi revisi). Trim Komunikata.
  26. Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan. Institut Teknologi Sepuluh November.
  27. Suryadi, N. R. S., Nurmegawati, L., Mu'aziyah, S. E. S., & Perdani, A. S. (2025). Rasch model for analysis of scientific attitude instruments in the context of secondary school science education. Equator Science Journal, 3(2), 98–106. https://doi.org/10.61142/esj.v3i2.234
  28. Turiman, P., Omar, J., Daud, A. M., & Osman, K. (2012). Fostering the 21st century skills through scientific literacy and science process skills. Procedia – Social and Behavioral Sciences, 59, 110–116. https://doi.org/10.1016/j.sbspro.2012.09.253
  29. Widiyanti, I. S. R., Winata, A., & Cacik, S. (2017). Pengembangan perangkat penilaian literasi sains berorientasi Programme for International Student Assessment (PISA). Education and Human Development Journal, 2(1). https://doi.org/10.33086/ehdj.v2i1.377

Downloads

PDF

Statistic

Abstract Views
: 0 times
PDF
: 0 times