Machine learning and multimodal AI for predicting athlete psychological states: a systematic review

  • Anisa Sholiha Mia 
    Universitas Negeri Padang
    Indonesia
  • Bram Sujadesman
    Universitas Negeri Padang
    Indonesia

Abstract

Athletes’ emotional, cognitive, and motivational states can change rapidly, yet conventional assessment often relies on intermittent self-report. This systematic review synthesized evidence on (1) psychological states targeted by machine learning (ML) and multimodal artificial intelligence (AI), (2) data modalities and model architectures used for prediction, and (3) methodological maturity of the evidence base. Following PRISMA 2020, Scopus was searched in TITLE-ABS-KEY fields for studies published from 2020 to 2025. Of 113 records identified, 4 duplicates were removed, 109 titles/abstracts were screened, 25 full texts were assessed, and 10 studies met the eligibility criteria. Screening used predefined criteria; no independent duplicate-screening record was retained, so a retrospective inter-rater coefficient could not be calculated. Evidence was synthesized narratively through thematic coding and descriptive bibliometric analysis, with study reporting quality appraised using a four-domain FICO rubric (0–2 per domain; retention threshold ≥4/8). Four themes emerged: multimodal emotion recognition, deep hybrid architectures, continuous/personalized affective-state prediction, and limited ecological validation. Multimodal fusion generally outperformed unimodal approaches, but small samples, heterogeneous labels, limited external validation, and weak interpretability constrain confidence in reported accuracy. Future work should prioritize participant-level validation, standardized psychological constructs, explainable models, and prospective field testing.

Keywords
  • Computational sport psychology, Machine learning, Multimodal artificial intelligence, Athlete psychological states, Predictive analytics
Lisensing Information
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

References
  1. Ahmed, N., Al Aghbari, Z., & Girija, S. (2023). A systematic survey on multimodal emotion recognition using learning algorithms. Intelligent Systems with Applications, 17, 200171. https://doi.org/10.1016/j.iswa.2022.200171
  2. Al Imran, M. A., Nasirzadeh, F., & Karmakar, C. (2024). Designing a practical fatigue detection system: A review on recent developments and challenges. Journal of Safety Research, 90, 100–114. https://doi.org/10.1016/j.jsr.2024.05.015
  3. Balasamy, S., Atchudan, R., Arya, S., & Sundramoorthy, A. K. (2025). Emerging trends in wearable and non-invasive cortisol sensing technologies: A review. Journal of Materials Chemistry B. https://doi.org/10.1039/d5tb01519g
  4. Bhardwaj, A., & Kumar, M. (2025). Automatic sports video highlight detection using machine learning. 2025 IEEE 7th International Conference on Computing, Communication and Automation (ICCCA 2025). https://doi.org/10.1109/ICCCA66364.2025.11325680
  5. Bieńkiewicz, M. M. N., Smykovskyi, A. P., Olugbade, T., Janaqi, S., Camurri, A., Bianchi-Berthouze, N., Björkman, M., & Bardy, B. G. (2021). Bridging the gap between emotion and joint action. Neuroscience and Biobehavioral Reviews, 131, 806–833. https://doi.org/10.1016/j.neubiorev.2021.08.014
  6. Brunyé, T. T., Okano, K., McIntyre, J., Sandone, M. K., Townsend, L. N., Lee, M. M., Smith, M., & Hughes, G. I. (2025). Inferring mental states via linear and non-linear body movement dynamics: A pilot study. Sensors, 25(22), Article 6990. https://doi.org/10.3390/s25226990
  7. Christ, L., Amiriparian, S., Baird, A., Kathan, A., Müller, N., Klug, S., Gagne, C., Tzirakis, P., Stappen, L., Meßner, E.-M., König, A., Cowen, A., Cambria, E., & Schuller, B. W. (2023). The MuSe 2023 multimodal sentiment analysis challenge: Mimicked emotions, cross-cultural humour, and personalisation. MuSe 2023 – Proceedings of the 4th Multimodal Sentiment Analysis Challenge and Workshop, 1–10. https://doi.org/10.1145/3606039.3613114
  8. Christ, L., Amiriparian, S., Baird, A., Tzirakis, P., Kathan, A., Müller, N., Stappen, L., Meßner, E.-M., König, A., Cowen, A., Cambria, E., & Schuller, B. W. (2022). The MuSe 2022 multimodal sentiment analysis challenge: Humor, emotional reactions, and stress. MuSe 2022 – Proceedings of the 3rd International Multimodal Sentiment Analysis Workshop and Challenge, 5–14. https://doi.org/10.1145/3551876.3554817
  9. Cui, C. (2022). Intelligent analysis of exercise health big data based on deep convolutional neural network. Computational Intelligence and Neuroscience, 2022, Article 5020150. https://doi.org/10.1155/2022/5020150
  10. de la Cal, E., Sedano, J., Gallucci, A., & Valderde, P. (2023). A comparison of two speech emotion recognition algorithms: Pepper humanoid versus bag of models. Lecture Notes in Networks and Systems, 531, 635–644. https://doi.org/10.1007/978-3-031-18050-7_62
  11. Fan, Z., Xu, L., Chen, S., & Wang, Y. (2025). A CNN-Transformer fusion model-based EEG signal-driven method for game video emotion classification. Journal of Mechanics in Medicine and Biology, 25(8), Article 2540069. https://doi.org/10.1142/S021951942540069X
  12. Fauzi, A., Ohnishi, A., Terada, T., & Tsukamoto, M. (2025). Evaluation of wearable sensors for mental fatigue level estimation under physical fatigue. International Conference on Intelligent Informatics and BioMedical Sciences (ICIIBMS), 173–180. https://doi.org/10.1109/ICIIBMS66230.2025.11316703
  13. Feng, R., Zou, L., Wang, Y., Xu, Y., & Zhou, H. (2025). Towards athletes’ emotion state recognition with the fusion of EEG and facial expression based on ResNet-LSTM. Proceedings of the 2024 International Conference on Sports Technology and Performance Analysis (ICSTPA 2024), 24–31. https://doi.org/10.1145/3723936.3723941
  14. Fuente, C., Castellanos, F. J., Valero-Mas, J. J., & Calvo-Zaragoza, J. (2023). Multimodal recognition of frustration during game-play with deep neural networks. Multimedia Tools and Applications, 82(9), 13617–13636. https://doi.org/10.1007/s11042-022-13762-7
  15. Ganiti-Roumeliotou, E., Dias, S. B., Khalaf, K., Jelinek, H. F., & Hadjileontiadis, L. J. (2024). Beyond the game: Multimodal game-experience recognition during dynamic affective game environments. 2024 IEEE 22nd Mediterranean Electrotechnical Conference (MELECON 2024), 1042–1047. https://doi.org/10.1109/MELECON56669.2024.10608557
  16. Gao, Z., Hamanaka, S., Inui, Y., Goto, K., Bekku, H., Sasaki, W., Okoshi, T., & Nakazawa, J. (2023). Bodily condition estimation of archery learners using interoceptive accuracy. UbiComp/ISWC 2022 Adjunct – Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 173–177. https://doi.org/10.1145/3544793.3563408
  17. Gumus, M., Mack, M. L., Green, R., Khodadadi, M., Wennberg, R. A., Crawley, A., Colella, B., Tarazi, A., Mikulis, D., Tator, C. H., & Tartaglia, M. C. (2022). Brain connectivity changes in postconcussion syndrome as the neural substrate of a heterogeneous syndrome. Brain Connectivity, 12(8), 711–724. https://doi.org/10.1089/brain.2021.0127
  18. Guo, D. (2025). Fine grain emotional intelligent recognition method for athletes based on multi physiological information fusion. International Journal of Biometrics, 17(1–2), 15–30. https://doi.org/10.1504/IJBM.2025.143721
  19. Gutiérrez-Martín, L., Romero-Perales, E., de Baranda Andújar, C. S., Canabal-Benito, M. F., Rodríguez-Ramos, G. E., Toro-Flores, R., López-Ongil, S., & López-Ongil, C. (2022). Fear detection in multimodal affective computing: Physiological signals versus catecholamine concentration. Sensors, 22(11), Article 4023. https://doi.org/10.3390/s22114023
  20. Hashimoto, Y., Noto, K., Tada, S., & Nishida, Y. (2025). Wearable multimodal sensor probe for monitoring core body temperature, electrocardiogram, heart rate, and sweat rate. IEEE Access, 13, 70769–70778. https://doi.org/10.1109/ACCESS.2025.3561288
  21. Huang, Y., Huang, S., Wang, Y., Li, Y., Gui, Y., & Huang, C. (2022). A novel lower extremity non-contact injury risk prediction model based on multimodal fusion and interpretable machine learning. Frontiers in Physiology, 13, Article 937546. https://doi.org/10.3389/fphys.2022.937546
  22. Jong, H. Y., & Kuan, G. (2025). The role of intonation in designing machinery for mental sports psychology. Neuroscience Research Notes, 8(3), Article 449. https://doi.org/10.31117/neuroscirn.v8i3.449
  23. Karunakaran, M., Kumar, A., Smitha, K., Dua, N., Shaktawat, T., Sarpal, S. S., & Deshpande, A. (2025). Predictive AI for rhythm synchronization in training. ShodhKosh: Journal of Visual and Performing Arts, 6(4S), 11–21. https://doi.org/10.29121/shodhkosh.v6.i4s.2025.6828
  24. Khare, S. K., Blanes-Vidal, V., Nadimi, E. S., & Acharya, U. R. (2024). Emotion recognition and artificial intelligence: A systematic review (2014–2023) and research recommendations. Information Fusion, 102, 102019. https://doi.org/10.1016/j.inffus.2023.102019
  25. Kimball, J. P., Zia, J. S., An, S., Rolfes, C., Hahn, J.-O., Sawka, M. N., & Inan, O. T. (2021). Unifying the estimation of blood volume decompensation status in a porcine model of relative and absolute hypovolemia via wearable sensing. IEEE Journal of Biomedical and Health Informatics, 25(9), 3351–3360. https://doi.org/10.1109/JBHI.2021.3068619
  26. Kourtesis, P. (2024). A comprehensive review of multimodal XR applications, risks, and ethical challenges in the metaverse. Multimodal Technologies and Interaction, 8(11), Article 98. https://doi.org/10.3390/mti8110098
  27. Li, H., & Sun, H. (2024). Emotional analysis of joint sports quality expansion tasks based on multi-modal feature fusion. Systems and Soft Computing, 6, Article 200092. https://doi.org/10.1016/j.sasc.2024.200092
  28. Li, S., Hou, Z., Amjad, K., & Mushtaq, H. (2025). Multi modal fusion of medical imaging and biomechanical data using attention based Swin-UNet and LSTM for sports injury prediction. Frontiers in Physiology, 16, Article 1687895. https://doi.org/10.3389/fphys.2025.1687895
  29. Liu, G. (2024). Multimodal analysis and optimisation strategy of teaching behaviour in physical education classroom. Applied Mathematics and Nonlinear Sciences, 9(1), Article 20241506. https://doi.org/10.2478/amns-2024-1506
  30. Ma, W., Liu, Y., Yi, Q., Liu, X., Xing, W., Zhao, R., Liu, H., & Li, R. (2025). Table tennis coaching system based on a multimodal large language model with a table tennis knowledge base. PLoS ONE, 20(2), Article e0317839. https://doi.org/10.1371/journal.pone.0317839
  31. Madanian, S., Chen, T., Adeleye, O., Templeton, J. M., Poellabauer, C., & colleagues. (2023). Speech emotion recognition using machine learning—A systematic review. Intelligent Systems with Applications, 20, 200266. https://doi.org/10.1016/j.iswa.2023.200266
  32. Mavromoustakos-Blom, P., Bakkes, S., & Spronck, P. (2020). Multi-modal study of the effect of time pressure in a crisis management game. ACM International Conference Proceeding Series, Article 86. https://doi.org/10.1145/3402942.3403006
  33. Nguyen, P.-T., Nazmudeen, M. S. H., Dao, M.-S., & Le, D.-D. (2022). Adaptive learning models for getting insights into multimodal lifelog data. Proceedings – International Conference on Knowledge and Systems Engineering (KSE). https://doi.org/10.1109/KSE56063.2022.9953616
  34. Ochab, J. K., Węgrzyn, P., Witaszczyk, P., Drążyk, D., & Nalepa, G. J. (2025). Mobile game evaluation method based on data mining of affective time series. Sensors, 25(9), Article 2756. https://doi.org/10.3390/s25092756
  35. Ogundokun, R. O., Damasevicius, R., & Maskeliunas, R. (2025). OptiMobileX: Optimizing deep transfer learning model for accurate human posture recognition using a deep feature fusion technique. IEEE Sensors Journal, 25(6), 9759–9766. https://doi.org/10.1109/JSEN.2024.3522105
  36. Pei, G., Li, H., Lu, Y., Wang, Y., Guanxiong, H., & Li, T. (2024). Affective computing: Recent advances, challenges, and future trends. Intelligent Computing, 3, 0076. https://doi.org/10.34133/icomputing.0076
  37. Pereira, R., Mendes, C., Ribeiro, J., Ribeiro, R., Miragaia, R., Rodrigues, N., Costa, N., & Pereira, A. (2024). Systematic review of emotion detection with computer vision and deep learning. Sensors, 24(11), 3484. https://doi.org/10.3390/s24113484
  38. Quintero, L., Fors, U., & Papapetrou, P. (2025). Personalized feature importance ranking for affect recognition from behavioral and physiological data. IEEE Transactions on Games, 17(3), 594–603. https://doi.org/10.1109/TG.2023.3263070
  39. Raj, F. K., Rahad, R. H., Afnan, M. D., Ur Rahman, A., Ahmed, M. S. U., Rhaman, M. K., & Alam, M. G. R. (2025). Analyzing emotion patterns in gaming using CNNs on facial and vocal features. 2025 IEEE 7th International Conference on Sustainable Technologies for Industry 5.0 (STI 2025). https://doi.org/10.1109/STI69347.2025.11367592
  40. Roy, S., Molla, M. T. I., Hossain, F., Hasan, M. N., Barua, H., & Hasan, K. (2025). Efficient exercise monitoring with reduced sensor usage: A machine learning approach. 2025 IEEE International Conference on Quantum Photonics, Artificial Intelligence, and Networking (QPAIN 2025). https://doi.org/10.1109/QPAIN66474.2025.11171701
  41. Samal, P., & Hashmi, M. F. (2024). Role of machine learning and deep learning techniques in EEG-based BCI emotion recognition system: A review. Artificial Intelligence Review, 57, 50. https://doi.org/10.1007/s10462-023-10690-2
  42. Song, M., Mallol-Ragolta, A., Parada-Cabaleiro, E., Yang, Z., Liu, S., Ren, Z., Zhao, Z., & Schuller, B. W. (2021). Frustration recognition from speech during game interaction using wide residual networks. Virtual Reality and Intelligent Hardware, 3(1), 76–86. https://doi.org/10.1016/j.vrih.2020.10.004
  43. Sun, H., Soh, K. G., Mohammadi, A., Toumi, Z., Zhang, L., Ding, C., Gao, X., & Tian, J. (2024). Counteracting mental fatigue for athletes: A systematic review of the interventions. BMC Psychology, 12, 67. https://doi.org/10.1186/s40359-023-01476-w
  44. Tan, M., Xiao, Y., Jing, F., Xie, Y., Lu, S., Xiang, M., & Ren, H. (2024). Evaluating machine learning-enabled and multimodal data-driven exercise prescriptions for mental health: A randomized controlled trial protocol. Frontiers in Psychiatry, 15, Article 1352420. https://doi.org/10.3389/fpsyt.2024.1352420
  45. Tao, X., Sáenz-Lechón, N., & Eckert, M. (2025). Mapping the landscape of artificial intelligence for serious games in health: An enhanced meta review. Computers in Human Behavior Reports, 18, Article 100696. https://doi.org/10.1016/j.chbr.2025.100696
  46. Tileubay, S., Yerekeshova, M., Baiganova, A., Janyssova, D., Omarov, N., Omarov, B., & Baiekeyeva, Z. (2024). Development of deep learning enabled augmented reality framework for monitoring the physical quality training of future trainers-teachers. International Journal of Advanced Computer Science and Applications, 15(3), 329–339. https://doi.org/10.14569/IJACSA.2024.0150334
  47. Tillem, M., & Gün, A. (2025). The impact of color blindness on player engagement and emotional experiences: A multimodal study in a game-based environment. Multimodal Technologies and Interaction, 9(6), Article 62. https://doi.org/10.3390/mti9060062
  48. Tiwary, G., Chauhan, S., & Goyal, K. K. (2024). Multimodal attention CNN for human emotion recognition. Lecture Notes in Networks and Systems, 918, 161–177. https://doi.org/10.1007/978-981-97-0641-9_11
  49. Udahemuka, G., Djouani, K., & Kurien, A. M. (2024). Multimodal emotion recognition using visual, vocal and physiological signals: A review. Applied Sciences, 14(17), 8071. https://doi.org/10.3390/app14178071
  50. Vec, V., Tomažič, S., Kos, A., & Umek, A. (2024). Trends in real-time artificial intelligence methods in sports: A systematic review. Journal of Big Data, 11, 148. https://doi.org/10.1186/s40537-024-01026-0
  51. Wang, Y., Wang, H., & Lu, X. (2025). Research on the impact of explosive martial arts training on emotion regulation and attention based on questionnaire data. Scientific Reports, 15(1), Article 37957. https://doi.org/10.1038/s41598-025-21790-0
  52. Wang, Y., Zhang, L., & Ding, G. (2025). A survey on expressive action quality assessment driven by aesthetics and biomechanics. Proceedings of SPIE – The International Society for Optical Engineering, 13709, Article 1370907. https://doi.org/10.1117/12.3073579
  53. Wu, W. (2024). Multimodal emotion detection of tennis players based on deep reinforcement learning. International Journal of Biometrics, 16(5), 497–513. https://doi.org/10.1504/IJBM.2024.140775
  54. Xiaoyan, L., Nazarudin, M. N., & Mansor, A. Z. (2025). The long-term impact of running applications on physical fitness among vocational college students: A longitudinal analysis. Journal of Physical Education and Sport, 25(5), 1060–1070. https://doi.org/10.7752/jpes.2025.05117
  55. Xu, J., Guo, X., Zhang, Z., Liu, H., & Lee, C. (2025). Triboelectric mat multimodal sensing system (TMMSS) enhanced by infrared image perception for sleep and emotion-relevant activity monitoring. Advanced Science, 12(6), Article 2407888. https://doi.org/10.1002/advs.202407888
  56. Yang, F., & Wang, Z. (2025). An intelligent taekwondo coaching system based on augmented reality technology with real-time feedback mechanisms. Scientific Reports, 15(1), Article 40832. https://doi.org/10.1038/s41598-025-24608-1
  57. Yang, Y., Caldwell, K., Schlagal, R. C., DeCelle, S., Simmons, S., Yang, L., Abduljawad, S., Ayar, J., Singh, J. R., Sheeler, R., Flynn, P. R., Knapp, K. A., & Härtl, R. (2025). “No pain, more gain”: Subjective responses to a T’ai Chi and Qigong program for back pain. Global Advances in Integrative Medicine and Health, 14. https://doi.org/10.1177/27536130251384679
  58. Younis, E. M. G., Mohsen, S., Houssein, E. H., & Ibrahim, O. A. S. (2024). Machine learning for human emotion recognition: A comprehensive review. Neural Computing and Applications, 36, 8901–8947. https://doi.org/10.1007/s00521-024-09426-2
  59. Zhang, H., Li, X., Sun, Y., Fu, X., Qiu, C., & Carroll, J. M. (2024). VRMN-bD: A multi-modal natural behavior dataset of immersive human fear responses in VR stand-up interactive games. Proceedings – 2024 IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2024), 320–330. https://doi.org/10.1109/VR58804.2024.00054
  60. Zhang, L., & Tian, H. (2024). Construction of a risk assessment and prediction model for athlete doping use based on bioinformatics. MCB Molecular and Cellular Biomechanics, 21(4), Article 446. https://doi.org/10.62617/mcb446
  61. Zhao, B., Yang, Z., & Wu, Y. (2025). SFNet: Fusion of spatial and frequency domain features for improved posture classification in yoga poses. 2025 IEEE 8th International Conference on Mechatronics and Computer Technology Engineering (MCTE 2025), 145–149. https://doi.org/10.1109/MCTE67374.2025.11213654

Downloads

PDF

Statistic

Abstract Views
: 14 times
PDF
: 9 times