Student Expression Detection Based on Facial Image Using Convolutional Neural Network (CNN)
Keywords:
Affective Computing, Emotion Recognition, Facial Expression Analysis, Deep Learning, Convolutional Neural Network
Abstract
The widespread adoption of electronic learning (e-learning) in higher education has brought significant changes to how knowledge is delivered. Despite its advantages, many implementations remain focused solely on content dissemination, often neglecting learners’ emotional engagement. Emotional states, particularly in academic contexts, influence concentration, motivation, and comprehension. One of the most effective and intuitive indicators of emotion is facial expression. This research investigates the use of Convolutional Neural Networks (CNN), a deep learning approach, to automatically detect student emotions through facial image analysis. A dataset of facial expressions was constructed and divided into training and testing sets, each containing five distinct emotional categories: anger, happiness, fear, neutrality, and surprise. The CNN model was trained for 100 epochs, resulting in a training accuracy of 89% and a testing accuracy of 88%. These results demonstrate that CNN-based emotion recognition has strong potential to enhance e-learning platforms by providing instructors with real-time emotional insights. By integrating emotional feedback, educators can adapt instructional strategies more effectively to improve student engagement and learning outcomes. This study contributes to the growing field of affective computing and emphasizes the importance of emotional awareness in digital learning environments.
Downloads
References
Adesola, S. A., Li, Y., & Liu, X. (2019). Effect of Emotions on Students Learning Strategies. Proceedings of the 2019 8th International Conference on Educational and Information Technology. https://doi.org/10.1145/3318396.3318408
Alexander, D. (2019). Deteksi Emosi Menggunakan Citra Ekspresi Wajah Secara Otomatis.
Arkorful, V., & Abaidoo, N. (2015). The role of e-learning, advantages and disadvantages of its adoption in higher education. Procedia - Social and Behavioral Sciences, 3, 29–35. https://doi.org/10.1016/j.protcy.2014.10.067
Azhary, L., Handoyo, E., & Khafid, M. (2018). The Implementation of Integrated Character Education in Policy Design at SD Muhammadiyah (Plus) Salatiga.
Azis, M. (2022). The Role of Higher Education in Building Student Character: A Moral and Religious Approach. Jurnal Pendidikan Ilmu-Ilmu Sosial, 11(2), 151–160. https://doi.org/10.15294/jpii.v11i2.38439
D’Errico, F., Paciello, M., & Cerniglia, L. (2016). When emotions enhance students’ engagement in e-learning processes. Journal of E-Learning and Knowledge Society, 12, 9–23. https://doi.org/10.20368/1971-8829/1144
Eka Putra, W. S. (2016). Klasifikasi Citra Menggunakan Convolutional Neural Network (CNN) pada Caltech 101. Jurnal Teknik ITS, 5(1). https://doi.org/10.12962/j23373539.v5i1.15696
Grover, R., & Bansal, S. (2025). Efficient Facial Expression Recognition Through Lightweight CNN Technique on Public Datasets. SN Computer Science, 6(15). https://doi.org/10.1007/s42979-024-03557-y
Gupta, S., Kumar, P., & Tekchandani, R. K. (2022). Facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models. Multimedia Tools and Applications, 81, 11365–11394. https://doi.org/10.1007/s13278-023-01181-x
Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020). The difference between emergency remote teaching and online learning. Australasian Journal of Educational Technology, 36(3), 20–34. https://doi.org/10.1080/01587919.2020.1756120
Isen, A. M. (2008). Positive affect and decision processes: Some recent theoretical developments with practical implications. American Psychologist, 63(10), 791–803. https://doi.org/10.1037/0003-066X.63.10.791
Khan, M. A., Akram, T., Sharif, M., Javed, K., & Raza, M. (2021). Facial expression recognition using optimized deep CNN features. Sensors, 21(16), 5386. https://doi.org/10.3390/s21165386
Li, S., & Deng, W. (2020). Deep facial expression recognition: A survey. Neurocomputing, 429, 215–233. https://doi.org/10.1016/j.neucom.2020.01.081
Li, X., Yue, R., Jia, W., Wang, H., & Zheng, Y. (2021). Recognizing Students’ Emotions based on Facial Expression Analysis. 2021 11th International Conference on Information Technology in Medicine and Education (ITME), 96–100. https://doi.org/10.1109/ITME53901.2021.00030
Li, Y., Song, Y., Luo, Z., Liu, H., & Liu, Q. (2020). Facial Expression Recognition for Intelligent Healthcare: A Review. Frontiers in Psychiatry, 11, 607890. https://doi.org/10.3389/fpsyt.2020.607890
MacAllister, G. P. (2025). The Impact of Emotion-Based Teaching Strategies on Motivation in Collaborative Learning. Journal of Advanced Research in Education. https://doi.org/10.56397/jare.2025.01.02
Mukhtar, K., Javed, K., Arooj, M., & Sethi, A. (2020). Advantages, Limitations and Recommendations for online learning during COVID-19 pandemic era. Heliyon, 6(9), e03163. https://doi.org/10.1016/j.heliyon.2020.e03163
Ngo, D., Nguyen, A., Dang, B., & Ngo, H. (2024). Facial Expression Recognition for Examining Emotional Regulation in Synchronous Online Collaborative Learning. International Journal of Artificial Intelligence in Education, 34, 650–669. https://doi.org/10.1007/s40593-023-00378-7
Oliver, M. M., & Amengual Alcover, E. (2020). UIBVFED: Virtual facial expression dataset. PLOS ONE, 15(4), e0231266. https://doi.org/10.1371/journal.pone.0231266
Pekrun, R., Loderer, K., & Götz, T. (2017). Affective science in education: Theoretical contributions and empirical applications. Contemporary Educational Psychology, 54, 1–4. https://doi.org/10.1016/j.cedpsych.2016.10.002
Putra, G. D., & Purwaningsih, Y. (2021). E-learning as a transformation in higher education during the Covid-19 pandemic. ETERNAL (English Teaching Journal), 12(2), 212–223. https://doi.org/10.26877/eternal.v12i2.8320
Putra, I. N. T. A. (2015). Perancangan dan Pengembangan Sistem Absensi Realtime Melalui Metode Pengenalan Wajah. JST Jurnal Sains Dan Teknologi, 3(2). https://doi.org/10.23887/jst-undiksha.v3i2.4480
Shan, J., & Eliyas, S. (2024). Exploring AI Facial Recognition for Real-time Emotion Detection: Assessing Student Engagement in Online Learning Environments. 2024 3rd International Conference on Artificial Intelligence For Internet of Things (AIIoT), 1–6. https://doi.org/10.1109/AIIoT58432.2024.10574587
Tilaar, H. A. R. (2004). Pendidikan, Kebudayaan, dan Masyarakat Madani Indonesia. Remaja Rosdakarya.
Ullah, S., Ou, J., Xie, Y., & Tian, W. (2024). Compound Facial Expressions Recognition Approach Using DCGAN and CNN. Multimedia Tools and Applications, 83, 85703–85723. https://doi.org/10.1007/s11042-024-20138-6
Wahyudin, A. Y., & Suryadi, A. (2022). Educational values and character building in the Indonesian education system. Lingua Cultura, 18(1), 1–8. https://doi.org/10.23917/lingua.v18i1.18635
Wang, H., Wang, Y., & Li, S. (2023). Unpacking the relationships between emotions and achievement of EFL learners in China: Engagement as a mediator. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1098916
Wu, R., & Yu, Z. (2022). Exploring the effects of achievement emotions on online learning outcomes: A systematic review. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.977931
Zhao, X., Peng, Y., Song, Y., & Song, Y. (2020). Feature extraction and facial expression recognition based on CNN. Computer Methods and Programs in Biomedicine, 197, 105469. https://doi.org/10.1016/j.cmpb.2020.105469
Zhou, Y., Liu, Q., Lin, Y., Yang, H., Tang, Q., & Wang, W. (2023). Emotion Recognition in Education: A Facial Expression Recognition Approach for Real-Time Student Engagement Detection. Journal of Intelligent and Robotic Systems, 108(43). https://doi.org/10.1007/s00542-023-05420-1
Zhu, L., Yan, Y., Liu, S., & Tang, X. (2021). The Research of Online Classroom Status Feedback Based on Facial Expression Recognition Technology. 2021 2nd International Conference on Information Science and Education (ICISE-IE), 1329–1332. https://doi.org/10.1109/icise-ie53922.2021.00297
Published
How to Cite
Issue
Section
Copyright (c) 2025 Muh Riyaldi Pratama, Erwin Gatot Amiruddin, Kamaruddin Kamaruddin, Tamus Bin Tahir, Muhammad Qadri, Kumar Vivek

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.