ARCHIVES
Original Article
Deep Learning Based Facial Emotion Recognition System
Kiran Kumar Raja1
P Sanjay Kumar2
Ch Venkata Gowtham3
1 Assistant Professor, Department of Computer Science and Engineering, Vignan’s Foundation for Science, Technology and Research, Andhra Pradesh, India. 2 3 Department of Computer Science and Engineering, Vignan’s Foundation for Science, Technology and Research, Andhra Pradesh, India.
Published Online: May-August 2026
Pages: 466-478
Cite this article
↗ https://www.doi.org/10.59256/indjcst.20260502053
References
[1] P. Ekman and W. V. Friesen, “Constants across cultures in the face and emotion,” J. Personality Social Psychol., vol. 17, no. 2, pp. 124–
129, 1971.
[2] B.-C. Ko, “A brief review of facial emotion recognition based on visual information,” Sensors, vol. 18, no. 2, p. 401, 2018.
[3] M. Pantic and L. J. M. Rothkrantz, “Automatic analysis of facial expressions: The state of the art,” IEEE Trans. Pattern Anal. Mach. Intell.,
vol. 22, no. 12, pp. 1424–1445, Dec. 2000.
[4] P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The extended Cohn-Kanade dataset (CK+): A complete dataset
for action unit and emotion-specified expression,” in Proc. IEEE CVPR Workshops, San Francisco, CA, 2010, pp. 94–101.
[5] D. Lundqvist, A. Flykt, and A. Öhman, “The Karolinska Directed Emotional Faces (KDEF),” CD ROM, Dept. Clinical Neuroscience,
Karolinska Institutet, Stockholm, Sweden, 1998.
[6] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted residuals and linear bottlenecks,” in Proc.
IEEE/CVF CVPR, Salt Lake City, UT, 2018, pp. 4510–4520.
[7] T. Ahonen, A. Hadid, and M. Pietikäinen, “Face description with local binary patterns: Application to face recognition,” IEEE Trans.
Pattern Anal. Mach. Intell., vol. 28, no. 12, pp. 2037–2041, Dec. 2006.
[8] M. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba, “Coding facial expressions with Gabor wavelets,” in Proc. 3rd IEEE Int. Conf. Automatic
Face and Gesture Recognition, Nara, Japan, 1998, pp. 200–205.
[9] N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proc. IEEE CVPR, San Diego, CA, 2005, pp. 886–893.
[10] T. F. Cootes, G. J. Edwards, and C. J. Taylor, “Active appearance models,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 6, pp. 681–
685, Jun. 2001.
[11] A. Mollahosseini, D. Chan, and M. H. Mahoor, “Going deeper in facial expression recognition using deep neural networks,” in Proc. IEEE
WACV, Lake Placid, NY, 2016, pp. 1–10.
[12] E. Barsoum, C. Zhang, C. C. Ferrer, and Z. Zhang, “Training deep networks for facial expression recognition with crowd-sourced label
distribution,” in Proc. ACM ICMI, Tokyo, Japan, 2016, pp. 279–283.
[13] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE/CVF CVPR, Las Vegas, NV, 2016, pp.
770–778.[14] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proc. IEEE/CVF
CVPR, Las Vegas, NV, 2016, pp. 2818–2826.
[15] Y. Li, J. Zeng, S. Shan, and X. Chen, “Occlusion aware facial expression recognition using CNN with attention mechanism,” IEEE Trans.
Image Process., vol. 28, no. 5, pp. 2439–2450, May 2019.
[16] A. Dosovitskiy et al., “An image is worth 16×16 words: Transformers for image recognition at scale,” in Proc. ICLR, Virtual, 2021.
[17] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proc. ICLR, San Diego, CA,
2015.
[18] A. G. Howard et al., “MobileNets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861,
2017.
[19] S. L. Happy and A. Routray, “Automatic facial expression recognition using features of salient facial patches,” IEEE Trans. Affect.
Comput., vol. 6, no. 1, pp. 1–12, Jan. 2015.
[20] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proc. ICML, Lille,
France, 2015, pp. 448–456.
[21] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdi-nov, “Dropout: A simple way to prevent neural networks from
overfitting,”K.Mach. Learn. Res., vol. 15, pp. 1929–1958, 2014.
[22] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. ICLR, San Diego, CA, 2015.
[23] S. Du, Y. Tao, and A. M. Martinez, “Compound facial expressions of emotion,” Proc. Natl. Acad. Sci., vol. 111, no. 15, pp. E1454–E1462,
Apr. 2014.
[24] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA: MIT Press, 2016.
[25] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp.
2278–2324, Nov. 1998.
[26] I. A. Bachelder and M. Waxman, “Visual object recognition using Gabor filters and a priori constraints,” in Proc. IEEE CVPR, San Juan,
PR, 1997, pp. 413–418.
129, 1971.
[2] B.-C. Ko, “A brief review of facial emotion recognition based on visual information,” Sensors, vol. 18, no. 2, p. 401, 2018.
[3] M. Pantic and L. J. M. Rothkrantz, “Automatic analysis of facial expressions: The state of the art,” IEEE Trans. Pattern Anal. Mach. Intell.,
vol. 22, no. 12, pp. 1424–1445, Dec. 2000.
[4] P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The extended Cohn-Kanade dataset (CK+): A complete dataset
for action unit and emotion-specified expression,” in Proc. IEEE CVPR Workshops, San Francisco, CA, 2010, pp. 94–101.
[5] D. Lundqvist, A. Flykt, and A. Öhman, “The Karolinska Directed Emotional Faces (KDEF),” CD ROM, Dept. Clinical Neuroscience,
Karolinska Institutet, Stockholm, Sweden, 1998.
[6] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted residuals and linear bottlenecks,” in Proc.
IEEE/CVF CVPR, Salt Lake City, UT, 2018, pp. 4510–4520.
[7] T. Ahonen, A. Hadid, and M. Pietikäinen, “Face description with local binary patterns: Application to face recognition,” IEEE Trans.
Pattern Anal. Mach. Intell., vol. 28, no. 12, pp. 2037–2041, Dec. 2006.
[8] M. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba, “Coding facial expressions with Gabor wavelets,” in Proc. 3rd IEEE Int. Conf. Automatic
Face and Gesture Recognition, Nara, Japan, 1998, pp. 200–205.
[9] N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proc. IEEE CVPR, San Diego, CA, 2005, pp. 886–893.
[10] T. F. Cootes, G. J. Edwards, and C. J. Taylor, “Active appearance models,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 6, pp. 681–
685, Jun. 2001.
[11] A. Mollahosseini, D. Chan, and M. H. Mahoor, “Going deeper in facial expression recognition using deep neural networks,” in Proc. IEEE
WACV, Lake Placid, NY, 2016, pp. 1–10.
[12] E. Barsoum, C. Zhang, C. C. Ferrer, and Z. Zhang, “Training deep networks for facial expression recognition with crowd-sourced label
distribution,” in Proc. ACM ICMI, Tokyo, Japan, 2016, pp. 279–283.
[13] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE/CVF CVPR, Las Vegas, NV, 2016, pp.
770–778.[14] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proc. IEEE/CVF
CVPR, Las Vegas, NV, 2016, pp. 2818–2826.
[15] Y. Li, J. Zeng, S. Shan, and X. Chen, “Occlusion aware facial expression recognition using CNN with attention mechanism,” IEEE Trans.
Image Process., vol. 28, no. 5, pp. 2439–2450, May 2019.
[16] A. Dosovitskiy et al., “An image is worth 16×16 words: Transformers for image recognition at scale,” in Proc. ICLR, Virtual, 2021.
[17] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proc. ICLR, San Diego, CA,
2015.
[18] A. G. Howard et al., “MobileNets: Efficient convolutional neural networks for mobile vision applications,” arXiv preprint arXiv:1704.04861,
2017.
[19] S. L. Happy and A. Routray, “Automatic facial expression recognition using features of salient facial patches,” IEEE Trans. Affect.
Comput., vol. 6, no. 1, pp. 1–12, Jan. 2015.
[20] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proc. ICML, Lille,
France, 2015, pp. 448–456.
[21] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdi-nov, “Dropout: A simple way to prevent neural networks from
overfitting,”K.Mach. Learn. Res., vol. 15, pp. 1929–1958, 2014.
[22] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. ICLR, San Diego, CA, 2015.
[23] S. Du, Y. Tao, and A. M. Martinez, “Compound facial expressions of emotion,” Proc. Natl. Acad. Sci., vol. 111, no. 15, pp. E1454–E1462,
Apr. 2014.
[24] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA: MIT Press, 2016.
[25] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp.
2278–2324, Nov. 1998.
[26] I. A. Bachelder and M. Waxman, “Visual object recognition using Gabor filters and a priori constraints,” in Proc. IEEE CVPR, San Juan,
PR, 1997, pp. 413–418.
Related Articles
2026
Artificial Intelligence in Learning and Teaching
2026
Admin Assist: An AI – Driven Configuration and Orchestration for Enterprise Application
2026
Enhancing Blood Group Identification using pigeon inspired optimization: An Innovative Approach
2026
Eco-Genius: Power Up Smart, Power Down Waste
2026
Crowd-Sourced Disaster Response and Rescue Assistant
2026
