A Deep Learning Driven Simulation Analysis of the Emotional Profiles of Depression Based on Facial Expression Dynamics
Taekgyu Lee1, Seunghwan Baek2, Jongseo Lee1, Eun Su Chung1, Kyongsik Yun3,4, Tae-Suk Kim5, Jihoon Oh5
1College of Medicine, The Catholic University of Korea, Seoul, Korea
2UniAI Corporation, Daejeon, Korea
3Computation and Neural Systems, California Institute of Technology, CA, USA
4Bio-Inspired Technologies and Systems, Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA
5Department of Psychiatry, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
Correspondence to: Jihoon Oh
Department of Psychiatry, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, 222 Banpo-daero, Seocho-gu, Seoul 06591, Korea
E-mail: ojihoon@gmail.com
ORCID: https://orcid.org/0000-0003-1114-976X
Received: January 27, 2023; Revised: March 31, 2023; Accepted: April 10, 2023; Published online: June 8, 2023.
© The Korean College of Neuropsychopharmacology. All rights reserved.

This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
Objective: Diagnosis and assessment of depression rely on scoring systems based on questionnaires, either self-reported by patients or administered by clinicians, and observation of patient facial expressions during the interviews plays a crucial role in making impressions in clinical settings. Deep learning driven approaches can assist clinicians in the course of diagnosis of depression by recognizing subtle facial expressions and emotions in depression patients.
Methods: Seventeen simulated patients who acted as depressed patients participated in this study. A trained psychiatrist structurally interviewed each participant with moderate depression in accordance with a prepared scenario and without depressive features. Interviews were video-recorded, and a facial emotion recognition algorithm was used to classify emotions of each frame.
Results: Among seven emotions (anger, disgust, fear, happiness, neutral, sadness, and surprise), sadness was expressed in a higher proportion on average in the depression-simulated group compared to the normal group. Neutral and fear were expressed in higher proportions on average in the normal group compared to the normal group. The overall distribution of emotions between the two groups was significantly different (p < 0.001). Variance in emotion was significantly less in the depression-simulated group (p < 0.05).
Conclusion: This study suggests a novel and practical approach to understand the emotional expression of depression patients based on deep learning techniques. Further research would allow us to obtain more perspectives on the emotional profiles of clinical patients, potentially providing helpful insights in making diagnosis of depression patients.
Keywords: Depression; Facial expression; Deep learning; Diagnosis; Dynamic

This Article

e-submission

Archives