A Deep Learning Driven Simulation Analysis of the Emotional Profiles of Depression Based on Facial Expression Dynamics
Taekgyu Lee1, Seunghwan Baek2, Jongseo Lee1, Eun Su Chung1, Kyongsik Yun3,4, Tae-Suk Kim5, Jihoon Oh5
1College of Medicine, The Catholic University of Korea, Seoul, Korea
2UniAI Corporation, Daejeon, Korea
3Computation and Neural Systems, California Institute of Technology, Pasadena, CA USA
4Bio-Inspired Technologies and Systems, Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA
5Department of Psychiatry, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, Seoul, Korea
Correspondence to: Jihoon Oh
Department of Psychiatry, Seoul St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, 222 Banpo-daero, Seocho-gu, Seoul 06591, Korea
E-mail: ojihoon@gmail.com
ORCID: https://orcid.org/0000-0003-1114-976X
Received: January 27, 2023; Revised: March 31, 2023; Accepted: April 10, 2023; Published online: June 8, 2023.
© The Korean College of Neuropsychopharmacology. All rights reserved.

This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
Objective: Diagnosis and assessment of depression rely on scoring systems based on questionnaires, either self-reported by patients or administered by clinicians, and observation of patient facial expressions during the interviews plays a crucial role in making impressions in clinical settings. Deep learning driven approaches can assist clinicians in the course of diagnosis of depression by recognizing subtle facial expressions and emotions in depression patients.
Methods: Seventeen simulated patients who acted as depressed patients participated in this study. A trained psychiatrist structurally interviewed each participant with moderate depression in accordance with a prepared scenario and without depressive features. Interviews were video-recorded, and a facial emotion recognition algorithm was used to classify emotions of each frame.
Results: Among seven emotions (anger, disgust, fear, happiness, neutral, sadness, and surprise), sadness was expressed in a higher proportion on average in the depression-simulated group compared to the normal group. Neutral and fear were expressed in higher proportions on average in the normal group compared to the normal group. The overall distribution of emotions between the two groups was significantly different (p < 0.001). Variance in emotion was significantly less in the depression-simulated group (p < 0.05).
Conclusion: This study suggests a novel and practical approach to understand the emotional expression of depression patients based on deep learning techniques. Further research would allow us to obtain more perspectives on the emotional profiles of clinical patients, potentially providing helpful insights in making diagnosis of depression patients.
Keywords: Depression; Facial expression; Deep learning; Diagnosis; Dynamic
INTRODUCTION

Depression is one of the most under-diagnosed and under-treated mental illnesses but is among the leading cause of disability globally [1,2]. It is estimated that the prevalence of depression is 5.3% in Korea [3] and 7.5% in the United States [4], and there is an increasing trend in the prevalence over time [2]. Currently, diagnosis and assessment of major depressive disorder (MDD) rely on qualitative questionnaires such as Patient Health Question-naire-9 (PHQ-9), Composite International Diagnostic Inter-view (CIDI), Center for Epidemiological Studies Depres-sion Scale (CES-D), and Hamilton Depression Rating Scale (HAM-D), either self-reported by patients or administered by clinicians [2]. They serve as practical tools for clinicians, but they also have limitations as such questionnaire-based methods are prone to subjectivity and recall biases of respondents [5]. Overcoming such limitations can be a way to reduce the possibility of under-diagnosis and under-treatment [6].

Due to such limitations, observation of facial expres-sions and recognition of their changes during the interviews often plays a crucial role in making impressions of depression in clinical settings [7]. For example, reduced ability to express emotion through facial expressions is notably associated with depression and anxiety [8]. De-pression is also associated with impaired ability to recognize the emotions of others and respond accordingly [9]. To capture the subtle changes in facial expressions and emotions, techniques such as electromyography and eye movement trackers were attempted to quantify the movements, but simpler methods would be more useful for clinical purposes.

With regards to depression, there has been much progress in assessing such traits of depression patients with audio and video recordings on the back of deep learning techniques. Symptom severity of depression was measured based on the speech and 3D facial scans on the DAIC-WOZ dataset [10] and dynamic facial manifestations through the motion history images on AVEC’14 dataset [11]. Algorithms have also been proposed to identify potential depression risk based on the video recordings of patients conducting various structured tasks [12] and classify remission of MDD over time [13]. To have practical value in clinical settings, however, it is necessary to come up with an algorithm based on simpler data that can be obtained more easily during the first interviews with patients. Such an algorithm should be able to detect sustained negative affect and loss of pleasure, the two major features of MDD according to the Diagnostic and Statisti-cal Manual of Mental Disorders, 5th edition (DSM-5) [14].

The current diagnostic criteria of MDD based on the DSM-5 [14] calls for a clinical judgment on listed symptoms, such as depressed mood, markedly diminished interest or pleasure, slowing down of thought, a reduction of physical movement, fatigue or loss of energy, and diminished ability to think or concentrate. Quantitative dimensions and scoring systems on the listed symptoms above could be used, but there is a room for judgment calls to intervene by clinicians as the diagnosis is made in a rather categorical basis [15]. Deep learning driven approaches can assist diagnosis of depression by providing robust quantitative background in determining whether the listed symptoms have been satisfied for the diagnosis of depression based on subtle recognition of facial expressions in depression patients.

The present study aimed to differentiate emotional profile of MDD based on video recordings of structured interviews with simulated patients in controlled settings. We extracted the emotions from each frame of the recordings based on the deep learning techniques and deduced the differences in distributions of emotions between the depression and the control. An additional diagnostic tool based on facial dynamics alongside the conventional questionnaire may result in increased diagnostic precision, leading to improved reliability of clinicians equipped with consistent, albeit convenient way to further objectivity. Understanding the emotional profiles of simulated depression patients would provide useful perspec-tives upon further investigations with clinical patients.

METHODS

Participants

Seventeen simulated patients (5 male, 12 female) participated in the study. They are professionally trained actors who regularly participate as simulated patients in the clinical performance examination training at the medical schools for years. They acted as depressed patients with moderate depression in accordance with a scenario prepared in advance and as non-depressed normal control without depressive features. All participants did not have a psychiatric history, but when they performed as depression patients, they imitated the typical characteristics such as expressions and tones of depression patients. This study was approved by the Institutional Review Board of the Catholic University of Korea (KC19ENSI0198).

Data Acquisition

A psychiatrist performed an interview with each of the simulated patients in a separate room that was blocked from outside noises. Each interview was conducted for about 10 minutes based on items on the Hamilton Depres-sion Scale. The interview process was filmed using an iPhone X (1080p HD, 60 frames per second; Apple Inc.), and the subjects participated in the interview without being aware of the camera.

Data Analysis

Each frame of all thirty-four video recordings from the seventeen subjects was analyzed based on the emotion classification model using DeepFace [16]. DeepFace is a framework developed to provide libraries for face recognition and facial attribute analysis, where the attributes are age, gender, emotion, and race. In this study, we focused on emotion, while the other attributes are collected for additional references. DeepFace contains several state- of-the-art models, including WSCNet [17], WILDCAT [18], MldrNet [19], ResNet-50 [20], VGG-16 [21], etc.

The WSCNet, also known as “Weakly Supervised Coupled Convolutional Network”, automatically selects relevant soft proposals given weak annotations such as global image labels. The emotion analysis model uses a sentiment-specific soft map to couple the sentiment map with deep features as a semantic vector in the classification branch. The WSCNet outperforms the state-of-the-art results on various benchmark datasets. The model is trained with 3 million different photos to distinguish facial emotions and the accuracy is over 99.6%, which exceeds the human cognition.

If such a model is newly trained, there is a risk of under-performance by further training the model. Thus, pre-trained weight was used in this study without fine- tuning. Accuracy of the pre-trained model was verified by feature importance analysis to identify the most differentiated areas of the facial images for each emotion and saliency map to review the areas of the images that contribute the most to the model’s predictions.

WILDCAT (Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation) is another deep learning method which jointly aims at aligning image regions for gaining spatial invariance and learning strongly localized features. MldrNet (Multi-Level Deep Representation Learning Network) is an architecture for multi-label classification that outperforms existing methods on several benchmark datasets, achieving state-of-the-art performance on the Microsoft COCO and PASCAL VOC datasets. MldrNet uses a multi-level architecture that integrates features from different layers of a pre-trained convolutional neural network to learn high-level representations for the classification task. It has achieved top-1 accuracy of 53.4% on the COCO dataset and mean average precision of 48.7% on the PASCAL VOC dataset, which are significant improvements over previous state-of-the-art methods.

ResNet-50 (Residual Network with 50 layers) and VGG- 16 (Visual Geometry Group with 16 layers) are deep convolutional neural network models proposed to alleviate existing problems and improve training performances, effective for image classifications and face recognition tasks.

The environment under which data for training was collected greatly affects the accuracy of models [22,23]. For instance, a model for facial emotion recognition in DeepFace achieves 98.9% in a controlled environment, while the accuracy drops to 55.3% in an uncontrolled environment. Uncontrolled factors that can attribute to the reduced accuracy include small faces, unequal positions of faces, occlusion or blurring of faces, varying resolution, different degree of illumination, and so on. Con-trolled environment, on the other hand, is the well-designed experimental environment which excludes such factors. In this study, subjects were mostly tested in an environment similar to that of CK+ (ViT+SE, 99.8%) and JAFFE (ViT, 94.83%) datasets. Considering our dataset was created in a static, controlled environment as in CK+ and JAFFE datasets, it is reliable to apply DeepFace to the dataset created in this study.

The process to classify emotion from each frame using facial emotion recognition was performed largely in three steps. First, image frames were undergone through pre- processing, where each frame was reviewed if one and only one face was detected on the given frame and further validated if the images were taken under a consistent en-vironment. Then, the images were converted to gray scale for face detection and emotion classification through facial contours. The facial contour and the coordinates of the two eyes were detected in the images. The detected face area was resized to (48 × 48) pixels in order to match the size before input to the emotion classification model. The detected face area was rotated so that the line connecting the centers of the detected eyes was parallel to the x-axis. Finally, the normalized images were put into the emotion classification model.

The proportion of each of the seven emotions (anger, disgust, fear, happiness, neutral, sadness, and surprise) was compared between the depression-simulated group and the normal group with paired ttest. The distribution of the emotions from the entire recording frames between the two groups was analyzed with chi-squared test of independence. The frequency of emotional change, mea-sured as a proportion of frames with a shift in emotions compared to the previous frames, was compared with one-sided paired ttest.

RESULTS

Distribution of Emotional States

To examine whether depression is associated with increased or muted expression of any emotions compared to the normal control, we calculated the proportion of frames represented by each of the seven emotions amongst the total number of frames analyzed.

Sadness was expressed significantly more frequently among the depression-simulated group (p value = 0.012) based on the paired ttest. The proportion of sadness was 44.2% on average among the depression-simulated group, compared to 23.9% among the normal group. Neutral and fear were expressed significantly more frequently among the normal group (p value = 0.015 and 0.044, respectively). The proportion of neutral was 29.0% on average among the depression-simulated group, compared to 46.3% among the normal group. The proportion of fear was 4.4% on average among the depression-simulated group, compared to 7.1% among the normal group. Pro-portions of other emotions such as anger, disgust, happiness, and surprise did not show a significant difference between the two groups (Table 1).

We also examined the proportion of each emotion in all recordings in each group, 17 recordings of 132,304 frames for the depression-simulated group and the other 17 recordings of 85,217 frames for the normal group (Table 2). Within the depression-simulated group, sadness was the most frequently observed emotion (45.0%), followed by neutral (29.5%), happiness (11.8%), and anger (6.2%). In contrast, neutral was the most frequently observed emotion in the normal group (45.2%), followed by sadness (23.6%), happiness (15.7%), and fear (8.3%). The distributions of emotional states of the two groups in total were significantly different based on a chi-squared test of independence (p < 0.001).

Change in the Emotional States

As depression is known to be associated with reduced expressions of emotions [24], we assumed that the variance in the observed emotional states would be less in the depression group. The variance was defined as the proportion of frames where emotion changes from the previous frames.

In the depression-simulated group, changes in emotional states were observed in 32.7% of the recorded frames, whereas in the normal group, changes were observed in 38.3%. The depression-simulated group showed a smaller variance in expressed emotions than the normal group (p value = 0.022) on a one-sided paired ttest, suggesting that the blunted emotion among the depressive patients can be found in frame-by-frame emotion recognition based on patients’ facial expression (Table 3).

DISCUSSION

In this study of 17 simulated subjects participating in both the depression and the normal group, the depression-simulated group showed a higher proportion of sadness and a lower proportion of neutral and fear than the normal group based on the frame-by-frame deep learning detection of emotions on facial video recordings. The distribution of emotions was different, and the suppression of emotions among the depression group could also be detected.

Depression is related to persistent sadness and lack of happiness [14] and previous research found that depressed patients tend to show deficits in positive emotions and suppress the expression of emotions [25]. This study's significant expression of sadness in the depression group coincides with the previous findings, although a lack of positive emotions was not so apparent. More simulated patients could have focused on emphasizing the depressive traits as the recording only went on for a limited duration. A lack of positive emotions could also be noticed if the interviews were recorded for an extended period to allow the subjects to express various emotions shown by depression patients. Further classification of what was labeled as “neutral” by the deep learning methods could also mitigate the unclear results on positive emotions. A higher proportion of fear among the normal group was also distinctive, as fear, by intuition, is an emotion classified as negative rather than positive. This could imply suppressed fear among depression patients, but further discussions with the subjects could provide more insights.

We also tracked changes in emotions over time and considered that the frequent emotional changes would reflect a higher level of emotional expressions. Significantly less variation in emotion in the depression group confirmed that the suppression can be measured quantitatively by analyzing the facial expressions of each frame. Further studies can be performed on whether suppression of emotions can be detected separately for both positive and negative emotions.

Existing methodologies to diagnose depression are mostly based on self-reported questionnaires and interviews in clinical settings, and both provide an adequate degree of confidence [26], but they are prone to self-bias, subjective interpretations, and indirect extraction of emotions by interviewers. This novel study aimed to suggest the possibility of adding a quantitative tool to making a diagnosis of depression. Extraction of emotions from facial expres-sions on the back of deep learning technologies allows us to improve the accuracy of diagnosis by not only having an additional diagnostic tool but also removing the involvement of human subjectivity in the process of diag-nosis. Extraction of emotions directly from the subjects could also exclude various biases that can arise in the courses of diagnosis by integrating the two-step process of interviewing and interpreting.

Although this study suggested the potential to utilize the quantitative method in the diagnosis of depression patients, it has several limitations. First, the subjects were not clinical patients, but professionals who have served as simulated depression patients for a long term at medical institutions. Although it is difficult to guarantee that there is no difference between the facial expressions shown by the simulated patients, we believe that there is a consensus among psychiatrists on the facial expressions and emotions of depression patients, and the enrolled simulated patients are highly trained professionals who can mimic such traits to reflect the consensus. They expressed behaviors, emotions, verbal and non-verbal expressions, languages, and other features of actual depression patients based on their experiences and analysis.

Secondly, the number of subjects recruited for this study was rather small. Despite the significant differences in proportions of certain emotions, emotional distribu-tions, and emotional variances between the depression group and the normal group with simulated patients, a larger number of subjects could have revealed more findings. Further research could be performed with a larger number of actual patients to reaffirm the findings of this study.

Thirdly, in a way to confirm the suppression of emotional expressions of depression patients, the variance of emotion was measured in terms of the number of changes in emotions in each frame. However, there could be different ways to measure the variance of emotion, reflecting the positivity and negativity of each emotion. For example, let us consider hypothetical sequences of emotions such as (1) “sadness-neutral-sadness” and (2) “sadness- happiness-sadness”. When comparing (1) and (2), the methodology that was used in this study would yield the same variance of emotions, as there were equally two frames with emotion changes within the three frames recorded. However, sequence (1) did not show any positivity, while sequence (2) moved across positive and negative emotions back and forth, indicating a higher variance of emotions. Delicate assignment of adequate numbers to each emotion could provide a better picture of the variance of emotions and describe the suppressive feature of depression patients in a more quantifiable manner.

Fourthly, depression is a globally arising mental health problem, but a facial expression can be interpreted differently by regions, cultures, languages, races, or ages. The deep learning technique used in this study extracted emotions from the facial expressions of subjects recruited from Korea. However, applying the same methodology in different countries could result otherwise.

In conclusion, this study suggests a novel and practical approach with an easily accessible device and a facial emotion recognition algorithm to understand the emotional expression of depression patients based on deep learning techniques. Emotional profiles of simulated depression patients showed a distinctive pattern, both in proportions of certain emotions and in the frequency of emotional changes, compared to the normal control group. Further research would allow us to obtain more perspectives on the emotional profiles of clinical patients, potentially providing helpful insights in making diagnosis of depression patients as an additional diagnostic tool alongside the current diagnostic approaches.

ACKNOWLEDGEMENTS

The authors thank to START center, The Catholic Uni-versity of Korea, for their support in recruiting simulated patients and performing experiments.

The funder had no role in the design, collection of data, analysis, interpretation of results, and drafting the manuscript.

Conflicts of Interest

No potential conflict of interest relevant to this article was reported.

Author Contributions

Concept and design: Taekgyu Lee, Jihoon Oh. Data Acquisition: Jihoon Oh. Analysis: Seunghwan Baek, Kyongsik Yun. Interpretation of data: Taekgyu Lee, Seunghwan Baek, Kyongsik Yun. Drafting of the manuscript: Taekgyu Lee, Jihoon Oh, Jongseo Lee, Eun Su Chung. Critical revision of the manuscript: Tae-Suk Kim, Kyongsik Yun, Jihoon Oh, Taekgyu Lee. Obtained funding: Jihoon Oh. Jihoon Oh takes all responsibilities for this manuscript.

Tables

Proportion of emotions

Emotion Total (n = 34) Depressive-state (n = 17) Normal (n = 17) p valuea
Anger 0.427
Mean (95% CI) 5.3% (0.03−0.08) 6.2% (0.02−0.10) 4.4% (0.02−0.07)
Disgust 0.566
Mean (95% CI) 0.8% (0.00−0.01) 0.6% (0.00−0.01) 1.0% (0.00−0.02)
Fear 0.044
Mean (95% CI) 5.7% (0.03−0.08) 4.4% (0.02−0.07) 7.1% (0.03−0.11)
Happiness 0.686
Mean (95% CI) 15.0% (0.10−0.20) 13.9% (0.06−0.22) 16.0% (0.11−0.21)
Neutral 0.015
Mean (95% CI) 37.7% (0.30−0.45) 29.0% (0.17−0.41) 46.3% (0.39−0.54)
Sadness 0.002
Mean (95% CI) 34.1% (0.26−0.42) 44.2% (0.30−0.59) 23.9% (0.18−0.30)
Surprise 0.320
Mean (95% CI) 1.4% (0.01−0.02) 1.6% (0.01−0.03) 1.2% (0.01−0.02)

CI, confidence interval.

aTwo-sided paired ttest on depression-simulated group vs. normal group.

Distribution of emotions

Emotion Total (n = 34) Depressive-state (n = 17) Normal (n = 17) p valuea
Number of frames classified as < 0.001
Angry 12,432 (5.7) 8,175 (6.2) 4,257 (5.0)
Disgust 1,820 (0.8) 1,031 (0.8) 789 (0.9)
Fear 13,777 (6.3) 6,721 (5.1) 7,056 (8.3)
Happy 28,939 (13.3) 15,592 (11.8) 13,347 (15.7)
Neutral 77,567 (35.7) 39,047 (29.5) 38,520 (45.2)
Sad 79,546 (36.6) 59,472 (45.0) 20,047 (23.6)
Surprise 3,440 (1.6) 2,266 (1.7) 1,174 (1.4)  
Total number of frames analyzed 217,521 (100.0) 132,304 (100.0) 85,217 (100.0)  

Values are presented as number of frames (%).

aChi-squared test of independence for all recorded frames on depression-simulated group vs. normal group.

Emotion changes by frame

Emotion change Total (n = 34) Depressive-state (n = 17) Normal (n = 17) p valuea
Frames with emotion change 0.022
Mean (95% CI) 35.5% (0.31−0.40) 32.7% (0.26−0.40) 38.3% (0.33−0.43)
Observed emotion changes
< 20% 4 (11.8) 3 (17.6) 1 (5.9)
20−29.9% 6 (17.6) 4 (23.5) 2 (11.8)
30−39.9% 13 (38.2) 4 (23.5) 9 (52.9)
40−49.9% 7 (20.6) 4 (23.5) 3 (17.6)
≥ 50% 4 (11.8) 2 (11.8) 2 (11.8)

Values are presented as number of subjects (%).

CI, confidence interval.

aOne-sided paired ttest on depression-simulated group vs. normal group.

References
  1. Zomer E, Rhee Y, Liew D, Ademi Z. The health and productivity burden of depression in South Korea. Appl Health Econ Health Policy 2021;19:941-951.
    Pubmed CrossRef
  2. Moreno-Agostino D, Wu YT, Daskalopoulou C, Hasan MT, Huisman M, Prina M. Global trends in the prevalence and incidence of depression: A systematic review and meta-analysis. J Affect Disord 2021;281:235-243.
    Pubmed CrossRef
  3. Kim GE, Jo MW, Shin YW. Increased prevalence of depression in South Korea from 2002 to 2013. Sci Rep 2013;10:16979.
    Pubmed KoreaMed CrossRef
  4. Rhee TG, Wilkinson ST, Steffens DC, Rosenheck RA, Olfson M. Prevalence of treatment for depression among US adults who screen positive for depression, 2007-2016. JAMA Psychiatry 2020;77:1193-1195.
    Pubmed KoreaMed CrossRef
  5. Paulhus DL. Socially desirable responding on self-reports. In: Zeigler-Hill V, Shackelford TK, editors. Encyclopedia of personality and individual differences. Springer International Publishing;2017. p.1-5.
    CrossRef
  6. Mitchell AJ, Vaze A, Rao S. Clinical diagnosis of depression in primary care: A meta-analysis. Lancet 2009;374:609-619.
    Pubmed CrossRef
  7. Trémeau F, Malaspina D, Duval F, Corrêa H, Hager-Budny M, Coin-Bariou L, et al. Facial expressiveness in patients with schizophrenia compared to depressed patients and nonpatient comparison subjects. Am J Psychiatry 2005;162:92-101.
    CrossRef
  8. Gaebel W, Wölwer W. Facial expression and emotional face recognition in schizophrenia and depression. Eur Arch Psychiatry Clin Neurosci 1992;242:46-52.
    Pubmed CrossRef
  9. Surguladze SA, Young AW, Senior C, Brébion G, Travis MJ, Phillips ML. Recognition accuracy and response bias to happy and sad facial expressions in patients with major depression. Neuropsychology 2004;18:212-218.
    Pubmed CrossRef
  10. Haque A, Guo M, Miner AS, Fei-Fei L. Measuring depression symptom severity from spoken language and 3D facial expressions. arXiv. 1811.07216 [Preprint]; 2018 [cited 2022 Jul 24]. http://arxiv.org/abs/1811.08592.
  11. Pampouchidou A, Pediaditis M, Kazantzaki E, Sfakianakis S, Apostolaki IA, Argyraki K, et al. Automated facial video-based recognition of depression and anxiety symptom severity: Cross-corpus validation. Mach Vis Appl 2020;31:30.
    CrossRef
  12. Guo W, Yang H, Liu Z, Xu Y, Hu B. Deep neural networks for depression recognition based on 2D and 3D facial expressions under emotional stimulus tasks. Front Neurosci 2021;15:609760.
    Pubmed KoreaMed CrossRef
  13. Jiang Z, Harati S, Crowell A, Mayberg HS, Nemati S, Clifford GD. Classifying major depressive disorder and response to deep brain stimulation over time by analyzing facial expressions. IEEE Trans Biomed Eng 2021;68:664-672.
    Pubmed KoreaMed CrossRef
  14. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, fifth edition. 5th ed. American Psychiatric Association Publishing;2013.
    CrossRef
  15. Hyman SE. The diagnosis of mental disorders: The problem of reification. Annu Rev Clin Psychol 2010;6:155-179.
    Pubmed CrossRef
  16. Taigman Y, Yang M, Ranzato M, Wolf L. DeepFace: Closing the gap to human-level performance in face verification. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH;2014. p.1701-1708.
    CrossRef
  17. She D, Yang J, Cheng MM, Lai YK, Rosin PL, Wang L. WSCNet: Weakly supervised coupled networks for visual sentiment classification and detection. IEEE Trans Multimedia 2020;22:1358-1371.
    CrossRef
  18. Durand T, Mordan T, Thome N, Cord M. WILDCAT: Weakly supervised learning of deep ConvNets for image classification, pointwise localization and segmentation. In: 2017 IEEE Con-ference on Computer Vision and Pattern Recognition (CVPR). p.5957-5966.
    CrossRef
  19. Rao T, Li X, Xu M. Learning multi-level deep representations for image emotion classification. Neural Process Lett 2020;51:2043-2061.
    CrossRef
  20. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. arXiv. 1512.03385 [Preprint]; 2015 [cited 2023 Mar 21]. http://arxiv.org/abs/1512.03385.
  21. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv. 1409.1556 [Preprint]; 2015 [cited 2023 Mar 21]. http://arxiv.org/abs/1409.1556.
  22. Gan Y, Chen J, Xu L. Facial expression recognition boosted by soft label with a diverse ensemble. Pattern Recognition Letters 2019;125:105-112.
    CrossRef
  23. Holder RP, Tapamo JR. Improved gradient local ternary patterns for facial expression recognition. J Image Video Proc 2017;2017:42.
    CrossRef
  24. Gehricke J, Shapiro D. Reduced facial expression and social context in major depression: Discrepancies between facial muscle activity and self-reported emotion. Psychiatry Res 2000;95:157-167.
    Pubmed CrossRef
  25. Beblo T, Fernando S, Klocke S, Griepenstroh J, Aschenbrenner S, Driessen M. Increased suppression of negative and positive emotions in major depression. J Affect Disord 2012;141:474-479.
    Pubmed CrossRef
  26. Stuart AL, Pasco JA, Jacka FN, Brennan SL, Berk M, Williams LJ. Comparison of self-report and structured clinical interview in the identification of depression. Compr Psychiatry 2014;55:866-869.
    Pubmed CrossRef


This Article

Close ✕


Cited By Articles

Author ORCID Information

Funding Information

Services
Social Network Service

e-submission

Archives