Convergent evidence indicate that cognitive impairment is prevalent in patients with major depressive disorder (MDD) [1]. Among patients with MDD, 25−70% of persons manifesting with depression exhibit cognitive deficits [2,3], and the effect sizes of cognitive deficits relative to healthy controls (HCs) in meta-analyses were reported to be 0.32 to 0.97 [4,5]. It has been reported that persons experiencing a major depressive episode (MDE) have experienced impairment in the following cognitive domains: alertness, psychomotor speed, sustained attention, memory, and executive functioning [4-6]. Moreover, persons in remission of MDD have reported cognitive deficits especially in the domain of selective attention, working memory, and long-term memory [7]. Cognitive impairment in patients with MDD is associated with functional outcomes and could be a principal mediator of functional impairments [8,9]. Moreover, cognitive remediation is reported to significantly improve depressive symptoms as well as cognitive function in patients with MDD [10]. Hence, it is crucial to screen for cognitive impairments in persons with MDD with a validated cognitive screening tool that is feasible for clinical practice.

In Korea, efforts have been consistently made to standardize various tools for assessing the diverse symptoms of mood disorders in the Korean language [11-14]. As part of these endeavors, we have also decided to standardize a Korean version of a tool for assessing cognitive function in MDD. Among the recent screening tools for cognitive impairment in MDD, the THINC-integrated tool (THINC-it) is a computerized screening tool to assess measures of cognition developed by the THINC task force with an international intent [15,16]. The tool was validated only in English [15,17] and Chinese [18], although it can be applied in 15 languages, including English, German, French, Italian, Spanish (Spain and Mexico), Portuguese, Korean, Chinese, Dutch, Danish, Polish, Japanese, Arabic and Hungarian.

Hence, the purpose of this study was to test the reliability and validity of the Korean version of the THINC-it tool (THINC-it-K) in adult patients with MDD in Korea.

Subjects

Patients with MDD visiting the outpatient department of the three branch hospitals of The Catholic University of Korea (Yeouido St. Mary’s Hospital, Uijeongbu St. Mary’s Hospital, and Daejeon St. Mary’s Hospital) from May 2017 to December 2019 were screened for eligibility for study participation. For the convenience of sampling, healthy volunteers from three respective communities adjacent to the three branch hospitals were incorporated as control subjects. The criteria for the enrollment of patients with MDD were: (1) patients aged 19−65 years who visited the outpatient department; (2) patients diagnosed with MDD according to the Diagnostic and Statistical Manual of Mental Disorders-5th edition (DSM-5) as confirmed by MINI International Neuropsychiatric Interview; (3) patients with the Montgomery–Åsberg Depression Rating Scale (MADRS) [19] score of 22 or more; (4) patients with a current MDE lasting 3 or more months; (5) patients have been receiving a stable antidepressant dose or regimen for a minimum of 2 weeks prior to the study visit; and (6) patients who signed the informed consent form.

The exclusion criteria were: (1) patients with other psychiatric or physical diseases that might affect cognition including alcohol/substance use disorders; (2) patients with comorbid psychiatric disorders of primary clinical concern; (3) patients who were taking any drugs that might affect cognitive function (e.g., glucocorticoids, β-blockers, opioid analgesics, and central stimulants); (4) patients who took benzodiazepines within 12 hours before the THINC-it-K test; (5) patients who took alcohol within 8 hour before the THINC-it-K test; (6) patients who had physical, cognitive, or language impairment severe enough to adversely affect the validity of the data; (7) patients with history of diagnosis of a reading disability, dyslexia, or clinically significant learning disorder; (8) patients who received electroconvulsive therapy in the last 6 months; or (9) patients with history of moderate or severe head trauma (e.g., loss of consciousness for more than 1 hour), or other neurologic disorders or unstable physical disorders likely to affect the central nervous system determined by investigators.

The criteria for the enrollment of the HCs were: (1) individuals aged 19−65 years without current or past history of mental illness; (2) individuals had no first-degree relatives with a mental illness diagnosed by a medical doctor; (3) individuals without neurologic disorders or unstable physical disorders likely to affect the central nervous system determined by investigators; (4) individuals who signed the informed consent form. The exclusion criteria were: (1) individuals with psychiatric or physical illness that might affect cognition; (2) individuals who were taking drugs that might affect cognitive function (glucocorticoids, β-blockers, opioid analgesics, and central stimulants); (3) individuals who took alcohol within 8 hours before the THINC-it test; or (4) individuals who could not read and understand the informed consent form. The study was approved by the ethics committee of The Catholic University of Korea (IRB number: XC16FIMI0051S).

Assessment

The English version of applied cognitive assessment and calibration tools were described elsewhere in detail [15,16]. As calibration tests, the pen-and-paper versions of the Korean version of Perceived Deficits Questionnaire for Depression–5-item (PDQ-5-D) [14], Digit Symbol Substitution Test (DSST) [20] and Trail Making Test–Part B (TMT-B) [21], and CogState battery (https://www.cogstate.com/digital-cognitive-assessment/) including the Identifi-cation Task (IDN) using the Choice Reaction Time paradigm [22], and the One-Back test (ONB) [23] were per-formed. The Korean version of THINC-it (THINC-it-K) set was used for digitalized cognitive assessment, which included ‘Spotter’ (corresponding to IDN), ‘Symbol Check’ (corresponding to ONB), ‘Codebreaker’ (corresponding to DSST), ‘Trails’ (corresponding to TMT-B), and the PDQ-5-D, In addition, we performed clinical assessments including MADRS to establish severity for depression, and Sheehan Disability Scale (SDS) [24,25] as a measure of psychosocial function.

Study Procedure

Structured interviews with the HCs and patients with MDD were administered by qualified and trained psychiatrists at each investigational sites to obtain the demographic and clinical data. The sequence of the calibration tests and THINC-it-K component scales remained identical for all subjects throughout the study as the following order: cognitive function assessment using the THINC-it-K tool (PDQ-5-D, Spotter, Symbol Check, Codebreaker, and Trails), and cognitive function assessment of calibration (IDN and ONB using CogState software followed by the pen-and-paper version of DSST, TMT-B, and PDQ-5-D in the same order of administration as in the THINC-it-K). The order of administration of the THINC-it-K and the calibration tests using CogState software/pen-and-paper version were alternated between subjects to account for potential order effects. Patients with MDD underwent all assessments on one occasion during a single visit. HCs completed the full set of cognitive assessments three times on the first visit to assess practice effects, and were retested one week later to assess test-retest reliability of THINC-it-K. The analyses herein utilized data from baseline measures of HCs and attempt 1 of patients with MDD.

Statistical Analysis

Results of descriptive analysis was presented as mean (standard deviation [SD]) for the continuous variables, and as frequency (percentage) for the categorical variables. The calculation from the raw score to z score was performed according to the original study [15]. All z scores were sign-adjusted, such that higher z scores denote better performance. We compared a composite z score defined as the equally weighted (assigned a weight of 0.20) mean of z scores of all THINC-it tasks performed (i.e., Spotter, Symbol Check, Codebreaker, Trails, and PDQ-5-D) of HCs and patients with MDD to evaluated whether cognitive dysfunction in patients living with MDD could be detected. In addition, objective composite z score (i.e., Spotter, Symbol Check, Codebreaker, and Trails, each weighed of 0.25) was compared between HCs and patients with MDD. The z score for the PDQ-5-D derived from THINC-it-K was compared to evaluate whether subjective cognitive complaints in patients with MDD could be detected when compared to the HCs.

For comparisons between the groups, the chi-square test was used for categorical variables, and the independent t-test was performed for the continuous variables. Mean differences (MDs), standard errors (SEs), and 95% confidence intervals (CIs) are presented. Pearson correlation analysis comparing the THINC-it-K subtests to corresponding cognitive function tests was conducted to examine concurrent validity. Internal consistency of the PDQ-5-D and THINC-it-K composite scores was assessed using the Cronbach α. Analyses of concurrent validity and internal consistency were delimited to patients with MDD.

The data were analyzed using SPSS 18.0 for window (IBM Co.) set at the level of significance as 0.05.

Subject Characteristics

A total of 88 subjects (44 patients with MDD and 44 HCs) were included in the analysis. The demographic and clinical characteristics of the two groups are presented in Table 1. There was no difference between HC group and MDD group in age, sex, education and marital status. It was observed that fewer persons were employed in the MDD group (p < 0.001) and showed higher MADRS (p < 0.001) and SDS (p < 0.001) total scores when compared to the HC group (Table 1).

Differences in the Composite THINC-it-K Score

A significant difference was detected between the MDD group and HC group when comparing objective cognitive performance according to the total composite z score of the THINC-It-K (MD [SE] = −1.31 [0.15], p < 0.001; 95% CI = −1.69 to −1.00). The mean differences in performance between HC group and MDD group for Spotter, Codebreaker, PDQ-5-D and objective THINC-it-K composite score were significant (Table 2). However, no significant differences between the 2 groups were observed in Symbol Check and Trails (Table 2).

In addition, 13.6% (n = 6) of MDD group demonstrated cognitive performed between 0.5 and 1 SD below the mean of the HC group, while 75.0% (n = 33) performed at 1 SD or lower than the HC group’s mean for THINC-it-K total composite z score. Conversely, every subject of HC group performed better on the THINC-it-K total composite z score when compared to the mean for MDD group.

Reliability and Validity of the THINC-it-K

Concurrent validity of each test, and objective and total composite score evaluated with Pearson r were presented in Table 3. Concurrent validity was the highest between THINC-it-K and pen-and-paper PDQ-5-D (r = 0.928, p < 0.001) and the lowest between Symbol Check and Cogstate ONB (r = 0.076, p = 0.625). Internal consistency for PDQ-5-D, 4 objective tests and all 5 tests of pen-and-paper test and THINC-it-K evaluated with Cronbach’s alpha for MDD group were presented in Table 4. The internal consistency reliability of the PDQ-5-D was acceptable with Cronbach’s alpha of 0.876, but the internal consistency coefficients for the objective composite score and total composite score were low (0.490 and 0.362, respectively).

Numerous investigations have revealed substantial impairments in various domains of cognitive function among individuals diagnosed with MDD when compared to HCs (e.g., information processing speed, attention, verbal learning and memory, working memory, and executive function) [26]. The findings presented in this study demonstrate that the THINC-it-K is a sensitive tool for identifying cognitive dysfunction in adults with MDD. The proportion of individuals demonstrating clinically significant cognitive deficits in our sample (75.0%) is comparable to what has been reported in other studies [9,26], which used more comprehensive and time-consuming testing methods. However, the proportion of patients with MDD and cognitive deficits was higher than a previous study by McIntyre et al. [15] (i.e., 44.4%). The foregoing discrepancy could be attributed to the difference in sample characteristics, specifically the mean age, proportion of patients with recurrent episodes and past hospitalization being higher in our MDD sample compared to the previous study [15].

The THINC-it instrument has undergone validation in a Caucasian and Chinese sample and has demonstrated the ability to identify cognitive impairment in individuals with MDD [15,18]. The present study replicates and expands upon previous research findings by indicating that Korean adults with MDD exhibit cognitive deficits, including attentional deficits measured with Spotter and Codebreaker, reduced information processing speed evaluated with Codebreaker, and impairment in executive function indicated by Codebreaker. Furthermore, subjective reports of cognitive difficulties were found to be significantly elevated among individuals with MDD compared to HCs. These results are in line with that of previous studies, wherein the MDD group had demonstrated decreased performance in the Spotter and objective composite score as well as greater objective cognitive impairment when compared to the HC group [15,18]. However, the findings of the present study and those of Hou et al. [18] indicate lower performance on the Codebreaker task in the MDD group compared to the HC group.

It is noted however, McIntyre et al. [15] did not report a significant difference in performance on this task between the two groups. The reasons for this discrepancy in performance between the MDD and HC groups on the Codebreaker task cannot be fully determined because of the differences in sample characteristics. However, it is possible that the relatively older age and higher proportion of patients with recurrent episodes in the MDD group of our sample may have contributed to this significant difference. It should be noted that effect sizes for the DSST correspond to the Codebreaker task in THINC-it tool have been shown to be influenced by both age and the number of episodes in previous research [7,27].

The present study evaluated the internal consistency reliability of the PDQ-5-D and found it to be acceptable with Cronbach’s alpha of 0.876, consistent with previous studies [18,28]. Additionally, the internal consistency coefficient for the four objective tests administered in this study (THINC-it-K objective composite score) was calculated to be 0.490; this may be related to the varying cognitive domains assessed by the objective tests. Furthermore, it was observed that the internal consistency reliability of the five subtests of the THINC-it-K (THINC-it-K total composite score) decreased when subjective measures were included (Cronbach’s α = 0.362), which may be attributed to the differential contributions of subjective and objective tests to cognitive screening as the previous studies have shown [15,18].

The THINC-it-K showed good concurrent validity in most tests. Pearson r was ranged from 0.422 for the Spotter between IDN to 0.928 for pen-and-paper PDQ-5-D between THINC-it-K PDQ-5-D; (p < 0.005). A correlation of substantial magnitude was observed between THINC-it-K and CogState software/pen-and-paper version in the Trails and TMT-B (r = 0.764) and the Codebreaker and DSST (r = 0.650) among objective tests. These validities of tasks suggests that THINC-it-K could be reliable proxy measures of the standard software/pen-and-paper version of cognitive tests in patients living with MDD. However, the lowest and non-significant concurrent validity was reported for the Symbol check and ONB (r = 0.076, p = 0.625). In previous studies, the correlation between the Symbol check and ONB was the lowest among objective tests in previous studies. The Pearson correlation coefficient was 0.19 in Harrison et al. [28], −0.146 in McIntyre et al. [15], and 0.343 in Hou et al. [18], as well. As discussed by Hou et al. [18] and Harrison et al. [28], this discrepancy may be attributed to the differing cognitive demands of the two tasks. In the One-Back Task, the participant is asked to indicate whether it is the same as the one presented one step earlier which required a binary “yes” or “no” decision. However, the Symbol Check task necessitates the study participant to indicate their response by selecting the symbol presented immediately prior, among a selection of five options. This requires the participant to rapidly alternate their attention between the sequence of stimuli and the response choices. Conversely, the traditional binary decision version of the task does not typically necessitate visual attention to be directed towards the potential responses. It is plausible that the Symbol Check task imposes a greater demand on attentional and executive resources in comparison to traditional implementations of the One-Back paradigm.

One possible limitation of this study was that IQ was not measured. In a previous study [18], there was significant difference in IQ between MDD group and HC, and the differences in performance in the Spotter and the Codebreaker were no longer significant after adjusting the analysis for IQ. This means that IQ may affect the ability to detect a difference between MDD group and HC. Moreover, it has been reported that HCs with higher IQ have been associated with better performance in neurocognitive tests [29]. One of the well-researched theories which explain the contribution of IQ on neurocognitive performance is the idea of cognitive reserve [30,31]. This theory suggests that when the brain is facing problems, it tries to find new, more effective ways to compensate it, recruiting more efficient alternative neural network such as premorbid IQ, and therefore people with higher IQ may be able to better cope with brain pathology. Also, research shows that people with higher cognitive reserve tend to perform better in various cognitive domains [32]. However, it is possible that IQ does not significantly affect cognitive performance in patients with MDD. For example, it was reported that patients with MDD differed from HC on neurocognitive tests of verbal learning and memory as well as attention and executive function, and the results remained significant even after controlling for premorbid IQ [33]. The relationship between premorbid IQ and neurocognitive outcomes in depression has not yet been fully elucidated, therefore, the results from the present study should be interpreted with caution.

An additional limitation of the present study is small sample size. In addition, because our subjects with MDD were relatively chronic, recurrent, moderate-to-severe depression, the results from this study may not be generalized to patients with less severe depression. A larger sample would benefit a future study to improve generalizability of the results.

In summary, the psychometric properties, subjective reliability and validity, and objective validity of the THINC-it-K were deemed appropriate for adults with MDD. This marks that the THINC-it-K could be a validated tool to evaluate cognitive performance for Korean patients living with MDD within a healthcare setting.

Roger S. McIntyre has received research grant support from CIHR/GACD/National Natural Science Foundation of China (NSFC) and the Milken Institute; speaker/con-sultation fees from Lundbeck, Janssen, Alkermes, Neumora Therapeutics, Boehringer Ingelheim, Sage, Biogen, Mitsubishi Tanabe, Purdue, Pfizer, Otsuka, Takeda, Neurocrine, Sunovion, Bausch Health, Axsome, Novo Nordisk, Kris, Sanofi, Eisai, Intra-Cellular, NewBridge Pharmaceuticals, Viatris, Abbvie, Atai Life Sciences. Dr. Roger McIntyre is a CEO of Braxia Scientific Corp. Kayla M. Teopiz has received fees from Braxia Scientific Corp. All other authors have no conflict of interest to report.

Conceptualization: Young Sup Woo, Won-Myong Bahk. Study design: Young Sup Woo, Won-Myong Bahk. Data acquisition: Young Sup Woo, Won-Myong Bahk, Kyoung-Uk Lee, Changtae Hahn. Formal analysis: Young Sup Woo, Won-Myong Bahk. Data interpretation: Young Sup Woo, Won-Myong Bahk, Kyoung-Uk Lee, Changtae Hahn. Writing—original draft: Young Sup Woo, Won-Myong Bahk. Writing—review & editing: Roger S. McIntyre, Kayla M. Teopiz. Final manuscript approve: Young Sup Woo, Won-Myong Bahk.