The facial expression of schizophrenic patients applied with infrared thermal facial image sequence

Bo-Lin Jian, Chieh-Li Chen, Wen-Lin Chu, Min-Wei Huang

Schizophrenia is a neurological disease characterized by alterations to patients' cognitive functions and emotional expressions. Relevant studies often use magnetic resonance imaging (MRI) of the brain to explore structural differences and responsiveness within brain regions. However, as this technique is expensive and commonly induces claustrophobia, it is frequently refused by patients. Thus, this study used non-contact infrared thermal facial images (ITFIs) to analyze facial temperature changes evoked by different emotions in moderately and markedly ill schizophrenia patients. Schizophrenia is an emotion-related disorder, and images eliciting different types of emotions were selected from the international affective picture system (IAPS) and presented to subjects during ITFI collection. ITFIs were aligned using affine registration, and the changes induced by small irregular head movements were corrected. The average temperatures from the forehead, nose, mouth, left cheek, and right cheek were calculated, and continuous temperature changes were used as features. After performing dimensionality reduction and noise removal using the component analysis method, multivariate analysis of variance and the Support Vector Machine (SVM) classification algorithm were used to identify moderately and markedly ill schizophrenia patients. Analysis of five facial areas indicated significant temperature changes in the forehead and nose upon exposure to various emotional stimuli and in the right cheek upon evocation of high valence low arousal (HVLA) stimuli. The most significant P-value (lower than 0.001) was obtained in the forehead area upon evocation of disgust. Finally, when the features of forehead temperature changes in response to low valence high arousal (LVHA) were reduced to 9 using dimensionality reduction and noise removal, the identification rate was as high as 94.3%. Our results show that features obtained in the forehead, nose, and right cheek significantly differed between moderately and markedly ill schizophrenia patients. We then chose the features that most effectively distinguish between moderately and markedly ill schizophrenia patients using the SVM. These results demonstrate that the ITFI analysis protocol proposed in this study can effectively provide reference information regarding the phase of the disease in patients with schizophrenia.