Universitas Gadjah Mada Innovation: Multimodal AI Detects Depression Through Facial Expressions and Intelligent Interviews
Large Language Models for Screening Depression Through Facial Emotion Recognition and Open-Ended PHQ Testing
YOGYAKARTA – A research team from Universitas Gadjah Mada (UGM), consisting of Samudera Fadlilla Jamaluddin, S.Psi., M.Sc., Riangga Novrianto, S.Psi., M.Psi., Psychologist, and Muhammad Oriza Nurfajri, S.Kom., M.IT., is developing a revolutionary framework for depression screening. This research, focused on AI for Healthcare, leverages a combination of Large Language Models (LLM) and computer vision to detect depression in a multimodal manner, offering an automated, efficient, and scalable solution to overcome the limitations of conventional methods.
Depression is a mental health disorder with high global prevalence, significantly impacting quality of life, productivity, and increasing the risk of morbidity and mortality. Early detection is crucial, yet traditional methods such as face-to-face clinical interviews and standardized questionnaires often require substantial time, cost, and limited professional resources.
Addressing this challenge, advancements in artificial intelligence (AI) open new possibilities. This ongoing research proposes a depression detection framework that integrates three main components to deliver more accurate and reliable results compared to single-method approaches.
The three components are:
- Facial expression analysis based on Vision Transformer: This technology enables the system to analyze and interpret facial expressions, which serve as key indicators of emotional states.
- Exploratory open-ended interview based on LLM: By leveraging Large Language Models, the system conducts open-ended interviews and analyzes linguistic responses to identify patterns associated with depression.
- Patient Health Questionnaire (PHQ-9): This clinically validated questionnaire is integrated to provide comprehensive quantitative assessment.
The integration of visual, linguistic, and quantitative data enables automated, efficient, and scalable depression screening, representing a significant breakthrough in the field of mental health.
With the potential to provide more accessible and efficient screening tools, this AI-based depression detection system is expected to become a crucial solution in addressing mental health challenges. Its successful implementation could support early identification of depression cases, enable faster intervention, and ultimately improve the quality of life for many affected individuals.
