AI Tool SeeMe Detects Consciousness in Patients
A new study published in Nature Communications Medicine led by neurosurgery researchers Sima Mofakham, PhD, and Chuck Mikell, MD, from the Renaissance School of Medicine (RSOM) at Stony Brook University, introduces a remarkable breakthrough in assessing consciousness in patients suffering from traumatic brain injury (TBI). This innovative research may revolutionize personalized care strategies in critical care and rehabilitation settings.
Every year, thousands of individuals with brain injuries are classified as “unresponsive” within U.S. hospitals. However, recent findings reveal that approximately one-quarter of these patients might actually be conscious but are unable to communicate this fact. This condition, termed cognitive motor dissociation (CMD), presents a significant diagnostic challenge that urgently needs addressing.
To tackle this issue, Drs. Mofakham and Mikell developed SeeMe, an AI tool specifically designed to detect covert consciousness by analyzing subtle facial movements that are otherwise imperceptible. Remarkably, their study indicates that SeeMe can identify signs of awareness four to eight days earlier than traditional clinical assessments.
This addresses a crucial concern highlighted by a landmark 2024 study published in The New England Journal of Medicine, which discovered that between 15 to 25 percent of patients deemed unresponsive in intensive care may actually retain substantial brain function. Unfortunately, current bedside tests lack the necessary sensitivity to uncover this possibility. As a result, misdiagnosis can delay critical treatments and rehabilitation for patients who might otherwise make a recovery.
Dr. Mofakham emphasized the thoughtful intent behind the creation of SeeMe: “We developed this tool to bridge the gap between patient capabilities and clinician observations. Just because a patient is unable to move or speak does not imply a lack of consciousness. Our tool reveals the hidden efforts made by patients to communicate their awareness.”
In a clinical study involving 37 patients with acute brain injury and coma, SeeMe captured high-resolution video and utilized computer vision to assess involuntary facial reactions to verbal prompts such as “open your eyes” or “show me a smile.” These subtle responses, typically undetectable by medical personnel, were analyzed through machine learning techniques. Intriguingly, in many cases, SeeMe was able to detect meaningful movements several days before clinicians observed physical responses.
This kind of innovation demonstrates that the future of medicine lies at the intersection of distinct disciplines, where applications of AI and engineering can yield significant improvements in medical practice. Dr. Mofakham underlined this point: “We aim to transform intricate data into practical tools enabling doctors to make informed judgments when every moment counts.”
Importantly, patients who exhibited early responses as detected by SeeMe were found to have higher chances of regaining consciousness and achieving improved functional outcomes by the time of discharge.
Dr. Mikell described this tool as more than a mere diagnostic instrument; it could serve as a prognostic marker. He stated, “Families often inquire about the likelihood of their loved ones waking up. This study equips us with data-driven insights to address those questions with enhanced confidence, rather than relying solely on intuitive judgment. This information empowers us to tailor care, inform families, and optimize rehabilitation efforts effectively.”
The study’s authors also point to significant ethical implications regarding TBI patients and their potential recovery. Mislabeling patients as unresponsive can lead to premature withdrawal of care, limited neurorehabilitation access, and missed therapeutic opportunities.
The 2024 study underscored the pressing need for reliable tools to detect CMD at the bedside—tools that do not require costly imaging or invasive tests. SeeMe represents a viable solution, being noninvasive, affordable, and scalable. It requires only a standard camera and open-source software, making it suited for use even in under-resourced hospitals and ICUs.
As the team embarks on larger clinical trials and seeks regulatory approval, they envision incorporating SeeMe into standard ICU practice, integrating it alongside EEG and other data streams to establish a comprehensive consciousness-monitoring platform. In this regard, SeeMe illustrates a powerful opportunity for AI to enable communication for patients who are silent yet still conscious.
This research was made possible through multiple institutional seed grants that foster collaboration between the Departments of Neurosurgery and Electrical and Computer Engineering at Stony Brook University.
For more information on the extensive neuroscience research surrounding consciousness conducted at RSOM, visit the Mofakham Mikell Laboratory.
Source:
Stony Brook University
Journal reference:
Cheng, X., et al. (2025). Computer vision detects covert voluntary facial movements in unresponsive brain injury patients. Communications Medicine. doi.org/10.1038/s43856-025-01042-y