At the 2020 ESRS Congress, Nox Medical Research Manager Jon Agustsson, PhD, shared valuable, thought-provoking insights on the growing role of AI and machine learning in sleep medicine and how Nox Medical is putting this technology into practice with its new Nox BodySleep algorithm*, available in the company’s new home sleep testing device, the Nox T3s.
Before diving in, Dr. Agustsson noted that it is important for anyone using AI and machine learning technologies to be aware of their capabilities and limitations: “There is a lot of hype about AI, machine learning and big data, but there is no clear definition for what these words mean.”
Machine Learning: The Basics
Dr. Agustsson explained that AI and machine learning are typically used to describe models where the parameters are tuned by fitting known input to some known output. He provided linear regression as an example of such models, though more complex models like artificial neural networks are gaining more attention today.
Contrary to what the name suggests, Dr. Agustsson remarked that “AI models are not intelligent in any way but can instead recognize complicated patterns and map them onto an output.”
Looking at the difference between conventional programming and machine learning, Dr. Agustsson notes that in conventional programming, a programmer enters rules or a program into a computer where rules are used to process data and generate an answer. The rules can be arbitrarily complicated, but they are generated by the programmer and can, in principle, be understood by a human.
In contrast, machine learning is based on using an algorithm to infer rules by presenting it with data and answers, called supervised learning. According to Dr. Agustsson, this can be a helpful approach when it is difficult to write the rules to a program, or when the rules are unknown.
“Once we have the model in which we trust, we can use the model to predict the answers on new data where we don’t know the answers.”
Getting Started with ML
Dr. Agustsson notes that to leverage machine learning successfully, it is important to have a deep understanding of the tools, the subject matter, the data, and the questions you want to answer.
“At Nox Medical, we use machine learning to automate manual scoring of sleep studies and to develop novel analyses to apply to the studies,” he shared.
According to Dr. Agustsson, sleep staging is a great place to start building confidence in applying machine learning in sleep medicine because the task is so well defined.
“We know that we have to classify each 30-second epoch into one of five classes, we know the signals to use, humans generally agree on how to score sleep studies, there is an abundance of data available and there are many papers out there comparing how human experts agree.”
After defining the goal, Dr. Agustsson recommends collecting data which is representative of the data you expect to be used in the future.
Key Learnings
“When I started at Nox Medical five years ago, we had a machine learning algorithm in place,” said Dr. Agustsson. “The problem was, it was trained on data from people with severe sleep apnea, so these people did not have much N3 and REM sleep, so the model could not learn how to score those sleep stages. It scored almost every epoch as N2. So, the first machine learning task our group took on was to update the model to the one we have today.”
According to Dr. Agustsson, the way to develop and validate a model is to split the data into a training set, validation set, and a hidden test set, which will be used to test the model later on, “however, the true test of the model is when people start using it in the real world.”
Dr. Agustsson says to start with the state-of-the-art: the AASM Manual for the Scoring of Sleep and Associated Events, which outlines exactly how to score sleep stages. It tells us what signals to use, and what to look for in the signals.
“But anyone who has scored a sleep study knows reading the manual is not enough,” he noted. “You need a lot of practice, and this is exactly why we cannot just write the program based on the AASM manual but need to train the model to do our scoring.”
A simple place to start applying machine learning to sleep staging is to use the information provided in the AASM scoring manual to determine which features in the signals are of importance. The features are parameters that can be calculated from the signals such as the signal amplitude, signal power in the different frequency bands, or any other signal feature we can think of. This allows us to incorporate the state-of-the-art knowledge into our machine learning models and can allow us to develop high-performance models using limited data.
When we have access to larger data sets, it becomes feasible to apply a different approach called convolutional neural networks or deep learning. Convolutional neural networks are capable of learning to extract the important features from the raw signals. These networks allow us to go beyond what is described in the state-of-the-art since we do not need to handcraft descriptive features from the signals we use.
Detecting sleep from non-EEG signals
Once we’re confident in applying machine learning to sleep staging, why not use other signals such as actigraphy or breathing to classify sleep?
“Sleep stages are defined by what is seen in the EEG signals, so if we want to claim to be detecting sleep stages, I think we have to be measuring EEG; however, there are many things in the body that change during sleep, and these things can help us determine something about the sleep of a person.”
By providing more information, including information on sleep from non-EEG signals, from a home sleep apnea test (HSAT), Dr. Agustsson says we can create greater efficiencies for sleep clinics and improve patient care.
The Nox BodySleep* uses the abdomen and thorax RIP signals and actigraphy to estimate wake, non-REM and REM.
“One thing to keep in mind: since we’re using RIP signals and actigraphy, there are no rules established to verify correct scoring of the data we’re using,” Dr. Agustsson noted. Research has shown, however, that manual scoring and scoring through the Nox BodySleep* algorithm seem to agree.
To give the users of the Nox BodySleep* a better insight into the model output, we decided to present the output in a plot called the hypnodensity plot. This was inspired by work done by Prof. Emmanuel Mignot and team who presented the hypnodensity plot and showed how they could replicate the level of agreement between multiple human scorers. The output from a neural network is typically some kind of a likelihood of each of the expected output classes. In the case of the Nox BodySleep, the output is the likelihood of having wake, NREM or REM sleep states. If the model is correctly calibrated, these probabilities can represent the likelihood of a certain sleep state being present in a certain epoch. An example of this could be that the output for an epoch could be 10% wake, 80% NREM, and 10% REM, this would mean that 80% of the time an epoch with this output would truly be NREM, 10% of the time it would be wake, and 10% of the time it would be REM. The hypnodensity plot presents this information allowing the user to find areas where they should maybe look at the data and make sure the model prediction makes sense.
Ultimately, machine learning and AI technologies have tremendous potential to improve diagnostic accuracy, increase efficiency and provide cost-effective home sleep apnea testing for patients with suspicion of sleep disordered breathing.
Tune into Dr. Agustsson’s full ESRS Congress presentation below and learn more about Nox Research here.
*Nox BodySleep is not available in the U.S.
Topic: Research & Publications
