ABOVE: © ISTOCK.COM,
ERGIN YALCIN
When Joe Brew worked for the Florida Department of Health as an epidemiologist for two years starting in 2013, he helped with syndromic surveillance, meaning he had the arduous job of reviewing the symptoms of patients coming into the emergency departments from all across the state. The goal of such work: to detect an abnormal spike of symptoms in an area that may indicate there’s a public health concern.
Public health authorities worldwide continue to use this type of surveillance. The outbreak of a novel pathogen in Wuhan, China in late 2019, for instance, was in part detected by a large uptick of patients coming to the hospital with symptoms of a respiratory infection, with unknown etiology. But Brew says this system fails to prevent the transmission of a virus like SARS-CoV-2 because by the time patients arrive at the hospital, they have likely already been infectious for a matter of days. COVID-19 tests, too, often fail to return a result in time for patients to properly isolate while they’re infectious.
This realization led Brew to turn to a device that billions have in their pockets—a smartphone—to provide public health authorities with real-time symptomatic data from the community. Brew and several colleagues founded Hyfe, a free phone application that uses artificial intelligence to detect and track users’ coughs, a hallmark of many respiratory conditions including COVID-19.
Cough has long been a symptom that physicians record, yet the method for monitoring it is typically limited to a self-report during a clinic visit.
“The way you beat COVID is by acting fast—by being ahead of it,” says Brew, who is now the CEO of Hyfe. “Those places that very quickly identified clusters and outbreaks were able to shut things down and basically control the pandemic even without a vaccine.”
People who are curious to monitor their cough frequency trends, say, if they have a respiratory condition or want to share these data with loved ones or medical professionals, can download Hyfe onto their smartphones. When the application hears a loud, abrupt noise, such as a cough, it captures that approximately half-second snippet of sound and converts it into a 3-D image called a spectrogram that represents the pitch and intensity of the sound over time. The spectrogram is then processed through a machine learning algorithm known as a convolutional neural network, which has been trained on a dataset of more than 270,000 sounds—a cough, laugh, grunt, burp, or a fork hitting a plate, for example—that two human listeners have labeled as a cough or not a cough that help the algorithm determine whether the abrupt noise was, indeed, a cough. The app also learns an individual’s unique cough, so if the app detects another person’s cough that significantly deviates from their previous coughs, it won’t log it.
Brew and a team of scientists aim to take this concept into the field to see if they can detect an outbreak of an infectious respiratory pathogen. Their pilot project will examine the cough data at a community-wide level in a small municipality on the outskirts of Pamplona, Spain.
“There’s a lot of people thinking about diagnostics, but everyone is thinking individually,” ISGlobal Barcelona Institute for Global Health epidemiologist Carlos Chaccour, who is leading the study, tells The Scientist. “But so far, the community perspective has not been pursued.”
While Brew admits that there could be many non–infectious disease causes for a cough—air quality, asthma, allergies, fumes from cooking, to name a few—he says the lack of specificity is a feature not a bug. In the case of wastewater surveillance, which is another tool for public health officials to use to indirectly detect and monitor the spread of infectious disease, “you’re already looking for the virus, and you’re already in an epidemic situation,” he says, whereas Hyfe could, in theory, detect a surprise flare-up from an novel disease before it progresses to that point.
The great thing about this cough monitoring is it will be passive—the patient won’t actually have to do anything.
—Alyn Morice, Hull York Medical School
Chaccour and his team have enrolled more than 60 people in the community so far and aim for as many as 500 to test the concept. As part of the study, participants grant researchers access to their hospital records and their Hyfe data to determine if a rise from a baseline level of coughing among the participants correlates to more diagnoses of respiratory conditions, including COVID-19.
If Hyfe can successfully demonstrate that its detection of a higher community incidence in coughing precedes more respiratory diagnoses in the clinic, Chaccour says, he envisions users could then view a heat map of anonymized data showing which communities have the highest prevalence of coughing—a tool that could come in handy for public health officials but also people hoping to understand the risk of infection in a community.
A missed opportunity
Cough has long been a symptom that physicians record, yet the method for monitoring it is typically limited to a self-report during a clinic visit. Previous research suggests that patients often underreport how much they cough, which has led epidemiologists such as Brew to think that there is untapped potential in using cough data.
Alyn Morice, who is the head of respiratory medicine at Hull York Medical School in the UK and specializes in the diagnosis and treatment of cough, says that patients answering questionnaires are entirely unreliable. He’s even seen patients fabricate data from peak flow meters, portable devices that patients use at home to measure how efficiently air flows through their lungs.
“The great thing about this cough monitoring is it will be passive—the patient won’t actually have to do anything,” he says.
University of Washington infectious disease expert Peter Small, who has advised Hyfe and provided seed funding to the firm when he was the senior health director of global health technologies at Global Good, tells The Scientist he is optimistic about this new frontier of technology, particularly in the context of eradicating tuberculosis. “Patients seek care very late in the disease and part of that is because we, as a society, tend to ignore cough in adults,” he says.
With the help of AI-assisted cough tracking, he envisions a world in which users who have been coughing at a higher rate than normal for, say, two weeks would receive a text notifying them of their symptoms with directions to a public clinic that can test them for tuberculosis.
Even in patients with a confirmed TB diagnosis, Small says, the technology could show patients’ recovery progress. “I’ve been around a lot of TB patients and it’s a very disconcerting diagnosis,” he says. “Even though it’s almost always curable, it’s psychologically difficult on patients, and having objective evidence that their cough is getting better can help with their spirits.”
Cough counting would have been interesting in 1990, but right now we have some five billion humans carrying a microphone with them at all times every day, everywhere.
—Joe Brew, Hyfe
The Hyfe group is not the only one working to integrate cough into a more effective public health tool. Morice, for instance, developed an alert system using cough tracking through an external monitor users wear around their necks for impending chronic obstructive pulmonary disease (COPD) exacerbations—severe episodes that can often lead to hospitalization. His research team detected 45 percent of these flare-ups an average of four days prior to diagnosis, according to data they presented at this year’s European Respiratory Society virtual conference. With early intervention, patients who take steroids or bronchodilators can prevent or lessen the severity of these exacerbations, Morice says.
“If you’re able to prevent hospital admissions in these folks, it’s much better for the patient but it’s also much better for the health economy because [treating the] exacerbation is an expensive thing,” he says.
An MIT group tried to develop a tool to determine if it’s possible to identify a COVID-19–specific cough. The researchers processed more than 70,000 forced-cough audio samples, of which 2,660 were submitted by people with COVID-19, through a machine learning algorithm, which they claim accurately identifies 98.5 percent of coughs from people who were confirmed to have COVID-19, including 100 percent of forced coughs from those who were asymptomatic, according to their paper published in October in IEEE Journal of Engineering in Medicine and Biology.
In other endeavors, researchers are soliciting healthy and COVID-19–infected individuals to help train their AI models to eventually allow users to understand if they have the virus based on their cough. These projects include a Bill and Melinda Gates Foundation–funded initiative, Cough Against Covid, at the Wadhwani Institute for Artificial Intelligence in Mumbai, the University of Cambridge’s COVID-19 Sounds project, and the Coughvid project at the Swiss Federal Institute of Technology Lausanne.
Morice remains skeptical of apps that claim they can diagnose users’ coughs: “Frankly, I don’t believe them. You can tell a wet cough from a dry cough, but that’s about it from the cough sounds.” Several coauthors of the MIT study declined requests for an interview to discuss their work.
Brew says he not only wants to better understand the acoustic signature of different ailments, but also track the diurnal pattern of cough—do people with COVID-19, for example, tend to cough more during a certain part of the day or night? “When do they begin coughing? Does a change in cough frequency indicate a certain prognosis?” Brew asks. “These are super basic questions that no one really knows [the answer to] at this point.”
Privacy concerns of recording audio
More than 40 countries and 21 US states and territories use official state-sponsored COVID-19 applications that aid contact tracers in stemming the spread of the virus. Despite the promise of these apps, few people in the US have downloaded them, in part because users worry about handing over their detailed location history to their government, says Chaccour.
Any app monitoring cough would also require permission to record audio through users’ smartphones and, in Hyfe’s case, track their location to measure cough at a community level. Brew says he hopes to reassure users that the app would only record roughly half-second snippets following an abrupt noise. Still, he says he thinks there needs to be some value offered back to the user. One idea is to design a dashboard of users’ personal data akin to what Fitbit does with step counts. “Nobody cared about step counts 15 years ago until Fitbit made it trendy,” he says.
To Chaccour, if the technology proves useful, the end goal is not to provide it to governments, but perhaps to third-party companies such as Apple or Google, which can integrate it into their phone operating systems. He’s noticed that in Spain, people aren’t very trustful of the official government COVID-19 app, but they don’t mind their phones listening for users to summon the voice assistant or tracking how much they’ve slept.
Brew says he thinks the present moment is a perfect opportunity to roll out the technology. People care about public health and “cough counting would have been interesting in 1990, but right now we have some five billion humans carrying a microphone with them at all times every day, everywhere.”
Correction (December 9): The article has been updated to note Peter Small's relationships to Hyfe. The Scientist regrets the error.
