Gib researchers compare Covid-19 self-assessment tools, finding some miss severe cases

A study by researchers in Gibraltar has assessed the tools used by five different countries to determine whether people with Covid-19 symptoms should seek medical care.

The study examined how “symptom checkers” in Japan, Singapore, Norway, the UK and the US worked against simulated cases of Covid-19, varying in risk and severity.

The study found that in some cases, the patient-led assessment tools could potentially worsen outcomes by delaying appropriate clinical assessment.

Symptom checkers are self-assessment tools — typically online or via computer application — through which individuals enter their symptoms into a predetermined platform and from there an algorithm produces an outcome, usually advice.

The researchers from the Gibraltar Health Authority and the University of Gibraltar found that while symptom checkers from Singapore and Japan advised onward healthcare contact for the majority of simulations - 88% and 77%, respectively – those in the UK and the US only referred patients to doctors in 38% and 44% of cases.

In simple terms, the UK and US symptom checkers for Covid-19 may have guided people to stay at home when they needed to seek medical help, the study suggested.

The authors said that if the NHS 111 Covid-19 symptom checker was used as a “sole initial point of healthcare contact” it could lead to people delaying in seeking care, which could increase the risk of disease or death.

For instance, the symptom checker would instruct a 72-year-old person who has had a fever and a cough for a week to “stay at home with no clinical, nursing or healthcare contact”.

The study, published in the journal BMJ Health and Care Informatics, was authored by university researcher Fatma Mansab, Dr Sohail Bhatti, the Director of Public Health, and Dr Daniel Goyal.

National symptom checkers were introduced by many countries in a bid to reduce the burden faced by health services.
People can input symptoms – usually online or via an app – and from the information entered the algorithm produces some advice.

It is hoped these assessments would effectively spot those who would benefit from being assessed by a medic.
Conversely, patients who do not require medical care would be able to recover at home, thus easing the strain on over-stretched healthcare systems and helping to curb the spread of infection.

“The upside of symptom checkers, particularly during a pandemic is difficult to ignore,” the authors of the study said.

“By reducing physical patient contacts, symptom checkers can potentially save valuable resources and avoid further viral transmission.”

“While telephone and telemedicine triaging also protects staff and reduces transmission, such services require more healthcare staff than symptom checkers and hence carries a greater financial and human resource burden.”

But identifying patients with Covid-19 who require medical treatment is challenging.

Academics set out to examine the symptom checkers created by five different nations – Singapore, Japan, Norway, the UK (though the English symptom checker) and the US.

Researchers came up with 52 simulated cases with varying risk factors and entered them into the symptom checkers.
The authors examined how many of the cases would have been referred patients to seek help.

Singapore had the highest overall referral rate at 88% and the US had the lowest at 38%. The NHS system triaged 44% of cases to healthcare contact.

The authors said the US and English system triaged a “significant number” of cases to “stay home” that would typically have required early clinical assessment.

“The UK’s 111 Covid-19 symptom checker frequently triaged possible severe Covid-19 and bacterial pneumonia to stay at home with no follow-up and is likely to have delayed treatment for sepsis and severe Covid-19,” the authors wrote.

They added: “The low case fatality nations’ (Singapore and Japan) symptom checkers triaged in twice as many cases for direct clinical assessment than the higher case fatality nations (the US and UK).

“Of clinical concern was the failure of both the US and UK symptom checkers to triage cases simulating bacterial pneumonia, sepsis and severe Covid-19 on to any healthcare contact.”

They said the wording of some of the questions on the 111 symptom tracker encourage the self-reporting towards lower categories of illness.

“Our case simulation demonstrated that answering the moderately severe answer still triages patients to self-isolate with no healthcare contact,” they wrote.

“As such, patients with cough and fever for seven days would have to be so severely unwell that they are unable to do anything they usually do to be triaged to any clinical contact.

“Our case simulation study indicates that both the ‘111 Covid-19 symptom checker’ and the [US] ‘CDC coronavirus symptom checker’, if used as the sole initial point of healthcare contact, are likely to delay presentations of serious medical conditions to appropriate care, and as such, are likely to confer an increased risk of morbidity and mortality.

“Both symptom checkers maintain a high threshold for referring onward to clinical contact, triaging the majority of patients to stay home with no clinical contact.”

In assessing why Singapore’s symptom checker performed well, the researchers found that the online webpage was well-presented and functioned with a simple underlying algorithm.

Advice to seek medical help was triggered by age – anyone over 65 – underlying health conditions or symptoms that had persisted over four days.

Any degree of shortness of breath also triggered advice to seek medical assistance.

“The Singapore Covid-19 Symptom Checker, if used by the public, is likely to reduce healthcare contacts by the young, fit patients who are early on in the illness, thus off-loading the healthcare burden to some degree while maintaining a relatively low risk to the public,” the authors concluded.

The researchers said their study supported the recommendation that symptom checkers should be subjected to the same level of evidenced-based quality assurance as other diagnostic tests prior to implementation.

Reacting to the study, NHS Digital said in a statement: “NHS 111 online is not a diagnostic tool. It allows people to assess their symptoms and signposts to other NHS services as required.”

“This simulation study was based on an early version of the tool.”

“Over the past year it has been continually revised and updated in response to new scientific information, government guidance and public health strategies.”

EDITOR'S NOTE: Our reporters are working round-the-clock to bring you the latest news on Gibraltar and the Covid-19 crisis. All our coverage on this critical issue is available free outside the paywall. If you find it useful, please help us reach more people by sharing our journalism. And if you want to support our work further, please consider subscribing to the digital version of our daily newspaper and all our premium online content. You can subscribe via our website or for iOS devices via the iTunes store. Thank you.