Diagnostic accuracy of thirteen COVID-19 (SARS-CoV-2) rapid antigen self-tests with very high sensitivity approved for home use in Australia

We are concerned that the approved rapid antigen tests (RATs) used in Australia to detect SARS-CoV-2, which causes COVID-19 infection, are being used for a purpose for which they were not intended. We are also concerned that there is a lack of understanding of the role that disease prevalence has on the accuracy of the results of screening tests.

The Australian Government Department of Health Therapeutic Goods Administration (TGA) had listed 32 approved rapid antigen self-tests (RATs) (some of which appear to be duplicates) for home use by the end of February 2022.¹ Information on different tests for COVID-19 and how they help understand infectiousness is provided by the United States Food and Drug Association (FDA).² The Australian TGA has provided information on the clinical sensitivity of each test and identified 13 of these as having ‘very high’ sensitivity (i.e. >95%). Level of sensitivity is said to have been based on the proportion of individuals that produced a positive test result using a COVID-19 RAT, in comparison with a positive result that was obtained using a more sensitive laboratory polymerase chain reaction (PCR) test. However, it is notable that no information is provided on positive and negative predictive values, either by the TGA or any of the manufacturers. Such information would be necessary to provide estimates of diagnostic accuracy under conditions of differing prevalence or pre-test probability rates. We acknowledge that the first stage in accuracy studies is to determine test sensitivity and specificity because this indicates how well the test works with known positive and negative samples. We also note from the manufacturers’ documentation the intended use of RATs is to detect SARS-CoV-2, which causes COVID-19, with specimens collected from symptomatic individuals who are suspected of being infected with COVID-19 within the first seven days of symptom onset. Nevertheless, RATs are being used in Australia by asymptomatic individuals with no known epidemiological links to COVID-19 such as, for example, those wishing to travel, enter educational facilities, visit aged care residences, attend family gatherings or return to places of employment. These asymptomatic individuals have been led to believe a one-off screening test with a RAT will accurately identify SARS-CoV-2, which causes COVID-19 infection.

We analysed the top 13 tests utilising Bayes’ Rule with the sensitivity and specificity data provided by the test manufacturers to obtain estimates of the positive and negative predictive value of their tests, assuming the prevalence of active COVID-19 cases at both 1% and 3% in the community. There are differing methods of calculating prevalence rates depending upon the unique circumstances of the person being tested.³ As we were interested in the interpretation of a RAT result for a random fully vaccinated asymptomatic individual with no close contacts or other risk factors for COVID-19, we estimated the population point prevalence rate at the peak of the Omicron wave in Australia in mid-January 2022 to have been around 3%. This was based on the total number of active cases at that time as a percentage of the population.⁴ As of the beginning of March 2022 the estimated point prevalence decreased to below 1% (allowance for under-reporting would likely be necessary).

There were marked discrepancies between the 13 “very high” sensitivity tests, with positive predictive values ranging from 38% to 100% when prevalence is at 1% and from 65% to 100% when prevalence is at 3%.⁵ Figure 1 shows the positive predictive values and numerous instances of wide 95% confidence intervals. Four of the RATs were reported by the manufacturers to have had zero false positives, which produced positive predictive values of 100%. Negative predictive values for all 13 tests at both 1% and 3% prevalence were uniformly high at 99%.

The numerous low positive predictive values and wide confidence intervals present a challenge to accurate test interpretation on a one-off basis for home use. This would be particularly true for low-risk asymptomatic individuals who are fully vaccinated, with no history of close contacts or other risk factors for COVID-19. A review and meta-analysis of a large number of international independent studies showed the pooled sensitivity and specificity values for RATs from asymptomatic samples to be 56.8% and 99.1% respectively.⁶ Using the metrics from this meta-analysis would produce positive predictive values that are considerably lower than those we have calculated. This would suggest that the metrics provided by the manufacturers are likely to be over-estimates. It is also unlikely that in real-world conditions RATs would produce zero false positives.

We support the existing recommendation for using imperfect tests to make clinical decisions.⁷ In addition, careful consideration of confidence intervals is essential. RATs that have positive predictive values at or below chance, or confidence intervals that include chance level, should not be relied upon for one-off testing in low-risk individuals. Chance in this context is a posterior probability of 50%, that is, only a 50% chance of being SARS-CoV-2 positive when obtaining a positive RAT result.

There is clear evidence of vastly differing diagnostic accuracy in the 13 approved RATs with “very high” sensitivity. It is likely that many of those lower ranked tests that the TGA has deemed as having “high” sensitivity and those having “acceptable” sensitivity will be found to have unacceptably low diagnostic accuracy. Government and health officials as well as the lay public who will be using these tests, even when used for the purpose for which they were intended, especially under conditions of low prevalence, should be cognisant of the nuances in their interpretation.