1 INTRODUCTION

In 2021, the World Health Organization (WHO) first issued the recommendation that countries should ensure regular DNA-based testing for human papillomavirus (HPV) to identify women who have or are at risk of cervical precancer.^{1, 2} The last decade has therefore seen a move toward primary HPV screening, with countries either shifting from existing programs based on cytology or visual inspection with acetic acid (VIA) or looking for sustainable solutions to implement HPV screening ex novo.³ As each of the 13 high-risk HPV (hrHPV) genotypes (types 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59 and 68) carries a different risk of progression to cervical cancer,^{4, 5} the addition of HPV genotype to triage HPV-positive women has been well studied with substantial prospective epidemiologic support. Positivity for HPV16/18 is currently being included in clinical algorithms (eg, US, the Netherlands, Italy) as an indication of high-risk supporting direct referral for colposcopy^{2, 6} and the inclusion of extended genotyping is being piloted (eg, Sweden).⁷ Extended or full genotyping refer, respectively, to the ability of the HPV test to identify some or all the hrHPV types other than 16/18 in groups or individually. However, most commercial HPV assays (i) yield a pooled result for types other than 16/18 (ie, partial genotyping), neglecting differences in risk between HPV16-related alpha-9 types and other hrHPV types, (ii) include HPV66 or HPV53, whose classification has been rectified by IARC as extremely rarely carcinogenic to humans,⁵ (iii) do not permit testing a large or flexible number of samples per run or (iv) are too expensive to be used in large-scale screening settings, especially in lower resource settings.

These limitations highlighted the need for more affordable and feasible extended genotyping tests. Extensive evidence supports categorizing HPV types globally into four groups reflecting the risk of progression to cervical precancer: HPV16, HPV18/45, HPV31/33/35/52/58 and HPV39/51/56/59/68.^{8, 9} Slightly different categorizations of genotypes have also been supported.^{10, 11} Given that new isothermal amplification techniques can lower the complexity of HPV testing without compromising sensitivity, HPV researchers from the US National Cancer Institute (NCI) collaborated, in the interest of public health, to help redesign a 15-type isothermal amplification assay (AmpFire) into a single-step 13-type assay (ScreenFire) that aims to target the four type groups in a hierarchical primer design: HPV16 is targeted as the highest design priority, else HPV18/45, else HPV31/33/35/52/58 and else HPV39/51/56/59/68.¹² The new version of the assay is meant to be a screening test that also serves as a risk-stratification test, thus allowing primary HPV testing and management of HPV-positive women by genotyping. The genotype channels are ordered such that the result supports clinical management recommendations in cervical cancer screening. These efforts were undertaken to help identify who, among HPV-positive women, needs more attention and to create more precise screening and management recommendations depending on established cancer risk.^{13, 14}

The aim of the present study is to evaluate the clinical accuracy of the novel HPV assay using a disease enriched dataset and to demonstrate the rationale for a risk-based HPV test.

2 METHODS

3 RESULTS

The characteristics of the study population consisting of 2076 women, including the histologic and virologic reference standards used for the evaluation of ScreenFire HPV test, are displayed in Table 1. The mean age was 30.9 years and most women were below age 50. The sample included 256 cervical cancer cases (43 adenocarcinomas and 213 squamous cell carcinomas), 350 CIN3 (13 AIS and 337 CIN3), 409 CIN2 and 1020 < CIN2.

The hierarchical HPV results by ScreenFire are shown in Table 2 stratified by histology and in Table 3 stratified by the histologic and virologic reference standards. The concordance of ScreenFire by the virologic reference standard was high for all type groups and higher for cancer, CIN3, and CIN2 than for <CIN2. The highest priority cases were those with HPV16. Among HPV16-positives by the virologic reference standard, ScreenFire detected HPV16 in 140 out of 143 cancers, in 236 out of 238 CIN3, in 177 out of 185 CIN2, and in 180 out of 219 < CIN2. ScreenFire missed no HPV16 cancer case (ie, 3 cancers HPV16-positive by the virologic reference standard and negative for ScreenFire HPV16 were positive for other ScreenFire channels) and picked up HPV16 in five additional cancers missed by the reference HPV tests (ie, hrHPV negative by the virologic reference standard and HPV16-positive by ScreenFire). Although ScreenFire was designed to have highest sensitivity for HPV16, the sensitivity of the HPV18/45 channel, tabulated in HPV16-negative specimens, was virtually as high for CIN2+ (Table 3). The other channels were similarly very sensitive among women with CIN2+. There was, as expected, more variation in channel positivity among women with <CIN2, who are established to have lower viral loads than those with CIN2+ for HPV16 and the HPV16-related types.²⁶ Overall, Table 3 shows a strong association of the hierarchical approach with the histologic diagnosis. HPV16 with precancer and cancer were the highest priority and, moving from the highest priority HPV16 channel to the lower risk channel, a lower number of precancer and cancer cases were observed.

Similar results were obtained when repeating the analysis after excluding equivocal positive results and after restricting to single-type infections by the virologic reference standard. Moreover, the level of agreement between ScreenFire and the virologic reference standard was excellent when the analysis was restricted to women aged 30 to 49 years, the main targeted screening age. The concordance of ScreenFire with the virologic reference standard was high also when the HPV results were analyzed non-hierarchically, both for each type group (Table S1) and for each individual type (Table S2).

The overall clinical sensitivity of the three HPV assays, also stratified by age group, is displayed in Table 4. For all assays, positivity was higher for CIN2+ than for <CIN2, and among women with CIN2+, for endpoints CIN2 and CIN3 (the best-established screening target) than for cancers. A subset of cases, particular adenocarcinomas, were negative for all assays. For endpoint CIN3+, the sensitivity was 94.7% (92.6-96.4) for ScreenFire, 92.3% (89.7-94.3) for Linear Array, and 96.0% (93.9-97.6) for TypeSeq. The CIN3+ sensitivity was higher for younger women (<30 years: 98.7% [96.2-99.7] for ScreenFire, 97.8% [95.0-99.3] for Linear Array, and 98.2% [95.4-99.5] for TypeSeq; 30 to 49 years: 96.0% [92.5-98.2] for ScreenFire, 93.3% [89.2-96.2] for Linear Array, and 96.9% [93.4-98.9] for TypeSeq) than for women >49 years of age (84.1% [76.0-90.3] for ScreenFire, 78.8% [70.1-85.9] for Linear Array and 89.1% [80.9-94.7] for TypeSeq).

We performed ancillary analyses on time-to-positive by ScreenFire. Figure S1 depicts the distribution of the time-to-positive by ScreenFire (non-hierarchical) separately for each of the 13 hrHPV types as detected by the virologic reference standard. Most HPV infections detected by ScreenFire gave a positive result within 30 min. The infections detected after 30 min were mostly with types HPV51 or HPV59. Higher odds of having a CIN2+ vs <CIN2 were seen for fast time-to-positive than for medium or slow time-to-positive in the three higher risk channels (Table S3). A markedly increased odds of CIN2+ was seen for the HPV16 channel, especially when fast-positive (high load). Lower odds of CIN2+ were seen for the lowest risk channel found alone as compared with the others, regardless of time-to-positive.

4 DISCUSSION

In the cross-sectional study presented here, we showed excellent agreement of ScreenFire HPV test designed for risk-based screening with two research reference typing tests that NCI has depended on for epidemiologic work on type-specific HPV natural history. The test population was a colposcopy clinic, highly enriched for cases of cervical cancer/precancer, and the comparison was based on stored provider-collected specimens. Most importantly, the read-out of the assay is redesigned for risk-based screening. The channels are read as hierarchical under multiple channel positivity, stressing the importance of types that cause most cervical cancers worldwide.

The agreement between ScreenFire, Linear Array and TypeSeq was excellent, for each of the four type groups defined by ScreenFire as well as for each of the 13 hrHPV types individually. ScreenFire had an overall clinical sensitivity for CIN3+ of 94.7%, similar to that of the other two tests. When the analyses were repeated in the subset of women aged 30 to 49 years, ScreenFire performed even better (96.0% overall clinical sensitivity for CIN3+), indicating that the newly redesigned HPV assay would work very well in screening settings. Our ancillary analysis on the time-to-positive by ScreenFire revealed markedly increased odds of CIN2+ for the HPV16 channel, especially when fast-positive (high load), and lower odds of CIN2+ for the lowest risk channel found alone regardless of time-to-positive. In other words, in particular for HPV16, time-to-positive was shorter in women with precancer. This suggests that time-to-positive could be used in combination with HPV genotyping to evaluate the risk of developing cervical precancer, but further evaluations are needed to support this indication.

We helped redesign ScreenFire to fit a risk-based screening strategy in low-resource settings, thus for use as primary HPV test on self-collected samples also allowing clinical management by genotyping. The reasoning behind this can be explained by giving more details on the hierarchical approach. The hierarchical approach presented in this article was designed to manage the reality that all HPV typing tests have at least a slight amount of error in type assignment. Slight differences in sensitivity by type are typically seen, and at least slight type-type competition occurs when multiple types are present. ScreenFire was designed by primer optimization to favor detection of the HPV types that cause most of the cervical cancer cases worldwide and particularly to detect HPV 16, 18 and 45. This means that the reading of ScreenFire HPV results should follow the hierarchical design (HPV16, else HPV18/45, else HPV31/33/35/52/58, else HPV39/51/56/59/68). This approach has clear consequences; hierarchical reading aims to direct the clinical management based on the risk channel. Subject to the type of triage under use and the follow-up capacity, positivity for HPV16 confers a much higher risk of progression to cancer and therefore closer monitoring is warranted as compared to a positive result for the HPV39/51/56/59/68 channel. Similarly, efforts to detect HPV18 have omitted the high risk of progression of HPV45, particularly for adenocarcinomas. Adding both types at the same level of risk is consistent with prospective data. In a sub-analysis presented in this article, the performance of ScreenFire was evaluated at the type-level and shown to be very good. Nevertheless, ScreenFire is not meant to be used as a genotyping test (apart from HPV16) but rather as a risk-stratification screening test. The risk of HPV infections is known to be measured best by the most carcinogenic type present. Multiple infections do not convey additive risk; in fact, HPV16 found alone is the highest risk result.²⁷ When used hierarchically, the four HPV type groups reflect different cervical cancer risks, thus they can be used to manage HPV-positive women accordingly giving always maximum risk level to HPV16 and the lowest (not null) to the channel detecting HPV39/51/56/59/68. Risk-based screening strategies, including the use of HPV genotyping, are gaining importance worldwide because they offer the possibility to maximize screening efficiency by better allocating available resources and directing them to women at highest-risk. A risk-based strategy combining extended genotyping and age is being piloted in Sweden⁷ and it is likely that more countries will adopt similar strategies as, in the near future, more HPV tests used for primary screening will also provide extended/full genotyping information without additional costs.

This independent evaluation of a commercial product supports the following conclusions. As compared to the currently available HPV typing tests, ScreenFire (i) provides extended genotyping sufficient for screening, (ii) focuses on the most carcinogenic hrHPV types without including HPV66 or HPV53, (iii) can process a 96-well plate without requiring batch testing (1 to 96 samples/controls without waste of reagents) and (iv) is relatively low-cost (6 USD per test not including the collection device, laboratory supplies, or labor).^{28, 29} Compared to many typing tests, the assay is simpler to run requiring basic pipetting skills for sample preparation, is rapid (60 min run time plus approximately 60 min of hands-on preparation time, depending on experience, for 96 samples/controls), and does not require DNA purification or collection medium,³⁰ which makes it compatible with self-sampled swabs transported dry to the laboratory. Additionally, reagents and equipment are low-volume making them easy to ship. Finally, the testing platform is suitable for other pathogens (eg, other sexually transmitted infections, COVID-19).^{28, 29} As for the assay's limitations, 6 USD per test is still expensive for some settings, pipetting skills are required, along with careful laboratory technique. ScreenFire is a table-top molecular test, thus an adequately clean environment is still needed and contamination is still possible especially with post-amplification products if technique is flawed and the seals or disposal are compromised. Efforts are ongoing to reduce the number of pipetting steps and the risk of contamination. For example, the use of reaction tube strips with pre-aliquoted reagents instead of an empty 96-well plate could be a technical solution to both those limitations.

Our study has two important limitations with regards to the study population and type of samples used. First, the study sample was a colposcopy referral population and not a general screening one. We chose this population with a high proportion of confirmed precancer and multiple infections because we were interested to test for sensitivity, also among women living with HIV who are known to have multiple infections. The study sample also had an age range wider than the targeted screening age. We repeated the analyses by age groups and showed that ScreenFire had an excellent performance in the age group 30 to 49 years, which is the main targeted screening age. Second, this study was done using provider-collected cervical samples and not self-collected vaginal samples. By using a specific protocol for dry swab and ensuring best practice of vaginal self-collection, we expect ScreenFire to perform well on self-collected material, but sensitivity could be an issue given the small volume of specimens assayed. Additionally, ScreenFire was run on a residual cell pellet while Linear Array and TypeSeq were performed on the same extracted DNA, thus, technically, the relative amounts of HPV could have been different. If that was the case, a bias in sensitivity could have been possible, but the three assays showed very high similar sensitivities.

Taking these limitations into account, this study does not provide validation of the assay for cervical cancer screening by self-sampling. This cross-sectional comparison shows that the assay has sufficient accuracy to move forward with it. ScreenFire is currently being used in nine low- and middle-income countries (LMICs), as part of a large study that we are conducting, called PAVE, aiming at validating a “screen-triage-treat” strategy comprising HPV genotyping of self-samples and artificial intelligence (AI)-assisted visual evaluation (AVE) on 100 000 women.³¹ The PAVE study will provide absolute risk (positive and negative predictive values) estimates for every test result (channel and time-to-positive as a measure of viral load), and show what the yield of precancer/cancer is for every result, alone and combined with visual assessment. We will validate the assay on self-collected samples and conduct extensive evaluations, including but not limited to: optimization of the workflow, optimization of the cutpoints to define how to guarantee sensitivity and specificity yielding good risk stratification, repeatability experiments, evaluation of the assay usability in LMIC settings. By the end of 2024, we will be able to confirm the ability of ScreenFire to be used as screening and management HPV typing test in real-world screening settings. Additionally, in the interest of making a screening test available for public health use, the data from PAVE, if convincing, will be submitted to scientific and regulatory bodies for formal consideration along with other trial data as required for approval and implementation. For the present, we can conclude that ScreenFire has the potential to bring an accurate HPV screening test to an increased number of women, irrespective of geography.

In conclusion, ScreenFire showed excellent agreement with other HPV assays and high sensitivity for each of the types individually and grouped. The risk-based type groups can be managed hierarchically for cancer risk allocation, thus facilitating clinical management of HPV-positive women. The potential role of time-to-positive as a risk stratifier in combination with genotyping should be further evaluated. ScreenFire could support implementation of HPV-based screening and management, particularly in low-resource settings, if it proves to be equally accurate and practically effective in real-life screening by self-sampling.

AUTHOR CONTRIBUTIONS

The study reported in the article has been performed by the authors, unless clearly specified in the text. Study conceptualization and supervision was carried out by Silvia de Sanjosé, Nicolas Wentzensen, Joel M. Palefsky, Mark Schiffman. Sample collection and clinical characterization was done by Rosemary E. Zuna, Joan L. Walker, Nicolas Wentzensen, Mark Schiffman. HPV testing was performed by Casey Dagnall, Amanda Hoffman, Sepideh Farhat Nozzari. Statistical, epidemiologic and/or clinical expertise was provided by all coauthors. Data analyses and writing of the original draft were done by Federica Inturrisi. The data were accessed and verified by Brian Befano and Mark Schiffman. The manuscript was reviewed and edited by all coauthors.

FUNDING INFORMATION

The study was funded by the NCI Intramural Research Program. NCI chose the specimen collection and the two laboratories for the ScreenFire HPV testing (NCI-CGR and UCSF). NCI invited UCSF to test half to rule out laboratory variability due to the NCI laboratory having previously tested these same specimens. The reported data were masked to prior results at NCI, and the half tested at UCSF were totally masked and previously unknown to that laboratory. UCSF was provided with equipment and reagents by Atila Biosystems for the sole purpose of this study. The authors declare no intellectual property or conflict of interest with regard to the manufacturer, whose scientists had no input into the masked testing, analysis, or reporting of the results.

CONFLICT OF INTEREST STATEMENT

Joel M. Palefsky reports institutional grant support from Merck & Co., Roche Diagnostics, Antiva Biosciences, Vir Biotechnologies and Virion Therapeutics, as well as honoraria for attending meetings, giving lectures, presentations, consultations from Merck & Co., Roche Diagnostics, Antiva Biosciences, Vir Biotechnologies, Gilead Pharmaceuticals and Janssen Pharmaceuticals. He has stock or stock options in Virion Therapeutics and he has received resources/services from Atila Biosystems. All other authors declare no competing interests.

ETHICS STATEMENT

Written informed consent was obtained from all women enrolled into the SUCCEED study and Institutional Review Board approval was provided by OUHSC and the NCI.