1 INTRODUCTION

Clinicians are becoming more aware of the importance of genetic testing and the impact on clinical care. However, non-geneticists have limited genetics training and many centers do not have access to genetics providers, which makes choosing the appropriate genetic test challenging (Maiese et al., 2019; Talwar et al., 2017). It is not surprising that clinicians without genetics training make genetic test order errors at a “significantly higher rate” (Mathias et al., 2016) and literature shows 8%–30% of genetic tests are ordered incorrectly (Kotzer et al., 2014; Miller et al., 2014). Ordering an inappropriate genetic test can have various untoward clinical effects including inaccurate result interpretation, inappropriate patient guidance, and even a missed diagnosis in addition to unwarranted cost expenditure (Miller et al., 2014). In addition, advancements in genetics are occurring at a rapid rate. Therefore, it is unrealistic to expect providers not well trained in genetics to fully understand the nuances in genetic testing, such as distinctions regarding which laboratory to utilize or the differential diagnosis for a presenting symptom. Genetic testing utilization is critical to ensure that each patient gets the appropriate test.

Ineffective genetic test utilization can be seen in a neonatal intensive care unit (NICU) setting, where multiple cytogenetic studies are often ordered simultaneously. A chromosomal microarray was the recommended first-line test for children with multiple congenital anomalies (Manning & Hudgins, 2010). Historically, karyotypes were ordered in this clinical scenario, but the yield of a microarray is greater than that of a karyotype, leading to microarray being the recommended test for this indication (Miller et al., 2010). Additionally, literature has shown that if a patient has a normal microarray, approximately 1% or less of patients would have an informative karyotype with some authors finding that of those with an abnormal karyotype, <0.001% would be clinically significant (Bi et al., 2013; Waggoner et al., 2018). Therefore, ordering a simultaneous chromosomal microarray and karyotype should not be routinely practiced; it is not cost effective with very minimal increased diagnostic yield. Studies have also shown that even if both tests are done sequentially, it is more cost effective to order a microarray first (Li et al., 2018). More recent guidelines suggest that exome or genome sequencing be considered as a first or second tier test in those with congenital anomalies (Manickam et al., 2021) and given that sequencing data can be used to detect copy number variation (Coutelier et al., 2022) this calls into question whether microarrays may need to be ordered at all (Miyatake et al., 2015).

We performed a preliminary investigation of cytogenetic studies at our institution to determine how providers were ordering cytogenetic studies in practice. Review of patient data from the Level IV NICU at Levine Children's Hospital in Charlotte, NC, from October 1, 2019, to October 31, 2020, indicated that 123 patients had genetic testing ordered during that period. At least two cytogenetic tests were ordered for 78% of cases. The most common combinations of tests were (1) karyotype and microarray, (2) FISH and karyotype, and (3) FISH, karyotype, and microarray. In addition, multiple orders were placed but not completed and duplicate microarrays were sent on the same patient, indicating inefficiencies in the ordering process. In total, an excess of at least 92 potentially unnecessary tests were ordered during the selected timeframe with the combination of karyotype and microarray accounting for 53% of the genetic tests ordered.

Other centers have experienced similar challenges and have shown success in reducing genetic testing redundancy, improving diagnostic yield, and decreasing cost through various measures including lab-based genetic counselors (GCs) or genetic testing utilization services (Miller et al., 2014; Riley et al., 2015; Schuler et al., 2023; Thomas et al., 2021). Often these services are run by GCs who review testing and can help advise on the appropriateness of test selection before it is sent to the laboratory, which can be particularly helpful for providers without formal genetics training. However, the creation of a genetic testing utilization service requires significant institutional commitment in terms of additional funding and personnel. In some institutions, limitations on ordering genetic testing are integrated into the electronic medical record (EMR), thereby restricting who can order certain genetic tests. Other centers have had success with algorithms that guide genetic testing recommendations (Floyd et al., 2019). The goal of these efforts is not to limit genetic testing or to be an impediment to providers but rather to ensure that the most appropriate test is chosen and sent to the correct laboratory. Our group aimed to improve genetic testing utilization in our NICU by reducing redundant or unnecessary cytogenetic testing through the implementation of an algorithmic decision support tool.

2 MATERIALS AND METHODS

2.3 Project methods

We built an interdisciplinary team from all stakeholders involved in the process including representatives from Medical Genetics, Neonatology, Pediatric Cardiology, Pediatric Cardiothoracic Surgery, pediatric residents, neonatal nursing, obstetrical nursing, laboratory, quality management, data analysis, and administration. PDSA cycles focused on the development, standardization, and implementation of a decision support tool that served as an algorithm to guide NICU team members when ordering genetic testing (Figure 1). The decision support tool was jointly developed by Genetics and Neonatology divisions with input from Pediatric Cardiology.

The initial algorithm (Figure S1) launched in February 2020. This initial iteration sought to empower NICU providers to make most genetic testing decisions themselves with genetics consultation reserved for complex cases. The algorithm focused on important reminders and common questions that the genetics team often receives. The primary reminder regarded prenatal genetic testing, as there is typically no need to duplicate if diagnostic prenatal testing was performed. The decision support tool underwent multiple revisions based on provider feedback and data review. Additional interventions included the creation of delivery room kits for the collection of cord blood samples in May 2020 and a Department of Pediatrics Grand Rounds presentation on genetic testing and appropriate utilization in December 2020.

In February 2021, we completed a more comprehensive revision of the decision support tool based upon provider feedback. The main concern with the previous iteration was that the algorithm was too complex. The Genetics team also reported that they were often consulted for cases that fell within the algorithm's guidelines. This revised version was much simpler and aimed to be more user friendly for both providers and nursing. It discouraged the simultaneous ordering of a karyotype and microarray, emphasized the common conditions (i.e., Trisomy 21) for which a karyotype alone was appropriate, recommended specific orders in the EMR, highlighted the reasons for which genetics should be consulted, and included the genetics team's contact information.

We performed another revision of the decision support tool in April 2023 (Figure 2). This revision further simplified the algorithm and updated many of the recommended orders as our institution had transitioned to a new EMR. In addition to the PDSAs and tests of change already discussed, multiple small-scale meetings targeting different subsets of providers (pediatric residents, neonatal nurse practitioners, obstetrical nursing, etc.) were held to increase awareness and provider engagement.

We evaluated each measure with run charts and SPC charts. SPC charts help differentiate between special cause variation and common cause variation. SPC charts include an average line as well as lines indicating the highest and lowest expected variation. Common cause variation is simply the normal variation in any process (i.e., status quo). Special cause variation reflects the impact of interventions or other unusual occurrences on the process (e.g., the implementation of the algorithm or changes in the EMR). There are five rules that help detect special cause variation. Rule Two (eight data points above or below the average line) informed line shifts.

3 RESULTS

Our cumulative interventions demonstrated improvement in all measures. The main outcome metric of percentage of NICU patients requiring genetic testing with only one cytogenetic test ordered improved from an average of 37% to 65%. The variation of the process is wide as marked by control limits; we attribute this to the low number of patients. As observed in Figure 3, the most effective interventions were the combination of initial algorithm, delivery room kits, and widespread education during grand rounds. The goal of 70% was met or exceeded during 24 of 50 months evaluated.

The first process measure of 50% reduction in cytogenetic testing orders with the combination of karyotype and microarray, improved from an average of five patients per month receiving both a karyotype and microarray to less than one (Figure 4). This decrease represents a 92% reduction of unnecessary tests being ordered. The earliest sign of improvement was observed after the first three interventions with additional improvement seen 8 months after the EMR was changed. The third iteration of the algorithm helped sustain the improvement and since January 2023 there have only been five patients with this combination test ordered. During the project, we observed the control limits narrowing, providing more information about greater stability of the process.

The consultation to genetics metric surpassed the 80% target and sustained at an average of 89% for the last 38 months evaluated (Figure S2).

The balancing metric of monitoring diagnostic rate showed an increase in diagnostic rate from an average of 14% to an average of 33% (Figure S3).

The balancing metric of canceled/discontinued orders did show greater variability later in the process as shown by the control limits, but the average sustained all throughout the project (Figure S4).

Lastly, we tracked the average monthly cytogenetic charges per NICU patient on whom genetic testing was performed or a consult was ordered. As shown in Figure 5, average monthly charges were reduced from $2988.59 to $1973.50 after the second iteration of the algorithm. This was sustained over 13 months and subsequently reduced to $1222.91 after the change of EMR with sustained improvement over 15 months. This represents a 59% decrease in charges. Based on the sustained performance over the last year, we estimate projected average savings of $21,000 per year.

4 DISCUSSION

As stated previously, our primary outcome measure was that 70% of NICU patients requiring genetic testing would have only one cytogenetic test ordered. Although we considered setting the goal higher, multiple cytogenetic tests are sometimes required for an accurate diagnosis and there was concern that artificially decreasing testing further could have a negative diagnostic impact. For this reason, we monitored the diagnostic rate as a balancing measure to observe any correlation between decreased testing and decreased diagnoses, although we hypothesized that more appropriate testing utilization would result in higher diagnostic yields. The fear of missed diagnoses did not materialize as we found that our diagnostic accuracy instead increased over time. During the project, we found that our primary outcome measure was the best proxy for tracking the overall testing decrease.

We had initially hypothesized that more frequent involvement of the Genetics team could help improve testing utilization and increase accuracy. However, we also monitored the number of consults for genetics providers to ensure that an increased number would not become unmanageable. By October 2020, we achieved and sustained the process measure of genetics consults for greater than 80% of patients requiring genetic testing. Shortly after, we saw our greatest improvement in the primary outcome measure following the Department of Pediatrics Grand Rounds presentation in December 2020 and the first revision of the decision support tool in February 2021. Because these two interventions were so close to each other in terms of proximity, it is difficult to clearly separate their individual impacts. After the Grand Rounds, we did observe an immediate and sustained decrease in the simultaneous ordering of both karyotype and microarray. However, there continues to be wide variation in the percentage of patients with more than one cytogenetic test ordered, though the overall improvement has been sustained. Possible explanations for the continued variability include normal variations with new team members (i.e., new NICU providers) or an increase in complex syndromic infants in the NICU. Interestingly, we did not observe any seasonality to the data, which implies that incoming classes of new pediatric resident interns rotating through the NICU were most likely not responsible for any detectable increase in testing.

Obtaining cord blood is important to both NICU and OBGYN providers at our institution. To support already established NICU protocols, cord blood collection was included within the algorithm but was not a primary focus of this project. We initially sought to track the availability of cord blood at the time of Genetics consultation. This proved to be exceptionally difficult to monitor due to differences in provider documentation and no uniform tracking function in the EMR. After only a few months, we ceased tracking cord blood availability as the difficulty with accurate data collection outweighed any possible utility. However, there were ongoing efforts to improve cord blood collection by collaborating with the OBGYN and Maternal-Fetal Medicine teams to include the use of cord blood collection kits in the algorithm. In addition, labels were created to hold cord blood at the bedside until genetic testing orders were placed by the genetics team and this was reinforced with NICU nursing to ensure patient samples were not lost or wasted.

During the conceptualization of the project, it was hypothesized that a reduction in testing would result in cost-savings. However, quantifying the actual cost of testing was found to be difficult due to two factors. First, the cost of testing sent to specialty laboratories varied, sometimes significantly. Additionally, due to differences in insurance coverage or the lack thereof, the calculated cost charged by the hospital or a specialty laboratory for genetic testing was not necessarily reflective of the cost to individual patients. However, in the setting of an overall decrease in testing, a decrease in cost for the overall health system was likely generalizable to the average patient. Given these caveats, the average monthly cytogenetic charges per patient have decreased by 59% throughout the course of the project.

Although our decision support tool did not yield immediate results as our first intervention, this could have been confounded by the timing of the algorithm's deployment, as it coincided nearly perfectly with the dawn of the novel COVID-19 pandemic. Nevertheless, it was arguably our most impactful intervention as the algorithm was refined through multiple iterations. Although the initial version was admittedly complex, we found that we achieved better results when the algorithm was simplified, primarily by removing the burden for genetic testing utilization decision-making from NICU providers and placing it on the Genetics team. This raises two points. First, while the initial complex algorithm was clearly designed for our specific tertiary care children's hospital, our final iteration was a simpler algorithm that could much more easily be adapted for use in a community hospital NICU. We recognize that many centers do not have access to geneticists and therefore inpatient genetic testing would likely not be sent from a community hospital NICU based on the most recent algorithm. However, the algorithm could help ensure that patients are referred appropriately after discharge or consideration could be given to transfer for genetics evaluation. Second, with the simplification of the algorithm placing a large burden on the inpatient Genetics team, this may become untenable in the future with further growth of our institution and health system. This led to our genetics division hiring an inpatient GC to help with the workload.

We encountered several challenges during the project. As mentioned earlier, implementing our first algorithm coincided with the COVID-19 pandemic. It is conceivable that many providers were likely distracted and not focused on genetic testing utilization during those uncertain times, which is illustrated by the variance in our data during the first few months of 2020. Additionally, we transitioned to a new EMR in May 2022. This created several issues. First, the names and orders for specific genetic tests changed within the new EMR, which created confusion as it differed from what had been codified in the algorithm. Additionally, the messaging platform for contacting the Genetics team also changed with the transition to the new EMR. These obstacles are likely reflected in the increased variation in our process measures that coincides with the initial stages of the EMR transition in January 2022. Furthermore, our team did not have access to data retrieval in the new EMR for several months before and after the transition, inhibiting our ability to evaluate data in a timely manner and quickly respond to trends. This also made planning and implementing PDSA cycles during this time difficult.

Lastly, we found that our provider education sessions were capturing a smaller percentage of NICU staff than we desired. This was a result of at least two factors, primarily the nature of shiftwork in the NICU and overnight staffing. Second, there appeared to be a fair amount of turnover within the nursing staff during this period, which possibly impacted our results as the nurses were primarily responsible for the collection of cord blood and ensuring it was held at the bedside while waiting for orders. Despite our difficulties, we received an institutional QI award for our work in August 2022, which did provide exposure and increase awareness of the project.

A limitation of this project was that we did not track all genetic testing used in our NICU and only focused on cytogenetic testing with the primary focus of karyotype and microarray. Given newer recommendations to consider exome or genome sequencing as a first or second tier test in those with multiple congenital anomalies, we often order rapid exome or genome sequencing when needed for this patient population. Nevertheless, these would have been cases where karyotype and microarray were ordered prior to our consultation of ordering exome and therefore, we suspect this project still ultimately has led to healthcare cost savings by minimizing unnecessary cytogenetic testing.

Several avenues are being considered for future expansion of this project. Spreading the project into the cardiovascular intensive care unit is the most likely next step, as cytogenetic testing is frequently performed on these infants. Secondly, given our success using a simpler version of the decision support tool, we are considering expanding the project to include community hospital NICUs within our health system. These facilities do not have in-hospital Genetics consultation and would be forced to rely on teleconsultation or transfer. This would only further increase the burden of work placed on our Genetics team. As other institutions have found success at alleviating this through the creation of a genetic testing utilization service staffed by GCs, we are currently in the beginning stages of establishing a similar program. Lastly, with the adoption of our new EMR and the establishment of a genetic testing utilization service, we are considering placing limitations or restrictions within the EMR for the ordering of genetic testing to further decrease the chances of inappropriate testing.

5 CONCLUSION

Genetic testing utilization is crucial in our current landscape with numerous genetic testing options and few genetics trained providers to help guide testing. Through our well-received QI work with interventions including a NICU genetic testing algorithm and ongoing education, we have shown a decrease in the number of cytogenetic tests ordered at a Level IV NICU in a large tertiary care children's hospital. By minimizing redundancy and inefficiencies while maintaining diagnostic accuracy, this effort has also resulted in substantial cost savings to both individual patients and the overall health system.

AUTHOR CONTRIBUTIONS

Andrew T. Jacobsmeyer: project implementation; chart review; writing – original draft; writing – review and editing. Talia L. Buitrago-Mogollon: methodology; project administration; data curation; writing – review and editing. Blanche White: data curation; writing – review and editing. Jasmyne-Rian Charles: data curation; writing – review and editing. Jessica P. Clarke-Pounder: project implementation; writing – review and editing. Jodi Amador: project implementation; writing – review and editing. Lauren B. Carter: conceptualization; methodology; chart review; project implementation; writing – review and editing.

CONFLICT OF INTEREST STATEMENT

The authors have indicated they have no potential conflicts of interest to disclose.