2.1 Geographic and taxonomic scope

All assays were validated for use throughout the continental United States (Figure 1), including previously published assays that were initially developed for narrower geographic applications. Nontarget taxa were defined as all freshwater and terrestrial confamilial species potentially occurring within this area, along with any sympatric extrafamilial taxa that were identified as potentially cross-amplifying via a Primer-BLAST search of GenBank and also had eDNAssay assignment probabilities ≥0.3 (as described below). We referred to primary literature and online databases for taxonomic assignments, including FishBase (https://www.fishbase.se) and the Integrated Taxonomic Information System (ITIS; https://www.itis.gov). State-level occurrence data were obtained from NatureServe (https://explorer.natureserve.org) and categorized into USFWS regions using a custom script in R (version 4.0.4; R Core Team, 2021). Note that regional borders coincide with state borders everywhere except the Klamath River Basin in southern Oregon and the Sheldon National Wildlife Refuge in northern Nevada; given our approach, these areas were assigned to Regions 1 and 8, respectively. Additional occurrence data were obtained from primary literature and online databases such as the Global Biodiversity Information Facility (https://www.gbif.org) and the Nonindigenous Aquatic Species database (https://nas.er.usgs.gov) as needed for complete documentation.

2.2 Assay design and optimization

Of the 46 total assays described here, 32 were newly developed, 12 were previously published, and two were previously published but modified to improve specificity (Table 1). Our newly developed assays were designed using one of two approaches. Three newly developed assays—green sunfish, channel catfish, and lake trout (see Table 1 for Latin names)—were designed prior to the availability of accurate in silico testing methods and followed a conventional, mismatch-based design process (sensu Wilcox et al., 2015). Specifically: (1) target and nontarget sequences were downloaded from GenBank (Sayers et al., 2019) using a custom script in R with package rentrez (Winter, 2017) and aligned using the ClustalW (Thompson et al., 1994) and MUSCLE (Edgar, 2004) algorithms in MEGA (version 11; Tamura et al., 2021); (2) candidate primer pairs were generated using the R package DECIPHER (Wright et al., 2012); (3) probes were inserted between primers in regions where target sequences were most conserved and divergent from those of nontarget taxa; and (4) sequences were modified and evaluated iteratively using base-pair mismatch counts to maximize generality and specificity. The design of the remaining 26 newly developed assays relied heavily on the eDNAssay classifier (Kronenberger et al., 2022), a machine-learning-based PCR model that accurately predicts qPCR cross-amplification using sequence alignment. All newly developed assays were located within the most commonly sequenced mitochondrial genes (COI and Cytb) to allow for more comprehensive in silico testing. For these assays: (1) sequences were downloaded and aligned as above, (2) alignments were visually inspected to identify areas where target sequences were most conserved and divergent from those of nontarget taxa, and (3) candidate primer and probe sequences were added, modified, and evaluated iteratively using eDNAssay to maximize generality and specificity. The best-performing assays were selected for further testing. Melting temperatures were determined using Primer Express 3.0.1 (Life Technologies). Primer melting temperatures were kept between 56 and 60°C, and primer pairs differed by <2°C whenever possible. Probes were designed to have melting temperatures ~10°C higher than primers. Secondary structures were evaluated using OligoAnalyzer (Integrated DNA Technologies; IDT), and potentially problematic dimers were avoided whenever possible.

Primer concentrations were optimized independently for each assay. To do so, we amplified target DNA, either extracted from tissue samples or purchased as gBlocks Gene Fragments (IDT), using 16 assay mixes with forward and reverse primer concentrations at all combinations of 100, 300, 600, and 900 nM and TaqMan minor-groove binding probes at 250 nM in the final reaction. Optimal primer concentrations were those that yielded the greatest change in normalized fluorescence (ΔR_n) and the lowest quantification cycle (C_q). Reactions were run on StepOnePlus and QuantStudio 3 Real-Time PCR Systems (Thermo Fisher Scientific) in triplicate. Each well contained 7.5 μL of TaqMan Environmental Master Mix 2.0 (2×; Thermo Fisher Scientific), 4 μL of target DNA, and 0.75 μL of assay mix, diluted to 15 μL with sterile water. The thermal cycling profile was 95°C for 10 min, followed by 45 cycles of 95°C for 15 s and 60°C for 1 min.

Finally, we estimated each assay's limit of detection (LOD; the lowest copy number an assay can detect in 95% of replicates) and limit of quantification (LOQ; the lowest copy number an assay can quantify with a coefficient of variation below 35%) using the curve-fitting method of Klymus et al. (2020). Standard curves were created using five-fold serial dilutions from either 31,250 to 2 copies per reaction or 15,625 to 1 copy per reaction. Six replicates of each dilution level and negative controls were run using the same reaction chemistry and thermal cycling profile as above. When all or nearly all replicates of all dilution levels were amplified for an assay, the curve-fitting method could not be used to determine LOD, and we instead used the discrete threshold method, which limits estimates to the particular standard curve dilutions tested. For simplicity, we report only discrete threshold estimates in Appendix S1: Assay Documentation. See Appendix S2: Standard Curve Parameters for modeled estimates and raw data.

2.3 In silico testing

We tested assays in silico using the eDNAssay classifier and sequences from as many target individuals as possible and at least one sequence from each nontarget taxon represented on GenBank (Figure 2; Appendix S3: GenBank Accessions). If there were many nontargets for a given assay or there was an overabundance of sequence data for a given nontarget, the number of sequences evaluated was truncated (typically 5–10 sequences) for expediency and to facilitate sequence alignment, curation, and error checking. Sequences, primers, and probes were aligned as described above and input into eDNAssay to produce class assignment probabilities for each template. The default threshold for assigning a template to the “amplify” class is 0.5; assignment probabilities greater than 0.5 predict amplification, and those less than 0.5 predict nonamplification. While this threshold produces accurate classifications, it can be lowered to safeguard against costly false negative model errors (i.e., when a template is predicted to not cross-amplify but it does). We utilized the more conservative threshold of 0.3 as recommended by Kronenberger et al. (2022). If the number of sequences was truncated for initial assessment (as described above) and the nontarget taxon had an assignment probability ≥0.3, it was reassessed using additional sequences (typically all available) to more fully evaluate cross-amplification risk.

Cross-amplification is very unlikely for nontarget taxa with maximum assignment probabilities <0.3 and very likely for nontargets with minimum assignment probabilities ≥0.7 (Kronenberger et al., 2022), so these were not necessarily tested in vitro. Nontargets with assignment probabilities between 0.3 and 0.7—unless they were of particularly low specificity concern (i.e., also a nonindigenous invasive species, potentially extirpated, or highly endemic)—were tested in vitro to confirm model predictions. In certain cases, a small subset of target sequences had minimum assignment probabilities <0.7 (low-to-moderate likelihood of amplifying), but the assay was nonetheless deemed valid because these sequences were rare (99 of 4670 target sequences assessed [2%]) and could have arisen from hybridization, species misidentification, or technical issues leading to erroneous sequences on GenBank.

Nontarget taxa without sequence data were tested in vitro whenever possible. Some assays had a substantial number of nontargets, for which we lacked both sequences and tissues. To help understand the risk of cross-amplifying these untested nontargets, when there were >10 confamilials with sequence data, we fitted a beta probability density function to the maximum eDNAssay assignment probabilities. We used the probability density function to predict the probability that an untested confamilial (i.e., one of the nontargets without available sequences) would exceed eDNAssay assignment probability thresholds of 0.3 and 0.5, as described in Wilcox et al. (2024).

The eDNAssay classifier is limited to evaluating sequences input by the user. Therefore, we identified more distantly related taxa that primers may amplify via Primer-BLAST, using the nr database with default settings. Sequences from any taxa that were flagged by Primer-BLAST and could potentially occur within the continental United States were input into eDNAssay as described above. Any Primer-BLAST hits with eDNAssay assignment probabilities ≥0.3 were considered to potentially reduce assay specificity and evaluated like confamilial nontarget taxa.

2.4 In vitro testing

We sought to perform in vitro testing on genomic or synthetic DNA from multiple target individuals and at least one representative of each nontarget taxon that could not be tested in silico or that in silico testing identified as potentially cross-amplifying (Figure 2). Nontarget haplotypes with the highest assignment probabilities were tested whenever possible. We prioritized in vitro testing of nontargets with maximum eDNAssay assignment probabilities ≥0.3 but performed broader testing of templates with known sequences to more fully evaluate model performance (Appendix S4: eDNAssay Performance). Nontargets that could not be tested in silico or in vitro were treated as described above—either by gleaning specificity information from sequenced confamilials (sensu Wilcox et al., 2024) or by adding caveats to the assay application. In cases where multiple species had identical sequences where oligonucleotides bind, all were considered to have been tested in vitro as long as one template of a known sequence was tested.

Genomic DNA was extracted from tissues using DNeasy® Blood and Tissue Kits (QIAGEN, Hilden, Germany) and diluted to 0.1 ng/μL, as determined using a Qubit 2.0 Fluorometer (Thermo Fisher Scientific). Tissue samples either already existed in-house or were obtained from other labs or museum collections (see Contributing Partners in Appendix S1: Assay Documentation for full acknowledgements). When tissue samples were difficult to obtain but sequences were available, we tested synthetic DNA (i.e., gBlocks), quantified using a Qubit 2.0 fluorometer or NanoDrop 2000 spectrophotometer (Thermo Fisher Scientific), and diluted to 5000 copies/reaction. Reactions were run in singlicate, included negative and positive controls, and followed the qPCR protocol described above. A template was considered to have been amplified if there was an unambiguous amplification curve, according to expert opinion, regardless of the software-assigned threshold. Any curve that was particularly depressed (with a low ΔRn) was noted to aid in the interpretation of questionable in situ results.

2.5 In situ testing

Assay performance was validated in situ using environmental samples from areas with known or suspected presence and absence of the target taxon. If a previously published assay was already tested in situ, we typically did not perform additional testing. In most cases, environmental samples were collected via the filtration method of Carim, McKelvey, et al. (2016) and either archived from other sampling initiatives or collected expressly for assay validation. Several assays have been extensively tested during routine field usage, and results can be found on the eDNAtlas (Young et al., 2018). These assays did not necessarily undergo formal in situ testing if detections occurred only in locations with reasonable likelihoods of target taxon occurrence (Appendix S1: Assay Documentation).

For new and previously published but modified assays, eDNA was extracted from filters following Franklin et al. (2019) and run in triplicate along with negative and positive controls using the qPCR protocol described above, but replacing 1.8 μL of water with TaqMan Exogenous Internal Positive Control reagents (IPC; Life Technologies; 1.5 μL of 10× IPC assay and 0.3 μL of 50× IPC DNA per reaction). If a sample did not amplify and its IPC was delayed ≥1 C_q from the negative control IPC, the reaction was considered inhibited and the extract was treated using a OneStep™ PCR Inhibitor Removal Kit (Zymo Research) and reanalyzed. A sample was considered positive if an unambiguous amplification curve was produced in at least one well. All eDNA sample handling and analysis was conducted in dedicated laboratory facilities in accordance with the standard operating procedures used by the eDNA program at the National Genomics Center for Wildlife and Fish Conservation.

3.1 Trends in sensitivity

All 46 probe-based qPCR assays (Table 1; Appendix S1: Assay Documentation) have high sensitivity. Standard curves demonstrated low LOD values using both methods described by Klymus et al. (2020): discrete threshold (mean = 5, range = 1–10 copies per reaction) and curve-fitting (mean = 3, range = 2–6 copies per reaction), although the curve-fitting method could not be used for 22 assays because all or nearly all replicates of the lowest standard were amplified (Appendix S2: Standard Curve Parameters). Estimates of LOQ were likewise low (discrete threshold mean = 22, range = 1–250; curve-fitting mean = 16, range = 1–190).

3.2 Trends in generality

We assessed generality in silico for 4670 target sequences (mean = 102, range = 3–625 in silico tests of generality per assay; Figure 3b). Generality was high for all assays, as evidenced by high minimum eDNAssay assignment probabilities (≥0.7) in all but several cases involving uncommon or outlier sequences (n = 34 haplotypes across 13 assays; Figure 3b). Along with the developers of previously published assays, we tested 741 target DNA templates in vitro (mean = 16, range = 1–212 in vitro tests of generality per assay). In vitro testing of target DNA produced strong amplifications, except for 2 of the 31 templates tested with one of the two virile crayfish assays (virile crayfish B), which had unusually low ΔRn values. Both samples were from Wyoming (Green River and Pathfinder Reservoir), and the remaining 19 samples from Wyoming amplified strongly.

Along with the developers of previously published assays, we confirmed amplification in situ of environmental samples from areas with known presence of the target taxon for 42 assays (Appendix S1: Assay Documentation). However, for three of these assays (marbled crayfish, sea lamprey, and feral swine), we did not have access to field samples and instead tested samples from aquaria or captive enclosures containing the target species. We were unable to confirm amplification in situ for four assays (northern/blotched snakehead, Wami tilapia, crucian carp, and golden mussel) because we lacked environmental samples from areas with known presence.

3.3 Trends in specificity

Our search for nontarget taxa—defined as all sympatric confamilials and any sympatric extrafamilials that were flagged by Primer-BLAST and had moderate-to-high cross-amplification risk (eDNAssay assignment probabilities ≥0.3)—identified 0–405 per assay (Figure 3a). Few of the extrafamilial taxa flagged by Primer-BLAST were both sympatric and had moderate-to-high cross-amplification risk: zero for 37 assays, ≤ 3 for eight assays (one of the two virile crayfish assays [virile crayfish A], opossum shrimp, smallmouth/spotted bass, snakehead, silver/bighead carp, black carp, sea lamprey, and feral swine), and 10 for one assay (Chinese mystery snail).

Across assays, there were 1094 nontarget taxa, yielding 5276 unique assay-nontarget taxon combinations. Of these, we tested 4206 (80%), including 3234 (61%) in silico only, and, along with the developers of previously published assays, 972 (18%) in vitro (Figure 4). Paired in silico and in vitro testing was accomplished for 916 assay-nontarget combinations. Specificity was high, with typically low nontarget eDNAssay assignment probabilities (mean of maximum assignment probabilities, 0.16 ± 0.10 SD; Figure 3b). Of 4150 assay-nontarget combinations tested in silico, 292 (7%) had moderate-to-high cross-amplification risk (eDNAssay assignment probabilities ≥0.3; Figure 3b). We tested 236 (81%) in vitro and, of these, 45 (19%) cross-amplified. Fifty-six assay-nontarget combinations were only tested in vitro because we lacked sequence data, and two (4%) cross-amplified. Cross-amplifications were rare overall, observed in 47 (1%) of the 4206 tested assay-nontarget combinations, distributed among 15 assays (Figure 3b,c). Cross-amplifying nontarget taxa were widespread (affecting most USFWS regions) for the American bullfrog, one of two virile crayfish assays (virile crayfish A), tilapia, and crucian carp assays (Figure 3c). The four tilapia assays primarily cross-amplified one another's target species, and the crucian carp assay only cross-amplified goldfish and Prussian carp, species that are also nonindigenous in the United States and may also be of interest to wildlife managers. Many cross-amplifications were “slight” (i.e., with severely depressed ΔRn values), as indicated in Appendix S1: Assay Documentation, and therefore are unlikely to be incorrectly deemed detections in practice.

We focused our in vitro testing efforts on nontarget taxa with moderate-to-high cross-amplification risk (eDNAssay assignment probabilities ≥0.3), but also tested nontargets with low cross-amplification risk (eDNAssay assignment probabilities <0.3) when practical to evaluate performance of the eDNAssay classifier. We only used synthetic gene fragments for this to eliminate sequence uncertainty. In total, we tested 649 unique assay-template combinations (Figure 5; Appendix S4: eDNAssay Performance). In silico predictions from the eDNAssay classifier were 96% accurate, similar to the accuracy reported by Kronenberger et al. (2022). Because Kronenberger et al. (2022) tuned their model to maximize recall and reduce costly false negative errors—that is, cross-amplification of a template predicted to not cross-amplify—accuracy was much higher for predictions of assay specificity (the true negative rate; 611 of 626 correct; 98%) than for predictions of cross-amplification (the true positive rate; 12 of 23 correct; 52%). Importantly, these results were produced using the default classification threshold of 0.5, but our cutoff for in vitro testing was 0.3; under this more conservative threshold, predictions of specificity were 100% accurate.

We identified 1173 “potentially problematic” assay-nontarget combinations—those that either were not tested in silico or in vitro (n = 1070; 91%), had eDNAssay assignment probabilities ≥0.3 but were not tested in vitro (n = 66; 6%; mean of maximum assignment probabilities, 0.45 ± 0.16 SD), or cross-amplified in vitro (n = 37; 3%; Figures 3 and 4c). In some cases, we decided not to test nontarget taxa with moderate-to-high assignment probabilities in vitro if they were highly endemic or rare (as detailed in Appendix S1: Assay Documentation) and therefore any specificity issues would be uncommon. Of the 1173 potentially problematic assay-nontarget combinations, 849 (72%) were associated with the seven crayfish assays and 143 (12%) with the six carp assays. There were typically few potentially problematic nontarget taxa per assay per USFWS region (Figure 3c); of 368 total assay-USFWS region combinations, the number of potentially problematic nontarget taxa was zero for 179 (49%), ≤2 for 279 (76%) and ≤10 for 342 (93%). A notable exception is the crayfish assays when applied in Region 4, where many nontargets are potentially problematic (n = 93–116 per assay; Figure 3c) due to very high biodiversity and a lack of sequence information. Excluding the crayfish assays in Region 4, there were 0–32 potentially problematic nontarget taxa per assay per USFWS region.

As described above, a large majority of potentially problematic assay-nontarget combinations (91%) were designated as such because they were not tested in silico or in vitro. Testing was not possible in these cases because we lacked both sequences and DNA. Therefore, for assays with >10 confamilials tested in silico (n = 33), we used available eDNAssay assignment probabilities to estimate the cross-amplification risk posed by confamilials that were not tested. The cross-amplification risk was generally low. If we fit a probability density function for each assay using the beta distribution (Sensu Wilcox et al., 2024), the likelihood of an untested confamilial exceeding the 0.3 conservative classification threshold is ≤27% in all cases except for smallmouth/spotted bass (44% chance) and the four tilapia assays (35–55% chance). The likelihood of an untested confamilial exceeding the 0.5 default classification threshold is ≤1% in all cases except one of the two virile crayfish assays (virile crayfish A; 4% chance) and the four tilapia assays (5–22% chance). However, high likelihoods for the tilapia assays stem from congeneric species that sometimes hybridize with the targets and are also nonindigenous. With congeners removed, the likelihood of an untested confamilial exceeding the 0.3 threshold is ≤37%, and the likelihood of exceeding the 0.5 threshold is ~0%. Furthermore, along with the developers of previously published assays, we confirmed nonamplification in situ of environmental samples from areas with suspected absence of the target taxon for all assays.

3.4 Assay-specific reporting

To enable efficient use of the 46 assays, descriptions in Appendix S1: Assay Documentation follow a structured format that provides detailed information on (1) oligonucleotide sequences, optimal primer concentrations, and LOD; (2) target taxonomy and relevant biology; (3) occurrence information for all target and nontarget taxa within each USFWS region; (4) the results of all in silico, in vitro, and in situ testing; and (5) special use considerations given the identity and geographic distribution of untested or cross-amplifying nontarget taxa. This structured format is meant to clearly articulate the extent of assay validation and facilitate future updates as additional sequence data and DNA templates become available. See How to Use This Document in Appendix S1: Assay Documentation for a hypothetical example and Troubleshooting for more information about dealing with sequence and sample missingness and advice for interpreting questionable amplifications. State-level nontarget specificity information can be found at https://nationalgenomicscenter.shinyapps.io/state-level-specificity.