Chapter 139
Quantitative Diagnostic Test Validation
Bente Flatland,
Bente Flatland
Search for more papers by this authorBente Flatland,
Bente Flatland
Search for more papers by this authorBook Editor(s):Marjory B. Brooks DVM, DACVIM,
Kendal E. Harr DVM, MS, DACVP,
Davis M. Seelig DVM, PhD, DACVP,
K. Jane Wardrop DVM, MS, DACVP,
Douglas J. Weiss DVM, PhD, DACVP,
Marjory B. Brooks DVM, DACVIM
Director, Comparative Coagulation Section
Animal Health Diagnostic Center, Cornell University, Ithaca, New York, USA
Search for more papers by this authorKendal E. Harr DVM, MS, DACVP
URIKA, LLC, Mukilteo, Washington, USA
Search for more papers by this authorDavis M. Seelig DVM, PhD, DACVP
Associate Professor, Clinical Pathology
Department of Veterinary Clinical Sciences, University of Minnesota, College of Veterinary Medicine, St. Paul, Minnesota, USA
Search for more papers by this authorK. Jane Wardrop DVM, MS, DACVP
Professor and Director, Clinical Pathology Laboratory
Department of Veterinary Clinical Sciences, College of Veterinary Medicine, Washington State University, Pullman, Washington, USA
Search for more papers by this authorDouglas J. Weiss DVM, PhD, DACVP
Emeritus Professor
College of Veterinary Medicine, University of Minnesota, St. Paul, Minnesota, USA
Search for more papers by this authorSummary
This chapter focuses on analytical and diagnostic performance and provides an overview of the study designs and statistical tools used in their assessment. Analytical performance assessment characterizes an assay's measurement performance through the study of analytical properties such as imprecision, bias, and linearity. Individual laboratories should carry out some assessment of analytical performance prior to implementing a new assay for routine clinical use. Analytical quality goals have been derived from various sources, including regulatory requirements, expert consensus, biological variation data, “state of the art”, and patient outcome studies. Analytical performance validation of assays involving unstable specimens or labile measurands may necessarily be limited by logistical concerns. Observed total error combines observed bias and imprecision into one quantity calculated with a stated statistical probability. Diagnostic test performance validation compares results from a test under evaluation, that is, index test, to another test deemed the “gold standard” for disease diagnosis.
REFERENCES
- Establishment and verification of method performance specifications . Federal Register 1992 ; 57 ( 40 ): 7164§ 493.1253.
- Medical Laboratories – Requirements for quality and competence . International Organization for Standardization. ISO 15189. Geneva, Switzerland : International Organization for Standardization , 2012 .
- Bland MJ , Altman DG . Statistical methods for assessing agreement between two methods of clinical measurement . Lancet 1986 ; i : 307 – 310 .
- Bossuyt PM , Reitsma JB , Bruns DE , et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies . Clin Chem 2015 ; 61 ( 12 ): 1446 – 1452 .
- Bossuyt PMM , Reitsma JB , Linnet K , et al. Beyond diagnostic accuracy: the clinical utility of diagnostic tests . Clin Chem 2012 ; 58 ( 12 ): 1636 – 1643 .
- Carey R , Anderson F , George H , et al. User verification of performance for precision and trueness; approved guideline , 2nd ed. Clinical and Laboratory Standards Institute. EP15-A2 . Wayne, PA : Clinical and Laboratory Standards Institute (CLSI) , 2005 ;25(17):1–49.
- Erb HN . Prior probability (the pretest best guess) affects predictive values of diagnostic tests . Vet Clin Pathol 2011 ; 40 ( 2 ): 154 – 158 .
- Flatland B , Breickner LC , Fry MM . Analytical performance of a dry chemistry analyzer designed for in-clinic use . Vet Clin Pathol 2014 ; 43 ( 2 ): 206 – 217 .
- Flatland B , Camus MS , Baral RM . Analytical quality goals: a review . Vet Clin Pathol 2018 ; 47 : 527 – 538 .
- Flatland B , Friedrichs KR , Klenner S . Differentiating between analytical and diagnostic performance evaluation with a focus on the method comparison study and identification of bias . Vet Clin Pathol 2014 ; 43 ( 4 ): 475 – 486 .
- Flatland B , Vap LM . Quality management recommendations for automated and manual in-house hematology of domestic animals . Vet Clin North Am Small Anim Pract 2012 ; 42 ( 1 ): 11 – 22 .
- Freeman KP , Bauer N , Jensen AL , et al. Introduction to ISO 15189: a blueprint for quality systems in veterinary laboratories . Vet Clin Pathol 2006 ; 35 ( 2 ): 157 – 171 .
- Gardner IA , Greiner M . Receiver-characteristic operating curves and likelihood ratios: improvements over traditional methods for the evaluation and application of veterinary clinical pathology tests . Vet Clin Pathol 2006 ; 35 ( 1 ): 8 – 17 .
- Harr KE , Flatland B , Nabity MB , et al. ASVCP guidelines: allowable total error guidelines for biochemistry . Vet Clin Pathol 2013 ; 42 ( 4 ) 424 – 436 .
- Hooijberg EH , Steenkamp G , du Preez JP , et al. Analytic and quality control validation and assessment of field performance of a point-of-care chemistry analyzer for use in the White rhinoceros . Vet Clin Pathol 2017 ; 46 ( 1 ): 100 – 110 .
- Jacobs RM , Lumsden JH , Grift E . Effects of bilirubinemia, hemolysis, and lipemia on clinical chemistry analytes in bovine, canine, equine, and feline sera . Can Vet J 1992 ; 33 : 605 – 608 .
- Jensen AL , Kjelgaard-Hansen M . Diagnostic test validation . In: DJ Weiss , KJ Wardrop , eds. Schalm's Veterinary Hematology . Ames, IA : Blackwell Publishing Ltd , 2010 : 1027 – 1033 .
- Jensen AL , Kjelgaard-Hansen M . Method Comparison in the Clinical Laboratory . Vet Clin Pathol 2006 ; 35 ( 3 ): 276 – 286 .
- Johnson BM , Fry MM , Flatland B , Kirk CA . Comparison of a human portable blood glucose meter, veterinary portable blood glucose meter, and automated chemistry analyzer for measurement of blood glucose concentrations in dogs . J Am Vet Med Assoc 2009 ; 235 : 1309 – 1313 .
- Kinns H , Pitkin S , Housley D , et al. Internal quality control: best practice . J Clin Pathol 2013 ; 66 ( 12 ): 1027 – 1032 .
- Kjelgaard-Hansen M , Jensen AL , Kristensen AT . Internal quality control of a turbidimetric immunoassay for canine serum C-reactive protein based on pooled patient samples . Vet Clin Pathol 2004 ; 33 ( 3 ): 139 – 144 .
- Lane MB , Flatland B , Olin SJ , et al. Analytic performance evaluation of a veterinary-specific ELISA for measurement of serum cortisol concentrations of dogs . J Am Vet Med Assoc 2018 ; 253 : 1580 – 1588 .
- Lester S , Harr KE , Rishniw M , et al. Current quality assurance concepts and considerations for quality control of in-clinic biochemistry testing . J Am Vet Med Assoc 2013 ; 242 ( 2 ): 182 – 192 .
- Linnet K . Estimation of the linear relationship between the measurements of two methods with proportional errors . Stat Med 1990 ; 9 : 1463 – 1473 .
- Linnet K , Bossuyt PMM , Moons KGM , et al. Quantifying the accuracy of a diagnostic test or marker . Clin Chem 2012 ; 58 ( 9 ): 1292 – 1301 .
- Linnet K , Boyd JC. Selection and analytical evaluation of method – with statistical techniques . In: CA Burtis , EA Ashwood , DE Bruns , eds. Tiezt Textbook of Clinical Chemistry and Molecular Diagnostics , 4th ed. St. Louis, MO : Elsevier Saunder , 2006 ; 353 – 407 .
- Magnusson B , Ossowicki H , Rienitz O , et al. Routine internal- and external-quality control data in clinical laboratories for estimating measurement and diagnostic uncertainty using GUM principles . Scand J Clin Lab Invest 2012 ; 72 ( 3 ): 212 – 220 .
- Nabity MB , Harr KE , Camus MS , et al. ASVCP Guidelines: allowable total error hematology . Vet Clin Pathol 2018 ; 47 : 9 – 21 .
- Nandagopalan S , Carey R , Levine J , et al. How to Construct and Interpret an Error Grid for Quantitative Diagnostic Assays; approved guideline . Clinical and Laboratory Standards Institute (CLSI). EP27-A . Wayne, PA : Clinical and Laboratory Standards Institute (CLSI) , 2012 ; 32 (12): 1 – 29 .
- Oosterhuis WP , Bayat H , Armbruster D , et al. The use of error and uncertainty methods in the medical laboratory . Clin Chem Lab Med 2018 ; 56 ( 2 ): 209 – 219 .
- Oosterhuis WP , Theodorsson E . Total error vs. measurement uncertainty: revolution or evolution . Clin Lab Med 2016 ; 54 ( 2 ): 235 – 239 .
- Pfutzner A , Klonoff DC , Pardo S , et al. Technical aspects of the Parkes Error Grid . J Diabetes Sci Technol 2013 ; 7 ( 5 ): 1275 – 1281 .
- Plaut D , Westgard JO. Monitoring quality with EQA and PT . In: JO Westgard , ed. Basic QC Practices Training in Statistical Quality Control for Medical Laboratories , 4th ed. Madison, WI : Westgard QC , 2016 : 253 – 269 .
- Rishniw M , Pion PD . Evaluation of performance of veterinary in-clinic hematology analyzers . Vet Clin Pathol 2016 ; 45 ( 4 ): 604 – 614 .
- Rishniw M , Pion PD , Maher T. The quality of veterinary in-clinic and reference laboratory biochemical testing . Vet Clin Pathol 2012 ; 41 ( 1 ): 92 – 109 .
- Sandberg S , Fraser CG , Horvath AR , et al. Defining analytical performance specifications: consensus statement from the 1st strategic conference of the European Federation of Clinical Chemistry and Laboratory Medicine . Clin Chem Lab Med 2015 ; 53 ( 6 ): 833 – 835 .
- Theodorsson E . Uncertainty in measurement and total error: tools for coping with diagnostic uncertainty . Clin Lab Med 2017 ; 37 ( 1 ): 15 – 34 .
- Tholen D , Kallner A , Kennedy J , et al. Evaluation of precision performance of quantitative measurement methods; approved guideline – second edition . Clinical and Laboratory Standards Institute (CLSI). EP5-A2 . Wayne, PA : Clinical and Laboratory Standards Institute (CLSI) , 2004 ; 24 (25): 1 – 39 .
- Westgard JO . Charts of operational process specifications (“OPSpecs charts”) for assessing the precision, accuracy, and quality control needed to satisfy proficiency testing performance criteria . Clin Chem 1992 ; 38 ( 7 ): 1226 – 1233 .
- Westgard JO. The detection limit experiment . In: JO Westgard , ed. Basic Method Validation , 3rd ed. Madison, WI : Westgard QC , 2008 ; 167 – 176 .
- Westgard JO. The interference and recovery experiments . In: JO Westgard , ed. Basic Method Validation , 3rd ed. Madison, WI : Westgard QC , 2008 ; 153 – 166 .
- Westgard JO . Internal quality control: planning and implementation strategies . Ann Clin Biochem 2003 ; 40 : 593 – 611 .
- Westgard JO. The linearity or reportable range experiment . In: JO Westgard , ed. Basic Method Validation , 3rd ed. Madison, WI : Westgard QC , 2008 : 101 – 112 .
- Westgard JO. A method evaluation decision chart (MEDx chart) for judging method performance . Clin Lab Sci 1995 ; 8 : 227 – 283 .
- Westgard JO. Method validation: the inner, hidden, deeper, secret meaning . In: JO Westgard , ed. Basic Method Validation , 3rd ed. Madison, WI : Westgard QC , 2008 ; 27 – 36 .
- Westgard JO. The replication experiment . In: JO Westgard , ed. Basic Method Validation , 3rd ed. Madison, WI : Westgard QC , 2008 ; 113 – 122 .
- Westgard JO , Westgard SA . Westgard QC . https://www.westgard.com/ . Accessed September 6, 2018.
- Westgard JO , Westgard SA . Measuring analytical quality: total analytical error versus measurement uncertainty . Clin Lab Med 2017 ; 37 ( 1 ): 1 – 13 .
- Westgard JO , Westgard SA . Quality control review: implementing a scientifically based quality control system . Ann Clin Biochem 2016 ; 53 : 32 – 50 .
- Whiting PF , Rutjes AW , Westwood ME , et al. A systematic review classifies sources of bias and variation in diagnostic test accuracy studies . J Clin Epidemiol 2013 ; 66 ( 10 ): 1093 – 1104 .
