Original Article
The dependability of electrophysiological measurements of performance monitoring in a clinical sample: A generalizability and decision analysis of the ERN and Pe
Scott A. Baldwin,
Michael J. Larson,
Peter E. Clayson,
Corresponding Author
Scott A. Baldwin
Department of Psychology, Brigham Young University, Provo, Utah, USA
Address correspondence to: Scott A. Baldwin, 268 TLRB, Department of Psychology, Brigham Young University, Provo, UT 84660. E-mail: [email protected]Search for more papers by this authorMichael J. Larson
Department of Psychology, Brigham Young University, Provo, Utah, USA
Search for more papers by this authorPeter E. Clayson
Department of Psychology, University of California, Los Angeles, Los Angeles, California, USA
Search for more papers by this authorScott A. Baldwin,
Michael J. Larson,
Peter E. Clayson,
Corresponding Author
Scott A. Baldwin
Department of Psychology, Brigham Young University, Provo, Utah, USA
Address correspondence to: Scott A. Baldwin, 268 TLRB, Department of Psychology, Brigham Young University, Provo, UT 84660. E-mail: [email protected]Search for more papers by this authorMichael J. Larson
Department of Psychology, Brigham Young University, Provo, Utah, USA
Search for more papers by this authorPeter E. Clayson
Department of Psychology, University of California, Los Angeles, Los Angeles, California, USA
Search for more papers by this authorThis research was supported by the Brigham Young University College of Family, Home, and Social Sciences.
Abstract
Psychometric studies of the ERN, CRN, Pe, and Pc ERPs are increasing. Coherent integration of these results is difficult with classical test theory because the definition of error depends on the measure of reliability. This study used generalizability theory, which extends the ideas of classical test theory, as a framework for evaluating the influence of psychopathology and number of trials on dependability of measurement. Participants included 34 people meeting criteria for major depression, 29 meeting criteria for an anxiety disorder, and 319 controls. For all ERPs, within-person variance was larger than between-person variance across groups, indicating many trials are needed for adequate dependability (at least 13). Slightly fewer trials were needed to achieve adequate dependability in the control group than the pathology groups. Regions of interest had higher dependability than single sensors.
References
- Aarts, K., Vanderhasselt, M. A., Otte, G., Baeken, C., & Pourtois, G. (2013). Electrical brain imaging reveals the expression and timing of altered error monitoring functions in major depression. Journal of Abnormal Psychology, 122, 939–950. doi: 10.1037/a0034616
- Bartholow, B. D., Pearson, M. A., Dickter, C. L., Sher, K. J., Fabiani, M., & Gratton, G. (2005). Strategic control and medial frontal negativity: Beyond errors and response conflict. Psychophysiology, 42, 33–42. doi: 10.1111/j.1469-8986.2005.00258.x
- Bates, A. T., Liddle, P. F., Kiehl, K. A., & Ngan, E. T. C. (2004). State dependent changes in error monitoring in schizophrenia. Journal of Psychiatric Research, 38, 347–356. doi: 10.1016/j.jpsychires.2003.11.002
- Brazdil, M., Roman, R., Daniel, P., & Rektor, I. (2005). Intracerebral error-related negativity in a simple go/no-go task. Journal of Psychophysiology, 19, 244–255. doi: 10.1027/0269-8803.19.4.244
- Brennan, R. L. (2001). Generalizability theory. New York, NY: Springer.
10.1007/978-1-4757-3456-0 Google Scholar
- Cassidy, S. M., Robertson, I. H., & O'Connell, R. G. (2012). Retest reliability of event-related potentials: Evidence from a variety of paradigms. Psychophysiology, 49, 659–664. doi: 10.1111/j.1469-8986.2011.01349.x
- Chen, S., & Melara, R. D. (2009). Sequential effects in the Simon task: Conflict adaptation or feature integration? Brain Research, 1297, 89–100. doi: 10.1016/j.brainres.2009.08.003
- Chiu, P. H., & Deldin, P. J. (2007). Neural evidence for enhanced error detection in major depressive disorder. American Journal of Psychiatry, 164, 608–616. doi: 10.1176/appi.ajp.164.4.608
- Clawson, A., Clayson, P. E., & Larson, M. J. (2013). Cognitive control adjustments and conflict adaptation in major depressive disorder. Psychophysiology, 50, 711–721. doi: 10.1111/psyp.12066
- Clayson, P. E., Baldwin, S. A., & Larson, M. J. (2013). How does noise affect amplitude and latency measurement of event-related potentials (ERPs)? A methodological critique and simulation study. Psychophysiology, 50, 174–186. doi: 10.1111/psyp.12001
- Clayson, P. E., Clawson, A., & Larson, M. J. (2011). Sex differences in electrophysiological indices of conflict monitoring. Biological Psychiatry, 87, 282–289. doi: 10.1016/j.biopsycho.2011.03.011
- Clayson, P. E., & Larson, M. J. (2011). Conflict adaptation and sequential trial effects: Support for the conflict monitoring theory. Neuropsychologia, 49, 1953–1961. doi: 10.1016/j.neuropsychologia.2011.03.023
- Dien, J. (2010). The ERP PCA toolkit: An open source program for advanced statistical analysis of event-related potential data. Journal of Neuroscience Methods, 187, 138–145. doi: 10.1016/j.jneumeth.2009.12.009
- Dien, J., Michelson, C. A., & Franklin, M. S. (2010). Separating the visual sentence N400 effect from the P400 sequential expectancy effect: Cognitive and neuroanatomical implications. Brain Research, 1355, 126–140. doi: 10.1016/j.brainres.2010.07.099
- Endrass, T., Klawohn, J., Gruetzmann, R., Ischebeck, M., & Kathmann, N. (2012). Response-related negativities following correct and incorrect responses: Evidence from a temporospatial principal component analysis. Psychophysiology, 49, 733–743. doi: 10.1111/j.1469-8986.2012.01365.x
- Endrass, T., Reuter, B., & Kathmann, N. (2007). ERP correlates of conscious error recognition: Aware and unaware errors in an antisaccade task. European Journal of Neuroscience, 26, 1714–1720. doi: 10.1111/j.1460-9568.2007.05785.x
- Endrass, T., Schuermann, B., Kaufmann, C., Spielberg, R., Kniesche, R., & Kathmann, N. (2010). Performance monitoring and error significance in patients with obsessive-compulsive disorder. Biological Psychology, 84, 257–263. doi: 10.1016/j.biopsycho.2010.02.002
- Eriksen, B. A., & Eriksen, C. W. (1974). Effects of noise letters upon the identification of a target letter in a non-search task. Perception & Psychophysics, 16, 143–149.
- Falkenstein, M., Hohnsbein, J., Hoormann, J., & Banke, L. (1991). Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. Electroencephalography and Clinical Neurophysiology, 78, 447–455.
- Falkenstein, M., Hoormann, J., Christ, S., & Hohnsbein, J. (2000). ERP components on reaction errors and their functional significance: A tutorial. Biological Psychology, 51, 87–107. doi: 10.1016/S0301-0511(99)00031-9
- Fallgatter, A. J., Bartsch, A. J., Strik, W. K., Mueller, T. J., Eisenack, S. S., Neuhauser, B., … Herrmann, M. J. (2001). Test-retest reliability of electrophysiological parameters related to cognitive motor control. Clinical Neurophysiology, 112, 198–204. doi: 10.1016/S1388-2457(00)00505-8
- Ford, J. M. (1999). Schizophrenia: The broken P300 and beyond. Psychophysiology, 36, 667–682. doi: 10.1111/1469-8986.3660667
- Foti, D., Kotov, R., & Hajcak, G. (2013). Psychometric considerations in using error-related brain activity as a biomarker in psychotic disorders. Journal of Abnormal Psychology, 122, 520–531. doi: 10.1037/a0032618
- Gehring, W. J., Goss, B., Coles, M. G. H., Meyer, D. E., & Donchin, E. (1993). A neural system for error detection and compensation. Psychological Science, 4, 385–390. doi: 10.1111/j.1467-9280.1993.tb00586.x
- Gehring, W. J., Himle, J., & Nisenson, L. G. (2000). Action-monitoring dysfunction in obsessive-compulsive disorder. Psychological Science, 11, 1–6. doi: 10.1111/1467-9280.00206
- Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis, 1, 515–533.
- Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–511.
10.1214/ss/1177011136 Google Scholar
- Georgiadi, E., Liotti, M., Nixon, N. L., & Liddle, P. F. (2011). Electrophysiological evidence for abnormal error monitoring in recurrent major depressive disorder. Psychophysiology, 48, 1192–1202. doi: 10.1111/j.1469-8986.2011.01198.x
- Gottesman, I. I., & Gould, T. D. (2003). The endophenotype concept in psychiatry: Etymology and strategic intentions. American Journal of Psychiatry, 160, 636–645.
- Hajcak, G., Franklin, M. E., Foa, E. B., & Simons, R. F. (2008). Increased error-related brain activity in pediatric obsessive-compulsive disorder before and after treatment. American Journal of Psychiatry, 165, 116–123. doi: 10.1176/appi.ajp.2007.07010143
- Hammer, A., Kordon, A., Heldmann, M., Zurowski, B., & Münte, T. F. (2009). Brain potentials of conflict and error-likelihood following errorful and errorless learning in obsessive-compulsive disorder. PLOS ONE, 4, e6553. doi: 10.1371/journal.pone.0006553
- Herrmann, M. J., Römmler, J., Ehlis, A., Heidrich, A., & Fallgatter, A. J. (2004). Source localization (LORETA) of the error-related-negativity (ERN/Ne) and positivity (Pe). Cognitive Brain Research, 20, 294–299. doi: 10.1016/j.cogbrainres.2004.02.013
- Hoffman, S., & Falkenstein, M. (2011). Aging and error processing: Age related increase in the variability of the error-negativity is not accompanied by increase in response variability. PLOS ONE, 28, e17482. doi: 10.1371/journal.pone.0017482
- Holmes, A. J., & Pizzagalli, D. A. (2008). Spatiotemporal dynamics of error processing dysfunction in major depressive disorder. Archives of General Psychiatry, 65, 179–188.
- Holmes, A. J., & Pizzagalli, D. A. (2010). Effects of task-relevant incentives on the electrophysiological correlates of error processing in major depressive disorder. Cognitive Affective and Behavioral Neuroscience, 10, 119–128. doi: 10.3758/CABN.10.1.119
- Holroyd, C. B., & Coles, M. G. H. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679–709. doi: 10.1037/0033-295x.109.4.679
- Huffmeijer, R., Bakermans-Kranenburg, M. J., Alink, L. R. A., & van Ijzendoom, M. H. (2014). Reliability of event-related potentials: The influence of number of trials and electrodes. Physiology & Behavior, 130, 13–22.
- Hughes, G., & Yeung, N. (2011). Dissociable correlates of response conflict and error awareness in error-related brain activity. Neuropsychologia, 49, 405–415. doi: 10.1016/j.neuropsychologia.2010.11.036
- Johannes, S., Wieringa, B. M., Nager, W., Rada, D., Dengler, R., Emrich, H. M., … Dietrich, D. E. (2001). Discrepant target detection and action monitoring in obsessive-compulsive disorder. Psychiatry Research, 108, 101–110. doi: 10.1016/S0925-4927(01)00117-2
- Junghöfer, M., Elbert, T., Tucker, D. M., & Braun, C. (1999). The polar average reference effect: A bias in estimating the head surface integral in EEG recording. Clinical Neurophysiology, 110, 1149–1155.
- Larson, M. J., Baldwin, S. A., Good, D. A., & Fair, J. E. (2010). Temporal stability of the error-related negativity (ERN) and post-error positivity (Pe): The role of number of trials. Psychophysiology, 47, 1167–1171. doi: 10.1111/j.1469-8986.2010.01022.x
- Larson, M. J., Clawson, A., Clayson, P. E., & Baldwin, S. A. (2013). Cognitive conflict adaptation in generalized anxiety disorder. Biological Psychiatry, 94, 408–418. doi: 10.1016/j.biopsycho.2013.08.006
- Larson, M. J., Clayson, P. E., & Clawson, A. (2014). Making sense of all the conflict: A theoretical review and critique of conflict-related ERPs. International Journal of Psychophysiology, 93, 283–297.
- Larson, M. J., Clayson, P. E., & Farrer, T. J. (2012). Performance monitoring and cognitive control in individuals with mild traumatic brain injury. Journal of the International Neuropsychological Society, 18, 323–333. doi: 10.1017/S1355617711001779
- Larson, M. J., Kaufman, D. A., Schmalfuss, I. M., & Perlstein, W. M. (2007). Performance monitoring, error processing, and evaluative control following severe TBI. Journal of the International Neuropsychological Society, 13, 961–971.
- Larson, M. J., & Perlstein, W. M. (2009). Awareness of deficits and error processing after traumatic brain injury. NeuroReport, 20, 1486–1490.
- Larson, M. J., South, M., Krauskopf, E., Clawson, A., & Crowley, M. J. (2011). Feedback and reward processing in high-functioning autism. Psychiatry Research, 187, 198–203. doi: 10.1016/j.psychres.2010.11.006
- Larson, M. J., Steffen, P. R., & Primosch, M. (2013). The impact of a brief mindfulness meditation intervention on cognitive control and error-related performance monitoring. Frontiers in Human Neuroscience, 7, 308. doi: 10.3389/fnhum.2013.00308
- Masaki, H., Gehring, W. J., Takasawa, N., & Yamazaki, K. (2004). The functional significance of the error-related negativity in action monitoring. Japanese Journal of Physiological Psychology and Psychophysiology, 22, 3–18.
10.5674/jjppp1983.22.3 Google Scholar
- Meyer, A., Riesel, A., & Hajcak, G. (2013). Reliability of the ERN across multiple tasks as a function of increasing errors. Psychophysiology, 50, 1220–1225. doi: 10.1111/psyp.12132
- Münte, T. F., Heldmann, M., Hinrichs, H., Marco-Pallares, J., Krämer, U. M., Sturm, V., & Heinze, H.-J. (2007). Nucleus accumbens is involved in human action monitoring: Evidence from invasive electrophysiological recordings. Frontiers in Human Neuroscience, 1, 11. doi: 10.3389/neuro.09.011.2007
- Nieuwenhuis, S., Nielen, M. M., Mol, N., Hajcak, G., & Veltman, D. J. (2005). Performance monitoring in obsessive-compulsive disorder. Psychiatry Research, 134, 111–122. doi: 10.1016/j.psychres.2005.02.005
- Nieuwenhuis, S., Ridderinkhof, K. R., Blom, J., Band, G. P., & Kok, A. (2001). Error-related brain potentials are differentially related to awareness of response errors: Evidence from an antisaccade task. Psychophysiology, 38, 752–760. doi: 10.1111/1469-8986.3850752
- Nocera, F. D., Ferlazzo, F., & Borghi, V. (2001). G theory and the reliability of psychophysiological measures: A tutorial. Psychophysiology, 38, 796–806.
- Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. New York, NY: McGraw-Hill.
- Olvet, D. M., & Hajcak, G. (2008). The error-related negativity (ERN) and psychopathology: Toward an endophenotype. Clinical Psychology Review, 28, 1343–1354. doi: 10.1016/j.cpr.2008.07.003
- Olvet, D. M., & Hajcak, G. (2009a). Reliability of error-related brain activity. Brain Research, 1284, 89–99. doi: 10.1016/j.brainres.2009.05.079
- Olvet, D. M., & Hajcak, G. (2009b). The stability of error-related brain activity with increasing trials. Psychophysiology, 46, 957–961. doi: 10.1111/j.1469-8986.2009.00848.x
- Olvet, D. M., Klein, D. N., & Hajcak, G. (2010). Depression symptom severity and error-related brain activity. Psychiatry Research, 179, 30–37. doi: 10.1016/j.psychres.2010.06.008
- Overbeek, T. J. M., Nieuwenhuis, S., & Ridderinkhof, K. R. (2005). Dissociable components of error processing: On the functional significance of the Pe vis-à-vis the ERN/Ne. Journal of Psychophysiology, 19, 319–329. doi: 10.1027/0269-8803.19.4.319
- Pailing, P. E., & Segalowitz, S. J. (2004). The effects of uncertainty in error monitoring on associated ERPs. Brain and Cognition, 56, 215–233. doi: 10.1016/j.bandc.2004.06.005
- Pontifex, M. B., Scudder, M. R., Brown, M. L., O'Leary, K. C., Wu, C.-T., Themanson, J. R., & Hillman, C. H. (2010). On the number of trials necessary for stabilization of error-related brain activity across the life span. Psychophysiology, 47, 767–773. doi: 10.1111/j.1469-8986.2010.00974.x
- R Development Core Team. (2012). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/
- Ruchsow, M., Herrnberger, B., Beschoner, P., Gron, G., Spitzer, M., & Kiefer, M. (2006). Error processing in major depressive disorder: Evidence from event-related potentials. Journal of Psychiatric Research, 40, 37–46. doi: 10.1016/j.jpsychires.2005.02.002
- Ruchsow, M., Herrnberger, B., Wiesend, C., Gron, G., Spitzer, M., & Kiefer, M. (2004). The effect of erroneous responses on response monitoring in patients with major depressive disorder: A study with event-related potentials. Psychophysiology, 41, 833–840.
- Scheffers, M. K., & Coles, M. G. H. (2000). Performance monitoring in a confusing world: Error-related brain activity, judgments of response accuracy, and types of errors. Journal of Experimental Psychology: Human Perception and Performance, 26, 141–151. doi: 10.1037/0096-1523.26.1.141
- Schrijvers, D., De Bruijn, E. R., Maas, Y., Vancoillie, P., Hulstijn, W., & Sabbe, B. G. (2009). Action monitoring and depressive symptom reduction in major depressive disorder. International Journal of Psychophysiology, 71, 218–224. doi: 10.1016/j.ijpsycho.2008.09.005
- Schrijvers, D., De Bruijn, E. R. A., Maas, Y., De Grave, C., Sabbe, B. G., & Hulstijn, W. (2008). Action monitoring in major depressive disorder with psychomotor retardation. Cortex, 44, 569–579. doi: 10.1016/j.cortex.2007.08.014
- Segalowitz, S. J., Santesso, D. L., Murphy, T. I., Homan, D., Chantziantoniou, D. K., & Khan, S. (2010). Retest reliability of medial frontal negativities during performance monitoring. Psychophysiology, 47, 260–270. doi: 10.1111/j.1469-8986.2009.00942.x
- Shalgi, S., Barkan, I., & Deouell, L. Y. (2009). On the positive side of error processing: Error-awareness positivity revisited. European Journal of Neuroscience, 29, 1522–1532. doi: 10.1111/j.1460-9568.2009.06690.x
- Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Thousand Oaks, CA: Sage.
- Stan Development Team. (2014). Stan: A C++ library for probability and sampling, Version 2.4. Retrieved from http://mc-stan.org
- Stemmer, B., Segalowitz, S. J., Witzke, W., & Schonle, P. W. (2004). Error detection in patients with lesions to the medial prefrontal cortex: An ERP study. Neuropsychologia, 42, 118–130. doi: 10.1016/S0028-3932(03)00121-0
- Stern, E. R., Liu, Y., Gehring, W. J., Lister, J. J., Yin, G., Zhang, J., … Taylor, S. F. (2010). Chronic medication does not affect hyperactive error responses in obsessive-compulsive disorder. Psychophysiology, 47, 913–920. doi: 10.1111/j.1469-8986.2010.00988.x
- Tang, Y., Zhang, X., Simmonite, M., Li, H., Zhang, T., Guo, Q., … Wang, J. (2013). Hyperactivity within an extensive cortical distribution associated with excessive sensitivity in error processing in unmedicated depression: A combined event-related potential and sLORETA study. International Journal of Psychophysiology, 90, 282–289. doi: 10.1016/j.ijpsycho.2013.09.001
- van Veen, V., & Carter, C. S. (2002). The anterior cingulate as a conflict monitor: fMRI and ERP studies. Physiology & Behavior, 77, 477–482. doi: 10.1016/S0031-9384(02)00930-7
- Vidal, F., Burle, B., Bonnet, M., Grapperon, J., & Hasbroucq, T. (2003). Error negativity on correct trials: A reexamination of available data. Biological Psychology, 64, 265–282. doi: 10.1016/s0301-0511(03)00097-8
- Vidal, F., Hasbroucq, T., Grapperon, J., & Bonnet, M. (2000). Is the “error negativity” specific to errors? Biological Psychology, 51, 109–128. doi: 10.1016/S0301-0511(99)00032-0
- Weinberg, A., & Hajcak, G. (2011). Longer term test-retest reliability of error-related brain activity. Psychophysiology, 48, 1420–1425. doi: 10.1111/j.1469-8986.2011.01206.x
- Weinberg, A., Olvet, D. M., & Hajcak, G. (2010). Increased error-related brain activity in generalized anxiety disorder. Biological Psychology, 85, 472–480. doi: 10.1016/j.biopsycho.2010.09.011
- Wiersema, J. R., van der Meere, J. J., & Roeyers, H. (2007). Developmental changes in error monitoring: An event-related potential study. Neuropsychologia, 45, 1649–1657.
- Yeung, N., Botvinick, M. M., & Cohen, J. D. (2004). The neural basis of error detection: Conflict monitoring and the error-related negativity. Psychological Review, 111, 931–959. doi: 10.1037/0033-295x.111.4.931
