SYMPOSIUM ON EXPERIMENTS AND THE STUDY OF THE PRESIDENCY
The Promise of Experiments for Studying the Presidency
Jeffrey E. Cohen,
Jeffrey E. Cohen
Search for more papers by this authorJeffrey E. Cohen,
Jeffrey E. Cohen
Search for more papers by this authorAUTHOR'S NOTE: I would like to thank Douglas Kriner, Andrew Reeves, and Jon Rogowski for comments on an earlier version of this article.
Abstract
Presidency scholars, like political scientists in general, have increasingly used experiments in their research. This article argues that, to differentiate itself from other experimental political science, presidency scholars should bring to bear their knowledge of the institution when using experiments and focus on issues of governance. Doing so, however, raises a host of issues and complications in designing experiments to study presidential governance. This article discusses some of these concerns and offers suggestions to improve and stimulate experimental research on the presidency.
References
-
Aberbach, Joel D.,
Robert D. Putnam, and
Bert A. Rockman. 1981. Bureaucrats and Politicians in Western Democracies. Cambridge, MA: Harvard University Press.
10.4159/9780674020047 Google Scholar
- Adar, Eytan. 2011. “Why I Hate Mechanical Turk Research.” CHI 2011 Workshop on Crowdsourcing and Human Computation, May 7–12.
- Aronson, Elliot, Marilyn B. Brewer, and J. Merill Carlsmith. 1985. “Experimentation in Social Psychology.” In Handbook of Social Psychology, eds. Gardner Lindzey and Ellitot Aronson. New York: Random House, 441–86.
- Barabas, Jason, and Jennifer Jerit. 2010. “Are Survey Experiments Externally Valid?” American Political Science Review 104: 226–42.
- Baum, Matthew A., and Tim Groeling. 2009. “Shot by the Messenger: Partisan Cues and Public Opinion Regarding National Security and War.” Political Behavior 31: 157–86.
- Berinsky, Adam J., Gregory A. Huber, and Gabriel S. Lenz. 2012. “Evaluating Online Labor Markets for Experimental Research: Amazon.Com's Mechanical Turk.” Political Analysis 20: 351–68.
- Bertrand, Marianne, Esther Duflo, and Sendhil Mullainathan. 2004. “How Much Should We Trust Differences-In-Differences Estimates?” Quarterly Journal of Economics, 119: 249–75.
- Bishop, George Franklin. 2013. “Instrument Design: Question Form, Wording, and Context Effects.” In The Oxford Handbook of American Public Opinion and the Media, eds. Robert Y. Shapiro and Lawrence R. Jacobs. New York: Oxford University Press, 348–66.
- Braman, Eileen. 2016. “Exploring Citizen Assessments of Unilateral Executive Authority.” Law & Society Review 50: 189–223.
-
Butler, Daniel M. 2014. Representing the Advantaged: How Politicians Reinforce Inequality. New York: Cambridge University Press.
10.1017/CBO9781139871969 Google Scholar
- Chong, Dennis, and James N. Druckman. 2010. “Dynamic Public Opinion: Communication Effects over Time.” American Political Science Review 104: 663–80.
- Chong, Dennis, and James N. Druckman. 2013. “Counterframing Effects.” The Journal of Politics 75: 1–16.
- Christenson, Dino P., and David M. Glick. 2015. “Chief Justice Roberts's Health Care Decision Disrobed: The Microfoundations of the Supreme Court's Legitimacy.” American Journal of Political Science 59: 403–18.
- Christenson, Dino P., and Douglas L. Kriner. 2017. “Constitutional Qualms or Politics as Usual? The Factors Shaping Public Support for Unilateral Action.” American Journal of Political Science. 61: 335–49.
- Cohen, Jeffrey E. 2015. “Presidential Leadership of Public Opinion: An Embedded Survey Experiment.” Political Communication 32: 345–55.
- Converse, Philip E. 1964. “The Nature of Belief Systems in Mass Publics.” In Ideology and Discontent, ed. David Apter. New York: Free Press, 206–61.
- Dafoe, Allan, Baobao Zhang, and Devin Caughey. 2015. “Confounding in Survey Experiments.” Paper presented at the Annual Meeting of the Society for Political Methodology, University of Rochester, Rochester, NY, July 23.
- Druckman, James N. 2003. “The Power of Television Images: the First Kennedy-Nixon Debate Revisited.” The Journal of Politics 65: 559–71.
- Druckman, James N., Donald P. Green, James H. Kuklinski, and Arthur Lupia. 2006. “The Growth and Development of Experimental Research in Political Science.” American Political Science Review 100: 627–35.
-
Druckman, James N.,
Donald P. Green,
James H. Kuklinski, and
Arthur Lupia., eds. 2011. Cambridge Handbook of Experimental Political Science. New York: Cambridge University Press.
10.1017/CBO9780511921452 Google Scholar
-
Druckman, James N., and
Lawrence R. Jacobs. 2015. Who Governs?: Presidents, Public Opinion, and Manipulation. Chicago: University of Chicago Press.
10.7208/chicago/9780226234557.001.0001 Google Scholar
-
Druckman, James N., and
Justin W. Holmes. 2004. “Does Presidential Rhetoric Matter? Priming and Presidential Approval.” Presidential Studies Quarterly 34: 755–78.
10.1111/j.1741-5705.2004.00222.x Google Scholar
-
Druckman, James N., and
Cindy D. Kam. 2011. “Students as Experimental Participants: A Defense of the ‘Narrow Data Base.’ ” In Cambridge Handbook of Experimental Political Science, eds. James N. Druckman, Donald P. Green, James H. Kuklinski, and Arthur Lupia. New York: Cambridge University Press, 41–57.
10.1017/CBO9780511921452.004 Google Scholar
- Druckman, James N., Thomas J. Leeper, and Kevin J. Mullinix. 2014. “The Experimental Study of Legislative Behaviour.” In Oxford Handbook of Legislative Studies, eds. Shane Martin, Thomas Saalfeld, and Kaare Strøm. New York: Oxford University Press, 194–210.
-
Dunning, Thad. 2012. Natural Experiments in the Social Sciences: A Design-Based Approach. New York: Cambridge University Press.
10.1017/CBO9781139084444 Google Scholar
- Edwards, George C. 2003. On Deaf Ears: the Limits of the Bully Pulpit. New Haven, CT: Yale University Press.
- Eggers, Andrew C., Anthony Fowler, Jens Hainmueller, Andrew B. Hall, and James M. Snyder. 2015. “On the Validity of the Regression Discontinuity Design for Estimating Electoral Effects: New Evidence From over 40,000 Close Races.” American Journal of Political Science 59: 259–74.
- Erikson, Robert S., and Laura Stoker. 2011. “Caught in the Draft: The Effects of Vietnam Draft Lottery Status on Political Attitudes.” American Political Science Review 105: 221–37.
- Feller, Avi, and Chris C. Holmes. 2009. “Beyond Toplines: Heterogeneous Treatment Effects in Randomized Experiments.” Unpublished manuscript, Oxford University.
- Frye, Timothy, Scott Gehlbach, Kyle L. Marquardt, and Ora John Reuter. 2017. “Is Putin's Popularity Real?” Post-Soviet Affairs 33: 1–15.
- Gaines, Brian J., James H. Kuklinski, and Paul J. Quirk. 2007. “The Logic of the Survey Experiment Reexamined.” Political Analysis 15: 1–15.
- Gaines, Brian J., James H. Kuklinski, Paul J. Quirk, Buddy Peyton, and Jay Verkuilen. 2007. “Same Facts, Different Interpretations: Partisan Motivation and Opinion on Iraq.” The Journal of Politics 69: 957–74.
- Gerber, Alan S., James G. Gimpel, Donald P. Green, and Daron R. Shaw. 2011. “How Large and Long-Lasting are the Persuasive Effects of Televised Campaign Ads? Results from a Randomized Field Experiment.” American Political Science Review 105: 135–50.
- Glynn, Adam N. 2013. “What Can We Learn with Statistical Truth Serum? Design and Analysis of the List Experiment.” Public Opinion Quarterly 77: 159–72.
- Hollibaugh, Gary E., Jr. 2017. “Presidential Appointments and Policy Priorities.” Social Science Quarterly 98: 162–84.
-
Howell, William G. 2003. Power Without Persuasion: The Politics of Direct Presidential Action. Princeton, NJ: Princeton University Press.
10.1515/9781400874392 Google Scholar
-
Howell, William G. 2009. “ Quantitative Approaches to Studying the Presidency.” In The Oxford Handbook of the American Presidency, eds. George C. Edwards III and William G. Howell. New York: Oxford University Press, 9–29.
10.1093/oxfordhb/9780199238859.003.0002 Google Scholar
- Iyengar, Shanto, and Donald R. Kinder. 1989. News That Matters: Television and American Opinion. Chicago: University of Chicago Press.
-
Kalinin, Kirill. 2016. “The Social Desirability Bias in Autocrat's Electoral Ratings: Evidence From the 2012 Russian Presidential Elections.” Journal of Elections, Public Opinion and Parties 26: 191–211.
10.1080/17457289.2016.1150284 Google Scholar
- Kane, James G., Stephan C. Craig, and Kenneth D. Wald. 2004. “Religion and Presidential Politics in Florida: A List Experiment.” Social Science Quarterly 85: 281–93.
-
King, Gary,
Robert O. Keohane, and
Sidney Verba. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton, NJ: Princeton University Press.
10.1515/9781400821211 Google Scholar
-
Kriner, Douglas L., and
Andrew Reeves. 2015. The Particularistic President: Executive Branch Politics and Political Inequality. New York: Cambridge University Press.
10.1017/CBO9781139839341 Google Scholar
-
Kriner, Douglas L., and
Eric Schickler. 2016. Investigating the President: Congressional Checks on Presidential Power. Princeton, NJ: Princeton University Press.
10.23943/princeton/9780691171852.001.0001 Google Scholar
- Levendusky, Matthew S. 2010. “Clearer Cues, More Consistent Voters: A Benefit of Elite Polarization.” Political Behavior 32: 111–31.
- Lewis, David E., and Abby K. Wood. 2012. “The Paradox of Agency Responsiveness: A Federal FOIA Experiment.” Working Paper 6-2012, Center for the Study of Democratic Institutions, Vanderbilt University, http://www.vanderbilt.edu/csdi/research/papers.php (accessed December 23, 2016).
- Light, Paul C. 1982. The President's Agenda: Domestic Policy Choice from Kennedy to Carter. Baltimore: Johns Hopkins University Press.
- Lockerbie, Brad, and Stephen A. Borrelli. 1990. “Question Wording and Public Support for Contra Aid, 1983–1986.” Public Opinion Quarterly 54: 195–208.
- McDermott, Rose. 2002. “Experimental Methods in Political Science.” Annual Review of Political Science 5: 31–61.
-
McDermott, Rose. 2011. “ Internal and External Validity.” In Cambridge Handbook of Experimental Political Science, eds. James N. Druckman, Donald P. Green, James H. Kuklinski, and Arthur Lupia. New York: Cambridge University Press, 27–40.
10.1017/CBO9780511921452.003 Google Scholar
- Mcfarland, Sam G. 1981. “Effects of Question Order on Survey Responses.” Public Opinion Quarterly 45: 208–15.
- Mondak, Jeffery J. 1993. “Source Cues and Policy Approval: The Cognitive Dynamics of Public Support for the Reagan Agenda.” American Journal of Political Science 37: 186–212.
- Mondak, Jeffery J., Christopher J. Lewis, Jason C. Sides, Joohyun Kang, and J. Olyn Long. 2004. “Presidential Source Cues and Policy Appraisals, 1981-2000.” American Politics Research 32: 219–35.
-
Morton, Rebecca B., and
Kenneth C. Williams. 2010. Experimental Political Science and the Study of Causality: From Nature to the Lab. New York: Cambridge University Press.
10.1017/CBO9780511762888 Google Scholar
-
Mullinix, Kevin J.,
Thomas J. Leeper,
James N. Druckman, and
Jeremy Freese. 2015. “The Generalizability of Survey Experiments.” Journal of Experimental Political Science 2: 109–38.
10.1017/XPS.2015.19 Google Scholar
-
Mutz, Diana C. 2011. Population-Based Survey Experiments. Princeton, NJ: Princeton University Press.
10.1515/9781400840489 Google Scholar
- Palfrey, Thomas R. 2009. “Laboratory Experiments in Political Economy.” Annual Review of Political Science 12: 379–88.
- Reeves, Andrew, and Jon C. Rogowski. 2015. “Public Opinion toward Presidential Power.” Presidential Studies Quarterly 45: 742–59.
- Reeves, Andrew, and Jon C. Rogowski. 2016. “Unilateral Powers, Public Opinion, and the Presidency.” The Journal of Politics 78: 137–51.
- Rose-Ackerman, Susan. 2017. “What Does ‘Governance’ Mean?” Governance 30: 23–27.
- Rosen, Corey M. 1973. “A Test of Presidential Leadership of Public Opinion: The Split-Ballot Technique.” Polity 6: 282–90.
- Rottinghaus, Brandon, and Kent L. Tedin. 2012. “Presidential ‘Going Bipartisan’ and the Consequences for Institutional Approval.” American Behavioral Scientist 56: 1696–717.
- Sekhon, Jasjeet S. 2009. “Opiates for the Matches: Matching Methods for Causal Inference.” Annual Review of Political Science 12: 487–508.
- Sigelman, Lee, 1981. “Question-Order Effects on Presidential Popularity.” Public Opinion Quarterly 45: 199–207.
- Sigelman, Lee, and Carol K. Sigelman. 1981. “Presidential Leadership of Public Opinion: From ‘Benevolent Leader’ to ‘Kiss of Death’?” Experimental Study of Politics 7: 1–22.
-
Sigelman, Lee, and
Carol K. Sigelman. 1986. “Shattered Expectations: Public Responses to ‘Out-Of-Character’ Presidential Actions.” Political Behavior 8: 262–86.
10.1007/BF01002101 Google Scholar
- Sigelman, Lee, and Dan Thomas. 1984. “Opinion Leadership & the Crystallization of Nonattitudes: Some Experimental Results.” Polity 16: 484–93.
-
Sirin, Cigdem V., and
Jose D. Villalobos. 2011. “Where Does the Buck Stop? Applying Attribution Theory to Examine Public Appraisals of the President.” Presidential Studies Quarterly 41: 334–57.
10.1111/j.1741-5705.2011.03857.x Google Scholar
-
Sniderman, Paul M. 2011. “The Logic and Design of the Survey Experiment.” In Cambridge Handbook of Experimental Political Science, eds. James N. Druckman, Donald P. Green, James H. Kuklinski, and Arthur Lupia. New York: Cambridge University Press, 102–14.
10.1017/CBO9780511921452.008 Google Scholar
- Sovey, Allison J., and Donald P. Green. 2011. “Instrumental Variables Estimation in Political Science: A Readers’ Guide.” American Journal of Political Science 55: 188–200.
- Streb, Matthew J., Barbara Burrell, Brian Frederick, and Michael A. Genovese. 2008. “Social Desirability Effects and Support for A Female American President.” Public Opinion Quarterly 72: 76–89.
- Tedin, Kent, Brandon Rottinghaus, and Harrell Rodgers. 2011. “When the President Goes Public: The Consequences of Communication Mode for Opinion Change Across Issue Types and Groups.” Political Research Quarterly 64: 506–19.
- Thomas, Dan, and Lee Sigleman. 1984. “Presidential Identification and Policy Leadership: Experimental Evidence on the Reagan Case.” Policy Studies Journal 12: 663–75.
- Van Bavel, Jay J., Peter Mende-Siedlecki, William J. Brady, and Diego A. Reinero. 2016. “Contextual Sensitivity in Scientific Reproducibility.” Proceedings of the National Academy of Science, www.Pnas.Org/Cgi/Doi/10.1073/Pnas.1521897113 (accessed May 29, 2016).
- Van De Walle, Steven, and Gregg G. Van Ryzin. 2011. “The Order of Questions in a Survey on Citizen Satisfaction with Public Services: Lessons from A Split-Ballot Experiment.” Public Administration 89: 1436–50.
- Villalobos, José D., and Cigdem V. Sirin. 2012. “Agenda Setting From the Oval Office: An Experimental Examination of Presidential Influence over the Public Agenda.” International Journal of Public Opinion Research 24: 21–41.
- Villalobos, José D., and Cigdem V. Sirin. 2017. “The Relevance of Emotions in Presidential Public Appeals: Anger's Conditional Effect on Perceived Risk and Support for Military Interventions.” Presidential Studies Quarterly 47: 146–68.
-
Zaller, John. 1992. The Nature and Origins of Mass Opinion. New York: Cambridge University Press.
10.1017/CBO9780511818691 Google Scholar
