Research ethics beyond the IRB: Selection bias and the direction of innovation in applied economics
Abstract
Principles for ethical behavior in the context of research are codified into rules that may change over time to meet peoples’ needs in specific institutions, including universities and professional associations. This paper aims to spark discussion about a set of ethical choices beyond those addressed by an IRB or recent association policy statements. Our specific focus is topic selection and the role of researchers’ interests and incentives in determining the kinds of research that we do. Using the principle of induced innovation, we show how changing incentives can influence the direction of research effort and thereby affect the kinds of policies or technologies that are supported by available evidence. With this paper, we hope to generate discussion among applied economists about selection bias in research and how we can use insights from economics itself to guide topic selection.
JEL CLASSIFICATION
A11; A14; B41; Q00
Social norms about ethical behavior can change over time and differ among individuals and groups. Among moral philosophers such as Rawls (1971), Nozick (1974), Scanlon (2000), and Parfit (2013) there are deep divisions but also broad agreement about what constitutes ethical behavior. When general principles are codified into rules for specific institutions such as universities, governments, and professional associations, seeking common ground and adapting to new circumstances can help individuals overcome past differences to achieve a common purpose.
This paper focuses on topic selection in research to spark discussion among applied economists about a specific kind of choice that may have ethical dimensions that are not typically made explicit. Our primary interest is how applied economists’ research topics might affect the direction of institutional and policy change in ways that may be similar to how the direction of technological change is influenced by topic selection in agronomic and veterinary research. Our interest in topic selection goes further than the longstanding concerns about the influence of research funders and conflicts of interest to address the possibility of selection bias when researchers are free to choose their own hypotheses, datasets, and methods.
These ethical dimensions of topic selection go beyond conflicts of interest and the work of any organization's institutional review board (IRB), as well as rules regarding conflict-of-interest disclosure, or transparency, reproducibility, and replicability.1 We focus on societal incentives in the market for ideas that might unconsciously influence the direction of research in the applied economics profession as a whole. As in economics generally, our interest is in the interaction between individual decision making and social choice, focusing especially on modifiable factors that drive research outcomes and the adoption of new ideas.
The paper begins by describing the life cycle of research activities to highlight how decisions made early in the life cycle have profound effects on the scientific evidence available to policymakers and the public. We then consider the drivers of topic selection and study design alongside influences on research implementation. To fix concepts, we develop a simple model of induced innovation in the development of research output and explore its impact on applied economics research. We carry these ideas forward to explore how selection bias at the early stage of the research life cycle manifests itself in the dissemination and impact of applied economic research. Finally, we note how the market for ideas forms incentives for outcomes at the end stages of research, which feed back into the early stages of developing new research projects.
Explicit discussion of research ethics beyond the IRB and other institutional regulation is helpful for a number of reasons. A first reason is the value of learning about and overcoming our individual cognitive biases. The rise of behavioral economics has trained us to look for systematic errors in how others evaluate their choices, revealing patterns with deep psychological roots including present bias, availability heuristics, and motivated reasoning. It is instructive to turn that lens inward, to address our own failures and shortcomings, and to reexamine our choices in the light of more information. Kuhn (1962) provides an early application of what we now call motivated reasoning to scientific activity, arguing that researchers tend to interpret data within existing paradigmatic structures and fail to pursue, or even discard, other kinds of data.
A second reason why research ethics merits further discussion is that the research environment for applied economics is changing rapidly. We collect vastly more data than ever before, and advances in computing power allow for easier data use but also easier data misuse, such as that which occurs through p-hacking and data dredging. Beyond computing, the academic environment is shifting: researchers are increasingly rewarded for attracting media attention and have an ever-wider array of outlets to reach popular audiences and policymakers (Scheiber, 2015). New arrangements for scholarly publishing have sharply altered how research is disseminated, and new institutions, such as public-private partnerships, alter how it is funded. As Josephson and Michler (2018) observe, new ethical dilemmas have already begun to be met by new ethical solutions, creating a need for these solutions to be critically evaluated, discussed, and debated.
A final reason why research ethics remains important is the multiplicity of societal demands being met by research, and the need for researchers to serve multiple stakeholders and audiences. This is particularly true for applied economists, who frequently conduct research within land grant institutions designed to serve a specific interest group within society, and within a profession that aims to take account of many other interest groups as well. A central challenge is how to be both mission driven and responsive to one particular clientele (be they farmers, business owners, or governments) while also being true to ourselves and others in society. Increases in transparency and social media coverage of academic life, as well as reliance on funding sources that demand highly visible impacts, creates more opportunity for real or perceived conflicts among our various constituencies. The choice of where to stand, given where we sit, is difficult and offers no easy solutions.
THE LIFE CYCLE OF RESEARCH
On those occasions when research ethics are discussed in the social sciences, and particularly in agricultural and applied economics, the focus is primarily on the conduct of research. For powerful and intrenched historical reasons, institutions have invested heavily in steps to protect participants in research projects (Josephson & Smale, 2021). Examples include the mandatory completion of courses provided by the Collaborative Institutional Training Initiative (CITI, 2020), mandatory approval of project proposals by one or more IRBs, and compliance with other rules regarding publication and disclosure (Josephson & Michler, 2018).
Given the importance of conduct and misconduct within research projects, the prevailing wisdom seems to be that if a researcher has received ethical approval from their IRB and complied with other rules, then ipso facto the research is ethical. But IRBs are focused on protection of research participants and are often motivated by legal concerns as much as ethical judgments (Schrag, 2010). Similarly, publication policies apply only at the publication stage, long after most of the critical research decisions have been made. There are numerous other stages in the life cycle of a research idea that have, by and large, been omitted from the discussion of research ethics.
As a way to examine research ethics, we begin by laying out the life cycle of a research idea (Figure 1). We divide the life cycle into six stages, recognizing that decisions at each stage may be made simultaneously rather than sequentially. Although these stages are interconnected and overlapping in real life, arraying them along a timeline is useful for ease of exposition. The life cycle starts with the genesis of an idea and the formulation of a research design that will allow the researcher to test a hypothesis. The second stage of the life cycle is project development in which the researcher builds their research team and seeks funds to support the work. This is followed by the data collection, data analysis, and dissemination stages, which may involve some overlap, depending on the project. The final stage of the life cycle is the impact that the research idea may have, and this can extend far into the future, long after the researcher has moved on to other ideas. The life cycle presented is, of course, a stylized rendering of the research process and any given research project need not map exactly to this structure. We discuss each of these stages in more detail, highlighting some of the ethical considerations in each.
The life cycle of research begins with the germ of an idea. The researcher seeks to develop this germ into actionable research by formulating an appropriate research design. In recent years, applied economists have begun using pre-analysis plans to help guide research design and help ensure ethical, transparent, and reproducible research (Janzen & Michler, 2021). As Angrist and Pischke (2009) suggest, steps taken at this initial stage narrow the range of questions that can be addressed later. Another key determinant of which research ideas grow and which ones fail to flourish occurs at the second stage, when the researcher seeks external funding for the project. Ideas attract or fail to attract funding for a number of reasons, which we address in subsequent sections. The ability to attract funding for a project often determines what research questions progress into research projects.
Once a research idea has been developed into a project and funding has been secured, the next stage in the life cycle is the collection of data. At this point, the researcher typically begins to interact with their research subjects. Though there are innumerable types of research subjects, due to the breadth of research in the field of applied economics, the process and procedures of these interactions are generally governed by IRBs. Many view IRB policies as burdensome and a poor fit for much of social science research (Josephson & Smale, 2021; Schrag, 2010; van den Hoonaard & Hamilton, 2016). Yet, the lack of reported ethical violations of subjects’ rights in economic research suggests that IRB policies are reasonably effective at ensuring participant protection and confidentiality.
The fourth stage in the life cycle of a research idea, analysis, frequently overlaps with the fifth stage, dissemination. Initial results can be presented at conferences, seminars, and workshops where feedback is sought to improve the analysis and perhaps expand the initial research idea and objective. During this stage of the life cycle, the researcher will interact with colleagues at academic events and begin the process of publication. Both of these activities are governed by codes of conduct and policies, both explicit and implicit (Lybbert & Buccola, 2020). Codes of conduct stipulate behavior at institutional and association sponsored events, and are more concerned with the ethics of interpersonal interactions than with the ethics of research (AAEA, 2019; AEA, 2018). Additionally, publication policies define a variety of criteria that must be met for research to be deemed ethical and thus acceptable for publication. Currently, the American Journal of Agricultural Economics has policies that define pre-publication, financial disclosure, replication, research involving human subjects, and plagiarism.2
The last stage of the life cycle of a research idea is the impact it has on researchers and media, government, policymakers, industry, and, finally, the public. Generally, this impact begins during the process of dissemination with fellow researchers and the scientific community. In recent years, the media has played an increasingly important role in disseminating research, amplifying and impacting the policy and outcomes that result from research work. Administrators at universities and other institutions have taken notice and begun rewarding researchers who gain fame by appealing directly to the media. It may be the case that the push at research institutions for “impact” beyond the scientific contribution is changing how and what research is done.
Within this life cycle, the researcher is required to, at several stages, verify their adherence to explicit ethical criteria. Without IRB approval or signed conflict of interest statements, research is likely to die prematurely. However, there are many other stages in the research life cycle that are not governed by explicit ethical criteria, though implicit norms within the field may exert influence on the researcher. Potentially the most critical stage in the life cycle is the initial one, where ideas are formed and research is designed. Many of the considerations and pitfalls of ethical behavior that a researcher faces during the life cycle arise at this earliest of stages. In the subsequent sections, we use the life cycle of research as a framework to investigate specific stages of research in more depth.
INDUCED INNOVATION IN RESEARCH; OR, BE CAREFUL WHERE YOU LOOK
As we focus on research ideas in this paper, we naturally turn our attention to the early stages of the research life cycle. This allows us to investigate ethical issues that arise in the initial idea and project development stages of applied economics research. We focus on ethics in these early stages for two reasons. First, ethical issues arising at this stage are least likely to be governed by institutions. Idea and project development occur prior to review by IRB or journal editors and reviewers, and would only be tangentially touched by professional codes of conduct, which are typically concerned with interaction at association events that come later in the research life cycle. Second, there can be significant path dependency in the research life cycle. Decisions made during the initial idea and project development stages will substantially impact decisions and outcomes at later stages. This means that reflection on ethical issues arising at these early stages can be particularly fruitful for researchers.
The key ethical issue arising in the initial idea and project development stage concerns which research questions get asked (and funded) and which questions do not. If research ideas and funding decisions were made at random, we could be confident that, on average, applied economics research represented the true state of the world. Although this might generate unbiasedness in funding for research, it would clearly also be inefficient in that the system would fund much worthless research. However, funding decisions are not made at random. Both researchers and funding agencies select which research ideas to propose. The fact that non-random selection exists suggests that the scientific record may be over-represented on some topics and under-represented on others.
A policeman sees a drunk man searching for something under a streetlight and asks what the drunk has lost. He says he lost his keys, and they both look under the streetlight together. After a few minutes the policeman asks if he is sure he lost them here, and the drunk replies no and that he lost them in the park. The policeman asks why he is searching here, and the drunk replies, “this is where the light is.”
Researchers are often accused of conducting a drunkard's search, looking for answers only in the most convenient places, even when we know the truth lies elsewhere. But if this were researchers’ only bias, we would just not discover very much.
A more interesting kind of selection bias in research and policymaking follows the same kind of selection bias that we have observed in technological change. We have taught generations of students the theory of induced innovation: the direction of research and development (R&D), as well as technology adoption, is often driven by price changes, thereby shaping the direction of innovation and resource use (Binswanger, 1974; Hayami & Ruttan, 1985; Hicks, 1932). Much of the literature on induced innovation focuses on input use. For example, where labor is becoming more expensive relative to capital and fuel, innovators will develop labor-saving, fuel-using machinery. When irrigation and fertilizer prices are falling, farmers and crop breeders will look for water-using, fertilizer-hungry new varieties. When innovators anticipate higher costs of carbon emission, low-carbon technologies will be developed and adopted. We are also interested in induced innovation among outputs. When entrepreneurs anticipate rising consumer demand for alternative meats, they will develop lower cost and higher quality items in that product category.
Figure 2 adapts the induced innovation model to the research topics of applied economists. Along the horizontal axis is a community's potential output of public goods and services, such as schooling or infrastructure. Along the vertical axis is their potential output of private goods and services, such as clothing or houses. There may be some increasing returns when shifting resources out of one sector into the other, so the production possibilities frontier (PPF) may not be concave everywhere. This community's cost of taxing private activity to fund public goods is shown by the slope of its revenue line, which shows the total level of income as the sum of private and public activity. In public finance, that slope is known as the cost of public funds. If the society uses its resources efficiently, it will move along its PPF to the highest possible level of income, where the cost of public funds just equals the slope of that PPF.
No one imagines that a two-dimensional model of public finance could be anything more than a crude cartoon, but the sketch is useful for a simple thought experiment. If innovators expect the cost of public funds to be higher than in the past, they will look to save public funds and improve the market for private goods. Conversely, if innovators believe that the cost of public funds could be lower than in the past, they will research ways of providing more public goods and services. A historical analogy to this thought experiment is the focus on a smaller federal government that started around 1980, aiming to end the high inflation seen during the 1970s with monetary contraction and fiscal restraint. Many factors contributed to the desire of many Americans for lower public expenditure in recent decades, including distributional concerns about who was benefiting from that spending, but recent changes suggest that American voters and political leaders may now be looking to spend more rather than less of our national income on public goods (Long, 2020; Tankersley, 2020).
The story of induced innovation begins with price changes but includes the way in which research institutions and funding agencies do or do not transmit price signals to individual researchers, research projects, and the flow of innovation from discovery to application. At the prevailing cost of public funds in America, whose beliefs about the desired future size of the public sector will drive the path of innovation? How do we decide which type of research to pursue, and what type of evidence about social welfare and cost effectiveness we might discover? The market for ideas is sometimes influenced by a measurable variable such as the cost of public funds, but it has a complex structure with many different kinds of incentives. A practical example of research and innovation taking different directions despite little difference in relative prices is illustrated by Figure 3, in which a familiar kind of technology took two very different directions in recent years. The figure graphs the change in maize yields over time for the United States and France. Over the 18-year period from 2000 to 2018, maize yields in the U.S. grew by 164 kg/ha each year, a statistically significant annual increase. By comparison, maize yields in France were essentially static. Maize yields grew, on average, by 44 kg/ha each year, not significantly different from zero. There are some differences in relative prices between the U.S. and France, but the different technological trajectories are ultimately due to very different political judgments about what kinds of crop technology are socially desirable.
Society plays a role in influencing institutional priorities, which in turn set agendas for R&D. This priority setting induces innovation in technological change, ideally resulting in socially desirable outcomes. Although this is likely true for any R&D process, including private-sector firms directing R&D based on expected market sentiments, it is surely true for public-sector institutions, such as government research agencies (NSF, NIH, NIFA, etc.) and state universities. Voters elect representatives who then make funding decisions and set priorities for research. At land grant universities, the core mission is to provide education, produce research, and disseminate that research, often via extension, to the community, with a focus on agriculture, science, and engineering. This makes the community, and in the case of agricultural and applied economists, farmers, a key constituency in setting the R&D agenda for land grant universities, outside of community members’ role as voters. The reduction of state funding at many land grant universities has given rise to academic freelancing for industry or foundations. This has meant that private sector concerns play a larger role in guiding R&D decisions of those in the academy. None of this is a criticism of the status quo; rather, it is an acknowledgment of the many stakeholders seeking to induce innovation in one direction or another.
The influence of priority setting also goes in the opposite direction. Applied economics research shapes knowledge about options for public policy, which is then used to set new priorities for governments and institutions, both public and private. The applied economics researcher is not fully at the whims of institutional decision making in choosing which path for public policy should be investigated. If, in the research process, we sometimes find what we seek, we should be careful what we look for.
THE MARKET FOR IDEAS
The principle of the drunkard's search, that where we look influences what we find, raises three important questions. First, to what extent do (or should) a researchers’ own personal priorities define where to look? Second, to what extent do (or should) researchers follow funder priorities in defining where to look? Finally, to what extent do (or should) other institutions that purport to speak for the combined public interest define where to look? By engaging with each of these questions, we seek to investigate the market for research ideas and how it is influenced by our own interests, the interests of those funding our work, and the interests of those with a broader public stake in our work.
Personal priorities
We start by examining the extent to which a researcher should follow their personal priorities in choosing where to look for new ideas. There are three primary ways through which personal priorities and perspectives can create selection bias in the ideas that are pursued. The first is through a researcher's own tastes and preferences. If a researcher holds a particular taste or preference, that partiality may impact the research questions they ask and the outcomes they publish. Ioannidis and Trepanowski (2018) argue that, in addition to disclosing financial conflicts of interest, researchers should disclose conflicts of interest arising from their personal tastes, preferences, advocacy, and/or political persuasion. They present the case of a hypothetical nutritionist who adheres to the Atkin's diet and undertakes research that demonstrates the nutritionist's dietary choices are in fact beneficial. In this case, the nutritionist has a personal conflict of interest and should disclose their dietary habits. As Ioannidis and Trepanowski (2018) write, “such disclosure should not be seen as an admission of lack of integrity. To the contrary, disclosure strengthens the perceived integrity of the author.” The concern is that our tastes and preferences might influence the type of research questions we ask and the results that we find, meaning that this potential bias is disclosure-worthy information.
Although bias resulting from a researcher's tastes and preferences can be attenuated through conflict of interest disclosure, it is often complicated by implicit bias in the researcher. As economists who work primarily in developing countries, this is an issue with which all three of the current authors struggle. Both implicit bias and a lack of identity and lived experience have profound consequences on minorities peoples of all types. In the academy, this includes everything from the systematic and persistent omission of minority people from medical and health studies (Nazha et al., 2019; Scharff et al., 2015) to the omission of left-handed people from neurological studies (Willems et al., 2014). A detailed discussion of this problem and its implications for selection bias in applied economic research requires a paper, or papers, of its own.3 It is important to acknowledge that these issues may create implicit biases that lead researchers to select topics, which leave marginalized people under-represented.
The final way that researchers’ priorities and perspectives may impact where they look for ideas is through the recent push for “impact” in areas other than scientific contributions. That is, the area at the far end of the research life cycle, where dissemination impacts policymakers and the public at large. As researchers we all want our work to influence those outside of our narrow field of work. Yet, the growing incentives for impact beyond the scientific contribution of the work may introduce selection bias into the research process. The website RePEc/IDEAS now publishes rankings for economists based on the most Twitter followers, in addition to its ranking of economists with the most citations. The AAEA rewards outside press attention through the weekly Members in the News announcements. Together, these trends suggest that there may be expansive career rewards for simply garnering media attention, regardless of the quality or substance of the research. Because of the shifting incentives toward attention beyond publications in academic journals, prestige outside of academia plays an increasingly large role in shaping what research gets done. As a result, researchers are subject to these influences in determining where they look for research ideas and what they find. Of course, this attention may be a double-edged sword. A recent example within our profession is Dr. Brian Wansink, formerly a member of the Applied Economics and Management Department at Cornell University, who attracted substantial press attention through a series of clever and highly influential studies around food and nutrition (Engber, 2018; Lee, 2018). Ultimately these studies, once touted in newspapers, magazines, and TED Talks, drew ire from the press as evidence of misconduct emerged and a number of studies were retracted. Because of the shifting incentives toward attention beyond publications in academic journals, prestige outside of academia plays an increasingly large role in shaping what research gets done. As a result, researchers are subject to these influences in determining where they look for research ideas and what they find.
Funder priorities
We next consider the extent to which a researcher should follow funder priorities in choosing where to look for new ideas. Although good research can be done without external funding, funding can open opportunities to new, and hopefully better, research. In addition to new research opportunities, funding may also provide career opportunities. In many universities and other research organizations, grantsmanship is becoming a key component in evaluation for promotion and tenure. As a result, there are strong incentives to look for new ideas in areas that directly appeal to funder interest. Of course, this presumes that researchers would do different research if funding was unlimited and unconstrained. Nor should this be taken to suggest some purity in unfunded research ideas. Rather, it underscores the existence of the principle of the drunkard's search in our research process: we search where the light (funding) is.
Funding for sponsored programs comes primarily from three sources: public funding from governments, private funding from industry, and philanthropic funding from organizations. Researchers may choose to pursue research ideas that appeal directly to a funder's interests, even if the idea has little scientific merit. Alternatively, the funding organization may prioritize certain topics, not out of any scientific interest but in order to build a portfolio of evidence in support of the funder's priorities and perspectives. These two sources of selection bias may result in a redirection of research effort. Ultimately, the types of innovation induced by this sort of selection bias may be socially sub-optimal.
Governments have traditionally been the largest source of research funds. In an ideal world, public funding would perfectly represent some social optimum. Yet, public funding may be misallocated due to governance failures in representing public interests. These failures come from a variety of sources. They may be due to regulatory capture or capture by the current electorate, which discounts the government's mandate to protect the citizenry, which is long lived. A related potential bias in public funding is that governments tend to view change as a negative and that their incentive is for the continuation of the status quo. Thus, governments can seek to mitigate departures from prevailing norms and procedures. This suggests that researchers need to consider the bias for research that preserves the status quo created by where government funding shines its light.
Although research priorities for industry can be intertwined with government through regulatory capture, there are additional concerns when considering funding from industry. Many of these have been covered by Zingales (2013) in his discussion of economist capture. Bias in research associated with funding from industries may result from a quid pro quo in which a researcher, either implicitly or explicitly, forms an agreement with the funder regarding the type of results that the research will generate. The funder provides resources with the expectation that the researcher will produce results in line with the funder's business interests. Although the most common occurrences of this sort of capture involve medical research (e.g. Lexchin et al., 2003), there are documented cases in economics and agriculture. Zingales (2013) and the Academy Award winning Inside Job both document the existence of implicit quid pro quos between economists and the financial industry. Similar evidence has emerged in the agrochemical sector (e.g. Hakim, 2017; Marcus, 2018). This by no means suggests that biases do not result from other funding sources, but rather, researchers must reflect on the degree to which the ideas they search for are a function of where funding is shining a light.
Finally, philanthropic funding itself can be intertwined with industry and government funding. More than government or industry funding, philanthropic funding may be driven by the interests—or whims—of the donor. The philanthropic interests of the donor may, in turn, affect the types of questions asked by the researcher. As philanthropic funding plays are larger and larger role in financing research, concerns have arisen about the influence donors may exert on research outcomes (Mervis, 2017; NSF, 2016). Potential bias may exist whether taking money from the Bill and Melinda Gates Foundation, the Chan Zuckerberg Initiative, the Clinton Foundation, the Charles Koch Foundation, or any other philanthropic agency. What is important to consider is the extent to which, as independent researchers, we choose to pursue ideas, not because of any scientific value, but because they speak to the idiosyncratic interests of a given donor.
Public priorities
Given the preceding arguments, one might be inclined to conclude that the ethical researcher should shun their personal interests as well as the interests of funding agencies. What is left, then, is to only look for new ideas that maximize social welfare. But, to what extent is this feasible or desirable? Economists know better than most the strong assumptions necessary to aggregate individual utilities into that of a representative agent. Even if only searching for new ideas that were in the best public interest was feasible, would it be efficient for researchers to completely sacrifice their autonomy in searching for new discoveries?
Both medical and agricultural research provide clear examples of the tension between individual or funder research priorities and social welfare. In terms of medicine, using the public interest to define where researchers look for new ideas would mean placing corporate health ahead of individual health. The result would be a myopic focus on diseases that afflict the largest number of people to the detriment of all else. How to strike the right balance between medical research, which tends to focus on the health of a single person, with public health research, which is concerned with the health of the entire population, is an ongoing debate in bioethics (Buchanan & Miller, 2006). Obviously, a cure for cancer or malaria would impact millions of more people than the cure for any given orphan disease, which typically afflicts less than 200,000 people. Yet, few would argue that a researcher looking to cure a rare childhood illness is unethical because they are not looking for a cure for cancer. A similar argument exists in the agricultural sciences. Norman Borlaug's research has saved millions of lives, yet a researcher who looks for new ideas to improve a specialty crop is not unethical because they are not engaged in research on staple crops.
Further, it is not obvious that a world in which ethical norms dictate that researchers only look for ideas that had the potential to maximize social welfare would be any better off than a world in which researchers could exert some autonomy regarding the research they conduct. The reason is well known to economists from the principle-agent literature. If the researcher is the sole proprietor of their research, there are no incentive alignment problems. The researcher maximizes their own benefits from research while also bearing all the costs. The researcher can choose to search for new ideas that interest them most, maximizing their benefit while lowering any perceived costs to the research process. However, if there is some principle maximizing social welfare from research, incentive alignment problems may arise. The researcher may engage in research that the socially minded principle finds valuable but that has a high subjective cost to the researcher, relative to pursuing their own research interests. The principle then must design a mechanism to incentivize the researcher to put in effort or face the efficiency loss from the misalignment of incentives. Although far from a formal model of research efficiency, the intuition should be clear. Researchers got into the scientific endeavor for a variety of personal reasons, most of which involve the ability to ask questions that they find interesting. As we have discussed, following personal priorities may result in selection bias. However, a cadre of researchers forced by ethical norms to search for ideas they find uninteresting is hardly a recipe for efficient production of new research ideas.
As the principle of the drunkard's search suggests, where we look influences what we find. For researchers in agricultural and applied economics it is imperative that we evaluate the ethics of our research process. This involves the balancing of competing research interests, each of which have their own perspectives. A failure to reflect on the potential selection bias induced by personal or funder priorities can result in outcomes that move society in an unintended direction. This is not to say that the only ethical researcher is the one who pursues hoped-for discoveries. Looking only for what one hopes to find is a form of blindness. Even if one applies rigorous methods and avoids confirmation bias, researchers who seek only a certain kind of discovery may miss other opportunities. Individual researchers can and should investigate many questions beyond their own interests. But a researcher's interests matter, so having a heterogeneous research community pursuing diverse hypotheses is likely to result in better, less-biased research than adherence to a single set of research objectives.
FINAL THOUGHTS
In this paper, we aim to spark and encourage discussion among applied economists regarding the ethical principles we might share, beyond existing rules that institutions, such as IRB, establish for us. In particular, we focus on where new research ideas come from, the role of induced innovation in economic research, and how competing priorities may induce selection bias in which ideas are pursued.
Selection bias in research drives which kinds of research gets done and, as a result, which policies have evidence of success or failure. When researchers look for X, they might find it. Without a similar effort to find Y, all evidence will point to X. Unless we as researchers reflect on the potential for selection bias in the research ideas we choose to pursue, the resulting sample of published evidence will fail to accurately represent the underlying scientific truth. Selection bias in policy communication compounds these problems, driving which evidence is widely known and available. When policy platforms reward X we will talk about it. Without similar emphasis on Y, all available evidence points to X.
This form of induced innovation in research and policy is similar to induced innovation in technological change. Turning the lens of agricultural economics back onto our own objectives suggests opportunities to discuss the ethics of choosing what we look for—because we might find it.
ACKNOWLEDGMENTS
We are particularly grateful to Marc Bellemare, Sara Folta, Travis Lybbert, Gerald Shively, Melinda Smale, Parke Wilde, Norbert Wilson, and the AEPP editor, Craig Gundersen, who were all very generous with their time and provided helpful thoughts and comments on the paper.
