Do Policy Makers Use Academic Research? Reexamining the “Two Communities” Theory of Research Utilization

Universities are powerhouses of research, but by many accounts, policy makers do not use academic research to its fullest potential (Nutley, Walter, and Davies 2007). Over many decades, the study of how policy decisions can be based on—or impervious to—the outputs of academic research has grown, inspiring subgenres with names such as “research utilization,” “knowledge transfer,” “knowledge brokering,” and “evidence-based policy.” Much of this international literature assumes that the knowledge produced by university researchers has value and merit and should be consumed more heartily than it currently is by those who contribute directly to the decisions that govern society (e.g., Banks 2009, 16). The goal, then, is to discover how academic research can be used more effectively to inform the formulation and implementation of public policy.

In the 1970s, early scholars of research utilization argued that policy makers and academics formed two separate communities that were poorly connected and operated under different rules, spoke different languages, and were motivated by different rewards systems (Caplan 1979; Dunn 1980). This “two communities” construct has been adopted by many as a way to describe the disconnect between the worlds of academia and policy (e.g., Carden 2004, 138; Edwards 2005, 68; Lavis, Ross, and Hurley 2002, 145). However, although it is generally accepted that the supply of academic research greatly exceeds its demand among policy makers, the reasons for this imbalance are not widely agreed upon, and the most effective strategies to improve the use of academic research have yet to be identified. Accordingly, the two communities theory of research utilization has faced some criticism (Bogenschneider and Corbett 2010; Jacobson 2007; Kalmuss 1981; Wingens 1990).

This article aims to add to the international debate by exploring the validity of the two communities construct in an empirical context. A survey of more than 2,000 policy workers and in-depth interviews with 126 public servants at the state and federal levels in Australia are used to explore the relationship between academics and policy makers and to generally evaluate the two communities approach to understanding this relationship. While this article focuses on a single country, the results are likely to be generalizable beyond Australia, as aggregate responses to more general questions in the survey were similar to results from previous studies in Canada (Belkhodja et al. 2007), the United States (Lester 1993), and the United Kingdom (Talbot and Talbot 2014). Australian bureaucracies are similar to those in other developed democracies, and comparisons have been made in the past (e.g., Considine and Lewis 2003).

The survey and interview responses provide interesting results that pose a significant challenge for the two-communities approach. A substantial number of survey respondents indicated that they do not value academic research very highly and do not often use the findings of academic research when constructing advice for making policy, which, at first glance, seems to support the notion of distinct policy and research communities. However, a significant subset of respondents indicated that they do in fact use academic research for the purposes of policy making. Furthermore, the interview responses suggest that there is a range of opinion on the use of research and that attitudes toward the utility of academic outputs for policy can vary widely.

The Two-Communities Approach to Understanding Research Utilization

In the 1970s and 1980s, a body of scholarship began to emerge on the relationship between social science research and the formulation and implementation of public policy. The emphasis in those days was on how research could be used by policy makers (C. H. Weiss 1979), how different policy sectors might use research differently (J. A. Weiss 1979), and whether research is actually used by policy makers at all (Bardach 1984; Caplan, Morrison, and Stambaugh 1975; Kalmuss 1981). More recently, there has been a renewed international interest in the use of research in policy making, but this time the lens is focused on the policy end of the research–policy relationship. The evidence-based policy movement, as it is now referred to, increasingly encourages governments to be concerned with “what works,” especially since the U.K. government under Tony Blair first used this term as its slogan for public management reform (Legrand 2012; Sanderson 2003). Politicians (e.g., O'Malley 2014) and bureaucracies (European Commission 2010; OECD 2013; Productivity Commission 2010) have expressed an interest in increasing the use of research evidence to inform policy decision making, with the assumption that increased information will lead to more popular and more sustainable policy outcomes.

In spite of the more recent shift in attention from a focus on research to a focus on policy, the fundamental question is, and has always been, why is academic research not used more by policy decision makers? In attempting to answer this question, and following C. P. Snow's (1959) characterization of the humanities and the natural sciences as “two cultures,” Caplan (1979) separated the worlds of academic research and policy making in what he termed the “two communities” theory of knowledge utilization. According to Caplan, “government policy makers are action-oriented, practical persons concerned with obvious and immediate issues” while social scientists are intellectuals who are “concerned with ‘pure’ science and esoteric issues” (1979, 459). These qualities, according to the theory, are cultural and ingrained in the nature of the two communities, and they constitute the primary reason why academic research is not as valuable to policy making as it ultimately could be.

The notion that policy makers and academics have different incentives, rules, obligations, and interests has proven persuasive over the last 30 years. Edwards (2005, 68), for instance, argues that researchers often see governments as risk averse, too focused on the short term, anti-intellectual, and motivated by ideology, while governments see policy research as lacking relevance to current policy debates and to day-to-day issues in program delivery. Head (2010, 88) warns that the research and policy communities must overcome mutual ignorance and indifference if policy is to be based on research evidence. Frenk (1992, 1398) argues that researchers and policy makers communicate differently—researchers must communicate in complicated, scientific language, while policy makers need to make quick, accessible decisions that can be understood by everyone. Others have pointed to the different goals pursued by the two communities, as researchers aim to increase knowledge while policy makers’ objectives are more concretely attached to “operational matters” (Hemsley-Brown 2004, 539). It has also been suggested that although interaction between the communities may improve the uptake of academic research by policy makers, the communities are themselves resistant to interacting with each other (Lavis, Ross, and Hurley 2002, 145). In general, these authors believe that there are fundamental differences between researchers and policy makers and that these differences impede a process that would otherwise see academic research inform and influence policy directly and more abundantly (Raadschelders 2011, 144–45).

The two-communities thesis has been supported by numerous empirical studies in several countries, showing that policy makers do not appear to use or value academic research to its fullest potential. One study, in which 155 officials in U.S. mental health agencies were interviewed, found that instrumental use of academic research “seems in fact to be rare, particularly when the issues are complex, the consequences are uncertain, and a multitude of actors are engaged in the decision-making process” (Weiss 1980, 397). Another study, in which 113 American state-level public servants were surveyed about their use of academic research for policy making, found that “decision makers are reluctant to draw from university faculty and research organizations for their policy advice” (Lester 1993, 276). In Canada, a survey of 833 officials in federal and provincial government agencies, using Likert-scale questions about the use of academic research in policy making ranging from 1 (never) to 5 (always), found average scores as low as 2.15 for the adoption of research and 2.25 for the influence of research on policy outcomes (Landry, Lamari, and Amara 2003, 198). In another Canadian study, an analysis of survey responses from 1,512 policy workers in nine provincial governments found that Canadian policy workers “may not have the capacity required to practice a high level of evidence-based policy analysis and policy-making” (Howlett and Newman 2010, 131). In the United Kingdom, a survey of 340 senior civil servants from a variety of policy areas found that there is “a major disjunction … between what senior civil servants perceive as being important disciplines and what academic institutions have seen as important” (Talbot and Talbot 2014, 15). And in another survey of 916 managers in Canadian health services agencies, it was found that although most respondents admitted receiving academic research, a significant number also reported that “research never or rarely influenced their decisions” or that research “was never or rarely transformed into concrete applications” (Belkhodja et al. 2007, 405).

However, none of the studies cited found a total absence of research use by policy makers. In fact, in all of the studies, there were significant subsets of respondents who reported that academic research was valuable to them and had a practical influence on their policy advice and decision making. Some authors found that research use by policy makers even exceeded expectations (e.g., Talbot and Talbot 2014, 10). Furthermore, survey data from the United States suggests that policy makers may use research to different degrees depending on the questions they are asking, the level of risk involved, and the issue area in which they work (Hall and Jennings 2008; Jennings and Hall 2012). There is reason to doubt the validity of a theory, therefore, that models policy makers and academics as two homogenous groups with little interaction between them.

Despite challenging the two-communities thesis, previous studies do not go far enough. For instance, Webber's (1986) study, in which Indiana lawmakers were surveyed about the connection between job roles and research use, provides an interesting starting point but it is not broad enough to be generalizable to other kinds of policy makers. Much of the critical research in this area is from the 1980s and may require support from a more current empirical study. Of course, the fact that most of the initial work was done more than 30 years ago has not deterred scholars from continuing to use the two-communities approach to understand the use of research by policy makers (e.g., Lavis, Ross, and Hurley 2002, 145; Shonkoff 2000; Tseng and Nutley 2014; Ward, House, and Hamer 2009, 271).

Furthermore, much of the two-communities literature presents a policy community that is poorly defined. Most glaringly, the policy community is usually understood to include both politicians and bureaucrats. However, these two groups of actors have different rules, motivations, and organizational structures, such as constitutional rules rather than bureaucratic organization, incentives to seek reelection as opposed to those of job promotion, and adversarial party politics instead of hierarchy. For analytical purposes, the use of academic research in the bureaucratic and political elements of policy making should be examined separately (Page 2012).

In fact, even within the bureaucratic component of the policy community, it has been suggested that decision-making bureaucrats might work in various roles that would require them to use academic research in different ways. For example, Cunningham and Weschler (2002) argue that some policy workers are responsible for the technical problem solving of day-to-day policy issues, in which case they would not be required to engage with academic theory on public policy because they are occupied by short-term issues, technical policy emergencies, and problems of implementation. Others, whose job roles might involve innovative thinking and long-term strategic planning, would have to be familiar with policy theory and would need to know how to apply it to specific areas of policy formulation and implementation. Of course, by dividing the bureaucracy into binary subgroups in the above manner, one runs the risk of further enforcing the error of homogenization that the two-communities theory presents in the first place. As will be argued later, these internal differences in the roles of decision makers and their various levels of engagement with academic research should be more appropriately envisioned as existing along a spectrum rather than in a small number of distinct groups.

In general, then, the two-communities metaphor may not be a useful way of imagining the relationship between academic research and policy. Any way that the communities are defined, some subset of professional policy workers will report that they engage with academic research and consider it vital to their policy development work. We tested some of these matters in a large empirical study in order to shed some more light on these international debates.

The Survey and Its Methodology

Beginning in November 2011 and ending in March 2013, a research team at the University of Queensland conducted a survey of 2,084 public servants at the state and federal levels in Australia on their use of academic research in their policy-related work (for details, see Cherney et al. 2013; Cherney et al. 2015; Head et al. 2014). The survey was aimed at policy-relevant personnel working in government departments and public sector agencies that administer and deliver human service programs. A questionnaire was delivered electronically to public servants in 21 departments and agencies within the national government and the state governments of Queensland, Victoria, and New South Wales, which are the three most populous states and together represent more than three-quarters of the Australian population. In addition, one-hour semistructured interviews were conducted in person or by telephone with 126 participants representing a variety of policy areas and geographic locations.

It must first be acknowledged that the survey instrument does not use a probability sample. Ideally, the research team would have been able to obtain a list of all potentially relevant participants in the entire Australian public service and then would have selected some smaller number of participants randomly and in some way compelled them to complete the survey. In a practical world, this is simply not possible. For one thing, it is impossible (not to mention unethical) to compel people to participate in social research. For this reason, this survey and all similar studies must contend with the problem of self-selection bias.

Second, and relatedly, the research team was not able to obtain complete contact information for all of the potentially relevant participants. This is because the participating organizations were concerned with protecting the anonymity of their employees. In many cases, the survey was delivered to managers who then distributed the survey internally. This increases the problem of selection bias and also makes calculating response rates difficult, if not impossible. It also means that the sampling technique proceeded more like a census than a statistically representative probability sample (Fricker 2008).

However, there is good reason to consider the results of the survey to be empirically useful. The ready and enthusiastic cooperation of managers in the participating organizations suggests that they understood the purpose and utility of the project and likely directed the survey to appropriate personnel. In addition, the way in which the questionnaire and the instructions to the survey were worded made it difficult for unintended or inappropriate recipients to complete the survey and thereby bias the survey results in any meaningful way. Finally, if self-selection bias were a problem, one would expect that those who value academic research and use it in their policy work would be more likely to opt in to the study, which would skew the outcome by overrepresenting research users. In fact, as will be shown later, research users were decidedly in the minority among the survey's respondents. Therefore, even if self-selection resulted in an overrepresentation of research use, this has not necessarily altered the aggregate results of the survey.

Strictly speaking, then, the results of the survey component of this study should be interpreted as applying only to Australian policy workers who would be interested in participating in a survey on the use of academic research in policy making and should not be scientifically extrapolated to include people outside this population. However, given that the large quantity of respondents came from a diverse background of policy areas, jurisdictions, ages, and stages of their career, it is likely that this study's results should provide interesting suggestions beyond the immediate population of respondents. Furthermore, the method used in this study is probably the best method for conducting survey research on a sector that is structurally designed at an institutional level to protect individuals from being identified, that traditionally operates with a unified voice, and that is intended to be resistant to outside influence. These characteristics apply to the public service in most modern democracies, in Australia as well as internationally.

The Results of the Survey: Is the Two-Communities Thesis Supported?

The survey asked a number of questions about the use of academic research for the purposes of policy making. Many of the survey questions were adapted from survey instruments used in other countries in previous studies on research utilization, in particular the survey used by a team at Laval University in Quebec (see Amara, Ouimet, and Landry 2004; Belkhodja et al. 2007). The adapted survey instrument was then tested in a pilot study, and feedback from this pilot group resulted in further modifications. Questions that pertain to individual use of academic research in policy-related work and summaries of responses are shown in table 1.

It is clear from the results in table 1 that the public servants who completed the survey have access to academic research. The majority of respondents (58 percent) reported using electronic databases in addition to Google to download academic research, with only 30 percent reporting that they always or usually experience difficulty in accessing full-text versions of academic articles. Moreover, more than 60 percent of respondents reported drawing on academic research for use in their written documents.

However, while policy workers may have access to academic research, and they may even devote some space to it in their written work, it is less clear to what extent academic research ultimately influences policy (Head et al. 2014). A question asking whether academic research was used to improve the understanding of policy, with responses ranging from 1 (frequently) to 4 (never), resulted in a mean answer of 2.062, which is very close to “occasionally” and hardly an emphatic endorsement of academic research. In a series of questions with scaled responses ranging from 1 (always) to 5 (never), respondents were asked how often they receive relevant academic research, how often they cite academic research directly, and how often they adapt academic research for use in crafting policy. All of these questions yielded mean responses very close to 3.0, indicating, on average, no special inclination toward using research for policy making purposes. Of course, means are summary indicators only and do not reflect the spread of responses. In fact, only 28 percent of respondents indicated that they always or usually adapt academic research for informing policy.

A second series of questions asked participants to indicate their perceptions of the use of research in policy making on a scale of 1 (strongly agree) to 5 (strongly disagree). The responses in this section seem to suggest that pressures of scarce financial resources and short time frames provide little opportunity for long-term conceptual uses for the outputs of academic research (see table 2). For example, the mean response to the statements “Policy-making is driven by budgetary considerations” and “Responding to urgent day-to-day issues takes precedence over ‘long-term’ thinking” were both very close to 2.0, indicating that, on average, respondents agreed with these statements.

What is striking about all of these responses is that in very few cases was there a trend toward responses that might support a two-communities view of the policy–academia relationship. If policy and academia really were two separate communities with difficulties in communication and barriers to interaction, one would expect mean responses to the survey questions to converge on low levels of use of academic research in policy making. Instead, the survey responses trended toward very neutral responses, with no marked emphasis on either ignorance of academic research or structured uptake for policy making purposes. Mean responses trended toward the middle of the range of values, and there were substantial numbers of participants indicating responses across the spectrum in all areas.

The apparent suggestion, then, is that the two-communities theory is probably not the best way to understand the relationship between policy and academia. More likely, there is a range of activities involving the use of academic research in policy making, and different individuals will use research in different ways and to various degrees. In fact, this alternative view is supported by the responses given in the in-depth interviews.

Interview Responses: A Wide Range of Opinions Regarding Research

Semistructured interviews were conducted with senior managers and other highly experienced personnel representing a variety of public sector organizations and working in numerous policy areas. The interview sample was recruited in a number of ways. First, after the completion of the survey process within each respective agency, each agency contact assigned to help the academic team with recruiting the survey sample was invited to identify and nominate a small number of senior staff in relevant positions who were willing to participate in an in-depth interview. Not all of the selected interviewees had previously completed the survey. In addition, a number of current and former senior public servants, including some in the partner or collaborating agencies providing cash and in-kind support to the research, were identified by the project team and directly contacted with an invitation to participate in an interview. A total of 126 interviews were conducted from July 2012 to December 2013. This sample was composed of 74 males and 52 females across different state and federal government departments.

The interview schedule was divided into a number of sections and involved asking interviewees their opinions regarding the use of academic research in policy making, their personal and professional connection to academia, and the timeliness, usefulness, and relevance of academic research to decision making in public policy. Interview data were coded in NVivo 10, with a research assistant and members of the research team reviewing the data and formulating themes and subthemes to which segments of interview data were then allocated. Four major themes were identified that relate to the present discussion: academic contextual factors such as incentive structures and institutional priorities; research dissemination; knowledge brokering; and competing pressures on policy formulation, such as those coming up from stakeholders or down from the political executive.

Discussion

If this study's respondents are correct, and there is in fact rich potential for improved use of academic research in policy making, it is probable that this improvement might come about through the work of specialized bridgers, as many have noted previously in international studies (Hargadon 1998; Knight and Lightowler 2010; Williams 2002). These “knowledge brokers” or “boundary spanners” may have the capacity to demonstrate the utility of academic research within the policy process to policy makers, but only if they understand the needs of policy makers and can translate the outputs of academia into forms that are recognizable as being useful to policy decision makers. This may involve selecting academic research that is most relevant to policy makers’ needs, but it could also require packaging and presentation of academic outputs into formats that are best and most quickly absorbed by policy makers, who might require structured summaries of research because they do not have the capacity to review the research themselves (Maynard 2006). The fact that many policy makers are already using academic research and that knowledge brokering already occurs provides encouragement to those who would like to see a greater uptake of academic research by those who are tasked with directing public policy.

One of the central arguments of the two-communities thesis is that the lines of communication between academia and policy are broken. This is demonstrably not true, neither in the Australian context nor in other Western countries (Bogenschneider and Corbett 2010). Both survey respondents and interview subjects in this study reported that they are receiving academic research, that they have access to academic articles and reports, and that they are aware of academic research topics.

The two-communities approach also emphasizes the diverging rewards structures and incentives in policy and academia. Specifically, the academic focus on publishing is seen as not being conducive to policy impact (Cherney et al. 2012). This may be true, and this sentiment is reflected in this study's survey and interview responses. However, because academic research is available to policy makers even if it is not specifically directed to them, perhaps the policy sector does, as many interview respondents seemed to suggest, have a greater responsibility to access and ultimately use academic research for policy making purposes without having to be prompted directly by academic researchers. As is evident in this study's interview responses, many public servants see academic research not as a plentiful resource of knowledge about a multitude of policy-relevant topics but rather in terms of whether the research helps them address immediate policy priorities. In addition to instrumental purposes of academic research, such as helping identify solutions to policy problems, academic work can have a conceptual purpose as a source of intellectual output as well. Pushing ideational boundaries requires looser time frames and complete intellectual freedom, as opposed to an instrumental view of academic research as work that is available on request, to purpose, and by contract. Perhaps if public sector decision makers saw academic research more often as conceptual enlightenment with some applied benefits rather than as a form of contract labor—to be hired when a policy problem needs solving—they might be less frequently disappointed with academic outputs (Hammersley 2014).

Conclusion

Analysis of international literature and the data from this study's survey and in-depth interviews with Australian public servants seems to suggest that the two-communities conceptualization of the relationship between policy and academia is misleading and that a much more nuanced account of the relationship is necessary. Dissemination practices do vary within academia (Cherney et al. 2013), as do research uptake practices within the policy domain and the types of relationship-building activities that are adopted (Bogenschneider and Corbett 2010; Cherney 2013; Haynes et al. 2011). Thus, a more realistic approach would acknowledge that there is a range of interactions between policy and academia, with different individual policy makers valuing and using academic research more than others. Many public servants see the potential for increased interaction and cooperation. Importantly, there is a perception among bureaucrats that political decision making affects policy differently than administrative decision making, and therefore perhaps the political and administrative levels ought to be recognized as separate communities in their own right.