Between context and comparability: Exploring new solutions for a familiar methodological challenge in qualitative comparative research
Abstract
enFinding the balance between adequately describing the uniqueness of the context of studied phenomena and maintaining sufficient common ground for comparability and analytical generalisation has widely been recognised as a key challenge in international comparative research. Methodological reflections on how to adequately cover context and comparability have extensively been discussed for quantitative survey or secondary data research. In addition, most recently, promising methodological considerations for qualitative comparative research have been suggested in comparative fields related to higher education. The article’s aim is to connect this discussion to comparative higher education research. Thus, the article discusses recent advancements in the methodology of qualitative international comparative research, connects them to older analytical methods that have been used within the field in the 1960s and 1970s, and demonstrates their analytical value based on their application to a qualitative small-N case study on research groups in diverse organisational contexts in three country contexts.
Abstract
deDie Balance zwischen der adäquaten Beschreibung der Besonderheiten von Untersuchungsfällen, und der Herstellung einer hinreichenden Basis für Vergleichbarkeit und analytische Verallgemeinerung zu finden, wird in der Diskussion zu Vergleichsmethodologie als eine zentrale Herausforderung angesehen. Methodologische Überlegungen dazu, wie die Besonderheiten von Fällen und Vergleichbarkeit in Studien gleichermaßen adäquat gefasst werden können, wurden für die quantitative Forschung bereits ausführlich diskutiert. In jüngster Zeit wurden zudem vielversprechende methodische Überlegungen für die qualitativ vergleichende Forschung in komparativen Forschungsfeldern, die zur Hochschulforschung benachbart sind, gemacht. Der Artikel will diese Diskussion in die vergleichende Hochschulforschung einführen. Er diskutiert daher neuere methodologische Vorschläge in der qualitativen, international vergleichenden Forschung, stellt eine Verbindung zu älteren analytischen Ansätzen her, die in der Hochschulforschung in den 1960er und 1970er Jahren zum Einsatz kamen, und demonstriert den analytischen Wert dieser Vorschläge anhand einer qualitativen Fallstudie zu Forschungsgruppen in unterschiedlichen Organisationskontexten in drei verschiedenen Länderkontexten.
1 INTRODUCTION: CHALLENGING THE PATHS OF (QUALITATIVE) COMPARATIVE RESEARCH
International comparative research is one of the key methodologies within the field of higher education research and has become a growing type of research in recent years (Cantwell, 2020; Dobbins, Knill, & Vögtle, 2011; Kosmützky & Krücken, 2014; Tight, 2012; Weiler, 2008). The significant benefit of international comparative studies is their ability to broaden our horizon: they give us the opportunity to reflect upon phenomena within our own higher education system through the lens of other systems. They also help to deconstruct narrow and often parochial national perspectives, as well as to overcome single-country myopia, where one assumes that all other systems follow the logic of the system in one’s own country. By providing a picture of what is going on elsewhere in the world, these studies also foster the analysis of the growing international dimension as well as the transnational and global entanglement of higher education. The comparison of similarities and differences across higher education systems can also enable us to see more general patterns of phenomena within higher education, and, consequently, or simultaneously, in other domains of society or public policy. We can use these patterns to theorise and build new theoretical assumptions about higher education’s mode of operations and social dynamics (Diogo, 2015, 2019; Kogan, Bauer, Bleiklie, & Henkel, 2006; Kosmützky, 2018; Teichler, 2014). However, comparative research (in higher education and beyond) also faces particular methodological challenges (for higher education, see, e.g., Antonucci, 2013; Kosmützky & Nokkala, 2014; Reale, 2014; and beyond, see, e.g., Hantrais, 2009; Hölscher, 2017; Smelser, 1976). Finding the balance between adequately describing the uniqueness of the context of the studied phenomena and maintaining sufficient common ground for comparability and analytical generalisation has widely been recognised as a key challenge in international comparative research for both variable-oriented quantitative research and case-based qualitative research (see, e.g., Goldthorpe, 1997; Przeworski & Teune, 1970; Ragin, 1987; Scheuch, 1967; Smelser, 1976; and for comparative higher education in particular, see, e.g., Goedebuure & van Vught, 1996; Teichler, 1996; Välimaa & Nokkala, 2014).
Research on both comparative traditions faces complementary problems. Different paths of methodological guidelines, as well as reflections on how to adequately cover context and comparability, have been discussed in the research methods-related literature and in the literature on various (comparative) neighbouring fields (e.g., comparative education, comparative politics, science and technology studies (STS), and the sociology of science). However, such a debate is mostly missing in higher education research. Qualitative methods are frequently used in comparative studies within the field of higher education research (Kosmützky, 2016). Nevertheless, not many methodological reflections that can help higher education researchers to cope with the methodological issues of qualitative comparative studies are available within the knowledge base of the field. This is not surprising when we bear in mind that higher education research is not a discipline with a strong tradition of methodological development. Rather, it is an object- or problem-oriented field of study (Cantwell, 2020; Teichler, 1996). The aim of our article is to reinvigorate the debate on comparative methodology in higher education research by importing some knowledge on comparative methodology from disciplines and comparative sub-disciplines that form the intellectual and institutional base of our interdisciplinary field (Schwarz & Teichler, 2000).1
Among the neighbouring fields, comparative politics plays a major role when it comes to comparative methodologies for qualitative data sets and case studies. Two methods specifically designed for qualitative small-N studies have been developed within the field: qualitative comparative analysis (QCA) and comparative historical analysis (CHA). A lot of literature exists on QCA (for overviews, see, e.g., Berg-Schlosser, De Meur, Rihoux, & Ragin, 2009; Rihoux & Ragin, 2009; Schneider & Wagemann, 2010), but QCA is not a ‘true’ qualitative (and inductive) method. Rather, it is a variable-oriented (and hypothesis-driven) method based on set-theoretical thinking (Buche, Buche, & Siewert, 2016). Thus, it does not meet the requirements for qualitative comparative research (for similar arguments, see, e.g., Bergene, 2007). CHA is a well-developed qualitative case-based method for temporal analysis of macro-social configurations. It uses process tracing as a core method. Like QCA, CHA only partially meets the requirements for comparative higher education research, as it focuses on macro-social change. Many research questions in the higher education research field focus on the meso-level or micro-level aspects of higher education (for overviews, see, e.g., Mahoney, 2007; Mahoney & Rueschemeyer, 2003b; Mahoney & Thelen, 2015). Furthermore, a vivid comparative (international) research debate is taking place in the field of the comparative politics of education. Methodological innovations that may suit higher education research might spring from this field in the future (Gift & Wibbels, 2014; Moe & Wiborg, 2017). Interestingly, a pronounced lack of methodological considerations has existed for qualitative comparative research in comparative education, which has a strong qualitative tradition (Phillips & Schweisfurth, 2014).
However, a discussion has taken place on the advancement of the methodological challenges of qualitative international comparisons in the qualitative branch of STS and in the sociology of science. These fields are as close to higher education as the field of comparative education is (Kosmützky, 2016; Manzon, 2011). Scholars from these fields who work ethnographically have made innovative suggestions in several articles and in special issues and anthologies, and these suggestions may be fruitful for higher education research (Deville, Guggenheim, & Hrdličková, 2016b; Hine, 2007; Morita, 2014; Niewöhner & Scheffer, 2008, 2010b). Central to their research paradigm is the goal of maintaining the openness and context sensitivity of qualitative social research, and of not giving it up through a fixation on a priori defined comparative criteria. Their methodological suggestions for comparative research maintain this openness, even as they, at the same time, grapple with comparability and aim to produce comparable case descriptions.
In this paper, Sørensen’s approach to process-oriented, multi-sited comparability (Sørensen, 2008, 2010), and Schmidt’s method of explorative comparison (Schmidt, 2008, 2012) are utilised. Both basically propose an inversion of the standard comparative process (defining comparative criteria—comparing cases—denoting differences/similarities). Sørensen (2008, 2010) additionally suggests a rearrangement of the spatial constellation. The plan here is to connect their propositions to comparative higher education research that also uses many other methodological approaches than ethnography and that also utilises a range of qualitative data (e.g., interviews, documents and website material) and various methods of data analysis (coding, qualitative content analysis and a variety of reconstructive analyses—grounded theory methodology, narrative analysis, sequential analysis and the documentary method) (Tight, 2012, 2013; Wells, Kolek, Williams, & Saunders, 2015). Furthermore, they will be related to older and ‘comparable’ approaches to comparison that have been used in higher education. These include the four-step model of comparison by Hilker (1962) and Bereday (1964), as well as the emic/etic approach by Berry (1969, 1989, 1999). Furthermore, the value of Sørensen’s and Schmidt’s suggestions will be demonstrated via utilising them for a qualitative small-N interview-based case study on research group orientation in competitive institutional environments. This article is not first and foremost about the empirical findings of this study (Nokkala & Diogo, 2019). Rather, the study will be used to discuss how comparability (and generalisations) can be achieved in qualitative comparative case studies in a different way.
The article is structured as follows: first, we begin with remarks on the role of comparative criteria for a comparative intellectual operation (the so-called tertii comparationis) and on the tension between context and comparability. We also introduce the two aforementioned approaches to qualitative international comparative research (Schmidt and Sørensen), and we remind the reader of the two older approaches in qualitative comparative studies from 1960 that used a similar method of comparison (albeit with a different epistemic approach). Second, we present an empirical case study on research groups in diverse organisational contexts in three country contexts that have benefited from an explorative and flexible approach that reverses the order of the usual comparative operation. In the conclusion, we discuss the advantages and prospects of these conceptual and empirical explorations for comparative research in the field of higher education.
2 INVERTING THE CONVENTIONAL SEQUENCE OF COMPARISON: COMPARATIVE CRITERIA AS A RESULT AND NOT AS A PRECONDITION
Generally speaking, comparisons are intellectual operations that establish relationships between units or events. Therefore, Heintz (2010; referring to Luhmann, 1995) labels comparisons as ‘instruments of observation’: every comparison is based on two operations that can be analytically distinguished: on the determination of (partial) sameness (commensurability), and, building on this, on the determination of differences. Comparisons are based on the assumption that the compared units are identical in at least one fundamental respect and presuppose a comparative criterion—the so-called tertium comparationis—that makes differences in the (partially) identical units observable. We typically justify comparisons by saying that two units are comparable in this or that respect. Conversely, units that have nothing in common cannot be compared with each other (and thus are incommensurable) (Heintz, 2010).
Many studies are comparative without being internationally comparative in nature, and they share essentially the same comparative logic of simultaneously observing similarities and differences. In the field of higher education, they compare, for example, organisations within a higher education system, groups of students, types of higher education institutions, academic disciplines, or reform developments in different time periods. Moreover, the same processes of data collection and data analysis are carried out within a number of analytical units that are more or less systematically compared. However, only international comparative studies collect data in different cultural or country contexts, and thus, they typically proceed at varying levels and have to deal with different types of context (Kosmützky, 2015). The term ‘context’ originally stems from the tradition of hermeneutical text interpretation and its holistic perspective, whose aim is to fully understand its objects (Dilley, 2002; Morita, 2014; Otto & Bubandt, 2010). In comparative empirical social science research context can be seen as ‘a set of connections between the object of inquiry and its surroundings’ (Morita, 2014, p. 215). Particularly in international comparative case studies aimed at grasping the specificity of the context of individual cases (George & Bennett, 2005)—whether these cases are universities, groups of students or types of governance regimes—multiple dimensions of contexts are at play (Bray & Thomas, 1995). Thus, higher education research is concerned with establishing comparability across multiple dimensions of contexts.
Both qualitative and quantitative approaches to comparison have their particular advantages and challenges concerning the tension between context and comparability, and they align with different goals and norms of research practice (see, e.g., Brady, Collier, & Seawright, 2010; Ebbinghaus, 2005; Hantrais, 2014; Lieberson, 1991; Lijphart, 1971; Mahoney & Goertz, 2006). The challenge of comparative research in the variable-oriented tradition (the so-called nomothetic tradition, which is aimed at finding general rules and works deductively) is a lack of context sensitivity. Thus, its concepts, measurements, constructs and items can be culturally biased and distorted (see, e.g., Ebbinghaus, 2005; Lieberson, 1991; Lijphart, 1971). It also suffers from a ‘segmentation problem’ because it dissolves the totality of comparative cases and its contexts into components and corresponding variables (Patzelt, 2005, p. 21, translation by the authors). However, the challenge of qualitative comparative research (in the so-called ideographic tradition, which is aimed at understanding cases in depth) is comparability. This type of research approaches its objects holistically and typically produces dense and unique case descriptions, as well as within-case analyses by focusing on historical, cultural, social and economic contexts (Bergene, 2007; Collier & Mahoney, 1996; King, Keohane, & Verba, 1994; Mahoney, 2007; Otto & Bubandt, 2010). It also has a ‘focusing problem’ because it attempts to capture the specifics of cases and thus suffers from providing conceptualisation to guide future studies or from producing generalisations (Patzelt, 2005, p. 21, translation by the authors).
Both camps were based on a positivist ontology and epistemology, and they were far from constructivist or critical realist notions in the 1960s and 1970s. They had in common that they conceived comparability (and incomparability) as a shared inherent characteristic of the units of comparison (and as a property of the empirical data) and thus as something that researchers discover, not something they actively construct.2 The perception that comparative criteria simply have to be discovered and identified at the beginning of the research process to establish comparability has proliferated in the methodological literature on comparative research. In fact, it still guides the quantitative comparative camp to a large extent. Meanwhile, the qualitative comparative camp is densely populated by scholars whose work is informed by methodological choices based on constructivist and critical realist ontologies and epistemologies (Bennett & Elman, 2006; Cantwell, 2020; Suspitsyna, 2019).
Both constructivist and critical realist comparative scholars are concerned with context and comparability. Still, critical realists have a tendency to be more focused on elaborating on mechanisms and on establishing causality. Meanwhile, constructivists lean towards thick descriptions and analytical induction. However, as Bergene (2007) explains, critical realists use a contextualised concept of context and consider contextual aspects to be important for the explanation of effects.3 Both critical realists and constructivists are interested in using abduction to identify patterns within empirical data and to make generalisations that derive from fine-grained evidence of particular and concrete cases. They also both favour new conceptual understandings and theory development. Both additionally combine the examination of similarities and differences across cases with in-depth or within-case analyses, and they use recontextualisation or relocations as a means for new perspectives and insights (see, e.g., Bergene, 2007; Danermark, 2002; Hine, 2007; Mahoney, 2007; Morita, 2014; Niewöhner & Scheffer, 2008). We do not want to overstretch the comparison and emphasis on commonalities between the two at this point. However, we wanted to link the two constructivist methodological propositions that explore new ways of comparison—those of Schmidt (2008, 2012) and Sørensen (2008, 2010)—with older methods (Bereday, 1964; Berry, 1969; Hilker, 1962) based on realist notions that we discuss in the following to broaden the debate on comparative methodology.
Schmidt follows a ‘constructivist, open and experimental approach to comparison’ (Schmidt, 2008, p. 340) and offers a procedural comparative method on the basis of grounded theory methodology (Schmidt, 2012). From this perspective, he suggests inverting the conventional sequence of comparison: instead of defining comparative criteria at the beginning of the research study and thus treating comparability as a prerequisite, the researcher should straightforwardly start with the comparison and equate objects and cases to each other. Typically, comparative research focuses on the equivalence of the units of comparison to ensure comparability (for an overview on the uses of equivalence in comparative research, see, e.g., Johnson, 1998). In Schmidt’s approach, the question of comparability is answered only in the course of the comparative process by exploring comparability and incomparability, or by uncovering otherwise unknown aspects of the objects under investigation. He furthermore suggests that the researcher should experiment with theoretical/analytical questions or with other preconstructed criteria that are chosen for analytical and descriptive purposes and at an explicit distance from the objects (this is done instead of seeing the criteria of comparison as inherent qualities of the object of research under study). He calls these questions and criteria the ‘analytical tertium comparationis’ (Schmidt, 2008, p. 341), and he strongly recommends experimenting with them to question familiar perspectives and to illuminate the objects of investigation in a new way. The aim of the entire inversion of the process is to ‘make use of contrasts and differences, to gain insights from incomparability and inadequacy’ (Schmidt, 2008, p. 358).
Like Schmidt, Sørensen (2008, 2010) in her likewise constructivist perspective suggests inverting the standard process of comparison and treating the tertium comparationis as a result and not as a precondition. She calls this approach process-oriented comparability. However, she also illuminates another important aspect: the spatiality of the comparative constellation between the researcher and the research object. Her starting point is the observation that the ethnographic method and comparative method based on a tertium comparationis represent two poles of a dichotomy: the former implies ‘inside views, process orientation and mobility’, and it ‘takes an ex-post approach and aims at producing inside descriptions’ of case specificities. The latter favours ‘outside views, result orientation and immobility’, and it takes ‘an ex-ante approach and is oriented towards producing an (at least partly) outside description by reducing the compared field sites’ specificities’ (Sørensen, 2010, pp. 43, 47). Ethnographers base their empirical research on interpretive inside perspectives that Sørensen calls ‘inside descriptions’. A tertium comparationis that is defined upfront as a common quality of the comparative cases thus would not match the inside descriptions of the cases (and would compromise the basic methodological principles of ethnography). To achieve inside descriptions, ethnographers move ‘“away” from home to produce an inside account of an “away” culture’ (Sørensen, 2010, p. 47).4 International comparative research encompasses several home-away movements through multiple field sites and cultures. Although the idea of a simple inside-outside perspective has been replaced with the idea of degrees of insiderness-outsiderness in the field of ethnography (e.g., Dyck, 2000; Marcus, 1998), Sørensen (2010) argues that the spatial constellation of the ethnographic method can still be seen as a spatiotemporal and conceptual movement. Moreover, the comparative method based on a tertium comparationis does not include temporal movements and inside views, and thus, it includes no conceptual change of perspective.
To combine inside descriptions with a tertium comparationis, Sørensen suggests treating comparisons as synonymous with juxtapositions. Contrasts and juxtapositions primarily serve the purpose of describing specificities of cases (respectively, field sites) but with a focus on their differences and not on their similarities. However, as she argues, throughout the course of the research, the researcher might find that some of the inside descriptions of cases are actually variations of the same organisational pattern. ‘While the ethnographic juxtapositions and contrasts help describe the specificities of the research object, organising patterns characterise “what the hell is going on”’ across field sites, although organising patterns can take different forms in various field sites (Sørensen, 2010, p. 56, referring to Clifford Geertz’s famous quote). Such organising patterns that emerge ex-post from thick ethnographic descriptions and multi-sited comparisons are ex-post tertii comparationis. They are the result of comparability that has been established from the inside (and not comparative criteria chosen upfront) and do not generalise in a universalist sense. As Sørensen emphasises, ‘postponing the definition of tertii comparationis until the end of the research allows us to find comparability between field sites we would intuitively never think could be compared’ (Sørensen, 2010, p. 75).
Although focused on ethnographic research, the methodological proposals of Schmidt and Sorensen are extremely fruitful for comparative qualitative research, which we will show based on our empirical case. However, we would like to contextualise these suggestions with older models of comparison developed or used within higher education research, which we discuss in the following paragraphs referring to Hilker’s (1962) and Bereday’s (1964) four-stage model of comparison, as well as Berry’s (1969) emic/etic approach. Both Hilker and Bereday also invert the typical sequence of comparison (defining comparative criteria—comparing cases—denoting differences/similarities) and see the tertium comparationis as a result of an initial description and interpretation of the characteristics of comparative units. Berry also utilises differences in spatiality for his methodological approach.
At the beginning of the 1960s, Hilker (1962) and Bereday (1964) both independently developed an analytical four-stage model consisting of a sequence of (a) description; (b) interpretation; (c) juxtaposition; and eventually (d) comparison.3 Their methodology is based on a realist epistemology; they thus would see comparability as a property of the comparative units that one has to discover through a comparative analysis. However, like the new suggestions by Schmidt and Sørensen, they also see comparison as a dynamic process that should start with a description of the cases and with the interpretation of material according to preconceived analytical categories. This process continues with a juxtaposition and comparison of the cases in various respects and constellations. Comparative criteria are also not defined upfront but rather are developed in the third stage through a preliminary juxtaposition and comparison of the data from different countries. For both Hilker (1962, p. 121) and Bereday (1964, p. 22), the actual comparative method begins only in the third stage with the juxtaposition of empirical material. As Hilker (1962, p. 100) notes, ‘similarity or dissimilarity does not mean much, as long as they are not related to another superordinate value’ (translation: authors). This is the stage of the search for a unifying concept and hypothesis, the tertium comparationis, upon which further comparisons can be made to prove the hypothesis that has derived from the juxtaposition.
Similarly to Sørensen’s home-away movements, Berry (1969) utilises inside-outside perspectives that he calls emic-etic perspectives. Emic indicates an internal perspective from within a particular culture or environment (or the insider’s view), whereas etic stands for a perspective from the outside (representing the view of the neutral, authoritative observer). His analytical method also alternates in a circular movement between both perspectives (Berry, 1989).5 The researchers typically start with assumptions and concepts rooted in their own contexts, and they use them to familiarise themselves with cases in another country, culture or geographical context. Thus, Berry labels this perspective an imposed etic. In the second step (the actual empirical analysis), researchers set aside their ‘cultural backpack’ (the imposed etic) and try to grasp the internal perspective from within particular contexts and environments (which should allow for the identification of context-specific aspects). The goal of this step is to gradually change the imposed etic and to match an emic viewpoint from the context of the cases under investigation. Only after research has passed through the emic phase should it eventually lead to the formulation of shared and common, or at least prevalent, aspects and features of the cases under investigation, which Berry calls derived etics. These aspects and features can then be used as comparative criteria (the tertii comparationis) for the actual comparison. A derived etic differs from a universal perspective because it is limited to a set of countries, cultures and contexts, as well as their specificities (Berry, 1969, 1989, 1999).
The older studies already suggest an important shift away from studies that are based on the a priori formulation of comparison criteria that protects openness as the core characteristic of qualitative research: they conceptualise comparative research as an analytical process with various stages. Then, they move the formulation of comparative criteria into a later phase of the process. This phase precedes descriptive and interpretative analytical steps, as well as tentative comparisons. One could say that the more recent methodological proposals radicalise these older proposals by taking comparative criteria not as a starting point but rather as an intermediate (analytical) result and device (Schmidt), or as a final result (Sørensen). In the next section, we discuss the application of this methodological strategy on the basis of an empirical study on research group orientations.
3 A CONVENTIONAL AND THEN INVERTED COMPARISON OF RESEARCH GROUP ORIENTATIONS
The empirical case application is based on a comparison of 11 research groups in three small European countries. The aim of our study (reported in Nokkala & Diogo, 2019) was to examine how research groups operating in the policy-relevant interdisciplinary research field, nanosciences, perceive the mission and beneficiaries of their work, and organise their work and collaborations. These groups work in varying organisational contexts and in three countries. The research questions that guided the study (although phrased differently in the article (Nokkala & Diogo, 2019) for reasons related to the theory chosen at a later stage) were phrased as follows: ‘How do research groups operating in the same research area but in different organizational and national contexts organize their work and collaborations?’ (descriptive question). Also, ‘Are the modes of organizing knowledge production and networking, as well as the self-understanding of the research group different in different organizational contexts, or do they converge regardless of the organizational context?’ (analytical question).
The study was thus built on the assumption that countries and especially organisational settings would determine the way in which the research group would operate and perceive its role. The case selection thus followed the idea that research groups were nested in multiple contexts at the same time (Hüther & Krücken, 2016) and that multiple dimensions of contexts are at play in any given comparative study (Bray & Thomas, 1995). Starting from the conventional comparative assumption—that the researchers choose the relevant comparative contexts and criteria in advance—our case selection featured Bray and Thomas’s geographic/locational (country, organisation) dimension. The perceived mission, beneficiaries, collaborations and competitors of the research groups represented the third dimension, which highlights the specific aspects of the system or society to be studied (Bray & Thomas, 1995). The case selection was based on the application of Seawright and Gerring’s (2008) diverse case strategy, which, they argue, is suitable for the exploratory research common in small-N studies. The aim was to explore a range of variations among dimensions that were of interest to the investigation. In this case, the countries chosen represented similar types of countries and thus a most-similar-system design (see, e.g., Landman, 2008), as our aim was to analyse differences. We focused on three small European Union members in different parts of Europe. Meanwhile, the context organisations of the research groups we selected represented a broad range of types of organisations and thus a most-different-systems design (see, e.g., Landman, 2008). This is because our aim was to analyse similarities across the types of organisations: universities, research institutes, technology development centres and companies. This was based on the assumption that the institutional logics (Thornton, Ocasio, & Lounsbury, 2012) of different types of organisations differ (e.g., Ocasio & Radoynovska, 2016; Potì & Reale, 2000). Still, the shared disciplinary field (Becher & Trowler, 2001) in which the research groups operate might have led to a certain similarity.6
The data were collected through semi-structured group interviews comprising three to five people who represented various career stages, ranging from junior to senior researcher, to a more managerial role. They were analysed in three larger stages (each comprising more than one analysis step): organising, reducing and comparing. In the following, we outline the analysis process and the challenges that arose from the conventional sequence of comparison based on the predefined comparative criteria. We also spell out the steps we took to invert the process and experiment with alternative analytical comparative criteria. Retrospectively speaking, these steps can be seen as following the propositions of Sørensen (2008, 2010) and Schmidt (2008, 2012), as well as Hilker’s and Bereday’s model of description, interpretation, juxtaposition and comparison (Bereday, 1964; Hilker, 1962). Alternatively, in Berry’s (1969; see also Halilovich, 2016) terminology, we moved from an imposed etic to a derived etic. However, all of these retrospective interpretations of our method highlight the importance of understanding the rich internal lives of the research groups themselves, which was at the core of our qualitative case study approach.
The conventional approach and its emerging problems
The aim of the first organising stage was to organise the data to be able to decipher its contents. We did that by country, research group and interview question. We wrote a short summary for each research group and described the most prominent features arising from the interview for each group. This was followed by conducting a qualitative content analysis (Hsieh & Shannon, 2005) using a bricolage of deductive concept-driven and inductive data-driven coding (Brinkmann & Kvale, 2018; Gibbs, 2018) aimed at identifying all elements that could be said to pertain to the topic of the study. These were organised into categories (broad thematic focus areas of the study), dimensions (topics addressed in the interview questions) and themes (different themes arising as responses to the interview questions).
The aim of the second stage, the reducing stage, was to reduce the complexity of the data by focusing on and abstracting elements of data to create case profiles (Nokkala & Diogo, 2019). They were used to distinguish the research groups from each other and to juxtapose elements of their profiles. This stage focused on identifying combinations of themes for each research group. It also stemmed from creating ‘group-theme composites’ based on what kinds of combinations of the themes the research groups exhibited. The reducing stage was completed by using the group-theme composites to construct three research group profiles: basic, broad and applied profiles. Finally, the research groups were categorised into the constructed profiles. At least five of the eight criteria for a given profile had to be met for the research group to be categorised into a given profile. Most research groups fit only into one profile, but two research groups fit into two profiles. The profiles are elaborated in Nokkala and Diogo (2019).
Until the third stage, the comparing stage, the analysis thus focused on descriptive knowledge about the cases. Our preconception was that we should find more differences among the research groups in different types of organisations in a given country, versus among the research groups in a given organisational context but in different countries. However, we discovered that neither the organisational context nor the country context clearly determined the profiles (Nokkala & Diogo, 2019). Thus, although the typology of profiles can in itself be an interesting tool, we could not advance beyond the typology to more explanatory findings using the conventional approach of predetermined comparative criteria.
Inverting the process: Back to emic perspectives and new analytical comparative criteria
At this point, we decided to change our strategy to solve the problems that arose from the previous approach: we introduced some of the dimensions that were left out of the profile construction, and we used the research group profiles as the new analytical comparative criteria. Being constructed based on the thick descriptions of the research group operations, the profiles thus took seriously the agency (Fumasoli & Huisman, 2013) and emic perspectives of the research groups themselves (Berry, 1969; see also Halilovich, 2016).
In using the profiles as the comparative criterion, we first formed a new element, geosocial orientation, based on two dimensions pertaining to the research groups’ perceptions of the geosocial operational level (global, national and local) for their own as well as their competitors’ operations. We constructed three geosocial orientations: global, local and mixed. In comparing the research group profile with its geosocial orientation, we found that the local level was more relevant to the applied research groups, which variably held a local or mixed geosocial orientation. In contrast, all basic and broad research groups were oriented towards the global level with the exception of one group with a mixed geosocial orientation.
Although only the comparison of geosocial orientations against the profiles was reported in the original study for reasons of brevity (Nokkala & Diogo, 2019), we also tried to test the profiles as a comparative criterion with another previously not selected dimension: the perceptions of change that the research groups encountered while focusing on perceived internal versus external changes. Research groups with broad or applied profiles—but not the basic research groups—perceived internal changes, such as the group’s internal restructuring, changing autonomy, and the restructuring of the staff and their new tasks. They also perceived the growth of the research group, as well as the emergence of the international dimension in the research projects. In contrast, the basic and broad research groups—but not the applied groups—perceived external changes, such as changing pressures from the operational context, including national or institutional regulations, policies or funding. These changing pressures from the operational context of the groups include, among others, the pressure to work more with companies, the experience of more bureaucracy and demands from administration or leadership, and less time to do research, as well as more competition in terms of funding, namely in terms of competitive/strategic funding. We therefore concluded that applied research groups perceive more internal changes, and basic groups perceive more external changes. The broad research groups lie somewhere in the middle, perceiving both types of changes.
An organising pattern emerges: Self-positioning and self-perceptions of research groups
In a retrospective view, when evaluating the empirical case on the backdrop of the suggestions for new methodological moves by Sørensen (2008, 2010) and Schmidt (2008, 2012), one could say that the three research group profiles constituted our operationalisation of the emic perspectives of the research groups themselves, and were a result of our descriptive in-depth case analysis (Bereday, 1964; Berry, 1969; Hilker, 1962; similarly, Hine, 2007; Morita, 2014, and others). They were used as new analytical comparative criteria and emerged as the new organising pattern of the comparison. The organising pattern that we found are the self-constructions of such groups as groups with basic, applied or mixed research profiles, as well as their self-positioning in a global/national or local competitive and collaborative context. It emerged through inverting the conventional comparative process at a dead-end stage of the analysis and by reconstructing the contexts of groups from an emic perspective (or as an ‘inside description’, in Sørensen’s words). Thus, the answer to the analytical research question, ‘Are the modes of organizing knowledge production and networking, as well as the self-understanding of the research groups different in different organizational contexts, or do they converge regardless of the organizational context?’ could be: yes, the self-understandings depend on other variables besides the organisation (and country), and these other variables can be heuristically represented with research group profiles. We cannot infer clear causality, but research group profiles clearly helped with broadening our understanding of the factors that contribute to the differences between the research groups while at the same time generating contextual knowledge that is more meaningful to the research process compared with the preselected comparative contexts.
4 CONCLUSION AND DISCUSSION: GOING BEYOND THE FAMILIAR
In this paper, we demonstrated with an empirical example how inverting the conventional comparative process—a suggestion that we derived in the conceptual part of the paper from recent ethnographic STS studies and older comparative studies that have been used within comparative education—helped with maintaining the interpretative openness of the qualitative comparative study, enabled balancing between context and comparability, and helped with coming up with generalisations based on in-depth case descriptions.
Instead of figuring out ‘what the hell is going on here’ (Sørensen, 2010, p. 44), the empirical analyses planned in the original research design of the study used as an example in his article came to a dead end and to a ‘what is the point here?’ situation halfway through the analysis. Methodologically, the original study veered between having theoretical preconceptions that guided the analysis and a grounded theory ‘light’ (Strauss & Corbin, 1990) and followed the stages of the conventional comparative process as follows: (a) define comparative criteria; (b) organise the material; (c) reduce and sort the material into categories; and (d) compare and juxtapose the material according to categories. However, the results achieved felt to us somewhat uninteresting on their own. Inspired by Schmidt’s (2008, 2012) and Sørensen’s (2008, 2010) suggestions for new ‘methodological moves’ in comparative ethnographic analyses, the stages of the analysis were reversed in this situation: the original comparative criteria (the tertii comparationis) were discarded, and new analytical comparative criteria (Schmidt, 2008) based on emic/inside context descriptions were used for enhancing the comparative analysis. These criteria broadened the understanding of the research groups’ relevant contexts. Through further comparisons, an organising pattern (Sørensen, 2010)—the research group profiles —emerged as the result of the analysis. The profiles revealed decisive differences in the self-perceptions of the research groups and the self-positioning in their specific fields of action across countries. Such a process-oriented comparability approach was not planned from the beginning. Rather, it was applied halfway through the research process as a result of encountering unforeseen challenges in the research design. We can conclude that it enriched the understanding of the research groups, provided us with new empirically-based comparative criteria and helped us to come up with a satisfying generalisation of the empirical cases. In fact, we could say that we created comparability by taking context seriously and by taking it into account in a new way.
-
New ways and aims for comparing: the methodological pathways that the abovementioned scholars suggest point a way out of the ‘iron cage’ of mainstream comparative approaches, or, as Krause (2016, p. 45) puts it, ‘free the academic practice of comparison from its theory’. The traditional ‘theory’ of comparison, respectively comparative methodology, has been obsessed with establishing equivalence in multiple dimensions of the comparison, and thus, it is aimed at comparing ‘like with like’ (Krause, 2016, p. 45). Correspondingly, Johnson (1998) outlines more than 50 varieties of pairings of term equivalence (e.g., conceptional equivalence, operational equivalence, interpretative equivalence, etc.) in the available literature on international comparative research in various social science disciplines from the 1960s onwards. Comparative research, mainly in the variable-oriented tradition, followed the notion that international comparative research is ‘quasi-experimental’ research and is aimed at analysing similarities and differences to test and build theory based on a specific type of causal explanation. Thus, different cultural contexts were basically treated as different ‘experimentation’ or ‘treatment’ grounds, and, as Krause puts it, all kinds of causes as forms of ‘treatment’ (Krause, 2016).7 This logic has also influenced qualitative comparative research (as, e.g., in CHA (Mahoney & Rueschemeyer, 2003a) or QCA (Ragin, 1987, 2000)) explicitly or implicitly, as in the original research design of the study used as an example in this article and many other qualitative studies in the field of higher education research. Although, we don’t know whether such articles have maintained the openness of qualitative research in the process of discovery but have adjusted the description of their research to the conventional comparative logic in the context of justification.8 However, the process-oriented comparability approach in combination with emic perspectives, inside descriptions, thick within-case descriptions and in-depth case analyses (Bereday, 1964; Berry, 1969; Hilker, 1962; Schmidt, 2008, 2012; Sørensen, 2008, 2010; similarly, Hine, 2007; Morita, 2014) are legitimate comparative aims and ways of comparing that help to illuminate the complexity of real-life cases. However, it is important to note that an ‘inside description’ is not ‘more authentic or truthful than approaches that apply a priori and “outside” classifications to empirical data. An “inside description” is fabricated in intra-action among researcher, instruments, research object, etc., just like any other research account’ (Sørensen, 2010, p. 44).
However, for higher education research, the recognition of the validity of a process-oriented comparability and comparative explorations based on in-depth case analysis and thick within-case descriptions may alleviate some common challenges. These include the tendency of a) institutional parochialism, for example, focusing on the tightly defined entities of structure or practice without embedding in complex environments with the recognition of their social dynamics (Dale, 2005; Diogo, 2019; Välimaa & Nokkala, 2014); and b) viewing the lexical equivalence of higher education systems or institutions as sufficient grounds for the unproblematic comparability of these entities (Dale, 2005). Not merely the expertise and experience of a comparative scholar but also the reflection of the researcher’s own positionality (Torres-Olave & Lee, 2019) may help with overcoming these challenges. The challenges with this in the original study arose especially from the assumption that a given organisational setting would render research groups similar. Of course, ‘organisational setting’ itself is a code word for a whole host of assumptions about structures, practices and institutional logics. The initial approach of the study that we used as an empirical illustration in this paper fell short both in the proper conceptualisation of these and in the recognition of the analytical value of within-case analyses.
- The relation of methods and comparability: at the point where ethnographic research would work with more thick descriptions, the approach chosen as an example here used qualitative content analysis based on the coding of material. Scholars from the field of comparative education suggested a very similar approach in the 1960s (Bereday, 1964; Hilker, 1962), although with a realist notion that takes for granted that comparability is a characteristic of the research object and can be discovered through an empirical analysis. From a constructivist perspective, one would rather say that ‘the comparator’ (Deville, Guggenheim, & Hrdličková, 2016a, p. 29)—the one who investigates, and the comparative devices and comparative methods used for the analysis—is creating comparability, or as Robinson (2016, p. 310) puts it, is ‘making comparability’. From such a perspective, one also has to consider that different methods of data collection and data analysis create comparability in different ways. Using content analysis and coding instead of an ethnographic thick description as methods of data analysis and as the basis for the comparison leads to a comparability that is different from the ‘thick comparison’ that comparative ethnographers target (Niewöhner & Scheffer, 2010a, p. 2). Such differences indicate the limitations of our study and lead to the last point of this discussion.
- The application and methodological reflection of qualitative methods: to further develop qualitative comparative higher education studies on the basis of the approaches proposed here, it would be a) necessary to design and carry out studies that incorporate these ideas and methodological suggestions from the outset—for example, right from the planning stage of the research design. It would be b) indispensable to explicitly reflect on their methodologies as well as their analytical benefits and feasibility, and it would be beneficial to compare the analytical value of studies based on different methods of data collection and data analysis. It would be c) productive to broaden the range of qualitative methods used in comparative higher education research and to make greater use of the ethnographic studies. Although such studies have typically been used in the fields of the sociology of science and STS since the seminal laboratory study approach was developed in the late 1970s and early 1980s (Knorr-Cetina, 1981; Latour & Woolgar, 1979), they are quite rare in higher education research. However, international comparative ‘class-room life’ studies or ‘organisational life’ studies in universities might be able to offer compelling insights into the practices within higher education and would enhance the variety of methods used in comparative higher education research.
In closing, a few words on the commensurability and incommensurability of the suggestions by Sørensen, Schmidt, Bereday, Hilker and Berry should be added. The fact that their epistemological perspectives are incommensurable and are not equivalent enough to integrate parts of their analytical techniques and procedures is, of course, a valid point of critique. In this respect, we deliberately work with bricolages and patchworks to inspire a thinking towards creative yet equally methodologically strong qualitative ways of doing comparative analyses and making comparability.
ENDNOTES
- 1 Most extensively, the problem of context in comparative studies has been discussed and developed for quantitative survey or secondary data research in the method literature (for overviews, see, e.g., Davidov, Schmidt, Billiet, & Meuleman, 2018; Harkness, 2012; Harkness, Van de Vijver, & Mohler, 2003). Meanwhile, a pronounced lack of methodological considerations exists for qualitative comparative research in the method literature. A partial explanation for this lack of reflection of qualitative comparisons in the methods literature might stem from the fact that quantitative comparative procedures and techniques have largely become dominant with the proliferation of large cross-country data sets and also the data processing techniques itself have advanced enormously (e.g., Hölscher, 2017; Seeber, 2019). Another partial explanation might be related to the fact that hermeneutic and reconstructive methods of data analysis often consider a comparison to being their normal practice of research and basis procedure of analysis (e.g., in the grounded theory methodology (GTM), the documentary method, or the narrative analysis. Therefore, scholars who have developed these methods might not see the need for any particular reflection when it comes to their application in international comparative research, as the methods themselves are context-sensitive. For further discussion of the specificities of qualitative research methodology, see Gläser & Laudel (2013). A lot of the literature from the field of cross-language research focuses on problems involving working with translations (e.g., Berman & Tyyskä, 2011; Hole, 2007; Temple & Edwards, 2002). However, this type of literature offers only a little help for the challenges facing qualitative international comparative empirical research.
- 2 A realist/positivist ontology and epistemology would assume that the social world exists as it is, and it is structured by rules and regularities that the social scientist can discover. Thus, a realist methodologist would argue that if we are just getting the comparison and comparability right, we should be able to discover these rules. A critical realist ontology acknowledges the existence of a social world out there but sees the knowledge that we gain about it as socially produced. Accordingly, the comparative knowledge that comparative social science can gain is produced via its methods, and thus, the critical realist methodologist would emphasise method reflection and development. A constructivist ontology and epistemology would see the social world as made (‘constructed’—to what extent depends on how radical it is), and thus, a constructivist methodologist would argue that we are making the comparative object and the comparison.
- 3 Referring to Lawson (2003), Bergene (2007, p. 12) labels the types of regularities that a comparative methodologist, who works based on critical realist epistemologies, uses as ‘demi-regularities or demi-regs’, as they strictly belong to the empirical domain and not to a ‘real’ social domain.
- 4 Sørensen (2010) emphasises that an ‘inside description’ is not ‘more authentic or truthful than approaches that apply a priori and “outside” classifications to empirical data. An “inside description” is fabricated in intra-action among researcher, instruments, research object, etc., just like any other research account’ (Sørensen, 2010, p. 44).
- 5 One can say that through the ‘etic lens’, the researcher ‘views the data in tacit reference to a perspective oriented to all comparable events’, and through the ‘emic lens’, the research ‘views the same events, at the same time, in the same context, in reference to a perspective oriented to the particular function of those particular events in that particular culture, as it and it alone is structured’ (Pike, 1967);(quoted in Berry, 1999, p. 186).
- 6 At the same time, although the country/organisation contexts were selected following the aforementioned presuppositions, questions of feasibility and access also influenced the selection of the individual cases, namely the research groups, in these three relatively small countries (Nokkala & Diogo, 2019).
- 7 Przeworski and Teune (1970, p. 30), influential thinkers in comparative politics, even wanted the role and task of comparative research to ‘[consist] of replacing the names of social systems by the relevant variables’.
- 8 See for the distinction between the context of discovery and context of justification: Popper, 2008.