Grand Theories or Design Guidelines? Perspectives on the Role of Theory in Computing Education Research

Over the course of its maturation as a research discipline, the field of computing education research (CER) has seen a significant change in the field’s publication profile. From its early beginnings in the 1960s until the late 1990s, the CER literature base saw more than its fair share of anecdotal experience reports, course overviews, teaching tool descriptions, proofs of concept, essays, program surveys, nifty assignment expositions, and other similar non-empirical outputs [23, 75, 83, 93]. One of the more common types was the infamous “Marco Polo”-type article: “I went there and I saw this” [93].

But at the turn of the millennium, a sea change began, as an increasing number of conferences and journals started to require researchers to present empirical evidence for their claims. Many early calls for increasing the rigor of computing education research focused on the field’s research methods [75, 93]. The computing education symposia, which used to be “swap meets” for sharing one’s pedagogical techniques, useful assignments, or lecture props [31], started to request the authors base their statements on empirical evidence. At the same time—and not completely unrelated—a number of nascent CER research groups and capacity-building initiatives had focused on training bona fide computing education researchers, who were able to deliver just what the symposia wanted [23, 33]. As new methodological standards for research were increasingly adopted, the field saw a gradual transformation in the 2000s [47, 75, 83].

The movement to build CER on a solid foundation primarily focused on methodological standards, but there was an important parallel in many articles: many authors identified a need to critically evaluate the field’s theory base—or lack thereof. For example, Holmboe et al. [37] lamented the field’s lack of reference to pedagogical theory. Goldweber et al. [31] proposed that CER needed to establish relationships between observations and theories. Report after report surveys of CER showed a weak theoretical base in the field [83], and two pioneers of the field declared in 2006 that there is no Kuhnian-type research paradigm in CER [23].

Research studies have shown that CER publishing venues differ greatly in their approaches to the role of theory. Joy et al. [39] showed that CER conferences, with their strict page limits, tend to ignore educational theory more than journals do. Clancy et al. [9] argued that CER articles in education research conferences are better positioned to involve educational theory in the context of CER.

In the 2010s, a steady stream of essays and analyses kept alive discussions about the roles and relevance of theory in CER. Nelson and Ko [63] wrote that in a design-based field, a demand for theory can detract the researcher from the main aims of the field, inhibit the search for CER-specific theory, and create a publication bias that benefits neither theory testing nor the development of better designs for learning. Szabo et al. [89] reviewed the CER body of literature for theory-related research, identified popular theories used in the field, and proposed a template for visualizing various dimensions of theory. Papamitsiou et al. [67] found that despite the marked increase in theoretically driven research in CER, theory choice was rarely featured in the author keywords of published articles.

“Theory,” however, is not a well-circumscribed concept, and compared to fields like mathematics education research, which has seen the rise and fall of many theoretical “-isms” and lived through eras like New Math and the science wars [85], CER has a more limited history of theoretical debates in the field. Like many others, Malmi et al. [51] noted that theory discussions in CER have been complicated by many different meanings of the word “theory.” The variety of theories is broad: for examples of theoretically sound research, one group of conference panelists listed action research, phenomenography, socio-cultural perspectives, and cognitive theories [4]. Isomöttönen [38] theorized phenomena in computing using grounded theory. Pears et al. [70] distinguished between disciplinary, methodological, analytical, and interpretive uses of theory. On occasion, the theory discussants have felt a need to explicitly state whether they are for or against theory in CER [89, p.90]. An important aspect concerning multiple functions of theory complicates the matter: in addition to the many traditional functions of theory, theories also inform scientists about what questions are relevant and how to tackle them, and they serve as lenses for observation, or, as the philosophers of science put it: observations are theory-laden [43].

The discussions about the role of theory in CER feature a wide variety of descriptions of CER as a field [9, 23, 31]. Nelson and Ko [63] characterized CER as fundamentally a design field whose goal is to “discover designs that produce better learning of computing,” and literature reviews are in consensus that designing and implementing tools specifically for computing education is a major part of the field [39, 93]. At the same time, many discussants have emphasized the field’s nature as a research discipline that seeks a deep, fundamental understanding of phenomena related to computing education [50, 83]. Some of them have emphasized the field’s search for (possibly conditionally) generalizable theories that allow prediction [4, 22, 91]. And many combine the views, arguing that a solid foundation on theories about learning improves learning designs in computing.

The concerted push by the field’s opinion leaders and gatekeepers towards a stronger theoretical basis for CER has impressed a new dimension of quality-consciousness on many CER researchers. The aims of that push are laudable, such as moving away from practice articles to contributions of theoretical relevance and empirical grounding [50], optimizing the field’s search for better learning designs [63], enabling derivation of pedagogically useful insights from theory [4], and bringing CER closer to its sister fields in other disciplines, such as mathematics education research [50], physics education research [52], and other similar “trading partners” [23].

The above views on the role of theory in CER, many of which were advanced by the field’s pioneers, have relied on a range of philosophical positions and theories of scientific research. As such, those views exhibit the remarkable depth of disciplinary understanding, and all are well justified from their perspectives. The field has a wealth of philosophically rich contributions. For example, Isomöttönen [38] offered a discussion on theory in the context of grounded theory. Thota et al. defended paradigm pluralism [92]. In their pioneering textbook on CER, Fincher and Petre [22] presented a 76-page deep analysis of the nature and potential of theory in CER; theirs is a must-read starting point for any discussion on the use of theory in the field. Another fruitful analysis of the roles of theory can be found in Pears et al. [70, e.g., Figure.1]. But similar to Almstrum et al. [2] who were, justifiably, of the opinion that “too much of the research in computing education ignores the hundreds of years of education, cognitive science, and learning sciences research that have gone before us,” this article proposes that some theory discussions in computing education research ignore the latest 40 years of research on the philosophy of science, sociology of science, and science and technology studies (STS). This article is aimed at contributing some alternative perspectives to CER’s theory discussions by using results and experiences from fields that have recently contributed to those debates, most of all the philosophy and sociology of science. The article assumes some familiarity with basic vocabulary in the philosophy of science but attempts to avoid jargon and technicalities.

Firstly, the article presents that the historical baggage that comes with the myriad uses of the term “theory” complicates discussions about the role of theory in CER. The article proposes for CER a modern, model-based view of scientific research. Secondly, the article discusses some challenges with borrowing theory from other fields to be used in CER, as well as some open questions about the nature of theories in general. Thirdly, the article suggests that at the heart of theory debates in CER are disagreements that are strikingly similar to the 1980s disagreements about computing’s disciplinary nature and about the role of design in the discipline. The article notes that the conflicts regarding computing’s disciplinary identity dissolved as a consensus was established over computing’s tripartite nature and that the three branches of computing (theory, science, engineering) are irrevocably intertwined. Perhaps that is where the philosophy of CER is ultimately heading, too.

The article presents the above three topics in the context of the philosophy of science, a field with a long history in debates like CER’s theory debates. In that context, the article reminds the reader that many models of scientific explanation quoted in CER’s theory discussions were explicitly about natural sciences and never meant as a model of social sciences, education research, or psychology. Many philosophers of social science have nowadays largely abandoned the quest to claim the mantle of hard science for social science, and sought to understand the field in its own right and its own terms [17, pp.1–8]. Models of natural sciences are neither applicable to nor desirable for many sectors of social sciences; the concerns of the fields are fundamentally different [17]. The article also describes a rift between the early 1900s’ “received view” of science that prescribed a priori rules for how successful scientists should work (see [34]), and the late 1900s “naturalistic view” of science that relies on empirical (e.g., sociological, historical, and psychological) research on how successful scientists actually do work. The article advances the view that instead of appropriating any a priori paradigm from other disciplines, and instead of relying on examples of theory use from fields like physics and medicine, CER can gradually build an understanding of its own paradigm (a disciplinary matrix of exemplars and other elements of research) by investigating how to progress in CER has been made in its influential and most successful studies.

Some aspects of CER’s theory debates resemble mid-20th century discussions in the field of sociology, when the field was seeking a disciplinary identity and a new direction. Some sociologists yearned for a general, all-encompassing theory of sociology, while some other sociologists wished to follow the natural sciences. Both views, however, led to trouble—the former due to its overly broad scope and lack of straightforward applicability, and the latter much due to misconceptions about the natural sciences those sociologists wished to imitate, including overplaying the role of inductivist reasoning (from a lot of data to theory), ignoring the maturity phase of a discipline (thinking that sociology, a new field then, would function the same as more established fields of inquiry like physics), and speaking as if natural sciences had an all-encompassing “grand unified theory.” It seemed that grand theories invariably faced the dilemma of being either trivial or wrong [60, p.70]. In this context of the field’s methodological debates, the sociologist Robert K. Merton [55] presented the concept of middle-range theories.

Not unlike many CER discussions of today, Merton [55] described a scientific theory to be a set of propositions that serve as grounds for deriving “empirical uniformities.” But there is much more to the picture. A theory alone is never a sufficient basis to carry out such derivations: auxiliary propositions, assumptions, and hypotheses are necessary to carry out the intended cognitive processes (e.g., [73]). Methodologically speaking, experiments never reveal anything about a single theory, but about the whole experimental setup, including also the instruments involved; one’s theory about how the instruments work; a whole universe of auxiliary assumptions, hypotheses, and theories; and epistemological and ontological standpoints (e.g., [71]). Merton noted a large gap between a very general, high-level “grand” theory and what Merton called evolving working hypotheses that are used in daily research, close to data. He argued that to maintain a relatively uninterrupted chain between data and general theory, middle-range theories are needed.

Middle-range theories are separate from sweeping general theories, such as variants of constructivism in CER, as they are not logically derived from them—but they are also separate from data in the sense that they are not simple generalizations therefrom (such as statistics of eye-tracking data or keystroke patterns in CER). So, methodologically speaking, a middle-range theory is an entity that lies between theory and empirical observation (or data). It is logically (inferentially) independent from both the most abstract level of general theories as well as from data at the observation level. Middle-range theories stand in between abstract theories and empirical data also substancewise, adding domain-specific information to the picture and having, unlike general theories, a closer link to empirical observation. Different from the views of science of the early 1900s, such as logical empiricism or naïve inductivism, the image of science sketched by Merton was not purely inductive (theories are generalizations of observations) or purely deductive (observation sentences are derived logically from theory).

In CER, general theories are loose-knit, diverse, and overlapping (e.g., [55]). Take some of the many theories used in CER, for example, zone of proximal development, mental models, and Maslow’s hierarchy of needs [89]. Not all the theories used in CER are straightforwardly operationalizable for the characteristic uses of theory typically mentioned in CER literature (such as prediction or interpretation of findings. For the aims of theory in CER see, e.g., [52, 63, 89]). Not all the theories used in CER offer readily applicable prescriptions for generating new learning designs, either. The concept of middle-range theories was intended to bring about a focus on distinct problems, allow for new hypotheses, direct attention in novel ways, and thereby furnish empirical work [55, 57].

The descriptions of the theoretical underpinnings of CER by Malmi et al. [52] and Szabo et al. [89] reveal challenges similar to those that vexed Merton and his successors. The desire to come up with broader and more predictive theories in sociology was aimed at extending the reach and utility of the discipline [55], the mastery of society, predictive power, control, and “human engineering” [57]. But emulating a misconceived picture of natural sciences turned out to be counterproductive, both towards those aims and towards the other aims of the discipline as well [57]. Similarly, CER debates need not be structured around simplistic, idealistic views of scientific investigation or explanation. For instance, Popper’s falsificationism, the deductive-nomological model, and inductivist views of science—all referenced in CER’s theory debates—advance narrow views of the roles of theory, experiment, and data [36, 71, 94].

It has been pointed out that it will be important for the nascent philosophy of computing education research to analyze what kinds of meta-theory, or “theory of theory,” the various arguments for the role of theory in CER adopt (Malmi et al. [51] refer to this as the ontological nature of theory). The philosophy of science has seen upheavals since the 1980s, but those changes are not strongly visible in CER’s theory discussions. There is room for richer discussion on whether the theory of theory adopted in CER theory discussions is closer to Hacking’s [36] or Pickering’s [71] view of theories as interventional elements that integrate and are integral for technology development, or whether CER’s theory of theory is closer to the syntactic view of the 1920s and 1930s Vienna Circle (see, e.g., [7]). (For reference, instead of the classic 800-page tome on the topic by Suppe [87], Winther [96] offers a more accessible and more up-to-date overview of the structure of scientific theories).

From a modern perspective, Merton’s view of science is, like its predecessors, relatively simplistic and more concerned with ideals of the time than what really happens in successful science projects (e.g., [36, 44, 71]). His idea is narrowly defined as sets of propositions, but he does broaden the view, in the end, to include “general orientations toward data, suggesting types of variables which theories must somehow take into account, rather than clearly formulated, verifiable statements of relationships between specified variables” [55, p.52 in original; p.458 in reprint], so Merton clearly acknowledges a variety of cognitive aspects of scientific work. Yet, the search for middle-range theories describes the task for a computing education researcher, like the task for a sociologist, according to Merton, but with a proviso that an eye for general concepts and propositions is kept.

Middle-range theories may be what many CER theory discussions seem to primarily mean by “theory.” Isomöttönen [38] is explicit about that, but the view is implicit all around. A good example is Robins et al. [78, p.262], who present a hierarchy for describing CER, where theories at the functional level connect to a higher level of more abstract pedagogical concerns as well as to a lower level of cognition and behavior of individuals. Their view highlights an important point about focusing on the level that matters, even when higher and lower levels of abstraction set some necessary conditions and boundaries. How Merton saw the state of sociological research resembles in many ways how many people see CER today. Merton’s proposal was to introduce an element that can bridge the most abstract level of theory and close-to-observation level of data. The following sections present different approaches to that.

In the absence of a field’s homegrown theory, it is wise to investigate what others have done in similar situations [62]. For instance, mathematics education research has extensively discussed the roles of theory in the field [86], including theories “borrowed” from other fields, adapted, or developed into new “home-grown” theories particular to the discipline [85]. But in discipline-based education research (DBER) fields, the roles of learning theories and principles vary widely by field, and all DBER fields are tightly connected with their parent disciplines, complicating direct borrowing of theories and results [62, pp.52–53,188]. Still, where CER lacks theory, a neighboring field or a related field is a natural place to look for. The analogies needed to do that may be very near or far, they may prove to be useful and effective, or they may be ill-fitting and fail. For example, theories from cognitive psychology can help designers think about how to direct learners’ attention to critical components of the learning material [88]. Theories about developmental stages can help understand what reasoning skills can be expected at different grades in school [88]. But the adoption or adaptation of theories from other fields requires an understanding of context: the challenges of learning differ between learning multiplication tables, methods of proof, chemical reactions, playing the piano, musical composition, languages, or programming [62]. This section presents a number of concepts for the analysis of analogies, theory borrowing, and strength of the theory.

Many disagreements over the role of theory in CER seem to stem from a lack of consensus over the nature of the field. Those who have characterized CER as an experiment-based empirical science have evoked concepts, nomenclature, and relationships between concepts from the standard literature in the philosophy of (natural) science. Studies on the topic cite Popper’s falsificationism [22, 63], Kuhn’s paradigm theory [22, 23], Feyerabend’s anarchistic theory of science [37], positivism [92], or Hempel’s deductive-nomological model [22]. Those accounts of science involve four classical aims of science: to explore (develop an initial understanding of an uncharted phenomenon), to describe (systematically record and model a phenomenon and its relationships to other phenomena), to predict (use existing knowledge to predict phenomena that have not yet come to pass), and to explain (clarify the causes, relationships, and consequences of a phenomenon).

But there are also those who see the field as primarily a design and engineering field. Many hold that the aim of CER is not primarily that of building descriptive or predictive theories but something else—such as improved designs for learning [63], innovation [54], or development of tools for learning [49]. Of Fincher and Petre’s [22] ten areas that motivate research in CER, many are about developing and evaluating educational interventions for computing education, be they technical or pedagogical. A survey of ACM SIGCSE articles between 1983 and 2003 found that 22% of articles presented tools for learning, and the sample identified more “tools” articles than any other type in 1994–2003 [93]. A series of studies by Simon (reported in [83]) found similar results in various other CER publishing venues.

There are crucial paradigmatic differences between those two different views of the nature of CER. The status of theory differs between sciences, design/engineering fields, social sciences, and humanities. Research articles that focus on the development of artifacts—software, hardware, or learning environments, for instance—are not expected to conform to the ideals of theory-driven research presented by philosophers of (natural) science mentioned above. The most suitable reference for design and engineering in CER is not the philosophy of science but the philosophy of engineering and the philosophy of technology. Philosophers of engineering and technology have shown that the essence of design and engineering lies in their aims: to develop tools or artifacts that accomplish classes of tasks more efficiently (e.g., [25, 58, 95]). Carl Mitcham, a prominent philosopher of technology wrote that while engineers do apply theories from other fields, “artifact design is what constitutes the essence of engineering, because it is design that establishes and orders the unique engineering framework that integrates other elements” [58, pp.146–147]. In engineering, theory is not the goal; the theory is subservient to the engineering aims.

Instead of theories, much of engineers’ knowledge is expressed as technical maxims, state-of-the-art solutions, and guiding principles—and similar to how Tenenberg and McCartney [91] characterized CER, that knowledge is tentative, conditional, and contextual [58, 95]. Designers and engineers achieve their aims by following constructive and descriptive methods that aim at achieving change in the affairs of the world [42]—such as developing and testing explainable AI, new ways to facilitate lifelong learning in computing, tools that enable teacher trainees to understand machine learning concepts, or tools that substantively change how collaborative learning happens in programming. As contrasted with natural sciences, the engineering method is described as “the use of heuristics to cause the best change in a poorly understood situation within the available resources” [42]. Engineers and designers often have to get things done relying on information that scientists would consider inadequate for research purposes (cf. [95]).

Yet tinkering, toying, or just building things is neither engineering nor design. The engineering-specific knowledge base that technology and engineering researchers apply in their work comprises state-of-the-art concepts, heuristic prescriptions, best practices, and procedural knowledge of what works, embodied in (1) procedural knowledge (“know-how”), (2) technical maxims, rules of thumb, heuristic strategies, or “recipes,” (3) descriptive experience-based “If A then B”-type laws, and (4) technological theories on how to apply scientific theory to practice [58].

The field of computing has a wealth of examples on how to communicate design knowledge—take human-computer interaction or software engineering, for example. Design patterns in software engineering [27] are a way to communicate heuristic strategies to work out common situations object-oriented programmers encounter. Design hints in operating systems [46] are a way to communicate procedural knowledge and technical maxims at the level of complete systems. And design principles in information protection [79] are a way to communicate rules of thumb about how to reduce the risk of compromising sensitive information. It has been suggested that in design, theories best inform design as counterfactuals, through thought experiments of the kind “If design was <like this>, then interaction would be <like that>” [65]. In those counterfactuals, theories can direct design choices, help designers choose between design choices and expose new design spaces [65].

One central reason for the separation of the philosophy of engineering, technology, and social sciences from the philosophy of (natural) science has to do with the different ontological views of the fields they study. The subjects of physical sciences—such as properties of particles in fields of force—are mind-independent [81]. The subjects of social sciences—such as preferences, behaviors, and mental states—are mind-dependent but are separate from the researcher [81]. The subjects of engineering are human-made artifacts, which have to cater to the laws of nature, to value and worth in the human experience [11], and to a range of values from budgets and schedules to fashion statements. Insofar as one of the most common aims of CER is to develop artifacts to help the learning process in computing [49], those studies need to be judged in the framework of the philosophy of engineering and technology and not by the standards developed in the philosophy of (natural) science. And insofar as CER is concerned with socially constructed phenomena—take collaborative learning, sociocultural practices of learners, or inequality, for example—then the philosophy of social sciences [16, 17, 61] offers other fruitful starting points for analyzing the role of theory in CER.

This essay has proposed that many equivocations that bedevil discussions on the role of theories in CER arise largely from the adoption of a “received view” of what theory is [34] and from a simplified view of theory’s role in scientific explanation. “The models are broken, the models are broken!” would Allen Newell exclaim.

The “classical” models of scientific explanation—such as Popper’s [72], Kuhn’s [43], or Lakatos’s [45] accounts of science—were designed with natural sciences, not arts, humanities, social sciences, or education research, in mind. Even if a procedure for conducting science works in natural sciences, it does not follow that it works in CER, a field fundamentally different in kind. (It does not follow that it would not work, either.) What is more, each model has been later shown to be inadequate even in natural sciences (e.g., [20]). Disciplinary discussions that cite the classical models of science should also take into account their shortcomings, their original context, and changes in scientific practice in the past 60–100 years. Logical positivism reigned a hundred years ago, Popper presented his influential view in the 1930s, and Kuhn’s game-changing book was published in the 1960s.

Many models of scientific research mentioned in CER’s theory discussions are a priori models, formulated in specific intellectual and historical contexts as ideals of how natural science should ideally be done. Many of those models were developed without getting much involved in the messiness of the real world when formulating methodological prescriptions for science—take Popper’s, Hempel’s, and Carnap’s accounts of science, for instance. Their views, pioneering as they were, have been shown to be based on preconceived, idealistic ideas of science that insufficiently capture how (especially 2020s) science really works in practice [36, 43, 71] or how it should work [21].

As opposed to the earlier a priori models, towards the end of the 1900s, the naturalistic turn of the philosophy of science started to shift the field from the shibboleths of analytically oriented philosophers towards an image of science deeply rooted in the reality of scientific practice. Historians of science looked deeper into the archives and found controversy, obstinacy, and irrationalism [21, 43]. Psychologically oriented studies found ways in which many seasoned scientists neither adhere to the idealized norms and ethos of science nor believe in them (cf. [59]). Sociologically oriented philosophers opened the lab door and painted fresh, new images of how successful scientists actually work [36, 71, 94]. Naturalistic epistemology is an extension of sociology and psychology, and it treats construction of scientific knowledge as an empirical matter instead of a normative and a priori matter. Like other fields of research [20, 21, 59], CER may not be immune to many problems a priori models ignore. Discredited theories may go on for decades [32, pp.46–50], intuitively appealing folk theories may prevail despite any amount of evidence to the contrary [3], and organized dogmatism, bias, and self-interest may sometimes surface [3]. This essay suggests that it would be fruitful for the philosophy of computing education research to anchor the image of CER to naturalistic epistemology and to the naturalistic turn of the philosophy of science.

This essay presents that by shifting more towards a model-based view of science, CER can avoid some problems that arise from centuries of conceptual baggage related to the concept of theory. Compared to theory, model is a much broader and more accommodating concept, and it does not give rise to as many expectations and demands as theory does (even if those arise from misunderstanding the role of theory). For example, no one expects a didactic model be verified or proven correct [sic]. No one thinks it has to be universally true. No one expects it to be tested across all types of learning situations and circumstances [40]. Models, by virtue of their very nature, are not assumed to be “true.” The demand to anchor findings in a theoretical framework somehow sounds much more intimidating than the demand to relate the findings to models related to learning—yet the actual objects can be the same.

This essay also suggests that a different perspective on the role of theory is necessary for those parts of CER that are oriented towards improved or new (artifact) designs for learning. Compared to the philosophy of science, the philosophy of engineering and the philosophy of technology provide better-suited analyses of the role of theory in the development of software or hardware artifacts. Artifact design has a rich and important history in CER, and it should be judged in terms of its own intellectual tradition instead of that of hypothesis-driven science. And compared to the philosophy of science, the philosophy of social science frames theories in ways more amenable to the logic of certain types of research on the behavior of individuals in various communities.

This essay is aimed at providing alternative perspectives to theoretical discussions in CER, and as such, it is limited in several important ways. It specifies only a very limited number of models, objects of interest, and their relationships. It covers only a few select perspectives of models from the many viewpoints that the philosophers of science have presented. It uses many fewer examples from CER than one would hope: the authors work in narrow sectors of education. In its advocacy of a model-based view of science, it gives much less attention than it could to the enormous body of research on the structure of scientific theories. It does not discuss the nature of products of research—and whether they are of logical type, descriptive statements, recommendations, or something else. It does not use results from the philosophy of computing education research or from the broader philosophy of computer science education. And because the strengths of theory-driven computing education research and technology development are already well covered by numerous other authors in CER and broader DBER, the essay probably fails to give enough credit to the fact that theory (of some kind) is often at the heart of excellence in research.

Despite its many shortcomings, we hope the essay gives rise to discussions about whether even a moderately naturalistic, model-based approach to the “theory of theory” would pay off in terms of better fit between CER theory discussions and how practicing researchers in CER see their own work. The essay suggests, not in a Quinean strand of naturalism, but perhaps in a Kuhnian strand of naturalism, that we take seriously what happens in CER in action, even when researchers do not adhere to a physics textbook depiction of theory fit in research.

Just as Kuhn’s major contribution was to reveal the importance of social structure of science communities, scientists, and their relative agreement, it would be an important contribution to CER to study the generally accepted landmark studies of the CER field and analyze how those studies actually use theory and models (instead of how philosophers of natural science prescribe they should be used). For instance, one can study the field’s most cited articles, or the most impactful ones, or perhaps the winners of best article awards in the field’s major conferences. That would help build a consensus of CER’s paradigm and describe the field’s take on theory in CER’s own terms. One aspect to keep in mind in that undertaking is not to let an unwarranted preconception of science be the criterion of good research.

Matti Tedre thanks January Collective for their meticulous but constructive critique on this manuscript.