1 Introduction
HCI has long been concerned with finding ways to address the societal impact of computing systems [13, 47, 49, 64, 77, 87, 92]. While engineers and computer scientists building systems often ground them in visions of positive societal impact, these systems often have unintended consequences (e.g., [8, 99]). As computing technologies have become ever more ubiquitous and their unintended effects more obviously problematic, HCI researchers continue to explore ethical issues in computing systems development and how to better integrate considerations of societal impact into their technical design [11, 14, 69, 95, 105]. Fundamentally, this work is grounded in the idea that making computer scientists aware of the possibility of particular societal consequences of technical choices will enable them to make better decisions about what to build. This paper examines what happens to ideas about societal impact when they become caught up in the everyday realities of producing computer science (CS) research. How do CS researchers envision societal impact through the process of envisioning, building and publishing about research systems? How are they orienting towards these ideas of societal impact in the decisions they take in their work? What factors shape the forms of societal impact they take into account, and that they claim for their systems? How and to what degree do these ideas shape what is actually built? This work aims at uncovering technical, institutional, and disciplinary factors that shape how particular research systems are understood to “have” societal impact and the work these claims do to produce particular emerging sociotechnical worlds.
Our work suggests that such structural factors torque how societal impact is imagined, how those imaginations become embodied in concrete systems, and how these systems become enrolled in claims of potential societal impact that motivate publication and further research. These structural dimensions complicate the idea that changing computer scientists’ thinking about and valuing of societal impact is adequate to produce systems that make a positive impact on the world. They also suggest new avenues for both technical and HCI researchers to shape societal impact by making topical and structural interventions into how technical research is funded, organized, and reviewed.
To make this argument, we integrate social observation and analysis with hands-on technical development. Our work builds on Agre’s critical technical practice [
3,
4]: a unified practice of technical system-building paired with critical reflection on sociotechnical factors that shape a system’s development.
We build on four years of critical reflection paired with ongoing technical development of a networking research system to articulate how structural factors shape the ways technical researchers orient to societal impact, and their material consequences. This paper, then, presents an insider’s perspective on the day-to-day practice of building computing systems with societal impact in mind.
The context for this paper is an area of CS research called “systems,” which aims to improve the performance and reliability of computer systems and networking. Systems research
focuses on computational infrastructure, including operating systems, mobile computing, and interprocessing management; the research area we focus on in this paper is networking.
1 Specifically, we investigate the context of building a technical platform for digital agriculture (or DA), which entails data collection, transmission, analysis, and actuation on rural American farms [
12,
66,
89,
98]. The platform, dubbed the
Software-Defined Farm (or SDF), was originally envisioned as supporting societal impact by addressing the need to feed a growing world population. As the platform has developed, our understanding of societal impact issues in digital agriculture has refined to encompass addressing infrastructural inequalities between rural and urban spaces and reducing risk of vendor lock-in, whereby consumers become locked in by design to particular hard- and software suppliers. Systematic reflection as we build this emerging infrastructure has revealed aspects that enable, shape, constrain, or reorient how we envision societal impacts.
Our study of the SDF relies primarily on critical technical practice because it allows an extraordinary level of access to detailed technical decision-making, while viewing those decisions through a critical lens which can bring powerful new insights. As critical maker Ratto argues,
[C]urrently the distance between [technical and social] areas of knowledge remains vast — often even within the geography of the university campus. However, there is an important need for critical makers that can reintegrate technical and social work and thereby innovate both. [
80, p. 258]
To ensure that we are not blinded by assumptions inherent in a personal narrative, we supplement the critical technical practice with an outside perspective through ethnographic observations of SDF technical work. Building on both perspectives, our analysis examines how ideas and claims about societal impact are enacted and oriented towards within the everyday practices of envisioning, building, and publishing about research systems.
Our key argument is that, even when computing systems researchers envision and work toward pre-determined societal impacts, they work towards these envisioned impacts within institutional, infrastructural, and disciplinary structures that gradually rework this envisionment in both direct and subtle ways. The results section presents a detailed, reflexive account of processes of envisioning societal impacts with differently situated stakeholders, and describes how those envisioned impacts are transformed as we set technical goals and build infrastructure, disseminate our findings and navigate CS peer review, and produce accounts of the success of our work for different stakeholders. We show how, throughout these activities, we must continuously work across technical, institutional, and disciplinary seams. We characterize such maneuvering across sociotechnical structures as ‘seam work’. Further, we argue that navigating these seams continuously refactors the way the system and its envisioned impacts are represented, narrated, and oriented towards, in response to structural pressures at each seam. Thus, we argue that, while researchers do have agency to work towards societal impact, our agency is compromised [
63] by our placement within institutional structures that subtly guide research agendas and outputs toward outcomes that benefit or reinforce existing financial, legal, intellectual property, or other structures. As critical technical practitioners, we are far from being innocent outsiders to such pressures; we must also work with(in) these institutional and disciplinary powers to attempt to maintain course toward envisioned societal impacts. We articulate how critical technical practice supports such navigation.
We argue that convincing computer scientists to care about different values may be ineffective, given the technical, institutional, and disciplinary constraints on how such care can be enacted in technical work. We propose three alternative ways that the HCI and computer systems communities can work together to better support technical research aimed at improved societal impact: by redesigning our institutional and disciplinary systems, by systematically deploying critical technical practice as an effective tool for investigating infrastructure development, and by fostering what we term trilinguals of the future — i.e., individuals bringing together computing technology, domain understanding, and critical reflection — as a way to improve the broader impacts of computing.
2 Related Work
This paper is particularly concerned with reflections on societal impact within academic CS research. A key
HCI approach to exploring how computer scientists do and could orient to the societal impact of computational systems frames these issues as questions of values and design [
39,
73,
75,
76,
90,
93]. This work focuses on how technology developers often unconsciously build their own values into systems, long before they become user-facing end products. Often, these approaches pair social scientists with expertise in ethical and social issues with developers with technical expertise, with the aim of collaboratively identifying and altering assumptions in early system design that may have negative consequences downstream. For example, Cheon and Su worked with robotics researchers to envision their own future autobiographies as a way of exploring values issues in robotics [
22].
As this work has evolved, researchers have identified recurrent challenges in engaging computing researchers in discussions around values and unintended societal impacts that arise from their disciplinary and institutional situation. For example, Shilton et al. describe challenges in engaging CS researchers in values discussions that arise because these researchers frame the goal of building general-purpose systems as requiring values neutrality [
91]. More generally, as Ribes et al. have described [
81], CS researchers often frame their work in terms of a separation between more general-purpose technology and specific domains in which that technology would be applied. In this framing, computer scientists oversee the general-purpose technology, while domain experts are in charge of domain-specific details. This separation would tend to suggest to practitioners that societal impacts are part of an application domain, and thus not in computer scientists’ purview.
Recently, Do et al. explored how academic computer scientists orient towards unintended consequences of their work [
28]. They showed that while computer scientists did care about unintended consequences, they rarely took action to address them during technical development. Causes included a lack of structured proactive methods, faith in their own estimation of the societal benefit of their work, the siloing of CS from community and other disciplinary experts who may be better placed to anticipate unintended consequences, and the pressure to publish quickly.
In this paper, we build on this prior work to explore the factors that enable, shape, constrain, and often blinker the consideration of eventual societal impacts in the everyday practices of academic CS. While Do et al.’s work relied on retrospective accounts by computer scientists, we here focus on how these pressures manifest within the everyday, mundane work involved in CS research, in ways that often escape computer scientists’ notice as they focus on the difficult task of producing publishable research.
We particularly focus on the envisionment of impact, rather than the eventual actual impact of the systems being built. By “societal impact”, we refer to broader impacts of technical systems in end users’ lives beyond the simple technical functioning of the system. We understand such societal impacts to be produced by an unfolding, long-term interaction between technical possibilities, political-economic forces, and the actions of stakeholders in navigating each of these (e.g., [68]). This process is incompletely predictable at design time; while engineers often aim towards specific possible impacts for their work, its actual impact will not be known until long after the work has been taken up in society [35]. Thus, our paper engages with the prospective imagining of societal impact in technoscientific research, also termed speculation, expectation, envisionment, or sociotechnical imaginaries [61]. Such speculation plays a central role in technoscientific research, since imaginaries guide engineering ideas about what can and should be built [65] and are used to enroll funding, labor, and other resources to ongoing projects [52, 57]. Researchers can and must use visions of societal impact to argue for the legitimacy of their project, in ways that are constrained by the present-day logics of the stakeholders to whom they are accountable [52]. Visions of societal impact matter, not because they predict the future, but because they performatively shape it, though not always in the ways that researchers intend [61]. Our work focuses specifically on reflection on societal impact within academic networking research. Here, we build on a small but significant literature in HCI and CSCW exploring social dimensions of computing from the perspective of networking researchers. Such researchers have explored how communities develop and maintain their own networks [
30,
32,
33,
34,
42], cultural and societal challenges in efforts to establish new networking infrastructure [
15,
16,
17,
31], how networking design choices do and should land in users’ lives [
23,
24,
25,
26,
36,
43,
48] and considerations for the design of new networking infrastructure from communities that are structurally underserved [
18].
Taken as a whole, this body of work showcases a deeper engagement with social science among (some) academic networking researchers that run against the previously reported narrative of disengagement with questions of societal impact [
28]. Further, it illustrates the complex sociotechnical dimensions and consequences of networking infrastructure. In so doing, it demonstrates the power of integrating deep familiarity with the technical dimensions of networking systems-building with social-scientific consideration for HCI/CSCW. As Dourish demonstrates, the material specificities of networking systems are deeply consequential [
29]. As Edwards points out, these “lower layers” of infrastructure are crucial for HCI to reflect on, yet access to them is mostly outside the HCI methodological toolkit [
37]. Our work adds to this interdisciplinary conversation by developing approaches to critically explore the structural possibilities and limitations of academic networking research itself during the everyday practice of doing such research.
To do so, we build on Agre’s notion of critical technical practice [
3,
4]. Critical technical practice is a methodology that integrates computational development with critical reflection such that “rigorous reflection upon technical ideas and practices becomes an integral part of day-to-day technical work itself” [
3, p. 3]. Agre used critical technical practice as a tool for identifying inherent conceptual limits in how Artificial Intelligence (AI) was framing the problem of producing intelligent action, identified alternative conceptualizations of such action, and developed technical innovations which embodied these alternatives [
5,
6]. Similarly, critical technical practice has been drawn on in HCI to explore and alter values and assumptions built into computational systems [
1,
38,
58,
60,
86,
87]. But Agre did not just use critical technical practice to reflect on and improve technical systems; he also used it to explore the structure of AI as a field, as everyday practice, and as a discourse which enables certain forms of work and sidelines others [
4]. It is in this vein that we use critical technical practice here as a tool to reflectively explore what happens when academic CS researchers aim to develop technical research systems with the goal of achieving positive societal impact.
4 Results
Because the results section was produced by Gloire as a self-reflexive analysis of his work and experiences, this section is written from his first-person perspective. In this section, the pronoun “we” refers to the technical team Gloire was working with, which included Hakim and other actors, but not the other co-authors.
In this section, I describe how we oriented towards envisioned societal impacts in different stages of computer systems and networking research in response to opportunities, pressures, and expectations from ourselves, other stakeholders, and our discipline. The first four subsections explore different stages in research development: from project specification, to technical development, to writing up results, to navigating peer review. The final subsection explores how we came to define success to ourselves and project partners at different stages of the project. Each subsection delves into how and why our envisioned societal impacts were in focus, out of sight, re-framed, or inadvertently forgotten at each stage.
4.1 Pitching a Project
The process of orienting towards societal impact started with the inception of our project. The research question in the abstract of the SDF GitHub repository is: ‘how do we sustainably feed 10 billion people in 2050?’ This question was adopted from our university’s collaborative research thrust for digital agriculture, which had inspired our team to begin working in the agricultural domain. As initially formulated, this vision was untethered from technical possibilities. As our research progressed, our ideas of how to contribute to feeding the world through our research evolved in response to funding opportunities, accessible stakeholders, and the needs and desires of different collaborators.
We began our project by listing societal impact goals, defining technical goals, enrolling institutional stakeholders, and submitting proposals for funding.
The initial version of SDF (i.e., SDF 1.0), became funded through grants from the US National Science Foundation (NSF), the US Department of Agriculture (USDA), and CloudAndMore, a large tech company which runs proprietary cloud computing services. We articulated the intended societal impact of our system by developing a pitch that spoke to these stakeholders. We came to envision the SDF 1.0 as providing new networking capabilities while anticipating the societal impacts of high-bandwidth networking on American farms in rural areas. Given the paucity of Internet in these areas, the networking would rely on innovative modalities such as long-range radio (LoRa) [
7,
102] and TV White Spaces (TVWS) [
10,
82,
84]. The envisioned applications for this first iteration of the platform were
developed in collaboration with agricultural researchers at our university; as a result, we intended to support early cow disease detection in dairy farms, early virus detection in vineyards, and measurement of plant water stress at our university’s apple orchards and greenhouses. The realization of the platform thus brought together computer scientists and engineers, animal scientists, biomolecular and chemical engineers, plant scientists, and researchers and engineers from
CloudAndMore,
whose interests were all reflected in the platform pitch.
In order to elicit funding from governmental agencies and industrial partners, visions of the SDF often tout ubiquitous, end-to-end, scalable platforms that speak directly to funders’ keywords. For example, one grant pitch described a system grounded in “(1) tech-enhanced innovation, (2) data integration, and (3) data analytics and decision support” (Research Diary, 01/20/2023), with the planned research system producing a data pathway from corn to cow and back:
As an example, it would be good to monitor the conditions under which the corn which is fed to cows is grown. From there, the cows can also be monitored, and any conditions can be traced back to the growing conditions for the corn. Finally, the manure from the cows can also be sent back to the corn fields. This, in essence, is a closed-loop system that is supposed to achieve the [USDA’s] farm of the future. - Research Diary, 01/20/2023
The apparent success of
SDF 1.0 was a catalyst to enlist new funding and collaborations
for SDF 2.0, which in turn reshaped the societal impacts towards which we were orienting. The
SDF 2.0 project brings together computer scientists, animal scientists, plant pathologists, and research and engineers from another technology giant. It is funded through grants from NSF, National Aeronautics and Space Administration (NASA), and USDA. These stakeholders all therefore play a role in determining what the SDF 2.0 should be for. This version of the
SDF developed in the context of a new university research center focused around a vision of digital agriculture enabling better plant-human communication. The
SDF 2.0 is thus intended to support holistic, two-way communication with plants and animals in an imagined Internet of living things (IoLT). Additionally, based on our experiences managing vendor lock-in during the development of
SDF 1.0,
SDF 2.0 is intended to address the issue of vendor lock-in, where consumers are locked into one (cloud) provider, and moving to a new provider presents numerous technical, security, scalability, and other issues [
78]. In the agricultural context, vendor lock-in leads to challenges in user autonomy and privacy [
41]; it also leads to technical difficulties in farm data aggregation and analytics because sensor data are collected and processed in silos [
83].
In summary, the evolution of the SDF is predicated upon and shaped by long-term partnerships between computer scientists and stakeholders in other domains to work toward shared visions. These partnerships are formally structured through shared vocabularies (e.g., ‘broader impacts’), principal investigators (PIs) and technical leads, and reporting mechanisms intended to keep stakeholders aligned on particular envisioned societal impacts. While the PIs are in charge of formal reports, the system design, implementation and deployment processes are characterized by continuous negotiations across institutional and domain structures done mostly by technical leads (i.e., graduate students, software engineers, and research scientists). This work shapes how the systems are envisioned and pitched, and which societal impact goals are considered relevant and addressable through project work.
4.2 Building Computer Systems
Envisionment of societal impact is not only shaped by stakeholders’ intentions at the start of the project, however. Even as our team was explicitly working toward specific desired impacts, the possible scope of that eventual societal impact was being reshaped during the mundane, day-to-day work of building computer systems through low-level software, hardware, and infrastructural negotiations.
As is typical in networking research, to produce SDF 1.0 I spent years of effort deep in the technical trenches dealing with low-level technical issues. Such everyday technical work requires persistent trials-and-errors that often take many hours. For example, low-level technical issues derailed me repeatedly as I tried to build an Ubuntu image for the Raspberry Pi 4B (Research Diary, 05/20/2022). Sometimes such trouble-shooting even takes months or years:
After a year of troubleshooting, I think I figured out the bug with resending of the [cow] records... it’s because... the protobuf was reducing the precision of the timestamp to be sent to the compute module... The fix was simply changing a float to a double in the protobuf definition of a sensor update. - Research Diary, 01/06/2023
Caught up in these low-level frustrations, it became increasingly difficult to notice how the infrastructure-building process itself could be shaping our eventual impacts. For example, we might decide to work with certain cloud providers or operating systems because they are more amenable to our software development or hardware deployments. A simple hardware vendor choice (i.e., Raspberry Pi 4B) influences, in increasing order of importance, the hardware architecture, the operating system image, the programming languages, the middleware packages, and the cloud provider. These choices can all influence how the ensuing system works, or does not work, to address our societal impact goals, for example by putting us in a situation where vendor lock-in is an unavoidable outcome.
Producing a complex, long-term system like the SDF requires the enrollment of many stakeholders, including undergraduate research assistants, graduate students, faculty members, and partnerships within/across institutions (e.g., industrial partners, building managers, carpenters, etc), who then both support and constrain how our project can unfold. Thus, issues arising from technical constraints are compounded by the number of stakeholders involved in our technical work whose concerns need to be addressed. Our technical infrastructure in principle was defined around a set of desired principles and knowledge production processes we believed would support our technical and societal impact goals, but sometimes these principles had to be revised. For example, we planned to maintain all software artifacts emerging from the research as open source, as is often mandated by funding agencies such as the NSF. Maintaining an open source and reproducible knowledge production process across domains is rife with negotiations, especially in teams involving computer scientists, who will need to coordinate on GitHub repository structures, branch naming and pull requests, etc. Moreover, navigating open-source goals is tricky when working with domains such as animal science that engage directly with commercial vendors providing proprietary hardware and software:
We had the meeting regarding [the progress on] open-sourcing the SDF [1.0] code. Giancarlo’s [research] group raised concerns about the embedding of their commercial [sensor] providers’ names in the dairymanager code. Therefore, we have decided to open-source the portions of the code sans dairymanager, and then the plan is to sanitize the dairymanager code of any commercial [vendor] names before adding it back to the rest of the open-source code. - Research Diary, 12/23/2022
Giancarlo is a PI leading an animal science research lab, and their concerns are valid given their direct relationships with commercial sensor vendors. However, removing vendor information from open source code also stripped the software artifacts of any contextual information on commercial vendors. This removal, in turn, subtly inhibit our efforts to address vendor lock-in by dulling the modular power of the SDF’s software abstractions. Further, it complicated our efforts for reproducible research by slowing down the open-source process.
4.3 Writing Across Disciplines and Academic Rungs
So far, we have seen that visions of societal impact are produced as projects are conceptualized and pitched, and that orientations to societal impact become complicated and reframed during the process of building test systems. But visions undergo a further transformation in the process of producing publications about those systems and their implications. Such publications are an integral part of institutional practices for career success, tied to factors such as student graduations and faculty promotions. They also form the major avenue by which academic systems researchers conceptualize the path to societal impact for our work, with the idea that published insights may be adapted by the community through further research, and potentially adopted by commercial actors to be integrated into widely available products and services.
During the course of data collection, we wrote and submitted two technical papers - one for the systems audience and one for plant pathology. My plant pathology collaborator and fellow PhD student Frank and I co-led this second paper, which presented a vineyard disease application of the SDF and was submitted for review to a plant pathology journal in December 2022. This paper was co-written with our respective advisors, Laura (a tenure-track professor in plant pathology), and Hakim, who is tenured in systems. This paper relied on a parallel paper led by Frank to another plant pathology journal. While his paper presented the plant pathology advancements, our joint paper described how the SDF enabled these advancements.
As Ph.D. students leading the writing efforts, Frank and I found ourselves negotiating across competing domain value systems to produce technical content that is digestible by both disciplinary audiences. The need to navigate multiple domains in writing was further complicated by power differences among authors at different rungs of the academic ladder: Ph.D. students, tenure-track professors, and tenured professors. These aspects came together in an incident that occurred while Frank and I were writing this paper. Upon sharing a draft of the paper with my co-advisor Phoebe, who is tenured in HCI, she challenged my use of the technical term ‘abstraction’ in the manuscript, arguing the plant pathology audience would be unlikely to know what this term meant. Phoebe’s identification of abstraction as an issue in this paper was ironic because we were simultaneously working on a social-scientific paper that was examining the role that abstraction plays in the societal impact of digital agriculture systems, thus I should have been sensitized to trickiness around claims of abstraction. I responded to her email comment as follows:
‘Here, again, Hakim rewrote what I had described... I tried to stay away from words like abstraction. [W]hen Hakim made that suggestion... I incorporated it. In retrospect, that sentence is more digestible to a CS audience than a general one. What’s ironic is that, as a collective of writers (i.e., Hakim, Laura, me, Frank), we went through the review phase with a goal of writing our paper such that both of our communities can grasp everything... [I realize] certain words are simply seeped so deeply in our CS minds that we take them for granted as part of lingua franca’ - Research Diary, 12/09/2022
This incident highlighted several factors. First, it demonstrated how difficult it is to speak comprehensibly across multiple domains, and how the process of negotiating across domains can lead to blinkering of social scientific insights. This led me to put aside my critical perspective to produce technical content for an interdisciplinary audience. As another example, I failed to notice a questionable claim in the draft that “food security through consistent and scalable agriculture is the invisible foundation of modern society.” This discomforted me, since I felt it important to integrate my critical and technical identities and disturbing that I could abandon my critical identity without realizing it.
Second, the blinkering of critical perspectives was exacerbated by the academic power dynamics that led me to defer to Hakim’s favoring of the term abstraction, prioritizing faculty assertions and problem/solution framing. And because on this project it was one of the junior researchers who was the biggest advocate for critical reflection, it was relegated to the backseat. This led to criticism from the other faculty member I was accountable to, Phoebe, adding another stakeholder whose views I needed to manage in the writing process.
4.4 Navigating Peer Review in Computer Systems
The previous section demonstrated how the writing process is laden with challenges in finding the appropriate technical depth, taking CS concepts for granted, and avoiding critical reflection on writing claims in deference to faculty seniority. But challenges that reshape how societal impact is framed continue after drafting publications through pressures from peer review. I will describe this by reflecting on a technical paper whose writing I led, which described the SDF architecture under the pseudonym of Chameleon, Prior to data collection, we had already submitted this paper twice (Conference A and B); during the period covered in this paper, we submitted it for review three more times to different systems conferences (Conference X in July 2022, Conference Y in September 2022, and Conference Z in January 2023).
The systems research community operates under a set of implicit principles and value systems which are disciplined through the review process in ways that shape how work can orient to societal impact. One aspect that reviewers emphasize is novelty. Novelty is understood in systems research as producing new configurations of technology. But many systems designed for societal impact involve getting known technologies to work in different, often low-resource contexts, or altering the political-economic functioning of the system, innovations that reviewers may not see as novel. For example, the SDF replicated the functionality of commercial systems to connect sensors with cloud processing, but did so in a way that reduced the risk of vendor lock-in. This led one reviewer to comment that they “see plenty of description of what all technologies you used. But maybe there was a way to highlight the novelty even better” (Anonymous review, Conference A, 2022).
More generally, the networking community currently favors technology development related to running commercial, large-scale data centers, leaving other forms of domain-oriented research out in the cold.
The recent meeting of researchers that dabble in networking + HCI got us thinking that it may be time to push the community toward being more open to this kind of work. The common issues identified were that the community is too focused on certain kinds of work (ahem datacenters), and this in turn makes publishing and getting funding for other kinds of interdisciplinary [work] difficult. - Research Diary, 5/20/2022
In fact, the apparent hostility in the networking community to interdisciplinary, impact-oriented work was exactly what motivated this group of related researchers to band together for mutual support.
Another value in systems research that makes it challenging to produce research oriented to societal impact is that the community favors generalizability of system ideas over getting a system to work in any particular application domain. This produces challenges for impact-oriented research, which often addresses impact on particular communities or in a particular setting. In our case, systems reviewers characterized our primarily applied work as having insightful deployment experiences for the community, but the “paper itself is a bit under-cooked” (Anonymous review, Conference X, 2022). Comments like these imply that the paper does not contain enough generalizable insight to be worth publishing.
The mismatch between applied research concerned with societal impact and the framings of ‘novelty’ and ‘generalizable findings’ regularly led to negative, sometimes biting reviews. For example, one review of the Chameleon paper from Conference X was completely dismissive of the work and its authors:
One of three reviewers was particularly vicious 3, to the point of saying none of the authors, including our PIs, had any evident experience submitting to a conference like [X]. I’ve been trained by my advisor [Hakim] to build thick skin as a member of [the] systems/networking [community]. The community is very negative (typical acceptance rates for conferences <= 20%), but this [Conference X] reviewer hit a new level of negativity! - Research Diary, 09/30/2022
The typical outcome from such reviews is that paper drafts require multiple submissions before getting accepted for publication in systems conferences. In fact, at two separate top-tier systems conferences, a fellow Ph.D. student described submitting work more than three times before getting accepted.
In our case, this led the system paper to go through multiple double-blind submissions that effectively molded it into a beast that is familiar to the community. The continuous recrafting of the paper in response to review is evident from a reflection after receiving reviewer feedback for our Conference X (July 2022) submission:
We received reviewer feedback from... our third [re]-submission. The paper did not get accepted... [the] theme is... reviewers are still having a hard time deciphering our “scientific” contributions... we took [the] comments with a grain of salt... the surgical changes focused on having clearer takeaways at the beginning and end of each section especially the later sections - Research Diary, 09/30/2022
To counteract the emphasis of the community on novelty and generalizability, we started leveraging a different value in the systems community: the importance of deep system building and deployment of ‘real’ prototypes, as Jen had highlighted in her ethnographic work. My Ph.D. co-advisor, Hakim, often refers to this ethic as ‘getting nerdy with it’. In other words, the community appreciates novel systems that have been kept running in what is considered “the real world” for months or years. As Jen pointed out, during our ethnographic research meetings, Hakim routinely instilled this value as ‘don’t pull the wool over people’s eyes’. Ironically, the more times the paper was rejected, the more we could highlight the length of the running SDF deployments as a sign of quality for the system, which could counteract some of the demands of novelty.
A second tactic that we eventually used to counteract the perceived lack of ‘novelty’ and ‘practical insights’ was to follow up on suggestions from reviewers to submit to a different paper track:
We received 2 weak rejects and 1 weak accept from our [Conference Y]’23 reviewers. Overall, they were excited about the case studies and practical experience. In fact, two of them were surprised that the paper was not submitted under the Operational Systems track. This is leading us to cut down the paper to 11 pages and submit to [Conference Z] (Z ’23) under the Deployed Systems track. This would be the fifth submission of the work, and its potential acceptance could cement my position as a sociotechnical researcher. In other words, I don’t have to let go of my critical perspective. - Research Diary, 12/23/2022
The ‘Operational Systems’ (also known as ‘Deployed Systems’) track is a separate submission track primarily dedicated to applied work. Submitting there was particularly advantageous because those papers “need not present new ideas or results to be accepted; indeed, new ideas or results will not influence whether the papers are accepted” [
101]. However, with a few exceptions [
50,
51], this applied track is mostly dominated by large technology giants with billions of users [
45]
— systems at a scale that far outreaches the scope of what most societal-impact-oriented academic researchers are in a position to produce. As a junior researcher invested in societal impacts, competition against tech giants whose impact can easily reach billions of users, paired with vicious reviews, led to constant imposter syndrome.
While we were eventually able to successfully publish this work, an unintended consequence of the incremental content restructuring this required was the gradual sidelining of societal impact considerations as the paper moved closer to acceptance and took on a form more recognizable to the systems community:
The Chameleon paper got accepted to [Conference Z ’23]!!!! This was a big surprise given the last diary entry! The reviewers actually spent time reading our rebuttal, and it was the deciding factor because it outlined some of the ‘practical insights’ that the reviewers had been wanting... It took FIVE tries for the paper to be accepted, and the acceptance rate was 18%! The reviewers were fair in saying that the insights need[ed] to be pulled out from the deployments... However, our work with the abstraction paper described in the previous diary entry highlight again how the goal is to make the insights general (i.e., placeless) for other [system] designers. Overall, this paper has taken multiple years to get accepted, and the final product may end up [with] a lot of the context stripped out to accommodate the inclinations of the CS systems/networking community.” - Research Diary, 04/28/2023
The abstraction paper referred to above is a critical paper from our project led by social scientists. The paper, which is under review at the time of writing, examines how the disembedding of agriculture from its contexts associated with the rise of industrial agriculture may be replicated in contemporary digital systems for farming, and outlines the risks and costs of such disembedding. It was therefore ironic that a similar disembedding of our own technology was exactly what we had to do to get our work published. In the Chameleon case, we had gradually stripped back the societal impact considerations in the publication to reveal what reviewers would likely consider the “essential” systems/networking insights. In prioritizing publishing our work, and with the goal of having junior members of the team be recognized as members of the systems community, we had engaged in what we term “contextual disembedding:” i.e., we had decided to forgo reporting contextual information about relationships between places, people, and organizations involved in the research, in the hopes that this would make the work seem less “applied.”
4.5 (Re-)defining success
The prior results have shown how envisioned societal impacts become constructed and reshaped through setting visions, building infrastructure, writing up findings, and getting through peer review. Here, I step back from these individual stages to consider how, across these stages of research, we construct success of our project in moving towards desired societal impacts to different stakeholders. Agreements on milestones that define “success” must at least temporarily align expectations between academics, industrial partners and funding agencies. Here, I describe how developing notions of success towards societal impact goals is intertwined with publishing and funding gymnastics.
The envisioning process brings together different stakeholders (e.g., agricultural, technology, academic, and funding partners). First, for agricultural stakeholders, the envisioning involved conducting interviews and visiting their work sites to guide our system design for SDF 2.0. Even within the same industry (e.g., vineyard management), stakeholders face significantly different pressures (e.g., disease detection versus labor optimization) to be seamlessly addressed by the same system, let alone across application domains (e.g., dairy cow management and vineyard management). Thus, “success” for our system is co-defined with stakeholders, based on their specific domain goals.
Second, as envisioned impacts were being brainstormed with technology partners before developing a legally binding joint study agreement (JSA) which would govern intellectual property on the project, I faced a level of uncertainty about what they would consider progress:
... It is unclear if/how working with tech giant will dictate the kind of research questions we can ask. In particular, the tech giant would [likely] push the agenda for [their edge computing platform] and its use case in this context. However, it is still unclear to me if this will be more of an engineering or research effort. - Research Diary, 03/05/2023
It was important for me to understand the nature of anticipated efforts between engineering and research tasks because our reviewers care mostly about the systems and networking research questions and consider the engineering questions to be “merely applied.” Moreover, understanding our technology partners’ incentives is important to keep them engaged because they provide technical resources and help in using their platforms. Is it enough for them to claim that their platforms were used in the system implementation? Is the claim that their cloud services were used in a paper publication sufficient? This practice of probing corporate incentives in research partnerships is not new. I had previously observed similar comments in interviews with academic researchers working to realize the potential of digital agriculture in collaboration with another major tech corporation [
83].
Third, I observed that some academic researchers engage with the ‘broader impacts’ funders ask for in limited ways. Stated otherwise, the broader impacts appeared to be tacked onto research visions to secure and maintain funding:
Giancarlo presented the vision for the Farm of the Future (FotF) to the [tech giant] visitors... as a site for developing and showcasing future farm technologies. This involved four phases: data aggregation, data analytics, data processing, and assessing societal impact. In the meeting, we all joked at the fact that societal impact is mentioned as a last phase. This is funny when paired with the fact that [a sociologist involved in our project] had given a talk earlier that day on the importance of assessing impact in all phases of development/deployment. - Research Diary, 02/17/2023
During system building, to assess whether we are moving towards our envisioned impact, we need to frame it in terms of specific technical milestones. For instance, one envisioned societal impact in building SDF 2.0 described earlier is to address vendor lock-in. More concretely, our platform should not be tied to a particular cloud service provider. Indeed, our team partially achieved this feat:
Ron [undergrad researcher] developed an awesome web app that replicates the tech giant’s web app... [which] has components for displaying sensor box data... statuses,... configurations, and the notifications system. At the core of this project is the reuse of existing Azure services... (e.g. Az Tables, IoT Hub) to provide more cloud-agnostic functionality. - Research Diary, 05/20/2022
However, what counts as a sociotechnical win to us may be perceived as re-inventing the wheel and lack of ‘technical novelty’ by reviewers in systems, since such a system already exists commercially in closed form.
Conversely, what counts as a win to reviewers may not be what we would consider a real success. For example, a workaround I can reliably use to convince peer reviewers is to have long-running system deployments and package our findings from agricultural contexts as networking insights grounded in long-term, real-world experience in the field. But keeping experimental systems running long-term often involves hacks and kludges:
In the 3/31 technical meeting, [we] described our progress made over the last four years to aggregate data from so many [sensor] vendors. What’s striking about the progress is that, despite aggregating data successfully, we still find ourselves having “post-processing” scripts before the data can be fed to Machine Learning (ML) models. On a related note, the possibility of retraining ML models (which currently are using dummy models) sounds like an exciting use case in our problem definition for the SDF 2.0. - Research Diary, 04/03/2023
In this case, our implicit argument for validity is four years of use, but this use has been contextually disembedded to create the impression it runs seamlessly. Conversely, the actual lack of seamless operation can motivate a new funding proposal to address this technical limitation. In other words, the same research endeavor - cow disease detection using networked data and ML models - can masquerade as both insightful to a networking audience and forward-looking to a funding review panel. These aspects are symbiotic; if we successfully sell the idea to a networking audience to get published, then we can use the same finding and ensuing publication as evidence that retraining ML models for cow disease is an important issue deserving of a multi-million dollar grant to fund SDF 2.0.
Ultimately, envisioning and system building seem to center around maintaining the illusion of a single seamlessly operating system which is narrated in fundamentally different ways to different audiences. In the above research diary entry, for example, the agricultural context (i.e., identity of the sensor vendors, agricultural feature importance for training ML models) does not matter to systems peer reviewers. Rather, of interest to reviewers are the networking insights from the data aggregation and limitations of the “post-processing” scripts. In contrast, the technical contents (i.e., “post-processing” scripts, ML model training and deployments) do not necessarily matter to the funding agency to whom we are simultaneously pitching the next stage of the project. Rather, the four years of experience running apparently seamless technical systems for agriculture is proof enough that the seemingly performative societal impacts in the research proposals can be achieved with the funding. The key observation is that maintaining this illusion of seamlessness is precisely what simultaneously complicates our work toward the envisioned impacts and shapes what could be its eventual impacts.
5 Discussion
In this section, we analyze the intricate relationship between how we as systems researchers orient to societal impact and the technical, institutional, and disciplinary aspects that enable, shape, and constrain how we can do so. We do so by analyzing the central role of navigating technical, institutional, and disciplinary seams in systems research, and the ways this shapes how societal impact is imagined and worked toward.
In prior work integrating ethnographic and technical approaches to networking research [
83], we found that systems researchers’ visions of seamlessness for the eventual use of our technology contrasts sharply with the mundane everyday work of networking research, which requires navigating numerous material resistances and challenges in order to construct and maintain apparently stable test deployments. While some of the infrastructural seams we as researchers encounter do eventually disappear in the development process, others are forgotten or blinkered from view because of institutional and disciplinary factors. This means that our claims to seamlessness for such technologies are constructed over a reality of recalcitrant seams which may resurface when the technology is adopted in end-use contexts.
In this work we orient to seams at a higher level, considering them as metaphorical descriptions of misalignments between technical, disciplinary, and institutional systems that have to be navigated in order to successfully produce networking research. Working across technical, disciplinary, and institutional seams is a ubiquitous aspect of networking research, especially when working in other domains where the technology to be built is envisioned to create societal impact. In this section, we will argue that this need to adapt and work across seams continually reworks representations of the technical systems being built and how their societal impact is envisioned. Structural seams are marked by differences such as disparate publication requirements, conflicting motivations, and academic power differences. These seams are brought into alignment to produce temporarily stable images of a technology and its societal impact that can motivate stakeholders and be narrated as successful work deserving of continued funding. These changing representations therefore continuously rework how societal impact is oriented to in response to structural pressures.
In this section, we will characterize the nature of work across structural seams in domain-focused systems research, describe how the mechanisms to work across seams continually produce new representations of technical systems and their envisioned impact, demonstrate how this process torques the way societal impact is oriented to in computer systems research, and describe the role of critical technical practice in navigating the dilemmas that result.
5.1 Seam Work
In this section, we draw on STS work on how people navigate seamful infrastructure [
96,
103] to characterize how networking researchers navigate institutional, disciplinary, and organizational seams while building systems oriented towards societal impact in a particular domain. We build particularly on Vertesi’s [
103] emphasis on the agency people have to work around, with, and through the infrastructural seams they are confronted with, and her articulation of the creative work required to bring messy, overlapping systems into temporary alignment, which she aptly describes as “patchwork”.
We frame researchers’ efforts to align action across technical, institutional, and disciplinary seams as a form of seam work, by which we mean artful work across structural seams to produce products that can be narrated or experienced as “seamless”. Our results demonstrate that seam work is ubiquitous when doing domain-focused academic networking research. Most obviously, we must navigate technical seams within the emerging infrastructure being built, such as technical incompatibilities between different packages that need to be combined to produce the envisioned software. But we must also navigate many seams between organizations. When we set visions of societal impact and start working toward them, we continuously probe whether they are in alignment with university (e.g., student graduation), funding (e.g., NSF/USDA grants), corporate (i.e., our industry partners), and agricultural (e.g., vineyards and dairy) partner priorities. For example, while we may define success as a system that enables interconnection among many different platforms, an industry partner may define success of the project as a public demonstration of what their technical platform in particular can support. In order to continue to access the resources we need to move forward with our project, we need to show each set of stakeholders progress in ways that matter to them.
Because we are doing research in collaboration with academic researchers in other domains, we must also navigate disciplinary seams between what our discipline considers to be “CS research” and the concerns and interests of domain researchers. This requires us to actively translate between whatever our partners want us to do and research questions that can be made interesting to the networking/systems community. Because this community defines publication-worthiness as technical novelty independent of a particular domain, our work to build functioning deployments with our project partners must be renarrated before publication as producing such technical novelty.
We must also navigate seams within the academic hierarchy. Our interdisciplinary project collaborations, like many in our area, are driven by collaborations between PhD students. This makes us answerable to multiple PI’s with different disciplinary orientations and the power to veto ideas that are unfamiliar in their area, as when Gloire’s write-up of the goals of the team’s work became caught up in a conflicted accountability with Hakim, Laura, and Phoebe.
In each of these cases, these seams are quite visible to us as actors; we cannot move forward with our research successfully without solving the problems produced by working across each of these seams. When we say this seam work is oriented to producing seamlessness, then, what we mean is that the goal of seam work is to remove these seams as a visible problem. Success means rendering them, if not invisible, then at least unremarkable. We find ways, for example, to cobble together incompatible software into apparently smoothly functioning systems; to align with our industry partners into jointly articulated projects that smooth over differences in outlook and rewards; to produce deployments that knit together our own concerns and interests with those of domain researchers; to produce research papers that present coherent representations of our work to differing disciplinary audiences; and to adjudicate which views can and can’t be included in how we represent our project. This takes a lot of artful work!
In practice, seam work involves not simply working across a single seam, but navigating multiple seams at once. For example, as we build an infrastructure, we run into many material resistances, such as when a particular operating system does not work well with a particular machine-learning package. In navigating these issues and pivoting where needed over the course of years, we are continuously taking into account the many seams across which we are working. We consider institutional seams: if we make a technical pivot to manage this issue, is our work still in alignment with our institutional partners and whatever vision they had of societal impact? We consider disciplinary seams: what about our reviewers, what story can we craft that is ‘nerdy’ and makes it seem like this system has been running for a long time and can provide ‘networking insights’? Could we be painting ourselves into a corner of being too applied? At the same time, we need to maintain our plant pathology and animal science collaborations so that we have concrete case studies that we can use to motivate our system design.
5.2 Seam Work Produces Simulacra of Impact
Seam work, then, involves translation work, as we renarrate the system we envision building, its goals and its impact, based on pressures produced by differentials across various seams. Thus seam work produces shifting representations of what we are doing and why, that align with the needs of different stakeholders as the work develops.
One aspect of the project that becomes re-represented through seam work is the nature of the technical system that is being built. Academic systems research involves not only building concrete systems but using them to argue for a more generalized understanding of how future systems can be usefully produced. These understandings become reshaped and reworked in response to our audiences, as we pitch them to funding agencies, enroll industry partners, collaborate with domain partners, and test arguments through the peer review process. The possibilities we can argue for also become reworked as we run into technical, disciplinary, and other seams that shape how we can implement the technical deployments that provide the empirical grounding for our argumentation.
Another aspect of the project that becomes re-represented is the intended societal impact of the system. The actual societal impact our work will eventually have is, of course, not currently knowable to us. But our envisioned impact acts as an orientation point for our technical decision-making, and enables us to enroll stakeholders in our project. Still, these ideas of societal impact do not remain fixed; they must respond to ongoing contingencies that arise in our seam work. For example, when our funders’ insistence on open-source outcomes clashed with a domain collaborator’s industrial relationships, we had to compromise on our own preference for societal impact. Our understanding of what societal impact is and how to orient towards it, then, shifts repeatedly as we work across seams.
As such,
representations of the system and its intended impact are continuously being refactored as we work across different seams. Vertesi highlights the temporary, contingent, and local nature of the patchwork involved in working together infrastructural seams [
103]. But while such contingency also characterizes our work across structural seams as networking researchers, our goal is to produce apparently stable representations of the technical system we are producing and its societal impact, even when the image we present belies the actually cobbled-together, temporarily functional nature of our work. We therefore refer to these representations as
simulacra of the
SDF. By “simulacrum” we mean a
clear and coherent representation of a technical system and its societal impact and meaning. Seam work continually produces new simulacra as we narrate how the system should be understood and argue for its benefits to new audiences, and simultaneously to ourselves.
While these representations are often stated in the form of claims about reality, their relationship to empirical evidence is more complex. For example, our networking publications emphasize the length of time that we have been running SDF deployments, because this provides evidence of robustness. But the longevity of this system relies on so much maintenance and code revision that it is practically debatable whether it should be called the “same” system.
But simulacra are also not fake, in at least two ways. First, they provide the best view that we can currently muster of what we believe our system can be argued to do, based on factors such as empirical work, research insights, intuition, and the demands of the audience. Second, these representations become orientation points that guide ongoing activity, such as in the transformation from SDF 1.0 to SDF 2.0. They thus guide the system into being.
5.3 Reworking Impact Across the Seams
So far, we have argued that seam work is an inherent part of systems research, and that seam work produces shifting representations of a system and its intended impact. How does this matter for how we as academic systems researchers orient to societal impact in our everyday work? In this section, we will describe two examples of how seam work systematically torques the ways in which technical systems and their societal impact are represented and oriented to.
5.3.1 Pressures in bilingual research.
The recent proliferation of computing systems, especially AI, into many domains is leading to a policy push in the United States to train the ‘bilinguals of the future’ [
67]. These are understood as people in fields like biology, chemistry, politics, history, and linguistics who are also skilled in the techniques of modern computing that can be applied to them [
67]. For example, the US National Science Foundation (NSF) has committed millions of dollars to NSF Research Traineeships (NRT) to train such bilingual scholars in various domains across the US (e.g., [
27,
40,
59,
94]). In the
SDF context,
Gloire and his collaborator
Frank are bilingual products of such NRT training at our university, bringing together computing, engineering, and plant sciences. In other words, bilingual projects are a policy approach for fostering broader impacts for computing by producing what Ribes et al. call ‘boundary-spanning figures’ [
81] who are deeply familiar with both computing techniques and a specific domain.
This institutionalization of an approach to producing design for domain-specific impact produces endemic structural seams that shape how projects can be developed and narrated. Specifically, it (1) builds on interdisciplinary collaborations between CS and other disciplines which are often less powerful and resourced, and (2) places the conceptual integration of these disciplines in the hands of some of the most junior members of the academic hierarchy.
Gloire had to navigate these tensions during the interdisciplinary paper-writing progress. As described in the findings, for every project, his collaborators write a paper describing the advancements in plant pathology while he writes a paper describing the computer systems and networking advancements. In this writing, it was difficult to find the right technical depth when describing computing advancements. For instance, to a CS audience, ‘object pickling’ is a fairly obvious concept. In contrast, the phrase ‘pickling ML models’ may present a foreign concept to a plant pathology audience. One outcome of this tug-of-war was the unintended dominance of CS parlance (e.g., abstraction, cloud, etc.) in the problem/solution framing of interdisciplinary contributions.
The second factor considers the power dynamics involved in the writing process. By writing with stakeholders across academic rungs, the findings pointed to an implicit reverence for problem/solution framing from (senior) faculty. In a context where societal impact considerations abound, critical reflections inadvertently took the backseat because we were preoccupied with producing technical content that can speak to multiple audiences. This challenge is more pronounced when junior bilingual researchers are most concerned with and/or have the most expertise on societal impact considerations. In other words, this power dynamic can effectively cripple critical reflection.
5.3.2 Publish, perish, or compromise for societal impact.
Previously, we described how the HCI community often challenges computer scientists to grapple with the societal implications of their work [
28,
91]. A recurring critique is that computer scientists are detached from or ambivalent to the societal impacts of their work. Our findings present a more nuanced account of societal impact considerations from within computer systems research, suggesting that societal impact considerations become structurally marginalized through the current CS peer review process. Recently, Edward Lee, an eminent figure in CS, argued that computing research has developed a ‘toxic culture of rejection’ based on a feedback loop between high submission rates, overwhelmed reviewers, low acceptance rates, and resubmission of previously rejected work [
62]. We, too, faced significant challenges getting publications within computer systems, including five revisions and resubmissions of the same paper. A few of the typical reasons for rejections include ‘lack of novelty’ or results are ‘too obvious’ [
62]. These reviews were signs of the need for more work across the seam between technical deployments meaningful to domain partners and the intellectual commitments of current systems research.
This “publish or perish” [
62,
88] culture has two effects on societal impact considerations. First, we are caught up in gradual, persistent revisions to get publications. This leads us to continually re-represent our work and revise ongoing technical work to better align with the expectations of systems research. As Scott Shenker, an eminent figure in the networking community, argues “the publish-or-perish atmosphere inevitably leads us to alter our research so as to increase the chances of our work getting accepted” [
88, p. 28]. In doing so, we easily lose sight of institutional, infrastructural, and other factors that are shaping our research agenda and, by extension, the eventual societal impact. The irony is that we need those publications in order to achieve our intended societal impacts by making our ideas public. In other words, we will not (re)-consider the impacts until we have reached a twisted version of them, as was the case for the
Chameleon paper.
Second,
Gloire experienced a repeating cycle as a junior researcher jumping through hoops and dealing with negative reviews that take a mental toll. Lee argues that this “culture of rejection” can cause junior researchers to either leave the field or persist through stubbornness, eventually to haze the next generation [
62]. While these factors are relevant for all systems researchers, we note they place particular pressure on researchers interested in societal impact in ways that defy the “domain-independent” norms of computing research.
This cycle effectively presents three paths for young computer scientists interested in societal impact to consider: persistence, defeat, or compromise. The first path is a self-repeating cycle, where Gloire persisted through five submissions, and could therefore be likely to perpetuate the cycle by subjecting other younger researchers to the same ‘hazing’ when he joins PCs. The second path is defeat, where researchers working on societal impact leave the field altogether. The third path is compromise, where papers gradually lose their societal impact considerations on the way to publication. In rare cases, the papers may find relevance in sister communities, as observed in the findings where a sub-community of researchers interested in networking, HCI, and social science emerged. In the case of the Chameleon paper, we relied on a combination of persistence and compromise. This path of compromise is explored further below, as we consider the value of critical technical practice as a means for navigating these issues.
5.4 Navigating Compromised Agency in Critical Technical Practice
We do not claim critical technical practice to be a mainstream practice in CS research. Rather, we leverage it here to show how societal impacts are envisioned, worked toward, and rehashed within computer systems research. Building new infrastructure involves hours, months, or even years of charting technical goals and working in mundane ways across technical, institutional, and disciplinary seams. Critical technical practice offers us an important guide as we execute on and document life in the technical trenches. This journey in the trenches presents many opportunities for reflections on core values. Compared to values in design (VID) engagements that bring together teams of computer scientists and social scientists during the early stages of design, critical technical practice during infrastructure building presents a unique opportunity to consider alternatives as close to the moment of intervention as possible.
Moreover, critical technical practice offers more agency to pivot technical goals and modify the system as we navigate ‘deep’ infrastructures [
37], an agency which social scientists in VID efforts typically lack. For instance, we faced a lack of electrical plugs for a gateway device in an
SDF 1.0 greenhouse deployment; we pivoted to find new deployment locations in the greenhouse, but we also considered the role of unreliable power in
SDF 2.0’s system design. In other words, critical technical practice allowed us to recognize and build in the need for more seamful infrastructure designs.
Through reflections produced through critical technical practice, we have demonstrated how systems research intended to produce positive societal impact is constrained and shaped by the need to navigate technical, disciplinary, and institutional seams. But what implications does this have for critical technical practice itself as a methodology to support considerations of societal impact?
In describing their own work producing critically reflective scientific research into ocean plastic pollution, Liboiron argues that all agency is fundamentally compromised by one’s position within social systems that are characterized by asymmetrical power relations [
63]. In other words, there is no pure or innocent position from which to rework scientific practice and assure that it will be deployed in service of the good. Instead, recognition of and grappling with compromise must form the basis for critically oriented scientific work.
Similarly, we emphasize that critical technical practice as we have deployed it is not independent of the technical, disciplinary, and institutional pressures that we describe in this paper. Rather, it introduces yet another seam, one across critical and technical work, which has to be juggled in the process. It also adds new disciplinary pressures, embodied in this case through Phoebe’s interventions, to produce knowledge which is publishable to the HCI and CSCW audience; you have your own standards for novelty and empirical work to which this paper must hew.
What we would argue is that critical technical practice is of value to computational researchers who care about societal impact, not by enabling an escape from seam work, but by providing a means for intentionally navigating it. Critical technical practice enables us to anticipate structural factors that provide resistance to working towards envisioned societal impact goals and circumvent them, wherever possible. In other terms, critical technical practice entails reflectively navigating powerful institutions, mundane infrastructure issues, and compromises with disciplinary structures to make the best decision possible under the circumstances.
For example, our findings captured a research process that includes setting visions, building large sociotechnical infrastructures, writing for interdisciplinary audiences, and receiving feedback from the research community. An important aspect of the envisioning process is enrolling corporate / institutional stakeholders. Critical technical practice gave us a lens to continuously and critically probe corporate interests and power in these societal impact research engagements. We worked in our project to leverage their institutional power to work towards positive societal outcomes, while considering ways in which their perspectives might skew them. No doubt the results are, however, imperfect.
In summary, using critical technical practice to work toward societal impact produces a greater degree of agency to make technical change and to leverage institutional power towards our goals, but it also demands an awareness of how that agency is inevitably compromised. Critical technical practice helps us to look behind the simulacra and to recognize the illusion of seamlessness that our activities produce. It helps us to recognize our own seam work and to navigate the choices we make of how to bring seams into alignment.
