Profiling Chinese Student Interpreters’ Usage Intention of ChatGPT-assisted Translation by Q Methodology Based on Technology Acceptance Model

ChatGPT represents a cutting-edge language translation system that leverages advanced natural language processing techniques to provide more accurate and faster translations compared to other systems. Built on the GPT-2 model, which has been extensively pre-trained on a vast corpus of text data, ChatGPT has undergone evaluations for various machine translation tasks, including translation prompts, multilingual translation, and translation robustness (Jiao et al., 2023). Experimental results indicate that ChatGPT, when using specifically designed translation prompts, can deliver performance that is on par with or superior to commercial translation systems for high-resource languages (Gao et al., 2023).

Compared with ChatGPT-assisted translation studies, its application in interpreting is at the exploratory stage. The ChatGPT technology with the characteristics of matching terminology and context, interactive language learning and two-way knowledge construction may change the data resources, technology iterations and scientific paradigms in interpreting. This has brought some additional technological benefits to the development of AI interpreting. A more obvious technological advantage in interpreting is demonstrated by the near transfer of the theme-related knowledge in the pre-interpreting preparation. Secondly, the ChatGPT application in interpreting can reduce the cognitive processing anxiety of interpreters to a certain extent. Finally, ChatGPT can significantly enhance the quality and efficiency of interpreting assessment management (Zheng et al.).

Information systems researchers have proposed using intention models from social psychology as a theoretical foundation for studying the determinants of user behavior (Swanson, 1974). Among these models, Fishbein and Ajzen's (1975) Theory of Reasoned Action (TRA) stands out as particularly well-researched and effective in predicting and explaining behavior across various domains. TRA is highly general, designed to explain virtually any human behavior (Ajzen and Fishbein, 1980), making it suitable for examining the determinants of computer usage behavior specifically.

Davis (1986) introduced the Technology Acceptance Model (TAM), an adaptation of the Theory of Reasoned Action (TRA), specifically designed to explain computer usage behavior. TAM builds on TRA by establishing causal linkages between two key beliefs: perceived usefulness and perceived ease of use, and their influence on users’ attitudes, intentions, and actual computer adoption behavior. In China, researchers have begun applying TAM in educational contexts. Carey and Kacmar (2010) examined how culture and language affect technology acceptance and attitudes among Chinese students, revealing that interface features tailored to cultural preferences impact students’ perceived usefulness and usage intentions. Additionally, TAM has been applied to distance education via the Internet (Chen & Qiu, 2005) and to studying university teachers’ IT acceptance behavior in classroom teaching from an empirical perspective (Ren & Zhai, 2012).

With artificial intelligence technologies like ChatGPT becoming the future trend in IT development, it is essential to study the usage intentions and behaviors of ChatGPT users, particularly student interpreters. Drawing on the factors of technology-based self-directed foreign language learning proposed by Pan Xiaoquan (2021)—perceived usefulness, learning attitudes, ease of use, technological self-efficacy, and subjective norms—the following research questions were formulated:

When it comes to ChatGPT-assisted translation, people naturally associate it with machine translation. In China, The Teaching Guidelines for Undergraduate Translation Major released in 2020 lists “Translation Technology” as a compulsory course for the first time, which highlights its importance (Wang & Li, 2021). Generally speaking, Translation technology refers to the different types of technical means applied to human translation, machine translation and computer-assisted translation (Bowker, 2002). The translation function is only one of its by-products of ChatGPT.

Due to its translation function, ChatGPT can be preferred and used by student interpreters as a kind of translation technology. Although ChatGPT has the potential to assist instructors by generating course materials and providing suggestions, and to act as a virtual tutor for students by answering questions and facilitating collaboration, its use also presents challenges. These challenges include the generation of incorrect or fake information and the potential to bypass plagiarism detectors (Lo, 2023).

This study was conducted at the College of Foreign Languages and Literature of a prestigious comprehensive university located in a major southeastern city in China. The Department of Translation at this university offers bachelor's degrees in translation, as well as MTI and Ph.D. programs.

For the study, 30 students were recruited from Chinese-English interpreting courses or research groups. Most participants were native Mandarin speakers majoring in interpreting at the undergraduate, graduate, and Ph.D. levels.  In particular, 8 (27%) non-English undergraduate participants come from the optional interpreting course. According to the definition of Sawyer (2004: 72), the participants are at the entry and intermediate levels. Participant demographics are shown in Table 1.

*Notes: English majors include Translation (undergraduate), MTI (Master of Translation and Interpreting), MA, and Translation Studies (Ph.D). The rest non-English majors come from the optional interpreting course.

Their classification information is shown in Table 2.

From Table 2, the researcher finds that the undergraduates of translation major receive the least training of ChatGPT use. And the reason for that lies in their preference to do translation by themselves rather than by machine translation, which is obtained from the later interview. Another significance of Table 2 is their high attention of training related to ChatGPT's guide and application, which indicates their acute perception of new technology.

Q methodology studies do not seek to generalize from a large population, similar to qualitative research (Watts & Stenner, 2012). A successful Q methodological study can involve a relatively small number of participants (typically 20-40) whose perspectives are crucial to the research topic (Watts, 2015). It means that in this PQ analysis, all the viewpoints of participants will be considered thoroughly and in keeping with this methodological holism.

This study employed Q methodology, developed by British physicist and psychologist William Stephenson in the 1930s (Stephenson, 1935). This method allows researchers to systematically uncover key subjective viewpoints among participants (Watts, 2015). It helps explore significant feelings and opinions held by individuals and highlights core attitudes or feelings shared within a group or community (Irie, 2014). Q methodology is similar to qualitative inquiries but includes statistical analysis, making it a mixed method on the continuum of quantitative and qualitative paradigms (Watts & Stenner, 2012; Watts, 2015). It has been successfully applied in political science, marketing research, policy studies, and education (e.g., Yang & Montgomery, 2013), and recently introduced to L2 motivation research (Irie & Ryan, 2015; MacIntyre et al., 2017). A typical Q methodological study involves five steps: (1) collecting related statements (concourse); (2) developing the sample statements (Q set), usually around 40 but ranging from less than 20 to over 250 (Millar, 2022); (3) gathering participants’ opinions (Q sort); (4) factor analysis of these Q sorts; and (5) interpreting the emergent factors in relation to the statements. The factors represent distinctive viewpoints shared within the participant group, grouping individuals according to their shared opinions on a certain issue. The participant sample size in a Q study usually ranges from 20 to 40, though effective studies can be conducted with fewer participants (Watts, 2015).

To identify the complexity of student interpreters’ usage intention of ChatGPT-assisted translation, the researcher subdivides the topic into ChatGPT use and acceptance of technology. A diverse range of sources, including academic literature (e.g., Zhang et al., 2023; Zheng et al., 2023; Dwivedi et al., 2023; Gordijn et al., 2023, etc.), popular science magazines, mainstream media platforms, and conversations with educators and peers, were utilized in an unstructured manner to collect various items encompassing different discourses on usage intention. Subsequently, two domain experts and two seasoned Q researchers were consulted to assess the statements and capture the nuanced perspectives on usage intention. The statements formed the preliminary concourse and were imported into online Q sorting platform¹. Then, four of the participants were invited to do the pilot test, after which the final concourse was formulated from 40 to 36 statements by culling repeated, ambiguous or indirectly related ones according to their feedback. That's Step 1 Concourse and Step 2 Q set.

Step 3 Q sort. The researcher made appointments with the participants one by one on online Tencent Meeting (a reliable cloud video meeting product), whose screen sharing and recording functions facilitated the real-time watching, communicating and using playback for the researcher. Participants received detailed information about the research project and signed an electronic consent form. They then filled out a brief questionnaire to provide additional demographic information. Following this, participants engaged in a card-ranking activity involving 36 items on usage intention of ChatGPT-assisted translation. Each item was assigned a hierarchical position in a forced-choice, quasi-normal, and symmetrical distribution based on how well the statement was perceived to reflect the participant's understanding of the topic. After the ranking they would complete comments on 7 most agree, disagree or neutral statements. The face-validity dimension required items to be sorted from ‘most disagree’ to ‘most agree,’ with the most negative value (−5) on the left and the most positive value (+5) on the right. This dynamic process aimed to create a single, holistic configuration of all items, reflecting each participant's continuous comparison between them. The resulting product is viewed as a dynamic representation of their perspective on the subject (Lunberg, 2019). The sorting task for each participant took about 20-30 minutes to complete.

For Step 4, the researcher used PQMethod², a web-based platform that allows to manage Q methodology studies online, including inverted factor analysis. In brief, the analysis of the “Chinese Student Interpreters’ Usage Intention of ChatGPT-assisted Translation” Q sort began with an exploratory factor analysis adhering to standard statistical conventions to determine the identification and retention of factors (Rieber, 2020). A correlation matrix between and among Q sorts was generated using centroid extraction. After the centroid factor analysis was performed, 7 factors were seen as an output. And then a manual rotation of the factors was performed, thus Factor 1, Factor 2, Factor 3 and Factor 5 out of the former 7 factors were extracted. Therefore, the extracted 4 factors (eigenvalue > 1.00, according to Watts & Stenner, 2012) were performed as a varimax rotation, and 5 Varimax factors with high loading would be the output (due to the order of Factor 5 and the total number to be chosen). A significant loading value of 0.43 (p < .01) was calculated by the equation 2.58 × (1÷\(\sqrt {{\rm{No}}.{\rm{\ of\ items}}}\)) (here No. is 36), indicating that any Q sort with a single rotated factor loading above 0.43 could be taken as a member of this factor group, while confounding factors should be removed. After the successful completion of last routine run, a four-factor structure was identified (labeled as Factor 1, Factor 2, Factor 3 and Factor 4), explaining 63% of the total variance. Factor 1 included seven Q sorts, accounting for 21% of the variance, while Factor 2 comprised three Q sorts, explaining 9% of the variance, and Factor 3 of nine Q sorts with 17% of the variance and Factor 4 of seven Q sorts with 16% of the variance, shown in Table 3.

Notes: Due to the confounding factors, Q3, Q7, Q8, Q12, Q15, Q22, Q26, Q27, Q29, Q30 were excluded from any group.

The consensus statements mean those that do not distinguish between ANY pair of factors.

As to the channel of knowing ChatGPT, all agreed on the information pushed from social media (35). They have understood its technological advantage (5) and found the differences between ChatGPT and other search engines (30). In a word, they feel happy, interested and are willing to try ChatGPT-assisted translation (8, 9, and 18), which embodies their IT literacy (19). Students’ internal positive acceptance plays a crucial role in promoting their usage intention of ChatGPT-assisted translation (Man, 2023).

ChatGPT is widely utilized for various NLP tasks, including text generation, language translation, and answering numerous questions (Dwivedi, 2023). As an AI tool, it represents the integration of interpreting with technology, similar to the previously mentioned “training in the use of information technology for subject preparation before, during, and after assignments.” (Gile, 1995).

As to the usage of ChatGPT in interpreting, the researcher gets the data from the demographic questionnaires answered before Q setting. 70% of the participants would like to use ChatGPT in per-interpreting preparation, 20% during interpreting and 20% for post-interpreting assessment. Therefore, it can be concluded that most student interpreters use ChatGPT as a search engine at present. Furthermore, all participants share virtually unanimous attitude towards the usage effect shown in Table 8. Although translation quality is frequently imperfect, it generally allows users to access content that would otherwise be unavailable to them. (Carl, 2018).

As for the integration of interpreting and ChatGPT, it needs a long way to go due to the essence of interpreting— from speech recognition, code transfer to expression, which requires equipment and connection.

Concerning the weakness of ChatGPT, Farrokhnia et al. (2023) listed “insufficient depth of understanding,” “challenges in assessing response quality,” “potential for biases and discrimination,” “deficiency in high-order thinking skills,” “risks to educational integrity,” “inadequate contextual comprehension,” “threats to academic honesty,” “reinforcement of educational discrimination,” “facilitation of plagiarism in education and research,” and “decline in high-order cognitive skills.”

ChatGPT is an AI technology that is based on large amounts of data. The flow of the data inevitably involves data security, especially in the complex international environment. Data security matters to national security. It is necessary to strengthen the management of data and raise awareness of data security of the masses before ChatGPT is open sourced on a large scale. In addition, the core technology of ChatGPT is developed by OpenAI, a foreign company, which will result in the technical security. How to effectively avoid the problem of technology containment in the future is also one of the issues to be considered.

A potential risk is the impact of ChatGPT on the efficiency of knowledge-based tasks. As a generative tool, ChatGPT can produce new data rather than merely analyzing existing information. Consequently, the absence of notable enhancements in the productivity of knowledge work may lead to frustration among leaders and policymakers.