Project title:
Assessing the accuracy and acceptability of AI vs human translations of patient-facing clinical trial documents
.jpg){kind=spacer}
Project background:
Research participation by people from diverse backgrounds is essential to understand the generalisability of trial findings. However, people taking part in clinical trials do not always represent the race and ethnicity of the whole population or those affected by different diseases and this is likely to have detrimental consequences for the health of underserved groups, including those from ethnic minority and migrant backgrounds, and may increase social inequalities in health.
While there are many different barriers to diverse recruitment into clinical trials, the inability to speak English is an exclusion criterion for many trials. Recruitment and consent processes often rely heavily on written materials, therefore being unable to read or speak English, or read or speak it to a certain level, means people from ethnic minority and migrant backgrounds may be excluded. Two recent SCTU projects aimed at widening participation in PPI and increasing the ethnic diversity of people taking part in clinical trials also identified language as a key consideration when engaging ethnically diverse communities with clinical trials.
However, translation of trial documentation into other languages can be expensive and resource heavy, especially when translations to several different languages may be required, and this is often not covered by a trial budget. Artificial Intelligence (AI) offers quick, free machine translation but evidence for its accuracy and accessibility is lacking, particularly in medical fields where complex information needs to be conveyed accurately and clearly.
Through this project we aim to compare machine translation of trial documents using AI applications to translations done by professional human translators, in several different languages.
Project methods and results:
Patient-facing documents (Participant Information Sheets, Consent Forms, video subtitles) from a variety of Southampton Clinical Trials Unit studies were translated into three of the most common non-English languages spoken in the UK (Polish, Punjabi and Mandarin) by professional human translators and widely available AI applications (ChatGPT, Claude and Copilot), and reverse translated back to English.
We conducted a dual-layered qualitative evaluation: i) Linguistic Coding Analysis: Five independent reviewers compared reverse-translated documents against original English texts for accuracy, tone, and loss of nuance, with findings validated by a sixth reviewer; ii) End-User Acceptability: Native speakers evaluated the translated documents, providing experiential feedback through online surveys and qualitative interviews. All reviewers were blinded to which documents were AI or professional translations.
Analysis is currently ongoing and the results of the project will be presented at the International Clinical Trials Methodology Conference in September 2026.
Potential relevance and future impact:
AI is already being successfully used in clinical research in data analysis and diagnostics, but further, robust evidence is needed on how AI is used safely in other areas. This project will provide evidence on the implementation of AI for translating trial documentation.
Following this foundational pilot, we have secured funding to further evaluate AI translation tools in real-world trial settings, assessing uptake of translated documents and conducting qualitative interviews with recruitment staff and trial participants. This will provide further evidence on the accuracy and acceptability of AI translations, their potential impact on diversity of trial participants, and the creation of best practice guidelines for future use of AI translations by trial teams.
For further information on our methodology research, visit our Trial Methodology webpage.