Assessing the Effectiveness of Voice-Translation Tools for Translating Medical Information from English to Arabic
Main Article Content
Abstract
The study aims to assess the performance of Gemini, ChatGPT, and DeepSeek models in translating medical information from English to Arabic. Using a comparative descriptive design, the current study tests 40 English audio clips of medical information and evaluates their 120 Arabic translations using the Multidimensional Quality Metrics (MQM). The performance of each model was evaluated along two dimensions of translation quality: accuracy and fluency, including medical terminology, semantics, grammar, and clinical instructions. The comparative analysis revealed accuracy and fluency problems, including problems related to the accuracy of medical terms, omissions, clinical plausibility, and emergency communication. In the accuracy assessment, ChatGPT scored 18, Gemini 14, and DeepSeek 10 points. The current study stresses the importance of differentiating between linguistic fluency and medical accuracy and the significance of human supervision.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This open-access article is distributed under a Creative Commons Attribution (CC-BY) 4.0 license.
You are free to: Share — copy and redistribute the material in any medium or format. Adapt — remix, transform, and build upon the material for any purpose, even commercially. The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
No additional restrictions You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
How to Cite
Share
References
Almahasees, R., & Almahasees, Z. (2025). Breaking language barriers in physiotherapy: Ensuring consistent English-to-Arabic translations for better patient care. Research Journal in Advanced Humanities, 6(2), 1-12.
Almahasees, Z. (2023). A Synchronic Assessment of Google Translate, Microsoft Translator and Reverso in English<> Arabic Translation. In 3rd International Conference on Language and Education (pp. 1-6). Cihan University-Erbil.
Almahasees, Z., & Husienat, I. (2024). A comparative analysis of terminological inconsistency in scientific translation from English into Arabic across different medical fields. Training, Language and Culture, 8(3), 25-40.
Almahasees, Z., & Husienat, I. (2026). AI-Powered Voice Translation in Medical Settings: An Evaluation of English–Arabic Translation Accuracy. International Journal of Linguistics, Literature and Translation, 9(9), 203-214.
Barrault, L., Chung, Y.-A., Meglioli, M. C., Dale, D., Dong, N., Duquenne, P.-A., Elsahar, H., Gong, H., Heffernan, K., Hoffman, J., Klaiber, C., Li, P., Licht, D., Maillard, J., Rakotoarison, A., Sadagopan, K. R., Wenzek, G., Ye, E., ... Wang, S. (2023). SeamlessM4T: Massively multilingual & multimodal machine translation [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2308.11596
Boulesnam, I., & Boucetti, R. (2025). Arabic Language Characteristics that Make its Automatic Processing Challenging. International Arab Journal of Information Technology (IAJIT), 22(4), 814.
Carreras Tartak, J. A., Haimovich, A. D., Thomas, C. E., Bivens Moore, E., Slebonick, A., Haque, N., & Thomas, S. H. (2025). Evaluating the accuracy of Google Translate in emergency department discharge instructions. The Journal of Emergency Medicine, 79, 132–142. https://doi.org/10.1016/j.jemermed.2025.04.024
Delfani, J., Orasan, C., Saadany, H., Temizoz, O., Taylor-Stilgoe, E., Kanojia, D., ... & Schouten, B. (2024). Google translate error analysis for mental healthcare information: Evaluating accuracy, comprehensibility, and implications for multilingual healthcare communication. arXiv preprint arXiv:2402.04023.
Freitag, M., Foster, G., Grangier, D., Ratnakar, V., Tan, Q., & Macherey, W. (2021). Experts, errors, and context: A large-scale study of human evaluation for machine translation. Transactions of the Association for Computational Linguistics, 9, 1460–1474. https://doi.org/10.1162/tacl_a_00437
Haider, A. S. (2019a). The representation of Al-Megrahi's release in Arabic and English newspapers in 2009 and 2010: A corpus-assisted discourse study. Dirasat: Human and Social Sciences, 46(1, Suppl. 2), 297–317. https://doi.org/10.35516/0103-046-986-019
Haider, A. S. (2019b). Syrian–Lebanese relations: A corpus-based critical discourse analysis of Bashar Al-Assad's speeches and interviews. Dirasat: Human and Social Sciences, 46(4), 551–570. https://doi.org/10.35516/0103-046-004-040
Haider, A. S. (2019c). Using corpus linguistic techniques in (critical) discourse studies reduces but does not remove bias: Evidence from an Arabic corpus about refugees. Poznan Studies in Contemporary Linguistics, 55(1), 89–133. https://doi.org/10.1515/psicl-2019-0004
Haider, A. S. (2026a). AI as a language tutor: Jordanian university students' perceptions of AI-generated feedback on their English. Research Journal in Advanced Humanities, 7(4), 1–17.
Haider, A. S. (2026b). Trust in AI translation: Reception and adoption among Jordanian business professionals. Research Journal in Advanced Humanities, 7(4), 1–15.
Haider, A. S., Abu Akar, R., Saed, H., Abu Tair, S., & Ismail, H. H. (2025). AI-powered Arabic translation of business and finance acronyms and abbreviations: Evidence from Gemini, ChatGPT, and Google Translate. In Innovation, technologies, and business management (ICTIM) (pp. 519–528). Springer. https://doi.org/10.1007/978-3-031-84889-6_62
Haider, A. S., & AlKhatib, R. (2024). Subtitling English legal acronyms into Arabic: Human vs machine. Kutafin Law Review, 11(4), 810–842. https://doi.org/10.17803/2713-0533.2024.4.30.810-842
Haider, A. S., Saed, H., Albarakati, M., Abu Tair, S., & Jarrah, S. (2025). Phonetic and orthographic transliteration of borrowed words in Arabic: A translation-oriented study. WORD, 71(1), 1–19. https://doi.org/10.1080/00437956.2025.2455229
Hartung, T., & Kleinstreuer, N. (2025). Challenges and opportunities for validation of AI-based new approach methods. Altex, 42(1), 3-21.
Ismayilli, T. (2024). Navigating complexities in medical text translation: Challenges, strategies, and solutions. Acta Globalis Humanitatis Et Linguarum, 1(2), 170-176.
Karakus, I. S., Strechen, I., Gupta, A., Nalaie, K., Chen, C. L., Hassett, L. C., & Barwise, A. K. (2025). Bridging language gaps in healthcare: a systematic review of the practical implementation of neural machine translation technologies in clinical settings. Journal of the American Medical Informatics Association, 32(11), 1756-1766.
Karanikolas, N., Manga, E., Samaridi, N., Tousidou, E., & Vassilakopoulos, M. (2023). Large language models versus natural language understanding and generation. In Proceedings of the 27th Pan-Hellenic Conference on Progress in Computing and Informatics (pp. 278-290).
Khujakulov, R. (2026). Semantic transformations of Arabic-origin medical lexemes in “The Canon of Medicine”: A case study of Uzbek folk medicine and English equivalents. Asian journal of medical sciences and clinical research, 1(1), 42-48.
Kuhn, K., Kersken, V., Reuter, B., Egger, N., & Zimmermann, G. (2024). Measuring the accuracy of automatic speech recognition solutions. ACM Transactions on Accessible Computing, 16(4), 1-23.
Li, D. (2026). Post-editing strategy optimization and performance evaluation based on DQF-MQM error analysis. Discover Applied Sciences, 8(4), 357.
Lommel, A., Gladkoff, S., Melby, A. K., Wright, S. E., Strandvik, I., Gasova, K., ... & Nenadic, G. (2024, September). The multi-range theory of translation quality measurement: MQM scoring models and statistical quality control. In Proceedings of the 16th Conference of the Association for Machine Translation in the Americas (Volume 2: Presentations) (pp. 75-94).
Mahdi, A. I., & Mohammed, H. G. (2025). Accuracy of medical terminologies translated from English into Arabic: The case of audiovisual shows. International Journal of Arabic-English Studies, 147-172.
Mejaški, A. (2026). A Comparative Analysis of AI Translation Errors in Medical and Legal Texts (Master's thesis, Sveučilište u Rijeci, Sveučilište u Rijeci, Filozofski fakultet, Katedra za translatologiju).
Mingote, V., Gimeno, P., Vicente, L., Khurana, S., Laurent, A., & Duret, J. (2023). Direct text to speech translation system using acoustic units. IEEE Signal Processing Letters, 30, 1262-1266.
Muhanna, A. A. I. (2025). Assessing machine translation quality on uterine cancer content: MQM-based comparative study. CDELT Occasional Papers in the Development of English Education, 89(1), 421-458.
NLLB Team, Costa-jussà, M. R., Cross, J., Çelebi, O., Elbayad, M., Heafield, K., Heffernan, K., Kalbassi, E., Lam, J., Licht, D., Maillard, J., Sun, A., Wang, S., Wenzek, G., Youngblood, A., Akula, B., Barrault, L., Gonzalez, G. M., Hansanti, P., Hoffman, J., ... Wang, J. (2022). No language left behind: Scaling human-centered machine translation [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2207.04672
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2023). Robust speech recognition via large-scale weak supervision. In Proceedings of the 40th International Conference on Machine Learning (Vol. 202, pp. 28492–28518). Proceedings of Machine Learning Research. https://proceedings.mlr.press/v202/radford23a.html
Sapawi, M. S. M., & Yusoff, N. M. R. N. (2026). Arabic Language and Medical Terminology in Education: A Systematic Review. IIUM Medical Journal Malaysia, 25(01).
Trigka, M., & Dritsas, E. (2025). The evolution of generative AI: Trends and applications. IEEE access, 13, 98504-98529.
Vieira, L. N., O’Hagan, M., & O’Sullivan, C. (2021). Understanding the societal impacts of machine translation: A critical review of the literature on medical and legal use cases. Information, Communication & Society, 24(11), 1515–1532. https://doi.org/10.1080/1369118X.2020.1776370
World Health Organization. (2024). Ethics and governance of artificial intelligence for health: Guidance on large multi-modal models. https://www.who.int/publications/i/item/9789240084759
Yelahina, N. I., & Fedchyshyn, N. O. (2025). The role of applied linguistics in enhancing medical translation accuracy: a corpus-based approach. https://dspace.luguniv.edu.ua/xmlui/handle/123456789/13249