Assessing AI-driven dubbing websites: Reactions of Arabic native speakers to AI-dubbed English videos in Arabic
Main Article Content
Abstract
In the era of rapid Artificial Intelligence (AI) advancements, emerging tools have become essential components of our daily routine. AI-dubbing aims to speed up localisation by replacing the original soundtrack with AI-generated sounds. These developments raise the question of whether human dubbers could be replaced in the future. This study quantitatively examines viewers’ reactions to the AI Arabic-dubbed versions of the “Pride and Prejudice” movie using AI dubbing tools from two websites: ElevenLabs and Dübverse. The participants are asked to watch the original video along with two dubbed videos generated by the two websites. For data collection purposes, a three-point Likert scale questionnaire consisting of 19 items and five constructs —linguistic issues, technical issues, synchronisation, translation quality, and attitudes and future recommendations —was designed to elicit the reactions of 150 native Arabic speakers. The analysis shows that linguistic issues, technical issues, synchronisation, and translation quality significantly affected the participants’ attitudes and future recommendations regarding the use of AI-dubbing services. 80% of the respondents acknowledged that AI-dubbing is beneficial for making content accessible to a wider audience. The study found that ElevenLabs outperforms Dübverse, especially in areas such as voice cloning, maintaining both kinesic and isochrony, handling colloquial language, managing multiple speakers, and achieving overall better performance in translation. The findings showed that both websites lack lip-synchrony and require enhancements in other areas. This study is beneficial for content creators seeking to expand their reach globally. The study recommends conducting further research on AI-dubbing across different genres and languages.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This open-access article is distributed under a Creative Commons Attribution (CC-BY) 4.0 license.
You are free to: Share — copy and redistribute the material in any medium or format. Adapt — remix, transform, and build upon the material for any purpose, even commercially. The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
No additional restrictions You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
How to Cite
Share
References
Abulawi, F., Al-Salman, S., & Haider, A. (2022). Modern Standard Arabic vs. Egyptian Vernacular in Dubbing Animated Movies: A Case Study of A Bug’s Life. International Journal of Communication and Linguistic Studies, 21, 125-141. doi:10.18848/2327-7882/CGP66
Alrousan, F., & Haider, A. S. (2022). Dubbing television advertisements across cultures and languages: A case study of English and Arabic. Language Value, 15(2), 54-80. doi:https://doi.org/10.6035/languagev.6922
Baños, R. (2023). Key challenges in using automatic dubbing to translate educational YouTube videos. Linguistica Antverpiensia, New Series–Themes in Translation Studies, 22, 61–79.
Bigioi, D., & Corcoran, P. (2023). Multilingual video dubbing—a technology review and current challenges. Front. Signal Process, 3, 1-14. doi:10.3389/frsip.2023.1230755
Bollen, K. A. (2014). Structural equations with latent variables. New Jersey: John Wiley & Sons.
Brannon, W., Virkar, Y., & Thompson, B. (2023). Dubbing in practice: A large scale study of human localization with insights for automatic dubbing. Transactions of the Association for Computational Linguistics, 11, 419-435.
Byrne, B. M. (2013). Structural equation modeling with Mplus: Basic concepts, applications, and programming. London Routledge.
Chaume, F. (2004). Synchronization in dubbing: A translational approach. In P. Orero (Ed.), Topics in audiovisual translation (pp. 35-52). Amsterdam, The Netherlands: John Benjamins.
Chaume, F. (2020). Audiovisual translation: dubbing. London: Routledge.
Chronopoulou, A., Thompson, B., Mathur, P., Virkar, Y., Lakew, S. M., & Federico, M. (2023). Jointly Optimizing Translations and Speech Timing to Improve Isochrony in Automatic Dubbing. arXiv preprint arXiv:.12979.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. psychometrika, 16(3), 297-334. doi:https://doi.org/10.1007/bf02310555
Federico, M., Enyedi, R., Barra-Chicote, R., Giri, R., Isik, U., Krishnaswamy, A., & Sawaf, H. (2020). From speech-to-speech translation to automatic dubbing. arXiv preprint arXiv:.06785.
Federico, M., Virkar, Y., Enyedi, R., & Barra-Chicote, R. (2020). Evaluating and optimizing prosodic alignment for automatic dubbing. Paper presented at the INTERSPEECH, Shanghai, China.
George, D., & Mallery, P. (2003). SPSS for Windows step by step: A simple guide and reference. 11.0 update. . Boston: Allyn & Bacon.
Hair, J. F., Ringle, C. M., & Sarstedt, M. (2011). PLS-SEM: Indeed a silver bullet. Journal of Marketing theory and Practice, 19(2), 139-152.
Hayes, L. (2021). Netflix Disrupting Dubbing: English Dubs and British Accents. Journal of Audiovisual Translation, 4(1), 1-26. doi:https://doi.org/10.47476/jat.v4i1.2021.148
Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the academy of marketing science, 43, 115-135.
Iitola, M.-M. (2017). “Let It Go”. Subtitling and Dubbing Song Lyrics into Finnish in the Animation Film Frozen. (Master’s Degree Programme in Language expertise in specialized society). University of Vaasa, Finland Retrieved from http://osuva.uwasa.fi/handle/10024/880
Lakew, S. M., Federico, M., Wang, Y., Hoang, C., Virkar, Y., Barra-Chicote, R., & Enyedi, R. (2021). Machine translation verbosity control for automatic dubbing. Paper presented at the ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Canada.
Lee, A., Chen, P.-J., Wang, C., Gu, J., Popuri, S., Ma, X., . . . Tang, Y. (2021). Direct speech-to-speech translation with discrete units. arXiv preprint.
Nunnally, J. C. (1978). Psychometric Theory. New York: McGraw-Hill.
Öktem, A., Farrús, M., & Bonafonte, A. (2019). Prosodic phrase alignment for machine dubbing. arXiv preprint.
Trivedi, A., Pant, N., Shah, P., Sonik, S., & Agrawal, S. (2018). Speech to text and text to speech recognition systems-Areview. IOSR J. Comput. Eng, 20(2), 36-43.
Wahlster, W. (2013). Verbmobil: foundations of speech-to-speech translation. Berlin: Springer Science & Business Media.
Yahiaoui, R. (2016). Ideological constraints in dubbing The Simpsons into Arabic. Altre Modernità: Rivista di studi letterari e culturali(1), 182-200.
Yingmin, S. (2023). Realistic problems and innovative path of AI dubbing in the era of intelligent media. Int. J. Front. Socio, 5, 23-27.
Yuan, Z., & Jin, H. (2023). The application of machine translation in automatic dubbing in China: A case study of the feature film Mulan. Linguistica Antverpiensia, New Series–Themes in Translation Studies, 22, 80–94.