Cross-Domain Speech Synthesis for Personalized Communication Assistance in Patients With Speech Impairments

Authors

  • Ruy Infante Department of Computer Science, University of Houston, Houston, TX, USA.

Keywords:

cross-domain speech synthesis; personalized voice; dysarthria; voice conversion; intelligibility-aware modeling; uncertainty-aware communication assistance

Abstract

Speech impairments alter intelligibility, prosody, voice quality, and communicative timing, and conventional augmentative and alternative communication voices often impose a generic synthetic identity. Cross-domain speech synthesis offers a route to personalized assistance by transferring speaker, style, and prosodic information from limited patient speech and larger typical or atypical corpora. This manuscript develops a methodological framework for such synthesis that treats personalized communication as a constrained cross-domain generation problem rather than a generic text-to-speech task. The framework separates content, speaker identity, prosodic style, and channel factors; aligns source and target domains through adversarial representation learning; adapts prosody and voice quality with reference-based style objectives; and shapes generation with an intelligibility-aware objective and uncertainty-controlled acceptance rule. It specifies the variables, assumptions, loss terms, and decision logic required to synthesize speech that is intelligible, identity preserving, controllable, and safe for assisted communication. Analytical assessment examines trade-offs among disentanglement, naturalness, speaker similarity, latency, and clinical robustness, and identifies failure modes such as content leakage, adversarial instability, over-smoothed prosody, and intelligibility loss under atypical articulation. The framework does not report empirical outcomes; its contribution is a technically coherent design and evaluation logic that links cross-domain representation learning to the communication needs of patients with speech impairments. The resulting perspective supports clinician-supervised personalization, transparent fallback behavior, and reproducible technical evaluation before clinical deployment.

References

1. Taylor, P. (2009). Text-to-speech synthesis. Cambridge University Press.

2. Zen, H., Tokuda, K., & Black, A. W. (2009). Statistical parametric speech synthesis. Speech Communication, 51(11), 1039–1064.

3. King, S. (2014). Measuring a decade of progress in text-to-speech. Loquens, 1(1), e006.

4. Ling, Z.-H., Kang, S.-Y., Zen, H., Senior, A., Schuster, M., Qian, X.-J., Meng, H.-M., & Deng, L. (2015). Deep learning for acoustic modeling in parametric speech generation: A systematic review of existing techniques and future trends. IEEE Signal Processing Magazine, 32(3), 35–52.

5. Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., Le, Q., Agiomyrgiannakis, Y., Clark, R., & Saurous, R. A. (2017). Tacotron: Towards end-to-end speech synthesis. In Proceedings of Interspeech 2017 (pp. 4006–4010).

6. van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016). WaveNet: A generative model for raw audio. arXiv preprint arXiv:1609.03499.

7. Chou, J.-C., Yeh, C.-C., & Lee, H.-Y. (2019). One-shot voice conversion by separating speaker and content representations with instance normalization. In Proceedings of Interspeech 2019 (pp. 664–668).

8. Qian, K., Zhang, Y., Chang, S., Yang, X., & Hasegawa-Johnson, M. (2019). AutoVC: Zero-shot voice style transfer with only autoencoder loss. In Proceedings of the 36th International Conference on Machine Learning (Vol. 97, pp. 5210–5219).

9. Mohammadi, S. H., & Kain, A. (2017). An overview of voice conversion systems. Speech Communication, 88, 65–82.

10. Lorenzo-Trueba, J., Yamagishi, J., Toda, T., Saito, D., Villavicencio, F., Kinnunen, T., & Ling, Z. (2018). The voice conversion challenge 2018: Promoting development of parallel and nonparallel methods. In Proceedings of the Odyssey 2018: The Speaker and Language Recognition Workshop (pp. 195–202).

11. Rudzicz, F., Namasiwayam, A. K., & Wolff, T. (2012). The TORGO database of acoustic and articulatory speech from speakers with dysarthria. Language Resources and Evaluation, 46(4), 523–541.

12. Creer, S., Cunningham, S., Green, P., & Yamagishi, J. (2013). Building personalised synthetic voices for individuals with severe speech impairment. Computer Speech & Language, 27(6), 1178–1193.

13. Zhang, Y., Huang, R., Li, R., He, J., Xia, Y., Chen, F., ... & Zhao, Z. (2024, March). Stylesinger: Style transfer for out-of-domain singing voice synthesis. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 17, pp. 19597-19605).

14. Beukelman, D. R., & Mirenda, P. (2013). Augmentative and alternative communication: Supporting children and adults with complex communication needs (4th ed.). Paul H. Brookes.

15. Kent, R. D., Weismer, G., Kent, J. F., Klatt, D. H., & Rosenbek, J. C. (1989). Toward phonetic intelligibility testing in dysarthria. Journal of Speech and Hearing Disorders, 54(4), 482–499.

16. Toda, T., Black, A. W., & Tokuda, K. (2007). Voice conversion based on maximum-likelihood estimation of spectral parameter trajectory. IEEE Transactions on Audio, Speech, and Language Processing, 15(8), 2222–2235.

17. Saito, Y., Takamichi, S., & Saruwatari, H. (2018). Statistical parametric speech synthesis incorporating generative adversarial networks. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 26(1), 84–96.

18. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. In Advances in Neural Information Processing Systems (Vol. 27, pp. 2672–2680).

19. Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2016). Domain-adversarial training of neural networks. Journal of Machine Learning Research, 17(59), 1–35.

20. Kingma, D. P., & Welling, M. (2014). Auto-encoding variational Bayes. In Proceedings of the 2nd International Conference on Learning Representations.

21. Gal, Y., & Ghahramani, Z. (2016). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning (Vol. 48, pp. 1050–1059).

22. Kendall, A., & Gal, Y. (2017). What uncertainties do we need in Bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems (Vol. 30, pp. 5574–5584).

Downloads

Published

2026-09-12

How to Cite

Ruy Infante. (2026). Cross-Domain Speech Synthesis for Personalized Communication Assistance in Patients With Speech Impairments. Global Financial Analytics Research Review, 3(4). Retrieved from https://gfarr.org/index.php/home/article/view/173