Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-Speech
Although recent neural text-to-speech (TTS) systems have achieved high-quality speech synthesis, there are cases where a TTS system generates low-quality speech, mainly caused by limited training data or information loss during knowledge distillation. Therefore, we propose a novel method to improve...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9775804/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850190900438761472 |
|---|---|
| author | Yeunju Choi Youngmoon Jung Youngjoo Suh Hoirin Kim |
| author_facet | Yeunju Choi Youngmoon Jung Youngjoo Suh Hoirin Kim |
| author_sort | Yeunju Choi |
| collection | DOAJ |
| description | Although recent neural text-to-speech (TTS) systems have achieved high-quality speech synthesis, there are cases where a TTS system generates low-quality speech, mainly caused by limited training data or information loss during knowledge distillation. Therefore, we propose a novel method to improve speech quality by training a TTS model under the supervision of perceptual loss, which measures the distance between the maximum possible speech quality score and the predicted one. We first pre-train a mean opinion score (MOS) prediction model and then train a TTS model to maximize the MOS of synthesized speech using the pre-trained MOS prediction model. The proposed method can be applied independently regardless of the TTS model architecture or the cause of speech quality degradation and efficiently without increasing the inference time or model complexity. The evaluation results for the MOS and phone error rate demonstrate that our proposed approach improves previous models in terms of both naturalness and intelligibility. |
| format | Article |
| id | doaj-art-94c605add4db4b9fa8fb5f63361d4929 |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-94c605add4db4b9fa8fb5f63361d49292025-08-20T02:15:07ZengIEEEIEEE Access2169-35362022-01-0110526215262910.1109/ACCESS.2022.31758109775804Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-SpeechYeunju Choi0https://orcid.org/0000-0003-2192-2680Youngmoon Jung1https://orcid.org/0000-0002-4321-379XYoungjoo Suh2Hoirin Kim3https://orcid.org/0000-0002-8787-6982School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South KoreaSchool of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South KoreaVoice Group, Konan Technology Inc., Seoul, South KoreaSchool of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South KoreaAlthough recent neural text-to-speech (TTS) systems have achieved high-quality speech synthesis, there are cases where a TTS system generates low-quality speech, mainly caused by limited training data or information loss during knowledge distillation. Therefore, we propose a novel method to improve speech quality by training a TTS model under the supervision of perceptual loss, which measures the distance between the maximum possible speech quality score and the predicted one. We first pre-train a mean opinion score (MOS) prediction model and then train a TTS model to maximize the MOS of synthesized speech using the pre-trained MOS prediction model. The proposed method can be applied independently regardless of the TTS model architecture or the cause of speech quality degradation and efficiently without increasing the inference time or model complexity. The evaluation results for the MOS and phone error rate demonstrate that our proposed approach improves previous models in terms of both naturalness and intelligibility.https://ieeexplore.ieee.org/document/9775804/MOS predictionneural TTSperceptual lossspeech synthesis |
| spellingShingle | Yeunju Choi Youngmoon Jung Youngjoo Suh Hoirin Kim Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-Speech IEEE Access MOS prediction neural TTS perceptual loss speech synthesis |
| title | Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-Speech |
| title_full | Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-Speech |
| title_fullStr | Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-Speech |
| title_full_unstemmed | Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-Speech |
| title_short | Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-Speech |
| title_sort | learning to maximize speech quality directly using mos prediction for neural text to speech |
| topic | MOS prediction neural TTS perceptual loss speech synthesis |
| url | https://ieeexplore.ieee.org/document/9775804/ |
| work_keys_str_mv | AT yeunjuchoi learningtomaximizespeechqualitydirectlyusingmospredictionforneuraltexttospeech AT youngmoonjung learningtomaximizespeechqualitydirectlyusingmospredictionforneuraltexttospeech AT youngjoosuh learningtomaximizespeechqualitydirectlyusingmospredictionforneuraltexttospeech AT hoirinkim learningtomaximizespeechqualitydirectlyusingmospredictionforneuraltexttospeech |