Foreign words are not pronounced correctly when using German language

[dimitriosp]

Describe the bug

Hey there, I am facing the issue with all new neural and multilingual but also German focuses voices, that the pronunciation for foreign words inside German sentences is really bad and makes it unusable for our use cases.

f.e.

• Heute gibt es Lasagne.
• Heute gibt es Steak.
• Das Bouquet ist reintönig und frisch.
• Das ist ein schöner Cabernet Sauvignon

I remember that it was indeed better some time ago.
Our system does not support the use ssml or language tags since the text is ai generated and send through api calls.

I also checked all regions and countless voice models, all with the same result.

Any idea what to do here?

To Reproduce

Go to https://speech.microsoft.com/, select german language, add the examples above and listen to the issues.
f.e. Seraphina Multilingual

Expected behavior

Foreign words should pronounced correct when using a multilingual neural voice model.

Platform, Operating System, and Programming Language

Using speech studio and api calls

[ChuuFu]

Hi @dimitriosp , well received this feedback.

I noticed you mentioned that "it was indeed better some time ago", so may I ask if there has been a regression? When did it occur?

Additionally, we used the example sentences you provided(Heute gibt es Steak.) to synthesize audio and listened to it. The foreign word "Steak" in the sentence doesn't seem to present much of a problem in the audio, so we would like you to confirm what exactly you mean by "the pronunciation is bad". Are you referring to a mispronunciation, or is there noise or something else? If it is a pronunciation issue, could you please describe the current incorrect pronunciation and the expected pronunciation more specifically?
Also, could you provide the SSML used during the synthesis to help us investigate the issue?

[dimitriosp]

Hello @ChuuFu

I remember 1-2 years ago, that English words where pronounced correct when German voice was selected. If you use: "Heute gibt es Lasagne" it will be spoken "Heute gibt es Lasakne" or "Heute gibt es Stick" instead of "Heute gibt es Steak". When trying the same with an English voice it sounds good "Today we will have Steak" "Today we will have Lasagne". Or also "heute haben wir Cabernet Sauvignon." (Kabertnet Sauvignon) and try the english version "Today we will have Cabernet Sauvignon.". As soon as a foreign word is used, it is spoken in "German" and not in the foreign pronunciation.

Testing the exact same words in Google TTS shows that it is possible. I will try to get SSML as we do not use it in our system.

[dimitriosp]

Another test you could do to see the differences, is to add this code to the speech studio to see the difference. when choosing Italian for this one word, it sounds beautiful again. Using just German, its not usable.

(i cant add the code in plain text here)