Text to Speech on unity
Make your own game talk using windows SAPI voices
About SAPI
The Speech Application Programming Interface or SAPI is an API developed by Microsoft to allow the use of speech recognition and speech synthesis within Windows applications. To date, a number of versions of the API have been released, which have shipped either as part of a Speech SDK, or as part of the Windows OS itself. Applications that use SAPI include Microsoft Office, Microsoft Agent and Microsoft Speech Server. (wikipedia)
How to embed SAPI and Unity ?
Fon SAPI unity integration we need an external DLL : Interop.SpeechLib.dll
If you are already using Interop.SpeechLib.dll but you application still crashing after build click here
Code TTSUnityWin.Cs (c#)
For loading external XML, remember to put the files in the build resource folder (asset)
string loadXMLStandalone (string fileName) {
string path = Path.Combine("Resources", fileName);
path = Path.Combine (Application.dataPath, path);
Debug.Log ("Path: " + path);
StreamReader streamReader = new StreamReader (path);
string streamString = streamReader.ReadToEnd();
Debug.Log ("STREAM XML STRING: " + streamString);
return streamString;
}
voice.Speak(loadXMLStandalone ("external.xml"), SpeechVoiceSpeakFlags.SVSFlagsAsync);
Calling Speak Function Samples :
voice.Speak("<speak version='1.0' xmlns='http://www.w3.org/2001/10/synthesis' xml:lang='en-US'>"
+"English One, two, three"
+"<p xml:lang='it-IT'> Italiano uno, due, tre</p>"
+"<p xml:lang='ja-JP'>日本語 一二三、終わります。 </p>"
+ "</speak>",
SpeechVoiceSpeakFlags.SVSFlagsAsync|SpeechVoiceSpeakFlags.SVSFIsXML);
voice.Speak("NOW XML SAPI TEST", SpeechVoiceSpeakFlags.SVSFlagsAsync);
voice.Speak("Spelling: Chiaroscurist, <spell>Chiaroscurist.</spell>Normal Volume,<volume level='50'>Low volume</volume>", SpeechVoiceSpeakFlags.SVSFlagsAsync|SpeechVoiceSpeakFlags.SVSFIsXML);
voice.Speak("<lang langid='410'>Frase in italiano</lang>", SpeechVoiceSpeakFlags.SVSFlagsAsync|SpeechVoiceSpeakFlags.SVSFIsXML);
voice.Speak("ESTERNAL FILE READING TEST", SpeechVoiceSpeakFlags.SVSFlagsAsync);
// voice.Speak(loadXMLStandalone ("builtIn.xml"), SpeechVoiceSpeakFlags.SVSFlagsAsync);
// voice.Speak(BuiltAsset, SpeechVoiceSpeakFlags.SVSFlagsAsync);
voice.Speak(loadXMLStandalone ("external.xml"), SpeechVoiceSpeakFlags.SVSFlagsAsync);//must in the resource folder after build
voice.Speak("End of the test, enjoy this code", SpeechVoiceSpeakFlags.SVSFlagsAsync);
Userful Links about windows Sapi and XML formatting
https://msdn.microsoft.com/en-us/library/ms723602(v=vs.85).aspx
https://msdn.microsoft.com/en-us/library/ms717077(v=vs.85).aspx
https://msdn.microsoft.com/en-us/library/windows/desktop/dd318693(v=vs.85).aspx
https://msdn.microsoft.com/en-us/library/jj127898.aspx
Questions? Suggestions? Please leave a comment below.
Thank you for the work you have done with this. I am just learning about SAPI and SpVoice. This demo works pretty well, but my next step is trying to save these audio samples to a wav or mp3 file. Do you have any advice or direction you can point me in?
Hi Eric
You are more than welcome, I’m glad I was able to help
I have done something similar last Year but I lost the code, I’ll try to figure out how to save in mp3 by the end of next week an i let you know
As Eric mentioned, i’m trying to connect this with a live lypsync library that uses audiosources for calculating voice animations. A perfect way to do this would be to connect the voice.AudioOutputStream into a audio source from Unity. Or as a workaround convert the voice to wav/mp3 but haven’t find a working way for this. Does anyone have a clue of where to start?
Hello Marco
That is really great job !
Unfortunatly, on my side, I can’t build a standalone EXE, even directly with your project.
I have the following issue:
<>
and no EXE is built 🙁
Found in google that it is a regular issue with external DLL.
I can see that there is an EXE version to be downloaded, so you managed to build one from unity. How did you do it ?
Thanks for you help,
Benjamin
error is:
“The classes in the module cannot be loaded”
Hey there, first of all: wow great post, I like you!!!
I will use this in a lot of projects, BUT: In the current version of Unity 5.4 (not sure if this was the case before and you know how to solve it) I get the following error, when I try to build your project…
looks like something is not good with the DLL? But what would it be and how could I make it compile. Oh that would be so cool if you could help me please. Thanks!
——
ReflectionTypeLoadException: The classes in the module cannot be loaded.
System.Reflection.Assembly.GetTypes () (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/Assembly.cs:371)
UnityEditor.AttributeHelper+c__Iterator3`1[System.Collections.Generic.IEnumerable`1[UnityEditorInternal.PluginDesc]].MoveNext () (at C:/buildslave/unity/build/Editor/Mono/AttributeHelper.cs:277)
UnityEditor.PluginImporter+c__Iterator0.MoveNext () (at C:/buildslave/unity/build/Editor/Mono/PluginImporter.cs:41)
DesktopStandalonePostProcessor.CopyNativePlugins () (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/DesktopStandalonePostProcessor.cs:116)
DesktopStandalonePostProcessor.SetupStagingArea () (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/DesktopStandalonePostProcessor.cs:145)
DesktopStandalonePostProcessor.PostProcess () (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/DesktopStandalonePostProcessor.cs:23)
UnityEditor.WindowsStandalone.WindowsDesktopStandalonePostProcessor.PostProcess (BuildPostProcessArgs args) (at C:/buildslave/unity/build/PlatformDependent/WinPlayer/Extensions/Managed/WindowsDesktopStandalonePostProcessor.cs:118)
UnityEditor.PostprocessBuildPlayer.Postprocess (BuildTarget target, System.String installPath, System.String companyName, System.String productName, Int32 width, Int32 height, System.String downloadWebplayerUrl, System.String manualDownloadWebplayerUrl, BuildOptions options, UnityEditor.RuntimeClassRegistry usedClassRegistry, UnityEditor.BuildReporting.BuildReport report) (at C:/buildslave/unity/build/Editor/Mono/BuildPipeline/PostprocessBuildPlayer.cs:176)
UnityEditor.HostView:OnGUI()
Hi Ben, sorry for late reply and thank you very much.
This code was written with an old versione of unity 5, and i have to figure out whi ins not working anymore. I’ll try to find a way asap but if you have find a resolution please share it with us !
Great post.
Did you ever get Eric’s request solved with outputting to WAV or MP3? That would help big time.
I’ve been tearing through the internet looking for a way in which to set sapi as the audiosource in Unity, but the best solution I can find is to have sapi output to wav or mp3 and then set the file as an audiosource.
Has there been a fix/update to this?
Hello jasmine, i’m so sorry till using old unity version, i did not had time to find a wai to fix it in the new update, hope to do it by the end of the month
No problem! I look forward to your update, and thank you so much for your quick response!
i got these error:
COMException
>System.Runtime.InteropServices.Marshall.ThrowExceptionForHR(Int32 errorCode)(at /users/buildusers/bulidslave/mono-runtime-and-classlib/build/mcs/class/corlib/System.Runtime.InteropServices/Marshall.cs:1031)
(wrapper cominterop)SpeechLib.SpVoiceClass.Speak(strring,SpeechLib.SpeechVoiceSpeakFlags)TTSUnityWin.Update()(at D:/Unity
D/Text2Speech/FinalMarco_Text2Speech/Assetes/TTSUnityWin.cs:100)
can anyone help me?i’m an absolute beginner in unity. thanks a lot.
I think is caused by the new version of unity, i’m trying to figure out
Last working version is Unity 5.3.6.
Thnaks so much @Finalmarco for doing this!
glad to help 🙂
Did you ever get the error below resolved in newer versions of Unity?
ReflectionTypeLoadException: The classes in the module cannot be loaded.
System.Reflection.Assembly.GetTypes () (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/Assembly.cs:371)