Voice recognition / speech commands

[jitterjames]

I will post a version where turning the osd off is an option soon, maybe tomorrow. If you minimize the program though I don't think you'll see the osd popups.

how do you find the bluetooth headset? does it work well? does the computer usually understand you correctly? I've been thinking about getting one but wasn't sure it would work reliably enough unless I got something with a boom mic.

[andyss]

Actually, the Bluetooth headset works pretty good, when I can get it working that is. Lately, I can't get past this exception error and it refuses to connect to the BT headset:

Code: Select all

See the end of this message for details on invoking 
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.NullReferenceException: Object reference not set to an instance of an object.
   at VoxCommando.Main.CreateCommandGrammar()
   at VoxCommando.Main.InitializeSpeechRecognitionEngine()
   at VoxCommando.Main.Main_Load(Object sender, EventArgs e)
   at System.Windows.Forms.Form.OnLoad(EventArgs e)
   at System.Windows.Forms.Form.OnCreateControl()
   at System.Windows.Forms.Control.CreateControl(Boolean fIgnoreVisible)
   at System.Windows.Forms.Control.CreateControl()
   at System.Windows.Forms.Control.WmShowWindow(Message& m)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.ScrollableControl.WndProc(Message& m)
   at System.Windows.Forms.ContainerControl.WndProc(Message& m)
   at System.Windows.Forms.Form.WmShowWindow(Message& m)
   at System.Windows.Forms.Form.WndProc(Message& m)
   at VoxCommando.Main.WndProc(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.4927 (NetFXspW7.050727-4900)
    CodeBase: file:///C:/Windows/Microsoft.NET/Framework/v2.0.50727/mscorlib.dll
----------------------------------------
VoxCommando
    Assembly Version: 1.0.0.0
    Win32 Version: 1.0.0.0
    CodeBase: file:///C:/Program%20Files/EventGhost/Andy_extras/VoxCommando/VoxCommando.exe
----------------------------------------
System.Windows.Forms
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.4927 (NetFXspW7.050727-4900)
    CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Windows.Forms/2.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.4927 (NetFXspW7.050727-4900)
    CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.4927 (NetFXspW7.050727-4900)
    CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Drawing/2.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System.Speech
    Assembly Version: 3.0.0.0
    Win32 Version: 3.0.6920.1109 (lh_tools_devdiv_wpf.071009-1109)
    CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Speech/3.0.0.0__31bf3856ad364e35/System.Speech.dll
----------------------------------------
System.Xml
    Assembly Version: 2.0.0.0
    Win32 Version: 2.0.50727.4927 (NetFXspW7.050727-4900)
    CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Xml/2.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
    <system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.

Hopefully this will help you out because I have no idea what it means
Also, when I launch VoxCommando minimized, it shows the OSD. It won't show the OSD if I restore then re-minimize (?) VoxCommando. My BT headset is a Motorola HS820 I had lying around.

BTW, greetings from Toronto!

Andy

I will post a version where turning the osd off is an option soon, maybe tomorrow. If you minimize the program though I don't think you'll see the osd popups.

how do you find the bluetooth headset? does it work well? does the computer usually understand you correctly? I've been thinking about getting one but wasn't sure it would work reliably enough unless I got something with a boom mic.

[jitterjames]

@andy: is that exception happening with the latest version and not the one before?

is the error code that you included what you get when you click the details button (the one shown on error.png)?

I don't think that message means much anyway. We would need to figure out how to enable the jit "just in time debugger". I just started to program for windows about a year ago and there is a LOT that I don't know how to do.

If you change the default recording device to something else (a mic or line in) does it still give you the same error?

if you want you can send me a zip of your complete folder so I can test it to see if it has anything to do with the xml files or something.

a fellow Canuck! Did I give myself away earlier by saying eh? or something?

I'll make the osd an option today. I'm not that fond of it anyway but sometimes it comes in handy.

I'm working on user definable payload commands right now and I'm only maintaining a single version of the software at the moment (except for backups).

I guess I need to start creating version numbers at least. This is mostly new to me.

J

[andyss]

I tracked it down to a comment I made in the xml file between <VoiceCommands> </VoiceCommands>. All works good now. If you need me to test out versions, I'll do my best. Can't wait for the payload commands!

Andy

[jitterjames]

soon!

I will also try to start putting in some better error checking when reading the xml etc.

It takes time. Right now I am actually just going through the code putting in comments because it's getting harder to keep track of everything.

[jitterjames]

Does anyone have specific ideas of how you would like to use payload commands? I know how I would like to set it up and use them but maybe I am suffering from tunnel vision.

[andyss]

What's your plan in setting it up?

Does anyone have specific ideas of how you would like to use payload commands? I know how I would like to set it up and use them but maybe I am suffering from tunnel vision.

[jitterjames]

What's your plan in setting it up?

well the most obvious is to allow certain commands to be followed by a number range so you could define a command like this

Code: Select all

command = "channel", range (1,50)
command = "volume", range (1,10)

so you could say "channel 5" and it would generate an event "channel" with the payload "5", but it wouldn't accept "channel 60"

The artist, album, movie thing is already using payloads, but I could make it more flexible so that you could create a list and call it what you want.

Code: Select all

<command name= "playlist">
   <payload value= "andys songs">
   <payload value= "blues">
</command>
<command name= "website">
   <payload value= "weather">
   <payload value= "stocks">
</command>

then if you said "playlist blues" it would generate an event "playlist" with the payload "blues"

[andyss]

My opinion is that you should keep it as open as possible to allow as much flexibility as possible. I don't know what kind of hurdles you would have to face in implementing that philosophy/technique, but I like the idea of having <payload> and <range> attributes available to any/most command(s).

I appreciate the work you are doing and have some more suggestions if you don't mind looking at them. Just let me know.

One of my pet peeves is ppl demanding things from developers that develop FREE software and have a passion about the project they are working on. I like to see projects flourish but I am not a person that demands improvements because I think I'm entitled to it.

I just wish I was adept at code

Andy

[jitterjames]

I would be happy to hear more suggestions. If I don't like them I just won't use them, and I certainly won't consider it a demand. Ideas are a contribution and I would appreciate you taking the time to write them down.

I have not decided what I want to do with the software in the long term. I may decide to make it completely open source, or I may not. Mostly likely it will be free/donation based and I will charge to install and set it up for people. That said anyone who tests or contributes in any way will certainly be welcome to free updates for as long as I am making them.

It would be nice if I could make some money off this eventually because it is a lot of work, but for now I am enjoying myself a lot, making something very cool (imo) and learning much along the way.

[jitterjames]

Just a heads up that I will probably be release version 0.4 soon. I think it'll be a big step forward but will most likely require some major changes to the voicecommands.xml file. I think it will be close enough that you can just edit it but you may want to start from scratch.

I'll probably put together a zip with everything in it again including some sample xml.

J

[andyss]

Hope I'm not hijacking your thread James but I have a question that is in relation to your program but more to do with EG. I'm trying to get the Load Volume Grammar working to have Eventghost change the Master Volume. I'm getting the event to fire but I can't seem to figure out the necessary Eventghost commands. I've tried the system.SetMasterVolume as well as the eg.WinApi.SoundMixer.SetMasterVolume().

Here's my code:

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<EventGhost Name="Configuration Tree" Expanded="True" Version="1194" Guid="{37249E24-1E4C-40D5-B1CC-E50C4E950660}" Time="1252431590.31">
    <Autostart Name="Autostart" />
    <Macro Name="Set Master Volume" Expanded="True">
        <Event Name="speech.Set Volume" />
        <Action>
            EventGhost.PythonCommand(u'eg.globals.volLevel = eg.event.payload[0]')
        </Action>
        <Action Name="setvolume">
            EventGhost.PythonCommand(u'system.SetMasterVolume(volLevel, 0)')
        </Action>
    </Macro>
</EventGhost>

The error I get is:

Code: Select all

         Error in Action: "system.SetMasterVolume(volLevel, 0)"
         Traceback (most recent call last) (1194):
         File "<string>", line 1, in <module>
         NameError: name 'system' is not defined

Would anyone know how to change the Master Volume to an event payload value?

Andy

[jitterjames]

this is what I'm using. I don't actually know if it works with win7.

make sure you have got the event firing and that you can see the payload with a payload

it should look like this in the log: Broadcast.Set Volume '55'

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<EventGhost Version="1194">
    <Macro Name="set volume to payload" Expanded="True">
        <Event Name="Broadcast.Set Volume" />
        <Action>
            EventGhost.PythonScript(u"eg.plugins.System.SetMasterVolume(int(eg.event.payload), 0)\neg.globals.regularvolume =int(eg.event.payload)\neg.plugins.EventGhost.ShowOSD(u'volume: '+eg.event.payload, u'0;-48;0;0;0;700;0;0;0;0;3;2;1;34;Arial', (255, 255, 255), (0, 0, 0), 4, (0, 0), 0, 3.0, True)")
        </Action>
    </Macro>
</EventGhost>

[andyss]

Works perfectly in win7, thanks!

A.

this is what I'm using. I don't actually know if it works with win7.

make sure you have got the event firing and that you can see the payload with a payload

it should look like this in the log: Broadcast.Set Volume '55'

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<EventGhost Version="1194">
    <Macro Name="set volume to payload" Expanded="True">
        <Event Name="Broadcast.Set Volume" />
        <Action>
            EventGhost.PythonScript(u"eg.plugins.System.SetMasterVolume(int(eg.event.payload), 0)\neg.globals.regularvolume =int(eg.event.payload)\neg.plugins.EventGhost.ShowOSD(u'volume: '+eg.event.payload, u'0;-48;0;0;0;700;0;0;0;0;3;2;1;34;Arial', (255, 255, 255), (0, 0, 0), 4, (0, 0), 0, 3.0, True)")
        </Action>
    </Macro>
</EventGhost>

[andyss]

If anyone is interested in the flip-side of computer speech, not only commanding your computer by way of jitterjames' excellent speech-recognition interface, but actually having it respond back to you with speech, I've found a simple command-line program that uses the MS SAPI api and works incredibly well with Eventghost:

Say.exe - a simple command line text to speech program for Windows

I've only tested it with Win7 RTM 32 so YMMV if you use XP or Vista.

Andy