Voice recognition / speech commands

[andyss]

-brand and model of bluetooth headset you are using

Motorola HS820

-brand and model of bluetooth dongle

Belkin F8T017

-what stack. (vistas built in, or widcomm, or bluesoleil... don't know if there are others)

WIDCOMM Bluetooth Software from Broadcom (Win7/Vista/XP/2K)

Yeah I was having problems getting the headset to work with the default MS Bluetooth stack in Win7. Kept saying I needed drivers for it but there aren't any to my knowledge or that I could find. Did some Google'in and found a page somewhere that someone used the WIDCOMM stack, so I just tried it, made it the default and it worked. Hope that helps.

I have been trying to get it to show some events in EG but I can't figure it out. I tried using the HID Plugin, experimental Bluetooth Plugin here, even the Serial Port Plugin, but no luck. If you get yours working and get events such as when it connects to show up in EG, let me know. I want to autostart VC after my BT headset connects to the system and exit when it disconnects, how cool would that be!

Andy

[andyss]

@carz181:

You might have to install SAPI in XP if I'm not mistaken. Have you done this already? (Check here: Speech Recognition with Windows XP).

Also make sure you have a recording device set to default before you start VoxCommando.

im running xp sp3. I tryed running voxcammando but it says "error vox needs to close". Any help??

[jitterjames]

thanks, this morning, before I read this, I tried installing the widcomm stack and got it working. Unfortunately I'm finding the quality isn't good enough. I get ok recognition for some commands but in the end it is more annoying than anything since certain commands won't work no matter how clearly I speak. I tried recording some sentences and there is a lot of distortion on many consonants even though the volume level is not very high.

So I guess I'll take it back. Ideally I'd like to have either an array microphone ($$$) or a proper wireless headset with a boom mic that doesn't compress the audio too much. again probably ($$$)

I'm actually getting remarkably good results with the mic that is built into the logitech webcam I have and it was quite cheap, but you still need to be within about 7 feet and the room has to be pretty quiet.

[jitterjames]

@carz181:

You might have to install SAPI in XP if I'm not mistaken. Have you done this already? (Check here: Speech Recognition with Windows XP).

Also make sure you have a recording device set to default before you start VoxCommando.

im running xp sp3. I tryed running voxcammando but it says "error vox needs to close". Any help??

I don't even know if it will work in xp, but you might also want to update to the latest .net from microsoft.

I believe there is actually a plugin for EG for voice recognition that works in xp. Look around the forum. I think even this topic on page 1...

[jitterjames]

Implement a timer function (similar to intercom) so that after a certain amount of time of activating "Send Commands" it automatically turns off so that you could send it a command or two or three and not forget to manually turn it off and send commands you didn't want sent. I see it as another phrase that activates this e.g. to turn on Normal mode say "Start Listening" (or whatever) then you would need to turn it off manually by saying "Stop Listening" and to turn on "Timer" mode say something else, "Computer Timer" (or whatever) and it turns off after a user defined time limit like 5 seconds or a certain amount of time not getting a valid command or 5 seconds of silence.

currently there are two commands you can send from eventghost. one turns the intercom on and the other turns it off

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<EventGhost Version="1194">
    <Folder Name="intercom" Expanded="True">
        <Macro Name="reducevolume" Expanded="True">
            <Event Name="PS3.Button.Blue" />
            <Event Name="MceRemote.Mce.Start" />
            <Action Enabled="False">
                EventGhost.TriggerEvent(u'restoreVolume', 6.0)
            </Action>
            <Action>
                Window.FindWindow(u'VoxCommando.exe', None, None, None, None, None, False, 0.0, 2)
            </Action>
            <Action>
                EventGhost.NewJumpIf(XmlIdLink(86), 1, False)
            </Action>
            <Action>
                Window.SendMessage(273, 1, 325, 0)
            </Action>
            <Action>
                EventGhost.PythonScript(u'import time\nimport thread\n\ntry:\n    intercomactive = eg.globals.intercomactive\n    #print intercomactive\nexcept AttributeError:\n    eg.globals.intercomactive = False\n    #print "intercomactive was not initialized setting to false"\n        \n    \nif  (not eg.globals.intercomactive):\n    eg.globals.regularvolume =eg.plugins.System.ChangeMasterVolumeBy(0, 0)\n    eg.globals.intercomactive = True\n    reducedvolume=int(eg.globals.regularvolume*.2)\n    #print "regular vol: "\n    #print eg.globals.regularvolume\n    #print "target vol: "\n    #print reducedvolume\n    \n    iVol=eg.globals.regularvolume\n    while iVol>reducedvolume:\n        #print iVol\n        iVol = iVol *.8\n        eg.plugins.System.SetMasterVolume(iVol, 0)\n        time.sleep(0.04)\n    eg.plugins.System.SetMasterVolume(reducedvolume, 0)')
            </Action>
        </Macro>
        <Macro Name="restore volume" Expanded="True">
            <Event Name="Main.restoreVolume" />
            <Event Name="Broadcast.Restorevolume" />
            <Action>
                EventGhost.PythonScript(u'import time\nimport thread\n\ntry:\n    intercomactive = eg.globals.intercomactive    \nexcept AttributeError:\n    eg.globals.intercomactive = False\n    #print "intercomactive was not initialized setting to false"\n\nif (eg.globals.intercomactive):    \n    targetvolume = int(eg.globals.regularvolume)       \n    eg.plugins.System.SetMasterVolume(targetvolume, 0)\n    eg.globals.intercomactive = False')
            </Action>
            <Action>
                Window.SendMessage(273, 1, 326, 0)
            </Action>
        </Macro>
        <Macro Name="Start Program: VoxCommando.exe" id="86">
            <Action>
                System.Execute(u'C:\\VoxCommando\\VoxCommando.exe', u'', 0, False, 2, u'')
            </Action>
        </Macro>
    </Folder>
</EventGhost>

I could add another 2 that works like the "start listening" and "stop listening" command (leaving it in voice activated mode) if that would help you.

[andyss]

Is there a way to activate/deactivate the "send Commands" check box only without touching the Intercom toggles, as in through sendMessage? I already use the intercom script with my remote, but wanted one where it would turn on with a voice command and automatically turn off after a set amount of time without any intervention. I tried using Winspector to get the appropriate params but wasn't successful.

Implement a timer function (similar to intercom) so that after a certain amount of time of activating "Send Commands" it automatically turns off so that you could send it a command or two or three and not forget to manually turn it off and send commands you didn't want sent. I see it as another phrase that activates this e.g. to turn on Normal mode say "Start Listening" (or whatever) then you would need to turn it off manually by saying "Stop Listening" and to turn on "Timer" mode say something else, "Computer Timer" (or whatever) and it turns off after a user defined time limit like 5 seconds or a certain amount of time not getting a valid command or 5 seconds of silence.

currently there are two commands you can send from eventghost. one turns the intercom on and the other turns it off

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<EventGhost Version="1194">
    <Folder Name="intercom" Expanded="True">
        <Macro Name="reducevolume" Expanded="True">
            <Event Name="PS3.Button.Blue" />
            <Event Name="MceRemote.Mce.Start" />
            <Action Enabled="False">
                EventGhost.TriggerEvent(u'restoreVolume', 6.0)
            </Action>
            <Action>
                Window.FindWindow(u'VoxCommando.exe', None, None, None, None, None, False, 0.0, 2)
            </Action>
            <Action>
                EventGhost.NewJumpIf(XmlIdLink(86), 1, False)
            </Action>
            <Action>
                Window.SendMessage(273, 1, 325, 0)
            </Action>
            <Action>
                EventGhost.PythonScript(u'import time\nimport thread\n\ntry:\n    intercomactive = eg.globals.intercomactive\n    #print intercomactive\nexcept AttributeError:\n    eg.globals.intercomactive = False\n    #print "intercomactive was not initialized setting to false"\n        \n    \nif  (not eg.globals.intercomactive):\n    eg.globals.regularvolume =eg.plugins.System.ChangeMasterVolumeBy(0, 0)\n    eg.globals.intercomactive = True\n    reducedvolume=int(eg.globals.regularvolume*.2)\n    #print "regular vol: "\n    #print eg.globals.regularvolume\n    #print "target vol: "\n    #print reducedvolume\n    \n    iVol=eg.globals.regularvolume\n    while iVol>reducedvolume:\n        #print iVol\n        iVol = iVol *.8\n        eg.plugins.System.SetMasterVolume(iVol, 0)\n        time.sleep(0.04)\n    eg.plugins.System.SetMasterVolume(reducedvolume, 0)')
            </Action>
        </Macro>
        <Macro Name="restore volume" Expanded="True">
            <Event Name="Main.restoreVolume" />
            <Event Name="Broadcast.Restorevolume" />
            <Action>
                EventGhost.PythonScript(u'import time\nimport thread\n\ntry:\n    intercomactive = eg.globals.intercomactive    \nexcept AttributeError:\n    eg.globals.intercomactive = False\n    #print "intercomactive was not initialized setting to false"\n\nif (eg.globals.intercomactive):    \n    targetvolume = int(eg.globals.regularvolume)       \n    eg.plugins.System.SetMasterVolume(targetvolume, 0)\n    eg.globals.intercomactive = False')
            </Action>
            <Action>
                Window.SendMessage(273, 1, 326, 0)
            </Action>
        </Macro>
        <Macro Name="Start Program: VoxCommando.exe" id="86">
            <Action>
                System.Execute(u'C:\\VoxCommando\\VoxCommando.exe', u'', 0, False, 2, u'')
            </Action>
        </Macro>
    </Folder>
</EventGhost>

I could add another 2 that works like the "start listening" and "stop listening" command (leaving it in voice activated mode) if that would help you.

[jitterjames]

that's what I meant by "equivalent to start listening/stop listening", which turns the "send commands" on and off.

It doesn't exist but I can add it easily. I will also add an open dictionary payload.

I'd rather not do anything with timers at the moment since that can be done with eventghost. I may change my mind about that later though.

[jitterjames]

Here's the list of messages you can send.
wparam is always 1, lparam has four options:

325: intercom on (switches to "Intercom" mode and checks both "Listening" and "Send Commands")
326: intercom off
327: listen (switches to "Voice Activated" mode and checks both "Listening" and "Send Commands")
328: don't listen (unchecks "Send Commands")


Also, now you can do payloads with dictation. At some point I'll probably also add the ability to "spell it" for tough words. Here's an example to be able to search for anything:

Code: Select all

    <command name="google">
      <phrase>search web for,google search</phrase>
      <payloadDictation/>
    </command>

[andyss]

Awesome, I can't wait to try it out. Will let you know how it goes.

Andy

Here's the list of messages you can send.
wparam is always 1, lparam has four options:

325: intercom on (switches to "Intercom" mode and checks both "Listening" and "Send Commands")
326: intercom off
327: listen (switches to "Voice Activated" mode and checks both "Listening" and "Send Commands")
328: don't listen (unchecks "Send Commands")


Also, now you can do payloads with dictation. At some point I'll probably also add the ability to "spell it" for tough words. Here's an example to be able to search for anything:

Code: Select all

    <command name="google">
      <phrase>search web for,google search</phrase>
      <payloadDictation/>
    </command>

[andyss]

One more small request James, hope I'm not too much of a bother!

I've noticed that VC doesn't send an event to EG when the "Listen" command is triggered but events are triggered when the "Don't Listen" command is activated:

Code: Select all

Broadcast.Restorevolume
Broadcast.Don't Listen

Is it possible to generate an event on "Listen"?

Thanks,

Andy

[jitterjames]

a little advice.

be careful creating a bunch of open payload commands. I'm not sure how efficiently the sapi will handle that. You might be better off using 2 payloads, where the first indicates the type of command you want using a payloadList.

try both methods and see whether there is a difference in memory consumption.

e.g.
command = search
phrase = search,find
payloadList = google,disk,yahoo,recipes
payloadDictation
/end of command

then use the first payload in eg to decide where to send the dictation.

[jitterjames]

One more small request James, hope I'm not too much of a bother!

I've noticed that VC doesn't send an event to EG when the "Listen" command is triggered but events are triggered when the "Don't Listen" command is activated:

Code: Select all

Broadcast.Restorevolume
Broadcast.Don't Listen

Is it possible to generate an event on "Listen"?

Thanks,

Andy

yeppers

[jitterjames]

here's version 0.4.0000003

it broadcasts the "listen" event.

[jitterjames]

I want to autostart VC after my BT headset connects to the system and exit when it disconnects, how cool would that be!

Andy

I missed this before. That would in fact be VERY COOL.

[jitterjames]

Ability to define your own sounds with other files instead of replacing speechon.wav / speechoff.wav.

I will probably do this eventually but for now you can just replace the wav files in the
VC directory with whatever you want, keeping the file names.

Poll at intervals for the default recording device and change accordingly. If you don't select (or in my case activate) the correct default device before VC starts you need to do so and restart VC.

I think you may already be able to do this by turning intercom on and off. I will look into it further. I don't know if I'll do a poll, but at least there should a command via message that resets the device. Eventually if it is possible I'd like to be able to use a non-default device by dropdown selection.

Ability to edit your command groups/commands/phrases directly in the VC window.

don't expect this any time soon. Obviously it's a great idea and one that I want to implement eventually but we are talking about a huge amount of work (for me anyway) and it is probably best that I nail down exactly how the xml language will work (almost there) before starting. If someone knows of some existing code in c# that I could modify that would be great.

Minimize to tray.

yes. I need to think about exactly how I want this to work but it should be in there. I like being able to see the commands in the task bar, but that's just me and it's not always appropriate.