Voice recognition / speech commands

[jitterjames]

there's also an intercom feature. If you have a remote you can tell the computer when to listen to you using a button on the remote. it'll turn the volume down until you issue a command, then the computer will turn the volume back up and stop listening to you until you hit the button on the remote again.

to use it add this to your EG macros

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<EventGhost Version="1194">
    <Folder Name="intercom" Expanded="True">
        <Macro Name="reducevolume" Expanded="True">
            <Event Name="HID.Blue" />
            <Event Name="MCE_Vista.Mce.Details" />
            <Action>
                EventGhost.TriggerEvent(u'restoreVolume', 6.0)
            </Action>
            <Action>
                Window.FindWindow(u'VoxCommando.exe', None, None, None, None, None, False, 0.0, 2)
            </Action>
            <Action>
                EventGhost.NewJumpIf(XmlIdLink(40), 1, False)
            </Action>
            <Action>
                Window.SendMessage(273, 1, 325, 0)
            </Action>
            <Action>
                EventGhost.PythonScript(u'import time\nimport thread\n\ntry:\n    intercomactive = eg.globals.intercomactive\n    print intercomactive\nexcept AttributeError:\n    eg.globals.intercomactive = False\n    print "intercomactive was not initialized setting to false"\n        \n    \nif  (not eg.globals.intercomactive):\n    eg.globals.regularvolume =eg.plugins.System.ChangeMasterVolumeBy(0, 0)\n    eg.globals.intercomactive = True\n    reducedvolume=int(eg.globals.regularvolume*.2)\n    print "regular vol: "\n    print eg.globals.regularvolume\n    print "target vol: "\n    print reducedvolume\n    \n    iVol=eg.globals.regularvolume\n    while iVol>reducedvolume:\n        #print iVol\n        iVol = iVol *.8\n        eg.plugins.System.SetMasterVolume(iVol, 0)\n        time.sleep(0.04)\n    eg.plugins.System.SetMasterVolume(reducedvolume, 0)')
            </Action>
        </Macro>
        <Macro Name="restore volume" Expanded="True">
            <Event Name="Main.restoreVolume" />
            <Event Name="Broadcast.Restorevolume" />
            <Action>
                EventGhost.PythonScript(u'import time\nimport thread\n\ntry:\n    intercomactive = eg.globals.intercomactive    \nexcept AttributeError:\n    eg.globals.intercomactive = False\n    #print "intercomactive was not initialized setting to false"\n\nif (eg.globals.intercomactive):    \n    targetvolume = int(eg.globals.regularvolume)    \n    while eg.plugins.System.ChangeMasterVolumeBy(5, 0)<targetvolume-5:\n        time.sleep(0.1)\n    #print "restoring volume"\n    eg.plugins.System.SetMasterVolume(targetvolume, 0)\n    eg.globals.intercomactive = False\n')
            </Action>
            <Action Enabled="False">
                Window.FindWindow(u'VoxCommando.exe', None, None, None, None, None, False, 0.0, 0)
            </Action>
            <Action Enabled="False">
                Window.SendMessage(273, 1, 326, 0)
            </Action>
        </Macro>
        <Macro Name="Start Program: VoxCommando.exe" id="40">
            <Action>
                System.Execute(u'C:\\VoxCommando\\VoxCommando.exe', u'', 0, False, 2, u'')
            </Action>
        </Macro>
    </Folder>
</EventGhost>

[Bob]

It seems to function. I'm able to make it listen and unlisten.

I'm calling it Hal, in memory of Stanley !!!

But after testing it a while, i'm not able to make it recognise any commands except television.

Here is my commands configuration :

Code: Select all

<?xml version="1.0" encoding="utf-8"?>
<!--A List commands for VoxCommando-->
<!--When the computer hears "alias.name" it will broadcast the corresponding "command.name"-->
<VoiceCommands>
  <command name="Listen">
    <alias name="hal attention" />
  </command>
  <command name="Don't Listen">
    <alias name="hal terminer" />
  </command>
  <command name="television">
    <alias name="television" />
  </command>
   <command name="pause">
    <alias name="pause" />
  </command>
</VoiceCommands>

Thanks,

Bob

[jitterjames]

If I understand you correctly, everything is working except pause.

try adding a few more commands and see how it goes. I'm not exactly sure about the french. Try using a bunch of different words in both languages and see. Try saying pause with an english accent (i.e say paws, not pows - haha)

How well does the regular voice recognition in vista work for you? Are you able to do dictation?

Are your commands showing up in eventghost?

[Bob]

My problem is that only these three commands are recognized :

Listen
Don't listen
television

I tried many combination but nothing happen. Sometimes, the command is written by your software but is "not validated" and sent to EG.

For example thess commands are not well recognized :

test,
position,
lecture,
and whatever.

Tomorrow, i'll try to improve the speech recognition of Vista.

Do you know a free and simple software to test voice recognition ?

Bob

[jitterjames]

for the case where it is shown but not sent, I can make you a version that is less picky.

there is a "confidence" rating that I check before issuing a command. It is set at .9 (out of 1) but i can lower it for you.

let me check.

as for free, I would just try playing around with the built in features of vista. You should be able to do some dictation etc. The quality of sound on your microphone is also pretty important. Try recording your voice (click start and type recorder to get the windows sound recorder) and then play it back to make sure it is clear. If it is too loud it can become distorted, if it is too quiet or if you are in a very noisy room you will have problems.

I will send you a new version soon with lower confidence requirements.

[jitterjames]

Bob (should I call you Robert?),

Here's a version (just the .exe / overwrite the old one) that requires a lower confidence before broadcasting the command.

I also added an extra button at the top right. It's a big button with a "?" on it. Click it to check whether Vox is operating in the correct language! When I click it I get "en-US" which is correct for me.

let me know.

J

[jitterjames]

maybe this will help too...

[andyss]

Hi JJ, does your program work with Win7? I've tried and can't seem to get it working.

Andy

[jitterjames]

yes, I've run it on a couple of computers with win7 x86 without problems, and on vista x86 and vista x64.

can you be more specific? What doesn't work?

One thing I should mention, it uses whatever is marked as the default input device. Go to controlpanel - sound. Make sure your microphone is set to default and working. If vista's built in voice recognition is working, then Vox should work. Also, you shouldn't need to do much but little windows voice training is a good idea.

Also, my code does hardly any error checking yet so if you mess up the xml files it probably won't work.

The software's really not ready for the public yet, but I will try to help you set it up if you give me more details. it works really well for me, once properly set up.

[Bob]

Hi, i'm back.


Thanks for your dedicated version. I tried it and clicked on the new "?" button : the language used is the good one, ie French. No problem here.


Next, I changed my microphone with a toocam one and I improved speech recognition with Vista. Now it's much better and i gone back to the first version of VoxCommando.

Now, I can use much more commands. I noticed that the last syllable is the most important one. I don't know if it's a French particularity.

I'm impatient to test payload function.

Now, i think that your software has to improve his stabilty and usuability. I seen some exceptions thrown (Oups !)

If you need help to develop your sofware don't hesitate to send me a PM.


But please, don't call me Robert... I'm too young for that. Bob is not my real name. If you want you can call me Pack.

One more time, thanks.

Bob

[jitterjames]

can you be more specific about the exceptions? I really haven't done much in the way of error checking yet, but once all the xml files are properly read and the program is running i don't see any exceptions here.

I'm not sure about the end of the word being more important. I think certain sounds are easier to identify. "computer" and "enter" sometimes get mixed up if i don't speak load enough, but the computer can very easily identify longer phrases. "please go over to the sink and make me a cup of tea" it would understand very easily unless you also had a command like "please go over to the sink and mock me a couple toast"... though it would probably still get it right. I can reliably call up almost any of my 7000 songs by name, the longer the better!

I am already using some payloads, for example "set volume (1-100)". Also all the artist album song and movie commands are using payloads. All of these are currently hard wired. I plan to create more flexible ways of customizing this. Maybe you can tell me different ways you would like to use the payload so that I can take that into account while I'm devising a strategy.

One thing I would like to do for sure, is to be able to specify a command, something like this.

command:up
alias: up, go up, move up
number range: 1,10

so you can say "go up 4" and it would generate the up command with payload '4'

it would also be nice to be able to define longer sententences

(go, move, travel) (up, down, left, right) (1-20)

and to mark specific words as optional.

(go)* (up,down,left,right) (please)*

there are a lot of possibilities to consider, but I think the most important for now is to allow the user to easily add a number range in a command.

btw, I think you can generate payload commands already in the vox command xml just by using a question mark but you still have to enter a command for each possible payload, which is a bit cumbersome.

example: rating?1, rating?2, rating?3... will appear in eventghost as the command: rating with a payload of 1,2 or 3

if you want I can hard-wire you a french volume command with payload today. Just tell me what you want. same goes for move commands, just send me the translations you want.

thoughts?

[jitterjames]

I seen some exceptions thrown (Oups !)

do you mean the message box that pops up saying "Oooops"?

this is actually not an exception per se. If you say something that the computer thinks could be interpreted many ways it, executes the most likely command, but also sends EG a list of commands that you can display. the user can then say option 2, or option 3 and it will execute that command. if your original command only generated 3 options and you said "option 4" it would pop up the Ooops message.

Not very helpful to the end user though right? What comes before alpha? This is definitely pre-alpha software!

btw here's the EG macro to display messages sent by vox

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<EventGhost Version="1194">
    <Macro Name="Display VoxCommando Message" Expanded="True">
        <Event Name="Broadcast.Message" />
        <Action>
            EventGhost.PythonScript(u'print eg.event.payload')
        </Action>
        <Action>
            EventGhost.ShowOSD(u'{eg.event.payload}', u'0;-48;0;0;0;700;0;0;0;0;3;2;1;34;Arial', (255, 255, 255), (0, 0, 0), 4, (0, 0), 0, 5.0, True)
        </Action>
    </Macro>
</EventGhost>

of couse it's very simple, you can change it as you like or make your own.

[Bob]

You're right. There is pop-up message with "Ooops".

My problem is that if this pop-up is shown VC (VoxCommando) do not anymore respond, until I close with a mouse the pop-up. Not very usefull.


Further more, i observed a memory leak. The process start at 30mo and after a few time it is at more than 90mo. Maybe it is caused by the log in the window.


And to finish, I have always commands with movie? or artiste even if I do not say anything. Do you think that you can desactivate this ?

Sorry for my number of requests but i am very interested by your software.

Bob

[jitterjames]

This should be better. You can edit the options.xml file. It looks like this:

Code: Select all

<?xml version="1.0" encoding="utf-8"?>
<!--Options for VoxCommando voice commands.-->
<Options>
	  <option name="LoadMusicXml" value="true" />
	  <option name="LoadMovieXml" value="true" />
	  <option name="LoadMoveMultipleGrammar" value="true" />
	  <option name="LoadVolumeGrammar" value="true" commandText="Set Volume?"/>
</Options>

So Bob, you can set the LoadMusicXml and LoadMovieXml to false. Change "Set Volume?" to a french command if you want. REmember the ? at the end. It will then let you say "set volume xx" where xx is a number from 1 to 100, and will issue a command with a payload of xx.

here's the macro I use with it:

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<EventGhost Version="1194">
    <Macro Name="set volume to payload" Expanded="True">
        <Event Name="Broadcast.Set Volume" />
        <Action>
            EventGhost.PythonScript(u"eg.plugins.System.SetMasterVolume(int(eg.event.payload), 0)\neg.globals.regularvolume =int(eg.event.payload)\neg.plugins.EventGhost.ShowOSD(u'volume: '+eg.event.payload, u'0;-48;0;0;0;700;0;0;0;0;3;2;1;34;Arial', (255, 255, 255), (0, 0, 0), 4, (0, 0), 0, 3.0, True)")
        </Action>
    </Macro>
</EventGhost>

are you still getting memory leaks? I'm not seeing any here. I thought they were pretty rare when using .net.

[andyss]

HI James, I figured out the problem. I'm using a Bluetooth Headset and was turning it on after I started VoxCommando so it wasn't switching over. But it works great, I'm very impressed with the program. Can't wait for the upcoming releases. I know it's in alpha stage right now, but is there a way to turn off the OSD?

Andy

Hi JJ, does your program work with Win7? I've tried and can't seem to get it working.

Andy

yes, I've run it on a couple of computers with win7 x86 without problems, and on vista x86 and vista x64.

can you be more specific? What doesn't work?

One thing I should mention, it uses whatever is marked as the default input device. Go to controlpanel - sound. Make sure your microphone is set to default and working. If vista's built in voice recognition is working, then Vox should work. Also, you shouldn't need to do much but little windows voice training is a good idea.

Also, my code does hardly any error checking yet so if you mess up the xml files it probably won't work.

The software's really not ready for the public yet, but I will try to help you set it up if you give me more details. it works really well for me, once properly set up.