Notice: This forum has been recovered from an old backup, so some content, links, and dates may be outdated. The forum is currently read-only while we restore sign-in and registration functionality. Details

If you find this forum valuable and would like to help keep it online, donations to help cover hosting and domain costs are greatly appreciated, but never expected. You can support the forum through Buy Me a Coffee or Ko-fi. Thank you for helping preserve the EventGhost community.

TextGrab plugin

If you have a question or need help, this is the place to be.
Post Reply
Samcek
Posts: 14
Joined: Sat Oct 03, 2009 4:52 pm

TextGrab plugin

Post by Samcek »

Hi everybody. As this is my first post, I would first like to thank the author(s) for this great, useful, and altogether very nice piece of software. (As a software developer, I should know.) I've been using EventGhost for a few weeks now and so far, it's been working perfectly for me (using several IR remotes, USB-UIRT, RF communication on the hardware side as well as a bunch of applications on the software side).

Having said that, I did find a need to extend EG for some custom functionality. That's why I wrote the TextGrab plugin. As I find it useful, I think somebody else may find it useful too, and that's why I'm posting the plugin here.

So, what does the TextGrab plugin do? As you might have guessed, it grabs some text, to be further processed (displayed or read aloud or written somewhere) later. You might ask, what good is THAT for? Well... it can be pretty useful. For example, I use it to grab the currently playing song title from my favourite media player (MPC) and then having it read aloud through the speech plugin. If I'm listening to the playlist of mp3's, without the monitor/TV turned on, laying on the couch reading something, and an interesting song plays, I can press a button on my remote, the EventGhost grabs the current song title from the Media Player Classic, and then it reads it aloud for me to hear it. No visuals are involved in any of this (although they can be if you wish).

How does TextGrab plugin work? It's basically a programmatic glue between two other plugins/actions: the FindWindow action and some other arbitrary action, like Speech or Show OSD. Actually, TextGrab leverages FindWindow functionality heavily, and is basically only an extension of this one. FindWindow is a very powerful action.. it does not only find a certain window, it can actually browse the whole graphic subtree of objects below a certain application window as well. This is awesome - and this provides the real power behind the TextGrab plugin. Namely, using the FindWindow action you can target not just a whole window, but a certain component of a window - for example, a label (static text). Do you see where I'm getting at? When you successfully target a label, all you have to do is programmatically read (grab) its text - and that's exactly what TextGrab plugin does!

Installation guide and a configuration example
--------------------------------------------------------------
First, download the __init__.py file atached to this message, which contains the plugin. Then, create a new directory below your EventGhost\plugins directory called TextGrab, and copy the downloaded __init__.py in there. Restart EventGhost and activate the plugin via Add Plugin toolbar button (search for the TextGrab plugin in the 'Other' plugins section). This completes the installation part, now for the configuration...

Now, create/add a new macro via the Add Macro toolbar button and name it whatever you wish it to (e.g., "display currently playing song title"). Then, you will need to add 3 actions to the newly created macro:

1) add a FindWindow action (it's inside the 'Window' plugin) and configure it appropriately. This is the trickiest part of the configuration so let me spend a little more time on that.

FindWindow action is a very configurable and powerful action. In this case, we're hunting not necessarily for a window, but for some other graphic object that is displaying some text (which we are about to grab). In this example, we'll be hunting for a label (static text) that displays the current song title inside the Media Player Classic, but you can use any other application that displays any text in a similar manner. Let's assume the MPC is running and the configuration dialog of FindWindow is showing MPC among the running application. Now we click on the plus sign to expand the object tree below that application and we can see the something like this:

Image


After we expand the whole tree of graphic objects, the so-called "Static" objects (labels) appear in the tree bearing their current text values. Now click on the "static" (label) element to select it. All the checkboxes below will be automatically selected with appropriate values. They carry the full criteria for finding the selected graphic element, however, you may want to change them to be more flexible. In our case, we must uncheck the "Window name" and "Child name", as those two are different every time a new song plays (they may carry the song title), and we want fixed criteria for finding the appropriate label element. Lastly, we must enter the "n'th element" number. Looking at the fully expanded tree of graphic objects, we can see that the searched-for graphic element is the fifth in a row of all so-called "Static" elements (labels). Such is the case on my computer, with my configuration of MPC, but your situation may be different, depending on how the MPC is set up (what information it displays). In any case, just count the number of previous "Static"-s in the object tree and add one, then put the number in "only return n'th match" entry field. Phew...

What we have just done is configure the FindWindow action in such a way that it searches for (and hopefully finds) exact graphic element inside the Media Player Classic that is displaying the text we wish to grab. If the application we would be trying to read from was something else, we would need to configure the FindWindow action differently, although the same principle still aplies: find the appropriate graphic element in a subtree of elements.

2) now the easy part: add a GrabText action (from the TextGrab plugin). Just add the action using the Add Action button from the toolbar, no need to configure anything here.

3) lastly, the fun part: add an action that actually does something with your grabbed text. Good candidates are either the Show OSD action or Speech action. Let's have a look at the Speech action example. Add the Speech action (speech plugin must be activated before you do this) and the configuration window will open. Now, insert the text you wish to be read aloud, including the {eg.globals.grabbedText} , curly brackets included, which will be replaced by the grabbed text. For example, type in "now playing {eg.globals.grabbedText}", without the quotes of course. Here's an example screenshot:

Image

Alternatively, if you wish to SEE the song title instead of HEAR it, you may wish to use Show OSD action instead of Speech. In the config dialog of Show OSD, "Text to display" should contain {eg. globals.grabbedText} somewhere.

And that's basically it for the TextGrab plugin. The only thing left to do is to add a trigger event to the top of the macro we just created, something like a remote control button press that will trigger it. Now, the next time you listen to your internet radio station and an interesting song comes up, just press that button and magically you'll be informed what song is playing (assuming your player is displaying that information somewhere). :)

Here's a screenshot of what the final macro should look like:

Image

Here's the configuration code of the aforementioned example. Note that you may need to configure the FindWindow action differently in order to suit your player's configuration:

Code: Select all

<?xml version="1.0" encoding="UTF-8" ?>
<EventGhost Version="1194">
    <Macro Name="read currently playing song title in MPC" Expanded="True">
        <Event Name="USB_UIRT.Mce.Details" />
        <Action>
            Window.FindWindow(u'mplayerc.exe', None, u'MediaPlayerClassicW', None, u'Static', 5, False, 0.0, 0)
        </Action>
        <Action>
            TextGrabPlugin.GrabText()
        </Action>
        <Action>
            Speech.TextToSpeech(u'ATT DTNV 1.4 Crystal', 0, u'now playing {eg.globals.grabbedText}', 0, 100)
        </Action>
    </Macro>
</EventGhost>
p.s. I wanted to post this message inside the plugins section of the forum, but I'm not allowed to post there. Some admin should move it there if he seems it appropriate.
Attachments
__init__.py
The TextGrab plugin python file. Place it inside the EventGhost\plugins\TextGrab directory.
(1.49 KiB) Downloaded 366 times
User avatar
Pako
Plugin Developer
Posts: 2294
Joined: Sat Nov 11, 2006 1:31 pm
Location: Czech Republic
Contact:

Re: TextGrab plugin

Post by Pako »

I think that the idea is not bad. But not the way it is made now.
I do not know how they say in English, but in the Czech language they say "take a gun (cannon) to a sparrow".
Plugin, which contains only one line has no meaning. He can be completely replaced by actions "Python Command".
GrabMPCtitle.gif
GrabMPCtitle.gif (4.33 KiB) Viewed 7792 times
Plugin will make sense when will be able to read the texts more difficult to reach - for example, status-bar.
If you want to add it, look here.
Pako
Samcek
Posts: 14
Joined: Sat Oct 03, 2009 4:52 pm

Re: TextGrab plugin

Post by Samcek »

Hello Pako,

Yes I agree completely with your point - a single-line plugin is a bit funny, if it can be as easily achieved with a line of phyton script. But (and this is not insignificant) for many people it is much easier to use a plugin, however small, than to write a single line of phyton code. Not everybody is a programmer, you know. :)

Having said that, what I would really like to be done (if the author agrees) is to simply add this one simple line of code to the FindWindow plugin, so there is no need anymore to have a separate plugin for grabbing text anyway. This could be done by adding a new checkbox titled something like "export the text content of the found graphic element" to the configuration dialog of the FindWindow action. If you want to push it a little further, you could also add a textfield that takes the name of the variable you want to store the text to (no need to be a predefined one, e.g. eg.globals.grabbedText).
User avatar
Pako
Plugin Developer
Posts: 2294
Joined: Sat Nov 11, 2006 1:31 pm
Location: Czech Republic
Contact:

Re: TextGrab plugin

Post by Pako »

Samcek wrote:Yes I agree completely with your point - a single-line plugin is a bit funny, if it can be as easily achieved with a line of phyton script. But (and this is not insignificant) for many people it is much easier to use a plugin, however small, than to write a single line of phyton code. Not everybody is a programmer, you know. :)
Yes, this argument has its weight. Also, I applied it when I created a plugin, which then was named "File Operations". Author of the project EventGhost - Bitmonster - also claimed:
Bitmonster wrote:I think most people who knew how to use the eg.result variable and like to write some data to a file, will in most cases have no problem to write a simple script like:

Code: Select all

outfile = open("C:\\myfile.txt", "w")
outfile.write(str(eg.result) + "\n")
outfile.close()
or something like that.
However, this case (TextGrab) is really so simple that plugin is not needed. One possible solution is to create a collection of snippets of code and place it on the Wiki.
It would be very useful if someone searched the entire forum and a collection created.
I would do it, but for me it is very difficult to write an English text and in doing so it would be necessary.
Samcek wrote:Having said that, what I would really like to be done (if the author agrees) is to simply add this one simple line of code to the FindWindow plugin, so there is no need anymore to have a separate plugin for grabbing text anyway. This could be done by adding a new checkbox titled something like "export the text content of the found graphic element" to the configuration dialog of the FindWindow action. If you want to push it a little further, you could also add a textfield that takes the name of the variable you want to store the text to (no need to be a predefined one, e.g. eg.globals.grabbedText).
Yes, in this way would also have to do. But we have to wait and see, will tell Bitmonster.
Apparently, he is just away for a while and does not contribute to the forum. That happens sometimes, but so far have always returned.
Otherwise I would, however, chose a different (more expected) solutions.
I would in this case the text save to the variable eg.result (instead of list windows handles).
Pako
User avatar
jitterjames
Experienced User
Posts: 677
Joined: Thu Aug 13, 2009 4:36 pm
Location: Quebec, Canada
Contact:

Re: TextGrab plugin

Post by jitterjames »

Either way I am happy to be exposed to the concept and to see the code in case I want to adapt it. It's also nice to know how simple it would be to do my own file writing without having to use the file plugin! You learn something new (almost) every day.

In any case I think the idea has merit and vote to have its functionality added to "find window" - or at least it should be considered.

I do some coding and have been writing some python scripts but had no idea that this would be so easy to achieve with 1 line of code.

btw Pako: your expression "use a canon to kill a sparrow" translates very well. I'm sure there is a similar saying in English, though I don't know what it is.
User avatar
Pako
Plugin Developer
Posts: 2294
Joined: Sat Nov 11, 2006 1:31 pm
Location: Czech Republic
Contact:

Re: TextGrab plugin

Post by Pako »

I think I found the best method of application of your plugin:
GrabText.gif
But two things still apply, which I have already stated:
1) Plugin (action) should be able to get the text from the status-bar
2) We must wait for Bitmonster
Pako
Samcek
Posts: 14
Joined: Sat Oct 03, 2009 4:52 pm

Re: TextGrab plugin

Post by Samcek »

Excellent idea, Pako. I think this is really the best solution, if Bitmonster agrees. The whole functionality is then encapsulated as a separate action, no need to burden FindWindow action with that, and no need to add another plugin just for this one-liner. Great!

Speaking of proverbs, in Slovenian language we would describe such solution as "the wolf is not hungry and the goat is still whole", but it sounds much better in original than in english, hehe.

Regarding your request about adding the get-statusbar-text functionality: yes I would be prepared to add it to this new action, however I need to learn how to create the configuration dialog for such action first. I'm a java developer and have no previous experience in python, I need to look into this. I'll do it, it'll just take me a day to get acquainted with the GUI development in python.

In the meanwhile, you could help me with an explanation or two. I need to understand more thoroughly what your piece of code (GetStatusBarText) does... is it only a method of getting the text from the status bar objects or is it a more general method of accessing data inside any (or just gui?) Windows API object? I'm not familiar with Win32 api and have no idea how SendMessage mechanism (which you are using in this case) works. I'm sure I could look it up, but you would save me some time if you just explain it to me.
User avatar
Livin
Experienced User
Posts: 792
Joined: Wed Oct 08, 2008 4:56 am

Re: TextGrab plugin

Post by Livin »

This is a really nice idea... I can see a lot of use for it when making decisions on what action to trigger... the text can be used as a 'state' of an application to be used in a decision tree.

e.g.
If PAUSE in WindowXYZ then trigger ActionABC

very cool
setup... XBMC, W7MC for DVR & Live OTA TV, JRMC for multi-zone audio, EG, MiCasaVerde Vera3, USB-UIRT IR receiver, Harmony remote, 5.2 home theater system
User avatar
zian
Site Admin
Posts: 569
Joined: Wed Jun 24, 2009 3:54 pm

Re: TextGrab plugin

Post by zian »

I think I would like to try that function that Pako posted in the above image.

But until then this plug-in has come in handy for me.
Now my issue is that in the MPC "find window" it is a different window depending on what type of file I'm playing. For mp4 files it would be the "only find 17th match" option. For mkv files there is no Title text window (that I can find).
Pako wrote: If you want to add it, look here.
Pako
I did try this for the status bar text grab but couldn't get that either. I think I may have messed up with the code names/files/folders etc.. Any body else use this nifty action (the one Pako linked to...Status Bar Text Grab?)
eventghost.net
Be there or be square.
User avatar
Pako
Plugin Developer
Posts: 2294
Joined: Sat Nov 11, 2006 1:31 pm
Location: Czech Republic
Contact:

Re: TextGrab plugin

Post by Pako »

@Zian
I'm very sorry, but I did not understand your question.
I am an older man and learning foreign languages, I'm not very good.
When I was still in school, we had Russian as a foreign language.
Today I have quite often understand simple English text without the dictionary.
But sometimes happens that I am not understand a text even with the dictionary.
When I write English text, I helping with Google Translator.
When I create an English text, I always re-translate him into Czech.
Thus I check, whether my English text makes sense.
If this is wrong, I change the English text. Then again translate into Czech.
Thus I proceed to as long as the resulting Czech text not give the correct sense.
If you want me to I can well understand, you must write simple sentences, if possible.

@Samcek
For the above reasons that I am not able to make some explanation.
I can get the job done for much less time than I should explain something.
I can only recommend something to study:
Manning Publication wxPython in Action (pdf file) and win32-docs-demos.

Pako
User avatar
Pako
Plugin Developer
Posts: 2294
Joined: Sat Nov 11, 2006 1:31 pm
Location: Czech Republic
Contact:

Re: TextGrab plugin

Post by Pako »

I can not afford to change plugin "Windows" and so meantime I have made enhancement to plugin "TextGrab".
I do not know if that follows will actually work reliably.
It is also possible that it will be necessary to make some special arrangements for some other classes
(not just for TStatusBar a msctls_statusbar32).
Pako
Attachments
__init__.py
(4.6 KiB) Downloaded 270 times
User avatar
zian
Site Admin
Posts: 569
Joined: Wed Jun 24, 2009 3:54 pm

Re: TextGrab plugin

Post by zian »

Pako,
No worries friend.
Sometimes my English is like a foreign language also.
I thought my words were simple but I'll try to make them easier to read.

I'm going to try that new file you have just posted.
Thanks again.
eventghost.net
Be there or be square.
shady
Posts: 3
Joined: Sun Jan 18, 2009 8:49 pm

Re: TextGrab plugin

Post by shady »

*** Deleted ***
Wrong topic, sorry! :oops:
Post Reply