Stops receiving UDP once per week - restart fixes after hang

Found a bug in EventGhost? Report it here.

Stops receiving UDP once per week - restart fixes after hang

Postby davidmark » Tue Apr 12, 2016 12:44 am

Been using this thing for a couple of years now and for the first year or so I never had to restart EventGhost.

I use the VoxCommando application to send various UDP packets on voice commands, X10, etc. About six months ago, I noticed that EventGhost wasn't responding to (or even receiving per the console log) the packets. Restarting EventGhost always snaps it out of it, so don't think I can blame VoxCommando for this one.

Only other clue is that EventGhost always hangs now (i.e. have to use Task Manager to stop it). That happens whether it is fouling up or not. Always prompts me to save my "changes" as well, despite not changing anything for months.

Happens almost like clockwork once per week. Haven't bothered trying to diagnose it further, but do recall that UDP packets were received by NodeJS at a time when EventGhost was ignoring them. Haven't tried sending UDP from other apps, but doubt that will change anything as the fix is to restart EventGhost.

All UDP settings on both sides are defaults. All other EventGhost functions work as usual; just UDP that vanishes (and the mysterious hang on closing). Only thing I can think of that I've touched in last six months to a year is I turned off the SunTracker weather poll as Yahoo! Weather was fouling up. Never turned that back on and it seems like that's right about when the trouble with the hanging started. Perhaps I'll try disabling that module altogether, but seems like a red herring. More likely something else has gotten fouled up in the EventGhost data. What would make it think it always has changes to save?

Anybody seen any of this before? I suppose I should check to see if there is a new version of EventGhost and perhaps reboot the PC (been at least six months since I've done that).
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby piert » Tue Apr 12, 2016 10:18 am

I assume you are sending UDP packets from other equipment to the PC running Eventghost.
Just to exclude some possible issues:
-double check the IP addresses of the connected equipment and the settings in the plugin in Eventghost. Could be that IP addresses were changed by your router.
-I have had hangs of Eventghost when the IP routing in my network got messed up for whatever reason (for instance after an electrical blackout; at resume the router can give other IP addresses to the network equipment).
-Even if settings appear correct, a reboot of PC's, router and other equipment in your network in the right order may be worth the effort. Again, just to exclude simple causes.
piert
Experienced User
 
Posts: 248
Joined: Tue Jun 14, 2011 2:53 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby davidmark » Wed Apr 13, 2016 6:39 am

piert wrote:I assume you are sending UDP packets from other equipment to the PC running Eventghost.
Just to exclude some possible issues:
-double check the IP addresses of the connected equipment and the settings in the plugin in Eventghost. Could be that IP addresses were changed by your router.
-I have had hangs of Eventghost when the IP routing in my network got messed up for whatever reason (for instance after an electrical blackout; at resume the router can give other IP addresses to the network equipment).
-Even if settings appear correct, a reboot of PC's, router and other equipment in your network in the right order may be worth the effort. Again, just to exclude simple causes.


No, EG and VC are on same box and the IP address for that box, though dynamically allocated, hasn't changed in a year and a half or so. If it ever does change, I'll have to change it in a few places to keep everything working; definitely hasn't happened recently.

By "hang" I meant it could not be closed. It continues to work (minus UDP) and hangs around in memory after I exit, requiring Task Manager to shut it down completely.

Not holding out much hope on a reboot, but will give it a try one of these days. Thanks!
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby kgschlosser » Wed Apr 13, 2016 8:59 am

what plugin are you using??

i had a similar problem with a plugin i made just didn't like how it was situated with the stop and close definitions.


i don't remember exactly how i sorted it but it wasn't a problem really.


now the other problem with it hanging after a week or so.

i have had this happen to me with another plugin and it was almost like clockwork

if the plugin you are using is using the asyncore for threading, there is some kind of a problem with it or memory leak or something.

the plugin then needs to be converted to using threading instead of the asyncore.
If you like the work I have been doing then feel free to Image
User avatar
kgschlosser
Site Admin
 
Posts: 2128
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Stops receiving UDP once per week - restart fixes after

Postby kgschlosser » Wed Apr 13, 2016 9:08 am

i think i found the plugin you are using.

broadcaster

and if it is broadcaster it's asyncore i looked,

i can convert it for ya but you will have to remember to not copy over it or maybe i should rename it so that if you upgrade it won't get overwritten

it is a core plugin.


and there is no close definition in the plugin either

that would be why it hangs

so i am going to go ahead and just update the thing, and maybe it will replace the exiting one in the cores

K
If you like the work I have been doing then feel free to Image
User avatar
kgschlosser
Site Admin
 
Posts: 2128
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Stops receiving UDP once per week - restart fixes after

Postby davidmark » Wed Apr 13, 2016 10:05 am

Very interesting. I believe it is the Broadcaster plugin that I use for UDP. Will confirm...

Maybe I just got lucky or was restarting EG more often for other reasons during the first year or so. Perhaps I wasn't paying close enough attention at the time as this failure only breaks a couple of minor features (and the VC voice recognition that I virtually never use).
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby davidmark » Wed Apr 13, 2016 6:09 pm

It is the Broadcaster add-on and it also appears to have problems with Unicode characters. See sporadic error messages regarding news headlines with fancy quotes and such.
Last edited by davidmark on Thu Apr 14, 2016 8:58 pm, edited 1 time in total.
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby davidmark » Thu Apr 14, 2016 2:58 am

So anybody know of a Python function to strip out all but the ASCII character set (or convert as appropriate)? That's actually what I need for this as these notifications go to an old alpha sign.

Maybe if the thing didn't throw an exception every five minutes (have a lot of notifications here), it wouldn't cause EG to hang or lose the "connection" to VC. Maddening that I could be having UDP issues on the same box, not to mention Unicode.
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby davidmark » Thu Apr 14, 2016 3:57 am

kgschlosser wrote:i think i found the plugin you are using.

broadcaster

and if it is broadcaster it's asyncore i looked,

i can convert it for ya but you will have to remember to not copy over it or maybe i should rename it so that if you upgrade it won't get overwritten

it is a core plugin.


and there is no close definition in the plugin either

that would be why it hangs

so i am going to go ahead and just update the thing, and maybe it will replace the exiting one in the cores

K



Would be interested in testing a new version (of any name). Please look at what it is doing with Unicode characters as well. Thanks!
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby davidmark » Thu Apr 14, 2016 9:07 pm

Have analyzed the data and it appears that the UDP problem started (or became more pronounced) when the number of notifications (and log messages) increased significantly a month or two ago. Don't ever recall having to restart EventGhost before that. Now it is usually once per week, which is a deal-breaker for an automation application.

So routed the log messages elsewhere to relieve some of the burden on EventGhost/Broadcaster. Of course, it's the notifications that come from outside that have the problematic characters in them. Suspect that every time it throws an exception it leaks.

Let me know when the new version is ready. Thanks!
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby davidmark » Fri Apr 15, 2016 10:42 am

No Python expert and haven't confirmed, but this would appear to be the root of the Unicode issues:

Code: Select all
UDPSock.sendto(eg.ParseString(eventString)+self.plugin.payDelim+eg.ParseString(payload),addr)


Always blows up with the message that - sendTo - takes three arguments, but only two were provided. Can see the escaped characters in the payload (displayed prefixed by backward slashes in the left panel) and apparently eg.ParseString can't handle them. Suppose I can substitute for these problematic characters with regular expressions, but have to figure there's an existing solution out there somewhere.

Don't need anything but ASCII for the signs and can just send the log emails from somewhere else; can't stand VoxCommando for that anyway. It's like a minefield when sending rapid-fire emails. Of course there's no documentation for the proper way to trigger emails via UDP, but do it "wrong" (e.g. two sends in same event) and see that it sends two copies of one message and loses the other. :(

Suppose that's another question: is there a reliable IMAP add-on for EventGhost? Couldn't be too involved to write one, but never looked into it.

Also should mention that EventGhost seems to have no problems displaying these strings on the screen (haven't looked into how). It's just when it tries to use this - ParseString - method that things go awry:

Code: Select all
def ParseString(text, filterFunc=None):
    start = 0
    chunks = []
    last = len(text) - 1
    while 1:
        pos = text.find('{', start)
        if pos < 0:
            break
        if pos == last:
            break
        chunks.append(text[start:pos])
        if text[pos+1] == '{':
            chunks.append('{')
            start = pos + 2
        else:
            start = pos + 1
            end = text.find('}', start)
            if end == -1:
                raise SyntaxError("unmatched bracket")
            word = text[start:end]
            res = None
            if filterFunc:
                res = filterFunc(word)
            if res is None:
                res = eval(word, {}, eg.globals.__dict__)
            chunks.append(unicode(res))
            start = end + 1
    chunks.append(text[start:])
    return "".join(chunks)


The error displayed makes no sense to me as it appears that sendTo will get two arguments regardless. Again, not a Python expert.
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby davidmark » Fri Apr 15, 2016 11:48 am

Had to reverse engineer this as, for whatever reason, found every article about this Python subject to be useless:

Code: Select all
message = message.replace(u"\xe2\x80\x98", u"'");
message = message.replace(u"\xe2\x80\x99", u"'");
message = message.replace(u"\xe2\x80\x9c", u"\"");
message = message.replace(u"\xe2\x80\x9d", u"\"");


...but then I am no Python expert. Can't stand it actually and that's one reason I've been moving away from EventGhost to NodeJS; of course, Node can't deal with serial devices as easily, but have it doing most of the other chores at this point (and know it can send emails without issues, unlike VC).

Can now see the cleaned up messages on the signs and screen (was incorrect when I said it was displaying these on the screen before).

That does single and double quotes anyway. Obviously, there are lots more to deal with. Anybody know of an existing script?

Will post back if the UDP module continues to go AWOL each week. Can live with the hangs on closing as I don't close this application very often (unless forced to). Come to think of it, there should be a way to restart this thing without closing it (know VC has such a command, as did MisterHouse).
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby kgschlosser » Sat Apr 16, 2016 4:00 am

I have just found that for some reason the asyncore locks up and as far as the unicode problem i would have to see the exact error to track it down.

I am going to work on that UDP Broadcaster for ya to see if we can get it to stop locking up on ya and fix the close issue.

if you are passing a variable directly into a dialog with the {eg.event.payload} kinda deal I handle that a little differently than using the parser, i have used it tho but handled any exceptions it may throw out.

but normally i do an if on the string coming in to see if it's an eg variable

example:

Code: Select all
testStr = "{eg.event.payload}"

if testStr[:4] == "{eg.":
    try:
        testStr = eval(testStr[1:-1])
    except:
        pass


i do that pretty much to look for a variable.
i have thought about going a little more safe with it and looking into the globals() to see if the variable is in it. but i figured if someone was going to run malicious code on your computer it would have to be you.

LOL

so i never bothered with it
i think i may have read somewhere that unicode causes EG a problem if it's getting printed to the log. not sure on that one. I have never tested it and i don't ever recall using unicode for anything anyways.

well i am going to tear apart that UDP plugin and lets see what i come up with.

K
If you like the work I have been doing then feel free to Image
User avatar
kgschlosser
Site Admin
 
Posts: 2128
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Stops receiving UDP once per week - restart fixes after

Postby davidmark » Sat Apr 16, 2016 5:03 am

kgschlosser wrote:I have just found that for some reason the asyncore locks up and as far as the unicode problem i would have to see the exact error to track it down.


See previous post of mine. I was able to patch my code to deal with the fancy quotes.

kgschlosser wrote:I am going to work on that UDP Broadcaster for ya to see if we can get it to stop locking up on ya and fix the close issue.


Thanks very much. Now that it doesn't throw exceptions every hour, it may take longer to go AWOL (if at all). Will post back if it happens again.

kgschlosser wrote:if you are passing a variable directly into a dialog with the {eg.event.payload} kinda deal I handle that a little differently than using the parser, i have used it tho but handled any exceptions it may throw out.


Not doing anything with dialogs. Just sending the messages received over HTTP to VC via UDP, as well as OSD and to the serial ports for the alphanumeric signs. None of those operations handled the fancy quotes properly; threw exceptions complaining about the Unicode characters.

Here's my theory on the UDP outages: I increased the number of messages considerably about a month or so back and many of these messages come from news services and some of them use fancy quotes in their headlines. That's when the exceptions started (or at least became far more frequent). Each exception is likely resulting in a failure to clean up behind the scenes and after enough of them it fails completely, which requires a restart (any way to do this without closing the app?) and that's when I started noticing it hanging (and always prompting to save nonexistent changes).

Just my theory. Thanks again!
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby kgschlosser » Sat Apr 16, 2016 5:31 am

ok your gonna be my guinea pig.

i don't have anything that i deal with in UDP Broadcasts.

so ya have to tell me if it works.

I do know the server starts and i get no compilation errors.

file is attached.

i changed all the registration information including a new guid as well as a new class name as not to interfere with what you have going on already

Thanks

K
Attachments
NewBroadcaster.zip
(6.64 KiB) Downloaded 48 times
If you like the work I have been doing then feel free to Image
User avatar
kgschlosser
Site Admin
 
Posts: 2128
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Next

Return to Bug Reports

Who is online

Users browsing this forum: No registered users and 3 guests