Stops receiving UDP once per week - restart fixes after hang

Found a bug in EventGhost? Report it here.

Re: Stops receiving UDP once per week - restart fixes after

Postby davidmark » Mon Jul 11, 2016 9:13 pm

Thank very much for the quick service. I'll try at least one of them this weekend and let you know how it goes. Will set it up to send an email if/when it fails again.

I'm sure the problem is with the sender, which I think would have to be VoxCommando. Can't think of anything else sending UDP on that port. Perhaps I did a bad paste on that end or something. Regardless, will be good to keep the plugin going in this event (and log the data). Recommend updating the repo the error handling solution.

Thanks again!
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby kgschlosser » Tue Jul 12, 2016 3:37 am

i will submit a fix to the team working on EG. and see if they add it.


i have a pile of them i will do them all at once. but in the fix i will just have it throw an error stating that the incoming data has to be in the ascii table and has to be between decimal 31 and 128 (Human Readable Non ANSI characters)

i do not know as to why EG can show them i understand control characters like EOL and NULL and things of that nature (below 32) but most of the ones above 127 are ANSI characters i would have to test to see if the EG log is capable of handling it.


let me know if this suits you. I will change it up a little to have it just pop a notice or a one line error. and have it do a full blown report to the EG debug log. that way there is a full description of who it came from what the whole message saw and what the hex value is for the bad character. like the one that is triggering your error decimal 192 and is this character. don't know it may help you track down the problem

here is the character


so if you happen to find out what is sending that character then you are good to go.
A loved one and Time, The 2 things that can never be replaced.

Family, The only thing you don't get to choose in life.
User avatar
kgschlosser
Site Admin
 
Posts: 1467
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Stops receiving UDP once per week - restart fixes after

Postby kgschlosser » Tue Jul 12, 2016 4:31 am

i cleaned it up some and also added it to log in the debug log if the -debug switch is used when running EG

this method will just inform you of some bad data. but all the rest of the information including the full traceback and the senders is and also the message that was send with the bad data converted to hex.

i have the plugin do a print error with a generic sort of message. but i also threw in the senders ip into that one as well.

this will require you to add an import at or about line 57

Code: Select all
import traceback


and here is a more updated version cleaned up and prints all the goodies to the debug log file.

Code: Select all
        if (not my_addr) or self.selfBroadcast:
            bits = data.split(str(self.payDelim))
            commandSize=len(bits)
            try:
                if commandSize==1:
                    self.plugin.TriggerEvent(bits[0])
                if commandSize==2:
                    self.plugin.TriggerEvent(bits[0],bits[1])
                if commandSize>2:
                    self.plugin.TriggerEvent(bits[0],bits[1:])
            except Exception:

                tmperr = 'suffix: ' + bits[0] + '\n'
                if commandSize == 2:
                    tmperr += 'payload: ' + bits[1] + '\n'
                if commandSize > 2:
                    tmperr += 'payload: ' + str(bits[1:]) + '\n'

                err = ''
                for char in list(tmperr):
                    if (ord(char) > 127 or ord(char) < 32) and ord(char) != 10:
                        err += '  **' + hex(ord(char)) + '**  '
                    else:
                        err += char

                err = str(traceback.format_exc()) + '\n' + err
                err += 'sender address: ' + str(addr[0]) + '\n'
                eg.PrintError('Broadcaster: Data received is outside of the printable range\nsender address: ' + str(addr[0]))
                eg.PrintDebugNotice(err)


again as before this compiles but i have not tested it. i will be adding this to the bugs section of the forum. and also passing it on to the powers that develop EG.

so if this works properly please let me know that way i can say it has been tested and it works properly

so if you wouldn't mind running EG with the -debug switch for a while and let it log the data. and if you get the error. you can either dig through the log or just send the whole log to me so i can make sure it came out properly

add the log as a file tho not a copy and paste. or if you prefer to e-mail, just send me a PM asking for my e-mail addy and we can go about it that way.

I am trying to put together all of the little things that i correct to pass them forward. and testing is a huge part of it and i don't always have the setup that caused the problem in the first place and to reproduce the problem would take some time to get everything in order to make a fake scenario replicate it. it's easier to have you do it if you don't mind at all.

the debug log file gets saved into the %appdata%\EventGhost folder.

now you may run into complications if you have stray files in your plugins directory if you do any tinkering. this logs all problems even in files that are not "loaded" into eg but reside in the EG directory.

you will be able to tell if there is a problem because it will cause EG to hang. and if it does kill the task and go and read the log. it will tell you what file is causing the problem. i had tons of code samples all over the place in my plugins directory. or botched code but i wanted to save for a part of it. took me a while to do a clean out to run the debug.

i am going to at some point look into fixing that portion of it so EG doesn't hang. it will log the problem and keep on going. but that is another project

Thanks Again for providing ample information to be able to trace the cause of the problem.

I know it took a while and restarting the plugin was just a band aide. and we now have the root of the issue and i believe it has been successfully fixed

and you were right on track with the problem being with the parsing of the data. tho it was being parsed in a different way it was still a right proper assessment :D

if you have any other glitches or issues I am always about or just PM me and i will do my best to help ya out. and if i don't know about it. i am a pretty motivated individual, and i will spend the time to learn it if i don't know anyone who might have more information and would be a better help.
A loved one and Time, The 2 things that can never be replaced.

Family, The only thing you don't get to choose in life.
User avatar
kgschlosser
Site Admin
 
Posts: 1467
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Stops receiving UDP once per week - restart fixes after

Postby davidmark » Thu Jul 14, 2016 8:15 pm

All sounds good. Glad that you are still working on this thing. Find it to be an excellent platform for home automation. Run indefinitely without interaction now; will implement your latest changes to the Broadcaster plugin this weekend and remove my workaround.

As to the inability of handling printable characters out of the ASCII range, I'm fine with that; however, I find that much of the Python ecosystem seems incapable of handling these characters. First noticed this when OSD attempts failed. On using various Python Email libraries, I see the same thing. Just throws an exception as soon as it hits an out-of-range character. Clearly there should be a way to send these characters in an email message. Poked around a bit in the docs and appears there are plenty of methods for dealing with different encoding methods, but don't care enough at the moment to delve into them.

Speaking of email (and I'll likely post another thread for this), I would like to be able to send messages asynchronously (e.g. in another thread). Currently relegated to running a second instance of EG to send email "log entries". System sends so many emails that it "hitches" constantly while GMail completes the transaction. Doesn't cause it to miss anything, but found it annoying enough to implement a proxy.

Am in process of encapsulating the core of my system as plugins. Will post when they are ready and will likely have more questions in the interim. Thanks again!
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby kgschlosser » Fri Jul 15, 2016 1:47 am

That is because python below version 3 is separated into str and unicode. once you get to 3+ it's 100% unicode and i believe it eliminates all of these issues. This is a major overhaul of EG to accomplish this and is currently being worked on. so in the future hopefully if everything goes well we will see a version of EG with python 3+
A loved one and Time, The 2 things that can never be replaced.

Family, The only thing you don't get to choose in life.
User avatar
kgschlosser
Site Admin
 
Posts: 1467
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Stops receiving UDP once per week - restart fixes after

Postby kgschlosser » Fri Jul 15, 2016 1:53 am

davidmark wrote:Speaking of email (and I'll likely post another thread for this), I would like to be able to send messages asynchronously (e.g. in another thread). Currently relegated to running a second instance of EG to send email "log entries". System sends so many emails that it "hitches" constantly while GMail completes the transaction. Doesn't cause it to miss anything, but found it annoying enough to implement a proxy.

Am in process of encapsulating the core of my system as plugins. Will post when they are ready and will likely have more questions in the interim. Thanks again!



Yup i find this to be a particular annoyance with EG. it is because there is one thread for Events processing. now this wouldn't normally be an issue but if something has to sit there and spin it's wheels for a little bit it stop eg from doing anything. Now unfortunately because of how EG was written each macro does not have the capability of running it's own thread. even when using the Python Scripts. this is one of those things i would like to make happen. automatically actually. all behind the scenes. with an option to turn it on or off per script. I have put some thought into how to do it but this is something that would have to be done in the core. it would be a real hassle to get it done by using a plugin. tho it can be done. I have already made a nice Python Script editor. well its got a couple bugs i am still trying to work out of it. but it acts more like an IDE then a simple text editor.

gotta start off somewhere. LOL.
A loved one and Time, The 2 things that can never be replaced.

Family, The only thing you don't get to choose in life.
User avatar
kgschlosser
Site Admin
 
Posts: 1467
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Stops receiving UDP once per week - restart fixes after

Postby davidmark » Fri Jul 15, 2016 1:05 pm

Yes, a plugin was what I had in mind. The VoxCommando thing can send async emails, but haven't looked into how open their code is (certainly it uses Python scripts). However, I originally used that thing as a proxy to send emails (via Broadcast) and it was a complete disaster. Sending multiple broadcasts per macro caused constant drops and dupes (e.g. send A, B, C and see something like A, A, C in the "log"). So their related code could only serve as an inspiration.

Thanks again for all the help. Will be trying out your Broadcaster fix this weekend. Will test by sending a non-ASCII character from VC.
davidmark
Experienced User
 
Posts: 85
Joined: Thu Jan 01, 2015 5:25 pm

Re: Stops receiving UDP once per week - restart fixes after

Postby kgschlosser » Fri Jul 15, 2016 6:25 pm

davidmark wrote:Yes, a plugin was what I had in mind. The VoxCommando thing can send async emails, but haven't looked into how open their code is (certainly it uses Python scripts). However, I originally used that thing as a proxy to send emails (via Broadcast) and it was a complete disaster. Sending multiple broadcasts per macro caused constant drops and dupes (e.g. send A, B, C and see something like A, A, C in the "log"). So their related code could only serve as an inspiration.

Thanks again for all the help. Will be trying out your Broadcaster fix this weekend. Will test by sending a non-ASCII character from VC.


ok cool. i also modified the Python Script plugin for EG. it now allows for Threading. it really wasn't that difficult for me to do. i just have to polish it up alittle more and make sure it is pretty much bug free.
A loved one and Time, The 2 things that can never be replaced.

Family, The only thing you don't get to choose in life.
User avatar
kgschlosser
Site Admin
 
Posts: 1467
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Previous

Return to Bug Reports

Who is online

Users browsing this forum: No registered users and 2 guests