EG and Stackless

[kgschlosser]

I have been running some tests with stackless python and i would like to confirm something,

as stackless is a dying flavor of python. I wanted to see how difficult it would be to remove stackless from EG.

and it really was not that difficult. and something unexpected also happened

and i just want to confirm it is not a computer issue.

here is the setup. and what is happening.

I have 2 computers.

#1 8 core 3.2 ghz, 64 gigs ram, 4 SSD raid 0 and each ssd is rated at up to 550mbs read and 520mbs write and these drives are controlled by a dedicated raid controller with an intel iop 1.2ghz processor with 512mb memory

#2 6 core 2.8ghz 8gb ram and a single ssd rated at 480mbs read and 430mbs write


i have a version of EG with stackless removed and threading has been used to replace it. that version of EG is running on machine #2

starting eg with an empty tree computer #1 has an xml load time and plugin scan time of

11:45:21: 0 ActionThread: XML loaded in 0.215043 seconds.
11:45:20: 0 MainThread: PluginManager.ScanAllPlugins :0.19594453014844029

and on computer #2 without stackless xml and plugin load time of

12:23:49: 7052 ActionThread: XML loaded in 0.002979 seconds.
12:23:48: 8568 MainThread: PluginManager.ScanAllPlugins :0.21741138411702612

now this is pretty definitive on my end.

but i would like to know what others have for load times.
this can be done by turning on debugging. and reading the log file

i am going to run more tests to see. I am going to time total EG load time.

but the slower computer is far faster at file access without stackless thus far.

I have run benchmarks to check hdd performance. and these are the 2 computers. and i have pretty much narrowed it down to stackless

the first image in each set is a 1gig test and the second is a 50mb test and the first column in each is the read tests. and the second is write tests.

as far as the write tests go they are very close in numbers. but there is one difference. computer 1 has write caching turned off and computer 2 has it turned on.
this is done because before i had a battery backup i didn't want to have any data corruption on computer 1 if i lost power. this can happen because of the raid. i honestly forgot up until now that it is in fact turned off. so the numbers being the same indicates that computer 1 is faster at writes than computer 2 because basically computer 2 is intentionally handicapped and the numbers are almost the same

computer 1

computer 2

[kgschlosser]

well i used one of my virtual computers and i installed Version: 0.4.1.r1722

ok i timed the whole EG start process with and without stackless. and the numbers are almost identical after running the thing 10 times.

there has to be something with windows and file access that is causing the file load time differences. after running it a bunch of times they flatten out to almost the same.


so running stackless seems to make no performance impact what's so ever.

[Sem;colon]

Ah, good to know, thank you kgschlosser!

[jprovan]

I am not so sure that "stackless is a dying flavor of python". Looking at https://bitbucket.org/stackless-dev/sta ... ommits/all, there have been commits as recent as 9/10/2016.

[kgschlosser]

I know it is still being updated. but they really don't have a website. and trying to find information about use and examples if challenging to say the least. and the developer of stackless has shifted his focus to other things and just does minor tweaks and changes. nothing big. this is the information i have gotten from the internet. so take it as that.


I have a rather complex tree and it would be nice if EG was designed to spawn threads for things like running a macro. that way EG could go about it's business and not hold thing up if you are processing an event.

example. i have a central server that runs EG. and if i wanted to set a polling loop for when something turns on it binds up EG and no other event can come in. unless i implement threading in each and every script. which is not so much a problem. but for the folks that do not know how to program in python it would be nice if it was just that way

[krambriw]

I think that when EG once was designed, the idea was that plugins should be the actual abstraction layer between normal users & developers. And in plugins you would definitely use threading for such tasks as you describe. The evolution has since then moved forward, more and more users are exploring the possibilities with python scripting (Kids of today are learning Python programming using RPi's). I think that your idea having an option to run a macro in separate thread is very good

[Sem;colon]

I used to use threading in scripts as well, but meanwhile I moved everything that used threading into plugins.
Plugins are great!
I don't think EG was intended to have too much intelligence in the .xml part, after all it's more a GUI tool than a scripting tool.
Having scripts running in different treads may confuse unexperienced users, as you cannot "see" them.
It's still a very nice idea for prototyping and I like it.

[kgschlosser]

oh yeah it would be the way to go especially for developing. I should send ya over my Python Script Editor. it allows for super simple threading in scripts. with just the click of a button. and also allows for listing all scripts that are made, which ones are running, and which ones have threading enabled. and allows for stopping the script if it is running.

I find it very useful. I have stopped adding things to it because I got tied up with playing around with the core EG GUI. the agw AUI is awesome. but i have to do some major recoding of how the dialogs are done in order to get them to be dockable which i do like. i can get them to show but with the dialogs coded the way they are it causes EG to hang. i was trying to add a selection to be able to select "Old Style Dialogs" or new ones and have it point to a different set of dialog files. but it's proving to be very challenging. because by nature a modal dialog causes wxapp to bind the GUI and wait for a response and you cannot add a dialog to an AUI Manager. so what happens is if i change up the dialog to say a scrolled panel it no longer binds the GUI to wait for a response and this causes the tasklet to continue running with no answer provided and EG it set up so that if a None is returned it cancels the action. i am almost to a workaround but the @AssertInActionThread and @AssertInMainThread when debugging is turned on raises an exception. which causes a problem. I am working on a way to add a wrapper to actually insert the called function into the proper thread like the actionThread.Func but for some reason that is actually what is causing most of my problems. and I am not sure as to why. i keep on getting further along. but what i have had to do is to pass the function to be called after the dialog closes to the actual dialog so it can execute it instead of it waiting. the hang i believe is due to the fact that the dialog is no longer a dialog and doesn't bind EG the proper way.

I am not sure as to why EG was coded in a way that raises the exception on the Assert methods only when in debug. instead of just printing a message

I just looked at the code for the Assert methods yesterday and noticed that. and i have not tested it with just printing the message instead of raising the exception. that may fix the issues i have been having. but I think i am going to have to go back to scratch and make the changes again one at a time.

I am going to get rid of the option to use the old style dialogs because its quite difficult to be able to have both sets of code in place because both require different means of functioning

[krambriw]

Having scripts running in different treads may confuse unexperienced users, as you cannot "see" them.

Once this is introduced, I think EG needs some means (or view) to allow the user to see a list & being able terminate threads. So EG needs to keep track of the threads it has started

[kgschlosser]

Having scripts running in different treads may confuse unexperienced users, as you cannot "see" them.

Once this is introduced, I think EG needs some means (or view) to allow the user to see a list & being able terminate threads. So EG needs to keep track of the threads it has started

without a doubt. that is one of the reasons why when i made my script editor i added the threading as a simple button press for the user and a couple of functions that could be used in the script to manage the thread easily. and also be able to terminate a looping thread easily from anywhere inside of EG

for instance. on a looping thread they would do

Code: Select all

print "starting script"
i = 0
while RunScript():
    i += 1
    if i == 1000:
        print i
    if i == 10000:
        ExitScript()

print "exiting script"

there is also
Wait()
Clear()
IsRunning()

the cool thing is that these can be called from anywhere even outside the script. so if there is a script that is looping and you want it to exit. a call can be made to script.SOMESCRIPTNAME.ExitScript()

and if it's a thread loop that call will exit the loop. and if called a second time will terminate the script

and because of how i set the thing up if you wanted to pass keyword arguments to a script this can be done by script.SOMESCRIPTNAME(keyword1=None, keyword2='Test')

and it will create 2 variables that you can access from inside of the script called keyword1 and keyword2


but because all of this is managed by the script editor it makes it a clean close when you exit EG because a call to script.ExitAll() will close any loops that are in place

and the fun doesn't stop there.

[Sem;colon]

Awesome! You should test it for a while and if you think it's working properly, post the modified files here so other people can test it as well.

[kgschlosser]

Awesome! You should test it for a while and if you think it's working properly, post the modified files here so other people can test it as well.

I have been using it. and there are a couple of glitches but it seems to run good.

I have a plugin version of it. that will allow both the original script editor and the new one to run at the same time.

and it has an option to override the stock one. but users be warned, once you do the override you cannot go back.

give me a day or 2 to make sure it's up to snuff. i know i do want to rework the whole thing because it's very much so a proof of concept and needs a heavy overhaul. the code is sloppy and has been patched a bunch of times to fix things.

and with time and learning i can shrink the code while making it better. and the reason i have not released it is because I know i am going to change the api about and i didn't want someone to upgrade it and end up in a real debockle.

if you send me a PM i will gladly e-mail it to you. i think that is a better way so users don't have problems.

[krambriw]

i think that is a better way so users don't have problems.

I think it is better you finish & clean it up and then just publish it. But better fix your house first before snow comes as you wrote in another thread

[kgschlosser]

I think it is better you finish & clean it up and then just publish it. But better fix your house first before snow comes as you wrote in another thread

yeah i know. but that is why i haven't published it yet. but i would like some input on how it functions as well. so i am kind of stuck between a rock and a hard place with the plugin.

and trust you me i am working on the house. i just got the electric back up to all of the house. a part of it was down for a while while i gutted all of the wiring out moved electrical sub panels. changed outlet locations. added more lights and switches.


but everything is back up and functioning once again.

i got the house sealed up a couple of days ago so no water can get in. have it all insulated. i put up a couple more sections of the exterior sheathing yesterday. i have to close up the roof section over my office as soon as i replace a section of the sill plate of the main house. that is today's project

[WoLpH]

When Stackless Python was first introduced it was definitely a very useful method of reducing the overhead of threading on Windows systems which made it a very good choice for EventGhost.

These days however... with asynchronous I/O fully supported by the Python core I would argue it's not the best solution anymore. Additionally, Stackless is dying somewhat... the latest version of Stackless for Windows is running on Python 3.3 which is a 4 year old Python release.

Do we have any compelling reasons to keep stackless?

I would personally argue that as soon as we make the switch to Python 3 we drop stackless as well. EventGhost would function amazingly well on a asyncio based system: https://docs.python.org/3/library/async ... coroutines

Code: Select all

import asyncio

async def compute(x, y):
    print("Compute %s + %s ..." % (x, y))
    await asyncio.sleep(1.0)
    return x + y

async def print_sum(x, y):
    result = await compute(x, y)
    print("%s + %s = %s" % (x, y, result))

loop = asyncio.get_event_loop()
loop.run_until_complete(print_sum(1, 2))
loop.close()