EventGhost

I don't really kow,
You saw my test where I'm saving the same sensor data to ten different tables at the same time, I assume this is a bit stressful as well. I don't have a virtual machine either but I will try to find out how I could stress the system more somehow

How about using some CPU cycles by using Virtual Dub or similar to transcode a large video file?

I have reverted back to saving data without using threads (retries are still there) and now memory consumption looks ok again. For temperature data you will see a print when saving starts and ends so that we can get an idea of how long it takes (in my tests I do several savings per second).

Please use the current version of both files

(Maybe I can learn more about threading and then improve the plugin)

OK, back up. The first save took 35 seconds! Following save took around five. Then I had one for one second. And then it all stopped. No movement at all in the logger for five minutes, and I have a yellow EG icon. I can move around in EG, but the log isn't going anywhere, and I can't fire any actions. I have to kill EG and go back to the version before threading.

Edit: Oh, and I see that it uses around 50 % CPU, while memory is around 140 000. Maybe I should just give you access to the EG VM that's running at home? I can make a copy of it before you start and then you can play all you want via TeamViewer, and then I just bring back the original after that. Would that help? Mind you I haven't checked if that reacts the same way to this code, I am running stable there right now. But of course I can VNC over my VPN and do everything I want to do there as well.

Edit 2: And while I was writing this, EG self destructed! I didn't have to kill the process, it has taken it's own life. I'll give it one more run just to see if maybe this was a fluke.

No fluke. One minute and 13 seconds for first capture, then one second, a few more around 1-2 seconds, then five seconds, and now stuck. But I thin it may lock up while the reports are being made. I'll deactivate that and try again. The reports haven't messed up anything befor, though.

Final word on this version: Locks up even without any reports created or any FTP-uploading. THis time I even saw a "Database locked" error before it stopped. It runs for a couple of minutes, and that's it. Again I wonder if this can be because I'm running a VM for it. I haven't had problems with any other programs, or any other plugins, for the five years or so that I've been doing that, so it's strange. But then I haven't used anything that works on SQL before either. Just a thought: Would it be worth a shot running a second EG instance on the host, and doing all this capture and reports stuff there? So have something send the temperatures from the EG VM to the EG on the host, just to see if this is a VM problem?

I think, next week we can have a teamviewer session if we do not find the problem before. I do suspect the VM being the reason, isn't there a setting somewhere where you can give the VM direct access to the HD? Why would there otherwise be such a big difference when your system is saving compared to mine (my temp data base is currently around 38 Mb)? One thing, when we save we are first reading the full collection but still, it is very fast in both my machines.

Interesting thought, to have a separate instance in the host itself doing the crunching

I'm at this very moment doing a full image backup of the server, and I will then use the backup partition (I always have one main and one backup partition, so it's possible to get it up and running again very fast) to install EG, the plug-in and the receiving end of the USB over TCPIP boxes. Then I will fire up the system and see how it captures with this last version. Stay tuned!

I'm trying to download the last version from you again for the server (I copied it over with the working version in the VM), but I can't get into the website sto.hopto.org/index.html

Rebooting the apache webserver...

Latest version time is 12:59

I'm afraid I'll have to stop for now and boot back to the regular system while we're away on the family thing. I'll try tonight or tomorrow. Btw could you upload the latest threaded version as well, so I can see if that too works better on a physical server?

latest threaded version

New version with threads for saving, trying some new ideas
Please rename to __init__.py when testing

New threaded version is not working well, I think I got the same type of problem as you saw. I used an older library for threading named 'thread', the newer library I used before is named 'threading' and worked better in my systems but anyway, I could also see memory consumption growing.

As conclusion, only stable right now seems to be the version with saving without threads

I think I have found an optimization possibility in querying the sql ...

Currently, when I do save, the code is first fetching ALL rows. Actually this is not necessary. I only need the last number of added rows that are used in the moving average calculation (default 5). So I'm re-designing the code now trying to improve the performance...