Page 2 of 37

Re: ClimateDataCalculation

Posted: Fri Nov 14, 2014 8:49 am
by krambriw
Hello Mastiff,
I have uploaded a new version with a lot of new features (I will describe all of that later). However, as indicated before, the database you have now is not compatible anymore :( since I have added numerous of data needed for all new features...

So to start:

1) Take a screen shot or make a note of your macro configurations you have done for the rain collection part
2) Delete those macros...
3) Stop EG and delete your databases found in the EventGhost folder (rainData.db, humData.db, tempData.db)
4) Update the ClimateDataCalculation plugin files with the latest, you find them here
http://sto.hopto.org/Under%20constructi ... 20version/
5) Start EG again, re-configure your previous macros and data will start to be collected again

When you configure an action to capture data, you will get a dialog like below. I'll just explain some new settings, the plugin is getting more mature:

- you can now configure how frequently you want to save readings (default every 10 minutes)
- the plugin calculates two moving averages, one fast and one slow. This is then used to create events when the lines are crossing each other. The more readings you select, the slower the response to changes. This is a very useful function to avoid too many status changes when for instance controlling heaters (you might temporary open a door or window but the heater should not react immediately)
- in addition you can add a hysteresis value that will be subtracted from the fast moving average
- you can select if you want to use the rule evaluation that will generate those events or not. Currently I have only one rule 'equal-less-greater' that will generate events when the fast moving average gets equal, less or greater than the slow moving average.
- all this works for rain, temperature and humidity data
- rain, temperature and humidity data can be collected with a RFXtrx
- temperature and humidity data can be collected with a TellStick Duo (I am lacking a TellStick Duo log sample from a rain sensor)
- data from other devices like z-wave and 1wire can easily be added but I need sample events from the EG log.
Image2.jpg
Now, your eg.globals....

I think the script below should work. Just create a macro where you put it and change the name of the table 'RainSensor on roof' to what you have named to.
Everytime you trigger it, your eg.globals will be updated (so you need to add a suitable event that triggers it on a regular base, i.e. you could use the SchedulGhost plugin for that purpose. If you need more or other intervals you can easily add more or change them. You get the idea when you look at the syntax.

Code: Select all

import datetime 
now = datetime.datetime.now().strftime("%Y-%m-%d %H:%M")
start_of_today = datetime.date.today().strftime("%Y-%m-%d %H:%M")
sevenDaysAgo = (datetime.datetime.now() + datetime.timedelta(-7)).strftime("%Y-%m-%d %H:%M")
thirtyDaysAgo = (datetime.datetime.now() + datetime.timedelta(-30)).strftime("%Y-%m-%d %H:%M")

eg.globals.Regn_dag = eg.plugins.ClimateDataCalculation.RainRequestQuery(u'RainSensor on roof', start_of_today, now)
eg.globals.Regn_uke = eg.plugins.ClimateDataCalculation.RainRequestQuery(u'RainSensor on roof', sevenDaysAgo, now)
eg.globals.Regn_maaned = eg.plugins.ClimateDataCalculation.RainRequestQuery(u'RainSensor on roof', thirtyDaysAgo, now)
Good luck,
Walter

Re: ClimateDataCalculation

Posted: Fri Nov 14, 2014 9:42 am
by Mastiff
Thanks, mate! :mrgreen: I will have to check this out during the weekend, today I'm scrambling to get the cabin ready after three weeks alone her doing all kinds of work at the same time, often not finishing one project before I go to the next. So there's a bloody mess to clear up between now and around 8 P.M., when my wife will arrive! :shock:

Re: ClimateDataCalculation

Posted: Mon Nov 17, 2014 2:03 pm
by Mastiff
Sorry, haven't gotten around to it. :( Some stuff happened, so I had too much to finish off, but I'm going back to the cabin next weekend. And the weekend after that. So I will test it!

Re: ClimateDataCalculation

Posted: Wed Nov 26, 2014 7:43 am
by Mastiff
Well, it works, thanks! :) I have tested collecting rain data and the quick report, and no problems there. I will get to the code this coming weekend as well (because a colleague had an accident I had to do the work of a colleague last weekend, so not much time, but enough to test collecting and quick report). But I'm still coming up with new ideas. Have you seen the website Templight? I have used that for a while, it shows graphs of your sensor's readings (collected from a Telldus.net). Here's my outdoor zone at home:

http://www.templight.se/scenario_extern ... 87IPAE1RQE

Is this something that could be done locally instead, with EG? That would be very nice, there often is a delay on the Templight website, and I would prefer to have it all in house. :mrgreen:

Re: ClimateDataCalculation

Posted: Fri Nov 28, 2014 11:07 am
by krambriw
Hello,
I have made a new version that is somehow providing a graphical view of the data. You can download it from the normal link where you find the current version.

I have updated the Rain Report action so that it generates a html page that you can open in a browser. The config dialog looks like this
Image1.jpg
You can select another path that points to your webserver virtual directory if you like.

The html code generation is generic and put into a separate file, CreateHtml.py, so it can be customized. In this beta version I have only made available from the Rain Report action but it is easy to extend the plugin and add report actions for Temperature, Humidity etc.

If you put the action in a macro you can update the html page automatically by providing a suitable event in the same.

What I wanted to show is that now when the data is in a database you can create any kind of web pages presenting it. I did it in python but you can of course use other languages of your choice. It is just a matter of effort you would like to put into it.

Best

Re: ClimateDataCalculation

Posted: Sat Nov 29, 2014 7:31 am
by krambriw
New beta version available
- bugfix in rain report
- actions to generate html reports also for temperature and humidity data

PS @Mastiff: you have disabled your PM function somehow so I cannot respond to your question...

Re: ClimateDataCalculation

Posted: Sat Nov 29, 2014 9:22 am
by Mastiff
Cool, tonight I hope to have a chance to check it out! :D And sorry, the PM's working now. Wrong setting (or maybe paranoid setting, I have had forums with some hysterical bull and spam in PM's before, so I often choose not to activate that when I first go into a forum).

Re: ClimateDataCalculation

Posted: Sat Nov 29, 2014 4:24 pm
by Mastiff
OK, here my total lack of understaing of programming hits me in the ass again. :oops: I have no idea what this means:
17:23:03 Error in Action: "ClimateDataCalculation: RainReport: Regnmåler"
17:23:03 Traceback (most recent call last) (1669):
17:23:03 File "C:\Programfiler\EventGhost\eg\Classes\ActionBase.py", line 170, in CallWrapper
17:23:03 return self(*args)
17:23:03 File "C:\Programfiler\EventGhost\plugins\ClimaDataCalculation\__init__.py", line 938, in __call__
17:23:03 CreateHtml.CreateHtml(dbTableId, "Rain", fPath, rows, ['Time', 'Rain Total'], 4)
17:23:03 File "C:\Programfiler\EventGhost\plugins\ClimaDataCalculation\CreateHtml.py", line 69, in CreateHtml
17:23:03 fileHandle.write(logstr)
17:23:03 UnicodeEncodeError: 'ascii' codec can't encode character u'\xe5' in position 26033: ordinal not in range(128)
Only thing I can think of is that it doesn't like the "å" in the name of the table. Is that a probable reason? I'm unable to check that now because I don't have contact with the rain sensor at the moment, don't know why

Re: ClimateDataCalculation

Posted: Sat Nov 29, 2014 5:57 pm
by krambriw
Is that a probable reason?
Yes, that is very likely, I'll look into that

Re: ClimateDataCalculation

Posted: Sat Nov 29, 2014 6:49 pm
by krambriw
A quick fix: just replace with the attached file
CreateHtml.py
(2.22 KiB) Downloaded 122 times

Re: ClimateDataCalculation

Posted: Sun Nov 30, 2014 11:50 am
by Mastiff
Yessir, thanks! :) That works! Btw is it possible to get this to create a regular interval that keeps up with the week, day and so on? It seems like I have to choose the period from the dropdown list with specific points, instead being able to say "last week", "last 24 hours" and so on.

Re: ClimateDataCalculation

Posted: Sun Nov 30, 2014 2:51 pm
by krambriw
Yes, sure, anything is possible, this first shot was just to verify the technique. I can make some actions that are doing what you suggest.

Another thing I noticed is that when I open an action with 'a lot of' data behind (temperature and humidity in this case), it takes a short while for it to load. I was looking for a way to inform the user 'Please stand by...' but my attempts so far has been blocked by the heavy load the db access is causing...but it is working anyway and that's the good part.

So in summary, have I understood that it would be desirable to have actions to generate html reports
- for the last 1-365 days
- for the last 1-24 hours

??

Should I keep the current action as well or should it be removed?

Best regards

Re: ClimateDataCalculation

Posted: Sun Nov 30, 2014 4:15 pm
by Mastiff
For my part you can drop the one you have, it's kind of counter intuitive to have to edit the action itself to change the period you see in the graph. In that case it's much better just to have it set to regular periods like you say. :) I haven't seen that slowdown yet, but it may be because I don't have that much data yet.

Re: ClimateDataCalculation

Posted: Mon Dec 01, 2014 8:43 am
by krambriw
I have made a change accordingly. The config of the report actions typically looks like this now:
Image1.jpg
- you select the number of days & hours of history to cover

So it is calculating the total number of hours backward from now and then retrieving the data / creating the html file

Best regards

Re: ClimateDataCalculation

Posted: Wed Dec 03, 2014 8:12 am
by Mastiff
Thanks! Of course that introduced another problem again: The "å". I keep giving you grief here... I get this log output:

Code: Select all

09:08:45   Webserver error /regnm%C3%A5ler.html
09:08:45   Exception 'ascii' codec can't decode byte 0xc3 in position 6: ordinal not in range(128)
09:08:45   ----------------------------------------
09:08:45   Exception happened during processing of request from ('192.168.0.27', 56312)
09:08:45   Traceback (most recent call last):
09:08:45     File "SocketServer.pyc", line 558, in process_request_thread
09:08:45     File "SocketServer.pyc", line 320, in finish_request
09:08:45     File "SocketServer.pyc", line 615, in __init__
09:08:45     File "BaseHTTPServer.pyc", line 329, in handle
09:08:45     File "BaseHTTPServer.pyc", line 323, in handle_one_request
09:08:45     File "C:\Programfiler\EventGhost\plugins\Webserver\__init__.py", line 702, in do_GET
09:08:45       self.SendContent(path)
09:08:45     File "C:\Programfiler\EventGhost\plugins\Webserver\__init__.py", line 455, in SendContent
09:08:45       fsPath = self.translate_path(path)
09:08:45     File "C:\Programfiler\EventGhost\plugins\Webserver\__init__.py", line 744, in translate_path
09:08:45       path = os.path.join(path, word)
09:08:45     File "ntpath.pyc", line 108, in join
09:08:45   UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 6: ordinal not in range(128)
Another thing: Can you please make it possible to specify not just the directory for the output html but the name of it as well? That way I can have one called 24hours.html, one called week.html, one called month.html and so on. Is that possible?