ClimateDataCalculation

[krambriw]

ValueError: time data '25.03.2016 11:12' does not match format '%Y-%m-%d %H:%M'

Well, this indicates that the format of the date stored in the database has (for some reason) the wrong format. It is represented as dd.mm.yyyy instead of the required yyyy-mm-dd

I cannot see why this is stored wrong but try to run this script (edit it for the correct tableId) to check how the stored data looks like

Code: Select all

import sqlite3

def GetAllData(
 dbName,
 dbTableId
):
    dbName = str(dbName)
    conn = sqlite3.connect(dbName)
    c = conn.cursor()
    t = ((dbTableId),)
    try:
        print 'Export started, please wait...'
        c.execute('SELECT * FROM '+dbName.split('.')[0]+' WHERE tableId=?', t)
        coll = c.fetchall()
        conn.close()
        return coll
    except:
        conn.close()
        return None

# edit this line with name of db and table id
rows = GetAllData('tempData.db', 'Temperature ground')

fileHandle = None
fileHandle = open('dbExport.txt', 'a')
for row in rows:
    #print row
    fileHandle.write(str(row)+'\r\n')
fileHandle.close()
print 'Export done'

[Mastiff]

This is what I got:

Code: Select all

(u'2015-04-13 21:56:13', 1428954974, u'Ute', 3.2999999999999998, u'', 20.0, 20.0, 3.3599999999999999)
(u'2015-04-13 21:57:01', 1428955022, u'Ute', 3.2999999999999998, u'', 20.0, 20.0, 3.3399999999999999)
(u'2015-04-13 21:59:25', 1428955166, u'Ute', 3.2999999999999998, u'', 20.0, 20.0, 3.3199999999999998)
(u'2015-04-13 22:00:13', 1428955214, u'Ute', 3.2999999999999998, u'', 20.0, 20.0, 3.2999999999999998)
(u'2015-04-13 22:01:49', 1428955310, u'Ute', 3.2999999999999998, u'', 20.0, 20.0, 3.2999999999999998)
(u'2015-04-13 22:02:37', 1428955358, u'Ute', 3.2000000000000002, u'', 20.0, 20.0, 3.2799999999999998)
(u'2015-04-13 22:03:25', 1428955406, u'Ute', 3.2000000000000002, u'', 20.0, 20.0, 3.2599999999999998)

And another thing, I have made a script with a 30 days export for each zone (with zone number as filename), and for some reason some of them has a report with 30 hours, not days! Here are two:

Code: Select all

eg.plugins.ClimateDataCalculation.CombinedReport(u'Temperaturrapport', (u'WC',), (), (), (), (), 30, 0, True, u'D:\\EventGhost\\Webserver for EventGhost', u'2', u'30-dagers temperaturrapport for WC', (), True)
eg.plugins.ClimateDataCalculation.CombinedReport(u'Temperaturrapport', (u'Soveromsgang',), (), (), (), (), 30, 0, True, u'D:\\EventGhost\\Webserver for EventGhost', u'3', u'30-dagers temperaturrapport for soveromsgang', (), True)

I have used "Copy as python" to change regular plug-in commands into this, but I just checked the plug-in command, and the same happens there. WC gets the correct 30 days, while Soveromsgang has a 30 hour report. As far as I can see the code is identical. Or am I wrong there?

Finally, is it possible to put in URLs in the report in any way? I would like to put in one URL for a 360 day report for each room and one URL to take you back to the root page. This setup is really getting fancy!

[Mastiff]

OK, really finally: Is there a way to generate really big reports without locking up the system? I have created a set of one year (365 days) reports, but that locks up everything totally while they're running. I could do them at 03 or something, with waits inbetween them, but is there a more elegant solution?

[krambriw]

This is what I got

I was hoping you should show the results around those hours you reported problems for, i.e. how does it look like around 25.03.2016 11:12

for some reason some of them has a report with 30 hours

What is the result if you run the report 'manually'? Is it still 30 minutes instead of 30 days? Are you sure you are saving data every 10 minute as well?

Finally, is it possible to put in URLs in the report in any way?

I suppose you mean a field in the config dialog where you can type in a url to go back to, in this case your root page?
If you create all reports into your web server document root somewhere, you can link directly from the root page to the reports I suppose?

Is there a way to generate really big reports without locking up the system?

Well, such report creation is most likely very resource heavy stuff. I will check if there could be any possibility to release the cpu regularly to avoid this total lockup. But I fear that it will then take longer to create a report

[krambriw]

Is there a way to generate really big reports without locking up the system?

I tried to identify what takes most time by inserting a print when access to database starts and ends. As you see from below, the reading of data from the database is very fast (generating a 365 days 10 min comparison report) and most time is spent in generating the actual html page. In total it took 16 seconds, the size of the html file is 1200 kbyte. Also, my system is not locking up, events are popping in during the creation as you can see and I could work with other items in the tree

06:46:20 RFXtrx.Type: THGN122/123, THGN132, THGR122/228/238/268 id: 8194 ' temperature: +21.2 deg C humidity: 58 %RH status: comfort signal: 6 battery: 9'
06:46:22 MQTT./zwave u"Device_6_Instance_3_temperature, 23.3700016"
06:46:22 fetching started
06:46:22 RFXtrx.Type: THGN122/123, THGN132, THGR122/228/238/268 id: 1796 ' temperature: +15.2 deg C humidity: 92 %RH status: wet signal: 7 battery: 9'
06:46:22 TellStickDuo.oregon.1A2D.7.Temperature.Humidity '15.2|92'
06:46:23 MQTT./zwave u"Device_6_Instance_4_temperature, 26.5"
06:46:24 fetching ended
06:46:24 fetching started
06:46:26 fetching ended
06:46:36 RFXtrx.Type: Viking 02035,02038 id: 46848 ' temperature: +20.6 deg C humidity: 79 %RH status: wet signal: 7 battery: 9'
06:46:36 TellStickDuo.fineoffset.temperaturehumidity.183.Temperature.Humidity '20.6|79'
06:46:38 ClimateDataCalculation.comp 'Comparison report created'
06:47:00 SunTracker.Weather Condition: 'Mostly Cloudy'
06:47:01 RFXtrx.Type: THGN122/123, THGN132, THGR122/228/238/268 id: 8194 ' temperature: +21.2 deg C humidity: 58 %RH status: comfort signal: 6 battery: 9'

If you wanna try the same, edit the __init__.py file, see below, browse the editor to line nbr 403 where you find the function below

Code: Select all

    def LastData(
        self, 
        dbName, 
        dbTableId, 
        dateTimeFrom
    ):
        dbName = str(dbName)
        conn = sqlite3.connect(dbName)
        c = conn.cursor()
        v = ((dbTableId),(dateTimeFrom),)
        try:
            c.execute('SELECT * FROM '+dbName.split('.')[0]+' WHERE tableId=? AND date>=?', v)
            print 'fetching started'
            rows = c.fetchall()
            conn.close()
            print 'fetching ended'
            return rows
        except:
            conn.close()
            return None

[krambriw]

Finally, is it possible to put in URLs in the report in any way?

Coming back to this question, I guess you would have some kind of portal from where you can call the various reports. One solution that I have experimented with is what you can see in the video following the link below
http://screencast.com/t/tuxKA3hDDxuw

[Mastiff]

Sorry I fell out again... I have now tested the code, and that works perfectly! Now I can run the reports without locking up! Thanks a lot! Is that something that can be put into the code base, or will I have to change it in coming versions?

As for the 30 hours, not 30 days, for some reason that just disappeared! So I get the correct amount of days now. No idea why, I haven't changed anything myself. But at least it's working.

Finally, I can probably use html pages with frames to put a menu and stuff into the report page. I guess that will be the simplest way.

[Mastiff]

K-man, quick question: I have finally set up the system at the cabin (it's been a crazy half year since we moved...), and everything works as it should, except for one thing: I do not get anything in the wind rose report. It's empty. I have had data capture running since Tuesday, and everything else works nicely. I see the wind and all the temps in the combo report. Is it because the wind rose needs data from more time to draw?

[krambriw]

That is strange. there are no changes made what I know. Pls zip & email me your wind database and I will check

[Mastiff]

Will do! Thanks!

[Mastiff]

Did you get my mail? I forgot to ask if you're still using the bredband address!

[krambriw]

Yes got it,
It is strange, also my old generated reports does not work any longer. Data inside looks ok (if you view source). Something with HighCharts...
Currently debugging, will take some more time

[Mastiff]

Thanks! No rush at all! The main thing for me is that I can see the strength and direction in my web page for the house/cabin, and that I can see if the wind is getting stronger or weaker at the moment in the combo report. I need to know that before I go out and try to pull my crayfish (sj├Âkr├ñfta) pots!

[krambriw]

Found the problem, just verifying & final checking

[Mastiff]

Good man!