Re: ClimateDataCalculation
Posted: Mon May 25, 2015 3:02 pm
Hello, I have made a new version supporting gaps (it was actually a supported HighCharts api feature)
Page 22 of 37
Re: ClimateDataCalculation
Posted: Mon May 25, 2015 9:22 pm
How nice, thanks! I'll get to that tomorrow!
Re: ClimateDataCalculation
Posted: Thu May 28, 2015 11:59 am
Weird, I still have those lines down to zero. Also it seems to be very sensitive to the gaps, probably too sensitive (a few minutes, and there's a gap). But worse: Opening the graph locks up Firefox for a minute or two. Any idea what that can be? I'm PM'ing you the address to one of my graphs.
Re: ClimateDataCalculation
Posted: Sun May 31, 2015 3:41 pm
So here is a new version that supports making charts with 10 minutes data point interval (default). To update, replace with the new files, restart EG and open/save existing actions for Combo reports.
To select/deselect the new setting, tick the check box as below
To select/deselect the new setting, tick the check box as below
Re: ClimateDataCalculation
Posted: Tue Jun 02, 2015 7:49 am
Wow, than you for that one! 1/10 of the data really sped up the loading of those graphs! Very nice, sir! A couple of things:
1. What program do you use to look at the database? I have zero experience with databases (well, close to zero - I was doing a bit of dBase work in the early 90's, but I would imagine stuff has changed a bit since that...). I was looking for the reason for a gap in the graph, and I printed the full database for "Loftet". That locked up EG for abour 10 minutes...
2. I found the area with data for the gap (in the file log, it was long gone in the log window because of the mass of data, which is why I'd like to have a program to look at this in a better way), it's like this:
This looks like this in the graph, zoomed in:
So there's a gap between 18.19 and 19.09, but there is a lot of data, just not on the exact 10 minute. Is it possible in any way to read the closest data to the ten minutes instead of getting that gap, or is that impossible? I'm just asking, I have no idea about this.
Oh, btw: Do you get mails warning you about new posts in your threads? I don't get that anymore and I'm wondering if it's my account or a general error on the board.
Very nice, sir! A couple of things:1. What program do you use to look at the database? I have zero experience with databases (well, close to zero - I was doing a bit of dBase work in the early 90's, but I would imagine stuff has changed a bit since that...). I was looking for the reason for a gap in the graph, and I printed the full database for "Loftet". That locked up EG for abour 10 minutes...
2. I found the area with data for the gap (in the file log, it was long gone in the log window because of the mass of data, which is why I'd like to have a program to look at this in a better way), it's like this:
Code: Select all
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:18:45', 1432916325, u'Loftet', 25.600000000000001, u'', 10.0, 10.0, 25.579999999999998)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:21:09', 1432916469, u'Loftet', 25.600000000000001, u'', 10.0, 10.0, 25.579999999999998)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:21:57', 1432916517, u'Loftet', 25.5, u'', 10.0, 10.0, 25.57)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:22:45', 1432916565, u'Loftet', 25.5, u'', 10.0, 10.0, 25.550000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:24:21', 1432916661, u'Loftet', 25.5, u'', 10.0, 10.0, 25.550000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:25:09', 1432916709, u'Loftet', 25.5, u'', 10.0, 10.0, 25.530000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:25:57', 1432916757, u'Loftet', 25.5, u'', 10.0, 10.0, 25.52)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:26:45', 1432916805, u'Loftet', 25.5, u'', 10.0, 10.0, 25.5)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:27:33', 1432916853, u'Loftet', 25.5, u'', 10.0, 10.0, 25.5)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:28:21', 1432916901, u'Loftet', 25.5, u'', 10.0, 10.0, 25.5)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:29:09', 1432916949, u'Loftet', 25.5, u'', 10.0, 10.0, 25.5)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:29:57', 1432916997, u'Loftet', 25.399999999999999, u'', 10.0, 10.0, 25.48)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:30:45', 1432917045, u'Loftet', 25.399999999999999, u'', 10.0, 10.0, 25.469999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:31:33', 1432917093, u'Loftet', 25.399999999999999, u'', 10.0, 10.0, 25.449999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:32:21', 1432917141, u'Loftet', 25.399999999999999, u'', 10.0, 10.0, 25.43)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:33:09', 1432917189, u'Loftet', 25.399999999999999, u'', 10.0, 10.0, 25.420000000000002)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:33:57', 1432917237, u'Loftet', 25.399999999999999, u'', 10.0, 10.0, 25.399999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:34:45', 1432917285, u'Loftet', 25.399999999999999, u'', 10.0, 10.0, 25.399999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:35:33', 1432917333, u'Loftet', 25.399999999999999, u'', 10.0, 10.0, 25.399999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:36:21', 1432917381, u'Loftet', 25.399999999999999, u'', 10.0, 10.0, 25.399999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:37:09', 1432917429, u'Loftet', 25.399999999999999, u'', 10.0, 10.0, 25.399999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:37:57', 1432917477, u'Loftet', 25.300000000000001, u'', 10.0, 10.0, 25.379999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:38:45', 1432917525, u'Loftet', 25.300000000000001, u'', 10.0, 10.0, 25.370000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:39:33', 1432917573, u'Loftet', 25.300000000000001, u'', 10.0, 10.0, 25.350000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:40:21', 1432917622, u'Loftet', 25.300000000000001, u'', 10.0, 10.0, 25.329999999999998)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:41:09', 1432917669, u'Loftet', 25.300000000000001, u'', 10.0, 10.0, 25.32)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:42:45', 1432917765, u'Loftet', 25.300000000000001, u'', 10.0, 10.0, 25.300000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:43:33', 1432917813, u'Loftet', 25.300000000000001, u'', 10.0, 10.0, 25.300000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:44:21', 1432917861, u'Loftet', 25.300000000000001, u'', 10.0, 10.0, 25.300000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:45:09', 1432917909, u'Loftet', 25.300000000000001, u'', 10.0, 10.0, 25.300000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:46:45', 1432918005, u'Loftet', 25.300000000000001, u'', 10.0, 10.0, 25.300000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:47:33', 1432918054, u'Loftet', 25.199999999999999, u'', 10.0, 10.0, 25.280000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:48:21', 1432918101, u'Loftet', 25.199999999999999, u'', 10.0, 10.0, 25.27)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:49:09', 1432918150, u'Loftet', 25.199999999999999, u'', 10.0, 10.0, 25.25)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:49:57', 1432918198, u'Loftet', 25.199999999999999, u'', 10.0, 10.0, 25.23)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:51:33', 1432918293, u'Loftet', 25.199999999999999, u'', 10.0, 10.0, 25.219999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:52:21', 1432918341, u'Loftet', 25.199999999999999, u'', 10.0, 10.0, 25.199999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:53:09', 1432918389, u'Loftet', 25.199999999999999, u'', 10.0, 10.0, 25.199999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:54:45', 1432918486, u'Loftet', 25.100000000000001, u'', 10.0, 10.0, 25.18)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:55:33', 1432918534, u'Loftet', 25.100000000000001, u'', 10.0, 10.0, 25.170000000000002)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:56:21', 1432918581, u'Loftet', 25.100000000000001, u'', 10.0, 10.0, 25.149999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:57:09', 1432918629, u'Loftet', 25.100000000000001, u'', 10.0, 10.0, 25.129999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:57:57', 1432918677, u'Loftet', 25.100000000000001, u'', 10.0, 10.0, 25.120000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:58:45', 1432918726, u'Loftet', 25.0, u'', 10.0, 10.0, 25.079999999999998)
2015-06-02 09:26:36 INFO: (u'2015-05-29 18:59:33', 1432918774, u'Loftet', 25.0, u'', 10.0, 10.0, 25.07)
2015-06-02 09:26:36 INFO: (u'2015-05-29 19:00:21', 1432918822, u'Loftet', 25.0, u'', 10.0, 10.0, 25.050000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 19:01:09', 1432918869, u'Loftet', 25.0, u'', 10.0, 10.0, 25.030000000000001)
2015-06-02 09:26:36 INFO: (u'2015-05-29 19:01:57', 1432918917, u'Loftet', 25.0, u'', 10.0, 10.0, 25.02)
2015-06-02 09:26:36 INFO: (u'2015-05-29 19:02:45', 1432918966, u'Loftet', 25.0, u'', 10.0, 10.0, 25.0)
2015-06-02 09:26:36 INFO: (u'2015-05-29 19:03:33', 1432919014, u'Loftet', 25.0, u'', 10.0, 10.0, 25.0)
2015-06-02 09:26:36 INFO: (u'2015-05-29 19:04:21', 1432919062, u'Loftet', 25.0, u'', 10.0, 10.0, 25.0)
2015-06-02 09:26:36 INFO: (u'2015-05-29 19:05:09', 1432919110, u'Loftet', 25.0, u'', 10.0, 10.0, 25.0)
2015-06-02 09:26:36 INFO: (u'2015-05-29 19:05:57', 1432919158, u'Loftet', 25.0, u'', 10.0, 10.0, 25.0)
2015-06-02 09:26:36 INFO: (u'2015-05-29 19:08:21', 1432919301, u'Loftet', 24.899999999999999, u'', 10.0, 10.0, 24.98)
2015-06-02 09:26:36 INFO: (u'2015-05-29 19:09:09', 1432919350, u'Loftet', 24.899999999999999, u'', 10.0, 10.0, 24.969999999999999)
2015-06-02 09:26:36 INFO: (u'2015-05-29 19:09:57', 1432919398, u'Loftet', 24.899999999999999, u'', 10.0, 10.0, 24.949999999999999)Oh, btw: Do you get mails warning you about new posts in your threads? I don't get that anymore and I'm wondering if it's my account or a general error on the board.
Re: ClimateDataCalculation
Posted: Tue Jun 02, 2015 9:30 am
I have nothing special, normally I just write a suitable python script if we are talking about the databases for this plugin. But I guess there could be many other alternatives. Have you tried with excel?1. What program do you use to look at the database?
Hmm, it looks like there should not be any gap here...2. I found the area with data for the gap
It works that way (or should). In the python code I am looking at the time when it is changing. Lets say it is now 18:19, I will then pick the first value found recorded at that time. The next will be the first value found at 18:20. And so onIs it possible in any way to read the closest data to the ten minutes instead of getting that gap
Maybe you could PM me the link to download your db where this error is happening and I will have a look at it
Sorry, I don't use that featureOh, btw: Do you get mails warning you about new posts in your threads?
Re: ClimateDataCalculation
Posted: Tue Jun 02, 2015 9:47 am
1. No, Excel does not want to open them. But it would help to limit the scope. Here's your code I use to read the database, maybe you could show me how to limit on time/date as well as sensor, or even how to export it to a text file, if you do that:
2. I'm glad you already had thought of that, I was scared thinking that only I had that idea! You'll find the file tempData.zip at the same URL as you got the previous temp database of mine. I zipped it this to shave off some time with the crappy upload I have here.
As for the mail thing I actually got a mail warning for this one! Maybe because I was into my control panel, changed and changed back and saved a couple of settings. They're just like they were before, but maybe they were garbled in the internal system at the forum.
Code: Select all
import sqlite3
def GetAllData(
dbName,
dbTableId
):
dbName = str(dbName)
conn = sqlite3.connect(dbName)
c = conn.cursor()
t = ((dbTableId),)
try:
c.execute('SELECT * FROM '+dbName.split('.')[0]+' WHERE tableId=?', t)
coll = c.fetchall()
conn.close()
return coll
except:
conn.close()
return None
rows = GetAllData('tempData.db', 'Loftet')
for row in rows:
print rowAs for the mail thing I actually got a mail warning for this one! Maybe because I was into my control panel, changed and changed back and saved a couple of settings. They're just like they were before, but maybe they were garbled in the internal system at the forum.
Re: ClimateDataCalculation
Posted: Tue Jun 02, 2015 11:24 am
In your db for 'Loftet', I can see the gap from '2015-05-29 21:52:21' until '2015-05-29 23:44:22'.
I realize however that I'm not handling gaps larger than one hour...so maybe this is messing up the graph (I'll have to fix this)
As for exporting data to a file, I have adjusted the script a bit and here you can set the starting/ending date & hour
I realize however that I'm not handling gaps larger than one hour...so maybe this is messing up the graph (I'll have to fix this)
As for exporting data to a file, I have adjusted the script a bit and here you can set the starting/ending date & hour
Code: Select all
import sqlite3
def GetAllData(
dbName,
dbTableId
):
dbName = str(dbName)
conn = sqlite3.connect(dbName)
c = conn.cursor()
t = ((dbTableId),)
try:
c.execute('SELECT * FROM '+dbName.split('.')[0]+' WHERE tableId=?', t)
coll = c.fetchall()
conn.close()
return coll
except:
conn.close()
return None
rows = GetAllData('tempData.db', 'Loftet')
fileHandle = None
fileHandle = open('dbExport.txt', 'a')
for row in rows:
if row[0].find('2015-05-29 21')> -1 or row[0].find('2015-05-29 23')> -1:
fileHandle.write(str(row)+'\r\n')
fileHandle.close()
print 'Export done'
Re: ClimateDataCalculation
Posted: Tue Jun 02, 2015 11:39 am
Nice, thanks for the script! 
As for the gap the missing bit is from 18:18 to 19:08 in the graph, and there seems to be data in the graph in the period where there actually are no data in the database. So you're probably right, as usual.
As for the gap the missing bit is from 18:18 to 19:08 in the graph, and there seems to be data in the graph in the period where there actually are no data in the database. So you're probably right, as usual. Re: ClimateDataCalculation
Posted: Tue Jun 02, 2015 2:23 pm
I found the problem (handling more than 1 hour gaps). Updated the CreateComboHtml.py file
Now I think it looks correct
Now I think it looks correct
- Skärmklipp.PNG (7.71 KiB) Viewed 3892 times
Re: ClimateDataCalculation
Posted: Tue Jun 02, 2015 3:36 pm
Are you certain? On my system the gap seems to be between 19.13 and 21.03 now, not 21.52 and 23.44 (when zooming in and holding the mouse over the data points).
Re: ClimateDataCalculation
Posted: Tue Jun 02, 2015 3:39 pm
Re: ClimateDataCalculation
Posted: Tue Jun 02, 2015 3:44 pm
OK, it seems to be something weirder. It saves a new report, but the data in the new report stops at 14.51 today.
Re: ClimateDataCalculation
Posted: Wed Jun 03, 2015 5:19 am
I think I see your point, you are correct, something is wrong in the calculation & gap compensation I am trying to achieve. It is easy to see if you generate a chart for five days, then it should match ok but for 30 days it gets corrupted...
I need to see if I can improve the gap compensation
I need to see if I can improve the gap compensation
Re: ClimateDataCalculation
Posted: Wed Jun 03, 2015 6:52 am
OK, thanks! So this time it's not just me being crazy!