ClimateDataCalculation

[Mastiff]

Sounds great, man! I will try that out myself! So how about the ghosts? I'm a bit scared of ghosts since I saw "Thirtheen Ghosts"...

[krambriw]

You are correct, the ghost thing is bug

[Mastiff]

Phew, then it wasn't a ghost in the machine that escaped from the movie!

[krambriw]

Bug is fixed, new file uploaded

Just replace the CreateComboReport.py file and run the report again

One notice regarding using a cloud service: The more data you have in the chart, the more the performance will degrade. Best performance is always to have HighCharts and the generated report on the local machine. You can see this when moving around in the chart. With 'reasonable' amount of data it works ok. An additional feature could be that you have some plugin that copies the generated report from the cloud to a local folder if it gets too slow working directly in cloud

[Mastiff]

Thanks! I'm starting to wonder if the two weeks is the thing that actually slows me down, I see that the HTML is rather large. But I have hit a few snags (mostly because I haven't used Google drive before). First of all the app won't even start on my 2008 server, and I have my automation running to a network drive, which is not accepted by Google Disk app. Second when I try to open the file from the google disk, I see it in the totally useless raw mode, as the HTML code. I'll have to check this out tonight, when I have the time. I really should start my workday now, or my boss will be rather angry. Luckily I'm he!

[Mastiff]

Time to turn on the brain... I actually have my own web adress, with a webhotel on one.com, which is very fast from Norway! F** Google Disk, i will get it all to my site tonight!

[Mastiff]

My boss is now rather mad at me (and I'm still him...) but it does really help that the loading time has gone down to maybe three seconds! I thought out that I could just as well have the graph on my web hotel as the Bix drive that One.com has, and that did it! Which leads me to: Is it possible to ask for a new feature again? Could the reports be uploaded with FTP instead of just saved to the directory of choice? Or is that just too complicated? I have no idea myself if it's even possible...

Edit: Btw I have no idea why, but it's faster to open the graph from the web hotel then on the local computer where it's stored! I see that the HTML combo report is around 2 megabyte, maybe that's why, if it has to send that data up to Highcharts first on the slow upload I have?

[krambriw]

I found (with Google) two possibles ways you could try:

1) Try to associate a network drive letter with the FTP server path (e.g. \ftp_server\HighCharts). Then use the network drive as path for report creations

2) If you want to save a text file with Python to a FTP server, you could use this in a python script (just put the script as next action after the report creation action):

Code: Select all

from ftplib import FTP
ftp = FTP("myhost.com")  
ftp.login("username", "password")
# open the file to read it as a file like object
f = open("myreport.html", "r")
# Open directory
ftp.cwd("/HighCharts")
# save file like object as to the ftp path /HighCharts
ftp.storlines("STOR myreport.html", f) 
ftp.quit()
f.close()

[Mastiff]

Thanks! I'll try that script. One thing, though: Will the execution of that script lock up EG until it's done? That's something I thought of just now. If it is I think I'm probably better off just doing this with an external program like FreeFileSync. I wouldn't not like to tie up EG for 20-30 seconds every ten minutes.

[krambriw]

I tested the script variant with a ftp server and it works fine (my disk quota is however small) but it took about 9 seconds to transfer a 2.1 Mbyte report. As you suspected EG locks up during the transfer but I think that could be solved by transferring via a thread.

[Mastiff]

Well, I'm on a 600 kb up (12/0,6 slowband), so it takes a bit more for me. But threading sounds nice. if I knew how to do it, that is. Would it be possible to move the execution of everything that takes time (collecting the data and making the report) to a separate thread?

[Mastiff]

I found a neat trick, something called NetDrive that maps an FTP as a drive. I'll play with that a bit.

[Mastiff]

Yeah, that works. It seems it's not freeware as it once was, but it will function with one drive only even after the trial is gone out, so I think I'll just go for this for now. With that and FreeFileSync copying the reports I don't lock up anything. Still the threading of making of reports and stuff would be nice, if possible. As usual I have no idea how much work it is! But if it's not all that bad, it could open up for a yearlong report updated once every night, which would now lock EG up for a stupid amount of time!

[krambriw]

Good you found a solution.

This is a sample script using a thread for the actual transfer. It is not locking up EG while it runs. Time is printed in the beginning and at the end so you can see how long the transfer actually takes. You have to set the file name and path correctly (in my case, just testing with a file on the C drive) and I am deleting the file before I upload a new one just because of space limits

Code: Select all

import time
from threading import Thread, Event

def fileTransfer(transferCommandThreadEvent, fname):
    print time.time()
    from ftplib import FTP
    ftp = FTP('ftp.bredband.net')
    ftp.login('user', 'password')
    f = open("C:\\"+fname, 'r')
    ftp.cwd('/HighCharts')
    try:
        ftp.delete(fname)
    except:
        pass
    ftp.storlines('STOR '+fname, f)
    ftp.quit()
    f.close()
    print time.time()
    transferCommandThreadEvent.set()

fname = 'TG.html'

transferCommandThreadEvent = Event()
transferCommandThread = Thread(
    target=fileTransfer,
    args=(transferCommandThreadEvent, fname)
)
transferCommandThread.start()

[Mastiff]

Interesting! I will try that as well, thanks! Of course it's better to use as litte software as possible.