Notice: This forum has been recovered from an old backup, so some content, links, and dates may be outdated. The forum is currently read-only while we restore sign-in and registration functionality. Details

If you find this forum valuable and would like to help keep it online, donations to help cover hosting and domain costs are greatly appreciated, but never expected. You can support the forum through Buy Me a Coffee or Ko-fi. Thank you for helping preserve the EventGhost community.

ClimateDataCalculation

Questions and comments specific to a particular plugin should go here.
krambriw
Plugin Developer
Posts: 2570
Joined: Sat Jun 30, 2007 2:51 pm
Location: Stockholm, Sweden
Contact:

Re: ClimateDataCalculation

Post by krambriw »

But I guess I may have to go down on the frequency of logging in the end.
Yes, and maybe distribute the report creation around the hour. If you have a report with numerous nbr of tables involved, a heave reading of the db occurs. Even if there is no writing to the db, a number of threads are started. Since we do not really know where the limit of concurrent running threads are (especially inside a VM), I think you can adjust the upper limit until we see the "old" problem coming back.

Since the Q goes up that high, I fear that there are some threads that are not terminating correctly (could be threads used for report creation).

But spreading the execution a bit would not harm...
Mastiff
Experienced User
Posts: 872
Joined: Thu May 03, 2012 10:43 am

Re: ClimateDataCalculation

Post by Mastiff »

It seems like something must be done, yeah. I have another VM that does some work related video conversion, which is a bit heavy on the CPU, and it seems like that's when it happens. Because just now it has gone up to the 45 and locked again. I'll change the report generation a bit and then we'll see.
Mastiff
Experienced User
Posts: 872
Joined: Thu May 03, 2012 10:43 am

Re: ClimateDataCalculation

Post by Mastiff »

Weird, I was working on the new setup (each report triggers the next after two minutes), and suddenly the queue just started to tick again.
Mastiff
Experienced User
Posts: 872
Joined: Thu May 03, 2012 10:43 am

Re: ClimateDataCalculation

Post by Mastiff »

And now it's down to 1! Well, I have made a system now with the two minutes delay for each of the five reports that are triggered every hour, and then finally triggers the FTP upload. I'll try that until tomorrow and see what happens.
krambriw
Plugin Developer
Posts: 2570
Joined: Sat Jun 30, 2007 2:51 pm
Location: Stockholm, Sweden
Contact:

Re: ClimateDataCalculation

Post by krambriw »

Great, that will be interesting
Thanks!
Mastiff
Experienced User
Posts: 872
Joined: Thu May 03, 2012 10:43 am

Re: ClimateDataCalculation

Post by Mastiff »

Changing the report generation to 10 minutes, not 2. Two weeks worth of data takes a bit of time, it seems...
Mastiff
Experienced User
Posts: 872
Joined: Thu May 03, 2012 10:43 am

Re: ClimateDataCalculation

Post by Mastiff »

After the generation is done the que seems to fall again. Now I'm starting a new video conversion while running reports. And then I'm going to check in half an hour to see the damage! :mrgreen:
krambriw
Plugin Developer
Posts: 2570
Joined: Sat Jun 30, 2007 2:51 pm
Location: Stockholm, Sweden
Contact:

Re: ClimateDataCalculation

Post by krambriw »

My friend, you are really hitting the machine hard!
Mastiff
Experienced User
Posts: 872
Joined: Thu May 03, 2012 10:43 am

Re: ClimateDataCalculation

Post by Mastiff »

Yep, I am! I work hard, I play hard, I fu... Well, never mind! :oops: The queue is up to 41 now, it seems to keep growing even when I'm only running one report and video at the same time. I guess I'll have to increase the size of the queue to around 100! Would that be dangerous? The video conversion is only taking about 20 minutes, and I do them 3-4 times a day.
krambriw
Plugin Developer
Posts: 2570
Joined: Sat Jun 30, 2007 2:51 pm
Location: Stockholm, Sweden
Contact:

Re: ClimateDataCalculation

Post by krambriw »

Would that be dangerous?
No it's not, the thing that can happen is that we eventually get that old exception back again ('Can't start thread...') due to limited resources. But otherwise, just try, it might be that 100 is still far from the upper limit. And if you get that out of resource problem, it could be that giving the VM more RAM could also help.
Mastiff
Experienced User
Posts: 872
Joined: Thu May 03, 2012 10:43 am

Re: ClimateDataCalculation

Post by Mastiff »

Up to 141 before the video conversion was done at 13.27, after around 27 minutes. Then it started to shrink rather slowly, bobbing up and down like 132-133-134-133-132-131 and so on.

I can try more memory since I have so much available. CPU use is around 60-70 when the queue is shrinking, but it's only using a flatline of 1,26 gig RAM, so I'm not sure if that can help.

But some days the video conversion goes almost non stop for a couple of hours. So maybe I have to put the queue at 600? :mrgreen: It hasn't gone any lower than 75 after 15 minutes, so I guess this will take some time to get under control again!
krambriw
Plugin Developer
Posts: 2570
Joined: Sat Jun 30, 2007 2:51 pm
Location: Stockholm, Sweden
Contact:

Re: ClimateDataCalculation

Post by krambriw »

Yep, and soon there will be another round coming up...

Maybe considering physically separating home automation from video amusements...I mean it is not possible to make a standard Volvo hitting 310 km/h even if downhill, longing for home and need for a toilet visit.
krambriw
Plugin Developer
Posts: 2570
Joined: Sat Jun 30, 2007 2:51 pm
Location: Stockholm, Sweden
Contact:

Re: ClimateDataCalculation

Post by krambriw »

...or if possible, set the priority of your video VM to very low
Mastiff
Experienced User
Posts: 872
Joined: Thu May 03, 2012 10:43 am

Re: ClimateDataCalculation

Post by Mastiff »

But it's possible for it to reach 712.76 km/h in free fall! :lol: Which in my opinion is all a Volvo is useful for... I'd rather have a Koenigsegg! But seriously, this is work, not amusement. That means that when I'm at the cabin it has to be easily accessible from both home (when my wife works from home) and the cabin. I have fiber at home, bad DSL at the cabin, so sending to the cabin is fast, the other way is 1/50 of the speed... It's down to 20 now. :mrgreen:

Edit: Almost sync posting (which is a lot like sync swimmning, except for that I don't wear a bathing suit, I have no idea what you're wearing...)! That could be possible, but I have the video stuff at normal, and so is the VM. Oh, and now it's down to 5! So now let's see what happens when there's another round of reports in five minutes, this time without the video conversion.
Mastiff
Experienced User
Posts: 872
Joined: Thu May 03, 2012 10:43 am

Re: ClimateDataCalculation

Post by Mastiff »

Reporting without the video conversion is no problem. :mrgreen: So I guess I'll have to make a change in collecting data when I convert video.
Post Reply