OpenNetHomeServer

[Mastiff]

1) Where and how are you running NHS itself?
Same computer, and it's never had a similar problem before this last version.

2) Have you checked the NHS log itself?
Not really, I didn't think about that.

3) I run NHS in a RPi and the TCP port connection sometimes dies after a week or so. With the latest version, it stays alive longer
I have never had the TCP dying on me, even when I have ran for almost a month (but that's rare, I usually change stuff more often than that).

4) The error for CDC: wind sensor, are you expecting data from NHS for this?

No, it''s coming from the RFX, I just wondered if that error could have created a cascade in any form.

5) Here we see a lost sensor event that I believe is correct?

OpenNetHome.'Lost contact with sensor: FineOffset|1276'

That's very possible. I'm not really sure about that one.

6) I think the TCP port in NHS has died. Did you restart NHS yesterday?

Actually I think I didn't. I have done a full reboot now, so let's see tomorrow if it's all working. If not it's time for another bughunt I'm afraid.

Oh, btw, Bert has made a new firmware (95) for the RFX which should take care of the new kind of Viking sensors. Could you please have a look and see if you can update the RFX plug-in? Enjoy your weekend!

[krambriw]

Could you please have a look and see if you can update the RFX plug-in?

I have just this morning received a new sdk from Bert (that is necessary to have to update the plugin) but the Viking 02813 is not in there yet

BR

[Mastiff]

Moved to the correct thread...

[krambriw]

Pls discuss the rfx in the rfxtrx topic

[Mastiff]

It happened again, I'm afraid. It started here, without any error messages ahead of it:

Code: Select all

23:12:19            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 25643, 25593)
23:15:20            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 25698, 25643)
23:18:21            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 25750, 25698)
23:21:23            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 25798, 25750)
23:24:24            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 25850, 25798)
23:27:25            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 25904, 25850)
23:30:26            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 25958, 25904)
23:33:27            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 26008, 25958)
23:36:29            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 26060, 26008)
23:39:30            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 26106, 26060)
23:42:31            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 26162, 26106)
23:45:32            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 26215, 26162)
23:48:33            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 26261, 26215)
23:51:35            ('OpenNetHome', 'Checking connection with...', 'Tellstick Duo', 26314, 26261)

Events still come in to NetHomeServer. THis is the full log of NHS:

Code: Select all

16.02.07 09:00:00 	WeekTimer 	error,3,Could not find instance specified
	16.02.06 21:40:00 	WeekTimer 	error,3,Could not find instance specified
	16.02.06 17:00:00 	WeekTimer 	error,3,Could not find instance specified
	16.02.06 16:43:07 	TCPCommandPort 	Connection from 192.168.2.100
	16.02.06 16:43:02 	TCPCommandPort$Session 	Disconnected from 192.168.2.100
	16.02.06 16:43:02 	TCPCommandPort$Session 	Failed reading from socket in TCPCommandPort java.net.SocketException: Connection reset
	16.02.06 14:43:06 	TCPCommandPort 	Connection from 192.168.2.100
	16.02.06 14:43:01 	TCPCommandPort$Session 	Disconnected from 192.168.2.100
	16.02.06 14:43:01 	TCPCommandPort$Session 	Failed reading from socket in TCPCommandPort java.net.SocketException: Connection reset
	16.02.06 10:43:03 	TCPCommandPort$Session 	Disconnected from 192.168.2.100
	16.02.06 10:43:03 	TCPCommandPort 	Connection from 192.168.2.100
	16.02.06 10:43:02 	TCPCommandPort 	Connection from 192.168.2.100
	16.02.06 10:42:39 	HomeServer 	Activated 49 of 49 Items
	16.02.06 10:42:27 	HomeItemFileLoader 	Failed to load Item nr77from file. Could not create class: UsbScanner
	16.02.06 10:42:24 	HomeItemFileLoader 	Loading Items from default.xml
	16.02.06 10:42:24 	HomeManagerStarter 	Logging to: C:\Users\Administrator/HomeManager%g.log
	16.02.06 10:42:24 	HomeManagerStarter 	**Starting HomeManager 2.1-SNAPSHOT-ad869**

So nothing special around the time where EG looses the Tellstick. And the big problem is of course that it not only looses the tellstick, it stops processing everything. The stuff I show above is everything I see in the log, and the temperatures are totally off, depenting on wether the oven was on or off when the problem happened. I have to go back from the beta version to the one before since I'm not there when it happens. I'll let you know if anything changes by that. Is there anything else I can do to find out what's happening? I have a Tellstick.Net there as well as a backup, and I see that it's more than 20 degrees in some of the rooms, where it should have been around ten. But nothing is dangerous yet, so I can let it run for a few hours and wait for an answer from you before I replace the ONH plug-in version and restart EG.

[krambriw]

As you suggest, try to go back to previous version. For the moment I cannot think of anything changed that could cause this. Only thing changed was that it now correctly reports lost sensors of type fineoffset. You can verify with a compare documents (with UltraEdit or other tool)

[Mastiff]

Solved! I think...or at least hope. I suddenly understood the problem, I believe. It wasn't ONH, it was CDC. Or more like a combination. I remembered that I had upgraded the CDC plug-in as well, and I saw that it showed the queue size, which it shouldn't unless it's above 90 % after your latest update. I suddenly discovered I had not changed the max queue size. The standard is 15, and that's not enough for when I run the reports on my VM's. So the reason for the lockup was actually the CDC. But the combination comes in that before ONH was updated, everything else kept ticking along when the max queue size was reached. Now it's only doing the ONH stuff that is shown above. But I don't see any reason to look for the reason for that, I'm probably the only one in the world who runs those two plugins together on a computer that needs a substantial max queue size not to lock up!

[krambriw]

16.02.06 10:43:03 TCPCommandPort Connection from 192.168.2.100
16.02.06 10:43:02 TCPCommandPort Connection from 192.168.2.100

You have 2 connections to the TCP port?

[krambriw]

well, yeah

that's not enough

when resources are out, it's hard to tell what parts will be affected

[Mastiff]

Well, I now have set the queue to a ridiculous amount (1000), so this should not happen again.

[Mastiff]

This is incredible! Murphy strikes with a vengeance here! In exactly one week I'm locking the door of this house for good, and throwing the key to the new owner. Everything has been running totally smooth for a long time now (everything essential, at least), and he's very thrilled with the smarthome stuff. He's not a techie, but I am keeping a TeamViewer access to the system, and I am building a server clone, so he can move the cables over and be up and running within 20 minutes if something really bad happens. Hardware bad, that is. I thought I had killed all the software gremlins. A few days ago, when it was the hottest day this summer, 26 degrees and sun all day long, I thought I found the last hardware gremlin. Suddenly the RFX started acting up, but I found the problem quite soon: The box in the attic where the USB hub sits (protection against chewing mice) was too hot when it got over 50 degrees up there. So I cut a hole in the top with a wire mesh to close it to those mice, and the problem was solved. The good thing is that the last year I've had all kinds of weather, and I have only found this one problem. Until yesterday or today. Here comes the problem:

Every 20 minutes or so EG looses the connection to NHS. A reset of the plugin (which I have automated, of course - self healing is really the only way to go) usually fixes it, but every now and then it's necessary to restart NHS and EG. I run them both on the same VM, and it makes no difference if I use 127.0.0.1 or 192.168.0.100. I have not done anything to the system that should affect this connection at all. I tried to switch control port from 8005 to 8999 to see if that was the problem, and it went down from evvery 10 to around every 20 minutes, but that may of course have been a total coincidence. Part of the time it seems like it happens on 01 and 31 minutes past the hour, but that may also be a total fluke. Do you have any idea what this can be? I know that it's not NHS loosing the Tellstick, because that would have given another error message, one that I have set to restart EG, NHS and Girder at once if it happens. That is maybe once a week or so. The error message I get is:

Code: Select all

OpenNetHome.Connection with NetHomeServer TCP interface failed

OpenNetHome.Connection with NetHomeServer lost

It can happen seconds after the previous sensor was captured, so it's not anything that times out. I'm running exactly the same versions of EG, NHS and the plug-in as I have been running totally stable for long. Any ideas would be very much appreciated!

[krambriw]

This is because of a socket error, something with your network (card?)

Do you run the latest version of NetHomeServer? I know Stefan has improved the communication channel that we are using for the EG plugin. I think it is very stable using 127.0.0.1 but more likely to be disturbed when connecting over the network.

I cannot see anything wrong in the plugin code, the behavior is correct if you would have networking problems. The two error messages you get are related to the same problem. There is something with the networking that is not working perfect. Even if you use 127.0.0.1 networking will be used for communication

What timeout value do you have for the socket connection?

[Mastiff]

Thanks for answering so quick! Seems like I'm not the only one who's up late today, even if I'm trying to sleep now, but I did see the phone blink about the mail about this answer and another warning about a reset of the plugin. But it' hard to sleep with a thousand things to think about and finish. Not just technical, we are moving from a big house to our cabin, and we are gettting rid of 80 % of all the stuff we have, and the rest is moved one trip at a time. I really hope it isn't one of those mystical hardware problems that are impossible to find, and then goes away and comes back at a later time. The guys at the garbage duump have started saying: "Again?!" So it's typical that the network should act up now... I was running 2.1-SNAPSHOT-8fde5, but I downloaded the latest nightly build and now have 2.1-SNAPSHOT-554b0. I have no idea how many versions are between them, though. the build numbering is a mystery to me.

I have two network cards in to the VM, one for outside my server (for access to the webserver from outside) and one for the internal LAN (for just about everything else). I tried earlier today to remove the outer one, and that didn't change anything. I can't really remove the internal one, it would cripple my setup completely. But do you think it's a network card problem, or can it be some device on the network sending rogue packets or something (if that's even a thing, I remember that I had a faulty switch maybe ten years ago that messed up everything, even when the packets shouldn't have been routed through it). Maybe this could be another one of those deals.

I have a 60 seconds timeout for socket connections, but as I mentioned I had one of those lost contact things only a few seconds after the previous successfully received package, so I don't think that should be related.

[Mastiff]

Oh, btw, the network card in quetion is an Intel I210 on the Asus motherboard. I have not changed antyhing there either, with drivers or stuff like that.

[krambriw]

This might be tricky to track down but for the moment I am still believing there is a socket error problem. When this happen, the plugin is also trying to reconnect to ONH but obviously failing

If you look into the plugin code (in my code at line 1325) it looks like this

Code: Select all

                except socket.error, e:
                    #print 'socket.error', self.atStart
                    self.connectionError = True

We could try to see if we can get more info about the error by changing to this

Code: Select all

                except socket.error, e:
                    print 'socket.error', self.atStart, e
                    self.connectionError = True