Monitor log file for error

[ijourneaux]

I noticed there were a couple of directory/file monitoring plug-ins. I was wondering if there might be one that would allow monitoring a log file for certain error strings.

I suppose I could monitor the log file and create a that could be monitored by the file plugin.

[jinxdone]

This sounds like a good idea for a plugin. In the plugin's configuration options you should have the file name(s) to monitor, and regexp strings to watch for. If a monitored string is found it should raise an event like LogMonitor.SomeWatchedEvent with payload being the log entry.. It would be simple but functional.

Is this what you had in mind as well? It should be pretty straight forward to create a plugin that does this.

-jindxone

[Pako]

You can very well use the plugin "File Operations" and his action "Start periodical reading":

Once into the file is writed a new line, is released an event that carries a payload.
Payload is content of a new line.
Just then write a simple script that will scan the contents of payload.
Pako

[jinxdone]

It would be very useful to combine the FileOperations plugin with the DirectoryWatcher (or perhaps a filewatcher plugin...), that way you don't need to use the periodic read in FileOperations (which by the way I don't see how it's ever useful.. seems like a "brute force" way of doing things).

Only thing stopping you from using the DirectoryWatcher in combination with the FileOperations is that currently there is no way to pass the FileOperations "read from file" -action a variable as the filename. ex. {eg.result}

Other than that the FileOperations plugin seems to be very handy. I hadn't used it before.

[ijourneaux]

Thanks everyone for the comments and interest. In terms of monitoring a file, the only special thing is that you would have to monitor log files that are opened daily. Last created or alphabetic would work just fine.

I will explore Pako's comments more over the next week.

Creating a Tail type addin for EventGhost would probabably be a great addition. As you could monitor log files for codes and control your computer based on what appears in the log file.

[jinxdone]

You can choose whether to read from the beginning or the end of the file in the FileMonitor. If you choose end it should work like tail..

[ijourneaux]

this is likely exactly what i was looking for.

i did notice what may be a couple of bugs.

If or disable it, it stll keeps running. see attachment. i disabled the plugin and stll got an event

if you make changes to the plugin configuration, the settings aren't applied until after you restart eventghost. perhaps related?

a couple of other questions
- how would i monitor the newset log file in a directory? in my case the log files are xxx2010-4-12.log, xxx2010-4-11.log, etc. I only need to monitor the latest one
- i am still getting my feet wet with eventghost? are there any examples f creating a script similiar to what i need to analyze the payload from fileoperations?

[Pako]

If or disable it, it stll keeps running. see attachment. i disabled the plugin and stll got an event

You must use the action "Abort periodical reading" or "Abort all periodical reading"

- how would i monitor the newset log file in a directory? in my case the log files are xxx2010-4-12.log, xxx2010-4-11.log, etc. I only need to monitor the latest one

Please download a new version of the plugin "File Operations" from the SVN. I was there did a very simple but important improvements. Now you can write (like file to observation) not just a specific file, but also a Python expression (such {eg.result} or {eg.event.payload}).

i am still getting my feet wet with eventghost? are there any examples f creating a script similiar to what i need to analyze the payload from fileoperations?

An example of a script I can write you, but I need an example of content logs.
Pako

[ijourneaux]

ok got the latest version of file operations from the svn. Got the concept of aborting periodic reading. Thanks

The application who's log file I want to monitor creates a new log file every day in the format Application-yyyy-mm-dd.log so I would need to be able to select the newest log file not just specify a specific log file.

I have attached a sample log file. The error code I want to respond to is

Adding error: '12'

that occurs at 2:00:02

When I see that error message, i want to output an IR code via the Transmit IR function in the MCE Remote plugin to turn a device off then turn the device back on in a couple of minutes.

[Pako]

I have looked at your sample log file. I found that sometimes it is written in short succession a few lines.
It follows that we must always check the last few lines - for example 50. This presents a complication,
because it is necessary to ensure that the same error was not processed more than once.
I created two versions of the complete solution. In the first solution is to use periodical reading,
but the second version the periodical reading not used it.
For the second solution I had to make a small change of plugin File Operations,
so you need to download the new version.

Pako

[ijourneaux]

Ok still trying the first version. I changed pth so that it pointed to

pth = "c:\Temp\LOG"

In that folder, I have a collection of log files BTV-2010-04-15.log to BTV-2010-04-12.log

I am getting an error

Python script "16", line 10, in <module>
lastLogFile = files[-1][1]
IndexError: list index out of range

on line 10

lastLogFile = files[-1][1]

[Pako]

It is strange to me, it works.
Can you please try this modified script,

Code: Select all

pattern = "BTV-XXXXXXXXXX.log"
pth = r"c:\Temp\LOG"

from os import path, listdir
#Find all the log files that match the pattern:
files = [(path.getmtime(path.join(pth,f)), path.join(pth,f)) for f in listdir(pth) if (f[:4] == pattern[:4] and f[-4:] == pattern[-4:])]
print "listdir =",listdir(pth)
print "filtered listdir =",[f for f in listdir(pth) if (f[:4] == pattern[:4] and f[-4:] == pattern[-4:])]
print "files =",files
#sort log sort files by time changes:
files.sort()
#last changed file:
lastLogFile = files[-1][1]
#if file exist, trigger event
if lastLogFile:
        eg.TriggerEvent("lastLogFile",prefix = "BTV",payload = (lastLogFile,))

and then copied here three lines of logger, arising out of the print commands (listdir =, filtered listdir = and files =)?
Pako

[ijourneaux]

The rror was caused by a case chagne in the log file name btv... versus BTV...

I confimed that the first approach seems to work. Will try the second one shortly,

[ijourneaux]

Looking at approach #1, The periodic read works as it should. Unfortuantely it seems to react to the error code even if it is not the last line in the log. Perhaps there needs to be a certain number of lines after the rror code so that it doesn't trigger again. I tried adding 10-15lines after the error and it still triggered the event.,\

Looking at approach #2
I can see that it is reading in many lines but it never seems to locate the matching string.

[Pako]

I do not understand, because of these problems, I concentrated and many times I've tried.
Please set EventGhost as shown in the picture:

EG_View_Settings.jpg (29.47 KiB) Viewed 6653 times

Then do the same test as before and copy here the contents of logger.
Please do the same with both versions of the configuration.
It is possible that you did not configure all the necessary adjustments. So please attach both your xml files.
Pako