[Solved] parsing logifles - suggestions - VisualCron

Please note that VisualCron support is not actively monitoring this community forum. Please use our contact page for contacting the VisualCron support directly.

Welcome Guest! To enable all features please Login or Register.

osirisja
Free support Topic Starter

2011-12-23T12:14:36Z

Hi all

Wondering if anyone has any recommendations on the best way to use VC to Parse Log Files created by third party applications?

For example, if we have a rolling log file containing log entries for multiple services over a period of 30 days. I only want to query a particular service (e.g. SVC001B) for the current day, and where the Log entry contains the word 'ERROR'. I then want to send this (or these) Records via Email.

What is the best way to achieve this? Using a script? Or are there methods that can be better utilised within VC to achieve this?

Looking forward to hearing suggestions

Cheers

Andy

Edited by moderator 2011-12-27T13:03:08Z | Reason: Not specified

Sponsor

Forum information

Wanna join the discussion?! Login to your Forum forum account or Register a new forum account

Support
Official support

2011-12-23T12:36:54Z

You are talking about reading an event log right?

Henrik
Support
http://www.visualcron.com
Please like VisualCron on facebook!

osirisja
Free support Topic Starter

2011-12-23T12:58:39Z

Hi Henrik

Not exactly - as in not a Windows Event Log. I am talking about a system log file similar to the Visual Cron Server log file, but from a different application, but essentially, it is the same thing.

For example - the entries below are from a VC Log file, but I have amended it slightly to more accurately reflect the type of logfile I am talking about:

===============================================================================
22/12/2011 09:10:35 SVC099 Info Task started: read log file (5410)
22/12/2011 09:10:36 SVC099 Err Exception in Task: Could not find file 'C:\@@Trigger\Cutas\Log\20111221_C018CNF01.TXT.LINT'.
22/12/2011 09:10:36 SVC099 Info Task completed: read log file (5410)
22/12/2011 09:10:35 SVC001 Info Task started: read log file (5410)
22/12/2011 09:10:36 SVC001 Err Exception in Task: Could not find file 'C:\@@Trigger\Cutas\Log\20111221_C018CNF01.TXT.LINT'.
22/12/2011 09:10:36 SVC001 Info Task completed: read log file (5410)
21/12/2011 09:10:35 SVC001 Info Task started: read log file (5410)
21/12/2011 09:10:36 SVC001 Err Exception in Task: Could not find file 'C:\@@Trigger\Cutas\Log\20111221_C018CNF01.TXT.LINT'.
21/12/2011 09:10:36 SVC001 Info Task completed: read log file (5410)
================================================================================

In the above Example, I only want to list the entries for today (e.g. 22/12/2011), that are flagged as an error (Err) and only for the service 'SVC001'.

So I would get in my 'Read File' entry only the following record:

================================================================================
22/12/2011 09:10:36 SVC001 Err Exception in Task: Could not find file 'C:\@@Trigger\Cutas\Log\20111221_C018CNF01.TXT.LINT'.
================================================================================

I can then Email this error to the system administrator.

Hopefully that makes sense?

Cheers

Andy

Support
Official support

2011-12-23T15:17:39Z

You probably need to write a code snippet in .NET Execute Task or create a batch file that does the same. It seems to specific to automate by itself - it require some extra, hands on, programming.

Henrik
Support
http://www.visualcron.com
Please like VisualCron on facebook!

osirisja
Free support Topic Starter

2011-12-23T16:45:41Z

Hi Henrik

Any pointers on where I can find some .NET references to help me? I'm afraid i'm very new to scripting especially in .NET.

Cheers (and have a great festive holiday)

Andy

Support
Official support

2011-12-23T23:23:29Z

You may have a lot of reading in front you. The System.IO namespace is about reading files.

But, I do recommend that you use vworker.com or similar. You will find a coder that do this for you for $10.

Henrik
Support
http://www.visualcron.com
Please like VisualCron on facebook!

ErikC
Free support

2011-12-27T08:24:59Z

Hi Andy,

Due to the fact that it's Christmas, I wrote the code for you: 😁

using System;
using System.IO;
using System.Text;

public class Test
{
	public static string readlogfile(string logfile)
	{
		// Use this variable to store all the ERR log lines form the logfile
		StringBuilder sb = new StringBuilder();

		try
		{
			using (StreamReader sr = new StreamReader(logfile))
			{
				String line;

				// Read and display lines from the file until the end of 
				// the file is reached.
				while ((line = sr.ReadLine()) != null)
				{
					// Get the log type
					string type = line.Split(new string[] {"\t"},StringSplitOptions.None)[1];

					// Add the line if it's an error
					if(type.ToUpper()=="ERR")
						sb.AppendLine(line);
				}
			}
		}
		catch (Exception e)
		{
			// Let the user know what went wrong.
			sb.AppendLine("The file could not be read:");
			sb.AppendLine(e.Message);
		}

		// Return all the lines
		return sb.ToString();
	}
}

The variable 'logfile' is the full path to the VC log file (log_serveryyyymmdd.txt).

!! Beware !!
The current logfile is in use by Visualcron itself and therefore cannot be openened.

Regards,
Erik

Edited by user 2011-12-27T08:43:39Z | Reason: Not specified

Uses Visualcron since 2006.

ErikC
Free support

2011-12-27T08:46:15Z

Support wrote:
But, I do recommend that you use vworker.com or similar. You will find a coder that do this for you for $10.

There is also this site: http://fiverr.com/
Worth looking at.

Regards
Erik

Uses Visualcron since 2006.

osirisja
Free support Topic Starter

2011-12-27T10:45:27Z

Hi Erik

Thanks so much for this. I will examine the code and adapt where needed. Is this VB .NET? (although I see little difference between VB and C#). I also guess I can extend the logic 'if(type.ToUpper()=="ERR")' to check for multiple values, such as a particular date and a specific process? e.g.

if(type.ToUpper()=="ERR") & (type.ToUpper()=="22/12/2011") & (type.ToUpper()=="ST001")

Merry Xmas and a very good new year :-)

Cheers

Andy

ErikC
Free support

2011-12-27T11:38:18Z

#10

Hi Andy,

The code is in c# and what I did is splitting one line up in parts by using the 'tab' key as the part delimiter.

// Get the log type
string type = line.Split(new string[] {"\t"},StringSplitOptions.None)[1];

Your logfile differs from the VC logfile and it might not use the 'tab' as a delimiter. It might use a space.

You coude could be:

// Get the service and the log type
string service = line.Split(new string[] {" "},StringSplitOptions.None)[2];
string type = line.Split(new string[] {" "},StringSplitOptions.None)[3];

if(service.ToUpper() == "SVC001" && type.ToUpper() == "ERR")
{
    sb.AppendLine(line);
}

Regards,
Erik

Edited by user 2011-12-27T11:39:16Z | Reason: Not specified

Uses Visualcron since 2006.

osirisja
Free support Topic Starter

2011-12-27T12:00:08Z

#11

Hi Erik

Ahhh - that just all clicked in to place there - makes total sense to me now :-)

I've been putting off learning C# but think I need to bite the bullet and get right into it now I think. Thanks for giving me the insight.

Cheers

Andy