• Welcome to OS2World OLD-STATIC-BACKUP Forum.
 

News:

This is an old OS2World backup forum for reference only. IT IS READ ONLY!!!

If you need help with OS/2 - eComStation visit http://www.os2world.com/forum

Main Menu

Remove multiple occurances from a log file

Started by jep, 2010.01.08, 14:06:58

Previous topic - Next topic

jep

Marked as: Easy
Hi,

This script display the output from the log it read through StdErr and not the usual StdOut, so you can redirect the output to another file if you like.  ;D

Purpose (script):
Remove duplicate entries and display only one to the user. Output data as it appear in the log file.

Prerequisite:
A file that contain a lot of different information repeated over several times at various locations within the file. Each section has to begin with the text "ERROR:" or "WARNING:" (without the quotes).

Purpose (for you to learn about):
Difference between StdErr and StdOut and how to redirect output.
See Rexx Tips & Tricks for more information.

Usage (redirect errors to new log file named cleaned.log):
read_log my_log_with_duplicates.log 2 > cleaned.log

Output (screen):
Reading file: my_log_with_duplicates.log
File my_log_with_duplicates.log has been read

Output (log file "cleaned.log"):
Contents of the log file cleaned to only contain one occurrence of each log entry (See further sown for example).
Usage (all output to screen):
read_log my_log_with_duplicates.log


/* Short test script for the function below (read_log.cmd) */
Say 'Reading file:' ARG(1)
retval = parse_log( ARG(1) )
SAY 'File 'ARG(1)' has been read'
Return retval

parse_log: PROCEDURE
   IF ARG(2) <> '' THEN
       counter = counter + 1
   ELSE counter = 0
   IF STREAM( ARG(1), 'C', 'QUERY EXISTS' ) = '' THEN
       Return 4
   ELSE IF \STREAM( ARG(1), 'C', 'READABLE' ) THEN
   DO
        /* Log file could not be read */
       CALL SysSleep 1
        /* try for 5 minutes (expressed in seconds) */
       IF counter < 60 * 5 THEN
           CALL parse_log ARG(1), 1
       ELSE
           Return 5
   END
   f_size = STREAM( ARG(1), 'C', 'QUERY SIZE' )
   IF f_size > 0 THEN DO
       f_input = CHARIN( ARG(1), 1, f_size )
       CALL STREAM ARG(1), 'C', 'CLOSE'
       isProblem = 0
       delim.1 = 'ERROR:'
       delim.2 = 'WARNING:'
       delim.0 = 2
       DO i = 1 TO delim.0
           isDelim = POS( delim.i, f_input )
           isProblem = ( isProblem | ( isDelim > 0 ) )
           DO WHILE isDelim > 0
               INTERPRET 'PARSE VALUE f_input WITH "'||delim.i||'"pre"'||delim.i'"post'
               IF LENGTH( post ) > 0 THEN f_input = delim.i||post
               IF LENGTH( STRIP( pre ) ) > 0 THEN
               DO
                   pre = delim.i||pre
                   CALL LINEOUT 'STDERR', STRIP( pre )
                   count = COUNTSTR( pre, f_input )
                   DO count
                       f_input = DELSTR( f_input, POS( pre, f_input ), LENGTH( pre ) )
                   END
               END
               isDelim = POS( delim.i, f_input )
           END
       END
   END
Return isProblem


//Jan-Erik

Example file:
WARNING:
Please check the power cable

ERROR:
This is an error cause by a faulty drive...

ERROR:
This is an error cause by a faulty drive...

WARNING:
Please check the power cable

ERROR:
This is an error cause by a faulty drive...

ERROR:
This is an error cause by a faulty drive...

WARNING:
Please check the power cable

WARNING:
Please check the power cable


Output:
ERROR:
This is an error cause by a faulty drive...
WARNING:
Please check the power cable