Author Topic: Find missing files  (Read 609 times)

Per E. Johannessen

  • Full Member
  • ***
  • Thank You
  • -Given: 5
  • -Receive: 1
  • Posts: 178
  • Karma: +1/-0
    • View Profile
Find missing files
« on: April 18, 2021, 07:04:23 pm »
Hi,

I have some directories with lots of pdf-files like this:
10001.pdf
10002.pdf
10003.pdf
10005.pdf
The files are supposed to be consecutively numbered and this case only four files in the directory and it's easy to see that 10004.pdf is missing.
However, in a directory with thousands of files it's hard to manually find the missing ones.

Does anyone know of a program that takes "start file number ie. 10001.pdf" and "end file number ie. 10500.pdf" as
parameters and lists the missing ones?


David Graser

  • Hero Member
  • *****
  • Thank You
  • -Given: 0
  • -Receive: 81
  • Posts: 661
  • Karma: +22/-0
    • View Profile
Re: Find missing files
« Reply #1 on: April 18, 2021, 09:04:01 pm »
Unfortunately, it appears that Seek and Scan does not recognize wildcards.

However, using using SearchPlus by Keith Merrington, I had no problem finding a file(s) named as such with the following parameters.

1000*.pdf




Andreas Schnellbacher

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 36
  • Posts: 691
  • Karma: +7/-0
    • View Profile
Re: Find missing files
« Reply #2 on: April 18, 2021, 09:25:26 pm »
Does anyone know of a program that takes "start file number ie. 10001.pdf" and "end file number ie. 10500.pdf" as
parameters and lists the missing ones?
Most likely you have to script that. Not too difficult. May even possible with a non-REXX and non-4OS2 script.

Per E. Johannessen

  • Full Member
  • ***
  • Thank You
  • -Given: 5
  • -Receive: 1
  • Posts: 178
  • Karma: +1/-0
    • View Profile
Re: Find missing files
« Reply #3 on: April 19, 2021, 07:55:49 pm »
Thanks guys,

David;
I need to find out which files are missing so I think the parameter 1000*.pdf would simply list files numbered
10001.pdf up to 10009.pdf and not tell me that ie. 10004.pdf is missing

Andreas:
I believe REXX can do it, (but can I do it? ;) )

Andreas Schnellbacher

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 36
  • Posts: 691
  • Karma: +7/-0
    • View Profile
Re: Find missing files
« Reply #4 on: April 19, 2021, 08:35:23 pm »
OK, I'll try that. I guess that might be useful, it works on one dir only:

Parameters: <Dir> <StartName> <EndName> <LogFile>

John Hollemans

  • Newbie
  • *
  • Thank You
  • -Given: 0
  • -Receive: 0
  • Posts: 1
  • Karma: +0/-0
    • View Profile
Re: Find missing files
« Reply #5 on: April 21, 2021, 05:09:43 am »
_____
Hello

The easiest wat yo find your files is to use a "standard" DIR command like this:
"DIR C:\1000*.PDF /S"

You may use the "*" to generalise the search more if you like.  This command is found in the HELP pages fir the DIR command.  I fins that it is extremely fast.  If the list is of found files is very long, just use the "less" command like this:
"DIR C:\1000*.PDF /S | less"

Regards / John

Lars

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 76
  • Posts: 791
  • Karma: +23/-0
    • View Profile
Re: Find missing files
« Reply #6 on: April 21, 2021, 09:02:23 am »
You will need a REXX script. Use the "dir" command with the filename sorting option and pipe the command output into RXQUEUE. Then, set a counter to 1 and then, going through all files, construct an expected filename from the counter value and compare to real filename. If no match, issue warning. Increase counter by one and go to next file.

Andreas Schnellbacher

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 36
  • Posts: 691
  • Karma: +7/-0
    • View Profile
Re: Find missing files
« Reply #7 on: April 21, 2021, 06:09:40 pm »
Here's my script, roughly tested:

Code: [Select]
/****************************************************************************
* findmissingfilenames.cmd
*
* Version: 2024-04-21
*
* Syntax: findmissingfilenames <filemask> <startnum> <endnum> [<logfile>]
*
* o  <filemask> must have a part with '???' chars, where the amount of it
*    depends on the range <startnum> to <endnum>.
* o  <filemask> may include a path.
* o  <startnum> and <endnum> must be numbers.
* o  <logfile> may optionally be specified. The default <logfile> is
*    'missing.log'.
* o  Every arg may be surrounded with double quotes.
*
* Examples:
*    findmissingfilenames 1????.pdf 1 500
*    findmissingfilenames 10???.pdf 1 500
*
* By Andreas Schnellbacher
****************************************************************************/
/* Some header lines are used as help text */
HelpStartLine = 6
HelpEndLine   = 18

'@ECHO OFF'
CALL SETLOCAL

/* ----------------- Standard CMD initialization follows ----------------- */
SIGNAL ON HALT NAME Halt
SIGNAL ON NOVALUE NAME RexxError
SIGNAL ON SYNTAX NAME RexxError
SIGNAL ON FAILURE NAME RexxError
/*
SIGNAL ON ERROR NAME RexxError     /* Causes doubled error messages */
SIGNAL ON NOTREADY NAME RexxError  /* Causes error at encoding */
*/

env   = 'OS2ENVIRONMENT'
TRUE  = (1 = 1)
FALSE = (0 = 1)
CrLf  = '0d0a'x
Redirection = '>NUL 2>&1'
PARSE SOURCE . . ThisFile
GlobalVars = 'env TRUE FALSE CrLf Redirection ERROR. ThisFile'

/* Some OS/2 Error codes */
ERROR.NO_ERROR           =   0
ERROR.INVALID_FUNCTION   =   1
ERROR.FILE_NOT_FOUND     =   2
ERROR.PATH_NOT_FOUND     =   3
ERROR.ACCESS_DENIED      =   5
ERROR.NOT_ENOUGH_MEMORY  =   8
ERROR.INVALID_FORMAT     =  11
ERROR.INVALID_DATA       =  13
ERROR.NO_MORE_FILES      =  18
ERROR.WRITE_FAULT        =  29
ERROR.READ_FAULT         =  30
ERROR.SHARING_VIOLATION  =  32
ERROR.GEN_FAILURE        =  31
ERROR.INVALID_PARAMETER  =  87
ERROR.ENVVAR_NOT_FOUND   = 204

rc = ERROR.NO_ERROR

CALL RxFuncAdd 'SysLoadFuncs', 'RexxUtil', 'SysLoadFuncs'
CALL SysLoadFuncs
/* ----------------- Standard CMD initialization ends -------------------- */

GlobalVars = GlobalVars 'rc LogFile HelpStartLine HelpEndLine'

DO once = 1 TO 1

   DefaultLogFile = 'missing.log'

   /* Parse args */
   PARSE ARG Args
   Args = STRIP( Args)
   
   /* Show help */
   IF Args = '' THEN
      CALL ShowHelp
   ELSE
   DO
      IF WORDPOS( Args, '-? /?') > 0 THEN
         CALL ShowHelp
   END     
   
   /* Parse and check arg words */
   rest = Args
   LogFile = ''
   a = 0
   DO WHILE rest <> ''
      a = a + 1
      rest = STRIP( rest)
      IF LEFT( rest, 1) = '"' THEN
         PARSE VAR rest '"'next'"' rest
      ELSE
         PARSE VAR rest next rest
      SELECT
         WHEN a = 1 THEN
         DO
            FileMask = next
            IF FileMask = '' THEN
            DO
               rc = ERROR.INVALID_PARAMETER
               CALL ExitError 'Error: filemask not specified.'
            END
         END
         WHEN a = 2 THEN
         DO
            StartNum = next
            IF StartNum = '' THEN
            DO
               rc = ERROR.INVALID_PARAMETER
               CALL ExitError 'Error: startnum not specified.'
            END
            IF DataType( StartNum) <> 'NUM' THEN
            DO
               rc = ERROR.INVALID_PARAMETER
               CALL ExitError 'Error: startnum is not a number.'
            END
         END
         WHEN a = 3 THEN
         DO
            EndNum = next
            IF EndNum = '' THEN
            DO
               rc = ERROR.INVALID_PARAMETER
               CALL ExitError 'Error: endnum not specified.'
            END
            IF DATATYPE( EndNum) <> 'NUM' THEN
            DO
               rc = ERROR.INVALID_PARAMETER
               CALL ExitError 'Error: endnum is not a number.'
            END
         END
         WHEN a = 4 THEN
            LogFile = next
      OTHERWISE
         rc = ERROR.INVALID_PARAMETER
         CALL ExitError 'Error: Too many args specified.'
      END
   END
   IF LogFile = '' THEN
      LogFile = DefaultLogFile
     
   /* Find number of contiguous '?' chars in FileMask */
   NumPartLen = 0
   NumPartPos = 0
   pStart = 1
   PrevChar = '?'
   DO n = 1 TO LENGTH( FileMask)
      ThisChar = SUBSTR( FileMask, n, 1)
      IF ThisChar <> '?' THEN
         ITERATE
      IF PrevChar <> '?' THEN
         LEAVE
      /* Save pos of first '?' char */   
      IF NumPartLen = 0 THEN   
         NumPartPos = n
      /* Advance length */   
      NumPartLen = NumPartLen + 1
      PrevChar = ThisChar
   END   
   
   /* Check number of contiguous '?' chars in FileMask */
   IF NumPartLen < LENGTH( StartNum) THEN
   DO
      rc = ERROR.INVALID_PARAMETER
      CALL ExitError 'Error: Number of ''?'' chars in filemask too low for startnum.'
   END
   IF NumPartLen < LENGTH( EndNum) THEN
   DO
      rc = ERROR.INVALID_PARAMETER
      CALL ExitError 'Error: Number of ''?'' chars in filemask too low for endnum.'
   END
   
   /* Delete old LogFile */
   IF FileExist( LogFile) THEN
      CALL SysFileDelete LogFile

   /* Process range of numbers */
   m = 0
   f = 0
   DO n = StartNum TO EndNum
      /* Build Filename */
      NumPart = RIGHT( n, NumPartLen, '0')
      Filename = OVERLAY( NumPart, FileMask, NumPartPos, NumPartLen)
      f = f + 1
      /*SAY RIGHT( f, NumPartLen)': 'Filename*/
     
      /* Search filename */
      IF \FileExist( Filename) THEN
      DO
         m = m + 1
         /* Write to LogFile */
         CALL WriteLog 'Missing: 'Filename
      END
   END
   
   /* Close LogFile if opened */
   IF STREAM( LogFile) = 'READY' THEN
      CALL STREAM LogFile, 'C', 'CLOSE'
   
   /* Report number of missing files */
   IF m = 0 THEN
      SAY 'All 'f' numbers exist.'   
   ELSE
      SAY m' file(s) of 'f' are missing. See 'LogFile'.'

END

EXIT( rc)

/* ======================================================================= */

/* ----------------------------------------------------------------------- */
FileExist: PROCEDURE EXPOSE (GlobalVars)
   rc = ERROR.NO_ERROR

   PARSE ARG Filename

   IF FileName = '' THEN
      RETURN( 0)
   ELSE
      RETURN( STREAM( Filename, 'C', 'QUERY EXISTS') <> '')

/* ----------------------------------------------------------------------- */
WriteLog: PROCEDURE EXPOSE (GlobalVars)
   PARSE ARG Message
   
   StreamState = STREAM( LogFile)
   IF WORDPOS( StreamState, 'UNKNOWN NOTREADY') > 0 THEN
   DO
      StreamState = STREAM( LogFile, 'C', 'OPEN')
      PARSE VAR StreamState StreamState':'
   END
   IF StreamState = 'READY' THEN
      CALL LINEOUT LogFile, Message
     
   RETURN( '')   

/* ----------------------------------------------------------------------- */
ShowHelp: PROCEDURE EXPOSE (GlobalVars)
   DO l = HelpStartLine TO HelpEndLine
      SAY SUBSTR( SOURCELINE( l), 3)
   END
   EXIT rc

/* ----------------------------------------------------------------------- */

ExitError: PROCEDURE EXPOSE (GlobalVars)
   PARSE ARG Message
   CALL ShowError Message
   EXIT rc   

/* ----------------------------------------------------------------------- */
/* Show error message in a message box and/or write to STDERR */
ShowError: PROCEDURE EXPOSE (GlobalVars)
   PARSE ARG Message
   CALL LINEOUT 'STDERR', Message

   RETURN( '')

/* ----------------------------------------------------------------------- */
Halt:
   CALL ShowError 'Interrupted by user.'
   EXIT( 99)

/* ----------------------------------------------------------------------- */
/* Give a standard REXX error message. */
/* This is for REXX error conditions only. */
/* System error codes and REXX error codes are different. */
RexxError:
   /* SIGL must be saved to not get overwritten later. */
   ErrorLine = SIGL

   /* As an extension to the standard REXX error messages,    */
   /* the error condition will be appended to the error text. */
   ConditionText = 'Condition: 'CONDITION( 'C')
   ConditionDescription = CONDITION( 'D')
   IF ConditionDescription <> '' THEN
      ConditionText = ConditionText', Reason: 'ConditionDescription

   ErrText = ''
   IF SYMBOL( 'rc') = 'VAR' THEN
   DO
      IF rc > 0 & rc < 100 THEN
         ErrText = ERRORTEXT( rc)
   END
   IF ErrText = '' THEN
      ErrText = ConditionText
   ELSE
      ErrText = ErrText', 'ConditionText

   /* Ensure that rc is set and that rc <> 0 is returned */
   IF SYMBOL( 'rc') = 'VAR' THEN
   DO
      IF \( rc > 0 & rc < 100) THEN
         rc = 999
   END
   ELSE
      rc = 999

   ErrorMessage = ''
   IF ErrorLine > 0 THEN
      ErrorMessage = RIGHT( ErrorLine, 6)' +++  'SOURCELINE( ErrorLine)
   IF ErrorMessage <> '' THEN
      ErrorMessage = ErrorMessage''CrLf
   ErrorMessage = ErrorMessage ||,
      'REX'RIGHT( rc, 4, 0)': Error 'rc' running 'ThisFile', line 'ErrorLine': 'ErrText
   CALL ShowError ErrorMessage

   EXIT( rc)


Per E. Johannessen

  • Full Member
  • ***
  • Thank You
  • -Given: 5
  • -Receive: 1
  • Posts: 178
  • Karma: +1/-0
    • View Profile
Re: Find missing files
« Reply #8 on: April 21, 2021, 08:18:49 pm »
Thanks a lot Andreas, works like a dream.

Andreas Schnellbacher

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 36
  • Posts: 691
  • Karma: +7/-0
    • View Profile
Re: Find missing files
« Reply #9 on: April 21, 2021, 09:35:09 pm »
You're welcome.

That's almost the style from Christian. In 2002 and a few years later I had the chance to work on the same project with him and had learned a lot about programming styles and projects.