OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Applications => Topic started by: Shmuel (Seymour J.) Metz on November 28, 2018, 07:56:29 pm

Title: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on November 28, 2018, 07:56:29 pm
I have a script on my ArcaOS system that I use to backup all of my logical drives, except those used by openSUSE, to a USB thumb drive. It works well except for drive M:, a rather large collection of manuals. The script uses zip, and when the archive file exceeds 4 GiB, unzip can't handle it. I tried using -s and concatenating the pieces, but that just changed the symptoms.

Plan B was p7zip, with the command 7za w:\temp\M.7z M:\

I get a message that no files were found and an empty archive. I tried adding an * to M:\* and I tried the -r option; neither helped. What is the correct syntax for creating an archive of the entire volume?

Note: except for FAT C: and D: dries for freedoms, everything is JFS.

FWIW, Arctool was able to create an 11 GiB archive of the volume; I'm not sure how large it would be with default compression.

Thanks.
Title: Re: Creating archive > 4GiB from script
Post by: Doug Bissett on November 28, 2018, 10:04:36 pm
I use ZIP to backup my drives. One of mt ZIP files is about 12 GB, and it contains, at least one file that is about 5 GB. Be absolutely sure, that you are using the ZIP, and UNZIP, as distributed by RPM/YUM. Older versions of ZIP/UNZIP do not support files larger than 2 GB in, or out. The latest ZIP.EXE (from RPM/YUM) has a necessary fix for the -S parameter (capital 'S', parameters are case sensitive). I use the parameters -9yrS to do my backups. I have restored drives (including boot drives), a number of times, so I know that it works.

I do use Archive tool to manage the UNZIP activity. It makes it easy to pick out individual files, or directories, or do the whole thing. Be sure to set extract Full paths, to put files back in the original locations.

A typical backup command, for a full drive, is:
Code: [Select]
zip.exe -9yrS CDRIVE.ZIP C:\* -x@exclude.list 2>&1 | tee.exe CZIP.LOG
ZIP.EXE is in the path, the parameters use maximum compression, handle symlinks correctly, recurse through directories, and include system and hidden files. The exclude.list file contains a list of files that you don't need to back up. The rest redirects all output to TEE.EXE (available from RPM/YUM), which shows the progress in the window, and sends that to the log file. I use it to back up live boot drives, and there has never been a problem with that. Of course, that may depend on what else is actually running.
Title: Re: Creating archive > 4GiB from script
Post by: Bogdan on November 28, 2018, 11:43:35 pm
I have a script on my ArcaOS system that I use to backup all of my logical drives, except those used by openSUSE, to a USB thumb drive. It works well except for drive M:, a rather large collection of manuals. The script uses zip, and when the archive file exceeds 4 GiB, unzip can't handle it.
unzip can definitely handle large files. Also the klibc based build can unarchive a compressed zip archive larger than 4GB. The problem with the private klibc build is a broken directory handling combined with file name and path length problems. Almost every program that is linked dynamically against libcxx has problems with root directories and codepages in general. But the LARGEFILE stuff is handled in a proper way on supported file systems. So in real world you can use it for single file zip archives larger than 4 GB without issues, but there will be problems with large directory sizes, amount of files or deep path structures.

Quote
I tried using -s and concatenating the pieces, but that just changed the symptoms.

Plan B was p7zip, with the command 7za w:\temp\M.7z M:\

I get a message that no files were found and an empty archive. I tried adding an * to M:\* and I tried the -r option; neither helped. What is the correct syntax for creating an archive of the entire volume?
A simple solution or workaround when files can cause trouble for crappy tools is to avoid the usage of files completely or to some degree. You can do simply a complete disk dump of this volume and piping it through a compressor to create a large file for recovery. Or you collect all names in the volume (by a dir or find utility) and check if names and path length can be handled by those tools you mentioned earlier.
Title: Re: Creating archive > 4GiB from script
Post by: Silvan Scherrer on November 29, 2018, 11:07:15 am
I have a script on my ArcaOS system that I use to backup all of my logical drives, except those used by openSUSE, to a USB thumb drive. It works well except for drive M:, a rather large collection of manuals. The script uses zip, and when the archive file exceeds 4 GiB, unzip can't handle it.
unzip can definitely handle large files. Also the klibc based build can unarchive a compressed zip archive larger than 4GB. The problem with the private klibc build is a broken directory handling combined with file name and path length problems. Almost every program that is linked dynamically against libcxx has problems with root directories and codepages in general. But the LARGEFILE stuff is handled in a proper way on supported file systems. So in real world you can use it for single file zip archives larger than 4 GB without issues, but there will be problems with large directory sizes, amount of files or deep path structures.

Quote
I tried using -s and concatenating the pieces, but that just changed the symptoms.

Plan B was p7zip, with the command 7za w:\temp\M.7z M:\

I get a message that no files were found and an empty archive. I tried adding an * to M:\* and I tried the -r option; neither helped. What is the correct syntax for creating an archive of the entire volume?
A simple solution or workaround when files can cause trouble for crappy tools is to avoid the usage of files completely or to some degree. You can do simply a complete disk dump of this volume and piping it through a compressor to create a large file for recovery. Or you collect all names in the volume (by a dir or find utility) and check if names and path length can be handled by those tools you mentioned earlier.
as you seem to know exactly whats wrong with those so called private builds (even I have no idea why those should be called like that), I wonder where your bug reports are.
Exactly such ports give me the feeling I should completely give up on either reading anything in this forum at all, or give up completely with porting software.
Just pointing to software which should not behave right, but not adding bug reports is not the way to go. Sorry
Title: Re: Creating archive > 4GiB from script
Post by: Bogdan on November 29, 2018, 04:20:32 pm
as you seem to know exactly whats wrong with those so called private builds (even I have no idea why those should be called like that), I wonder where your bug reports are.

There were several bugs fixed and patches will make it to the next release (https://sourceforge.net/p/infozip/bugs/search/). But I cannot find any information about which patches were applied by your private build.

Quote
Exactly such ports give me the feeling I should completely give up on either reading anything in this forum at all, or give up completely with porting software.
Sorry I don't realise your point of view. Zip and Unzip have been portable software for very loooooong time and OS/2 is still supported. About which kind of porting you're talking?

Quote
Just pointing to software which should not behave right, but not adding bug reports is not the way to go. Sorry
By providing private built binaries derived from Zip and Unzip sources you can at least point to your patches or modifications, so we can check the issues.
Title: Re: Creating archive > 4GiB from script
Post by: Dave Yeo on November 29, 2018, 05:14:55 pm

By providing private built binaries derived from Zip and Unzip sources you can at least point to your patches or modifications, so we can check the issues.

Hi Bogdan, I don't know if private builds is the best description as these builds ship with the latest releases of ArcaOS, perhaps forks is a better term.
While the Bitwise people do make some choices that others don't agree with, they are some of the few developers actually doing something on our platform and though they do make some money doing it, considering the amount of time and effort they put in, much of the work is basically on a volunteer basis and it is hard to bitch about people doing stuff for free, even if not perfect.
One of the decisions they've made is not to spend time trying to get their patches accepted up stream. Be nice if they did, but understandable as getting patches accepted can and usually is, time consuming.
They have been very good about following licenses including being public with their fixes and other patches with many hosted at netlabs and others at Github.
In the case of zip and unzip, their trees are at http://trac.netlabs.org/ports/browser/zip/trunk (http://trac.netlabs.org/ports/browser/zip/trunk) and http://trac.netlabs.org/ports/browser/unzip/trunk (http://trac.netlabs.org/ports/browser/unzip/trunk) with issue tracking also hosted at netlabs.
Title: Re: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on November 29, 2018, 08:35:15 pm
Be absolutely sure, that you are using the ZIP, and UNZIP, as distributed by RPM/YUM. Older versions of ZIP/UNZIP do not support files larger than 2 GB in, or out. The latest ZIP.EXE (from RPM/YUM) has a necessary fix for the -S parameter (capital 'S', parameters are case sensitive). I use the parameters -9yrS to do my backups. I have restored drives (including boot drives), a number of times, so I know that it works[/code]

From zip -v

Code: [Select]
Copyright (c) 1990-2008 Info-ZIP - Type 'zip "-L"' for software license.
This is Zip 3.0 (July 5th 2008), by Info-ZIP.
Currently maintained by E. Gordon.  Please send bug reports to
the authors using the web page at www.info-zip.org; see README for details.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip,
as of above date; see http://www.info-zip.org/ for other sites.

Compiled with emx+gcc 4.9.2 for OS/2 2.x/3.x (32-bit) on Sep 28 2018.

Zip special compilation options:
ASMV
USE_EF_UT_TIME       (store Universal Time)
SYMLINK_SUPPORT      (symbolic links supported)
LARGE_FILE_SUPPORT   (can read and write large files on file system)
ZIP64_SUPPORT        (use Zip64 to store large files in archives)
[encryption, version 2.91 of 05 Jan 2007] (modified for Zip 3)

Encryption notice:
The encryption code of this program is not copyrighted and is
put in the public domain.  It was originally written in Europe
and, to the best of our knowledge, can be freely distributed
in both source and object forms from any country, including
the USA under License Exception TSU of the U.S. Export
Administration Regulations (section 740.13(e)) of 6 June 2002.

Zip environment options:
             ZIP:  [none]
          ZIPOPT:  [none]
             EMX:  [none]
          EMXOPT:  [none]

From unzip -v

Code: [Select]
UnZip 6.00 of 20 April 2009, by Info-ZIP.  Maintained by C. Spieler.  Send
bug reports using http://www.info-zip.org/zip-bug.html; see README for details.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip/ ;
see ftp://ftp.info-zip.org/pub/infozip/UnZip.html for other sites.

Compiled with emx+gcc 4.9.2 for OS/2 (32-bit) on Jun 12 2018.

UnZip special compilation options:
        ASM_CRC
        COPYRIGHT_CLEAN (PKZIP 0.9x unreducing method not supported)
        OS2_EAS
        SET_DIR_ATTRIB
        SYMLINKS (symbolic links supported, if RTL and file system permit)
        TIMESTAMP
        UNIXBACKUP
        USE_EF_UT_TIME
        USE_UNSHRINK (PKZIP/Zip 1.x unshrinking method supported)
        USE_DEFLATE64 (PKZIP 4.x Deflate64(tm) supported)
        LARGE_FILE_SUPPORT (large files over 2 GiB supported)
        ZIP64_SUPPORT (archives using Zip64 for large files supported)
        USE_BZIP2 (PKZIP 4.6+, using bzip2 lib version 1.0.6, 6-Sept-2010)
        VMS_TEXT_CONV
        [decryption, version 2.11 of 05 Jan 2007]

UnZip and ZipInfo environment options:
           UNZIP:  [none]
        UNZIPOPT:  -C
         ZIPINFO:  [none]
      ZIPINFOOPT:  [none]
             EMX:  [none]
          EMXOPT:  [none]

I thought that was using the version installed by ArcaOS, but I'll double check the PATH.

Meanwhile, any thoughts why 7za gives

Code: [Select]
Tue 11-27-18 22:47:32{1}[h:\vendors\7-zip\p7zip-16.02-os2-1\bin] 7za a w:\temp\m.7z m:\

7-Zip (a) [32] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.IBM-850,Utf16=on,HugeFiles=on,32 bits,2 CPUs x
86)

Open archive: w:/temp/m.7z
--
Path = w:/temp/m.7z
Type = 7z
Physical Size = 32
Headers Size = 0
Solid = -
Blocks = 0

Scanning the drive:

WARNING: No more files
m:

0 files, 0 bytes

Updating archive: w:/temp/m.7z

Items to compress: 0


Files read from disk: 0
Archive size: 32 bytes (1 KiB)

Scan WARNINGS for files and folders:

m: : No more files
----------------
Scan WARNINGS: 1

Title: Re: Creating archive > 4GiB from script
Post by: Doug Bissett on November 30, 2018, 03:42:23 am
Quote
I thought that was using the version installed by ArcaOS, but I'll double check the PATH.

What you show, is what the programs display, but it doesn't tell the whole story. It depends on which version of ArcaOS you have installed, and if you have properly updated the RPM/YUM stuff (use ANPM), to be sure that you have the correct versions. You may even need to install ZIP and UNZIP, using ANPM (RPM/YUM), to get the correct version installed. Look in the list in ANPM for that information. The description of your problems is exactly what happens if you are using an older version of ZIP/UNZIP. You also need to be sure that you do not have any older versions hanging around, just waiting to cause problems.

If it all looks correct, use ANPM to uninstall ZIP and UNZIP, then install them again, just to be sure that something didn't down level them in the background. Also, be aware, that ArcaOS has a copy of the older ZIP/UNZIP in the \SYS\INSTALL directory, but they are named ZIP2.EXE and UNZIP5.EXE. There is a reason for that, so don't mess with it.
Title: Re: Creating archive > 4GiB from script
Post by: Bogdan on December 01, 2018, 07:28:31 pm
Hi Bogdan, I don't know if private builds is the best description as these builds ship with the latest releases of ArcaOS, perhaps forks is a better term.
It's quite easy, these builds seem to be from a private codebase and not released officially by Infozip.

Quote
While the Bitwise people do make some choices that others don't agree with, they are some of the few developers actually doing something on our platform and though they do make some money doing it, considering the amount of time and effort they put in, much of the work is basically on a volunteer basis and it is hard to bitch about people doing stuff for free, even if not perfect.
They must be gods, usually only banks can make money in the Western world.

Quote
One of the decisions they've made is not to spend time trying to get their patches accepted up stream. Be nice if they did, but understandable as getting patches accepted can and usually is, time consuming.
I agree, applying some 2 minute patches to a 10 year old zip release can be very time-consuming. So there is no time left for compatibility checking any more or a functional test at all.

Quote
They have been very good about following licenses including being public with their fixes and other patches with many hosted at netlabs and others at Github.
In the case of zip and unzip, their trees are at http://trac.netlabs.org/ports/browser/zip/trunk (http://trac.netlabs.org/ports/browser/zip/trunk) and http://trac.netlabs.org/ports/browser/unzip/trunk (http://trac.netlabs.org/ports/browser/unzip/trunk) with issue tracking also hosted at netlabs.
Sorry we are in 2018 now, I cannot see any reason for a fork of 10 year old code, where created binary packages don't follow the terms of the licence agreement and additionally are incomplete or break compatibility in different ways.

So my question to the original poster here: Which type of archive that should be greater than 4GB-1 was actually created? Which storage method was utilised? Without proper record information nobody can seriously help.

For some of the remaining OS/2 hobbyists: You can simply read ISO/IEC 21320-1:2015 before making claims that are totally untenable.
Title: Re: Creating archive > 4GiB from script
Post by: Valery Sedletski on December 02, 2018, 12:57:01 am
2Bogdan:

> It's quite easy, these builds seem to be from a private codebase and not released officially by Infozip.

They are not private. They are publicly available from BWW GitHub.

> Sorry we are in 2018 now, I cannot see any reason for a fork of 10 year old code, where created binary packages don't follow the terms of the licence agreement and additionally are incomplete or break compatibility in different ways.

Do you think that fork is something bad? Anybody can fork any software with opensource license,
and add his own enhancements. Moreover, forks are encouraged with git, where multiple repositories
with the same program exist, and they may copy changes from one to another. Original OS/2 patches
from InfoZip are very old, and InfoZip can easily break the OS/2 version themself. I doubt they do
OS/2 version testing very much. So, there can be bugs. To fix these bugs, BWW needs extra patches.
Also, there are some enhancements like kLIBC symlinks support. I doubt that this feature present in
InfoZip version.

Binaries don't follow the license agreement? What do you mean? Both sources and binaries are
publicly available...

> I agree, applying some 2 minute patches to a 10 year old zip release can be very time-consuming. So there is no time left for compatibility checking any more or a functional test at all.

Why do you think it is 2-minute patches? Did you tried to build InfoZip yourself? I tried to build it
with OpenWatcom. It was very old version, and making the old OS/2 ACL patch work was very
hard. (There were some problems with calling 32-to-16-bit thunks. As some 16-bit API from
netapi.dll was called. I wonder how they work this around with 32-bit GCC build).
Title: Re: Creating archive > 4GiB from script
Post by: Bogdan on December 02, 2018, 03:07:57 am
2Bogdan:

> It's quite easy, these builds seem to be from a private codebase and not released officially by Infozip.

They are not private. They are publicly available from BWW GitHub.
I only know about the private Github company. Does it mean that BWW offers a similar service for the public?

Quote
> Sorry we are in 2018 now, I cannot see any reason for a fork of 10 year old code, where created binary packages don't follow the terms of the licence agreement and additionally are incomplete or break compatibility in different ways.

Do you think that fork is something bad? Anybody can fork any software with opensource license,
and add his own enhancements.
... under the restrictions of the licence agreement and the copyright. But that depends on local law.

Quote
Moreover, forks are encouraged with git, where multiple repositories
with the same program exist, and they may copy changes from one to another.
git is only a version control software that encourages nobody. I don't know about a special AI improvement or module that would offer this feature.

Quote
Original OS/2 patches from InfoZip are very old, and InfoZip can easily break the OS/2 version themself. I doubt they do
OS/2 version testing very much.
InfoZip releases of zip or unzip don't require any patches. They built on native OS/2 supported environments or can be cross-compiled from other development environments.

Quote
So, there can be bugs. To fix these bugs, BWW needs extra patches.
That's nothing special. Generally every software contains bugs that need to be fixed.

Quote
Also, there are some enhancements like kLIBC symlinks support. I doubt that this feature present in
InfoZip version.
Please provide the necessary information so I can answer the question. Do you mean symbolic links? The Zip format is from the PC DOS world and has only limited support for UNIX features. Hard link and symbolic link support can be provided with UNIX Extra Field, but seems to be useless for non-UNIX environments. Who wants to restore UNIX device names under OS/2? Can you provide a phone number of this support service or does it need a special contract?

Quote
Binaries don't follow the license agreement? What do you mean? Both sources and binaries are
publicly available...
Sorry I'm not a barrister or a spokesman for Infozip. Usually only 1% reads the licence agreement and 1% of them follows the terms.

Quote
Why do you think it is 2-minute patches?
Just a calculation of keystrokes by comparing with the sources of the release plus some extra time for creating a patch file. We use a plugin available for different environments which can automate this task, so we can monitor our Ukrainian workforce, and it saves a lot of money.

Quote
Did you tried to build InfoZip yourself?
I cannot count the platforms and different C compilers here, but mostly C/370 or the AIX C set.

Quote
I tried to build it
with OpenWatcom. It was very old version, and making the old OS/2 ACL patch work was very
hard. (There were some problems with calling 32-to-16-bit thunks. As some 16-bit API from
netapi.dll was called. I wonder how they work this around with 32-bit GCC build).
Which version of OpenWatcom was utilised? I remember of building it 20 years ago with Watcom C compiler for OS/2 and NT with the supplied makefile from an OS/2 hosted environment.
Title: Re: Creating archive > 4GiB from script
Post by: Valery Sedletski on December 02, 2018, 05:39:50 am
2Bogdan:

> I only know about the private Github company. Does it mean that BWW offers a similar service for the public?

What do you mean? Most Github projects are opensource and provide sources and binaries
for free, including BWW. So, BWW repos are public, not private (only commercial sources are not
public, like InJoy, for example).

>>> Sorry we are in 2018 now, I cannot see any reason for a fork of 10 year old code, where created binary packages don't follow the terms of the licence agreement and additionally are incomplete or break compatibility in different ways.

>>    Do you think that fork is something bad? Anybody can fork any software with opensource license,
 and add his own enhancements.

>... under the restrictions of the licence agreement and the copyright. But that depends on local law.

InfoZIP seems to be under BSD-like license. BWW provide packages under the same license. So, what
terms they don't follow?

>>  Moreover, forks are encouraged with git, where multiple repositories
with the same program exist, and they may copy changes from one to another.

>git is only a version control software that encourages nobody. I don't know about a special AI improvement or module that would offer this feature.

Git and GitHub makes it easy to create and merge forks. Forks are just a convenient way to create
modified versions of sources. Fork doesn't denote anything bad. So, there's nothing bad that BWW
created a fork. This fork could be in the future merged into an original version, but this requires time.

> InfoZip releases of zip or unzip don't require any patches. They built on native OS/2 supported environments or can be cross-compiled from other development environments.

These patches are old and InfoZip cannot thoroughly test the OS/2 code with newer versions. Simply
because they don't use OS/2 for everyday  use. So, they can break something in new versions, without
noticing it. That's why newer versions can cointain bugs (related to combination of old OS/2 patches
with newer InfoZip code). So, this require fixes, which is done by BWW. So, no wonder if they require
modifications.

The OS/2 patches to InfoZip are very old, they were created by Kai Uwe Rommel in 90'ths. And I doubt
that this code was updated much by InfoZip.

>> Also, there are some enhancements like kLIBC symlinks support. I doubt that this feature present in
InfoZip version.

> Please provide the necessary information so I can answer the question. Do you mean symbolic links? The Zip format is from the PC DOS world and has only limited support for UNIX features. Hard link and symbolic link support can be provided with UNIX Extra Field, but seems to be useless for non-UNIX environments. Who wants to restore UNIX device names under OS/2?

This is you are kidding, or serious?  Yes, symbolic links. Don't you know that kLIBC contains symlink support?
No, it is not useless on non-UNIX environment. It is very useful in case you backup/restore your UNIX ports tree.
This tree contains many symlinks in /@unixroot/usr/bin subdirectory, for example. I suspect that you'll not like
if after restoring the UNIXROOT from backup, your symlinks are broken? And yes, InfoZIP contains support for
symlinks on UNIX environments (the same as ACL and EA's support on OS/2 platform), so, it can be desirable
to have symlinks support on kLIBC too. This is one reason for creating patches to original version.

>>Did you tried to build InfoZip yourself?

>I cannot count the platforms and different C compilers here, but mostly C/370 or the AIX C set.

So, you didn't tried to build the OS/2 version, ok.

>Which version of OpenWatcom was utilised? I remember of building it 20 years ago with Watcom C compiler for OS/2 and NT with the supplied makefile from an OS/2 hosted environment.

I used OW 1.9 I mean that there were some problems with calling a 16-bit API from netapi.dll for OS/2
ACL support. OW is good with building 16<->32 bits thunks, at least, I was able to create 16-bit wrappers for
32-bit DLL (a FS utility DLL from fat32.ifs, for example). So, this should work, though, I was unable to build
wrappers for ACL code in zip.exe. That's why I wonder how this was worked around in GCC builds. Maybe, they
just dropped the ACL code. I don't know.
Title: Re: Creating archive > 4GiB from script
Post by: Bogdan on December 02, 2018, 06:55:08 pm
> I only know about the private Github company. Does it mean that BWW offers a similar service for the public?

What do you mean? Most Github projects are opensource and provide sources and binaries
for free, including BWW. So, BWW repos are public, not private (only commercial sources are not
public, like InJoy, for example).
Sorry, I don't want to start a discussion about ideological issues. The definition of "public" seems to be different under different legal systems.

Quote
InfoZIP seems to be under BSD-like license. BWW provide packages under the same license. So, what
terms they don't follow?
Yes, the licence changed back in 1999, but please:
Note that the lack of the BSD "advertising clause" and the presence of the third and fourth restrictions are the two principal differences from the BSD license. Restriction #3 is intended only to protect us from having to support other people's code and from getting blamed for other people's bugs.

Quote
These patches are old and InfoZip cannot thoroughly test the OS/2 code with newer versions. Simply
because they don't use OS/2 for everyday  use. So, they can break something in new versions, without
noticing it. That's why newer versions can cointain bugs (related to combination of old OS/2 patches
with newer InfoZip code). So, this require fixes, which is done by BWW. So, no wonder if they require
modifications.
So which code breaks OS/2? Please use: http://infozip.sourceforge.net/zip-bug.html

Quote
The OS/2 patches to InfoZip are very old, they were created by Kai Uwe Rommel in 90'ths. And I doubt
that this code was updated much by InfoZip.
You're speaking about a time that predates Info-ZIP.

Quote
This is you are kidding, or serious?  Yes, symbolic links. Don't you know that kLIBC contains symlink support?
Sorry, I'm not an OS/2 systems programmer. The Programming Reference doesn't mention anything.

Quote
No, it is not useless on non-UNIX environment. It is very useful in case you backup/restore your UNIX ports tree.
This tree contains many symlinks in /@unixroot/usr/bin subdirectory, for example. I suspect that you'll not like
if after restoring the UNIXROOT from backup, your symlinks are broken? And yes, InfoZIP contains support for
symlinks on UNIX environments (the same as ACL and EA's support on OS/2 platform), so, it can be desirable
to have symlinks support on kLIBC too. This is one reason for creating patches to original version.
Zip and Unzip are programs for archiving and compression. The support for backup/restore is limited by several factors of the container format. There exist different third party extensions which can be used and Info-ZIP implements it this way.

Quote
So, you didn't tried to build the OS/2 version, ok.
Should I send you some generated object code and compiler list files?

Quote
>Which version of OpenWatcom was utilised? I remember of building it 20 years ago with Watcom C compiler for OS/2 and NT with the supplied makefile from an OS/2 hosted environment.

I used OW 1.9 I mean that there were some problems with calling a 16-bit API from netapi.dll for OS/2
ACL support. OW is good with building 16<->32 bits thunks,
It seems to be that v1.9 has some code generation problems. Perhaps for this reason v1.8 is mostly used by OS/2 developers (uniaud?).
It should not be necessary to do thunking by your own for the 32-bit OS/2 target. Only IBMLS 3.0 or later is supported, so there's no reason for interfacing with 16-bit libraries directly.
Title: Re: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on December 02, 2018, 09:24:18 pm
The installed packages are zip-3.0-8.oc00.i686.rpm and unzip-6.0-7.oc00.i686.rpm from Netlabs.

To clarify, without -s, zip seems to run normally:

Code: [Select]
ZIP -$9rSuyq -b W:\  --display-counts J:\M.zip M:\
zip of M:\ took 00:42:58 with rc=0

but unzip fails with
Code: [Select]
Fri 11-30-18 14:56:02{9}[h:\] unzip -t J:\M
Archive:  J:/M.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.

See also http://trac.netlabs.org/ports/ticket/194 (http://trac.netlabs.org/ports/ticket/194)
Title: Re: Creating archive > 4GiB from script
Post by: Bogdan on December 03, 2018, 06:00:28 pm
The installed packages are zip-3.0-8.oc00.i686.rpm and unzip-6.0-7.oc00.i686.rpm from Netlabs.

To clarify, without -s, zip seems to run normally:

Code: [Select]
ZIP -$9rSuyq -b W:\  --display-counts J:\M.zip M:\
zip of M:\ took 00:42:58 with rc=0

but unzip fails with

What happens when using a different unarchiver?

At least python -m zipfile -e should be available for this purpose.
Title: Re: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on December 03, 2018, 10:29:21 pm
Quote
What happens when using a different unarchiver?

At least python -m zipfile -e should be available for this purpose.

Where do I download it and the documentation? Thanks.
Title: Re: Creating archive > 4GiB from script
Post by: Dave Yeo on December 04, 2018, 12:42:41 am
Python will be installed in ArcaOS or any system using YUM. Lots of documentation out there, perhaps start with https://www.python.org/ (https://www.python.org/). Just be aware that we're still on Python 2 and a lot of documentation is for Python 3. They're close but different in some ways.
Title: Re: Creating archive > 4GiB from script
Post by: Bogdan on December 04, 2018, 02:38:16 am
Quote
What happens when using a different unarchiver?

At least python -m zipfile -e should be available for this purpose.

Where do I download it and the documentation? Thanks.
It should be already installed as a requirement for this yum/rpm variation. Please don't ask for documentation. Non-OS specific things should work as long the runtime permits.

But under OS/2 zip archives have only a defined behaviour up to 2GB-1B. The compatibility problems start with files sizes between 2GB and 4GB-1B. And starting with 4GB the zip format cannot longer be used. That's the reason I was asking for the used storage access method. It seems you try to do some kind of backup & restore with two different tools which don't offer the same features. Please use a real backup tool to create a backup archive and do the compression later with the commands you're using.
Title: Re: Creating archive > 4GiB from script
Post by: Doug Bissett on December 04, 2018, 06:18:20 am
Quote
And starting with 4GB the zip format cannot longer be used.

Please don't tell my computer that. 12 GB (and larger) ZIP files work, with no problems, when you use the proper version of ZIP/UNZIP.

Please, update your software, properly, and stop posting incorrect information. Thanks...
Title: Re: Creating archive > 4GiB from script
Post by: Bogdan on December 04, 2018, 01:52:43 pm
Quote
And starting with 4GB the zip format cannot longer be used.

Please don't tell my computer that. 12 GB (and larger) ZIP files work, with no problems, when you use the proper version of ZIP/UNZIP.
Please read the appropriate documentation. I wrote about the zip format not about the special case of files. Of course you can rename large files to ZIP but that's just ridiculous.

Quote
Please, update your software, properly, and stop posting incorrect information. Thanks...
To 9 years old code with undocumented patches? ...and you will have the problems the original poster is talking about.
Title: Re: Creating archive > 4GiB from script
Post by: Doug Bissett on December 04, 2018, 06:40:57 pm
Please, update your software, properly, and stop posting incorrect information. Thanks...
Title: Re: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on December 07, 2018, 08:38:09 pm
 1. Arctool can create and read a 12 GiB zip file, although it takes more than 4 times as long

 2. what are the URLs for the zip and unzip builds that support > 4GiB in OS/2?

 3. Yes, of course ArcaOS includes Python, but what about the zipfile module and its documentation?

 4. Several of the other compression tools, e.g., gzip, xz, are intended to compress in place rather
     than to build a compressed archive.
Title: Re: Creating archive > 4GiB from script
Post by: Doug Bissett on December 07, 2018, 10:18:31 pm
Quote
1. Arctool can create and read a 12 GiB zip file, although it takes more than 4 times as long

It reads the ZIP file (I have done it with 60 GB files), just as fast. What takes the time, is formatting the screen output, so the user can see, and select, the files that are inside the zip file. Overall, much easier to use, but slower.

Quote
2. what are the URLs for the zip and unzip builds that support > 4GiB in OS/2?

I don't know, offhand. I use Arca Noae Package Manager (ANPM) to manage that stuff, so I don't need to do it manually. I expect that they are wherever they post the RPM packages. Whether they will work properly, without all of the other stuff that ANPM (RPM/YUM) installs, I don't care to know.

Quote
4. Several of the other compression tools, e.g., gzip, xz, are intended to compress in place rather than to build a compressed archive.

That doesn't make sense. How can you compress a folder (for example) "in place"? And what good are they, if they don't produce an archive file? Gzip and xz do exactly what ZIP does, they create an archive file out of (a) source file(s).
Title: Re: Creating archive > 4GiB from script
Post by: Dave Yeo on December 07, 2018, 10:33:10 pm
The Python zip documentation and link to module, https://docs.python.org/2/library/zipfile.html (https://docs.python.org/2/library/zipfile.html)

With gzip, bzip2, xv, the way to do it is to create a tar archive and then gzip, bzip2 or xz the tar archive. Tar usually supports both steps, read the documentation. Problem with these is no EA support unless you use really old versions, which don't support large files.
There is star, http://hobbes.nmsu.edu/download/pub/os2/util/archiver/star-1.5.73-os2.zip (http://hobbes.nmsu.edu/download/pub/os2/util/archiver/star-1.5.73-os2.zip), an alternative tar program that does support EAs, large files and gzip and IIRC, bzip2. Worth looking at as Yuri did a good job of porting, not just rebuilding.
In order of compression, from worst to best, gzip, bzip2 and xz. Note that xz is not considered as stable though I've never had a problem with xz compressed tarballs I've downloaded.
Title: Re: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on December 14, 2018, 08:00:34 pm
Quote
1. Arctool can create and read a 12 GiB zip file, although it takes more than 4 times as long

It reads the ZIP file (I have done it with 60 GB files), just as fast. What takes the time, is formatting the screen output, so the user can see, and select, the files that are inside the zip file. Overall, much easier to use, but slower.

It's the zip speed I'm concerned with, and that's 4 times as slow. The screen update time is negligible.
Quote

Quote
2. what are the URLs for the zip and unzip builds that support > 4GiB in OS/2?

I don't know, offhand. I use Arca Noae Package Manager (ANPM) to manage that stuff, so I don't need to do it manually. I expect that they are wherever they post the RPM packages. Whether they will work properly, without all of the other stuff that ANPM (RPM/YUM) installs, I don't care to know.

I also use ANPM to manage that stuff, and the version installed by ANPM is broken. That's why I asked for the  URL to a working zip version. I don't care whether it is an rpm, a wpi or a zip, as long as correctly builds a large zip.

Quote
Quote
4. Several of the other compression tools, e.g., gzip, xz, are intended to compress in place rather than to build a compressed archive.

That doesn't make sense. How can you compress a folder (for example) "in place"? And what good are they, if they don't produce an archive file?

You can't. The default behavior of those utilities is to create a compressed file and delete the original.

Quote
Gzip and xz do exactly what ZIP does, they create an archive file out of (a) source file(s).

Not even close. See the Dec 7 reply by David Yeo
Title: Re: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on December 14, 2018, 08:15:18 pm
The Python zip documentation and link to module, https://docs.python.org/2/library/zipfile.html (https://docs.python.org/2/library/zipfile.html)

Thanks.

Quote
With gzip, bzip2, xv, the way to do it is to create a tar archive and then gzip, bzip2 or xz the tar archive. Tar usually supports both steps, read the documentation. Problem with these is no EA support unless you use really old versions, which don't support large files.
There is star, http://hobbes.nmsu.edu/download/pub/os2/util/archiver/star-1.5.73-os2.zip (http://hobbes.nmsu.edu/download/pub/os2/util/archiver/star-1.5.73-os2.zip), an alternative tar program that does support EAs, large files and gzip and IIRC, bzip2. Worth looking at as Yuri did a good job of porting, not just rebuilding.
In order of compression, from worst to best, gzip, bzip2 and xz. Note that xz is not considered as stable though I've never had a problem with xz compressed tarballs I've downloaded.

I definitely need both EA and large archive support. I'll take a look at star, but in the meantime if you can give me a URL for a zip build that can creat a large archive under ArcaOS, I'd appreciate it.
Title: Re: Creating archive > 4GiB from script
Post by: Doug Bissett on December 14, 2018, 08:20:57 pm
Quote
The screen update time is negligible.

No it's not.

Quote
I also use ANPM to manage that stuff, and the version installed by ANPM is broken.

What is the problem? They work well, for me.

Have you made sure, that no other versions are sitting around in your system, just waiting to cause problems?

Have you reported your problem (if you really do have one)? It won't even be thought about, until you do. Posting here is NOT reporting a problem to anyone.

Quote
You can't. The default behavior of those utilities is to create a compressed file and delete the original.

Well, that isn't going to do you much good, as a backup method.  It does have limited uses, but not for backups.
Title: Re: Creating archive > 4GiB from script
Post by: Dave Yeo on December 14, 2018, 08:53:24 pm


Quote
You can't. The default behavior of those utilities is to create a compressed file and delete the original.

Well, that isn't going to do you much good, as a backup method.  It does have limited uses, but not for backups.

As I said, it's a two step process, tar which basically concats the files together with a table of contents, then compress the tarball. The old unix way is to pipe the commands, the new unix way is to use tar with a filtering option. tar --help | less for details.
Gzip, Bzip2 and xv are great for compressing individual files.
Compressing a 175MB log file, build log from TB, highly compressible.
gzip,  21.14 sec 5.67MB
bzip2 66.92 sec 4.42 MB
xz      26.79 sec 3.58 MB (Using 1 core)
xz      9.11 sec   3.61MB (using 4 cores) 
zip     5 sec        5.67MB (same compression method as gzip)

Title: Re: Creating archive > 4GiB from script
Post by: Bogdan on December 15, 2018, 04:42:12 am
As I said, it's a two step process, tar which basically concats the files together with a table of contents,
I don't know what's the reason of this misconception. No known tar implementation works in this way.

Quote
then compress the tarball. The old unix way is to pipe the commands, the new unix way is to use tar with a filtering option. tar --help | less for details.
Of course it's possible to use tar under UNIX. But it's not a tool for backup or restore. To label pipes old-stylish sounds strange. Btw. the given command will not work with any (standard) UNIX implementation. A simple "man tar" should be done, to read at least the section "Limitations" and to check its convenience for the desired process.

For backups usable in recovery situations there should be dump, cpio and pax. Unfortunately OS/2 is not UNIX and additional restrictions can occur.
Title: Re: Creating archive > 4GiB from script
Post by: guzzi on December 15, 2018, 09:53:44 pm
This may be a bit off-topic in this thread, but why use compression tools for backup purposes? Disk space is fairly cheap nowadays. Rsync and Keith's Easy Synch are imho far better tools for that, not in the least because they can do differential/incremental backups, it's easily possible to restore individual files etc. Are network copying times an issue?
Title: Re: Creating archive > 4GiB from script
Post by: Bogdan on December 16, 2018, 05:21:51 pm
This may be a bit off-topic in this thread, but why use compression tools for backup purposes?
There seems to be a bit of confusion between archiving and compressing, Neither the original PKZip nor Info-ZIP's Zip are compression tools but archivers. Tar is a tape archiver with block operation. The standard compression tools under OS/2 are PACK and PACK2, or under UNIX you have compress and pack and much more incompatible, mostly non-portable tools.

Quote
Disk space is fairly cheap nowadays. Rsync and Keith's Easy Synch are imho far better tools for that, not in the least because they can do differential/incremental backups, it's easily possible to restore individual files etc. Are network copying times an issue?
That's only data duplication or synchronisation. And how fast you can restore from those "backups"?
Title: Re: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on December 17, 2018, 08:57:38 pm
Quote
The screen update time is negligible.

No it's not.

Why do you believe that it takes hours to change a few dozen characters on the screen?

Quote
Quote
I also use ANPM to manage that stuff, and the version installed by ANPM is broken.

What is the problem? They work well, for me.

The problem is that I need to backup my data, not yours, and the zip provided with ArcaOS truncates the archive at 4 GiB - 1.

Quote
Have you made sure, that no other versions are sitting around in your system, just waiting to cause problems?

I don't understand how any of the other versions could have generated the date in "Compiled with emx+gcc 4.9.2 for OS/2 2.x/3.x (32-bit) on Sep 28 2018."

Quote
Have you reported your problem (if you really do have one)?

Who are you and what have you done with the real Doug Bissett? What is "See also http://trac.netlabs.org/ports/ticket/194", chopped liver? Please read what I've already posted here before accusing me of making up the problem.

Title: Re: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on December 17, 2018, 09:13:23 pm
This may be a bit off-topic in this thread, but why use compression tools for backup purposes? Disk space is fairly cheap nowadays. Rsync and Keith's Easy Synch are imho far better tools for that, not in the least because they can do differential/incremental backups, it's easily possible to restore individual files etc. Are network copying times an issue?

I want off-site backups, and the only devices that I have for removable media are DVD drives and USB ports, both slow. The capacity of a DVD is too small, so that leaves thumb drives or USB hard drives.

With zip or other compressed archive there are fewer data to write, which reduces the time required. I've rarely needed to restore an entire volume; usually I just need to restore one file to get the version prior to a finger check.

Of course, zip is also nice for porting my environment to another machine when my house loses power for weeks  >:(
Title: Re: Creating archive > 4GiB from script
Post by: Doug Bissett on December 17, 2018, 10:39:09 pm
Quote
Why do you believe that it takes hours to change a few dozen characters on the screen?

I know that PMMail has a serious problem populating the display container, when there are a few thousand entries to display. I see no reason why the Archive tool doesn't have the same problem. It is part of the way that OS/2 works, unless a programmer takes steps to work around it.

Quote
The problem is that I need to backup my data, not yours, and the zip provided with ArcaOS truncates the archive at 4 GiB - 1.

It occurs to me, that you may have a media problem (I know that ZIP/UNZIP will do it). What, exactly, are you backing up to? I know that FAT32 has some sort of maximum file size (probably 4 GB, and that is possibly fixed in the latest FAT32 versions). Try using JFS on the backup media.
Title: Re: Creating archive > 4GiB from script
Post by: Alex Taylor on December 19, 2018, 04:54:30 am
FAT32 does not support files larger than 4GB, period. This is a limitation of the filesystem, not the driver.

If you have to zip large archives onto a FAT32 drive, create a multivolume archive split at 4GB.

For example,
Code: [Select]
zip -r -S -AC -s 4g t:\mybackup_c.zip c:\ -x SWAPPER.DAT
Title: Re: Creating archive > 4GiB from script
Post by: Andi B. on December 19, 2018, 09:16:24 am
Quote
FAT32 does not support files larger than 4GB, period.
And older versions of our driver only supports up to 2GB. IIRC you need a pretty new version of fa32.ifs if you need >= 2GB.
Title: Re: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on December 19, 2018, 07:13:39 pm
Quote
Why do you believe that it takes hours to change a few dozen characters on the screen?

I know that PMMail has a serious problem populating the display container, when there are a few thousand entries to display. I see no reason why the Archive tool doesn't have the same problem. It is part of the way that OS/2 works, unless a programmer takes steps to work around it.

The only significant display updating is in the progress box; the update for each file is subsecond. There are only a bit over 4K files.

Quote
Quote
The problem is that I need to backup my data, not yours, and the zip provided with ArcaOS truncates the archive at 4 GiB - 1.

It occurs to me, that you may have a media problem (I know that ZIP/UNZIP will do it). What, exactly, are you backing up to?

M: is formatted JFS.

Quote
I know that FAT32 has some sort of maximum file size (probably 4 GB, and that is possibly fixed in the latest FAT32 versions). Try using JFS on the backup media.

The backup medium is formatted JFS.
Title: Re: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on December 20, 2018, 08:08:37 pm
http://hobbes.nmsu.edu/download/pub/os2/util/archiver/p7zip-9.20.1-os2.zip seems to be a viable workaround; the newer http://hobbes.nmsu.edu/download/pub/os2/util/archiver/p7zip-16.02-os2-1.zip seems to be broken.
Title: Re: Creating archive > 4GiB from script
Post by: Alex Taylor on December 20, 2018, 08:57:46 pm
Last I checked, the 7zip archive format did not support EAs.  That renders it unsuitable for backing up certain things, like OS/2 system volumes.

BTW, I use info-zip (v3) to back up my volumes all the time.  I've never had any problem creating zips much larger than 4 GB, as long as the target volume supports it. 
Title: Re: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on December 27, 2018, 09:06:10 pm
Last I checked, the 7zip archive format did not support EAs.  That renders it unsuitable for backing up certain things, like OS/2 system volumes.

I wound up using zip with exclude and include; that leaves me with more archive files for the volume, but the elapsed time is under an hour, far faster than with 7za. I'd still like a URL for a working version of zip.

Quote
BTW, I use info-zip (v3) to back up my volumes all the time.  I've never had any problem creating zips much larger than 4 GB, as long as the target volume supports it.

With what provenance? The one I'm using is from the ArcaOS and Netlabs repositories:
Code: [Select]
Installed Packages
Name        : unzip
Arch        : i686
Version     : 6.0
Release     : 7.oc00
Size        : 202 k
Repo        : installed
Summary     : A utility for unpacking zip files
URL         : http://www.info-zip.org/UnZip.html
License     : BSD
Description : The unzip utility is used to list, test, or extract files from a
            : zip archive.  Zip archives are commonly found on MS-DOS systems.
            : The zip utility, included in the zip package, creates zip
            : archives.  Zip and unzip are both compatible with archives created
            : by PKWARE(R)'s PKZIP for MS-DOS, but the programs' options and
            : default behaviors do differ in some respects.
            :
            : Install the unzip package if you need to list, test or extract
            : files from a zip archive.

Name        : zip
Arch        : i686
Version     : 3.0
Release     : 8.oc00
Size        : 524 k
Repo        : installed
Summary     : A file compression and packaging utility compatible with PKZIP
URL         : http://www.info-zip.org/Zip.html
License     : BSD
Description : The zip program is a compression and file packaging utility.  Zip
            : is analogous to a combination of the UNIX tar and compress
            : commands and is compatible with PKZIP (a compression and file
            : packaging utility for MS-DOS systems).
            :
            : Install the zip package if you need to compress files using the
            : zip program.


UnZip 6.00 of 20 April 2009, by Info-ZIP.  Maintained by C. Spieler.  Send
bug reports using http://www.info-zip.org/zip-bug.html; see README for details.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip/ ;
see ftp://ftp.info-zip.org/pub/infozip/UnZip.html for other sites.

Compiled with emx+gcc 4.9.2 for OS/2 (32-bit) on Jun 12 2018.

UnZip special compilation options:
        ASM_CRC
        COPYRIGHT_CLEAN (PKZIP 0.9x unreducing method not supported)
        OS2_EAS
        SET_DIR_ATTRIB
        SYMLINKS (symbolic links supported, if RTL and file system permit)
        TIMESTAMP
        UNIXBACKUP
        USE_EF_UT_TIME
        USE_UNSHRINK (PKZIP/Zip 1.x unshrinking method supported)
        USE_DEFLATE64 (PKZIP 4.x Deflate64(tm) supported)
        LARGE_FILE_SUPPORT (large files over 2 GiB supported)
        ZIP64_SUPPORT (archives using Zip64 for large files supported)
        USE_BZIP2 (PKZIP 4.6+, using bzip2 lib version 1.0.6, 6-Sept-2010)
        VMS_TEXT_CONV
        [decryption, version 2.11 of 05 Jan 2007]

A zip of the entire volume creates
Code: [Select]
12-23-18  10:37   4,294,967,295    124  M.zip
Attempting to unzip it gives
Code: [Select]
Archive:  K:/M.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.

Doing the zip in pieces gives
Code: [Select]
12-23-18   3:13   2,252,560,732    124  M.pdf.Galaxy.zip
12-23-18   3:41   3,468,746,085    124  M.pdf.IBM.zip
12-23-18   3:59          45,183      0  M.pdf.IBM370.log
12-23-18   4:22          28,300      0  M.pdf.IBM370.tlog
12-23-18   4:11   3,199,967,659    124  M.pdf.IBM370.zip
12-23-18   4:24     269,681,811    124  M.pdf.IBMz.zip
12-23-18   4:26     198,084,369    124  M.pdf.Math.zip
12-23-18   2:52   2,642,968,249    124  M.pdf.zip
12-23-18   2:33     676,719,403    124  M.zip

Title: Re: Creating archive > 4GiB from script
Post by: Alex Taylor on December 28, 2018, 12:18:54 am
The version on my backup partition reports itself ("zip -v") as:
Code: [Select]
Copyright (c) 1990-2008 Info-ZIP - Type 'zip "-L"' for software license.
This is Zip 3.0 (July 5th 2008), by Info-ZIP.
Currently maintained by E. Gordon.  Please send bug reports to
the authors using the web page at www.info-zip.org; see README for details.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip,
as of above date; see http://www.info-zip.org/ for other sites.

Compiled with emx+gcc 4.4.6 for OS/2 2.x/3.x (32-bit) on Jul 17 2013.

This is a slightly older-than-the-latest build of the bww code.  IIRC, I built it myself from the bww sources, after adding a patch to implement OS/2 support for the archive attribute switches (-AS/-AC).  I believe bww incorporated my patch in their subsequent releases. As such, there should be no real difference between my build and theirs, beyond theirs being based on slightly newer upstream code; still, if you want to try it, it's http://altsan.org/os2/utils/zip_300_os2_archive.zip (http://altsan.org/os2/utils/zip_300_os2_archive.zip).

My usual production volume uses the currently-latest version from the Netlabs yum repo, although I don't think I've tried zipping any massive archives with that build.
Title: Re: Creating archive > 4GiB from script
Post by: Shmuel (Seymour J.) Metz on December 30, 2018, 10:23:56 pm
The version on my backup partition reports itself ("zip -v") as:
Code: [Select]
Copyright (c) 1990-2008 Info-ZIP - Type 'zip "-L"' for software license.
This is Zip 3.0 (July 5th 2008), by Info-ZIP.
Currently maintained by E. Gordon.  Please send bug reports to
the authors using the web page at www.info-zip.org; see README for details.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip,
as of above date; see http://www.info-zip.org/ for other sites.

Compiled with emx+gcc 4.4.6 for OS/2 2.x/3.x (32-bit) on Jul 17 2013.

This is a slightly older-than-the-latest build of the bww code.  IIRC, I built it myself from the bww sources, after adding a patch to implement OS/2 support for the archive attribute switches (-AS/-AC).  I believe bww incorporated my patch in their subsequent releases. As such, there should be no real difference between my build and theirs, beyond theirs being based on slightly newer upstream code; still, if you want to try it, it's http://altsan.org/os2/utils/zip_300_os2_archive.zip (http://altsan.org/os2/utils/zip_300_os2_archive.zip).

My usual production volume uses the currently-latest version from the Netlabs yum repo, although I don't think I've tried zipping any massive archives with that build.

The bad news is that the same error exists in that build; the good news is that if I remove the working directory (-b) option, the archive is no longer truncated, although zip does take more than twice as long.
Title: Re: Creating archive > 4GiB from script
Post by: Andreas Schnellbacher on December 31, 2018, 04:44:20 am
Last I checked, the 7zip archive format did not support EAs.
The version used by Zippy or ArcaOS Archive Tool by "GlassMan" (http://www.5nets.ru/zippy.html) seems to support EAs. See the table "Archive Management Software Comparison", "Supports OS/2 EAs".