Author Topic: Stability Zip3 & Unzip6  (Read 717 times)

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 38
  • -Receive: 9
  • Posts: 371
    • View Profile
Stability Zip3 & Unzip6
« on: November 03, 2018, 07:20:20 pm »
The first ZIP v3 versions (plural) and/or UNZIP v6 versions (plural) for OS/2 were not stable. None of 'em. IIRC (I didn't record the bugs and downgraded every component) one of the problems was related to wildcards, and e.g. not all files were added or extracted. You couldn't use it for back-ups.

Is ZIP 3.0.8 & UNZIP 6.0.7 stable by now? The problems were easy to detect, so any Info-Zip power user may have noted one of the problems by now.

Doug Bissett

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 39
  • Posts: 1061
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #1 on: November 03, 2018, 07:50:38 pm »
I have been doing some of my backups, using ZIP/UNZIP, as supplied by ANPM (RPM/YUM), for about 5 years now. There was a problem, at one point, where system/hidden files were not saved, when the "-S" parameter was used, but that was fixed some time ago. I use "-r9yS" (without the quotes), to ZIP a whole drive, and I have restored, a number of times, using the Archive tool, as supplied by ArcaOS. It works well (better than BackAgain/2 ever did, and -y handles symlinks properly).

The main "problem", is that the backup files get pretty big, and they can be hard to manage, plus it is slow to load them for restore. It is most valuable to backup boot drives (even when booted from them), but try to keep all programs, and data, on a different drive (good practice anyway). On program/data drives, I also use RSYNC, to save individual files to an external disk drive (NAS, and/or USB). RSYNC manages copy, so that only changed, or new files get added, and removed files get deleted. It is a wonderful program, but use this one: http://home.earthlink.net/~steve53/betas/rsync-3.0.9.1-20171111-shl.zip, it has some fixes in it.

Neil Waldhauer

  • Sr. Member
  • ****
  • Thank You
  • -Given: 7
  • -Receive: 22
  • Posts: 377
    • View Profile
    • Blonde Guy
Re: Stability Zip3 & Unzip6
« Reply #2 on: November 03, 2018, 09:50:08 pm »
Maybe a power user will show up and post. :P

Until then, I'm using zip and unzip from Netlabs. Until the file dates were fixed in the most recent iteration, I didn't like to use them for backup. Like Doug B, i use the -y to preserve symbolic links.
Expert consulting for ArcaOS, OS/2 and eComStation
http://www.blondeguy.com

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 38
  • -Receive: 9
  • Posts: 371
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #3 on: November 04, 2018, 09:43:23 pm »
I didn't like to use them for backup
I'm using a legacy version of LZH, but its number of supported files is less than 15K and the newest version is as broken as the early versions of ZIP 3 & UNZIP 3 (IIRC both Paul's and of a German compiler). ZIP 2 and ZIP 5 are okay for me, all of the file sizes are less than 2 GiB, but it seems like ZIP 3 and UNZIP 6 should be good enough by now and I'll try to keep up.

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 38
  • -Receive: 9
  • Posts: 371
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #4 on: November 04, 2018, 10:04:17 pm »
I have been doing some of my backups, using ZIP/UNZIP, as supplied by ANPM (RPM/YUM), for about 5 years now. There was a problem, at one point, where system/hidden files were not saved, when the "-S" parameter was used, but that was fixed some time ago.

All early versions had different, non-trival problems, but presumably you should have noted those. My tests were basic tests of a user, without any test plan nor obscure situations. I did report the bugs, but to different authors. Thanks, I guess NetLabs 'is stable by now. I do use -S, so that may have been one of the initial bugs indeed. Both ZIP and UNZIP were involved w.r.t. bugs.

The main "problem", is that the backup files get pretty big, and they can be hard to manage, plus it is slow to load them for restore. It is most valuable to backup boot drives (even when booted from them), but try to keep all programs, and data, on a different drive (good practice anyway). On program/data drives, I also use RSYNC, to save individual files to an external disk drive (NAS, and/or USB). RSYNC manages copy, so that only changed, or new files get added, and removed files get deleted. It is a wonderful program, but use this one: http://home.earthlink.net/~steve53/betas/rsync-3.0.9.1-20171111-shl.zip, it has some fixes in it.

I'm the human package manager over here, which makes creating smaller backups easier. I do use ZIP for a few main directories with over 15K files. The number of other drives involved is about 10, and so on. I.e. not that relevant.

Thanks for mentioning RSYNC. I still have to review it. I'm sort of doing that manually, but not daily. Between backups I'll survive lost data (like fixed bugs or a few emails), but a DIFF'ish utility would help.

And I should create boot drive ZIP "clones. I do prefer to install the OS again, but that's not always reqiured.

Doug Bissett

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 39
  • Posts: 1061
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #5 on: November 05, 2018, 06:57:26 am »
Quote
I guess NetLabs 'is stable by now.

I use the ones that ANPM (RPM/YUM) installs. After they got the problem with -S fixed, I have not seen any problems. I don't think they are the same as what is at Netlabs.

Quote
I do use ZIP for a few main directories with over 15K files

One of my ZIP backups has 99480 files in it (this week). No problems, at all. In fact, when I do my backups, I usually do 4 or 5 drives (ZIP jobs), all at the same time. Each one takes a little longer than doing it alone, but the overall time is much less than doing them one at a time. The largest ZIP file is 10.67 GB.  It would be larger, but I split out the backup of my VBox installation, so it wouldn't take as long to do the main ZIP job. You do need to make good use of the include and exclude facility to split a section out, and omit files that really don't need to be included.

Quote
And I should create boot drive ZIP "clones. I do prefer to install the OS again, but that's not always reqiured.

I can restore a boot drive, in about 10 to 25 minutes (depending on which computer it is, and where the backup file is), including the time to format the drive, before restore. As I mentioned earlier, it is also easy to use the Archive tool, that comes with ArcaOS, to extract smaller parts, when necessary. I can also do a complete install, in about 4 hours (including all of my own programs, etc.). I get lots of practice because I do testing for the ArcaOS installer.

Quote
I'm the human package manager over here, which makes creating smaller backups easier.

I doubt that it is easier than what I do. The main problem with smaller backups, is that you need to figure out which one contains the file(s) that you need to restore. If I open the ZIP file in the Archive tool, the whole disk layout is presented, as it was on the disk, when it was backed up. Go for as much of it as you need, and it gets extracted, and put into the proper place (you do need to learn how to use the Archive tool, but that isn't difficult).

To be a little more specific, I ZIP all of each drive, on Monday morning. It takes about an hour to do three boot drives, my programs/data drive, and my VBOX setup (which has a 5 GB VDI file). Then, that gets RSYNCed to a USB drive (actually 2 of them, one for offsite storage), plus my NAS box (actually an old computer, running ArcaOS). That takes longer, because USB and network is slower than the HDD. Duplicate all of that to my 3 other main systems, and you can see that I use the new ZIP, and RSYNC, a lot.

In fact, I almost never use UNZIP, because the Archive tool is much easier to use. ZIP has come a long way, in the last couple of years. Try it, you will like it, but get the one that ANPM (RPM/YUM) installs, or all bets are off.

Silvan Scherrer

  • Full Member
  • ***
  • Thank You
  • -Given: 7
  • -Receive: 9
  • Posts: 159
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #6 on: November 05, 2018, 11:16:36 am »
Quote
I guess NetLabs 'is stable by now.

I use the ones that ANPM (RPM/YUM) installs. After they got the problem with -S fixed, I have not seen any problems. I don't think they are the same as what is at Netlabs.

It depends on which versions of netlabs one references. As no one told which version was referenced it's hard to tell. But rpm.netlabs.org for sure has the same zip as in the latest rpm. All other versions somewhere on netlabs servers, if there are any,  are most probably old outdated.

regards
Silvan
kind regards
Silvan
CTO bww bitwise works GmbH

Please help us with donations, so we can further work on OS/2 based projects. Our Shop is at https://www.bitwiseworks.com/shop/index.php

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 38
  • -Receive: 9
  • Posts: 371
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #7 on: November 05, 2018, 01:15:01 pm »
Quote
I use the ones that ANPM (RPM/YUM) installs.

So will I, having to assume that the RPM ZIP version is the same. I'm pretty sure ANPM keeps nearly full disks of OS/2 hardware in mind, so that's yet another reason to prefer a human package manager. But let's not start such a generic discussion again.

Quote
Each one takes a little longer than doing it alone, but the overall time is much less than doing them one at a time. The largest ZIP file is 10.67 GB.

Here smaller will win, for example because USB 1.1 drives aren't that fast. Excluding OS media I need less than 8 GiB, and your largest ZIP file will probably exceed the size of a whole disk over here. One archive was larger than 2 GiB (downloaded files for review), but that's solved by actually stating to review those files).

Quote
I can also do a complete install, in about 4 hours (including all of my own programs, etc.). I get lots of practice because I do testing for the ArcaOS installer.

Results cannot really be compared, but I'll win. When mirrors and images are banned, of course. The PITA is that people are using ANPM (and, in the case of FF/2, aren't supporting relevant OS/2 hardware). After having to deal with that, a human package manager with nearly full disks will have to do nothing extra to restore a bare minimum of files, which are located in common directories which is always being backupped anyway. On top of that, the human package manager has almost reduced the number of install script to a bare minimum. This still can be optimized. I do still have to drag a few objects to folders, there's no documentation to change settings of some apps, and so on.

What I've forgotten to do, looking back, is to record which non-OS packages requires which non-OS DLLs.

Quote
The main problem with smaller backups, is that you need to figure out which one contains the file(s) that you need to restore.

Such a problem doesn't exist over here, but YMMV. I do have several nearly identical installs, which is harder to manage. Creating a fullest backup will take most of the day.

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 38
  • -Receive: 9
  • Posts: 371
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #8 on: November 06, 2018, 09:32:59 pm »
Apparently no new UNZIP32.DLL (or rather UNZPI64.DLL, > 2 GiB).

I use "-r9yS" (without the quotes), to ZIP a whole drive, and I have restored, a number of times, using the Archive tool, as supplied by ArcaOS. It works well (better than BackAgain/2 ever did, and -y handles symlinks properly).

What are, in this case, symbolic links for OS/2? WPShadow objects? Backup the WPS object instead of the object it refers to? Just checking...

EDIT: seems to be a klibc/EA feature. Am I using symlinks without knowing it? IOW, is the -y a good default choice for all apps, because symlinks may have been used by its author?
« Last Edit: November 06, 2018, 09:37:51 pm by André Heldoorn »

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 9
  • -Receive: 110
  • Posts: 1611
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #9 on: November 07, 2018, 01:32:01 am »
Yes, the symlinks are supported at the LIBC level (and in Linux on HPFS) and it is possible you have some. Using -y shouldn't hurt and if you do have some symlinks, they'll be backed up.

Doug Bissett

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 39
  • Posts: 1061
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #10 on: November 07, 2018, 03:27:56 am »
Quote
What are, in this case, symbolic links for OS/2? WPShadow objects? Backup the WPS object instead of the object it refers to? Just checking...

It has nothing to do with the WPS. It is a method, used by Linux, to relate one file name, to a different file (usually executables, sometimes adding parameters). If you don't use -y ZIP will zip up the referenced file, not the symlink. That will actually work, for a while, but if the referenced file gets changed, you will still use the copy of the original file, and that can cause trouble.

Quote
Apparently no new UNZIP32.DLL (or rather UNZPI64.DLL, > 2 GiB).

There is no such concept as UNZIP64.*, in OS/2. Files (in and out), that are greater than 2 GB, are handled by the new API that does it properly, but that needs to be compiled into the executable. It is still a 32 bit program. If you are looking for something that other programs can use, I think that is done with Z.DLL (but I could be wrong).

Quote
I'm pretty sure ANPM keeps nearly full disks of OS/2 hardware in mind

I don't think that ANPM looks to see if you have a nearly full disk. If you do, it is likely that something will fill it up, and fail when it can't allocate disk space. ANPM would note the failure, if it was YUM, or Python, that failed, but every program that you run could possibly fill the disk. Even log files can fill the disk.

Quote
USB 1.1 drives aren't that fast.

If it is a laptop, you are probably stuck with that. If it is a desktop, check the used market for a USB 2.0 plug in card. Your patience will thank you. I have one old laptop, that only has USB 1.1. I find it faster to copy files to a newer machine, across my LAN, than to use USB 1.1.

Bogdan Szmalcownik

  • Newbie
  • *
  • Thank You
  • -Given: 0
  • -Receive: 2
  • Posts: 37
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #11 on: November 07, 2018, 04:20:45 pm »
There was only one release of UnZip and Zip form Info-ZIP that can be downloaded here: ftp://ftp.info-zip.org/pub/infozip/os2/

Sources for later beta releases can be obtained here: ftp://ftp.info-zip.org/pub/infozip/beta/

Point 3 of the licence should be mentioned:

Altered versions--including, but not limited to, ports to new operating systems, existing ports with new graphical interfaces, versions with modified or added functionality, and dynamic, shared, or static library versions not from Info-ZIP--must be plainly marked as such and must not be misrepresented as being the original source or, if binaries, compiled from the original source.  Such altered versions also must not be misrepresented as being Info-ZIP releases--including, but not limited to, labeling of the altered versions with the names "Info-ZIP" (or any variation thereof, including, but not limited to, different capitalizations), "Pocket UnZip," "WiZ" or "MacZip" without the explicit permission of Info-ZIP.  Such altered versions are further prohibited from misrepresentative use of the Zip-Bugs or Info-ZIP e-mail addresses or the Info-ZIP URL(s), such as to imply Info-ZIP will provide support for the altered versions.

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 38
  • -Receive: 9
  • Posts: 371
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #12 on: November 09, 2018, 03:20:16 pm »
There is no such concept as UNZIP64.*, in OS/2. Files (in and out), that are greater than 2 GB, are handled by the new API that does it properly

I don't know if the 32 of UNZIP32.DLL refers to the size it can handle (< 2 GiB). Hence the virtual UNZIP64.DLL.

Quote
I don't think that ANPM looks to see if you have a nearly full disk.

The human package manager can decide to only install HUNSPEL0.DLL, or whatever it's called, without any of the files this DLL requires to actually do something meaningful. If there's a restore speed contest, then such a minimal install will help too. But let's not discuss an installer of the happy few. I don't have it, I don't really want it, and it stops me from using several packages. If I want an Unix directory structure., I'll start using Unix.

Quote
If it is a laptop, you are probably stuck with USB 1.1

It's one of the shared devices of OS/2 and eCS 1.x notebooks indeed (if USB is broken or not available, then PCMCIA is the other one).  For e.g. IBM ThinkPad A/R/T/X notebooks it should be possible to use USB 2.0 via a port replicator.

Your network will be faster. WIth USB 2.0 creating a full backup costs about 1 hour, if I would have such a script.

Nevertheless speeds cannot be compared between us. In general the added value of the human package manager will win (only HUNSPEL0.DLL, a single INSTALL.CMD instead of several small ones, less WPI packages, and so on). Repositioning most of the objects takes most of the time over here, assuming a switch to new hardware, and e.g. IBM VisualAge's installation CD and FixPacks slows down the whole process significantly too. And I'm restricted to ~10 GiB, because of the maximum disk sizes of USB 1.1 hardware. So I do repackage e.g. LCSS, so I don't have to save (and backup) both such a package and the installed files. If you would replace the WPI archive by an INSTALL.CMD, then I'll still try to exclude that installer and append a few lines to mine. "It works for me", and my boot drive is quite clean.

As pointed out elsewhere, if e.g. eCS 2.x was available as an option to me then I may have been smiling while installing eCS 2.x and whatever installers it uses.

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 38
  • -Receive: 9
  • Posts: 371
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #13 on: November 09, 2018, 03:22:32 pm »
Yes, the symlinks are supported at the LIBC level (and in Linux on HPFS) and it is possible you have some. Using -y shouldn't hurt and if you do have some symlinks, they'll be backed up.

Thanks Doug & Dave, I'll replace my existing ZIP archives. If anything because it shouldn't hurt, and it may be good to get used to using it by default.

André Heldoorn

  • Sr. Member
  • ****
  • Thank You
  • -Given: 38
  • -Receive: 9
  • Posts: 371
    • View Profile
Re: Stability Zip3 & Unzip6
« Reply #14 on: November 09, 2018, 03:29:02 pm »
There was only one release of UnZip and Zip form Info-ZIP that can be downloaded here: ftp://ftp.info-zip.org/pub/infozip/os2/

There were at least 2 different compiles and versions, IIRC Australian and German, and each package was not stable.