XDF disk extractor

[Valery Sedletski]

2Rick:

[setup]
    help = setup.hlp

;   Place any programs here that should be run at the end of setup.
;   These apps will be run in order of their appearance here.
[run]

...

You can try searching these strings in the image with a hex viewer.

[David Graser]

So, vfdisk doesn't support XDF disks too, and disks of "6" format having
1840 bytes per sector, differ from XDF ones. But I think, it's possible to add
support of 23 sectors * 80 tracks * 2 sides disks to vfdisk, and it should
work.
[/quote]

Rick the source code is here if you want to take a look.

http://trac.netlabs.org/vfdisk

[Dave Yeo]

It's easy to publish something on Hobbes. Download the README.TEMPLATE from /pub/incoming, adjust the lines that need adjusting, rename it to the same name as your package, with txt rather then zip as a suffix and use ftp to upload both the zip and txt files to /pub/incoming. Best to stick to lower case as well.

Thank you, Dave.  Doing so now.  It will have two executables:

Code: [Select]

os2\xdflibsf.exe
win32\xdflibsf.exe
What other files should I add to it?  A readme.txt, etc?

Readme and a license at least.

[Dave Yeo]

About OpenWatcom, the version on Github is a fork. One of the developers got impatient with the review process etc and wanted to focus on 64bit and forked the code, he doesn't care about OS/2 as far as I know. Really they should have agreed to using different versioning.

[Valery Sedletski]

2Rick:

Hm, but why then diunpack doesn't work with these images? It should.

    L:\data\vm>diunpack WIN1.DSK -d aaa\


    DIUNPACK.EXE Release 3.00 12-20-95
    Copyright (C) IBM Corporation 1995
    All rights reserved.


    Reading disk image....


    ERROR: Image file fat is corrupt

This seems to be because, as you said, on track 0, only first 19 sectors are used. The
last 4 sectors are skipped. So, this place seems to be the hole in FAT table, this is why
diunpack says that FAT is corrupt.

I think, the hole should be handled specially. So, I wonder, how FAT32.IFS works
without the proper handling of that hole.

2Dave Yeo: Another fork at https://efbe.musca.uberspace.de/ seems to be better,
they seem to have OS/2 code not broken I tested it today. The beta3 on Netlabs' ftp
is that Steven Levine is working on. But it seems to be the same as Github fork, and
we get linker errors on some VBox code. The wl.exe and wrc.exe from yum repo seems
to be from Steven too.

[RickCHodgin]

2Rick:

[setup]
    help = setup.hlp

;   Place any programs here that should be run at the end of setup.
;   These apps will be run in order of their appearance here.
[run]

...

You can try searching these strings in the image with a hex viewer.

I had to begin at physical sector number 35 + the logical sector number from the FAT, rather than 33.  Is that what the two reserved entries are for in the FAT?  They also take up two physical sectors on disk?

[RickCHodgin]

The updated file's been posted.  Valery, please try this one to see if it works for you.

Direct download:  http://www.libsf.org/es2/xdflibsf.zip

Code: [Select]

xdflibsf\
xdflibsf\os2\
xdflibsf\os2\xdflibsf.exe
xdflibsf\win32\
xdflibsf\win32\xdflibsf.exe
xdflibsf\readme.txt
xdflibsf\license.txt
readme.txt

[RickCHodgin]

Rick the source code is here if you want to take a look.

http://trac.netlabs.org/vfdisk

Thank you, David.

[Valery Sedletski]

FAT entries correspond to clusters, not sectors. Though, a cluster on a floppy is usualy one sector. But
floppies with bigger clusters are possible too. FAT entries 0 and 1 are reserved. Entry 0 starts with the same
byte as the media byte in BPB. The media byte is usually 0xf8 on fixed disks, and 0xf0 on 1.44 MB floppies.
360 KB, 1.2 MB floppies have special values for that byte. In fact, the very old FAT disks had no BPB, they
identified media formats with a media byte in the 1st byte of FAT. Later, BPB was introduced as a more
universal mechanism of defining the disk formats. But the 0 FAT entry remained the same for compatibility.
Entry 1 is also reserved. Since Win95, it is used for storing the "dirty" bit of the FAT volume (it is ignored
on floppies, but used on hard disks). Entries starting from 2 describe clusters of data area. The data area
on FAT12/FAT16 start after the root directory. The root directory fas fixed size, specified in the BPB, it cannot
grow. So, the first are reserved sectors. Then follow the FAT copies. The FATs are followed by the root directory.
After the root directory ends, the 1st cluster begins. So, FAT entry #2 correspond to 0 cluster, FAT entry #3 --
to 1 cluster, and so on. So, you should subtract 2 from the FAT entry number, and you'll get 1:1 correspondence
(FAT entry no. - 2) -> cluster number. So, clusters 0 and 1 not correspond to any sectors on disk, they should
be skipped.

On FAT32 and exFAT, the root directory must not immediately follow FATs, and its size is not fixed, it can grow.
Also, in difference from FAT12/FAT16, the root directory on FAT32/exFAT can be fragmented. Also, root directory
area is not reserved on FAT32/exFAT. It is located in data area, and has cluster number assigned. On FAT32, if
the root directory starts after the last FAT immediately, it occupies cluster #2 (and corresponds to FAT entry #2).

[Valery Sedletski]

2Rick:

> The updated file's been posted.  Valery, please try this one to see if it works for you.

Now setup.inf extracted ok for me. Also, comparing file contents with the same files
in the image mounted with fat32.ifs, shows that the files are identical.

So, it now seems to work ok.

[Dave Yeo]

2Valery, https://efbe.musca.uberspace.de/ is Franks build machine, he builds from the official sources and targets OS/2 and is where I was pointed to by the OW developers, mostly Frank, a couple of years ago. Steven also works on the official tree and periodically applies OS/2 fixes and has done some work to support high memory amongst other stuff IIRC. I have no idea about the source on Netlabs or where it originated but Franks server is the go to place for semi-official OS/2 builds based on the daily snapshots

[Valery Sedletski]

2Dave Yeo: Good to know. So, it is an official build indeed.

2Rick Hodgin:

Looking at the diskette I uploaded, ftp://osfree.org/upload/win1.dsk, I tried to check
it with CHKDSK and got:

[l:\data\vm\bochs]chkdsk j:
The current hard disk drive is: J:
The type of file system for the disk is FAT32.
The FAT32 file system program has been started.
FAT32: Unicode translate table for CP 866 loaded.
Error: Next cluster for 2 = 00000FF7
Error: Next cluster for 3 = 00000FF7
Error: Next cluster for 4 = 00000FF7
Error: Next cluster for 5 = 00000FF7
Error: Next cluster for 6 = 00000FF7
FAT32 version 0.10.r378 compiled on Oct 16 2018
The volume label is WIN 1.
The Volume Serial Number is E300-0000.
The type of file system for the disk is FAT12.

CHKDSK is checking fats :  0%
SYS1374: File Allocation Table (FAT) is bad on drive J:.
The copies of the FATs do not match
Please run CHKDSK under Windows to correct this problem


Errors found.  The /F parameter was not specified.  Corrections will
not be written to disk.

Looking at it in the hex viewer, I can see that the 2nd FAT copy is truncated.
It begins at 0x1800. Sectors #1 and #2 of the 2nd FAT begins with:

00001A00:  F9 FF FF 03-40 00 FF 2F-00 00 00 00-00 00 00 00  ∙  @  /

What is this? Sector #3 starts with

00001E00:  57 49 4E 20-31 20 20 20-20 20 20 08-00 00 00 00  WIN 1      
00001E10:  00 00 00 00-00 00 00 00-00 00 02 00-00 00 00 00            
00001E20:  50 41 43 4B-49 4E 47 20-4C 53 54 01-00 00 00 00  PACKING LST

00001E30:  00 00 00 00-00 00 00 00-00 00 02 00-00 08 00 00              

This looks like a directory containing a volume label "WIN1", followed by a file
directory entry for "packing.lst". Sectors #4-#7 contain the contents of that file.
And sectors #8-#10 contain zeroes. Then, staring at offset 0x2e00, starts a root
directory. It also starts from a volume label "WIN1". Note that the volume label
should be in the root directory, only. So, I wonder why sector #3 contains another
root directory. And the real root directory doesn't have the "packing.lst" file. So,
what is that packing.lst file and the second root root directory for? It appears that
second FAT is used to store some special data: a separate private root directory
with a file which lists the files contained in the real root directory! And these file and
directory are preceded with some more unknown data.

Also, this diskette has 484352 bytes of free space, according to fat32.ifs.
According to your utility, it has 485888 of free space. The difference is 1536 bytes,
which is 3 sectors/clusters. Can you check what PCDOS shows on this diskette?

And diunpack, like CHKDSK, sees that two FAT copies are not the same, this is
why it fails to extract files.

[RickCHodgin]

Also, this diskette has 484352 bytes of free space, according to fat32.ifs.
According to your utility, it has 485888 of free space. The difference is 1536 bytes,
which is 3 sectors/clusters. Can you check what PCDOS shows on this diskette?

I don't have an easy way right now to do that.  My desktop machine blew out a capacitor (!!) this weekend.  Scared the whole family.  Popped.  Caught fire for a few seconds.  Really freaky.  Probably the dangers of using a 10+ year old desktop machine.  But, I'll try to get it setup so I can check.  I am able to borrow a desktop machine from my work and will set it up there.

Also, I think my algorithm is wrong, because it also reports 1536 bytes differently than the IBM PC DOS 7.0 disk 3 XDF floppy does in DOS 7.  I don't know why I'm off though.  I begin at FAT + 2 entries, and then proceed through to the end counting 0 entries.  Seems like a straight-forward algorithm. :-)

I'm going to re-visit my whole mounting logic and see if I can't make it generic enough to work with other things.  I'll create an xdfmount utility that is basically like a file viewer, so files can be manipulated when mounted by that utility, so it doesn't have to actually have a driver to mount as a hard drive on the machine.

I'll later expand it to libsfdsk for a generic disk mounter.  This will actually be how I create the FAT12/16/32 driver to use with ES/1 methinks. :-)  I'll get the logic working well in a solid environment, then translate it to physical disks.

[RickCHodgin]

What is this? Sector #3 starts with

Code: [Select]

00001E00:  57 49 4E 20-31 20 20 20-20 20 20 08-00 00 00 00  WIN 1      
00001E10:  00 00 00 00-00 00 00 00-00 00 02 00-00 00 00 00            
00001E20:  50 41 43 4B-49 4E 47 20-4C 53 54 01-00 00 00 00  PACKING LST

00001E30:  00 00 00 00-00 00 00 00-00 00 02 00-00 08 00 00              

This looks like a directory containing a volume label "WIN1", followed by a file
directory entry for "packing.lst". Sectors #4-#7 contain the contents of that file.
And sectors #8-#10 contain zeroes. Then, staring at offset 0x2e00, starts a root
directory. It also starts from a volume label "WIN1". Note that the volume label
should be in the root directory, only. So, I wonder why sector #3 contains another
root directory. And the real root directory doesn't have the "packing.lst" file. So,
what is that packing.lst file and the second root root directory for? It appears that
second FAT is used to store some special data: a separate private root directory
with a file which lists the files contained in the real root directory! And these file and
directory are preceded with some more unknown data.

I'm not sure.  It's almost as if it's a proprietary tweak to the normal FAT file system to include something the loader might check to make sure the disk has been created using a special tool and not just copied.

The "08" there on the first line after the WIN 1 plus spaces is a volume label indicator.  It sure looks like a special directory entry.

[Gregg Young]

2Valery, https://efbe.musca.uberspace.de/ is Franks build machine, he builds from the official sources and targets OS/2 and is where I was pointed to by the OW developers, mostly Frank, a couple of years ago. Steven also works on the official tree and periodically applies OS/2 fixes and has done some work to support high memory amongst other stuff IIRC. I have no idea about the source on Netlabs or where it originated but Franks server is the go to place for semi-official OS/2 builds based on the daily snapshots

I built the netlabs betas from the official sources as of the date they were built. Beta 3 includes adding support for the WINDOW keyword in wrc.exe, Those changes were all committed so if Frank's is a daily from the official sources it contains the same fixes. The code in the repository for wl and wrc is unchanged since the beta 3. 

The RPM packages are from the same build as the beta 3.