XDF disk extractor

[David Graser]

Also, this diskette has 484352 bytes of free space, according to fat32.ifs.
According to your utility, it has 485888 of free space. The difference is 1536 bytes,
which is 3 sectors/clusters. Can you check what PCDOS shows on this diskette?

I don't have an easy way right now to do that.  My desktop machine blew out a capacitor (!!) this weekend.  Scared the whole family.  Popped.  Caught fire for a few seconds.  Really freaky.  Probably the dangers of using a 10+ year old desktop machine.  But, I'll try to get it setup so I can check.  I am able to borrow a desktop machine from my work and will set it up there.

Rick, contact me at dwgras at swbell dot net

I have an old Toshiba Satellite notebook  M65-S9092 with Windows XP and eCS 2.1.  Send me your address in email and I will send it to you if the shipping is not exorbitant. I don't have the eCS installation disk because I threw them away when I moved to ArcaOS.  I haven't checked on the shipping price yet.

The battery is dead so I only use the power supply.  The right side of the screen has a few lines but is still good.  The screen is a 17 inch.

[RickCHodgin]

I've uploaded the xdflibsf package to Hobbes /pub/incoming/.

[RickCHodgin]

Rick, contact me at dwgras at swbell dot net

I have an old Toshiba Satellite notebook  M65-S9092 with Windows XP and eCS 2.1.  Send me your address in email and I will send it to you if the shipping is not exorbitant. I don't have the eCS installation disk because I threw them away when I moved to ArcaOS.  I haven't checked on the shipping price yet.

The battery is dead so I only use the power supply.  The right side of the screen has a few lines but is still good.  The screen is a 17 inch.

Thank you, David.  Much appreciated.  If shipping's too much I can PayPal you.

[RickCHodgin]

Valery, I'm looking at the BPB for the XDF disks, and they contain a total of 37 sectors:

Code: [Select]

root directory entries = 224
224 * 32 bytes per entry = 7168 bytes
7168 / 512 = 14 sectors

 1 = boot sector         ; 1
11 = sectors for fat1    ; 12
11 = sectors for fat2    ; 23
14 = root dir sectors    ; 37

According to this logic, data should begin at logical sector number 38.  However, when I am looking at the hex layout, I find the data begins at logical sector number 35.

Does that seem right?  Why would it begin at 35 and not 38 given the BPB data?

[Dave Yeo]

I'd suggest rereading http://www.os2museum.com/wp/the-xdf-diskette-format/ in particular how there is a disk image hidden in the second FAT that gets loaded if no driver and how the missing 3 sectors are really missing but marked as bad. At least if I'm following along correctly. Michal is pretty good with this stuff.

[RickCHodgin]

I'd suggest rereading http://www.os2museum.com/wp/the-xdf-diskette-format/ in particular how there is a disk image hidden in the second FAT that gets loaded if no driver and how the missing 3 sectors are really missing but marked as bad. At least if I'm following along correctly. Michal is pretty good with this stuff.

Will do.  I have dyslexia.  Large amounts of reading are difficult for me.  But, in the interest of doing it right, I'll copy-and-paste into my text-to-speech reader. :-)

[Valery Sedletski]

2Dave Yeo: Thanks for the link. MichalN blog is a good reading, indeed.

2Rick Hodgin: The root directory end at offset 0x4a00 (sector 37 (if sectors are counted from 0)). But
I see the "hidden root directory" repeating again here (a "WIN1" volume label and a "packing.lst" file again).
Why it's again here? We need to read the article in MichalN blog, and see, I think.

[RickCHodgin]

I'd suggest rereading http://www.os2museum.com/wp/the-xdf-diskette-format/ in particular how there is a disk image hidden in the second FAT that gets loaded if no driver and how the missing 3 sectors are really missing but marked as bad. At least if I'm following along correctly. Michal is pretty good with this stuff.

It looks like the important parts here:

Code: [Select]

When the XDF TSR was loaded, a 3½” HD XDF floppy looked like a
medium with 80 cylinders and 23 sectors per track.  When the XDF
TSR wasn’t loaded, the same medium looked like a teensy disk with
just a few sectors total, usually containing some text file
showing the XDF disk’s contents or a readme file.

The first cylinder of a XDF floppy used standard 512-byte sectors,
only with 19 sectors per track rather than the standard 18.  The
format used 9 sectors per FAT (for the 3½” HD XDF variant, by far
the one most widely used).  It abused the fact that the second
FAT is normally never read by DOS (as long as the first FAT copy
is readable).  The second FAT copy on a XDF disk instead contained
the small 8-sector backwards compatible image.  The XDF sectors
used high sector IDs and were thus not seen by standard DOS/BIOS.
Track 0 (cylinder 0, head 0) was laid out as follows: 1 boot
sector, 11 FAT sectors, 7 sectors of a backward compatible
micro-disk.  That is 19 sectors total.

The second track (cylinder 0, head 1) started with the root
directory (7 sectors) and was followed by the data area.  Now
there’s an obvious problem: The XDF disk claimed to have 23
sectors per track, but physically there were only 19.  What
happened with the missing 4 sectors?  Easy:  XDF marked the
corresponding clusters in the FAT as bad.

It is important to note that XDF images always contained full
XDF-sized tracks, even for cylinder 0.  Some sectors in a XDF
image were thus not written to a disk, but the XDF image was
uniform.

[RickCHodgin]

2Rick Hodgin: The root directory end at offset 0x4a00 (sector 37 (if sectors are counted from 0)). But
I see the "hidden root directory" repeating again here (a "WIN1" volume label and a "packing.lst" file again).
Why it's again here? We need to read the article in MichalN blog, and see, I think.

This portion appears to be in conflict:

Code: [Select]

The first cylinder of a XDF floppy used standard 512-byte sectors,
only with 19 sectors per track rather than the standard 18.  The
==>format used 9 sectors per FAT<=== (for the 3½” HD XDF variant, by far
the one most widely used).  It abused the fact that the second
FAT is normally never read by DOS (as long as the first FAT copy
is readable).  The second FAT copy on a XDF disk instead contained
the small 8-sector backwards compatible image.  The XDF sectors
used high sector IDs and were thus not seen by standard DOS/BIOS.
Track 0 (cylinder 0, head 0) was laid out as follows: 1 boot
sector, ==>11 FAT sectors<==, 7 sectors of a backward compatible
micro-disk.  That is 19 sectors total.
Hmmm....

I'm going to have to examine the disk using a good hex editor, one that can follow links.

[Valery Sedletski]

2Rick: Yes, it's 11 sectors per FAT. Not sure, where 9 sectors were used. 1 reserved sectors + two
11-sector FATs. 1+2*11 = 23 sectors == 1 usual track. Though, DOS/BIOS only sees the
hidden image which is 19 sectors long. Hm.

I didn't read the article to the end, yet, though

PS: Yes, 1.44 MB DS/HD indeed uses two 9-sector FATs. XDF format uses 11 sectors per FAT,
so here Michal is right.

[Valery Sedletski]

2Gregg Young: Oh, nice. So, you added the "WINDOW" keyword to wrc.exe? (Or it's Steven's fixes?)
IIRC, the version from OW 1.9 didn't supported adding resources to OS/2 binaries during the linking,
though, it did support that for windows binaries. But supported adding resources to existing OS/2
binaries (after they are linked). So, it's good to know if newer versions of wrc from OW 2.0 support
adding resources to OS/2 binaries during linking?

[Gregg Young]

2Gregg Young: Oh, nice. So, you added the "WINDOW" keyword to wrc.exe? (Or it's Steven's fixes?)
IIRC, the version from OW 1.9 didn't supported adding resources to OS/2 binaries during the linking,
though, it did support that for windows binaries. But supported adding resources to existing OS/2
binaries (after they are linked). So, it's good to know if newer versions of wrc from OW 2.0 support
adding resources to OS/2 binaries during linking?

2Valery Sedletski I added the "WINDOW" support. Steven did the high memory stuff for the linker. I also fixed some issues with wipfc.exe which is in the netlabs version but should also be in Frank's snapshot version since all the code was committed.

You said that the snapshot 2.0 linker worked for vbox. Did you use wlink? I didn't see wl in the package. I built wl from the latest code if you would like to try it I can send it to you. Contact me at ygk at qwest dot net.

[Valery Sedletski]

2Gregg Young: Is not wl.exe just a renamed wlink.exe? The only difference, it seems, is that wl.exe is built statically. wlink.exe is linked to wlinkd.dll. And wrc.exe from Watcom is linked with wrcd.dll. Wrc.exe from yum repository is built statically too. I just took wlink.exe from OW 2.0 (Frank's build) and renamed it to wl.exe, and relpaced the previous version of wl.exe with it. So yes, I built VBox successfully with these versions of wlink.exe and wrc.exe from Frank's site. The only parts built with OW 1.9 are BIOS and Video BIOS ROMs. They fail to link with wl.exe from OW 2.0. The error from linker is:

Error! E2011: file L:\src\vbox\prev\out\os2.x86\release\obj\VBoxPcBios32\VBoxPcBios32.lib(L:\src\vbox\prev\src\VBox\Devices\PC\BIOS\pci32.c): invalid object file attribute

[RickCHodgin]

I don't have an easy way right now to do that.  My desktop machine blew out a capacitor (!!) this weekend.  Scared the whole family.  Popped.  Caught fire for a few seconds.  Really freaky.  Probably the dangers of using a 10+ year old desktop machine.  But, I'll try to get it setup so I can check.  I am able to borrow a desktop machine from my work and will set it up there.

As of last night, I have my developer environment restored.  Someone at work gave me a 2004 desktop machibe with a 1.5 GHz Athlon XP. :-)  It's working wonderfully.  I have OS/2 4.52 installed, 106 kernel, IBM PC DOS 7, all my old folders and developer tools.

Need to get a green monochrome monitor now.

[RickCHodgin]

The XDF disk extractor has been posted on Hobbes (xdflibsf.zip):
http://hobbes.nmsu.edu/h-browse.php?dir=/pub/os2/util/disk

Source code:
http://www.libsf.org:8990/projects/LIB/repos/libsf/browse/es2

I also realized something today.  The inner-most track on a standard 3.5" floppy disk is about 6.5" in circumference, going out to the outer-most track being about 20.5" in circumference, resulting in about 1096 inches of continuous double-sided data.  A standard 3.5" 1.44MB floppy disk has 12,500 bytes per track per side unformatted.  Formatting to 1.44MB results in 18*512 = 9216 bytes per track per side, a slack of about 182(!!) bytes per sector.  XDF gained its larger space savings by reclaiming 14 of those 182 byte blocks by only using four sectors (one 8KB, one 2KB, one 1KB, and one 512B), or about 3650 bytes per inch.

If, however, the floppy drive itself were modified and used a spiral track like a CD/DVD, and employed variable RPM and kept the sector length consistent throughout the disk to that seen in the inner-most track, and switched to 8KB sectors, it would result in 1096 inches of 3650 bytes per inch with formatting overhead ... it would be about 3.75MB per disk formatted on the same 1.44MB floppies we used today.  The only variable is the speed it rotates keeping the sector length the same as that found on the inner tracks, but throughout the entire disk, and using 470 8KB sectors to reclaim the extra wasted space by sector markers.

I wonder why nobody ever created a drive like that?  Even if they used a wider format and only got 3MB per disk, it would still be better than even the 2.88MB disks they had, and all still using the same density media as the 1.44MB floppies.  It would've been a little slower to access, but I don't think people would've minded because it would've held that much more, requiring 1/2 to 1/3rd fewer disk changes during installation.  Imagine, OS/2 Warp 4 from IBM could've booted off one floppy disk rather than three.

OS/2 could've shipped on 1/3rd to 1/2 the number of floppies for machines with floppy drive support for a spiral track mode and variable RPM.