Author Topic: XDF disk extractor  (Read 5977 times)

RickCHodgin

  • Guest
XDF disk extractor
« on: November 02, 2018, 12:49:58 am »
Is there an XDF disk image extractor?  I've tried WinImage, WinEMT, DiskExplorer, and WinXDF.  DiskExplorer was able to read it, but it wouldn't read directories and wasn't able to reliably extract the files.  I couldn't find any tool to read the XDF format and export out the files.

I tried xdfcopy in OS/2, but I can't see where it extracts to a folder, but only to a physical floppy drive/disk.

I've ordered an external floppy drive for my laptop.  It will be here possibly this weekend, but probably Monday or Tuesday.
« Last Edit: November 02, 2018, 01:03:09 am by Rick C. Hodgin »

David Graser

  • Hero Member
  • *****
  • Thank You
  • -Given: 0
  • -Receive: 57
  • Posts: 505
    • View Profile
Re: XDF disk extractor
« Reply #1 on: November 02, 2018, 01:04:23 am »
Rick

Have you tried vfdisk.  Virtual floppy disk 6.0 and necessary control programs.

http://hobbes.nmsu.edu/download/pub/os2/system/drivers/filesys/vfdisk60.zip

RickCHodgin

  • Guest
Re: XDF disk extractor
« Reply #2 on: November 02, 2018, 01:50:49 am »
Have you tried vfdisk.  Virtual floppy disk 6.0 and necessary control programs.

http://hobbes.nmsu.edu/download/pub/os2/system/drivers/filesys/vfdisk60.zip

I got it to work.  But, I can't find the settings to set the drive letter.  It automatically assigns the next available letter.  I assume it's a device=...vfdisk.sys option.  I tried "A: 1" but that didn't do it.  It still goes to the next letter.  Does anyone know?

UPDATE:  When I run xdfcopy with H:, it fails.  If I run it with A: it asks me to put the disk in the drive.  It doesn't seem to recognize the virtual floppy as a physical device.
« Last Edit: November 02, 2018, 01:59:35 am by Rick C. Hodgin »

David Graser

  • Hero Member
  • *****
  • Thank You
  • -Given: 0
  • -Receive: 57
  • Posts: 505
    • View Profile
Re: XDF disk extractor
« Reply #3 on: November 02, 2018, 01:54:42 am »
I know when I installed WPS security on my system, it originally wanted a floppy disk drive.  I tried VFDisk and even though the virtual drive was not drive A or B, it had no problem finding and using the Virtual floppy disk to install itselp.

Neil Waldhauer

  • Sr. Member
  • ****
  • Thank You
  • -Given: 8
  • -Receive: 31
  • Posts: 460
    • View Profile
    • Blonde Guy
Re: XDF disk extractor
« Reply #4 on: November 02, 2018, 02:13:14 am »
This device driver provides support for eXtended Density Format (XDF) diskettes.

BASEDEV= ── XDFLOPPY.FLT ──────────────────────────────────────┤


Remarks

This device driver is required to use the XDFCOPY, command and to install this version of OS/2 using the OS/2 Installation diskettes.

Parameters

This device driver has no parameters.

Related Commands: BASEDEV, DEVICE, IBM1FLPY.ADD, IBM2FLPY.ADD, XDFCOPY.   
Expert consulting for ArcaOS, OS/2 and eComStation
http://www.blondeguy.com

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 11
  • -Receive: 149
  • Posts: 2115
    • View Profile
Re: XDF disk extractor
« Reply #5 on: November 02, 2018, 02:17:32 am »
There's a couple of other utilities to deal with xdf disks on Hobbes, http://hobbes.nmsu.edu/h-search.php?sh=1&button=Search&key=xdf&stype=all&sort=type_name&dir=%2F a port of mtools and another virtual disk I believe.
Note that A: and B: are hardwired to the physical floppies and can't be used for other things so we're limited to C:-Z:

RickCHodgin

  • Guest
Re: XDF disk extractor
« Reply #6 on: November 02, 2018, 04:19:29 am »
I think for my first OS/2 utility, I'm going to write a decent XDF disk extractor / converter.

Does anyone have knowledge of the OS/2 console window API?  Or an example of a simple text-based utility using one?  I want to make it a full-screen console app.  Is it like DOS in that you just write to 0xb8000, or is it like Windows with AllocConsole() and WriteLn(), etc, all API-function based?

XDF is 23 logical 512-byte sectors per track, with 39 128-byte physical sectors indicated for the track, broken out into an 8KB / 2 KB / 1 KB / 512 B format per track.  Additional info for physical access.

I think I could write an XDF floppy driver for ES/1 with relative ease as it works even with BIOS.  Will try that on my IBM PC DOS 7 machine (80486DX 33).
« Last Edit: November 02, 2018, 01:18:27 pm by Rick C. Hodgin »

RickCHodgin

  • Guest
Re: XDF disk extractor
« Reply #7 on: November 02, 2018, 04:42:25 am »
There's a couple of other utilities to deal with xdf disks on Hobbes, http://hobbes.nmsu.edu/h-search.php?sh=1&button=Search&key=xdf&stype=all&sort=type_name&dir=%2F a port of mtools and another virtual disk I believe.

I tried to install mtools, but couldn't get it installed.

Quote
Note that A: and B: are hardwired to the physical floppies and can't be used for other things so we're limited to C:-Z:

I'll have to change that for ES/2.  I actually want to give aliases to hard pathnames, so you can refer to things simply and as you want to.

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 11
  • -Receive: 149
  • Posts: 2115
    • View Profile
Re: XDF disk extractor
« Reply #8 on: November 02, 2018, 05:08:06 am »
mtools.exe seems to run fine here. I take it you have libc installed? The symlinks won't work from cmd.exe.
The console window API should be similar to windows. If you have the toolkit installed, there is an example program, consolio which creates a full screen console.

RickCHodgin

  • Guest
Re: XDF disk extractor
« Reply #9 on: November 02, 2018, 06:00:17 am »
mtools.exe seems to run fine here. I take it you have libc installed? The symlinks won't work from cmd.exe.
The console window API should be similar to windows. If you have the toolkit installed, there is an example program, consolio which creates a full screen console.

Thank you, Dave.  Will check it out tomorrow.  My floppy drive won't arrive until Tuesday, so maybe my XDF extractor tool could be completed before then.

I wonder why IBM didn't just use file compression?  Seems a better solution to save disk quantities.

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 11
  • -Receive: 149
  • Posts: 2115
    • View Profile
Re: XDF disk extractor
« Reply #10 on: November 02, 2018, 06:41:30 am »
Other then on the first few boot disks, most of the files are compressed. Couldn't get too fancy on a 386 with compression though. Use unpack.exe to decompress them.
I take it you're using an ISO. Wouldn't be easier to just examine the disk images in OS2IMAGE rather then the floppy images?

Bogdan

  • Jr. Member
  • **
  • Thank You
  • -Given: 0
  • -Receive: 4
  • Posts: 84
    • View Profile
Re: XDF disk extractor
« Reply #11 on: November 02, 2018, 09:00:47 am »
Is there an XDF disk image extractor?  I've tried WinImage, WinEMT, DiskExplorer, and WinXDF.  DiskExplorer was able to read it, but it wouldn't read directories and wasn't able to reliably extract the files.  I couldn't find any tool to read the XDF format and export out the files.
Most disk image tools work only with fixed 512 bytes/sector.

Quote
I tried xdfcopy in OS/2, but I can't see where it extracts to a folder, but only to a physical floppy drive/disk.
It copies from physical drive to an image or vice versa.

Quote
I've ordered an external floppy drive for my laptop.  It will be here possibly this weekend, but probably Monday or Tuesday.
External USB floppy drives will not work, because of 512 bytes/sector limitation used by their transfer method. External floppy drives connected through the dedicated external floppy connector found on ThinkPad port replicators or docking stations will do fine.

The only OS/2 tool that handles XDF images virtually is SVDISK: https://www.bmtmicro.com/BMTCatalog/os2/svdisk.html

Under Windows you can simply use Virtual PC which supports at least XDF image files - but not physically.

RickCHodgin

  • Guest
Re: XDF disk extractor
« Reply #12 on: November 02, 2018, 01:49:09 pm »
I take it you're using an ISO. Wouldn't be easier to just examine the disk images in OS2IMAGE rather then the floppy images?

I've recently learned about something called TTD (Time Travel Debugging, which I think I mentioned elsewhere).  It's a system where the entire state of the CPU is captured (Microsoft uses a "soft machine" (like Bochs) to capture it, but it can also be captured using a facility/feature of the x86 CPU, which is the ability to single-step every instruction by setting a flag value to signal an INT 1 interrupt).

I'd like to ultimately develop a set of primitive tools to enable this capture ability (initially across a COM port at bootup, then eventually to local disk files once drivers are loaded), to allow everything the CPU does to be intercepted, with all memory reads/writes being captured, and then an optional bit of storage to capture physical register changes as well as flags.

By doing this, any OS to be examined for security research would only need to be booted and arrive at the user interaction state.  It would capture the entire range of everything in memory, how it got there, which instruction did it, and everything.  This would allow for a comprehensive post-mortem analysis of the system after the fact.  Every aspect of which logic was chosen at each branch, how structures were updated, etc., would be captured.

It would then become a process of mining that operation to follow branches algorithmically, to categorize them, prioritize them as items of interest (oft-called, central branching points, etc.), and to extrapolate from the code being run exactly what is happening where, why, when, and how often.

I've never done this before, so I'm charting new waters.

I would prefer to modify a version of Bochs to handle this all virtually for me, but I've never been able to get Bochs setup to properly run / boot / install OS/2.  It's always reported a fatal error at some point.

In the alternative, Microsoft uses a custom-built virtual machine for this purpose.  It is integrated seamlessly with Windows and allows the full host of host machine attributes being exposed tot he virtual machine, so that when you create a Window it actually displays on the screen, you can interact with keyboard, mouse, network, etc.  It's quite a thing.

In any event, my goal is that instruction-by-instruction capture.  If I can do it with INT 1 it would be great because that facility would work on any machine, but bet slow (probably 200-500x slower than normal, so a normal 45 second boot would take over 6 hours).  But, it would only need to be done one time on each fixpack revision once everything is setup properly.

We'll see.  I'm in uncharted waters here (at least with regards to my own experience / expertise).  I proceed now only operating in theory, and it will require trial and error.  I will post my efforts / attempts / results as I go though.

Remember also, the goal here is security research.  I'm looking for where there are fatal flaws that could impact users resulting in data corruption, or the ability for some malicious software to take over / damage the system in some way.
« Last Edit: November 02, 2018, 02:14:53 pm by Rick C. Hodgin »

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 11
  • -Receive: 149
  • Posts: 2115
    • View Profile
Re: XDF disk extractor
« Reply #13 on: November 02, 2018, 03:31:04 pm »
Interesting, going to be a lot of data to go through :) Still not sure about the need to unpack XDF files.
Someone did suggest the diunpack program which is referenced here, https://wiki.gentoo.org/wiki/QEMU/OS2WarpV3_guest. Search for fastkick141.zip on the net.

RickCHodgin

  • Guest
Re: XDF disk extractor
« Reply #14 on: November 02, 2018, 03:35:47 pm »
Interesting, going to be a lot of data to go through :) Still not sure about the need to unpack XDF files.
Someone did suggest the diunpack program which is referenced here, https://wiki.gentoo.org/wiki/QEMU/OS2WarpV3_guest. Search for fastkick141.zip on the net.

Right now I am looking for unpacked XDF files for IBM PC DOS 7 disks for my guest machine partition in VirtualBox.  I'm going to extract them, then try to create them as 2.88 MB floppies so they'll load like normal and have enough space for their weird format.

If I get my XDF extractor written, I can probably just truncate some of the files, recompress them, add them to the floppies by number, and then add an extra step at the end to copy over the portions which were truncated, so it would install on regular 1.44 MB floppies.  As long as it's just copying and not running the programs it installs, truncating a few hundred bytes on a few files here and there would work, knowing they'll be copied over later.