Author Topic: Large Floppy Support Discussion - USBMSD.ADD  (Read 2112 times)

Doug Bissett

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 39
  • Posts: 1035
    • View Profile
Large Floppy Support Discussion - USBMSD.ADD
« on: November 24, 2017, 08:20:13 pm »
Quote
As I said multiple times this issue is already solved with my USBMSD.ADD driver.

Actually, it is not totally solved, and I know that that is the reason why AN has not picked up your changes for large floppy support. As I understand the problem (and I could be wrong), you wrap the contents of the drive in a software wrapper, but that doesn't include the MBR that is written on the device.  That works fine, until you decide to wipe the device, and recreate it, or even format it to a different file system. Since the "virtual MBR" is in memory, such changes modify the contents of memory, and not what is on the media. That works fine, until you eject it, and try it in another system. Then, the contents of the stick don't match. In other words, your approach does not allow an OS/2 user to properly manage media on USB sticks, because part of what needs to be on the stick, is only in memory (but only for FAT32 formatted sticks).

I think you have developed a "work around" for the real problem, but a "work around" that requires using some other OS, just to wipe the media, or even reformat it to some other file system, isn't up to the job.

Nothing wrong with what you did, but it is an incomplete solution, and not ready for prime time.


Lars

  • Full Member
  • ***
  • Thank You
  • -Given: 0
  • -Receive: 28
  • Posts: 234
    • View Profile
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #1 on: November 26, 2017, 05:56:13 pm »
Actually the most important problem WAS and IS solved. I agreed with David that if you want to change a large floppy stick to a partitioned stick under OS/2 you have to use DFSee (or its engine) to write all zeros for the first sector. (Think about why it requires a user decision to do that). USBMSD.ADD will detect that and will drop "large floppy" support so that you can subsequently partition and format the stick.

You can try that. It will work:
1) use DFSee to "write zeros to start of media"
2) eject the stick
3) run a lvm /rediscoverprm to have the system redetect the stick.
4) you can now freely partition the stick etc.

Not allowing to change the "large floppy" layout is a deliberate decision. If you think about it: you would not want OS/2 to mess around with your "large floppy" stick because that is a layout that stems from Windows. There are some subtleties in blindly formatting a "large floppy" stick to a different file system. Think about what would happen if someone deciced to format a "large floppy" stick to JFS.


Lars

Andreas Kohl

  • Sr. Member
  • ****
  • Thank You
  • -Given: 6
  • -Receive: 12
  • Posts: 280
    • View Profile
    • warpserver.de
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #2 on: November 27, 2017, 03:08:49 am »
Actually the most important problem WAS and IS solved. I agreed with David that if you want to change a large floppy stick to a partitioned stick under OS/2 you have to use DFSee (or its engine) to write all zeros for the first sector. (Think about why it requires a user decision to do that). USBMSD.ADD will detect that and will drop "large floppy" support so that you can subsequently partition and format the stick.
I can follow the point that it handles a subset of special cases. USBMSD.ADD should be an adapter device driver for the whole device class of Mass Storage Devices. I can realise that you're focussing on removable storage devices for several reasons. But how about fixed storage devices? For SCSI (that should not be a huge difference) there is a special filter driver IBMRSCSI.FLT to alter the behaviour.

Quote
You can try that. It will work:
1) use DFSee to "write zeros to start of media"
2) eject the stick
3) run a lvm /rediscoverprm to have the system redetect the stick.
4) you can now freely partition the stick etc.
I can confirm that it works for systems starting with Warp Server for e-business and later CP. But for the majority of the installed base it simply doesn't work in common configurations. Warp 4 RIPL requesters or WSOD don't have LVM support by default. So IBM's latest USB drivers work without this procedure as long applying only FAT or supported IFS for removable usage. And they have the advantage to be deployable for unattended installations.

Quote
Not allowing to change the "large floppy" layout is a deliberate decision. If you think about it: you would not want OS/2 to mess around with your "large floppy" stick because that is a layout that stems from Windows. There are some subtleties in blindly formatting a "large floppy" stick to a different file system. Think about what would happen if someone deciced to format a "large floppy" stick to JFS.
Originally JFS was neither designed nor supported for removable units. And I can not recommend to use JFS in conjunction with non-removable USB MSD under OS/2 for production level at the current state. My statements are based on my own surveillance only. For now we can still use device servers via local networking to work-around USBMSD and file systems support shortcomings at one blow. But this solution cannot be used for lone wolf systems without LAN connectivity.

Doug Bissett

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 39
  • Posts: 1035
    • View Profile
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #3 on: November 27, 2017, 05:44:28 am »
Quote
Actually the most important problem WAS and IS solved.

Actually, it is not. An effective work around has been developed, but the problem has not been solved. In fact, the USBMSD driver should be completely transparent to the file system. The problem exists because the FAT32 support is not properly written (probably not designed properly, in the first place). Some improvements have been made recently, but it is still not a proper file system, therefore it needs some special handling to make it work. No other (supported) file system needs help to make it work with USB.

Quote
And I can not recommend to use JFS in conjunction with non-removable USB MSD under OS/2 for production level at the current state.

That is flat out ridiculous. The current JFS is, IMO, the most reliable OS/2 file system to use with any device (removable, or not), that can use it. I find it far more reliable than HPFS ever was, and the performance is a whole level higher (especially with removable media). Personally, I would never use anything else, if there was no need to be compatible with other operating systems.

Having said that, I do use HPFS with the ARcaOS, or QSINIT, RAMDISK.  The reason is that the RAMDISK can be created with HPFS format, and I can turn the cache size down to minimum (64 KB), which helps performance (it would be better if it was zero, but that is not allowed by HPFS). I also use FAT32, to be able to interface with windows, which is still a necessary evil. I do turn off EA support to avoid problems caused because the FAT32 file system, in windows, does not know about them, and it causes nothing but problems, for very little gain. FAT (12 and 16) still has some limited use, but I try to avoid them, except for special things. The new addition of ExFAT support to the FAT32 driver, is interesting, but I don't know how useful that will be in the long run. I do know that there are serious questions about the legality of doing that, which is the reason why Arca Noae.removes that feature from the FAT32 driver that they distribute. There has been some discussion about porting the EXT file system to OS/2. I don't know much about that, so I won't comment, but word is that it is much like JFS. I also don't know much about HPFS386, but I suspect that JFS is more than a suitable replacement.

Lars

  • Full Member
  • ***
  • Thank You
  • -Given: 0
  • -Receive: 28
  • Posts: 234
    • View Profile
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #4 on: November 27, 2017, 07:45:10 am »
Unfortunately your answer is as wrong as you think my answer is:

1) OS/2 only supports "large floppy" if the file system is formatted with FAT16. Yes, there should be no special dealing in USBMSD.ADD to support "large floppy" (no matter what filesystem is in use). But it's the kernel (or OS2LVM.DMD/OS2DASD.DMD) that has this limitation and this can only be worked around by fixing the kernel (DMD drivers) large scale. By the way: USBMSD.ADD does not care a bit about the filesystem. It only cares about the difference of partitioned vs. "large floppy" media. It does that by reading the very first sector of a media which contains the MBR (master boot record) or the VBR (volume boot record), the latter containing the BPB (bios parameter block). This is also a dead end because with EXFAT Microsoft decided they are going to drop the BPB for good and describe the media layout at another place on the media. Another nail in the coffin of OS/2.

2) even the "large floppy" support for FAT12/16 media is done half-heartedly. The kernel has a bug in that it wants to write a DLAT sector to this kind of media nonetheless (if LVM is in use) if you tell lvm.exe (or minilvm.exe) to do that. What happens is that this will corrupt your USB stick as the kernel will blindly write that sector as sector 63 which will usually either fall into the FAT or into the data section of a "large floppy" formatted stick. Believe me, I have been testing and experiencing exactly this problem.

From the last comment you can tell why USBMSD.ADD is so paranoid about avoiding to mess with the real contents of a "large floppy" type stick. Changing data: Ok but changing vital disk structures: No.


In short: the kernel has never been written to support a layout without an MBR (that is: "floppy") on anything else than floppy disks (that's why FAT12/16 is supported for this layout). It has nothing to do with FAT32.IFS (or any other filesystem driver) whatsoever.


As to Mr. Kohl's response:

a) USBMSD.ADD deals with removable media as much as with permanently attached USB hard disk, USB CD-ROM drives and USB floppy drives. I am not focussing on anything, neither does USBMSD.ADD.

b) In the year 2017 it only makes sense to support systems with LVM. It's too late to make USBMSD.ADD compatible to an OS of the year 1996. There would be too many limitations.

c) it is true that even IBM writes in their JFS release note that JFS should not be used with removable media. Nonetheless it is the only filesystem that is usable on large partitions, with large files. It seems to work ok (at least for me) but of course you need to take care when removing a USB device formatted with JFS. You better do a proper eject or you are asking for trouble.
« Last Edit: November 27, 2017, 07:58:10 am by Lars »

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 8
  • -Receive: 106
  • Posts: 1453
    • View Profile
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #5 on: November 27, 2017, 08:44:12 am »
[There has been some discussion about porting the EXT file system to OS/2. I don't know much about that, so I won't comment, but word is that it is much like JFS. I also don't know much about HPFS386, but I suspect that JFS is more than a suitable replacement.

Ext2 has already been ported to OS/2. Problems include no LVM support and I understand people looked into adding it and failed. No EAs (xttrs), at least back then. Case sensitive file system, seems OS/2 actually upper cases most stuff when writing to a file system. The porter added a switch to make it case insensitive to allow running OS/2 on it, and yes, he wrote the mini and micro filesystems to make it bootable. And it was old, kernel 1.3 IIRC and Linux likes to change things.
I used to format my Linux partitions on OS/2 to have access to the Linux disks.
It would be nice if our JFS was once again compatible with the Linux JFS. Would be nice to port the Linux JFS to OS/2 as it has had more love over the years and supports links, both hard and soft as well as *nix permissions.
According to wiki, JFS is perhaps the best general purpose all around file system for Linux.

Valery Sedletski

  • Full Member
  • ***
  • Thank You
  • -Given: 0
  • -Receive: 11
  • Posts: 142
    • View Profile
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #6 on: November 27, 2017, 04:21:08 pm »
Quote
Actually the most important problem WAS and IS solved.

The problem exists because the FAT32 support is not properly written (probably not designed properly, in the first place). Some improvements have been made recently, but it is still not a proper file system, therefore it needs some special handling to make it work. No other (supported) file system needs help to make it work with USB.

What do you mean? Not designed properly? Is still not a proper file system? What special handling does it need?

Large floppy support does nothing with FAT32 specifically. As Lars correctly said, the problem is with OS2DASD.DMD, which thinks that Large floppies are FAT, but not IFS case. So, without Lars' workaround in USBMSD.ADD, the OS2DASD.DMD assigns a drive letter to it, and then the kernel calls MOUNT routine of in-kernel FAT driver, not any IFS-es. So, the proper Large Floppy support requires OS2DASD.DMD (and maybe, kernel too) rewrite. It is significantly harder. FAT32.IFS does not require anything special, compared to JFS or HPFS.

Quote
Quote
And I can not recommend to use JFS in conjunction with non-removable USB MSD under OS/2 for production level at the current state.

That is flat out ridiculous. The current JFS is, IMO, the most reliable OS/2 file system to use with any device (removable, or not), that can use it. I find it far more reliable than HPFS ever was, and the performance is a whole level higher (especially with removable media). Personally, I would never use anything else, if there was no need to be compatible with other operating systems.

Having said that, I do use HPFS with the ARcaOS, or QSINIT, RAMDISK.  The reason is that the RAMDISK can be created with HPFS format, and I can turn the cache size down to minimum (64 KB), which helps performance (it would be better if it was zero, but that is not allowed by HPFS).

You can also use the "/1" switch for hd4disk.add, to improve performance. It disables the strat2 routine, which simplifies things in ramdisk case, so it improves performance about 5 times.

Quote
I also use FAT32, to be able to interface with windows, which is still a necessary evil. I do turn off EA support to avoid problems caused because the FAT32 file system, in windows, does not know about them, and it causes nothing but problems, for very little gain.

I don't aware of any problems which are caused by EA support, except for EA files laying around everywhere. But this can be fixed in the future, as I plan to implement the "classic" EA support with "ea data. sf" file. Also, it should be noted that, with EA support, it is possible to store files, bigger than 4 GB, on the FAT32 volume, to transfer them between OS/2 systems. Indeed, support of big files up to 32 GB is possible without EA's, but bigger files require the EA support. The 64-bit file size is stored in three extra bits (previously, the reserved ones) of a directory entry. If it does not fit into 32+3 == 35 bits, the full 64-bit size is stored in the FAT_PLUS_FSZ EA. This feature could be enabled with  "/plus" switch. This feature is called FAT+ and it is suggested by DR-DOS developers. Also, copying files with EA's, including the desktop from the OS/2 boot partition, works fine. I saw no problems with it.

Quote
FAT (12 and 16) still has some limited use, but I try to avoid them, except for special things. The new addition of ExFAT support to the FAT32 driver, is interesting, but I don't know how useful that will be in the long run. I do know that there are serious questions about the legality of doing that, which is the reason why Arca Noae.removes that feature from the FAT32 driver that they distribute.

FAT12/FAT16 have very frequent use -- floppies, flash sticks < 2 GB, some old mp3 players, which could not support FAT32, SADUMP partitions (I also discovered that 4 GB FAT16 partitions, with 64 KB clusters, are supported by FAT os2dump, and it could be read with FAT32.IFS, so, such partitions could be used instead of DUMPFS. Read more in fat32.inf documentation on this topic). Regarding the legality of exFAT support, it is legal. In some countries, like USA, there are software patents. So companies like Arca Noae, should pay licensee fees to M$. Arca does not like this, so they don't want to include exFAT support in their distributions. I made a version without exFAT (#ifdef-ed from sources, but they want sources completely without exFAT. (and without FAT12/FAT16 too, but this is ridiculous, because it hurts nothing, and it is nonsense.) Also, they don't want to sponsor my developments, but want to include new version with FORMAT/autocheck/other features, without paying me, so I'll suspend implementing their wishes, for the time being.). Software patents are not valid in my country (Russia), and also it is implemented in many free opensource programs, like FUSE exFAT plugin in Linux, FatFS (which is the library QSINIT FAT support is based on). So far, there was no M$ claims about these projects. As I understand, they only sue commercial projects. Free projects have nothing to get from. You can freely use exFAT support for your own needs. It is Arca has problems with using it, not you or me. (But I made a version without exFAT, they are not satisfied with it, and not even thanked me for making it).

Quote
There has been some discussion about porting the EXT file system to OS/2. I don't know much about that, so I won't comment, but word is that it is much like JFS. I also don't know much about HPFS386, but I suspect that JFS is more than a suitable replacement.

JFS was developed earlier than IBM/Exigen USB stack, so they did not recommend to use it on removable media. I successfully use JFS with my USB hard drive for a long time. I've seen no problems with it. The only thing you need is to explicitly eject the volume before removal. But thic is the case with any file system. Also, JFS is journalled file system, so it is not recommended to be used on flash sticks, because of quick wearing-out of log area. So, it is better to use FAT32 or HPFS with flash sticks (or any other non-journalled FS)
« Last Edit: November 27, 2017, 06:49:58 pm by Valery Sedletski »

Valery Sedletski

  • Full Member
  • ***
  • Thank You
  • -Given: 0
  • -Receive: 11
  • Posts: 142
    • View Profile
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #7 on: November 27, 2017, 05:14:25 pm »
[There has been some discussion about porting the EXT file system to OS/2. I don't know much about that, so I won't comment, but word is that it is much like JFS. I also don't know much about HPFS386, but I suspect that JFS is more than a suitable replacement.

Quote
Ext2 has already been ported to OS/2. Problems include no LVM support and I understand people looked into adding it and failed.

it supports only very old ex2fs version, and does not mount newer ones. Yes, there are some problems with LVM. It works fine with danidasd.dmd. I don't know, where are incompatibilities with LVM. Probably, it is not ext2-os2.ifs itself is incompatible with LVM, but it is mwdd32.sys driver, which it is based on. Need to look at it better. Possibly, we'll just need to update mwdd32.sys.

Quote
No EAs (xttrs), at least back then. Case sensitive file system, seems OS/2 actually upper cases most stuff when writing to a file system. The porter added a switch to make it case insensitive to allow running OS/2 on it, and yes, he wrote the mini and micro filesystems to make it bootable. And it was old, kernel 1.3 IIRC and Linux likes to change things.
I used to format my Linux partitions on OS/2 to have access to the Linux disks.

You need to reformat the filesystem with special switch, changing its revision.

Quote
It would be nice if our JFS was once again compatible with the Linux JFS. Would be nice to port the Linux JFS to OS/2 as it has had more love over the years and supports links, both hard and soft as well as *nix permissions.
According to wiki, JFS is perhaps the best general purpose all around file system for Linux.

Our JFS is compatible with Linux. Linux sees OS/2 JFS partitions fine. And OS/2 can mount Linux JFS partitions, if you format them with -O (case insensitivity) option. OS/2 supports only FS'es, formatted as case-insensitive. Also, it requires BPB to be present in the boot sector. Linux JFS is case sensitive by default, and it has no BPB. But you can format it with the -O option and create the BPB with DFSee. There is Linux JFS ported to OS/2 already. It is OpenJFS project which is available from Netlabs. But Netlabs sources are a bit outdated. There are two versions at Netlabs svn, from 2003 and 2007. The IFS mostly works. The missing part is UJFS.DLL. But it could be created from fcsk/mkfs, which are present. Hard links should work in OS/2, like they do in ext2-os2.ifs. You only need a user mode utility to create them. For soft links, the kernel support is required, which is missing in OS/2. UNIX permissions require kernel support too. But I fear that permissions overlap with some OS/2-specific data in inode structure.
« Last Edit: November 27, 2017, 05:20:30 pm by Valery Sedletski »

Valery Sedletski

  • Full Member
  • ***
  • Thank You
  • -Given: 0
  • -Receive: 11
  • Posts: 142
    • View Profile
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #8 on: November 27, 2017, 05:57:58 pm »
2Lars:

Quote
b) In the year 2017 it only makes sense to support systems with LVM. It's too late to make USBMSD.ADD compatible to an OS of the year 1996. There would be too many limitations.

Danidasd.dmd is still quite usable. I use it in my Universal Boot Disk (a Live OS/2 system, capable of booting from ATA or USB CD's/USB flash sticks/USB hard drives/ATA hard drives/etc), to boot from USB sticks or hard drives. LVM is not usable in this case. Also, there are cases where the use of LVM is impossible. For example, I used to update Warp 3 to a newer fixpack at one plant, to allow them to use flash sticks, instead of floppies. The plant has a Warp 3 PC, which controls a glass-cutting bench. The people from that plant contacted me at a freelance site. They were not ready to update software to latest eCS system. So, if USB stack can work on such system, why not? So, it is desirable to support such systems, if we can. In fat32.ifs, I support Warp 3/Merlin systems, because 16-bit IFS should support them. No need to artificially limit it to the latest systems only. Nothing prevents the USB stack to work on such systems, too. And, some people use older systems in VM's. So it is better to support such systems too.

Also, regarding FAT/FAT32 only on big floppies. I think, that nothing prevents USBMSD.ADD to support Big floppies with any file system. Indeed, there can be Big Floppies, created by Linux, with ext2fs, or any other file systems. Nothing prevents me to create the Big Floppy with HPFS/JFS too. Indeed, creating partition table is not hard, but if you have flash sticks from other people, then reformatting/repartitioning is undesirable. The stick could be created by Linux, and not Windows.

As I understand, you check for the FS name field after the BPB, and if it is FAT, then you emulate a zero track with MBR and DLAT. But you can check for three first bytes instead, to be EB XX 90 (a two-byte JMP SHORT instruction, followed by the NOP instruction), and also the valid Partition table at 0x1BE offset, and the valid BPB otherwise. If JMP/NOP and a valid BPB exist, then it is a Large floppy, otherwise it is a PRM. As far as I can see, the valid partition type field is not mandatory. You can use 0x7 in any case. FAT32.IFS will mount the partitions with type 0x7 fine.

> This is also a dead end because with EXFAT Microsoft decided they are going to drop the BPB for good and describe the media layout at another place on the media. Another nail in the coffin of OS/2.

In exFAT, M$ has the BPB being zeroed-out, and it is followed by a new structure, which I call BPB2, which is equivalent to BPB, but has the 64-bit replacements for HiddenSectors and LargeTotalSectors fields. This is to support volumes of sizes larger than 2 TB. Also, for support of big clusters (which could be up to 32 MB), the Byte-sized shift values are used instead of BytesPerSector and SectorsPerCluster fields. Indeed, this does not prevent to mount exFAT partitions under OS/2. FS_MOUNT routine just parses the BPB2 structure, instead of the BPB. The problems could be with HiddenSectors, when booting OS/2 for such partitions. But indeed, this is not a big problem too. QSINIT can emulate the BPB in memory, and pass it to os2ldr, if Booting from exFAT, if you use QSINIT as a boot manager.
« Last Edit: November 27, 2017, 06:31:33 pm by Valery Sedletski »

Andreas Kohl

  • Sr. Member
  • ****
  • Thank You
  • -Given: 6
  • -Receive: 12
  • Posts: 280
    • View Profile
    • warpserver.de
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #9 on: November 27, 2017, 07:15:14 pm »
a) USBMSD.ADD deals with removable media as much as with permanently attached USB hard disk, USB CD-ROM drives and USB floppy drives. I am not focussing on anything, neither does USBMSD.ADD.
In my opinion it should not deal with removable media at all. I know it becomes complicated in real world to draw a line between removable device and removable medium. To the user it's uncertain which parts of the Mass Storage Class Specification are implemented. The drivers can make some trouble with permanently attached devices that support removable media. At least the user interface should distinguish between the logical and physical view of devices and media.

Quote
b) In the year 2017 it only makes sense to support systems with LVM. It's too late to make USBMSD.ADD compatible to an OS of the year 1996. There would be too many limitations.
I can see no difference between LVM and non-LVM enabled systems in kernel or device drivers. A Warp 4 or WSOD client can run the same bunch of applications. There's no need to use LVM for network stations at all. How many OS/2 user's operate with volumes that span across 50 partitions on different DASD?

Quote
c) it is true that even IBM writes in their JFS release note that JFS should not be used with removable media. Nonetheless it is the only filesystem that is usable on large partitions, with large files. It seems to work ok (at least for me) but of course you need to take care when removing a USB device formatted with JFS. You better do a proper eject or you are asking for trouble.
I'm aware of the limitations. Fortunately Валерий wrote elaborately about it, so I have nothing to add. Большое спасибо! Oчень эрудированнo.

Andreas Kohl

  • Sr. Member
  • ****
  • Thank You
  • -Given: 6
  • -Receive: 12
  • Posts: 280
    • View Profile
    • warpserver.de
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #10 on: November 27, 2017, 07:49:04 pm »
it supports only very old ex2fs version, and does not mount newer ones. Yes, there are some problems with LVM. It works fine with danidasd.dmd. I don't know, where are incompatibilities with LVM. Probably, it is not ext2-os2.ifs itself is incompatible with LVM, but it is mwdd32.sys driver, which it is based on. Need to look at it better. Possibly, we'll just need to update mwdd32.sys.
There are two different cases on LVM-enabled OS/2 systems. The orignal LVM partition support introduced by WSEB and the later CP1/CP2. It was enhanced in 2002 by the Extended Partition (EXPART) support to address larger hard drives and additional partition IDs (but only for uniprocessor ACP1/ACP2). The MWDD32.SYS based IFS implementations predate these developments. On non-LVM enabled systems the corresponding filter drivers ext2flt.flt or hfs.flt can still be used for recognition.

Valery Sedletski

  • Full Member
  • ***
  • Thank You
  • -Given: 0
  • -Receive: 11
  • Posts: 142
    • View Profile
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #11 on: November 27, 2017, 08:26:24 pm »
2Andreas Kohl:

> I'm aware of the limitations. Fortunately Валерий wrote elaborately about it, so I have nothing to add. Большое спасибо! Oчень эрудированнo.

Danke schoen :) (I know German a bit too :) Studied it at school.)

> There are two different cases on LVM-enabled OS/2 systems. The orignal LVM partition support introduced by WSEB and the later CP1/CP2. It was enhanced in 2002 by the Extended Partition (EXPART) support to address larger hard drives and additional partition IDs (but only for uniprocessor ACP1/ACP2). The MWDD32.SYS based IFS implementations predate these developments. On non-LVM enabled systems the corresponding filter drivers ext2flt.flt or hfs.flt can still be used for recognition.

Expart? Do you mean, partfilt.flt/ext2flt.flt? The second one was created for ext2-os2.ifs. The 1st one was its enhanced version for fat32.ifs, to support additional partition types, by default. They could be added with "/p" switch too. hfsflt.flt is a filter from hfs.ifs (Macintosh HFS file system for OS/2). It has nothing to do with partition types. It is needed for Macintosh floppies. They have no BPB, which is required for OS/2. So, this filter creates a virtual BPB on the fly, which makes OS/2 happy with these floppies. Why only uniprocessor ACP1/ACP2? partfilt.flt has some problems with SMP?

Or expart is something like partfilt.flt/ext2flt.flt, but for LVM systems? AFAIK, for LVM systems, such support is unneeded, because you can create the LVM info for any partition type, and the kernel will call all IFS'es for that partition, and it will be mounted if some IFS will recognize it. Or not all partition types are supported? 0x7/0xb/0xc supported, but Linux 0x83 is not? I am not sure. It is possible, indeed...

If so, it is also possible to assign a 0x7 partition type to an ext2 partition. Linux does not care about partition types, so it would not hurt it. So, we can try testing it.

Andreas Kohl

  • Sr. Member
  • ****
  • Thank You
  • -Given: 6
  • -Receive: 12
  • Posts: 280
    • View Profile
    • warpserver.de
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #12 on: November 27, 2017, 09:40:25 pm »
Actually I'm only 1/4 немецкий but 3/4 германский. I don't know how to translate it into English. :D

Expart? Do you mean, partfilt.flt/ext2flt.flt?
No, I was referring to this one:
http://service.software.ibm.com/os2dd/free/extpart/expartcp.exe
or non-LVM systems can use:
http://service.software.ibm.com/os2dd/free/extpart/expartw4.exe
which will also raise the kernel revison up to 14.091 (without SWC/PPA restricted features)

Quote
The second one was created for ext2-os2.ifs. The 1st one was its enhanced version for fat32.ifs, to support additional partition types, by default. They could be added with "/p" switch too. hfsflt.flt is a filter from hfs.ifs (Macintosh HFS file system for OS/2). It has nothing to do with partition types. It is needed for Macintosh floppies.
You're right (HFSFLT.FLT). I have to look through my archives. But I think we used rewritable CDs in the end to transfer stuff to the Macintosh world for preprint processing or NetWare. Although back in 1997 Zip and Jaz drives were quite popular with the same trouble - PRM vs. Super Floppy.

Quote
They have no BPB, which is required for OS/2. So, this filter creates a virtual BPB on the fly, which makes OS/2 happy with these floppies. Why only uniprocessor ACP1/ACP2?
Perhaps only at this point (2002) - I think it became fixed with later kernel, LVM and OS2DASD. It should be taken as a note that no original IBM installation media (WSEB, CP) included Extended Partition Support. The combination of later fixpaks and device driver updates made them play.

Quote
partfilt.flt has some problems with SMP?

Or expart is something like partfilt.flt/ext2flt.flt, but for LVM systems? AFAIK, for LVM systems, such support is unneeded, because you can create the LVM info for any partition type, and the kernel will call all IFS'es for that partition, and it will be mounted if some IFS will recognize it. Or not all partition types are supported? 0x7/0xb/0xc supported, but Linux 0x83 is not? I am not sure. It is possible, indeed...
I don't want to be confusing. EXPART support for CP simply exchanges the following files:
Code: [Select]
\os2\dll\ENGINE.DLL
\os2\dll\LVM.DLL
\os2\boot\OS2DASD.DMD
\os2\boot\OS2LVM.DMD
\os2\boot\IBM1S506.ADD
\os2\FORMAT.COM
\os2\LVM.EXE
\os2\LVMGUI.CMD
\os2\javaapps\LVMGUI.ZIP
\OS2DUMP
\OS2KRNL
For an end-user or operator it's only interesting that OS2LVM device mapper relies on OS2DASD.

Quote
If so, it is also possible to assign a 0x7 partition type to an ext2 partition. Linux does not care about partition types, so it would not hurt it. So, we can try testing it.
Sounds sane. Every IFS (ext2, hpfs, jfs) should also use the specified IFS partition id which is 7 (dual 0111). ;)

Valery Sedletski

  • Full Member
  • ***
  • Thank You
  • -Given: 0
  • -Receive: 11
  • Posts: 142
    • View Profile
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #13 on: November 28, 2017, 09:44:14 am »
2Andreas Kohl:

> Actually I'm only 1/4 немецкий but 3/4 германский. I don't know how to translate it into English. :D

What's the difference between немецкий and германский? I found them synonymous :)

Quote
I don't want to be confusing. EXPART support for CP simply exchanges the following files:

Code: [Select]
\os2\dll\ENGINE.DLL
\os2\dll\LVM.DLL
\os2\boot\OS2DASD.DMD
\os2\boot\OS2LVM.DMD
\os2\boot\IBM1S506.ADD
\os2\FORMAT.COM
\os2\LVM.EXE
\os2\LVMGUI.CMD
\os2\javaapps\LVMGUI.ZIP
\OS2DUMP
\OS2KRNL

Moveton from #os2russian said that his copy of EXPART package has os2krnl.smp file too. So, no SMP restriction, indeed. His package is, probably, is newer than yours.

> For an end-user or operator it's only interesting that OS2LVM device mapper relies on OS2DASD.

Of course. OS2DASD.DMD/OS2LVM.DMD pair from Aurora-level systems are used together. They could be exchanged to danidasd.dmd or os2dasd.dmd from Merlin, and everything will work. OS2DASD.DMD from Merlin and DANIDASD.DMD are 16-bit drivers. OS2DASD.DMD/OS2LVM.DMD from Aurora are 32-bit ones.

Quote
Quote
If so, it is also possible to assign a 0x7 partition type to an ext2 partition. Linux does not care about partition types, so it would not hurt it. So, we can try testing it.

Sounds sane. Every IFS (ext2, hpfs, jfs) should also use the specified IFS partition id which is 7 (dual 0111). ;)

Ok, I tried to create an ext2fs partition with type 0x7. Without ext2fs partitions, mwdd32.sys/ext2-os2.ifs/ext2_lw.exe are loaded syccessfully and everything boots fine. With an ext2fs partition created, it hangs on pmshell.exe start (or on cntrl.exe start). It looks that it hangs on an attempt to mount a partition (all partitions, except for the OS/2 boot partition, are mounted about this point). So, it tries to mount it, but hangs, for some reason. This needs further investigation.

Andreas Kohl

  • Sr. Member
  • ****
  • Thank You
  • -Given: 6
  • -Receive: 12
  • Posts: 280
    • View Profile
    • warpserver.de
Re: Large Floppy Support Discussion - USBMSD.ADD
« Reply #14 on: November 28, 2017, 03:04:09 pm »
What's the difference between немецкий and германский? I found them synonymous :)
It could become more complicated than with русский/росси́йский which censored dictionaries handle synonymously. Не в бровь, а в глаз.

Quote
Moveton from #os2russian said that his copy of EXPART package has os2krnl.smp file too. So, no SMP restriction, indeed. His package is, probably, is newer than yours.
Even the file expartcp.exe from the link I posted earlier includes this newer revision and removes the restriction. The changes were: OS2KRNL 14.092_SMP/UNI, additional OS2DUMP for SMP and a presence detection tool called ISSMP that's used by modified batch files. It's crucial parts should have been integrated into later fixpaks (XR_C003), so there's no need for the expartcp package anymore nowadays. But it simply visualises the difference between LVM and non-LVM environment.

Quote
Of course. OS2DASD.DMD/OS2LVM.DMD pair from Aurora-level systems are used together. They could be exchanged to danidasd.dmd or os2dasd.dmd from Merlin, and everything will work.
As long no real LVM features were used. A LVM-based partition scheme that only contains compatibility volumes can be migrated this way.

Quote
OS2DASD.DMD from Merlin and DANIDASD.DMD are 16-bit drivers. OS2DASD.DMD/OS2LVM.DMD from Aurora are 32-bit ones.
It should not be too complicated to distinguish between 10.xx and 14.xx (for 32-bit) revisions.

Quote
Ok, I tried to create an ext2fs partition with type 0x7. Without ext2fs partitions, mwdd32.sys/ext2-os2.ifs/ext2_lw.exe are loaded syccessfully and everything boots fine. With an ext2fs partition created, it hangs on pmshell.exe start (or on cntrl.exe start). It looks that it hangs on an attempt to mount a partition (all partitions, except for the OS/2 boot partition, are mounted about this point). So, it tries to mount it, but hangs, for some reason. This needs further investigation.
Do we talk about a non-LVM system regarding this experiment? I would recommend a simple case with OS/2 on a primary FAT and second primary IFS partition (type 7) which holds ext2. CONFIG.SYS should be modified to remove all other IFS FSD and to keep only the ext2 stuff. Booting to command prompt (single OS/2 session) works? Next step could be the creation of an extended partition that containes a logical IFS partition with ext2. It's a good idea to make now a complete dump of the whole DASD before fancy things happen with VCU or different Linux installation procedures. Alternatively several small (virtual) DASD can be helpful for further investigations. For similar procedures VirtualBox SCSI support (BusLogic emulation) is a time saver.