OS/2, eCS & ArcaOS - Technical > Storage

FAT32 - Netlabs or AN?

<< < (3/3)

Dariusz Piatkowski:
Thanks for all the feedback everyone. You all raised good points, some of them I agree with and some of them I do not, but I'll just leave it at that.

Alright, so I had a chance to test a few things out and there is nothing like a real number to tell the story, so here it goes.

1) Environment
I mentioned camera pics transfer because I'm talking about my Pentax K5-II DSLR. That simply accepts a storage card, which in my case is a Samsung Pro SDXC 32G card, that on my Android phone (Samsung S9) gets a 67M Read and 61M Write speeds. This card, using the very same USB stick but different hardware which runs a Win7 Pro install, shows a 13M Read and 7M Write speeds. The card is FAT32 formatted.

The card simply comes out of the DSLR's mount spot, I plug that into a reader stick, plug that puppy in and off I go. The card was previously configured to be LVM-aware, so it has it's own volume letter assigned.

The actual USB card reader specs are:


--- Code: ---<<< Device Description >>>
 Type            : 01
 USB Rev         : 200
 Class           : Reserved (0)
 Subclass        : Reserved (0)
 Protocol        : Reserved (0)
  Device Information is defined at interface Level
 Max. packetsize : 40
 Vendor  ID      : 0BDA
 Product ID      : 0119
 Device Release# : 1985
 Strings:
  Manufacturer Name : Generic
  Product Name      : USB2.0-CRW
  Serial number     : 20090815198100000
 Number of Configurations : 1

 Configuration 0 :
  Lenght     : 32
  Name       : CARD READER
  Value      : 1
  Attributes : 0x80
  Power      : 500 mA
  Interfaces : 1

  Interface 0
   Alt Setting : 0
   Name        : <NONE>
   Class       : Storage Device
   SubClass    : SCSI transparent command set
   Protocol    : Bulk-Only Transport
   Endpoints   : 2

   Endpoint 0:
    Address     : 01
    Attributes  : 02
    Packetsize  : 0200
    Interval    : 0

   Endpoint 1:
    Address     : 82
    Attributes  : 02
    Packetsize  : 0200
    Interval    : 0

--- End code ---

This is all plugged into my USB 2.0 port driven by the AOS USBEHCD 12.07 drivers.

2) Results
These are obtained by mounting the card and using LCMD filemanager to copy 12 JPG files off of the card onto a RAMDISK, files are approx 8-10M each in size.

OK, two stages here:

2a) 5.0.3.r171 AOS FAT32 install
- default settings, cache=2048, etc. shows a steady throughput of about 1M
- disabling cache and setting to 0 (zero) drops that throughput down to 250k

2b) 0.10.r383 NetLabs FAT32 install
- default settings, cache=2048, etc. shows a steady throughput of about 3M
- disabling cache and setting to 0 (zero) bumps that throughput up by about 500k to 3.5M
- what is interesting here is that with the cache disabled I am getting sporadic spikes of up to 6-7M, which drop off to about 2M and speed up again

OK, there it is.

Given these results I'm sticking with the Netlabs release, I simply cannot afford to waste time waiting for tens and occasionally hundreds of images to get moved off of my card. 3x the speed difference is just far too large of a gap.

I will say this (being part of the testing group and regularly testing the USB stuff and only intermittently the FAT32 stuff): Gregg put a LOT of effort into making the AOS FAT32 stuff work, so I do not want to take anything away from that and the stability improvements that has brought.

What I described above is a very limitted use-case, that being a fairly small storage card, this certainly pales in comparison to someone who may want to run a multi-Gig drive - I'm thinking the readily available mobile USB drives, and in that situation the AOS may in fact provide the better results. Heck, for all I know, this all may be a moot point when hooking up to a USB3 anyways...I did not test as I have no USB3 interface storage devices here.

I have not tested a from-scratch 'partition/LVM-mark/format/etc.' process either, so this test only covers the accessibility of the device and the throughput rates I'm seeing.

EDIT
====
Added the FAT32 card formatting reference and correct Netlabs r383 driver info.

Valery Sedletski:
Found this thread a bit late, but I'll answer some points

2Doug Bissett:
> I don't even consider using anything except the AN FAT32 driver. It seems to have a problem, or two, but nothing as bad as what 9.13 has (I don't even remember what the problems were). I did try the other one (with ExFAT, and FAT16, support), but decided that it wasn't really any better, and the FAT16 was interfering with the old OS/2 FAT16 support (not to mention the questionable legality of using it). At the time, I had no need for ExFAT, and still don't. It seems that nobody is using ExFAT anymore, even Microsoft doesn't seem to be pushing it.

How could FAT12/16 support interfere with IBM's FAT support, interestingly? Is not that Lewis Rosenthal who said you that nonsense? (I heard such a statement from him, withou any proof). If you would understand how IFS works, you'd know that only one IFS can mount a partition at a time, so they cannot conflict in any way.  If one IFS has mounted a partition, anoyther one will fail. And you can choose, which IFS should mount a FAT partition. If /fat switch is not specified on fat32.ifs command line, then IBM's kernel driver is in  use. If /fat is specified, then FS is remounted with fat32.ifs.

Regarding the exFAT, it is very useful if you use digital cameras. The SDXC cards are formatted with exFAT by default. So, if you need to copy data from someone else' camera without a reformat, you can use the Netlabs driver. if it's your own camers, you can reformat the card, of course. But no guarantee that the camera will support FAT32. In fact, it shouldn't, as the SDXC spec requires only exFAT support and FAT32 is optional. Also, exFAT supports files bigger than 2 GB. If you use FAT32 on latest cameras, you couldn't be able to copy videos larger than 2 GB, which lasts, with good quality, is about half an hour. So, you decide which FS to use. SDXC spec mandates exFAT support, but FAT32 support is optional. So, you're wrong saying that exFAT is not needed and even MS does not use it.

> (not to mention the questionable legality of using it)
What's the rubbish is that? Why it's not legal? Because AN fears Microsoft? There are several exFAT drivers for Linux. Nobody proved so far that they are illegal, the same for our driver.

2Rich Walsh: The BASEDEV is optional feature. QEMUIMG.DLL is optional too. You can delete it if you wish. The BASEDEV was created to be able to mount disk images with FS other than FAT/FAT32/exFAT. For example, ISO9660/CDFS works fine. Originally, I created support for mounting a FAT/FAT32/exFAT images to a subdirectory on FAT/FAT32/exFAT serving as a mountpoint. This is done by fat32.ifs itself as OS/2 cannot mount a FS to a subdirectory (only to a drive letter). Later, I created a BASEDEV for mouning other FS'es at a drive letter. The cachef32.exe daemon runs a thread which loads a QEMUIMG.DLL, monitors the IFS with FSCTL for open/read/write/close commands, and executes these requests via QEMUIMG.DLL. For a BASEDEV, I just decided to create a second similar thread monitoring the BASEDEV with IOCTL's. They are similar, so, both were put into cachef32.exe. Of course, it is possible to create a separate small daemon, but I just decided to use cachef32.exe to run worker threads. At least, fat32.ifs and cachef32.exe are present in any system, so no need to run more daemons if you need a loopback device support. So, I don't see a problem with this.

Regarding software patents and other rubbish. exFAT and FAT12/16 support is not more patented than FAT32. So, if M$ didn't sued us for using FAT32, then we could sleep in peace with FAT and exFAT as well. Specially for AN, I did created a version without exFAT support with all exFAT code ifdef'ed. So, if you fear M$ so much you could compile it without exFAT support at zero effort. No need to fork the IFS. But you forked it, not me. If someone wants a crippled version without disk image support as well -- no problem, I could #ifdef that too, if needed. Or I can create disk image support as a standalone package, as a compilation option. Unfortunately, I see almost no feedback, so no wonder.

Regarding not having performance improvements. This is indeed planned in the future, but there are some problems on the way. fat32.ifs is a 16-bit driver and has problems with bigger cache sizes. It's good that AN repaired (almost) the cache code. In my latest versions, I even backported the AN changes. But that code is still buggy so it's disabled so far. Also, my version is made sector-size independent. I successfully tested 2048-byte sectors on CDRW's formatted with FAT/FAT32/exFAT. But the cache code has 512-byte sectors hardcoded, that's another problem. I'd like to support other sector sizes in the cache as well. But then, caches with size bigger than 2048 KB, is required. But for that, I probably, need to wait until I convert the IFS to a 32-bit version (this is planned, and not only because of cache, it's also because 64-bit seek required for large files support is not working on 16-bit IFS-es, so I need to port it to 32 bits anyway. I also plan to port fat32.ifs to osFree in the future, so this is another reason to create a 32-bit IFS. I'd probably find a way to create a 16-bit or 32-bit version via an #ifdef).

PS: disk image/loopback support is still unfinished alpha-quality feature. I'd like some feedback regarding the image mounting feature and exFAT, but none so far. I am not aware of someone using these features, except myself.

Remy:

--- Quote from: Lars on November 28, 2020, 09:52:49 am ---I use the latest FAT32 from Netlabs.
For once, I don't have AN.
The other reason was, that I can finally format a dump partition up to 4 GB with FAT16 (by selecting a 64k cluster size) which allows to use the original OS2DUMP (that only dumps to FAT16 partitions) without the need to install DUMPFS.
There are also commandline switches that control what FAT16 partitions are served by FAT32 and which are controlled by the kernel built-in FAT16,you just have to use them.
The latest version is stable and works fine for me. The latest version is newer than the one announced on the Netlabs WIKI.

--- End quote ---

Hi Lars,

No issue with External SSD using 64k cluster size instead of 4k ?

Regards

Doug Bissett:

--- Quote ---How could FAT12/16 support interfere with IBM's FAT support, interestingly? Is not that Lewis Rosenthal who said you that nonsense?
--- End quote ---

No, I didn't hear it from Lewis. In fact I probably told him about it. This was early in the life of that driver, so things may have been fixed.


--- Quote ---Regarding the exFAT, it is very useful if you use digital cameras.
--- End quote ---

Perhaps, if you have an older camera. I haven't seen exFAT in anything that I ever bought, and looking at the information on the net, it seems that most companies have decided that it just isn't worth paying Microsoft for the rights to use it. True, it is available, but nobody trusts Microsoft.


--- Quote ---What's the rubbish is that? Why it's not legal? Because AN fears Microsoft? There are several exFAT drivers for Linux. Nobody proved so far that they are illegal, the same for our driver.
--- End quote ---


--- Quote ---Regarding software patents and other rubbish. exFAT and FAT12/16 support is not more patented than FAT32. So, if M$ didn't sued us for using FAT32, then we could sleep in peace with FAT and exFAT as well.
--- End quote ---

It is not legal to use FAT32, without paying Microsoft for the right to use it (even if it is only a dollar). They own the patent (as they do with exFAT). Just because they haven't, yet, gone after somebody for using it without the legal paperwork being done, doesn't mean they won't, in the future. IBM trusted Microsoft, and look what happened to OS/2. AN quite rightly fears Microsoft, and they fear IBM too. Their contract with IBM probably prohibits using things like exFAT. If either company decided that AN broke any contracts, they could put AN out of business with the stroke of a pen. AN cannot take the chance. They are a US company, and must follow US law. Obviously, you don't care much for laws, but they are necessary, even if they are stupid. AN do supply the FAT32 driver, which is old enough that they are pretty safe. I do know that they spent a lot of money checking out the legality of many parts of OS/2. That is one of the reasons why some of the GENMAC drivers are not included with ArcaOS. They couldn't get permission to include them.

The bottom line is that you can do what you want. Arca Noae cannot take the chance. If people want to use your driver, that is fine too. Just don't put AN at risk because you think you are above the law.


--- Quote ---The other reason was, that I can finally format a dump partition up to 4 GB with FAT16 (by selecting a 64k cluster size) which allows to use the original OS2DUMP (that only dumps to FAT16 partitions) without the need to install DUMPFS.
--- End quote ---

Well, DUMPFS is actually the FAT 16 driver, with a patch. You should also realize that ArcaOS 6.0 has new dump support. It will dump to FAT32 (or FAT16 if 2 GB is enough). It doesn't make much sense to make the dump partition more than 4 GB, because that is the largest size that a dump can be. Don't try to use the dump partition for anything else. The dump program doesn't care if you already have something in the drive.

Olafur Gunnlaugsson:

--- Quote from: Doug Bissett on March 17, 2021, 02:25:08 am ---It is not legal to use FAT32, without paying Microsoft for the right to use it (even if it is only a dollar).

--- End quote ---

Yes it is, they ran out years ago or were invalidated with the exception of the long filename patent that was invalidated everywhere except for the USA, and is easily circumvented anyway.

exFAT is another issue.

Navigation

[0] Message Index

[*] Previous page

Go to full version