OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Applications => Topic started by: xynixme on May 01, 2017, 11:31:46 pm

Title: Avoiding Windows 7's CHKDSK?
Post by: xynixme on May 01, 2017, 11:31:46 pm
A 256 MB flash drive (FAT) is sometimes used to transfer files from Microsoft's %*!*&#$ 7 (NTFS) to eCS (HPFS), using CMD.EXE. After the transfer the files are deleted, with CMD.EXE (DEL or RM).

If this empty USB flash drive is used again by %*!*&#$ 7, then %*!*&#$ 7 always claims that the drive may be damaged and it has to be checked. The check never repairs anything. Is there a way, for example by using a DFSee command, to stop %*!*&#$ 7 from believing that aliens have touched the drive?

The CHKDK procedure doesn't last that long, but the useless dialogs are annoying. Aliens haven't invaded the drive, and it's always okay. Protection or prevention plays no role.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: Mathias on May 02, 2017, 12:19:11 pm
Did you properly umount the flash drive after using it, or did you just remove it?

Win usually caches data when stuff gets transferred. This can be set up in the properties of the drive in hardware manager. On or off.
The default is ON, so you always need to properly umount the drive before removing it.

Running chkdsk with parameter /F should clear up the mess for now. Then properly umount the drive, wait some seconds and connect it again. Things should be fine now.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: drjohnnyfever on May 03, 2017, 02:17:43 am
Windows 7 and newer are supposed to not write cache removable drives by default allowing for "quick removal" but I have had the same thing happen to me multiple times even after deliberately un-mounting.

No corruption is ever found, same as you. I wonder if eCS is somehow marking FAT as dirty inadvertently.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: Andy Willis on May 03, 2017, 03:13:59 am
On OS/2, if you have cache enabled for a filesystem it includes for the USB drives.  If they are not properly ejected on OS/2 they will be marked dirty.  If you plug it back into an OS/2 system does it say it is dirty?
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: Dave Yeo on May 03, 2017, 05:02:01 am
Wonder if something about the EA's are confusing Windows. NT does/did support OS/2 EA's on FAT, I've even seen W2K chkdsk fix them. I wouldn't be surprised if something has changed in Windows FAT support and they never even tested with OS/2 EA's. It's awkward as both OS/2 and Windows uses some reserved bytes for different purposes, pointing at EA's with OS/2 and supporting (pointing at) long file names on Windows
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: xynixme on May 03, 2017, 03:44:36 pm
Did you properly umount the flash drive after using it, or did you just remove it?

The normal dirty-bit shouldn't be set. W7 accepts the drive, a FAT-named ZIP file (which contains *.DOCs, to be mailed with eCS) is moved to the USB flash drive (Ctrl-X, Ctrl-V), the drive is ejected properly. Of course "ejected properly" always includes waiting for darkness, without flashing LEDs.

eCS accepts the drive, and CMD.EXE is used to copy the ZIP file to a local drive and to delete the USB flash drive's files (DEL *). The drive is ejected properly, so eCS would be able to access it next time without requiring a CHKDSK.

It this USB flash drive is used by Win7 again, then Win7 wants to execute its CHKDSK dialog. As if the drive was touched by an alien. So far 100% of the Win7 CHKDSKs were 100% useless.

I do prefer a quick OS/2 solution above working with Win7, but IIRC it's possible to disable Win7's option to not having to eject the drive properly. So, assuming a default Win7 setup, is Win7 perhaps "confused" because eCS has no such option and cannot set some new, Microsoft-specific clean-bit?
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: xynixme on May 03, 2017, 03:55:18 pm
Wonder if something about the EA's are confusing Windows.

It's awkward as both OS/2 and Windows uses some reserved bytes for different purposes, pointing at EA's with OS/2 and supporting (pointing at) long file names on Windows

The WPS isn't used while working with the drive, other than to refresh removable drives XOR to eject the drive, FWIW.

Perhaps such a byte is used for a new "feature" of Win7 indeed, like not having to eject drives properly anymore. I don't know when which new Win7 "feature" was introduced, but apparently it doesn't bother OS/2 while Win7 detects a possible error.

The CHKDSK doesn't last long. Less than a second. Nevertheless a habit of answering "yes" may be dangerous if the next drive is a HPFS formatted drive and the question is related to formatting, by Win7, instead of the expected checking...
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: Andi B. on May 03, 2017, 04:05:58 pm
Quote
Of course "ejected properly" always includes waiting for darkness, without flashing LEDs.
For me proper eject means either -
- in the drives object right click and Eject and waiting until the drive letter disappers
- or from command line eject x: and waiting until the prompt reappears what indicates the drive was properly ejected
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: ivan on May 03, 2017, 04:41:16 pm
Maybe a stupid question but have you tried another USB stick? 

We use USB sticks from 1GB to 32 GB in size for transferring files to/from a couple of win 7 machines that a client uses and have never see any problems.  They were all setup using DFSee to provide the LVM information and drive letter and also do the FAT32 formatting.

There is also another possibility.  Depending on the age and amount of use your stick might be suffering from dead cells that the wear levelling isn't fixing.  While the OS/2, eCS FAT driver will generally ignore them win 7 will pick them up and report that it needs to run chkdsk which may not do anything if the dead cells are beyond what the wear levelling firmware can deal with (this is like the bad sector reports for floppy disks but because it is a flash drive it doesn't work the same as the floppy).
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: Pete on May 03, 2017, 05:16:31 pm
Hi

Is this a case of Win7 "remembering" that it wrote files to the drive but they are not there now?

Try deleting the files the next time the drive is inserted into Win7 rather than when using OS/2 to see what happens.


Regards

Pete
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: Doug Bissett on May 03, 2017, 07:32:50 pm
I haven't been following this thread, but win 10 will do something like that, if it is set for "fast startup" (or something like that, and it usually starts up slower anyway). What it does, is put win 10 to sleep, rather than properly shut it down. That means that the directory data is kept in sleeping memory (stored to disk), rather than getting written to the stick. When you start win 10 again, it loads the old data back into memory, and uses it. I wouldn't be surprised if win 7 (8 and 8.1)has something similar. Win 10 insists on turning the feature back on, without being asked, every once in a while. With win 7, you would need to enable it. In win 10, it is in the power control settings.

FWIW, if you share a system between win 10 and win 7, they will royally screw each other, if either one has the feature turned on.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: Olafur Gunnlaugsson on May 04, 2017, 02:33:40 am
I haven't been following this thread, but win 10 will do something like that, if it is set for "fast startup" (or something like that, and it usually starts up slower anyway). What it does, is put win 10 to sleep, rather than properly shut it down. That means that the directory data is kept in sleeping memory (stored to disk), rather than getting written to the stick. When you start win 10 again, it loads the old data back into memory, and uses it. I wouldn't be surprised if win 7 (8 and 8.1)has something similar.

Win 10 and 8.1 do it by default, 8.0 if it is set for fast boot which most systems are but 7 does not have the fast boot feature at all.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: xynixme on May 05, 2017, 11:20:18 am
Maybe a stupid question but have you tried another USB stick? 

We use USB sticks from 1GB to 32 GB in size for transferring files to/from a couple of win 7 machines that a client uses and have never see any problems.  They were all setup using DFSee to provide the LVM information and drive letter and also do the FAT32 formatting.

It's a FAT USB stick, so with a limited capacity. Clients can be yet another reason to fix it, because you don't want to be known for returning or delivering "damaged" drives. FAT32 isn't always an option, for example when an old FAT MP3 player is used.

Two images of the drive have been shipped to dfsee.com by now, and both files (same size) were different. Don't expect anything, but let's hope that some command can set OS/2's vritual "NumberOfEAs" bit from 0 to Windows' virtual "IsFlushed"-bit to 1.

If the average user of Windows gets used to answering Yes/Okay when an OS/2-drive is used, then one day this user may format your HPFS backup drive instead of CHKDSK'ing a FAT file transfer drive.

I could use a FAT32 drive for a test, but I'm not sure that OS/2 can access my device, and your test is as good as mine.

To reproduce, obvious steps excluded, FAT is classic FAT (FAT16): Use Windows 7 to save a FAT file, e.g. INVOICES.ZIP. Remove the USB FAT drive "safely". Boot eCS, use CMD.EXE to both "COPY x:\INVOICES.ZIP y:\" and "DEL x:INVOICES.ZIP". Remove the drive, and (try to) repeat the first step.

Quote
your stick might be suffering from dead cells that the wear levelling isn't fixing.

I cannot exclude anything, albeit I can use another, different FAT drive for testing purposes. The "IsFlushed"-bit is just a story too. BTW, I couldn't use an internet example to set or unset Windows flushing the drive. Perhaps that doesn't work with FAT. I'm using whatever the default setting is, and I'm used to manual flushing anyway. There drive is as old as 256 MB USB flash drives, but isn't iused that frequently. The images were produced with an "empty" drive (which still may include dead cells).

I do hope it's something like the usual DIRTY-bit of a drive, hence the "IsFlushed"-story. If so, then a command can be used to set it right for victims of Microsoft again. You don't want a client complaining about a damaged drive and/or a possible virus after you've borrowed their drive.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: xynixme on May 05, 2017, 11:46:31 am
Try deleting the files the next time the drive is inserted into Win7 rather than when using OS/2 to see what happens.

A slightly different, improved test shows that Win7 first had to find a driver (for a different FAT USB flash drive) before it displayed that the drive could be damaged. Additionally I skipped the CHKDSK, and it wasn't impossible to access the drive (didn't actually try that, but there was no fatal error).

The MP3 files of this FAT MP3 player were written by eCS 1.2, with a randomized order of a Rexx utility which first converts "Long Filename.MP3" to nnnnnnnn.MP3 and uses COPY.

I guess this also excludes dead cells, unless the same cells of two drives happened to be dead. So far my conclusion is that Windows 7 offers a(n useless) CHKDSK when an USB FAT flash drive was "touched" by OS/2.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: xynixme on May 05, 2017, 12:03:23 pm
Quote
Of course "ejected properly" always includes waiting for darkness, without flashing LEDs.
For me proper eject means either -
- in the drives object right click and Eject and waiting until the drive letter disappers
- or from command line eject x: and waiting until the prompt reappears what indicates the drive was properly ejected

I assume that compiling a perfect definition of "properly" has no use here, but after any of your additions the drive itself may still be active. If its LEDs are anything to go by, of course. And USB 1.0 flash drives aren't that fast. There is no relevant problem, but in this case my procedure includes both (one of) yours and mine. Hence the quoted "includes".

Or, IOW: there's no CHKDSK suggested nor required if the same OS is used again. Theoretically ejecting wrongly was a possible cause, it's about the same CHKDSK procedure, but IRL ejecting is unrelated.

I know people whom are conditioned by Windows. They switch on their computer, to boot Windows to remove a USB flash drive safely...
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: Valery Sedletski on May 05, 2017, 12:52:20 pm
For an old FAT16-based MP3 player, it is better to use new fat32.ifs with FAT16 support. The VFAT LFN's are now supported on FAT12/FAT16 too. So no need to use any REXX utilities to convert LFN's, or vfat2ea, it is all obsolete now. Please look at http://trac.netlabs.org/fat32/ for a download link.

And, it is possible that fat32.ifs will behave differently than the kernel FAT, so your error will be gone.

PS: The reserved bytes in directory entries which are used to store the EA's pointer by OS/2, are reused by M$ on FAT32 as a high word of a cluster number (because cluster number is 32-bit on FAT32). On FAT12/FAT16, these bytes are not used as cluster number is 12 or 16 bits, and no high word is needed. So, it should play no role.  And you say that the drive is considered *damaged*, not dirty by windoze, so I doubt it's something with unused bytes in dir entries. As I can see, the trash in dir entries does not prevent the disk to be mounted.

PPS: is it possible to upload the disk image somewhere? (I could suggest my ftp for upload, if so)
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: ivan on May 05, 2017, 06:08:40 pm
Quote
The word "damaged" (damaged disk, e.g. bad sector) occurs in a next dialog window, but is not relevant and blurred the memory.

The fact that flash disks don't have sectors still has to be taken into account.  If a flash cell becomes damaged, worn out, and the wear levelling firmware can't allocate a replacement then to the OS, especially windows, it will appear as a bas sector and trigger the chkdsk subsystem.

For a longer discussion on this see: http://wiki.laptop.org/go/How_to_Damage_a_FLASH_Storage_Device (http://wiki.laptop.org/go/How_to_Damage_a_FLASH_Storage_Device)
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: xynixme on May 06, 2017, 01:48:17 pm
I have removed my reply with the requested images.

Most likely it's solved. Unverified, but there may be more than one DIRTY-bit (/byte) since OS/2's FAT. If e.g. Windows 7 uses both, but OS/2 doesn't, then you may be triggering a CHKDSK. So FAT32.IFS can be a solution too, if FAT32 is aware of all modern flags and if it indeed doesn't break a FAT MP3 player.

DFSee should show that it's a DIRTY drive, but I wouldn't have noticed the DIRTY-flag now because images were created after a possible write operation, nor would I have noticed a "second" DIRTY-flag. DFSee shoudl be able to set the status to CLEAN.

No tests nor verifications yet, including a test of FAT32.IFS. So far a simplified conclusion is that the number of DIRTY-flags isn't one.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: Valery Sedletski on May 06, 2017, 02:13:45 pm
Dirty flag on FAT32 is the same as on FAT16 -- it is a bit set in the 1st FAT entry. On exFAT, though, it is set in the boot sector instead, but this is not the case.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: xynixme on May 06, 2017, 09:40:03 pm
Phrases like "second DIRTY bit" were simplified, and refered to a second setting near the expected (first) "DIRTY bit". I've now verified it (with anUSB flash drive, and FAT older than FAT32, and OS/2's FAT):

In FAT mode, DFSee's Set FAT status Clean sets both settings right. The expected classic DIRTY bit, and the unexpected second setting. DFSee prints both lines, one per setting.

The component (FAT and/or USB) of OS/2 may be outdated, because OS/2 may have to check or set both settings. Apparently Windows 7 checks both settings, and OS/2 sets one setting. DFSee checks and sets both.

If needed I can attach image files again, if one still wants to research the orginal wrong situation.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: xynixme on May 07, 2017, 02:38:23 pm
FAT32.IFS beta, with and without the /FAT switch: "could not find address of LoadTranslateTable" (ZIP distribution, replacing existing install of an old version of FAT32).

Edited output of DFSee (works), setting A and B:

Quote
Executing: dirty clean               
                                                         
(A) FAT boot record successfully updated with new DIRTY status
(B) FAT areas successfully updated with new DIRTY status     
                                                         
FAT DIRTY status  : 0x00 = CLEAN
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: Valery Sedletski on May 08, 2017, 07:39:42 pm
2Andre Heldoorn: What program writes that message? Which version? Do you have U*.DLL on LIBPATH? Are there older UFAT32.DLL on the LIBPATH? Did you unpacked all correctly?

PS: the required directory structure is preserved in the .ZIP archive.

PPS:
> What program writes that message?

Ah, I see, it is cachef32.exe. It cannot find LoadTranslateTable function in UFAT32.DLL. So, either, or no U*.DLL on LIBPATH, or an older version of UFAT32.DLL is used. Please clean up your LIBPATH from dups. eCS usually puts UFAT32.DLL in \ecs\dll, and I usually put it in \os2\dll. So, it should be only one newer copy.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: xynixme on May 10, 2017, 06:06:16 pm
Please clean up your LIBPATH from dups

Spot on, there was a second and older UFAT32.DLL in use.

Tested with FAT32. As expected, the latest version of FAT32 gets it right too, if the documented /FAT switch of FAT32.IFS is in use. No more CHKDSK dialog windows of Win7 after the FAT device, possibly including traces of VFAT data, was touched by OS/2.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: Rich442 on May 18, 2017, 05:01:14 pm
tried disk manager (Settings)?
 You can create/maintain or shrink FAT32 partitions that are readable on other systems (and assign a MBR and drive letter). Sometimes free progs like paragon free commander can work somewhat well on a volume as you describe. Sometimes I usedc a combination of tools. disk management performs many of the functions that we formally used fdisk and sfdisk for.
Title: Re: Avoiding Windows 7's CHKDSK?
Post by: xynixme on May 19, 2017, 02:51:57 am
tried disk manager (Settings)?
 You can create/maintain or shrink FAT32 partitions that are readable on other systems (and assign a MBR and drive letter).

In this specific case eCS doesn't support my FAT32 USB devices, due to at least one of the many poissible reasons why. Hence the use of classic, older FAT USB devices. A small, common shared FAT32 partition is an option too, but as such I'm not looking for a permanent shared device and the quality of my life actually will have been improved when I don't have to transfer files from Windows to OS/2 anymore (while remaining an user of OS/2). So FAT32 does look good, with its /FAT switch.