OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Storage => Topic started by: Lars on November 30, 2022, 01:01:18 pm

Title: JFS failing in a disastrous way - without the user knowing
Post by: Lars on November 30, 2022, 01:01:18 pm
Hallo,

I thought, I'd share this info with everybody.

Initial info:
0) as a standard case that is likely true for 99.9 % of all OS/2 systems in use today, my system/boot partition is JFS formatted
1) a long time ago, I had installed eCS as german NLS version. What that means for this error case is that some standard directories of the WPS are created with umlauts in their directory names, for example, "Desktop" becomes "Arbeitsoberfläche" in german.
2) as some of you might know, JFS.IFS saves its directory and file names in UNICODE and relies on the services of the UNICODE.SYS driver to properly convert between UNICODE and the default ASCII character set in use because effectively, the IFS interface only supports ASCII directory and filenames. The consequence of that is that you should NEVER hop between different character sets (by using the "chcp" command) when you create directories or files that have special characters in them (anything beyond the US437 character set).
3) for various reasons I have multiple boot configurations with different "config.sys" files, managed by the QSINIT selection menu. These files have different names: config.sys, config.smp, config.w4 etc.

Recently I decided to install this package:
https://hobbes.nmsu.edu/download/pub/os2/apps/wps/xworkplace/widgets/CPUTemp_wdgt_1-1-0.zip
That was the first error, because I have an AMD CPU and therefore this package was useless for me :-)

What followed next was the "cputemp" WPI package updating the default "config.sys" file by placing a driver into it. As it turned out later, it would not only add a driver load entry but also, apparently due to a bug in WarpIn (I later extracted the install script from the WPI file and found nothing wrong or suspicious in it), would remove the load line for UNICODE.SYS from the config.sys file. Because I had booted from a boot configuration that used another "config.sys" file (named, say "config.smp") this problem did not become apparent for as long as I was booting in the alternate boot configuration.

Eventually I booted into the standard boot config using file "config.sys". The base boot of loading all drivers and IFS (including JFS.IFS) went by without any problem but eventually, I was presented by the XWorkplace message that my desktop directory could not be found.
I was not too worried and suspected a simple OS2.INI or OS2SYS.INI corruption and decided to restore this set of files from a boot a few days earlier.

But on the next reboot, the error showed up again. From the commandline of the temporary desktop that had been created I decided to look directly at the filesystem naming of directories and files. And yes, "Arbeitsoberfläche" had been corrupted at the umlaut character. I then took the bad choice (in hindsight) and attempted to rename the directory back to what it was but it turned out that I was unable to do so. I was also unable to delete or rename that directory with any other tool like File Commander.
I decided to copy the desktop from a backup folder and was successful but intimidating enough, now I had two directories with the very same name "Arbeitsoberfläche" (on filesystem level !) but one of it could neither be renamed nor deleted. To remember: UNICODE.SYS driver was not loaded at this point.

It dawned on me that something had gone terribly wrong with the file system.Using DFSee, I triggered a chhdsk for the next reboot. The chkdsk would end without error but now I would directly run into a trap right after chkdsk was done. This was repeatable.

Fortunately, I had a completely different "development" boot partition that was completely intact (and both, UNICODE.SYS and JFS.IFS propery loading). And I had DFSee. But that was installed on the JFS partition that I could not boot from. Fortunately, as turned out later, that did not matter. The "broken" partition had directory naming problems but was intact as such and therefore I was able to readd the UNICODE.SYS load entry to "config.sys" on the "broken" partition.

I fired up the "development" boot partition and started DFSee. In one of its extended menus I found the possibility to rename directories and files on a very low file system level (so to say). Apparently, DFSee understands all low level structures of JFS and can directly overwrite sector content.

Subsequently it turned out that not only "Arbeitsoberfläche" had been "renamed" disastrously but also all OTHER DIRECTORIES below "Arbeitsoberfläche" that contained umlauts (yes there are quite a few directories below the "Desktop" directory ...).

For each of the affected directories, I had to hop back and forth between giving them an intermediate name like "Arbeitsoberflxxxx" and "Arbeitsoberfläche" to force the proper directory name (intermediate reboots required). After multiple reboots and some traps I was finally able to restore the partition to an acceptable state, but I had to recreate quite a few directories under the "Desktop" directory and move things to their proper place.


I think it would at least be helpful if JFS.IFS would, ideally already on bootup, report a warning if it fails in using UNICODE.SYS services instead of reporting nothing. That would have pointed me to the problem and would have saved me hours of work.
(I can understand that JFS.IFS will need to be able to continue nonetheless because otherwise you would not be able to fix the config.sys file and readd the UNICODE.SYS load entry ...).

This behaviour is surely an absolute NOGO for a new user (unless you only accept english speakers or people using the US/UK NLS version as new users).






Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Mike Kölling on November 30, 2022, 01:29:49 pm
Hi Lars,

Thanks for your detailed explanation about the JFS issue without the unicode.sys driver.
As a German user of ANOS with a lot of Umlaute, because I renamed a lot of the originally English named folders to common German names, I think it might help me to avoid similar waste of time for finding out the root cause of the problem.

Thanks and Greetings from Potsdam / Germany,
Mike
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Dariusz Piatkowski on November 30, 2022, 02:11:56 pm
Oh dude...I feel your pain!!!

1st of all, glad that you recovered, but yeah, as I read your post I likewise had the tell-tale feeling of "butterflies" in my stomach, old memories came back here.

A few years back I made the move from HPFS386 to JFS, but since this was NOT a fresh install I managed to convert stuff (standard XCOPY partition move approach, etc.) while UNICODE.SYS was NOT preceeding the JFS call in CONFIG.SYS. Bottom line being the NEW system was quite usable, appeared to work just fine, but soon enough strange things started to happen and eventually I had a major JFS problem on my hands.

In this case the root cause was my unfamiliarity with true JFS requirements and the fact that I missed tossing UNICODE.SYS into my CONFIG.SYS ahead of the JFS IFS call and ahead of the partition move process itself.

Your root cause is quite different from that, but at the time of my troubles I do recall thinking to myself: "well, if the darn thing is THAT important why isn't the IFS at least shouting - should really scream at me though - when it finds it missing?".
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Andi B. on November 30, 2022, 03:25:18 pm
These Umlaut problem hits us OS/2 users since the beginning. The root of the problem is IMO IBMs decision to translate important folders to something with special characters (Arbeitsoberfläche). A lot of us where bitten by this ages before JFS arrived when we simply played with codepage settings in config.sys or CHCP. It's the same with HPFS(386). When you change the codepage setting in config.sys, chances a good that on the next reboot you're Desktop is lost/destroyed. Only difference is - we learned not to do this. Never ever play with codepage setting (at least the first one). We also have to learn not the delete unicode.sys when using JFS. Maybe Martin can delete these package from hobbes. Or when it's really a warpin issue then I think it is solved in current releases. I remember reading about that problem (deleting unicode from config.sys) a while ago.

Quote
because I renamed a lot of the originally English named folders to common German names
I hope you don't make this problem worse and don't use umlauts. Even if the meaning of the translation sounds not as exact as it could be, it's far better to use another word and circumvent any special character in (system relevant) file/folder names.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Dave Yeo on November 30, 2022, 05:34:04 pm
I ran into the same issue, excepting with a English install, there were just some MP3's with umlauts. In my case everything was fine, including the umlaut files being fine, until I booted to a different install, where those file names became garbage (actually I forget if it was umlauts or other special characters). Took a while to figure out it was a missing unicode.sys and once I did figure it out, I renamed the problem files without unicode.sys installed.
This is a known problem that was reported to Digi long ago and he fixed the problematic CONFIGSYS="DEVICE=?:\OS2\rdmsr.sys | UNIQUE" line, which upon not finding rdmr.sys, replaced the first line containing .sys, usually unicode.sys. There's an issue, but it is private. Remy filed it. Unluckily it was quite a while before it was uploaded to Hobbes.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Remy on November 30, 2022, 07:46:47 pm
...
Subsequently it turned out that not only "Arbeitsoberfläche" had been "renamed" disastrously but also all OTHER DIRECTORIES below "Arbeitsoberfläche" that contained umlauts (yes there are quite a few directories below the "Desktop" directory ...).

For each of the affected directories, I had to hop back and forth between giving them an intermediate name like "Arbeitsoberflxxxx" and "Arbeitsoberfläche" to force the proper directory name (intermediate reboots required). After multiple reboots and some traps I was finally able to restore the partition to an acceptable state, but I had to recreate quite a few directories under the "Desktop" directory and move things to their proper place.


I think it would at least be helpful if JFS.IFS would, ideally already on bootup, report a warning if it fails in using UNICODE.SYS services instead of reporting nothing. That would have pointed me to the problem and would have saved me hours of work.
(I can understand that JFS.IFS will need to be able to continue nonetheless because otherwise you would not be able to fix the config.sys file and readd the UNICODE.SYS load entry ...).

This behaviour is surely an absolute NOGO for a new user (unless you only accept english speakers or people using the US/UK NLS version as new users).

Hi,

I had a similar issue after installing Lotus office program in French.
All directories having characters with accent where lost.
I have DFSee and with the help of the developper, I could find bad directories and correct them without use of a backup system.

In my case, they was a problem with unicode and my issue arrived after having put duplicated folder into the trash cam.

Of course, warning message would be nice in case of language with accent chars.   
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Martin Iturbide on November 30, 2022, 08:25:02 pm
I ran into the same issue, excepting with a English install, there were just some MP3's with umlauts. In my case everything was fine, including the umlaut files being fine, until I booted to a different install, where those file names became garbage (actually I forget if it was umlauts or other special characters). Took a while to figure out it was a missing unicode.sys and once I did figure it out, I renamed the problem files without unicode.sys installed.
This is a known problem that was reported to Digi long ago and he fixed the problematic CONFIGSYS="DEVICE=?:\OS2\rdmsr.sys | UNIQUE" line, which upon not finding rdmr.sys, replaced the first line containing .sys, usually unicode.sys. There's an issue, but it is private. Remy filed it. Unluckily it was quite a while before it was uploaded to Hobbes.

Hi

Dave. I didn't get right if the bug on WarpIn was fixed or not?
Lars, was this an specific version of WarpIn, or was it with the latest one (1.0.26)?

Regards
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Martin Iturbide on November 30, 2022, 08:31:31 pm
Hi

I quickly installed "CPUTemp_wdgt_1-1-0.zip" with Warpin 1.0.24 (ArcaOS 5.0.6) and I confirm it removed UNICODE.SYS from my config.sys.

Reverted everything, upgraded to WarpIn 1.0.26 and I got the same thing. UNICODE.SYS gets removed.

I also don't see anything strange on the CPUTEMP WarpIn script, just the "CONFIGSYS="DEVICE=?:\OS2\rdmsr.sys | UNIQUE" which should not remove UNICODE.SYS.

Should it be reported as WarpIn bug?

Regards
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Martin Iturbide on November 30, 2022, 09:05:49 pm
Hi

I just find out (thanks to this thread) that Digi updated the CPUTemp plugin to version 1.1.2 on the "System Load (https://hobbes.nmsu.edu/?search=SystemLoad_1-1-0.zip)" package.
I think he uses a different script to try to avoid this bug:
 EXECUTE="CONFIGSYS | $(102)\drvins.cmd | -d ?:\ -f $(102)\rdmsr.sys -i 1"
 DEEXECUTE="$(102)\drvins.cmd | -d ?:\ -f $(102)\rdmsr.sys -i 0"

Anyways, I think I will be removing "CPUTemp_wdgt_1-1-0.zip" from hobbes, because System Load (https://hobbes.nmsu.edu/?search=SystemLoad_1-1-0.zip) has a newer version.

Regards
 
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Dave Yeo on November 30, 2022, 10:01:34 pm
Hi

I quickly installed "CPUTemp_wdgt_1-1-0.zip" with Warpin 1.0.24 (ArcaOS 5.0.6) and I confirm it removed UNICODE.SYS from my config.sys.

Reverted everything, upgraded to WarpIn 1.0.26 and I got the same thing. UNICODE.SYS gets removed.

I also don't see anything strange on the CPUTEMP WarpIn script, just the "CONFIGSYS="DEVICE=?:\OS2\rdmsr.sys | UNIQUE" which should not remove UNICODE.SYS.

Should it be reported as WarpIn bug?

Regards

Warpin is working as it should based on discussion on the testers list. What happens is if rdmsr.sys is not installed, it matches up with the first *.sys in config.sys and replaces that with rdmsr.sys. Usually it is unicode.sys at the top of config.sys.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Martin Iturbide on December 01, 2022, 12:27:50 am
Hi Dave

Checking the WarpIn wpi_prg.inf file.
Code: [Select]
UNIQUE[(statement2)]
This makes sure that the statement part of the CONFIGSYS attribute occurs only once, i.e. will be unique in CONFIG.SYS, by searching for a similar line and replacing that line with the new line. If no such similar line is found, the new line is appended to the bottom of CONFIG.SYS (that position can be changed with ADDTOP, ADDBEFORE, ADDAFTER ).

All searching is case-insensitive.

This assumes that statement contains a "=" character, as all CONFIG.SYS statements do. The exact behavior of UNIQUE depends on the syntax:

If UNIQUE is specified all alone (without the brackets and statement2), CONFIG.SYS is searched for a line containing the part before the "=" char of statement.

This is only useful for single CONFIG.SYS statements such as SET xxx= whose left part (before "=") will occur only once in CONFIG.SYS.

Example: CONFIGSYS="PROTECTONLY=YES | UNIQUE" would replace any "PROTECTONLY=" statement with the new line. If no such line exists, a new line is added to the bottom (that position can be changed with ADDTOP, ADDBEFORE, ADDAFTER).

If UNIQUE is specified with the (statement2) part, CONFIG.SYS is searched for a line which contains the part before the "=" character of statement and statement2 after the "=" character as a substring.

Example:

CONFIGSYS="BASEDEV=DANIS506.ADD | UNIQUE(IBM1S506.ADD)"

would search for a line containing both BASEDEV= and IBM1S506.ADD and replace that line with BASEDEV=DANIS506.ADD.

This syntax is required for adding device drivers, because there will surely be more than one BASEDEV statement in CONFIG.SYS.

This also works with the DEVICE= statements which can have a full path specification, as follows:

CONFIGSYS="DEVICE=$(1)\bin\crashsys.drv | UNIQUE(stable.drv)"

This would replace a line containing both DEVICE= and stable.drv with DEVICE=[path_of_package_1]\bin\crashsys.drv, using macro resolution.

Do you mean that it should be used on the WarpIn script like...
  "CONFIGSYS="DEVICE=?:\OS2\rdmsr.sys | UNIQUE(rdmsr.sys)"
to avoid getting the Unicode.sys replaced?

Regards
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Dave Yeo on December 01, 2022, 01:22:21 am
Do you mean that it should be used on the WarpIn script like...
  "CONFIGSYS="DEVICE=?:\OS2\rdmsr.sys | UNIQUE(rdmsr.sys)"
to avoid getting the Unicode.sys replaced?


I'm not an expert with Warpin, it seems that would still not add rdmsr.sys if it wasn't in config.sys to begin with but would avoid replacing unicode.sys. Guess that is why Digi ended up with a bit of Rexx to do it.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Martin Iturbide on December 01, 2022, 02:10:33 pm
Thanks for the feedback.

The second thing.
Will it makes sense to request Arca Noae to put a "Boot Warning Text" for JFS.IFS if it does not find UNICODE.SYS loaded ? Does it makes sense? Can it be possible in boot time to detect this for JFS.IFS?

Regards
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Remy on December 01, 2022, 02:21:24 pm
Thanks for the feedback.

The second thing.
Will it makes sense to request Arca Noae to put a "Boot Warning Text" for JFS.IFS if it does not find UNICODE.SYS loaded ? Does it makes sense? Can it be possible in boot time to detect this for JFS.IFS?

Regards

Does it have sense to have unicode if language is EN only ?
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Lars on December 01, 2022, 04:22:45 pm
Thanks for the feedback.

The second thing.
Will it makes sense to request Arca Noae to put a "Boot Warning Text" for JFS.IFS if it does not find UNICODE.SYS loaded ? Does it makes sense? Can it be possible in boot time to detect this for JFS.IFS?

Regards

Yes, that would definitely make sense.

I would need to check the OpenJFS source code, but I would think that JFS.IFS uses the KEE API calls "KernCreateUConvObject", "KernStrToUCS" and "KernStrFromUCS" to create a UConv object, convert from an ASCII string to a Unicode string (taking into account active system codepage), convert from a Unicode string to an ASCII string, respectively.

I would suspect that "KernCreateUConvObject" is called very early, in the IFS loading/initialization stage (but not already on FS_INIT as that executes in Ring 3 whereas calling "KernCreateUConvObject" requires executing in Ring 0). The call will definitely return with an error if UNICODE.SYS is not loaded (these KEE API calls are the ones that require UNICODE.SYS to be loaded !!!) and that would be the indirect way to find out.

If it is not possible to already display a message on boot, then there are options:
1) have a daemon executing that could show a warning message. But that requires to implement a daemon interface in JFS.IFS and also to start the daemon application itself which of course can also go wrong.
2) Another, potentially better option is to write to the system log (the syslog.exe thing that nobody ever uses ...) because that would be a persistent logging (logging happens to a file) but that in turn also requires an additional device driver and daemon (log.sys and logdaem.exe) when the logging is to be done from a device driver or IFS. I also do not know if that conflicts with the role of the IFS giving access to files (if log.sys buffers to memory and logdaem.exe writing much later in the process, then that could work). In that case, AN should ensure that LOG.SYS and LOGAEM.EXE are always added to config.sys on each install.
3) last but not least to use the low level file system device driver calls to write a file in the root directory (say with fixed string: "Attention: UNICODE.SYS NOT LOADED !"). I guess that is the best option as it requires no further components to be up and running. But I seem to remember that also requires ring 0 execution which again would mean, you'd need to delay the logging until the first IFS call executing in Ring0 and also after the filesystem has been mounted (obviously).

But the problem should be solvable.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Lars on December 01, 2022, 04:29:15 pm
Hi

I quickly installed "CPUTemp_wdgt_1-1-0.zip" with Warpin 1.0.24 (ArcaOS 5.0.6) and I confirm it removed UNICODE.SYS from my config.sys.

Reverted everything, upgraded to WarpIn 1.0.26 and I got the same thing. UNICODE.SYS gets removed.

I also don't see anything strange on the CPUTEMP WarpIn script, just the "CONFIGSYS="DEVICE=?:\OS2\rdmsr.sys | UNIQUE" which should not remove UNICODE.SYS.

Should it be reported as WarpIn bug?

Regards

Warpin is working as it should based on discussion on the testers list. What happens is if rdmsr.sys is not installed, it matches up with the first *.sys in config.sys and replaces that with rdmsr.sys. Usually it is unicode.sys at the top of config.sys.

That is a very unfortunate side effect of specifying UNIQUE when "DEVICE" is the key string to look for because "DEVICE" will typically occur multiple times in config.sys. WarpIn should better be changed to do nothing if UNIQUE is specified and the key string is found multiple times. That is at least safe behaviour.
And if UNIQUE(xxxx) is specified, it should be changed to create the target string if no matching source string statement is found.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Andi B. on December 01, 2022, 05:16:10 pm
If you fear that something changes your config.xxx and you overlook to check the changes before reboot you can also start a rexx script from your startup.cmd which parses your current config.xxx. Though I don't know how to find out which config.xxx file is used by osinit.... But maybe simpler than changing jfs.ifs. Moreover chances are high that the output text from jfs.ifs while booting will be overlooked pretty easy. 
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Dariusz Piatkowski on December 01, 2022, 06:17:54 pm
If you fear that something changes your config.xxx and you overlook to check the changes before reboot you can also start a rexx script from your startup.cmd which parses your current config.xxx...

You bring up a reason for why for every install I actually save a copy of CONFIG.SYS first and then afterwards compare the new copy to that backup.

This of course could be automated just as you say with a little REXX script very easily.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Lars on December 02, 2022, 07:10:45 am
Sorry, but this is ridiculous.
If JFS.IFS can easily fail in a catastrophic way, then it is necessary to prevent that or at least to mediate the risk.
A new user would have no clue of what is going on. But maybe everyone has already accepted that there will be no new users...

Anyway, after thinking about it a bit more:
JFS.IFS can attempt to attach to UNICODE.SYS via a dehelp call and that can already be done on FS_INIT (call for initializing JFS.IFS) which is executed already on bootup.
If that attach fails you know that UNICODE.SYS is not loaded, display a warning message and even force the user to hit Return to continue boot process.
That takes around 10 lines of code in JFS.IFS.
But I am not the master of disaster. That is AN. They have the source code. And of course, it's closed source
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Andi B. on December 02, 2022, 10:23:21 am
I wouldn't call it ridiculous. I would say this is another shortcoming/bug/problem/nuisance of one part/program of our platform. Of course it may be easy to improve this situation with Lars proposed changes. No clue if David will implement it when Lars creates a ticket.

It's good that Lars shares his experience here to tell others about some problems and how to circumvent it. That's what a forum is good for.

But I also see a lot of other shortcomings. And when I were in the position to give priorities, this one would have a low priority. We only can wait and see what David says about this. Or is there any chance to get access to the latest JFS sources?

Quote
display a warning message and even force the user to hit Return to continue boot process
I fear this can be a problem with USB keyboards. The systems I have do not serve USB keyboards (with OS/2) until the end of the boot process. Every such "hit return to continue" condition while booting (while loading all the drivers from config.sys) needs the old PS/2 keyboard which luckily is also attached to step over. Here there is no chance to respond to such messages with the USB keyboard.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: OS4User on December 02, 2022, 10:33:37 am
It may have a sense to convert UNICODE.SYS from "DEVICE" to "BASEDEV".
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Lars on December 02, 2022, 12:10:00 pm
I wouldn't call it ridiculous. I would say this is another shortcoming/bug/problem/nuisance of one part/program of our platform. Of course it may be easy to improve this situation with Lars proposed changes. No clue if David will implement it when Lars creates a ticket.

It's good that Lars shares his experience here to tell others about some problems and how to circumvent it. That's what a forum is good for.

But I also see a lot of other shortcomings. And when I were in the position to give priorities, this one would have a low priority. We only can wait and see what David says about this. Or is there any chance to get access to the latest JFS sources?

Quote
display a warning message and even force the user to hit Return to continue boot process
I fear this can be a problem with USB keyboards. The systems I have do not serve USB keyboards (with OS/2) until the end of the boot process. Every such "hit return to continue" condition while booting (while loading all the drivers from config.sys) needs the old PS/2 keyboard which luckily is also attached to step over. Here there is no chance to respond to such messages with the USB keyboard.

There are device drivers that do not continue silently in case of error (I think MAC drivers often expect user confirmation to continue). So this is a general shortcoming and can hit you on all sorts of other occasions.
Does your system no longer have "legacy USB" support ? I can use the USB keyboard before OS/2 takes over the USB host
controllers ...

By the way: I cannot create a ticket as I am no paying AN customer. Somebody with an AN license would need to do that.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Lars on December 02, 2022, 12:11:39 pm
It may have a sense to convert UNICODE.SYS from "DEVICE" to "BASEDEV".

That is only going to remove the ordering dependency (UNICODE.SYS has to come before JFS.IFS). I can live with that ordering dependency but I cannot live with data loss because a vital driver wasn't loaded.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Andi B. on December 02, 2022, 12:59:15 pm
Quote
There are device drivers that do not continue silently in case of error (I think MAC drivers often expect user confirmation to continue). So this is a general shortcoming and can hit you on all sorts of other occasions.
Yes it is a problem with other drivers too. That's why I'm think about. Specifically with ethernet drivers I often observed this when f.i. they do not find the correct chip. But other drivers too. I think PAUSEONERROR (or similar) can prevent stop the boot process. But using this is no good solution too because you probably wouldn't recognize the error message then as it flies away much to fast on current (< 10 years :-) ) hardware.

Quote
Does your system no longer have "legacy USB" support ? I can use the USB keyboard before OS/2 takes over the USB host
controllers ...
It does as long as something (ACPI, USB, ...?) very early in the boot process takes over control until the time when all the USB stuff, including usbhid is initialized (very late). In any case there is a big window during boot where USB keyboards do not work. From all what I understand and all the discussions about that (testers list, AN tickets) this window (non working usb keyboard) can not be eliminated. I never figured out how to sort config.sys to make this window as small a possible. But I got the impression it's impossible to cover all these places where device drivers may fire up this "hit return" message. We have to life with this when we want (or have) to use USB keyboards instead PS/2.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Martin Iturbide on December 02, 2022, 02:19:34 pm
...
If that attach fails you know that UNICODE.SYS is not loaded, display a warning message and even force the user to hit Return to continue boot process.
...

Hi

I agree that it should be very useful for JFS.IFS to display a warning message if UNICODE.SYS is not loaded. For what I know UNICODE is very important for JFS. But I'm not complete sure about if had to pause the boot process forcing the user to hit "return".

But I think I can open the ticket and let the implementation decisions to Arca Noae, since it is their product.

By the way, as far as it was discussed AN JFS.IFS is closed source and they treat the IBM source code base of it as the IBM DDK license.

Regards
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Dariusz Piatkowski on December 02, 2022, 02:39:41 pm
Lars, everyone...

Sorry, but this is ridiculous.
...
But I am not the master of disaster. That is AN. They have the source code. And of course, it's closed source

Ahh, you see, and therein lies part of the challenge that causes majority of us to take a more pragmatic approach to dealing with these issues.

Yes, you are absolutely right about this being ridiculous, however, what other choice do we really have? There certainly isn't much ability for any of us to action it, even folks like yourself who have the technical know-how are finding themselves having their hands tied.

Sooo...instead of continuing the risk exposure I will do the best I can, with the tools/controls available to me today, to avoid that risk.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Lars on December 02, 2022, 03:44:46 pm
Quote
There are device drivers that do not continue silently in case of error (I think MAC drivers often expect user confirmation to continue). So this is a general shortcoming and can hit you on all sorts of other occasions.
Yes it is a problem with other drivers too. That's why I'm think about. Specifically with ethernet drivers I often observed this when f.i. they do not find the correct chip. But other drivers too. I think PAUSEONERROR (or similar) can prevent stop the boot process. But using this is no good solution too because you probably wouldn't recognize the error message then as it flies away much to fast on current (< 10 years :-) ) hardware.

Quote
Does your system no longer have "legacy USB" support ? I can use the USB keyboard before OS/2 takes over the USB host
controllers ...
It does as long as something (ACPI, USB, ...?) very early in the boot process takes over control until the time when all the USB stuff, including usbhid is initialized (very late). In any case there is a big window during boot where USB keyboards do not work. From all what I understand and all the discussions about that (testers list, AN tickets) this window (non working usb keyboard) can not be eliminated. I never figured out how to sort config.sys to make this window as small a possible. But I got the impression it's impossible to cover all these places where device drivers may fire up this "hit return" message. We have to life with this when we want (or have) to use USB keyboards instead PS/2.

1) it's not PAUSEONERROR that influences the driver behaviour. It is what error code a driver returns from its INIT routine. It can report either "OK" or "NON_CRITICAL_ERROR" or "SOME_ERROR" (various different errors). "OK" and "NON_CRITICAL_ERROR" means to continue without interruption, "SOME_ERROR" means to stop and wait for keypress.
Obviously, the idea is that a severe error should prevent booting from continuing without the user noticing.

2) I don' t have a long time window of USB not working. I have a short time window of USB not working where the OS/2 HC drivers are loading and turning off legacy USB (start of time window) and finally have taken over the HC HW and the relevant USB devices mouse/keyboard have been enumerated (end of time window). On my system, that time window is about 5 seconds, keyboard works before and after that time window (that is, in particular during the bootup process where the video is in text mode). But I am using my USB drivers and I don't know if the AN drivers have a different behaviour.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Lars on December 02, 2022, 03:48:55 pm
...
If that attach fails you know that UNICODE.SYS is not loaded, display a warning message and even force the user to hit Return to continue boot process.
...

Hi

I agree that it should be very useful for JFS.IFS to display a warning message if UNICODE.SYS is not loaded. For what I know UNICODE is very important for JFS. But I'm not complete sure about if had to pause the boot process forcing the user to hit "return".

But I think I can open the ticket and let the implementation decisions to Arca Noae, since it is their product.

By the way, as far as it was discussed AN JFS.IFS is closed source and they treat the IBM source code base of it as the IBM DDK license.

Regards

Waiting for a keypress was a suggestion. You could also do a simple delay of say 30 seconds or so (that would also be possible).
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Andi B. on December 02, 2022, 04:00:54 pm
Quote
Waiting for a keypress was a suggestion. You could also do a simple delay of say 30 seconds or so (that would also be possible).
That would be a very good work around. (Solution would be putting back unicode.sys into config.sys when it's not before jfs.ifs). Such 30s delay would be also very useful for ACPI or USB test versions when they outdate. A problem testers are bitten every now and then. F.i. even Keith at his presentation in Colonge...
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Martin Iturbide on December 02, 2022, 04:53:42 pm
Hi

I had added the suggestion on the Arca Noae Ticketing System.
- https://mantis.arcanoae.com/view.php?id=3364 (https://mantis.arcanoae.com/view.php?id=3364)

If it is not clear enough, I will appreciate if someone (with access) can add more information to the ticket.

Regards

Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: OS4User on December 02, 2022, 06:07:44 pm
Waiting for a keypress was a suggestion. You could also do a simple delay of say 30 seconds or so (that would also be possible).

It is very good suggestion. You proved that unicode.sys  is vital for system with JFS.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Dave Yeo on December 02, 2022, 10:39:28 pm
Waiting for a keypress was a suggestion. You could also do a simple delay of say 30 seconds or so (that would also be possible).

It is very good suggestion. You proved that unicode.sys  is vital for system with JFS.

OTOH, I went 18 months without unicode.sys with en_US locale. It was only when I found this,
Code: [Select]
2-17-22 11:04a         <DIR>      0 ----  Gel
 2-17-22 11:13a         <DIR>      0 ----  Sessions effac
 2-17-22 10:58a         <DIR>      0 ----  Verwijderde sessies
[\code]
instead of this,
[code]
2-17-22 11:04a         <DIR>      0 ----  Gelöschte Sitzungen
 2-17-22 11:13a         <DIR>      0 ----  Sessions effacées
 2-17-22 10:58a         <DIR>      0 ----  Verwijderde sessies

That I investigated. Seems without unicode.sys, you can write and read umlauts etc as long as always without unicode.sys.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Lars on December 03, 2022, 10:32:26 am
You can. Until you end up with a system where you can no longer remove directories where directory names were manipulated without UNICODE.SYS being loaded. That messes up the unicode character string on a very basic filesystem level. And then, DFSee is your last resort to rectify that.

If you always use en_US locale, you will very likely NEVER run into this problem, because codepage 437 is the default for about anything that relates to an ASCII string in OS/2. This is about people that either have to use a different codepage or where the OS installation installs a different codepage as the primary codepage. For a german OS/2, codepage 850 is installed as the default (with 437 as a secondary codepage) even though codepage 437 would also do for most of the things (german umlauts are also supported with codepage 437 and even have the same byte value as in codepage 850).
In short, for a few special characters, codepage 437 and 850 are identical. For the lower 128 characters, 437 and 850 are also identical (and that includes 0-9 a-z A-Z : \ .).
For a french codepage this will be comparable but certainly not for any cyrillic codepage.
Which also makes it clear why no modern operating system is using codepages anymore.

I repeat: this is an absolute NOGO for a commercial operating system.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Andi B. on December 03, 2022, 11:30:12 am
Quote
I repeat: this is an absolute NOGO for a commercial operating system.
Do you wanna send this complain to IBM or ArcaNoae? :-)

Lars, I absolutely understand your frustration. Really.

But IBM will never change this I fear. I also fear AN will not change the basics of any file system. Moreover I fear they have a lot of stuff to do with higher priorities. But we will see how they respond to the ticket.

For all with access to the ticket system - I think it's worth to read Stevens conclusion on this problem. The ticket should be linked now. Thanks Dave.

Edit - this ticket I meant. Scroll down to the end. https://mantis.arcanoae.com/view.php?id=1223
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Andreas Schnellbacher on December 03, 2022, 04:01:12 pm
For all with access to the ticket system
As an Arca Noae customer, I have access to the tickets. But #1223 (https://mantis.arcanoae.com/view.php?id=1223) seems to be hidden for me, while Martin's #3364 (https://mantis.arcanoae.com/view.php?id=3364) works for me.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Andi B. on December 03, 2022, 05:09:59 pm
Sorry to hear that. This ticket was from 2017 and dealt with the very same problem. But there it leads to a trap when reading DVDs. It's closed since a long time. I commented there back then. Maybe that's the reason why I see it. I don't want to copy text from there but the bottom line is - it would be good if the original CPUTEMP warpin package would have been tested better. But people make mistakes. The problems with this package are 1) it deletes UNICODE.SYS and 2) it puts rdmsr.sys at the top of config.sys. It should be better places after the IFS lines.

Unfortunately wic -t does not test such failures in a package. But a modified CPUTEMP package should be available from here - ftp://digi.os2.snc.ru/pub/SystemLoad/CPUTemp-20170504-2.zip. Mind I didn't test this. Use it on your own risk! Or search for the latest AND remember to check what an installer does with your config.sys.

By the way - this is a very good example that's not always a good idea to save old versions of packages/drivers...

I think it's not a big problem to copy this one sentence from the ticket - 'The only way for traps to never happen is for humans to never make
mistakes.'
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Lars on December 04, 2022, 11:31:36 am
Being an OS/2 user is a little bit like being a drug addict:
you know that it's bad for your health but you just can't let go.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Andi B. on December 04, 2022, 12:34:31 pm
Completely agree with you Lars ;-)

One note I want to add here - since a while I always check my config.sys (diff) after some installer wanted to change it. There are so many installers which changes libpath and/or path. At least since all these ported software luckily we have, it's a big problem when some installers put something in front of the %unixroot% entries.
Title: Re: JFS failing in a disastrous way - without the user knowing
Post by: Digi on December 04, 2022, 11:28:25 pm
it would be good if the original CPUTEMP warpin package would have been tested better

This issue was fixed a long time ago, right after the first reported problem. I recommend using the latest CPUTemp from the SystemLoad package: http://os2.snc.ru/product-SystemLoad.html (http://os2.snc.ru/product-SystemLoad.html)