Author Topic: OS/4 kernel and extended KEE  (Read 96076 times)

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #75 on: January 13, 2014, 01:57:26 am »
Quote
And how was made screenshot above? ;)

The .LONGNAME EA usage and such ugly name conversion - is WPS-related thing only. So, if you  format partition before WPS - it works normally with it - because it detects it as "long-name-file-system" at the own startup.

I don't know why your screen shot looks okay.

Try this:
Format the RAMDISK as HPFS, using whatever method you think works.
Use the WPS to copy a file with a LFN to the disk.

Now, see if it did it right:
The WPS shows the LFN.
Drop to a command line, and do a DIR on the RAMDISK. The file is stored with a short file name.

That is NOT correct. The DIR should show the LFN, because HPFS can store the file with the proper LFN. Only FAT needs to be managed by the short file name /EA long name kludge.

In addition, copy the file back to a normal disk. WPS still shows the LFN because of the EA, but DIR shows that it is saved as the short file name.

Then, use the command line to copy the file from the RAMDISK, using the LFN. You get File not found, because the file system knows that it contains the long file name, so no translation, using the EA, is necessary.

HPFS, and JFS, are supposed to store files using the LFN, not a shortened version of the name.  Both of them do store the LFN, if you use the command line (and FC is probably using that interface, which may explain why it works). However, The WPS seems to think that it needs to do the same as it does with FAT, and that is NOT correct operation. This is obviously a bug in the WPS, or the file system, or both. As I said, I can duplicate the problem on a real disk, switching between FAT format and JFS format. The problem persists until a reboot, then it gets it right.

I suspect that simply unmounting, and mounting, the drive would also fix the problem. YES! that does fix it. I started miniLVM (RMB on the Drives object, and select Manage Volumes). Use that to set the drive letter to <none>. Close miniLVM, saving the change. Open miniLVM again, and add the drive letter to the drive. Close miniLVM, saving the change (which will mount the drive). Now, it works properly. Now, to figure out how to do that with commands, which can be done after the WPS is active (preferably BEFORE the drive gets used), but even better would be BEFORE the WPS gets started.

The conclusion, that I come to, is that the proper format needs to be in place when the IFS starts, as well as when the WPS starts, or there will be a mismatch, and it will not work properly. Unmount then mount, causes the whole thing to be evaluated again, and it gets it right.

In addition, if I leave the format as FAT, and try to use the RAMDISK for %TEMP% I have problems if a program stores a temporary file, that should have a LFN. The file name gets shortened, and the program can no longer find the file that it stored (this is not unique to the RAMDISK, it also happens on a real disk that is formatted as FAT). That means that the disk needs to be either HPFS, or JFS, to be useful, and when it is one of those two formats, it doesn't work properly in all cases, until it gets unmounted, and mounted.

Quote
But bug still there.

I will have to look at this one later...

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #76 on: January 13, 2014, 05:25:36 am »
Quote
Now, to figure out how to do that with commands, which can be done after the WPS is active (preferably BEFORE the drive gets used), but even better would be BEFORE the WPS gets started.

Well, that was an interesting adventure.

The LVM drive letter cannot just be hidden, and restored. It must be changed to something else, then changed back. This must also be done AFTER the WPS has started, or it gets ignored by the WPS. The format can be done BEFORE the WPS has started.

I want the RAMDISK to have letter R:, and be formatted as JFS, so my solution is:
In QSSETUP.CMD (root of the boot drive):
Code: [Select]
RAMDISK R:
MD C:\TEMP
log save c:\log.txt
RAMDISK R: makes a default size RAMDISK (only memory above 4 GB), and gives it LVM letter R:.
MD C:\TEMP makes a directory on that RAMDISK (remember that this command file works in the QSINIT shell, so the RAMDISK letter is C: )
log save c:\log.txt saves the log file to the RAMDISK (I need a file on the disk, to prevent a problem later on).

Next, I need a way to do the format to either HPFS, or JFS. I do that with makeR.CMD, which contains:
Code: [Select]
COPY R:\* M:\RTRACKER

FORMAT.COM R: /FS:JFS /V:RAMDISK < RAMFMTRESP.TXT

md R:\TEMP
COPY R:\* M:\RTRACKER This copies the files from the root of the RAMDISK to a directory RTRACKER (which must exist). I do this for two reasons: 1) to be able to transfer files from the shell to OS/2. 2) If there is no file, the boot will stop demanding an Enter.
FORMAT.COM R: /FS:JFS /V:RAMDISK < RAMFMTRESP.TXT This does the format to JFS, in this case, and adds a volume label. (Simply substitute HPFS for JFS, if you wish to use HPFS.) RAMFMTRESP.TXT contains the single letter "Y" (without the quotes) as the first letter in the first line, followed by a blank line (this automatically responds Yes to the format command).
md R:\TEMP Since the drive was formatted, I need to restore the contents. Here i simply make a directory. You could UNZIP a fixed set of files, and directories, if you wish. REMEMBER that all of this will go away with a system shutdown, or crash.

Next, I put the lines:
Code: [Select]
CALL=C:\OS2\CMD.EXE /Q /C C:\makeR.cmd
SET tmp=r:\temp
SET temp=r:\temp
SET tmpdir=r:\temp
at the end of CONFIG.SYS.
CALL=C:\OS2\CMD.EXE /Q /C C:\makeR.cmd runs makeR.CMD, as described above.
The SET commands should replace any that may already exist, for those environment variables (you may want to make sure that it all works, before actually trying to use it for %TEMP%).

Now, for the part that makes it work properly:
At the beginning of STARTUP.CMD (the file may not exist, if not, create it). I put:
Code: [Select]
lvm /driveletter:[ Vol.00 ],X:

lvm /driveletter:[ Vol.00 ],R:

<whatever else you have/need in STARTUP.CMD>
lvm /driveletter:[ Vol.00 ],X: This changes the RAMDRIVE letter to X:, which is a drive letter that I use, but it must never exist at boot time. Note the spaces in "[ Vol.00 ]".
lvm /driveletter:[ Vol.00 ],R: This changes the RAMDISK letter back to R:, which causes a new drive mount, which resolves the previous discrepancies.

Note, that I cannot just assign a different drive letter (X:) when QSINIT makes the drive, and then change it to the final letter (R:) to mount the drive. That works okay, BUT if I do that the SET variables won't be assigned when CONFIG.SYS is executed, and they will demand an Enter because the drive doesn't exist. If the variables are assigned, after the WPS starts (in STARTUP.CMD), they are not set in the permanent environment. The double drive letter change is the easiest way around that, that I have found.

It is somewhat messy, but it works properly (I think). Any suggestions, or comments are welcome.

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #77 on: January 13, 2014, 06:18:52 am »
Quote
It is somewhat messy, but it works properly (I think).

Well, it worked on my Lenovo ThinkPad L530. It appears that I need to delay the LVM commands, in STARTUP.CMD, on my Lenovo ThinkPad T510, or it happens too fast (or something like that), and the mount fails to cause a reevaluation of the drive. I added a DIR command as the first entry in STARTUP.CMD, and now it is working. Obviously, a more elegant solution is needed.   :o

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4788
  • Karma: +99/-1
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #78 on: January 13, 2014, 07:55:31 am »
I use sleep for similar affects in startup.cmd, eg sleep 3 for a 3 second delay.
IIRC sleep is part of the binutils or now coreutils.

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #79 on: January 13, 2014, 05:11:59 pm »
I use sleep for similar affects in startup.cmd, eg sleep 3 for a 3 second delay.
IIRC sleep is part of the binutils or now coreutils.

Thanks. I know about Sleep in REXX, I didn't realize that it was also a command (although I was wondering why not).

It also occurred to me, that LVM /hide, and then LVM /driveletter, should do the job. It does, as long as there is more time between the commands. On my L530, I can use a a one second delay, and it works. On the T510, I need a 2 second delay. So, change the STARTUP.CMD section to:
Code: [Select]
lvm /hide:[ Vol.00 ]
sleep 1s
lvm /driveletter:[ Vol.00 ],R:

<whatever else you have/need in STARTUP.CMD>

Under some circumstances, the sleep might need to be changed to 2s. A quick test is to use the Drives object to copy a file with a long file name to the RAMDISK, then go to a command line, and do DIR. If the file name is a shortened name, add another second, reboot, and try again. With HPFS, and JFS, the file name should always be the long file name. I suppose that a test, to be sure that the drive really got mounted properly, would be a good idea. If it doesn't get mounted properly, some very strange things can happen.

I think I just fixed the final problem, but more testing is required.

I think those who do not have sleep.exe would have enough of a delay when the system says it can't find the program, but that could be bit confusing.

walking_x

  • Full Member
  • ***
  • Posts: 101
  • Karma: +0/-0
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #80 on: January 15, 2014, 10:33:10 am »
While in the QSINIT shell, it said it did it. When I boot OS/2, it hesitates for a second, or so, and reboots.
Try new version.... I am not sure - it will be worse or better.
This ugly "immediate reboot" bug buried somewhere in the deep of memory init and a kind of hero required to dig it out.

Code: [Select]
lvm /hide:[ Vol.00 ]

You can rename it to more friendly string in QSINIT (or I can make it - by default):
      lvm rename %ramdisk% 0 PAE_PT

And - this method is perfect, but I'm still does not understand - why HPFS is not ready from MPTSTART.
I'm added format after the :END label and that is all:
Code: [Select]
@ECHO OFF
....
CALL K:\MPTN\BIN\NBSETUP.CMD
:END
format q: /fs:hpfs <k:\mptn\bin\letter_y.txt

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #81 on: January 15, 2014, 08:39:26 pm »
Quote
Try new version.... I am not sure - it will be worse or better.
This ugly "immediate reboot" bug buried somewhere in the deep of memory init and a kind of hero required to dig it out.

Well, the latest version is working now, on the T510 (where I had the problem). I will need to try it on the other systems later.

When I do get a FAT32 drive created, it is not possible to format the drive as either HPFS, or JFS, but the long file name problem does not exist. I don't know how well it would work if I use it for %TEMP%. The normal FAT32 speed problem doesn't seem to exist (I don't use the FAT32 cache, and I don't use EAs with FAT32).

Can you add an option to force the file system to be FAT32, even with a small RAMDISK?

Quote
You can rename it to more friendly string in QSINIT

The LVM name really doesn't matter, but a name that means something to the user might make it easier to figure out what it is.

Quote
And - this method is perfect, but I'm still does not understand - why HPFS is not ready from MPTSTART.
I'm added format after the :END label and that is all:
Code: [Select]

@ECHO OFF
....
CALL K:\MPTN\BIN\NBSETUP.CMD
:END
format q: /fs:hpfs <k:\mptn\bin\letter_y.txt

It doesn't seem to matter when the format is done, the drive needs to be unmounted (hide), and mounted (assign a letter) again (with a delay between them), before the file system, and the WPS get synchronized. It also seems that the synchronization is not complete, until the desktop is fully populated, and all programs in the STARTUP folder are finished initializing. That is probably some sort of bug in the file systems, and/or the WPS. Doing the hide, then assigning a letter, seems to successfully get it fixed (probably something in the LVM mount procedure that fixes the problem).

I have tried doing the format as the very first CALL statement. I have done it, as you suggest, in MPTSTART (which is probably not a good place to do it), and I have done it as the last thing before the WPS starts up. The timing of the format doesn't matter. I have also tried doing the LVM /HIDE, and  LVM /DRIVELETTER command in that same section. The sleep command does not work until the WPS starts up, and the problem is not fixed without it. Doing the HIDE and DRIVELETTER , with a one, or two second delay between them, as the first thing in STARTUP.CMD does seem to fix the problem.

I did make a REXX program to do a WPS copy to the drive, and then check to be sure that the long file name was used. I use RexxAutoStart to cause a delay of one second, before it starts each program in the STARTUP folder, with a 0.75 second delay between the startups. That still sees the short file name, until RexxAutoStart has finished. Then, the long file name starts to work.

I am sure that we are seeing the results of some patching to make mounting disks possible (mostly USB, but other things as well). Normally, it is not physically possible to remount a drive as fast as it is done in this case, so the problem would not exist. It is also likely, that the RAMDISK is seen as being a permanently mounted disk, so some rediscovery code may not be used, until it is forced.

walking_x

  • Full Member
  • ***
  • Posts: 101
  • Karma: +0/-0
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #82 on: January 16, 2014, 11:11:49 am »
Can you add an option to force the file system to be FAT32, even with a small RAMDISK?
FAT32 can be made by reduction of cluster's size.
Only one thing defines type of file system - number of clusters.
If it greater 64k - it will be FAT32, if less 4096 - FAT12.
So, minimum FAT32 partition size - 32Mb (64k * 512) - and I can add key - "try to create FAT32 if possible".

p.s. i added key "fat32" for ramdisk - in new version.
« Last Edit: January 16, 2014, 03:24:49 pm by walking_x »

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #83 on: January 16, 2014, 06:33:17 pm »
Quote
p.s. i added key "fat32" for ramdisk - in new version.

Okay, I will try that later.

I have found that HPFS still shows a problem, that JFS does not show. I have a directory, on the real disk, that has various types of files, and a few directories. When I use the WPS to copy it to the HPFS RAMDISK, I quickly get a message that says "SYS0254: An incorrect extended attribute name was used." I can copy the same directory to a JFS RAMDISK, with no trouble.
Code: [Select]
SYS0254: An incorrect extended attribute name was used.

EXPLANATION: The extended-attribute name contains a character that is
incorrect.

ACTION: Correct the extended-attribute name and retry the operation.

Unfortunately, it doesn't tell me what it is trying to copy. I also see the problem when using XCOPY from the command line. That seems to indicate that unzip-6.0-5.zip is causing the problem. Looking at it with the FM2 EA editor, I don't see a long file name. I need to do more isolation, to see if I can figure this out. I also need to format a real partition to HPFS, and see if the problem happens there.

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #84 on: January 16, 2014, 11:44:18 pm »
Quote
I also need to format a real partition to HPFS, and see if the problem happens there.

Okay, the EA problem happens on a real volume, formatted as HPFS, so it must be a compatibility problem between JFS and HPFS, and not the RAMDISK. I can't find any long name EA that looks like it might cause such an error, but I haven't isolated the actual cause yet.

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #85 on: January 17, 2014, 12:04:34 am »
Quote
p.s. i added key "fat32" for ramdisk - in new version.

Well, it works on the T510. FAT32 doesn't show the EA problem either, although I don't enable EAs in the FAT32 driver.

walking_x

  • Full Member
  • ***
  • Posts: 101
  • Karma: +0/-0
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #86 on: January 18, 2014, 02:10:46 pm »
And one remark for all. If you downloaded QSINIT to "look at later" - update it.
At least for ram disk (a lot of serious bugs was fixed since "new year release" of it).
« Last Edit: January 18, 2014, 03:29:04 pm by walking_x »

Lars

  • Hero Member
  • *****
  • Posts: 1271
  • Karma: +65/-0
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #87 on: January 18, 2014, 04:02:53 pm »
And one remark for all. If you downloaded QSINIT to "look at later" - update it.
At least for ram disk (a lot of serious bugs was fixed since "new year release" of it).

It would be helpful to state where exactly the up to date version can be found.
There is one version here:
http://ru2.halfos.ru/core/downloads/qsinit/qs_ldr.zip

but obvously the current version can be found here:
ftp://212.12.30.18/public/QS/QS_LDR.ZIP

Are you going to move this latest version to http://ru2.halfos.ru ?

Lars

walking_x

  • Full Member
  • ***
  • Posts: 101
  • Karma: +0/-0
    • View Profile
Re: OS/4 kernel and extended KEE
« Reply #88 on: January 18, 2014, 04:17:51 pm »
halfos.ru have some "hosting in change" problems now...
qs_ldr.zip on it - is VERY old.

ftp://212.12.30.18/public/QS/QS_LDR.ZIP or
http://212.12.30.18/other/QS_LDR.ZIP are always up to date.

On later versions bldlevel can print its number.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 4714
  • Karma: +41/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: OS/4 kernel and extended KEE
« Reply #89 on: January 18, 2014, 04:24:26 pm »
Hi walking_x

Do you think it can be possible to start labeling the version of QSLoader, like putting it "QS_LDR-20140118.zip" ?
Just an idea to see it easier which one is newer. You are also free to set up a release/versino number for it.

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.