OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Setup & Installation => Topic started by: Dariusz Piatkowski on April 13, 2018, 05:58:25 pm

Title: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 13, 2018, 05:58:25 pm
So for a number of years I used to run a 4Gig RAM machine build. Then a couple of months ago I read a thread on the forum regarding the configuration of a RAM disk and I thought: "heck, why not, let's give it a try"...LOL. I had two spare DDR3 sticks laying around, installed them, for a total of 8Gig, configured the RAM disk to use just 1Gig and it's been pretty good so far.

I use the RAM disk (ramfs.ifs) for the following:

1) ZIP file processing
2) PMMail
3) GCC

...but I think there is more opportunity there.

So I'm curious, if you have a RAM disk deployed, what are you putting on there?

For example, I have the following 'temp' directories currently defined in my CONFIG.SYS:

SET TMPDIR=G:\var\tmp
SET TEMP=G:\TMP
SET TMP=G:\TMP

I think I could readily point TMP and TEMP to my ramdisk...not so sure about TMPDIR. That location in the Unix world is meant to be a temporary persistent storage location, meaning, it should survive a re-boot but may be occasionally purged. So far the YUM/RPM install is using this and I'd rather not mess around with it. However, TMPDIR is used by GCC, so for now I also have this set in my GCC environment config file.

The TMP and TEMP on the other hand store a bunch of runtime stuff, for example Firefox always creates etilqs_xxx files there. I noticed that if FF crashes they remain behind, it appears to be safe to delete them, at least I occasionally do since they are supposed to be just temporary SQLITE database files.

What else though? Any suggestions?
Title: Re: RAM disk and temp folders...what else?
Post by: David Graser on April 13, 2018, 07:20:38 pm
I had mentioned the following in another post.

I created a RAM drive named "X" that uses both low and high memory using ArcaOS 5.02 in the Computer\setup folder. I have it set to HPFS.

I moved my temporary directories to my RAM drive in my config.sys.
SET TMP=x:\temp
SET TEMP=x:\temp
SET TMPDIR=x:\temp

I also operate my system with my OS2.INI and OS2SYS.INI moved to my RAM drive during operatiion using a program called UPDINI12a.zip.

http://hobbes.nmsu.edu/download/pub/os2/util/wps/updini12a.zip

rem SET USER_INI=D:\OS2\OS2.INI
SET USER_INI=X:\TEMP\OS2.INI
rem SET SYSTEM_INI=D:\OS2\OS2SYS.INI
SET SYSTEM_INI=X:\TEMP\OS2SYS.INI

RUN=D:\OS2\UPDINI.EXE D:\OS2\ 10

Because I use this program, I added the following line as the first line in my config.sys.

CALL=D:\OS2\XCOPY.EXE D:\OS2\OS*.INI D:\VAR\TEMP /T /R /O

This backs up both files during bootup to another location in cause something drastic were to happen. I also archived my system before and after I made all these changes.
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 13, 2018, 08:14:57 pm
Hi David!

...I also operate my system with my OS2.INI and OS2SYS.INI moved to my RAM drive during operatiion using a program called UPDINI12a.zip.

http://hobbes.nmsu.edu/download/pub/os2/util/wps/updini12a.zip
...

Yes, I do remember reading this. I think the move of the OS2 INI files is a neat idea. Quite frankly though I would be very hesitant to do so only because these files (in my experience) are so sensitive to any changes. In particular the lack of an orderly "shut-down" functionality of the RAM disk storage which would allow you to bring these files back onto the disk in a major problem for me.

My INI files are often corrupted...and this is with just regular system use. Subsequently upon re-boot I often find that I bookt to messed-up Desktop and most WarpCenter icons being gone. This requires a shut-down, followed by a restore of the last good version, etc, etc...which I of course have, but it is tedious to say the least. All this happens despite the fact that I routinely run checkni and otherwise attempt to maintain them.
Title: Re: RAM disk and temp folders...what else?
Post by: Doug Bissett on April 13, 2018, 08:25:58 pm
Quote
I use the RAM disk (ramfs.ifs) for the following:

RAMFS.IFS is not very good, compared to the RAMDISK supplied by ArcaOS (or QSINIT). RAMFS.IFS uses real memory (from the 3.5 GB that OS/2 knows about). The ArcaOS, or QSINIT, RAMDISK will use memory above that limit. I use the ArcaOS RAMDISK, formatted as HPFS, and make the HPFS cache as small as possible (64 KB), so it doesn't get used much (the cache slows the RAMDISK, it would be faster without the cache, but HPFS insists on at least 64KB). I use JFS for my real disk, except for a FAT32 drive for communicating with windows. Using JFS seems to prevent a LOT of file corruption because of the way it works.

Quote
I use the RAM disk (ramfs.ifs) for the following:

1) ZIP file processing
2) PMMail
3) GCC

I am not sure what you are doing with these. If you zip a file to RAMDISK, you then need to copy it to a real disk, or it will be gone when you reboot. PMMail can use the RAMDISK for temporary files. It is probably not a good thing to use it for accounts etc. What do you do with GCC (which should be managed by RPM/YUM)?

Quote
I think I could readily point TMP and TEMP to my ramdisk...not so sure about TMPDIR.

I put all three to my RAMDISK. The big advantage is that it gets cleaned up at every boot, and you do need to remember that small detail.

The big disadvantage of using a RAMDISK, is that you need to load it when (actually before) the system starts, and copy anything that you want to keep, before the system is shut down (the ArcaOS / QSINIT RAMDISK, can, sometimes, retain data over a warm boot, but you can't count on that).

I have tried a number of things, but the only thing that is really safe, is to use it for the %TEMP% stuff.

Quote
I also operate my system with my OS2.INI and OS2SYS.INI moved to my RAM drive during operatiion using a program called UPDINI12a.zip.

Personally, I wouldn't even consider doing that. There are enough problems with maintaining the INI files, without complicating things by putting them on volatile storage.
Title: Re: RAM disk and temp folders...what else?
Post by: David Graser on April 13, 2018, 09:37:56 pm
I have used it for over a month with system hangs at times and it continues to work fine.  My machine is not a critical mission machine.  I have little to worry about and with the safeguards I have taken, I am not concerned.  I am always testing and experimenting with my system. 

David
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 13, 2018, 10:09:53 pm
Hey Doug!

Quote
I use the RAM disk (ramfs.ifs) for the following:

RAMFS.IFS is not very good, compared to the RAMDISK supplied by ArcaOS (or QSINIT). RAMFS.IFS uses real memory (from the 3.5 GB that OS/2 knows about). The ArcaOS, or QSINIT, RAMDISK will use memory above that limit.

Hmm...truth be told I never thought to pay enough attention during my testing to see where the memory being used by the ramdisk comes from. Buy yeah, you are right, I just moved a massive (600MB) video file over and yikes...not happy with the results!!!

OK, so I'm going to give the AOS RAMDISK a try...looks like EXE only, do you just toss that into your CONFIG.SYS and either RUN or CALL from there? I figured a run from there would be best to start up as early as possible, and if not then maybe startup.cmd?

Quote
I use the RAM disk (ramfs.ifs) for the following:
1) ZIP file processing
2) PMMail
3) GCC

I am not sure what you are doing with these. If you zip a file to RAMDISK, you then need to copy it to a real disk, or it will be gone when you reboot. PMMail can use the RAMDISK for temporary files. It is probably not a good thing to use it for accounts etc. What do you do with GCC (which should be managed by RPM/YUM)?

Ahhh, sorry, this needs more explanation for sure:

1) ZIP processing - I use 'RPF Zip Control', it's a GUI for one of the standard un/zip utils...so each time I open up a ZIP file it creates a temp dir and usually tosses some stuff in there. Clearly stuff I do not care to keep afterwards and would want it to run as quickly as possible. So when you open up a large ZIP, say something like the OS/2 Toolkit for example, the RAMDISK makes a huge difference

2) PMMail - just the standard attachments that you open up in the email, I see pmmail dumps them out into a file, so similar approach to #1 above, but because the filesizes are so much smaller it's barely perceptible

3) GCC - ahh, this is the biggest beneficiary actually...running any compile process (as I have it setup right now) causes the various object files to be created, I'm simply telling GCC to use the ramdisk instead of the disk, literally this almost gives a feel of an all in-memory process short of reading the sources and libraries
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 13, 2018, 10:39:14 pm
Ooppss ==> CORRECTION!!!

Hey Doug!

Quote
I use the RAM disk (ramfs.ifs) for the following:

RAMFS.IFS is not very good, compared to the RAMDISK supplied by ArcaOS (or QSINIT). RAMFS.IFS uses real memory (from the 3.5 GB that OS/2 knows about). The ArcaOS, or QSINIT, RAMDISK will use memory above that limit.
...
OK, so I'm going to give the AOS RAMDISK a try...looks like EXE only, do you just toss that into your CONFIG.SYS and either RUN or CALL from there? I figured a run from there would be best to start up as early as possible, and if not then maybe startup.cmd?...

Well, so I created a ramdisk using the AOS RAMDISK.EXE, re-did the test and discovered the same pattern of usage. Hmm...scratched my head and had a closer look at my RAMFS.IFS setup...as it turns out it is the very same RAMDISK.EXE that comes with AOS....so I'm a little confused Doug about your comment re: "RAMFS.IFS is not very good...".

Looking at the README with RAMFS it specifically calls out the install of RAMFS.IFS and creation of a disk with RAMDISK.EXE, which is precisely what I do today. I am not sure how the AOS process would be any different?
Title: Re: RAM disk and temp folders...what else?
Post by: Dave Yeo on April 13, 2018, 11:08:06 pm
I used ramfs for quite a while and it works well, the memory it allocates is swappable and doesn't affect things like shared memory. The problem is that it doesn't quite work right with symlinks for some reason.
Under System Setup, there is a ramdisk icon, this will allow you to use the memory above 4GB (actually more like 3.5GB) as a ram disk, so you should have something like a 4.5GB ram disk once you set it up. Depending on your system, this memory may survive a warm reboot as well, it does here.
You should be able to use it much like you use the ramfs volume, including putting tmpdir on it. Most Linux systems use a similar file system (tmpfs) for /tmp and I believe point tmpdir at it.
Title: Re: RAM disk and temp folders...what else?
Post by: Andy Willis on April 14, 2018, 12:29:29 am
I didn't use ramdisk.ifs much because it used otherwise valuable memory.  With the ramdisk in AOS, I have a 20MB ramdisk used from otherwise unused memory.  I have my swappath set to it as well as temp and Mozilla temp space.  My primary reason for using it for these is to limit writes to my SSD.
Title: Re: RAM disk and temp folders...what else?
Post by: Doug Bissett on April 14, 2018, 07:55:56 am
I see that Dave pointed you to the setup for the AOS RAMDISK. It is part of the new OS2LDR. RAMFS.IFS may be included in ArcaOS, but I recommend not using it.

Quote
1) ZIP processing - I use 'RPF Zip Control'

Ouch. That is an ancient program, that I quit using many years ago, simply because there are better ways to do zip/unzip. ARCHIVE Tool is one of them, and it is part of ArcaOS.

Quote
2) PMMail - just the standard attachments that you open up in the email, I see pmmail dumps them out into a file, so similar approach to #1 above, but because the filesizes are so much smaller it's barely perceptible

PMMail (3.x) has a setting to point to the drive, and directory, that you want to use for temporary files. Where it makes the biggest difference, is when you use the option to compress the message files, in a folder.

Quote
3) GCC - ahh, this is the biggest beneficiary actually...running any compile process (as I have it setup right now) causes the various object files to be created, I'm simply telling GCC to use the ramdisk instead of the disk, literally this almost gives a feel of an all in-memory process short of reading the sources and libraries

Okay. I thought that is probably what you meant, but what you said could confuse people.

Quote
I used ramfs for quite a while and it works well, the memory it allocates is swappable and doesn't affect things like shared memory. The problem is that it doesn't quite work right with symlinks for some reason.

Actually, there are a lot of things that don't work quite right, with RAMFS.IFS. I don't remember the details, but it wasn't possible to use it with some program that I was using when I found out about the limitations. The ArcaOS RAMDISK (which is the same as what QSINIT uses) is far better, and it can use the HPFS file system (it is one of the settings), which gives it the full capability of the HPFS file system. If you have enough memory, you can use more than one RAMDISK, with different file systems, but I can't think of any good reason to do that. The memory above 4 GB also has a method to use it as program storage (when not using it as a RAMDISK), but it needs an API to make that more useful. (look it up in the QSINIT documentation).

Quote
I didn't use ramdisk.ifs much because it used otherwise valuable memory.  With the ramdisk in AOS, I have a 20MB ramdisk used from otherwise unused memory.  I have my swappath set to it as well as temp and Mozilla temp space.  My primary reason for using it for these is to limit writes to my SSD.

The AOS RAMDISK is most useful when you have more than 4 GB of memory. I have 8 GB of memory, and use a 4.5 GB RAMDISK, without taking more than a few bytes (for the driver), away from my main memory. When you have 4 GB of memory, for OS/2, you will never swap. I would suggest, that 20 MB is pretty small, for a swap disk. One of my systems is only 3 GB, and I allocate a 1 GB RAMDISK from it, but I don't use it for swapping, mostly because a 2 GB machine will rarely swap anyway, and if it does, 20 MB isn't going to help much.
Title: Re: RAM disk and temp folders...what else?
Post by: Ben on April 15, 2018, 03:43:56 pm
I am just wondering if I have this straight; is the RAMDISK.exe that comes with the RAMIFS the same RAMDISK.exe that comes with ArcOS?

There seems to be some debate.

Right now I'm using RAMIFS loaded via CONFIG.SYS but if there is something better or issues with the former...
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 15, 2018, 05:33:15 pm
Hey Ben!

I am just wondering if I have this straight; is the RAMDISK.exe that comes with the RAMIFS the same RAMDISK.exe that comes with ArcOS?

There seems to be some debate....

LOL, great question...that was precisely what I was pointing out in my previous response to Doug's recommendation to use just RAMDISK.EXE. The RAMFS package which I have installed here is composed of:

1) RAMFS.IFS
2) RAMDISK.EXE

The README specifically instructs you to deploy RAMFS.IFS and use RAMDISK.EXE for the more "advanced" features, literally, here is the quote from the document:

Code: [Select]
...
Installation and use
~~~~~~~~~~~~~~~~~~~~
Add "IFS=d:\path\RAMFS.IFS" to CONFIG.SYS and reboot. During boot, it will
show a short version message.

Then, from an OS/2 prompt, use RAMDISK.EXE to create a RAM drive. To create a
drive R:, type "RAMDISK R:" To have a RAM drive created at every boot, you
can add "CALL=d:\path\RAMDISK R:" to CONFIG.SYS.
...

I have AOS 5.0.2 here, so all my comparisons are done against the install media only (no actual live installation) and all I can find is the RAMDISK.EXE file. Comparing that file to the one included in the RAMFS package yields a match, so they are the same files.

My suspicion therefore is that the use of RAMDISK.EXE is meant to allow you to create additional ramdisks following system boot, since you clearly could not mount RAMFS.IFS once you are up-and-running. That makese sense.

So my question is: can someone look up the details of the AOS configuration? As Dave Yeo pointed out:

Quote
...Under System Setup, there is a ramdisk icon, this will allow you to use the memory above 4GB...

???
Title: Re: RAM disk and temp folders...what else?
Post by: Doug Bissett on April 15, 2018, 06:38:03 pm
Quote
LOL, great question...that was precisely what I was pointing out in my previous response to Doug's recommendation to use just RAMDISK.EXE. The RAMFS package which I have installed here is composed of:

1) RAMFS.IFS
2) RAMDISK.EXE

The README specifically instructs you to deploy RAMFS.IFS and use RAMDISK.EXE for the more "advanced" features, literally, here is the quote from the document:

You seem to have completely missed the fact that the recommended RAMDISK has nothing, at all, to do with RAMFS. It is a completely NEW RAMDISK, based on the QSINIT project (look it up, if you feel that is necessary).

REMOVE all references to RAMFS. Then go to System Setup, and run the RAM Disk program to configure it. I recommend using the HPFS selection for most things. You will not find any IFS for it, since it exists in the new OS2LDR. You do need the BASEDEV=HD4DISK.ADD line, in CONFIG.SYS, but the RAM Disk program will do that for you.
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 15, 2018, 06:58:45 pm
Doug,

...oh boy... ::)

You seem to have completely missed the fact that the recommended RAMDISK has nothing, at all, to do with RAMFS. It is a completely NEW RAMDISK, based on the QSINIT project (look it up, if you feel that is necessary)...

I think it is you who have completely missed that fact that I am NOT running AOS...so the whole time you have been responding in this thread you are making a massive assumption that not just myself, but perhaps others as well, Ben for example, are running AOS. Meanwhile, I have clearly stated that I have not deployed AOS, I am running W4.52 FP6, 201SMP kernel (if you must know).

Sooo...that means the following:

1) QSINIT is not part of my install, unless i manually deploy it, which I have not done
2) RAMFS package deployed on my machine is composed of two elements:
- RAMFS.IFS
- RAMDISK.EXE

Therefore, given the above and having hopefully set the matter straight, is it starting to come together that regardless of what you say here, unless I have AOS deployed and can test/verify/assess I simply can not deploy your suggested setup regardless of how many times you tell me "...RAMDISK has nothing, at all, to do with RAMFS. It is a completely NEW RAMDISK...".
Title: Re: RAM disk and temp folders...what else?
Post by: Dave Yeo on April 15, 2018, 07:26:23 pm
It's easy to lose track of which versions of OS/2 people are running :)
Title: Re: RAM disk and temp folders...what else?
Post by: xynixme on April 15, 2018, 09:40:48 pm
Quote
1) ZIP file processing
2) PMMail
3) GCC

What else though? Any suggestions?
Data files, or EXEs, which are frequently accessed, or called, by a quick & dirty or unoptimized solution. If anything to avoid both physical harddisk abuse and the stress of the sounds of Microsoft's updates services.
Title: Re: RAM disk and temp folders...what else?
Post by: Doug Bissett on April 15, 2018, 10:14:24 pm
Quote
I think it is you who have completely missed that fact that I am NOT running AOS.

Yes, I did miss that. I still suggest using QSINIT, and it's RAMDISK (if you don't use ArcaOS). The small effort to switch is very much worth the effort, especially if your machine has 4 GB of memory, or more. The RAMDISK is far better than any other that is available, by a large margin.
Title: Re: RAM disk and temp folders...what else?
Post by: Ben on April 15, 2018, 10:23:01 pm
I have been using RAMFS.IFS for years. I have not have any problems that I could ever associate with it. However, if the RAMDISK that comes with QSINIT uses a smaller memory footprint then I have no trouble giving it a go, especially in these days of dwindling usable OS/2 memory.

I have an old version of QSINIT on a maintenance partition  that I can use for testing and I shall do that when I get the time.

Thanks for the heads-up.
Title: Re: RAM disk and temp folders...what else?
Post by: Lars on April 15, 2018, 11:13:32 pm
You are missing the point. QSINIT uses physical memory beyond the 4GB border. And THAT makes it so valuable. RAMIFS steals from the memory available to the OS.
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 17, 2018, 04:59:26 am
Quote
I think it is you who have completely missed that fact that I am NOT running AOS.

Yes, I did miss that. I still suggest using QSINIT, and it's RAMDISK (if you don't use ArcaOS)...

Dave is right, there are many versions and configurations of OS/2 floating around, so it's super easy to fork-off on that road, no biggie, easy miss, but you gave me the needed "push" ;D to re-visit the QSINIT utility.

I had previously installed it some time ago to investigate the functionality, but it was in it's early days and I didn't feel it offered much to warrant replacing the standard process.

So here is a bit of an update, and this will have specific info for those running HPFS386, because certain option available in the current latest version of QSINIT doesn't quite "play nice" with a HPFS386 machine...read on!

OK, so here is what I have done:

1) install QSINIT
2) add BASEDEV=HD4DISK.ADD to my CONFIG.SYS
3) add 'ramdisk Y: hpfs' to QSSETUP.CMD

...seemed like that's all that was required to get a plain-vanilla install. However, upon a re-boot I was greeted by numerous HPFS error messages pertaining to just that RAMDISK. I emailed dixie (QSINIT) author who confirmed this and explained that HPFS386 in particular has a problem with the RAMDISK in that it always thinks the volume is dirty because it always considers the volume to be a brand NEW volume.

Indeed, that required me adding the new 'Y' (my drive letter) to the CONFIG.SYS HPFS386.IFS line:
IFS=G:\IBM386FS\HPFS386.IFS /AUTOCHECK:FGY

Now with this auto-check in-place upon a re-boot there is a very quick CHKDSK that runs on the RAMDISK, because HPFS386 thinks it's NEW and therefore dirty and must be cleaned. The QSINIT dirty flag set utility does not help. But, this process is super quick and gives me a nice, nearly 5Gig ramdisk, plenty of storage.

Further on, I attempted to use the '/1' switch for HD4DISK.ADD, this however leads to a hard stop at bootup where the RAMDISK is not initialized and the device itself is only available as a DASD unit, the error message below is what I see:
Code: [Select]
HFS0109: The disk device driver for the 386 HPFS volume on drive Y: does not support memory above 16MB. The volume was mounted for DASD access only.

I do not understand the cause, but somehow the suggested '/1' parameter for use with HPFS formatted ramdisks does not work here. Playing with the 'useallmem' setting in HPFS386.INI does not help. Could be the size of my HPFS386 cache (only 64Meg), or it could be due to some difference between regular HPFS and HPFS386.

Anyways, I will let dixie know about this, maybe he is willing to debug further.
Title: Re: RAM disk and temp folders...what else?
Post by: Doug Bissett on April 17, 2018, 07:07:17 am
Quote
...seemed like that's all that was required to get a plain-vanilla install. However, upon a re-boot I was greeted by numerous HPFS error messages pertaining to just that RAMDISK. I emailed dixie (QSINIT) author who confirmed this and explained that HPFS386 in particular has a problem with the RAMDISK in that it always thinks the volume is dirty because it always considers the volume to be a brand NEW volume.

I never used HPFS386, so I have no idea what it might think of the RAMDISK. It shouldn't be running it anyway, because it is HPFS, and not HPFS386, but it probably doesn't know the difference. You could try using FAT, but that would limit you to a 2 GB partition. FAT32 might work, but it is very slow.

Quote
Further on, I attempted to use the '/1' switch for HD4DISK.ADD, this however leads to a hard stop at bootup where the RAMDISK is not initialized and the device itself is only available as a DASD unit, the error message below is what I see:

Code: [Select]
HFS0109: The disk device driver for the 386 HPFS volume on drive Y: does not support memory above 16MB. The volume was mounted for DASD access only
I do not understand the cause, but somehow the suggested '/1' parameter for use with HPFS formatted ramdisks does not work here. Playing with the 'useallmem' setting in HPFS386.INI does not help. Could be the size of my HPFS386 cache (only 64Meg), or it could be due to some difference between regular HPFS and HPFS386.

I never used "/1". You probably need "EARLYMEMINIT=TRUE" near the top of your CONFIG.SYS, but I seem to recall that that isn't friendly with HPFS386. Another possibility, is an OLD BIOS setting (something about supporting OS/2), which is only required with warp 3. Otherwise, it should be off

The way that I use the RAMDISK, is to use all memory above what OS/2 uses normally, for the RAMDISK. Then, I use HPFS for the RAMDISK (I also make the cache as small as possible - 64K- because the cache slows it down). All of my other partitions are JFS (which is probably better than HPFS386 anyway), except for one that is FAT32, for sharing files with windows.
Title: Re: RAM disk and temp folders...what else?
Post by: walking_x on April 17, 2018, 07:48:22 am
I do not understand the cause, but somehow the suggested '/1' parameter for use with HPFS formatted ramdisks does not work here. Playing with the 'useallmem' setting in HPFS386.INI does not help. Could be the size of my HPFS386 cache (only 64Meg), or it could be due to some difference between regular HPFS and HPFS386.
/1 forces system to use ancient STRAT1 method. This is fatal for most of file system drivers. Actually, only kernel's FAT implementation and HPFS.IFS able to use it.

HPFS386 looks more advanced as well as JFS. Nothing to do here.

Another bad thing with HPFS386 is that is may be slower, than common HPFS driver - on the RAMDISK. Because of smart caching, probably.

Any way, ramdisk may save your SSD, when used as TEMP. Also, there is a setting in seamonkey - path to cache location, it can point to the ram disk (nice thing for security too). Better than nothing ;)
Title: Re: RAM disk and temp folders...what else?
Post by: Lars on April 17, 2018, 08:22:37 am
Why would you use HPFS386 on a ramdisk anyway ?
Optimizing sector accesses is irrelevant for a ram disk. Caching will even make things worse: you waste additional memory for no gain in speed (after all, on a ram disk, each data sector is already in memory ...)
And it does not look like the OP cares about the file access protection features that HPFS386 offers on top of HPFS.

However, I don't know if you can load HPFS.IFS and HPFS386.IFS at the same time. Most likely not ...

As to STRAT1 method: as far as I can remember from the OpenJFS code I think JFS.IFS will fall back to the STRAT1 method when it cannot find a STRAT3 (flat addressing) entry point. Of course that is irrelevant nowadays as the 32-bit (most recent) OS2DASD.DMD supports the STRAT3 entry point. But the old 16-bit version of OS2DASD.DMD did not.
On the other hand, I think that JFS.IFS will only work with the new OS2DASD.DMD + OS2LVM.DMD.

Also: VFDISK.SYS (the virtual floppy disk driver, a legacy block device driver) only supports the STRAT1 entry point. But given the fact that you can only format virtual floppies to FAT anyways that should not be a problem.
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 17, 2018, 03:46:36 pm
Hey Doug,

...The way that I use the RAMDISK, is to use all memory above what OS/2 uses normally, for the RAMDISK. Then, I use HPFS for the RAMDISK (I also make the cache as small as possible - 64K- because the cache slows it down). All of my other partitions are JFS (which is probably better than HPFS386 anyway), except for one that is FAT32, for sharing files with windows.

Yup, similar configuration to what I currently have. My operating partitions are all HPFS386, the nightly backup partitions are HPFS386 as well but no CACHE on them. Neither is the RAMDISK, else it defeats the purpse.
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 17, 2018, 04:43:18 pm
Hi Lars,

Why would you use HPFS386 on a ramdisk anyway ?
Optimizing sector accesses is irrelevant for a ram disk. Caching will even make things worse: you waste additional memory for no gain in speed (after all, on a ram disk, each data sector is already in memory ...)
And it does not look like the OP cares about the file access protection features that HPFS386 offers on top of HPFS.

However, I don't know if you can load HPFS.IFS and HPFS386.IFS at the same time. Most likely not ...

You would think you are correct in that thinking. I would not use HPFS386 on a ramdisk if I had a choice, but it is a HPFS or HPFS386 type of a decision, so I have no choice in this matter, I am forced to run HPFS386 only. Since HPFS386 allows you to shut the cache off for the ramdisk created by QSINIT itself, as best as I can tell this should not a problem.

..and yet the numbers I see tell an entirely different story...

I am basing this last conclusion on the output I see from 'cache386 /o' command and throughput numbers I see from 'Lars Commander' during a file copy operation. If I then compare the speed throughput for the ramdisk with the cache ON and OFF I see the following:

1) HPFS386 cache ON - 600 Meg file copy from HD to ramdisk
Peak => 108,586K
Avg => 104,278K
Last => 108,420K

2) HPFS386 cache OFF
Peak => 53,079K
Avg => 26,325K
Last => 27,736K

I specifically selected a large source file for this initial test so that it is bigger than my HPFS386 cache of 64Meg. Yet despite this, I am seeing the same level of performance difference across both small and large files, that being nearly 4x faster throughput on cached ramdisk.

Lars, unless your throughput measure is being skewed by something else, the numbers here tell a different story. The only negative downside to keeping the HPFS386 cache on for the ramdisk is the 'pollution' of the true cache for the other real HDs in my system.

Sysbench (0.9.5d) 'Disk I/O' test is completely messed up by the ramdisk, the only number that actually comes back is the bus xfer which is 355MByte/s. However, the 'File I/O' results give 216157 for non-cache and 260371 for cached configurations.

Therefore, in the end, given what I see above and the fact that most files tossed on the ramdisk will normally be small (therefore implying the true cache pollution to be small) it actually seems like leaving the HPFS386 cache ON for the ramdisk gives the better performance.
Title: Re: RAM disk and temp folders...what else?
Post by: walking_x on April 17, 2018, 06:16:34 pm
You can try File i/o test in sysbench :) I found one, was launched on HPFS386 too. Something crazy a bit (may be, because of "lazy = v: OFF" in hpfs386.ini):
 File I/O - Drive V:
   4Kb seq.   Uncached w :    32837.920    Kilobytes/second
   4Kb seq.   Uncached r :   723265.941    Kilobytes/second
   4Kb random Uncached w :    10788.018    Kilobytes/second
   4Kb random Uncached r :   652518.493    Kilobytes/second
   4Kb seq.   Cached   w :    33585.820    Kilobytes/second
   4Kb seq.   Cached   r :  1252849.141    Kilobytes/second
....
   32K seq.   Uncached w :    44419.361    Kilobytes/second
   32K seq.   Uncached r :  2136524.963    Kilobytes/second
   32K random Uncached w :    44644.568    Kilobytes/second
   32K random Uncached r :  2346214.469    Kilobytes/second


Disk i/o subsubsystem adds a delay, any way. Real transfer speed on large block sizes is not far from PC memory speed.
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 17, 2018, 06:32:05 pm
You can try File i/o test in sysbench :) I found one, was launched on HPFS386 too...

So here are the results of the Sysbench 'File I/O' test that I reported the results for in my previous update. Take a look at them, they are pretty close, but at times significantly different. I think what may be getting muddied up by Sysbench is this whole 'Cached' vs 'Uncached' result...I have no idea how Sysbench controls that.
Title: Re: RAM disk and temp folders...what else?
Post by: Doug Bissett on April 17, 2018, 11:54:59 pm
I think that you will find different numbers every time that you run Sysbench. You must remember that there is a lot going on in the background (even stuff that the kernel doesn't know about), that adds to the timing of reads, and writes. to and from the RAMDISK. I find that the disk speed test, in DFSEE, gives the most consistent numbers, but they are also affected by what is going on in the background. For an interesting comparison, try using a FAT RAMDISK, compared to a HPFS RAMDISK. Then, try FAT32.

It doesn't really make any sense to use a FAT32 RAMDISK in OS/2, since it is really slow, and you can't use it to transfer files to / from windows. It may be useful for testing the FAT32 driver, but not much else. Some time ago, I was using the RAMDISK with JFS, but that gets tangled up with the JFS cache, and it was tricky to get it to work properly after the format was done to a FAT volume. Formatting a HPFS volume, as JFS, works okay, but you still have the cache problem.
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 18, 2018, 05:35:30 am
Hi dixie,

/1 forces system to use ancient STRAT1 method. This is fatal for most of file system drivers. Actually, only kernel's FAT implementation and HPFS.IFS able to use it.

HPFS386 looks more advanced as well as JFS. Nothing to do here...

One more thing I wanted to ask a clarification for.

You can create a ramdisk by using the following command in your QSSETUP.CMD: 'ramdisk y: hpfs', and this appears sufficient enough to give you a working HPFS formatted ramdisk. So is there any real need and/or benefit to running this command which I see mentioned several times in the readme as well: ' format Y: /quick /fs:hpfs /q'?

This is a stand-alone format of a ramdisk as a HPFS filesystem, I can only imagine one would want to perhaps use it in some situations, but I do not think this is necessary in any way, is that correct?
Title: Re: RAM disk and temp folders...what else?
Post by: walking_x on April 18, 2018, 08:38:19 am
This is a stand-alone format of a ramdisk as a HPFS filesystem, I can only imagine one would want to perhaps use it in some situations, but I do not think this is necessary in any way, is that correct?
Yes.

Actually, this is just a "format" command ;) You can mount a hard disk partition in QSINIT and format it. No JFS format, but HPFS, FAT & FAT32 are available. Some kind of additional tool for trouble times :)
Title: Re: RAM disk and temp folders...what else?
Post by: Lars on April 18, 2018, 02:25:39 pm
You would think you are correct in that thinking. I would not use HPFS386 on a ramdisk if I had a choice, but it is a HPFS or HPFS386 type of a decision, so I have no choice in this matter, I am forced to run HPFS386 only. Since HPFS386 allows you to shut the cache off for the ramdisk created by QSINIT itself, as best as I can tell this should not a problem.

..and yet the numbers I see tell an entirely different story...

I am basing this last conclusion on the output I see from 'cache386 /o' command and throughput numbers I see from 'Lars Commander' during a file copy operation. If I then compare the speed throughput for the ramdisk with the cache ON and OFF I see the following:

1) HPFS386 cache ON - 600 Meg file copy from HD to ramdisk
Peak => 108,586K
Avg => 104,278K
Last => 108,420K

2) HPFS386 cache OFF
Peak => 53,079K
Avg => 26,325K
Last => 27,736K


You leave it completely unclear if "cache386 /o" turns off the cache for ALL HPFS386 formatted partitions or only for the Ramdisk. Then you leave it completely unclear if you were copying from/to a HPFS386 formatted partition to/from a HPFS386 formatted partition or to/from a partition unaffected by the cache setting.
I have the impression that both source and target partitions were HPFS386 partitions ...
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 18, 2018, 05:25:45 pm
Hi Lars!

Umm...sorry...I completely do not get what led you to make the below statements, but allow me to re-iterate what I previously stated in my response to your post.

Originally you stated the following:

Why would you use HPFS386 on a ramdisk anyway ?

I then answered by providing the empirical evidence as per my tests:

You would think you are correct in that thinking. I would not use HPFS386 on a ramdisk if I had a choice, but it is a HPFS or HPFS386 type of a decision, so I have no choice in this matter, I am forced to run HPFS386 only. Since HPFS386 allows you to shut the cache off for the ramdisk created by QSINIT itself, as best as I can tell this should not a problem.

..and yet the numbers I see tell an entirely different story...

I am basing this last conclusion on the output I see from 'cache386 /o' command and throughput numbers I see from 'Lars Commander' during a file copy operation. If I then compare the speed throughput for the ramdisk with the cache ON and OFF I see the following:

1) HPFS386 cache ON - 600 Meg file copy from HD to ramdisk
Peak => 108,586K
Avg => 104,278K
Last => 108,420K

2) HPFS386 cache OFF
Peak => 53,079K
Avg => 26,325K
Last => 27,736K

...and that led you to conclude the following (which is what I do not understand at all):

You leave it completely unclear if "cache386 /o" turns off the cache for ALL HPFS386 formatted partitions or only for the Ramdisk. Then you leave it completely unclear if you were copying from/to a HPFS386 formatted partition to/from a HPFS386 formatted partition or to/from a partition unaffected by the cache setting.
I have the impression that both source and target partitions were HPFS386 partitions ...

So I thought it was pretty clear that in the context of the thread we were talking about the suitability of turning on the HPFS386 cache for a ramdisk, I mean after all, your original post response (which I quoted above for ease of reference) specifically stated so, or where you simply questioning whether one would want to use HPFS385 filesystem as opposed to something like FAT, FAT32 or JFS???

Alright, so let me confirm: YES, the HPFS386 cache is being turned ON for ramdisk and I am attempting to assess the impact on ramdisk performance with the HPFS386 cache being in either ON or OFF setting.

Further on, my reference to 'cache386 /o', given the context of the conversation, specifically relates to what cache386 output provides as applicable to the ramdisk, such as:

Code: [Select]
[G:\]cache386 /o
CACHE386 options
Cache Size    65536 KB

Drive F:
   Lazy write:             OFF

Drive G:
   Lazy write:              ON   MaxAge:          15000 msec
                                 BufferIdle:       5000 msec

In the above example this shows that HPFS386 cache is active for the G: drive (which is my boot drive), and it is not active for F: drive, which is my maintenance partition.

Therefore:

You leave it completely unclear if "cache386 /o" turns off the cache for ALL HPFS386 formatted partitions or only for the Ramdisk...

...as explained above, the 'cache386 /o' is applicable to the status of the HPFS386 cache for ramdisk only.

Further on:

...Then you leave it completely unclear if you were copying from/to a HPFS386 formatted partition to/from a HPFS386 formatted partition or to/from a partition unaffected by the cache setting...

...which is not the case since I specifically stated the following:

...
1) HPFS386 cache ON - 600 Meg file copy from HD to ramdisk
...

Given that I am looking to answer the 'should I turn ON the HPFS386 cache for ramdisk' question only, I would not be shutting OFF my physical HD HPFS386 cache, it simply would not make sense to do so since when the machine is in production mode it would never run without the HPFS386 cache being turned ON.

OK, I hope that clarified whatever may have been causing confusion.
Title: Re: RAM disk and temp folders...what else?
Post by: Lars on April 18, 2018, 06:36:49 pm
A lot of words that makes it yet more complicated. So my questions are:
1) do you ONLY have HPFS386 partitions on your HD ? Or do you also have JFS partitions ?
2) have you been copying from G: to ramdisk ?
3) What driveletter does your ramdisk have ? It seems that it is neither F: nor G: so what is it ?
4) What is the switch to cache386 to explicitely turn on/off caching for a specific drive letter ?
Title: Re: RAM disk and temp folders...what else?
Post by: Ben on April 19, 2018, 12:52:26 am
I am trying to get the HPFS386.IFS driver loaded, to get the QSINIT Ramdisk to work, but for some reason I get the following error:

"The file Y:\IBM386FS\HPFS386.IFS specified in the IFS statement on line 26 of the config.sys file does not contain a valid device driver or file system driver. Line 26 is ignored"

My config.sys entry is a simple one; "IFS=Y:\IMB386FS\HPFS386.IFS /A:R" where "Y:" is the boot drive and "R" is the RAMDisk.

I am using the default settings for HPFS386.INI and I put HPFS386.DLL in the OS2\DLL folder, (should it be needed)... does it require a config.sys entry? There is nothing in the instructions saying that it does.

Have I overlooded something obvious?
Title: Re: RAM disk and temp folders...what else?
Post by: Dave Yeo on April 19, 2018, 02:04:10 am
Why not just use the regular HPFS driver?
Title: Re: RAM disk and temp folders...what else?
Post by: Ben on April 19, 2018, 03:02:33 am
I may end up with no choice.

HPFS386 should work though maybe it doesn't work on newer software.

The RAMDisk is working fine otherwise as far as I can test it.
Title: Re: RAM disk and temp folders...what else?
Post by: Pete on April 19, 2018, 03:46:20 am
Hi Ben

My config.sys entry is a simple one; "IFS=Y:\IMB386FS\HPFS386.IFS /A:R" where "Y:" is the boot drive and "R" is the RAMDisk.



Is the "Y:\IMB386FS\" above a copy'n'paste typo?


Regards

Pete
Title: Re: RAM disk and temp folders...what else?
Post by: Valery Sedletski on April 19, 2018, 06:07:27 am
2Lars:

> As to STRAT1 method: as far as I can remember from the OpenJFS code I think JFS.IFS will fall back to the STRAT1 method when it cannot find a STRAT3 (flat addressing) entry point. Of course that is irrelevant nowadays as the 32-bit (most recent) OS2DASD.DMD supports the STRAT3 entry point. But the old 16-bit version of OS2DASD.DMD did not.
On the other hand, I think that JFS.IFS will only work with the new OS2DASD.DMD + OS2LVM.DMD.

(Open)JFS will fall back to strat2, not strat1, if strat3 is not supported. JFS does not support strat1 (as well as HPFS386 does not).  JFS.IFS works fine with 16-bit OS2DASD.DMD form Merlin/Warp3 (and DANIDASD.DMD), because it falls back to Strat2, I think.

Also, this is not irrelevant now, as 16-bit OS2DASD/DANIDASD can still be used on the modern system. I use DANIDASD in my universal bootable OS/2 stick, for example.

> Also: VFDISK.SYS (the virtual floppy disk driver, a legacy block device driver) only supports the STRAT1 entry point. But given the fact that you can only format virtual floppies to FAT anyways that should not be a problem.

BTW, using SVDisk.sys, you can easily format virtual floppy as HPFS (with SVDisk own HPFS format tool), then you can take a floppy image with savedskf.exe, and restore it to a physical floppy, or VFDISK.SYS virtual floppy, using loaddskf.exe. Also, I was able to format floppies with Macintosh HFS file system, and use them with HFS.IFS on OS/2. (HFS hybrid CD's work fine too, BTW).

2Dariusz Piatkowski:

> Yup, similar configuration to what I currently have. My operating partitions are all HPFS386, the nightly backup partitions are HPFS386 as well but no CACHE on them. Neither is the RAMDISK, else it defeats the purpse.

You still use HPFS386 as your main file system? Why not JFS? HPFS386 has limitation of 64 GB max partition size, like ordinary HPFS driver. So, how do you use it with modern hard disks?

I moved to JFS 17 years ago, since my partitions are all bigger than 64 GB.

2all: BTW, why nobody here is mentioned emsFS.IFS ramdisk? It uses memory above 4 GB too, but uses PSE36, instead of PAE for accessing this memory. It maps a 4 MB pages above 4 GB limit to a 4 MB window below the 4 GB limit, without switching PAE on/off and copying data. PSE36 is enabled permanently, and no copying is done, only mapping. Also, emsFS.IFS is based on ramfs.ifs. So, it has all the benefits of ramfs.ifs, like absence of block layer. So, it is IFS-based, hence no block layer and caching overhead. So, according to my tests, it is about 5 times faster than PAE ramdisk from QSINIT.

The disadvantages of emsFS.ifs is that it requires the OS/4 kernel as it uses special KEE's for PSE36 access. Also, there is no support for restoring the disk contents like QSINIT does with its "unzip" command. On the other hand, QSINIT is significantly slower.

Also, QSINIT advantage is that it unzip's files at the early stage, and then you can boot from this ramdisk, like with Veit's Memdisk. So, you can create bootable flash sticks.

But you can create bootable flash stcks and boot from emsFS/RAMFS too, if using my FreeLDR boot loader, together with bootos2.mdl OS/2 booter. This is done in my universal OS/2 boot disk (which can boot from flash/CD/harddisk media).

So, did anybody tried emsFS.IFS here?
Title: Re: RAM disk and temp folders...what else?
Post by: Lars on April 19, 2018, 03:29:28 pm
You already gave the answer why nobody uses emsFS.IFS:

1) I did not even know such a thing existed. I don't know where to get it. I am sure that others have the same problem.
2) it requires using the OS4 kernel. Unfortunately that kernel does not properly work on my system and tends to freeze my system.
For the VAC 3.0 profiler I have made a fix (a replacement for CPPOPA3.OBJ) to work around a bug in DOSCALL1.DLL so that the profiler finally works (at least with windowed applications). That works with the eCS and AN kernels but it fails with the OS4 kernel.


What I often lack for the OS4 project is some information page that tells me what stuff exists, how it works, what the restrictions are and where to get them. For example OS4APIC.PSD is largely undocumented and I only found it by chance on the OS4 web page. It allowed me to use multicore CPU but since it is not an ACPI implementation I don't know what the implications are.
There are a bunch of other drivers like CLOCK03.SYS etc. where it's also not clear if these drivers are mandatory or not and if they have to be added to config.sys or if they are loaded automatically by the kernel.
Title: Re: RAM disk and temp folders...what else?
Post by: ivan on April 19, 2018, 05:57:58 pm
Hi Lars,

I couldn't agree more with your statement.

I would like to try the OS4 kernel and other programs and drivers but there is nowhere enough information to let me do so - I have been told that it allows the AMD Ryzen processor to work, something I would very much like to happen.
Title: Re: RAM disk and temp folders...what else?
Post by: Valery Sedletski on April 19, 2018, 05:59:12 pm
2Lars: Where to get??? On Hobbes. If the kernel is not properly work on your system, you should report that to the developers, so that they will fix it. But you don't want. So, who is guilty?

All available stuff is either on Hobbes, or at official webpage, http://gus.biysk.ru/os4/. All, who needs it, knows about it. os4apic.psd is documented in kernel readme's. At least, available command line switches. What else is needed? It is not ACPI implementation, because ACPI is separate from PSD. ACPI is available for interested testers (including me). It is a special kind of driver (a PDD, but it is loaded before PSD, and it communicates to PSD, so that PSD gets ACPI tables info from it. Alternatively, IRQ routing info can be get from MPTable). Drivers like CLOCK03c.SYS are mandatory, but they are loaded by kernel without the need to be written into CONFIG.SYS (the same way as CLOCK01.SYS. CLOCK03c.SYS is loaded instead of CLOCK01.SYS, and contains both CLOCK and TIMER drivers. Plus, it contains support for hi-res timer, based on HPET). The same for SCREEN03.SYS. There are also OEMHLP.BPD and ACPI4.BPD, which contain OEMHLP$ and ACPICA$ implementations. All these should be not written into CONFIG.SYS. Everything should be described in readme's, otherwise you can ask developers. They are responsive.
Title: Re: RAM disk and temp folders...what else?
Post by: Ben on April 19, 2018, 08:00:27 pm
Hi Ben

My config.sys entry is a simple one; "IFS=Y:\IMB386FS\HPFS386.IFS /A:R" where "Y:" is the boot drive and "R" is the RAMDisk.



Is the "Y:\IMB386FS\" above a copy'n'paste typo?

Typo unfortunately.

I tried it on two different installations on two different machines with the same error.

I vaguely recall from the distant past, that HPFS386.IFS wouldn't load without some sort of memory restrictions... virtualmemorylimit adjustment or something like that. But I have that setting tweaked for other programs and I have no intention of changing it..

I take it that file system drivers, (*.IFS), can be loaded from anywhere not  just from config.sys path statements in the libpath, path dpath?

I guess that I have no choice but to use the basic HPFS.
Title: Re: RAM disk ... offtopic HPFS386
Post by: Andi B. on April 19, 2018, 08:16:12 pm
I've in config.sys

IFS=m:\IBM386FS\HPFS386.IFS /A:*
CALL=m:\OS2\CMD.EXE /Q /C m:\IBM386FS\CACHE386.EXE

And a few notes like -
"REM ! EARLYMEMINIT=TRUE   unreming may be needed for JFS chkdsk on large volumes but does not work with HPFS386"
"REM ! HPFS386 does not work anymore with VIRTUALADDRESSLIMIT=2048 - too less memory, going back to VA = 1536"



Title: Re: RAM disk and temp folders...what else?
Post by: Ben on April 19, 2018, 08:37:43 pm
Interesting, Andyy

With that in mind, I won't bother with HPFS386.
While it appears to have some performance increases over HPFS it's not worth the hassel for a RAM disk.

I'll add your comments to my CONFIG.SYS.

Thanks for that heads-up.
Title: Re: RAM disk and temp folders...what else?
Post by: OS4User on April 19, 2018, 10:05:23 pm
There are a bunch of other drivers like CLOCK03.SYS etc. where it's also not clear if these drivers are mandatory or not and if they have to be added to config.sys or if they are loaded automatically by the kernel.

I would like to try the OS4 kernel and other programs and drivers but there is nowhere enough information to let me do so.

Lars, Ivan

OS/4 distr contains HowTo file about installation process of OS/4 kernel.
Pls let me know if anything is unclear in this file and still creates complication.

If any technical questions arise, pls put them into https://www.os2world.com/forum/index.php?topic=1735

Ivan, at least two persons reported about successful run OS/4 on Rysen https://www.os2world.com/wiki/index.php/Phoenix_OS/4#OS.2F4_Tested_Hardware_List
Title: Re: RAM disk and temp folders...what else?
Post by: Lars on April 20, 2018, 01:32:14 pm
2Lars: Where to get??? On Hobbes. If the kernel is not properly work on your system, you should report that to the developers, so that they will fix it. But you don't want. So, who is guilty?

All available stuff is either on Hobbes, or at official webpage, http://gus.biysk.ru/os4/. All, who needs it, knows about it. os4apic.psd is documented in kernel readme's. At least, available command line switches. What else is needed? It is not ACPI implementation, because ACPI is separate from PSD. ACPI is available for interested testers (including me). It is a special kind of driver (a PDD, but it is loaded before PSD, and it communicates to PSD, so that PSD gets ACPI tables info from it. Alternatively, IRQ routing info can be get from MPTable). Drivers like CLOCK03c.SYS are mandatory, but they are loaded by kernel without the need to be written into CONFIG.SYS (the same way as CLOCK01.SYS. CLOCK03c.SYS is loaded instead of CLOCK01.SYS, and contains both CLOCK and TIMER drivers. Plus, it contains support for hi-res timer, based on HPET). The same for SCREEN03.SYS. There are also OEMHLP.BPD and ACPI4.BPD, which contain OEMHLP$ and ACPICA$ implementations. All these should be not written into CONFIG.SYS. Everything should be described in readme's, otherwise you can ask developers. They are responsive.

How do you know that I don't want ? In fact, I did install the OS4 kernel and along with it the OS4APIC.PSD.
The problem is that the OS4 components are spread across different places (there is no common zip, left alone an installation package). And it's unclear when and if you need ACPI4.BPD and OEMHLP.BPD.

Just to let you know: I have a 40-hour job and it has nothing to do with PC SW development. I don't have all day to find out how things work or where to find them. Then I have some own OS/2 related SW development going on. And then I have to maintain my existing OS/2 main installation and device driver development installation to ensure it is operational. Plus I have a couple of additional hobbies ...
Title: Re: RAM disk and temp folders...what else?
Post by: Valery Sedletski on April 20, 2018, 02:29:41 pm
2Lars:

> The problem is that the OS4 components are spread across different places (there is no common zip, left alone an installation package). And it's unclear when and if you need ACPI4.BPD and OEMHLP.BPD.

All required components are in kernel archive. acpi4.bpd is not yet released, because something should be fixed until the release. But it is promised to be released soon. BTW, it should work ok without acpi4.bpd (or, with ACPI.PSD older than 3.23.01)
Title: Re: RAM disk and temp folders...what else?
Post by: Neil Waldhauer on April 20, 2018, 03:01:01 pm
Pardon me for going off-topic a little bit.

Writing a good installer would supercharge the OS4 project. If I could have confidence that I could install OS4, and then uninstall it, I could be brave about trying it under various machines and tasks. I'm sure I'm not alone in feeling this way. I'd like to encourage the project, yet I have not got the time to invest to learn how it goes together.

It's not easy to write a good installer, and the skills have more to do with test engineering than software engineering. OS4 would be especially challenging because any error leaves the installation unable to boot. The target installations are diverse -- OS/2, eCS and ArcaOS.
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 20, 2018, 04:28:33 pm
...It's not easy to write a good installer, and the skills have more to do with test engineering than software engineering. OS4 would be especially challenging because any error leaves the installation unable to boot. The target installations are diverse -- OS/2, eCS and ArcaOS.

Neil's got a great point about the installer. Given that we now have several options available as far as picking the OS2 loader of choice, I would actually be hesitant to allow an installer to do this type of stuff.

Yeah, I know...may seem counter-intuitive, but I think there are far too many things that could go just 'wrong enough' to prevent my OS/2 machine from booting, that would make me feel comfortable doing this any other way.

Funny thing is, the request for additional info was literally my feedback to the OS4 folks in one of the other threads. But I think what I was talking about was a lot more like the stuff Lars mentioned here, that being: give me a better understanding of the internal design/implementation of the changes so that I can feel more confident in my decision to pursue this approach.

Having said all of the above, the actual (manual) install of the OS4 solution is not all that hard. The READMEs included do cover it fairly well, but yeah, go through them at least a couple of times. I think most of time would be spent in adjusting the default configuration, so getting yourself multiple CONFIG.SYS instances to give you the flexibility to either boot with the OS4 loader, or the default OS2 or maybe the AOS one.
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 20, 2018, 04:49:25 pm
Hi Lars,

OK, step by step explanation:

1) do you ONLY have HPFS386 partitions on your HD ? Or do you also have JFS partitions ?

Only using HPFS386 partitions. The only other IFSs deployed are: FAT32 & CDFS, but the CDFS is a standard these days and FAT32 is specific to just using memory cards/thumb drives.

Quote
2) have you been copying from G: to ramdisk ?

Yes, G: is my boot drive, I am copying various sized files from G: to ramdisk, which is drive Y:

Quote
3) What driveletter does your ramdisk have ? It seems that it is neither F: nor G: so what is it ?

Ramdisk is drive Y:

Quote
4) What is the switch to cache386 to explicitely turn on/off caching for a specific drive letter ?

Issuing 'cache386' @CLI shows the following:

Code: [Select]
[G:\]cache386
Valid options for CACHE386 (x is an optional drive letter):

/OPTIONS[:x]         Display current configuration
/STATS[:C|:D]        Display, clear, or dynamically display statistics
/LAZY[:x][:ON|:OFF]  Enable or disable lazy writing
/MAXAGE[:x]:n        Change msDataAge to n milliseconds
/BUFFERIDLE[:x]:n    Change msIdleBuf to n milliseconds

If no drive (x) is given, the option will apply to all HPFS drives.

Each option may be abbreviated to a single letter, for example:

CACHE386 /B:D:1000    (sets BufferIdle to 1000 for drive D)

So, to turn on the cache (lazy write really as per the option listing, but it appears to actually behave as if that controls the CACHE function) you issue 'cache386 /l:y:on', which can then be checked by issuing the following 'cache386 /o:y':

Code: [Select]
[G:\]cache386 /o:y
CACHE386 options
Cache Size    65536 KB

Drive Y:
   Lazy write:              ON   MaxAge:          15000 msec
                                 BufferIdle:       5000 msec

As best as I can tell issuing 'cache386 /l:y:off' disables or enables the HPFS386 cache:

Code: [Select]
[G:\]cache386 /o:y
CACHE386 options
Cache Size    65536 KB

Drive Y:
   Lazy write:             OFF

Therefore, the previously reported results are specific to the above method of turning on the HPFS386 cache (lazy write) for the ramdisk drive 'Y:'.

As far as I can tell there is nothing else in the HPFS386 module which allows one to explicitly control the cache function, instead I believe it is the 'LAZY write' control that actually does this.
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 20, 2018, 05:03:52 pm
Hi Ben!

Hi Ben

My config.sys entry is a simple one; "IFS=Y:\IMB386FS\HPFS386.IFS /A:R" where "Y:" is the boot drive and "R" is the RAMDisk.


Is the "Y:\IMB386FS\" above a copy'n'paste typo?

Typo unfortunately.

I tried it on two different installations on two different machines with the same error.

I vaguely recall from the distant past, that HPFS386.IFS wouldn't load without some sort of memory restrictions... virtualmemorylimit adjustment or something like that. But I have that setting tweaked for other programs and I have no intention of changing it..

I take it that file system drivers, (*.IFS), can be loaded from anywhere not  just from config.sys path statements in the libpath, path dpath?

I guess that I have no choice but to use the basic HPFS.

So the HPFS386 module is not solely made up of the IFS and the INI files, there of course are a number of DLLs which make up the full solution.

Here is what I currently have in my \IBM386FS directory:

Code: [Select]
Directory of G:\ibm386fs

10-29-17   7:07p     <DIR>           0  .
 4-19-18   1:25p     <DIR>           0  ..
 9-30-03   6:11p     25701         403  BOOTSH.EXE
 9-30-03   6:11p      1547         221  bsh.msg
 9-30-03   6:11p      3958         222  bshh.msg
 9-30-03   6:11p      3255         221  cac.msg
 9-30-03   6:11p      3497         222  cach.msg
 9-30-03   6:11p     27237         405  CACHE386.EXE
 9-30-03   6:11p     10804         224  DISKFT.SYS
 3-11-12   8:08p      2164           0  files_latest
 9-30-03   6:11p     10346         221  hfs.msg
 9-30-03   6:11p     33464         222  hfsh.msg
 9-30-03   6:11p      6845         404  HPFS386.DLL
 3-23-04  10:17a    259689           0  hpfs386.ifs
11-30-00   6:31p    259177           0  hpfs386.ifs.53_43
 9-30-03   6:11p    259691         225  hpfs386.ifs.53_44_15
 3-23-04  10:17a    259689           0  hpfs386.ifs.53_47
 4-17-18  10:49a     12232         268  hpfs386.ini
 8-30-14  10:42a      7154           0  hpfs386.log
 9-30-03   6:11p       462         225  hpfs386.txt
10-26-08  11:38a      1313          86  hpfsperf IP_8508.TXT
10-26-08  11:36a      8758          75  hpfsperf.cmd
 9-30-03   6:11p     39725         355  hva.exe
10-10-01   3:11p      7022           0  MAILSLOT.DLL
 9-30-03   6:11p      7020         405  MAILSLOT.DLL.53_43
 3-23-04   8:40p    363811          49  netapi.dll
 9-30-03   6:11p    348965         403  NETAPI.DLL.53_43
 3-23-04   8:40p    363811          49  NETAPI.DLL.53_47
 9-30-03   6:11p     23040         224  netapi.lib
 3-23-04   8:37p     63636          61  NETAPI32.DLL
 9-30-03   6:11p     63766         417  NETAPI32.DLL.53_43
 3-23-04   8:37p     63636          61  NETAPI32.DLL.53_47
10-10-01   6:44p      6588           0  NETSPOOL.DLL
 9-30-03   6:11p      6586         405  NETSPOOL.DLL.53_43
 1-01-13  10:51a     <DIR>        1037  newest
 1-01-13  10:51a     <DIR>        1144  original
 9-30-03   6:11p     26629         359  PREPACL.EXE
 9-30-03   6:11p      2081         255  readme.now
       38 file(s)    2583299 bytes used
                   3937073 K bytes free

Yeah, there are several versions listed there as a result of my previous work to narrow this configuration down to the most 'current' release.

Anyways, the IFS statement you show should be sufficient to load HPFS386. The HPFS386.INI file then brings specific controls into place, I've attached my version to the post for your review.

You certainly need the \IBM386FS to be present in your LIBPATH statement. I have it also included in the PATH and DPATH, not sure if this was necessary or not, but it seems to work fine here.

Further on, in your CONFIG.SYS you also need the following: 'CALL=G:\OS2\CMD.EXE /Q /C G:\IBM386FS\CACHE386.EXE >NUL', which will kick off the CACHE386 exec to, I believe, read in the HPFS386.INI definitions.

There are some issues you may run into, certainly the size of the HPFS386 cache has a lot to do with this. In the past, I used to run that cache as big as 500 Meg (only b/c the Win NTFS can run much bigger and I wanted to compare some things), but as more and more software piled onto my machine I had to bring this down. Currently I have this configured for 64Meg only, seems to work fine with VIRTUALADDRESSLIMIT=2560 in the CONFIG.SYS working fine. I have booted up the machine with VIRTUALADDRESSLIMIT=3072 as well w/o any problems.
Title: Re: RAM disk and temp folders...what else?
Post by: Dariusz Piatkowski on April 20, 2018, 05:08:59 pm
Hi Valery,

...2Dariusz Piatkowski:

> Yup, similar configuration to what I currently have. My operating partitions are all HPFS386, the nightly backup partitions are HPFS386 as well but no CACHE on them. Neither is the RAMDISK, else it defeats the purpse.

You still use HPFS386 as your main file system? Why not JFS? HPFS386 has limitation of 64 GB max partition size, like ordinary HPFS driver. So, how do you use it with modern hard disks?

I moved to JFS 17 years ago, since my partitions are all bigger than 64 GB....

Yeah, still using HPFS386 here mostly because of how long I have had a single install of my OS/2...literally since the Warp3 days, followed by Warp4 upgrade, then CP releases, move to SMP hardware and finally the ACPI stuff.

I did attempt to deploy JFS once, that caused massive data corruption, I suspect the HPFS386 and JFS simply had conflicts. It was a painfull recovery and since my hardware has remained the same (no HD changes, etc) I have not had a reason to re-do all of this.

Now, this may change very soon. I have a Samsung SSD (850 Evo) standing by and as soon as I off-load a bunch of the stuff from my DATA partitions onto the LAN NAS I will be deploying it. At that time it would make sense to take a stab at the JFS thing again.
Title: Re: RAM disk and temp folders...what else?
Post by: David Graser on April 21, 2018, 07:34:37 am
...It's not easy to write a good installer, and the skills have more to do with test engineering than software engineering. OS4 would be especially challenging because any error leaves the installation unable to boot. The target installations are diverse -- OS/2, eCS and ArcaOS.
.

Having said all of the above, the actual (manual) install of the OS4 solution is not all that hard. The READMEs included do cover it fairly well, but yeah, go through them at least a couple of times. I think most of time would be spent in adjusting the default configuration, so getting yourself multiple CONFIG.SYS instances to give you the flexibility to either boot with the OS4 loader, or the default OS2 or maybe the AOS one.

Having just recently moved to the OS4 kernel, it is pretty straight forward unless one is using a RAM drive utilizing QSINIT.  The one thing to remember is that when updating to the latest OS4 kernel, do not replace the QSINIT loader with the latest OS4 kernel loader.
Title: Re: RAM disk and temp folders...what else?
Post by: gamba on April 24, 2018, 12:18:17 pm
Having just recently moved to the OS4 kernel, it is pretty straight forward unless one is using a RAM drive utilizing QSINIT.  The one thing to remember is that when updating to the latest OS4 kernel, do not replace the QSINIT loader with the latest OS4 kernel loader.

I quote David. For me, this time installing OS/4 was easy: unzipped the files - except OS2LDR - from OS/4 package, replaced ArcaOS OS2LDR with the one from QSINIT, and then everything went as expected; just ACPI is not working, but I guess I just have to study a bit more the existing threads to find the answer :-)
Title: Re: RAM disk and temp folders...what else?
Post by: Lars on April 24, 2018, 12:33:57 pm
The answer is that for the OS4 kernel there currently only exist an experimental driver for ACPI called ACPI4.BPD (I seem to remember).
However that does not necessarily mean that you cannot use multiple cores.
OS4 comes with OS4APIC.PSD which allows to use multiple cores if your system still has the old style Intel Multiprocessor tables (Intel MP spec version 1.4) along with the newer ACPI tables.
Have you tried just using OS4APIC.PSD (preferrably with the /APIC switch) or are you forced to using ACPI on the system that you have ?
Title: Re: RAM disk and temp folders...what else?
Post by: gamba on April 24, 2018, 12:43:47 pm
...
Have you tried just using OS4APIC.PSD (preferrably with the /APIC switch) or are you forced to using ACPI on the system that you have ?

Hi Lars,

I have PSD=OS4APIC.PSD /APIC in CONFIG.OS4, but I just get one core instead of 4 (and of course I see the "acpi4.bpd not found or corrupted" message during bootup :-)

I should patch ACPI.PSD, I think, but I have ver. 3.23.04 installed, not 3.23.01.
Title: Re: RAM disk and temp folders...what else?
Post by: Dave Yeo on April 24, 2018, 04:45:49 pm
When I experimented, I also only had one core working until I went through the readme and set up a config.os4 exactly as described. Seems one of the other drivers was required to get both cores working.
I should test on this weird machine I'm using.
Title: Re: RAM disk and temp folders...what else?
Post by: Martin Vieregg on May 02, 2018, 11:13:26 pm
I have got here a new ArcaOS 5.0.2 installation and I know that the USB stick stating mechanism uses a Ramdisk. But how can I implement a Ramdisk via Config.sys and orStartup folder? I have only found a program named HIDISK.EXE, there comes only a small dialog form with some settings, and the procedure to setup is not explained.
Title: Re: RAM disk and temp folders...what else?
Post by: Dave Yeo on May 03, 2018, 01:08:53 am
Go into System Setup and open the Ram Disk object.
Title: Re: RAM disk and temp folders...what else?
Post by: Rich Walsh on May 04, 2018, 12:31:40 am
I have only found a program named HIDISK.EXE, there comes only a small dialog form with some settings, and the procedure to setup is not explained.

That dialog _is_ the setup procedure. You don't have to do anything except check a box, select a drive letter, then press "OK" (and reboot). If in doubt, you may want to press the "Help" button - read the "Overview" section first.

Title: Re: RAM disk and temp folders...what else?
Post by: Martin Vieregg on May 04, 2018, 02:06:58 pm
Thank you. Indeed, it works all fine. The first time, I forgot to check the "ramdisk" checkbox.

A cosmetic problem: The shadow to the folder on the ramdisk is broken, of couse, because I create the folder in a cmd file located in the startup folder. So I have to use a config.sys command:
c:\os2\cmd.exe /c createfolders.cmd   (is this correct?)
(haven't tried yet.) But how can I be sure that the ramdisk is generated before executing the cmd file?
Title: Re: RAM disk and temp folders...what else?
Post by: Doug Bissett on May 04, 2018, 07:48:26 pm
Quote
But how can I be sure that the ramdisk is generated before executing the cmd file?

The RAMDISK exists, before the kernel is loaded (created by OS2LDR). It is accessible as soon as the driver loads (BASEDEV=HD4DISK.ADD). The CMD executes just before the GUI starts up.

Personally, I create the desired folders in STARTUP.CMD, but you may need to do it earlier, if something will try to use them earlier. Your command should be a CALL or RUN (I don't remember which) statement in CONFIG.SYS:
CALL c:\os2\cmd.exe /c createfolders.cmd
I think (you may need to try a few variations).
Title: Re: RAM disk and temp folders...what else?
Post by: Andreas Schnellbacher on May 04, 2018, 07:59:16 pm
I recommend not to use the CONFIG.SYS for that, but a Startup Folder.

To the difference of the CONFIG.SYS statements CALL and RUN:

o  CALL runs the specified .exe synchronously. (The processing of the rest of the CONFIG.SYS statements is halted until the called .exe has finished its startup processing.)

o  RUN runs the specified .exe asynchronously. (The processing of the rest of the CONFIG.SYS statements is continued immediately and the processing of the .exe is processed in parallel.)
Title: Re: RAM disk and temp folders...what else?
Post by: Martin Vieregg on May 09, 2018, 10:02:59 am
I have got an idea how to make a "shadow icon" for a Ramdisk folder which is not broken:
- create a program icon, use a folder icon on the "icon" tab
- point to a cmd file which holds the string  "open z:\myfolder"
Title: Re: RAM disk and temp folders...what else?
Post by: Greg Pringle on May 09, 2018, 06:10:59 pm
I use Java extensively for applications.
One is a mail filter which I wrote that buffers the mail. To keep memory usage down it uses a virtual memory vector (FileVect) which I also wrote. This works fine but is much slower than real memory. It has the advantage of being able to buffer huge amounts of data. I recently redirected the virtual vector to use a 3.6 GB ram disk. It is working quite well. I have not checked the speed against a non-virtual vector but it seems close enough.
Title: Re: RAM disk and temp folders...what else?
Post by: Martin Vieregg on May 13, 2018, 11:19:26 pm
I made speed tests on my new current AM4 system with ArcaOS 5.0.2. I copied three files with together 550 MB with Larsen Commander which shows the average speed in MB/sec, SSD is JFS, RamDisk HPFS:

SSD -> SSD: 142 MB/sec
SSD -> RamDisk: 235 MB/sec
RamDisk -> RamDisk: 390 MB/sec

Anyway, the main benefit will be sparing the SSD with its limited write cycles.
Title: Re: RAM disk and temp folders...what else?
Post by: Andy Willis on May 14, 2018, 04:53:48 am
I made speed tests on my new current AM4 system with ArcaOS 5.0.2. I copied three files with together 550 MB with Larsen Commander which shows the average speed in MB/sec, SSD is JFS, RamDisk HPFS:

SSD -> SSD: 142 MB/sec
SSD -> RamDisk: 235 MB/sec
RamDisk -> RamDisk: 390 MB/sec

Anyway, the main benefit will be sparing the SSD with its limited write cycles.

To save the write cycles on my SSD, in Mozilla browsers I set:
browser.cache.disk.parent_directory
so that it writes to my RamDisk.  I also set the swappath in config.sys to use the RamDisk as it gets recreated at every boot.  I also set the temp, tmp, tmpdir to all be on the RamDisk for the same reason.
Title: Re: RAM disk and temp folders...what else?
Post by: gamba on May 14, 2018, 09:30:44 pm
I'm now trying to disable ArcaOS ramdisk (as I want to test emsifs.ifs).   I removed os2ldr.cfg file and remmed hd4disk.add in config.sys. I rebooted several times and also switched off the PC, but the drive letter used by ArcaOS ramdisk is still in use! LVM sees it as:

[ CDROM 2 ]          *->T:  Compatibility                   CDFS file system

and it fact it appears in Drives folder with a CD icon. I really guess how I can definitely remove it...
Title: Re: RAM disk and temp folders...what else?
Post by: Greg Pringle on May 14, 2018, 09:53:37 pm
I  am not sure what happens if you manually delete the files but you could have used the "HiDisk" utility used to create the ram disk in the first place. When using the utility "un-check" the create button and then "check" the drive you want removed and then press the "OK" button.
Then reboot and the ram disk will be gone.
Title: Re: RAM disk and temp folders...what else?
Post by: gamba on May 14, 2018, 09:54:49 pm
... I rebooted several times and also switched off the PC, but the drive letter used by ArcaOS ramdisk is still in use! LVM sees it as:

[ CDROM 2 ]          *->T:  Compatibility                   CDFS file system


It was USBCDROM.ADD: it seems that ArcaOS/QSINIT ramdisk - I had assigned T: to it - can "steal" the drive letter to USBCDROM.ADD, while emsifs can't.