Author Topic: RAM disk and temp folders...what else?  (Read 41953 times)

xynixme

  • Guest
Re: RAM disk and temp folders...what else?
« Reply #15 on: April 15, 2018, 09:40:48 pm »
Quote
1) ZIP file processing
2) PMMail
3) GCC

What else though? Any suggestions?
Data files, or EXEs, which are frequently accessed, or called, by a quick & dirty or unoptimized solution. If anything to avoid both physical harddisk abuse and the stress of the sounds of Microsoft's updates services.

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #16 on: April 15, 2018, 10:14:24 pm »
Quote
I think it is you who have completely missed that fact that I am NOT running AOS.

Yes, I did miss that. I still suggest using QSINIT, and it's RAMDISK (if you don't use ArcaOS). The small effort to switch is very much worth the effort, especially if your machine has 4 GB of memory, or more. The RAMDISK is far better than any other that is available, by a large margin.

Ben

  • Guest
Re: RAM disk and temp folders...what else?
« Reply #17 on: April 15, 2018, 10:23:01 pm »
I have been using RAMFS.IFS for years. I have not have any problems that I could ever associate with it. However, if the RAMDISK that comes with QSINIT uses a smaller memory footprint then I have no trouble giving it a go, especially in these days of dwindling usable OS/2 memory.

I have an old version of QSINIT on a maintenance partition  that I can use for testing and I shall do that when I get the time.

Thanks for the heads-up.

Lars

  • Hero Member
  • *****
  • Posts: 1271
  • Karma: +65/-0
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #18 on: April 15, 2018, 11:13:32 pm »
You are missing the point. QSINIT uses physical memory beyond the 4GB border. And THAT makes it so valuable. RAMIFS steals from the memory available to the OS.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #19 on: April 17, 2018, 04:59:26 am »
Quote
I think it is you who have completely missed that fact that I am NOT running AOS.

Yes, I did miss that. I still suggest using QSINIT, and it's RAMDISK (if you don't use ArcaOS)...

Dave is right, there are many versions and configurations of OS/2 floating around, so it's super easy to fork-off on that road, no biggie, easy miss, but you gave me the needed "push" ;D to re-visit the QSINIT utility.

I had previously installed it some time ago to investigate the functionality, but it was in it's early days and I didn't feel it offered much to warrant replacing the standard process.

So here is a bit of an update, and this will have specific info for those running HPFS386, because certain option available in the current latest version of QSINIT doesn't quite "play nice" with a HPFS386 machine...read on!

OK, so here is what I have done:

1) install QSINIT
2) add BASEDEV=HD4DISK.ADD to my CONFIG.SYS
3) add 'ramdisk Y: hpfs' to QSSETUP.CMD

...seemed like that's all that was required to get a plain-vanilla install. However, upon a re-boot I was greeted by numerous HPFS error messages pertaining to just that RAMDISK. I emailed dixie (QSINIT) author who confirmed this and explained that HPFS386 in particular has a problem with the RAMDISK in that it always thinks the volume is dirty because it always considers the volume to be a brand NEW volume.

Indeed, that required me adding the new 'Y' (my drive letter) to the CONFIG.SYS HPFS386.IFS line:
IFS=G:\IBM386FS\HPFS386.IFS /AUTOCHECK:FGY

Now with this auto-check in-place upon a re-boot there is a very quick CHKDSK that runs on the RAMDISK, because HPFS386 thinks it's NEW and therefore dirty and must be cleaned. The QSINIT dirty flag set utility does not help. But, this process is super quick and gives me a nice, nearly 5Gig ramdisk, plenty of storage.

Further on, I attempted to use the '/1' switch for HD4DISK.ADD, this however leads to a hard stop at bootup where the RAMDISK is not initialized and the device itself is only available as a DASD unit, the error message below is what I see:
Code: [Select]
HFS0109: The disk device driver for the 386 HPFS volume on drive Y: does not support memory above 16MB. The volume was mounted for DASD access only.

I do not understand the cause, but somehow the suggested '/1' parameter for use with HPFS formatted ramdisks does not work here. Playing with the 'useallmem' setting in HPFS386.INI does not help. Could be the size of my HPFS386 cache (only 64Meg), or it could be due to some difference between regular HPFS and HPFS386.

Anyways, I will let dixie know about this, maybe he is willing to debug further.

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #20 on: April 17, 2018, 07:07:17 am »
Quote
...seemed like that's all that was required to get a plain-vanilla install. However, upon a re-boot I was greeted by numerous HPFS error messages pertaining to just that RAMDISK. I emailed dixie (QSINIT) author who confirmed this and explained that HPFS386 in particular has a problem with the RAMDISK in that it always thinks the volume is dirty because it always considers the volume to be a brand NEW volume.

I never used HPFS386, so I have no idea what it might think of the RAMDISK. It shouldn't be running it anyway, because it is HPFS, and not HPFS386, but it probably doesn't know the difference. You could try using FAT, but that would limit you to a 2 GB partition. FAT32 might work, but it is very slow.

Quote
Further on, I attempted to use the '/1' switch for HD4DISK.ADD, this however leads to a hard stop at bootup where the RAMDISK is not initialized and the device itself is only available as a DASD unit, the error message below is what I see:

Code: [Select]
HFS0109: The disk device driver for the 386 HPFS volume on drive Y: does not support memory above 16MB. The volume was mounted for DASD access only
I do not understand the cause, but somehow the suggested '/1' parameter for use with HPFS formatted ramdisks does not work here. Playing with the 'useallmem' setting in HPFS386.INI does not help. Could be the size of my HPFS386 cache (only 64Meg), or it could be due to some difference between regular HPFS and HPFS386.

I never used "/1". You probably need "EARLYMEMINIT=TRUE" near the top of your CONFIG.SYS, but I seem to recall that that isn't friendly with HPFS386. Another possibility, is an OLD BIOS setting (something about supporting OS/2), which is only required with warp 3. Otherwise, it should be off

The way that I use the RAMDISK, is to use all memory above what OS/2 uses normally, for the RAMDISK. Then, I use HPFS for the RAMDISK (I also make the cache as small as possible - 64K- because the cache slows it down). All of my other partitions are JFS (which is probably better than HPFS386 anyway), except for one that is FAT32, for sharing files with windows.

walking_x

  • Full Member
  • ***
  • Posts: 101
  • Karma: +0/-0
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #21 on: April 17, 2018, 07:48:22 am »
I do not understand the cause, but somehow the suggested '/1' parameter for use with HPFS formatted ramdisks does not work here. Playing with the 'useallmem' setting in HPFS386.INI does not help. Could be the size of my HPFS386 cache (only 64Meg), or it could be due to some difference between regular HPFS and HPFS386.
/1 forces system to use ancient STRAT1 method. This is fatal for most of file system drivers. Actually, only kernel's FAT implementation and HPFS.IFS able to use it.

HPFS386 looks more advanced as well as JFS. Nothing to do here.

Another bad thing with HPFS386 is that is may be slower, than common HPFS driver - on the RAMDISK. Because of smart caching, probably.

Any way, ramdisk may save your SSD, when used as TEMP. Also, there is a setting in seamonkey - path to cache location, it can point to the ram disk (nice thing for security too). Better than nothing ;)

Lars

  • Hero Member
  • *****
  • Posts: 1271
  • Karma: +65/-0
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #22 on: April 17, 2018, 08:22:37 am »
Why would you use HPFS386 on a ramdisk anyway ?
Optimizing sector accesses is irrelevant for a ram disk. Caching will even make things worse: you waste additional memory for no gain in speed (after all, on a ram disk, each data sector is already in memory ...)
And it does not look like the OP cares about the file access protection features that HPFS386 offers on top of HPFS.

However, I don't know if you can load HPFS.IFS and HPFS386.IFS at the same time. Most likely not ...

As to STRAT1 method: as far as I can remember from the OpenJFS code I think JFS.IFS will fall back to the STRAT1 method when it cannot find a STRAT3 (flat addressing) entry point. Of course that is irrelevant nowadays as the 32-bit (most recent) OS2DASD.DMD supports the STRAT3 entry point. But the old 16-bit version of OS2DASD.DMD did not.
On the other hand, I think that JFS.IFS will only work with the new OS2DASD.DMD + OS2LVM.DMD.

Also: VFDISK.SYS (the virtual floppy disk driver, a legacy block device driver) only supports the STRAT1 entry point. But given the fact that you can only format virtual floppies to FAT anyways that should not be a problem.
« Last Edit: April 17, 2018, 08:26:27 am by Lars »

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #23 on: April 17, 2018, 03:46:36 pm »
Hey Doug,

...The way that I use the RAMDISK, is to use all memory above what OS/2 uses normally, for the RAMDISK. Then, I use HPFS for the RAMDISK (I also make the cache as small as possible - 64K- because the cache slows it down). All of my other partitions are JFS (which is probably better than HPFS386 anyway), except for one that is FAT32, for sharing files with windows.

Yup, similar configuration to what I currently have. My operating partitions are all HPFS386, the nightly backup partitions are HPFS386 as well but no CACHE on them. Neither is the RAMDISK, else it defeats the purpse.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #24 on: April 17, 2018, 04:43:18 pm »
Hi Lars,

Why would you use HPFS386 on a ramdisk anyway ?
Optimizing sector accesses is irrelevant for a ram disk. Caching will even make things worse: you waste additional memory for no gain in speed (after all, on a ram disk, each data sector is already in memory ...)
And it does not look like the OP cares about the file access protection features that HPFS386 offers on top of HPFS.

However, I don't know if you can load HPFS.IFS and HPFS386.IFS at the same time. Most likely not ...

You would think you are correct in that thinking. I would not use HPFS386 on a ramdisk if I had a choice, but it is a HPFS or HPFS386 type of a decision, so I have no choice in this matter, I am forced to run HPFS386 only. Since HPFS386 allows you to shut the cache off for the ramdisk created by QSINIT itself, as best as I can tell this should not a problem.

..and yet the numbers I see tell an entirely different story...

I am basing this last conclusion on the output I see from 'cache386 /o' command and throughput numbers I see from 'Lars Commander' during a file copy operation. If I then compare the speed throughput for the ramdisk with the cache ON and OFF I see the following:

1) HPFS386 cache ON - 600 Meg file copy from HD to ramdisk
Peak => 108,586K
Avg => 104,278K
Last => 108,420K

2) HPFS386 cache OFF
Peak => 53,079K
Avg => 26,325K
Last => 27,736K

I specifically selected a large source file for this initial test so that it is bigger than my HPFS386 cache of 64Meg. Yet despite this, I am seeing the same level of performance difference across both small and large files, that being nearly 4x faster throughput on cached ramdisk.

Lars, unless your throughput measure is being skewed by something else, the numbers here tell a different story. The only negative downside to keeping the HPFS386 cache on for the ramdisk is the 'pollution' of the true cache for the other real HDs in my system.

Sysbench (0.9.5d) 'Disk I/O' test is completely messed up by the ramdisk, the only number that actually comes back is the bus xfer which is 355MByte/s. However, the 'File I/O' results give 216157 for non-cache and 260371 for cached configurations.

Therefore, in the end, given what I see above and the fact that most files tossed on the ramdisk will normally be small (therefore implying the true cache pollution to be small) it actually seems like leaving the HPFS386 cache ON for the ramdisk gives the better performance.

walking_x

  • Full Member
  • ***
  • Posts: 101
  • Karma: +0/-0
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #25 on: April 17, 2018, 06:16:34 pm »
You can try File i/o test in sysbench :) I found one, was launched on HPFS386 too. Something crazy a bit (may be, because of "lazy = v: OFF" in hpfs386.ini):
 File I/O - Drive V:
   4Kb seq.   Uncached w :    32837.920    Kilobytes/second
   4Kb seq.   Uncached r :   723265.941    Kilobytes/second
   4Kb random Uncached w :    10788.018    Kilobytes/second
   4Kb random Uncached r :   652518.493    Kilobytes/second
   4Kb seq.   Cached   w :    33585.820    Kilobytes/second
   4Kb seq.   Cached   r :  1252849.141    Kilobytes/second
....
   32K seq.   Uncached w :    44419.361    Kilobytes/second
   32K seq.   Uncached r :  2136524.963    Kilobytes/second
   32K random Uncached w :    44644.568    Kilobytes/second
   32K random Uncached r :  2346214.469    Kilobytes/second


Disk i/o subsubsystem adds a delay, any way. Real transfer speed on large block sizes is not far from PC memory speed.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #26 on: April 17, 2018, 06:32:05 pm »
You can try File i/o test in sysbench :) I found one, was launched on HPFS386 too...

So here are the results of the Sysbench 'File I/O' test that I reported the results for in my previous update. Take a look at them, they are pretty close, but at times significantly different. I think what may be getting muddied up by Sysbench is this whole 'Cached' vs 'Uncached' result...I have no idea how Sysbench controls that.

Doug Bissett

  • Hero Member
  • *****
  • Posts: 1593
  • Karma: +4/-2
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #27 on: April 17, 2018, 11:54:59 pm »
I think that you will find different numbers every time that you run Sysbench. You must remember that there is a lot going on in the background (even stuff that the kernel doesn't know about), that adds to the timing of reads, and writes. to and from the RAMDISK. I find that the disk speed test, in DFSEE, gives the most consistent numbers, but they are also affected by what is going on in the background. For an interesting comparison, try using a FAT RAMDISK, compared to a HPFS RAMDISK. Then, try FAT32.

It doesn't really make any sense to use a FAT32 RAMDISK in OS/2, since it is really slow, and you can't use it to transfer files to / from windows. It may be useful for testing the FAT32 driver, but not much else. Some time ago, I was using the RAMDISK with JFS, but that gets tangled up with the JFS cache, and it was tricky to get it to work properly after the format was done to a FAT volume. Formatting a HPFS volume, as JFS, works okay, but you still have the cache problem.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #28 on: April 18, 2018, 05:35:30 am »
Hi dixie,

/1 forces system to use ancient STRAT1 method. This is fatal for most of file system drivers. Actually, only kernel's FAT implementation and HPFS.IFS able to use it.

HPFS386 looks more advanced as well as JFS. Nothing to do here...

One more thing I wanted to ask a clarification for.

You can create a ramdisk by using the following command in your QSSETUP.CMD: 'ramdisk y: hpfs', and this appears sufficient enough to give you a working HPFS formatted ramdisk. So is there any real need and/or benefit to running this command which I see mentioned several times in the readme as well: ' format Y: /quick /fs:hpfs /q'?

This is a stand-alone format of a ramdisk as a HPFS filesystem, I can only imagine one would want to perhaps use it in some situations, but I do not think this is necessary in any way, is that correct?

walking_x

  • Full Member
  • ***
  • Posts: 101
  • Karma: +0/-0
    • View Profile
Re: RAM disk and temp folders...what else?
« Reply #29 on: April 18, 2018, 08:38:19 am »
This is a stand-alone format of a ramdisk as a HPFS filesystem, I can only imagine one would want to perhaps use it in some situations, but I do not think this is necessary in any way, is that correct?
Yes.

Actually, this is just a "format" command ;) You can mount a hard disk partition in QSINIT and format it. No JFS format, but HPFS, FAT & FAT32 are available. Some kind of additional tool for trouble times :)