AOS RAM Disk

[Andy Willis]

You can put the swap file there too and in theory run a bunch of processes that use lots of memory and it would swap. In practice, IBM seems to have used 32 bit variables in too many places for this to work.

I put my swapper.dat on the RAM disk just so that it wasn't writing to the SSD drive every boot but given we can't use over 4G I don't see how the swapper could ever be used.

[Dave Yeo]

What happens if you start up 5 processes that each use a GB of memory?
In theory you should just get some swapping but as I mentioned, OS/2 seems to use 32 bit variables that limit the size of virtual memory. The swap file itself is limited by using signed 32 bit based on what happens if you grow it more then 2GB (same thing as if you run out of disk space) based on experience. It makes sense as all file systems had a 2GB file limit before JFS.
The i386 is quite capable of  handling 16 TB of virtual memory IIRC, but only 4 GB of address space

[Valery Sedletski]

I always thought, it is a 4 GB, not 4 MB limit. Am I not right?

You may not be right indeed: https://en.wikipedia.org/wiki/Gibibyte...

I don't use term "GibiByte". It is not "real" term. I use only real term "GigaByte". 1000 MB is not a gigabyte. Gigabyte was always 1024 Megabytes.

[Doug Clark]

Is the AOS RAM disk bound by the same limitations as file systems on "normal" disks,
or is there some magic going on in this app?

Without thinking I set my machine up some months ago with an 8GB HPFS RAM disk and have
been operating that way for months - although I don't think I have ever copied more that
2 GB to the RAM disk.

But what would happen if I did?

If I remove the line
IFS=C:\OS2\HPFS.IFS /CACHE:2048 /CRECL:4 /AUTOCHECK:*
from CONFIG.SYS I get the error

The specified disk or diskette cannot be accessed
C:\MPTN\BIN\VDOSCTL.EXE

The help for RAM disk says, in part
"The AOS loader can format the drive(s) it creates using FAT, FAT32, or HPFS; it can also leave them unformatted. Both drives will be formatted the same way. Note that if you choose FAT, any drive over 2gb will be left unformatted."
but it doesn't say anything about other file system types.

I tried using FAT32 but that makes the RAM drive VERY SLOW when copying files to and from the disk.

Finally - I tried setting the "Format partitions using" to none and using the WPS disk object
to format the RAM disk as JFS, and the application I was running (VLC) that was reading files
from the RAM disk started behaving badly.

And the final question:  If I don't have any other partitions, other than the RAM disk,
formatted as HPFS, do I really need the /CACHE:2048 clause the IFS statement in CONFIG.SYS?
Doesn't seem to make sense to cache a RAM disk in RAM.

[Dave Yeo]

I think you could remove the cache statement, or at worst shrink it to 64 if removing it seems to screw things up.
Personally, I reformat as JFS, see my other reply.

[Andi B.]

I've setup the RAM disk but don't use it for anything except testing. I tried formated with HPFS and JFS but no matter what it's slower than any of my SSDs. Also tried HPFS (or FAT?) strategy 1 (?) which should be faster but that does not work with the applications I tried. FAT32 of course is the slowest file system for use on OS/2. So don't even think about it could be a good option on anything else then data interchange with other OSes. IMHO when you have a SSD in your system it's not worth playing with RAM disk.

[Dave Yeo]

Yes, I've been fairly disappointed with the speed of the ram disk. Still it is fast enough that I use it for %TEMP% etc, the Mozilla cache and as a scratch work area. The speed isn't too bad when formatted with JFS, and I need JFS for temporary files as building Mozilla and likely other stuff can result in temp files over 2GB.
The problem with SSD's is they have a limited lifetime, measured by the number of writes. This is more of an issue on OS/2. We're missing a trim command so can't tell the SSD which blocks you're finished with, which makes garbage collection etc harder for the SSD. You can mount the file systems under Linux and trim them though. Or backup and do a secure erased and then restore. The secure erase will write zero's on the whole device including the spare blocks.
I also find that once the DRAM cache (1 GB on my 1TB SSD), things slow down and sometimes garbage collection or such happens and the SSD stalls, things like deleting a large directory,part of the way through it stalls, you think something has failed and eventually away it goes again.
Other problems is it is very hard to align the JFS 4k blocks with the SSD 4k blocks with CHS partitioning so even writing a 1 byte file might see 2 blocks used on the SSD. A good reason to use GPT partitioning.
JFS is also not a very good file system for SSD's. The journal is always getting written, and just looking at a file causes its atime (I think that's the one) to be updated. Linux has fixes for both, the journal can be on a different device, and updating the atime can be disabled. How often do you care when a file was last read?

[Remy]

I always thought, it is a 4 GB, not 4 MB limit. Am I not right?

You may not be right indeed: https://en.wikipedia.org/wiki/Gibibyte...

I don't use term "GibiByte". It is not "real" term. I use only real term "GigaByte". 1000 MB is not a gigabyte. Gigabyte was always 1024 Megabytes.

Yes, it was

Since a long time now: GB = GigaBytes = 1000KB
while GiB (Giga binary bytes) = 1024KB  "old GB"

[Andi B.]

.... IMHO when you have a SSD in your system it's not worth playing with RAM disk.

To correct myself this seems to be not valid for other/newer systems. The numbers Doug posted on the other thread are much better than on my system. On my old system I can get transfer rates up to 160-180MBytes/s for SSDs and conventional (spinning?) disks compared to the slow ~60MBytes/s to the RAM disk. Maybe I will change my mind when I build up a new system with NVM and faster RAM some times.

[Doug Clark]

Andi,

Your post made me curious about the speed for my 11 year old Thinkpad T530 laptop - i5-3320m, 2.6 ghz CPU.

 For that machine my SSD to RAM drive transfer speeds are pretty close to yours.

SSD to RAM
----------
70,153K
74,140K
73,819K

The SSD to SSD speed are copying a file from one directory to another on the same
drive - so it is probably limited by the write speed, age of the disk, and the conflicts of reading/writing
to the same disk.

SSD to SSD
----------
13,669k

I don't use my laptop all that often. But an ankle injury forced me onto the laptop these last few weeks and I am surprised how well  it performs - even now compared to my newer Ryzen 5.   The interesting thing is how different the same applications perform on the laptop and the desktop (Ryzen) - even though both are running AOS 5.1   On the new desktop (and on my previous desktop) I experience something similar to what Dave sees when he is compiling - the system seems to freeze for 4 - 8 seconds and then resume.  It is almost like it is taking a short rest.  I see it with VLC when watching movies. This doesn't happen with the laptop.  On the new desktop Win-OS2 applications freeze the machine when running seamless mode - on the laptop they run fine.

Just a reminder I guess how complex machines are now and how difficult it is to support all the hardware that is out there.

Take care.

[Doug Bissett]

How, exactly, are you measuing speed? Read? Read/write? Write? What are the cache parameters? Which file system?

There are too many variables to even start to estimate speed, never mind try to compare it.

Using the DFSEE read/write speed test, on my old Lenovo T510, with AHCI type SSD, I get about 41 MiB/s Same machine with ArcaOS Ramdisk, tells me about 397 MiB/s. No doubt about which is faster.

I don't have exact numbers for my new Asus X570, but the RAMDISK, and NVME SSD are pretty close to the same speed. The advantage to using the Ramdisk, is that it doesn't cycle out the memory cells, in the NVME SSD, as much.  Since memory, above 4 GB, isn't used much anyway, it doesn't really matter if those cells get cycled. An interesting thing, is that a 3.1 GB system dump, to the Ramdisk takes about 2 seconds. Then after I boot, that dump gets copied to more permanent storage, and that takes about 5 minutes (it is a FAT32 dump partition). FAT32 is VERY slow, even on a NVME type SSD.

[Doug Clark]

Yeah - the tests aren't all that scientific.  Each machine had a different processor, DRAM speed and type, and bus for the M.2/SATA SSD/SATA spinning.

The RAM disk is HPFS.  All others are JFS. 

The cache on HPFS (RAM disk) is
IFS=C:\OS2\HPFS.IFS /CACHE:2048 /CRECL:4 /AUTOCHECK:*

For JFS
IFS=C:\OS2\JFS.IFS /LW:5,20,4 /AUTOCHECK:*

The spinning disks have an internal cache of 32 MB.  One drive is Western Digital, the other Toshiba.

All tests were copying a file from one disk to another using Larsen Commander, the speed reported comes from Larsen reported Average

I used various file sizes when copying across a network.  The file size, destination drive and source drive didn't make much difference in the times.  I think the network was the limiting factor.

On the Ryzen machine (RAM to M.2, RAM to RAM, Spinning to Ram, Ram to Spinning) the file size was 2 GB, which should have been large enough to swamp any cache.

On the Lenovo t530 laptop the file size was a little smaller. HPFS and JFS on that machine are setup the same as the Ryzen desktop.

I also tried to test SATA SSD to RAM on the WSeB/eComStation machine.  But the RAM drive on that machine is only 64MB in size and a 64MB file copied so fast I could not get the time off the Larsen File Copy screen.  The file system for that disk says "RAMFS" and I don't think you can make it larger than 64MB.  But you can share it across a network with WSeB.  I think it also runs in lower memory only.


I used various file sizes when
The file size

[Doug Bissett]

I am not going to try to duplicate your configuration. there is really no point in doing that. I will comment on your configuration (which is similar to what I used a number of years ago);

RAMFS is not a good choice: It takes up lower shared memory space, which is in critical short supply. The ArcaOS RAMDISK (or use the QS Loader - available from HOBESARCHIVE, which is the same RAMDISK). It can use memory above what OS/2 can use, meaning that only a small driver uses system memory. A simple script, in STARTUP.CMD, can format it to JFS (start with selecting FAT32 as the format), which has full support for EAs and things like the SWAPPER (which doesn't seem to be used anyway, when you have more than 2 GB of memory).

HPFS is no longer a good choice. It also uses critical lower memory space. You likely use JFS anyway, which, on it's own, performs better than HPFS (especially when a CHKDSK is required at boot time).

If you can, use VIRTUALADDRESSLIMIT=3072. If that doesn't work, try 2560, which should work on all ArcaOS systems (and probably ECS and earlier versions of OS/2).

[Dariusz Piatkowski]

Not quite RAM disk related, but for the folks who are experiencing that "...system seems to freeze for 4 - 8 seconds and then resume..." behaviour: if you are using JFS, this may be a symptom of your JFS settings causing it to run out of free buffers and needing to purge the cache(s) to free up the buffers.

This is well documented by Sjoerd Visser in his "Dynamically Tuning the JFS Cache for Your job" presentation deck from way back in 2009.

Bottom line: this can be brought on by several 'system use' activities, but it basically causes the JFS code to write out dirty buffers to disk. The key to dealing with this on my rather large JFS cache (1G) was to watch the typical system use (log cstats output) and adjust the MIN & MAX free buffer settings in pair with the overall LazyWrite setups.

This is a trial & error thing as your machine will be heavily driven by what your usage patterns are.

For what it's worth, here is what I have:

CONFIG.SYS:
IFS=G:\OS2\JFS.IFS /CACHE:1048576 /LW:32,128,8 /AUTOCHECK:*
CALL=G:\OS2\CMD.EXE /Q /C G:\OS2\CACHEJFS.EXE /LW:32,128,8 /MINBUFFER:8000 /MAXBUFFER:24000 >NUL

5 Days into using my box (last time it was up for 27 days, normal desktop stuff, nothing fancy), castat shows:

Code: [Select]

[G:\]cstats
cachesize    262144   cbufs_protected       35795
hashsize     131072   cbufs_probationary    22467
nfreecbufs   101902   cbufs_inuse               0
minfree        8000   cbufs_io                  0
maxfree       24000   jbufs_protected      101075
numiolru          0   jbufs_probationary      894
slrun        136870   jbufs_inuse               0
slruN        174762   jbufs_io                  0
Other            11   jbufs_nohomeok            0

...with the nfreecbufs never dropping so low that they show zero (0) as slruN approaches slrun value.

I started with MIN=8000 and MAX=16000, and that gave a pretty solid system, although sometimes I would get that tell-tale "hang" feeling. So I moved to MIN=4000 thinking that would free up the buffers for caching duties..welllll...no sirr....wrong move...that completely resulted in a solid and repeatable "system hang". So back to the drawing board so to speak, I set my MIN=8000 and increased MAX=24000. RESULTS => SOLID, the most solid system I have had for years.

The thing that 'ruins' my JFS cache (spoils it actually) is the nightly disk copy (rcopy) run. If it wasn't for that activity my cbufs_protected would stay very large, which means I have a good amount of content that's being successfully cached. Of course, the jbufs_protected is equally important as that allows JFS to quickly figure out where to "go" to retrieve the content as opposed to having to read the data from the disk itself. Again, which one should be the focus for you entirely depends on what disk access patterns you see.

Anyways...balance, somewhere out there are the right settings for your machine.

Last but not least, my JFS formatted RAM DISK results are (diskio):

Drive cache/bus transfer rate: 629714 k/sec
Data transfer rate on cylinder 0   : 680303 k/sec
Data transfer rate on cylinder 634 : 680021 k/sec

meanwhile the SSD results are:

Drive cache/bus transfer rate: 125467 k/sec
Data transfer rate on cylinder 0   : 286615 k/sec
Data transfer rate on cylinder 30399: 253408 k/sec

The ram disk is 2-3x faster here.

EDIT
====
One more thing to add to this, albeit this is easily identifieable and most likely NOT the situation everyone else is seeing: I tried running AHCI driver here and on my hardware that would result in what seemed like a complete HARD lock for about 4-8 secs at a time. Once that "event" passed the system was available for use once again. I tried a boat-load of different setups and configs but none of it helped. Subsequently i went away from AHCI.

[Dave Yeo]

Dariusz, did you ever publish your updates to diskio, also have you considered adding them to Sysbench, which basically uses diskio for disk benchmarking.