AOS RAM Disk

[Andi B.]

...
There are too many variables to even start to estimate speed, never mind try to compare it.
...

True but not fully. You can compare your own SSDs and RAMDISK and cache settings very well when you use the same test method. You also can give others a clue what seems to be possible on other systems compared to yours. F.i. before the post here I never saw any system with only nearly that fast RAMDISK as posted here. The same goes with NVM compared to SSD which is interesting to me. I didn't bother playing with NVM before and so don't have any experience myself until now.

On the other hand I don't rely on values of LarsenCommander UNTIL I've
- rechecked the same copy operation from command line and
- I repeated the test numerous times and
- the whole copy process (number of files and/or size of files) takes more than 15-30 seconds

Remember I'm the one how tweaked the copy algorithm in LCMD :-) Although I trust the values LCMD sums up you've to be careful to extrapolate what you're really measure (filesystem cache performance, SSD write/read performance or only SSD cache performance, ...).

[Dariusz Piatkowski]

hey Dave!

Dariusz, did you ever publish your updates to diskio, also have you considered adding them to Sysbench, which basically uses diskio for disk benchmarking.

Nope...b/c I started the work on converting it to handle SSD's 4K sector sizes and ran into some problems with DosDevIOCtl32 return RC=87, which I wasn't able to resolve.

However, for what it's worth, here is the DISKIO version that I previously published in the forum here on a separate thread => https://www.os2world.com/forum/index.php/topic,2676.0.html.

[roberto]

Hello Dariusz
 I found your config.sys very interesting.!!
 At first I had no problems but it wasn't going well either. I have the feeling that I was missing something and maybe you are missing something.
I pass on my modifications to the config.sys, incorporating your slightly modified values.

Code: [Select]


IFS=C:\OS2\JFS.IFS /CACHE:1048576 /LW:32,128,8 /AUTOCHECK:*
CALL=C:\OS2\CMD.EXE /Q /C C:\OS2\CACHEJFS.EXE /LW:32,128,8 /MINBUFFER:8000 /MAXBUFFER:15000 >NUL
buffers=20
 SWAPPATH=C:\OS2\SYSTEM 0 512000
 SWAPPATH=C:\OS2\SYSTEM 2000 1045000
And surprise with the memory, I also attach the memory results with different values. And I really like what I see.

Code: [Select]

[C:\]mem /v

Total physical memory:      8 142 MB
Accessible to system:       3 054 MB
Additional (PAE) memory:    5 088 MB

Resident memory:              187 MB
Available virtual memory:   3 833 MB

Available process memory:
  Private low memory:         340 MB
  Private high memory:      2 240 MB
  Shared low memory:          276 MB
  Shared high memory:       2 227 MB

[C:\]cstats
cachesize     32768   cbufs_protected       10302
hashsize      16384   cbufs_probationary     2585
nfreecbufs    18938   cbufs_inuse               0
minfree        8000   cbufs_io                  0
maxfree       13000 **   jbufs_protected         681
numiolru          0   jbufs_probationary      255
slrun         10983   jbufs_inuse               0
slruN         21844   jbufs_io                  0
Other             7   jbufs_nohomeok            0

[C:\]mem /v

Total physical memory:      8 142 MB
Accessible to system:       3 054 MB
Additional (PAE) memory:    5 088 MB

Resident memory:              187 MB
Available virtual memory:   3 833 MB

Available process memory:
  Private low memory:         342 MB            *****
  Private high memory:      2 240 MB
  Shared low memory:          279 MB           ******
  Shared high memory:       2 227 MB

[C:\]cstats
cachesize     32768   cbufs_protected       10294
hashsize      16384   cbufs_probationary     2594
nfreecbufs    18941   cbufs_inuse               0
minfree        8000   cbufs_io                  0
maxfree       14000 **   jbufs_protected         747
numiolru          0   jbufs_probationary      185
slrun         11041   jbufs_inuse               0
slruN         21844   jbufs_io                  0
Other             7   jbufs_nohomeok            0

[C:\]mem /v

Total physical memory:      8 142 MB
Accessible to system:       3 054 MB
Additional (PAE) memory:    5 088 MB

Resident memory:              186 MB
Available virtual memory:   3 833 MB

Available process memory:
  Private low memory:         353 MB               *****
  Private high memory:      2 240 MB
  Shared low memory:          290 MB               *****
  Shared high memory:       2 227 MB

[C:\]cstats
cachesize     32768   cbufs_protected       10293
hashsize      16384   cbufs_probationary     2590
nfreecbufs    18944   cbufs_inuse               0
minfree        8000 **   cbufs_io                  0
maxfree       15000 **   jbufs_protected         750
numiolru          0   jbufs_probationary      184
slrun         11043   jbufs_inuse               0
slruN         21844   jbufs_io                  0
Other             7   jbufs_nohomeok            0

Pay attention that I'm using two swappaths, although only the last one is used, I haven't really tried to remove the first one, in case there is a difference or not.
Don't put the swappath on a ramdisk, it won't work well for you.
I get these memory values after restarting the computer.
Saludos

[Dariusz Piatkowski]

Hello roberto!

Code: [Select]

...
IFS=C:\OS2\JFS.IFS /CACHE:1048576 /LW:32,128,8 /AUTOCHECK:*
CALL=C:\OS2\CMD.EXE /Q /C C:\OS2\CACHEJFS.EXE /LW:32,128,8 /MINBUFFER:8000 /MAXBUFFER:15000 >NUL
...
[C:\]cstats
cachesize     32768   cbufs_protected       10302
hashsize      16384   cbufs_probationary     2585
nfreecbufs    18938   cbufs_inuse               0
minfree        8000   cbufs_io                  0
maxfree       13000 **   jbufs_protected         681
numiolru          0   jbufs_probationary      255
slrun         10983   jbufs_inuse               0
slruN         21844   jbufs_io                  0
Other             7   jbufs_nohomeok            0


So here is the thing: my JFS cache is really large, 1Gig, therefore the '/CACHE:1048576' setting.

When you attempted to run as big of a cache on your system, given that JFS wasn't able to allocate that much RAM it actually defaulted to 10% of your accessible RAM (or maybe 20%??? I don't remember the default at the moment), which gave you a JFS cache of about 128M.

This is confirmed by your cstats value of cachesize=32768, where each such buffer is 4096 bytes. You can get a true JFS cache size by running the following:

Code: [Select]

[G:\os2]cachejfs

          SyncTime:      32 seconds
            MaxAge:     128 seconds
        BufferIdle:       8 seconds
        Cache Size: 1048576 kbytes
        Min Free buffers:    8000 (   32000 K)
        Max Free buffers:   24000 (   96000 K)
Lazy Write is enabled

Therefore, if you are interested in increasing your JFS cache size you may want to inch-up on the 1Gig slowly...maybe go to 256M, than 512M, etc...

[roberto]

Hi Dariusz,
 You are right about many things, in fact at the beginning, it only showed me 131mb. The cachejfs.
 But reducing the virtualaddresslimit to 1024, it already shows me 1gb of cache.
 The price is the reduction of memory.
  I've still set up my ArcaOS NAS like this, but I don't see any performance improvements in file transfer.
 I'll have to do more tests.
 Your setup is very stable, I just reduced the value from 24000 to 15000.

Code: [Select]

[C:\]mem /v

Total physical memory:      8 142 MB
Accessible to system:       3 054 MB
Additional (PAE) memory:    5 088 MB

Resident memory:            1 143 MB
Available virtual memory:   3 809 MB

Available process memory:
  Private low memory:         344 MB
  Private high memory:        448 MB
  Shared low memory:          280 MB
  Shared high memory:         435 MB

[C:\]cstats
cachesize    262144   cbufs_protected       10288
hashsize     131072   cbufs_probationary     2594
nfreecbufs   248292   cbufs_inuse               0
minfree        8000   cbufs_io                  0
maxfree       15000   jbufs_protected         776
numiolru          0   jbufs_probationary      187
slrun         11064   jbufs_inuse               0
slruN        174762   jbufs_io                  0
Other             7   jbufs_nohomeok            0

[C:\]cachejfs

          SyncTime:      32 seconds
            MaxAge:     128 seconds
        BufferIdle:       8 seconds
        Cache Size: 1048576 kbytes
        Min Free buffers:    8000 (   32000 K)
        Max Free buffers:   15000 (   60000 K)
Lazy Write is enabled


[C:\]
Saludos

[roberto]

  I've still set up my ArcaOS NAS like this, but I don't see any performance improvements in file transfer.
 I'll have to do more tests.

Well, I answered me about the speed tests, and the improvement is noticeable.
A folder that normally took 23 minutes to copy is copied in 1 minute 37 seg. of 11gb.
This is from a 12tb gpt disk with 2tb x 6 partitions, to a 240mb gpt sdd disk on the same computer.
But yesterday from the network the result was poor, because the network is my bottleneck.
Saludos

[Dariusz Piatkowski]

Hello Roberto,

... But reducing the virtualaddresslimit to 1024, it already shows me 1gb of cache.
 The price is the reduction of memory...

Yup, this JFS thing is all about balancing the hardware resources in a manner most beneficial to the way YOU use YOUR machine. So there really are no generalizations, other than perhaps some starting points?

With my particular hardware combo I'm able to run this large of a JFS cache with 'VIRTUALADDRESSLIMIT=2048' setup. I found that going lower than that will start causing problems with our current applications, as others have pointed out as well.

So the 'balance' thing here is probably more heavily skewed towards 'being able to run an app' as opposed to 'having a fast FS cache'.

...I've still set up my ArcaOS NAS like this, but I don't see any performance improvements in file transfer.
 I'll have to do more tests...

I would NOT expect at all there to be file transfer performance improvements, other than I suppose the fact that the larger cache allows your target write operation to be cached.

...
[C:\]cachejfs

          SyncTime:      32 seconds
            MaxAge:     128 seconds
        BufferIdle:       8 seconds
        Cache Size: 1048576 kbytes
        Min Free buffers:    8000 (   32000 K)
        Max Free buffers:   15000 (   60000 K)
Lazy Write is enabled...

One last comment is re: SyncTime, MaxAge and BufferIdle. Be careful with these. I have my setups as large as they are because my machine is hooked up to a UPS fulltime. Therefore, the chance of power going out and the JFS cache NOT getting flushed out is very minimal. Still, a hard TRAP could still happen, so there is always some risk there.

Just for reference, here are my notes on this topic:

Code: [Select]

/LAZY:synctime,maxage,bufferidle
enables write cache with the following parameters:

- synctime : the interval at which the sync thread runs
default = 16

- maxage : is the longest time that a frequently modified file is kept in cache
default = synctime * 4

- bufferidle : is the time indicating a "recent" change. Changes newer than this
               value are not written unless the last write was older than maxage.
default = MIN(1,synctime/8)


[roberto]

Hi Dariusz
Thank you for sharing your experience and effort.
Modifying the LW about 15 or 20 years ago, experimenting sent me a computer to disaster,
and I was left with the idea of not touching it. That's why I know it's difficult.
But your values seem very good to me, the processors on the server are at ZERO or almost zero that's very good.
Now I'm trying to set up my test equipment with twice the standard cache without losing memory, using the virtualaddresslimit on 3072.
And this looks great, even with resource-depleting applications, but... requires more testing.
Saludos

[roberto]

Lines to modify the config.sys to have a JFS cache twice the current one:131072=2 x 65536 maintaining or
even increasing the available memory.
-------------------------
IFS=C:\OS2\JFS.IFS /CACHE:131072 /LW:32,128,8 /AUTOCHECK:*
...

CALL=C:\OS2\CMD.EXE /Q /C C:\OS2\CACHEJFS.EXE /LW:32,128,8 /MINBUFFER:655 /MAXBUFFER:3679 >NUL
...
BUFFERS=16
...
SWAPPATH=C:\OS2\SYSTEM 2000 1045000
...
VIRTUALADDRESSLIMIT=3072
...
--------------------------

Because of the things and notices of my experience:
How to calculate the minbuffer:
Putting Minbuffer: 0, the system will calculate the new minimum automatically returning the 655 value
How to calculate the Maxbuffer, I have been increasing the value while Private Low Memory: or the Shared Low Memory:
increase. And I reduce it, if it decreases. Looking for the values ​​of maximum memory available.

I believe that having the minimum, the important thing is the difference between these two values.

The buffers = 16, according to EDM, in case of configuring the cache, says reduce the buffers number.
20 It is also a good value.
I contribute link:
Now I can't find it.

Regarding SwaPPath, I remind you of this link, to be able to calculate Valid Swappat,
Another thing is that they work as it should, or that values ​​can work better than others:

https://www.os2world.com/forum/index.php/topic,3232.msg38051.html?PHPSESSID=4clqlps3of12k9kp8tc7rmr3cc#msg38051

In fact, if someone intends to try a 2048 1045000 swappath, you can fail the hard drive. Because
The 2048 value is not valid for the Swapper.

Also comment, that activating Hyperrating HT in many bios, 2MB are lost in Private Low Memory: and another 2MB in Shared Low Memory:
But the processors work better. Could it be the opposite? I don't know.

And that's it.

It's not all, I understand that for a simple NAS type server with few applications it can be interesting
Have less memory and more cachejfs to give better service. In this case I have the:
Buffers in 20
and the
SwaPPath = C: \ OS2 \ System 2000 2000000
and the Cache Size: 1048576 Kbytes
Minbuffer 0 or 655
Maxbuffer 7655
with the virtualaddresslimit in 1024. Only this is valid of the Page Boundary, others not and checked.
This conbination works quite well, but here there may be better.

Saludos