OS/2, eCS & ArcaOS - Technical > Storage

JFS cache sizing, and system "speed-up"

<< < (7/8) > >>

Dariusz Piatkowski:
Alright guys...I've got DATA!!!

Well, data is good, but visuals make it a tad easier to understand.

OK, so take a look at about 4 days of runtime stats. This is the capture of cstats that I mentioned earlier, and what i'm focusing on is the relationship between the metrics.

The thing to note is the nightly spikes...these are my RSYNC runs, which copy my data to NAS and also do a full disk RSYNC of the OS/2 partition to a local disk backup partition.

So I thought it would be interesting to deep-dive some more into these because if anything, these are going to more directly point out the cache behaviour given that a RSYNC copy will require a full disk scan, and therefore one would expect the METADATA to be heavily hit in the JFS cache.

This is shown in the 2nd capture where I basically narrowed things down to just these nightly spikes. For that reason ignore the 'plunge' you see in-between these runs, that's just a placemarker that I used to indicate the manual cut-off between each day.

I have gathered 7 different logs so far since I started experimenting with the larger JFS cache settings, each spanning about 3-4 days. The earlier logs have different JFS cache sizes along with different parameters.

Having said that, I honestly will tell you the following (keep in mind, this is just based on a single 4 day 1G cache cycle):

1) FF is just flat out faster, pages render quicker which is telling me that FF's cache (set to 350M max) persists in the JFS cache for longer
2) the faster application load is present for all other apps that I regularly use: Thunderbird, PMMail, Lotus 1-2-3, OpenOffice, Lucide, PMView, VSE and few others

Now here is the kicker which I did not exect to see: my shared memory over these past 4 days never dropped below 55M. This is VERY different from my prior experience where routinely after 3-4 days of normal system use (that typically includes daily FF re-starts - which i do once I see FF get to about 1G mem consumption as shown in Theseus) I basically exhaust all available shared memory and even shutting down all apps will not bring things back (because of the well known segmentation problem).

Sooo....I obviously need to re-do this cycle multiple times and attempt to better understand the cstats information.

Dariusz Piatkowski:
Here are the Theseus metrics from a Clean Boot (so basically a re-boot with my default application/utility mix: UPS stuff, Xit, XWP, etc...none of the other major apps):

1) System => Nonswappable Memory

--- Code: ---Nonswappable Memory analysis:
Apps & DLLs      = 00024000 ->     144K -> 0.141M
Process overhead = 0025B000 ->    2412K -> 2.355M
DD allocated     = 45615000 -> 1136724K -> 1110.082M
DOS              = 0001E000 ->     120K -> 0.117M
VDisk            = 00000000 ->       0K -> 0.000M
File system      = 00030000 ->     192K -> 0.188M
Kernel code      = 000B3000 ->     716K -> 0.699M
Kernel data      = 01292000 ->   19016K -> 18.570M
Kernel heap      = 003B9000 ->    3812K -> 3.723M

Total            = 46FE0000 -> 1163136K -> 1135.875M

--- End code ---

2) System => Free, Idle and Locked Memory

--- Code: ---Free, Idle, and Locked Memory:
Free                RAM = 7A902000 bytes (2008072K) (1961.008M)
Idle                RAM = 0001A000 bytes (  104K) ( 0.102M).
        (Dirty idle RAM = 00015000 bytes (   84K) ( 0.082M)).
Long  Term Locked   RAM = 0005E000 bytes (  376K) ( 0.367M).
Short Term Locked   RAM = 00000000 bytes (    0K) ( 0.000M).
Short & Long Locked RAM = 00000000 bytes (    0K) ( 0.000M).

--- End code ---

3) System => Kernel Information => System Object Summary

--- Code: ---  Object Allocated Committed   Present   Swapped
 address    memory    memory    memory    memory  Description
          --------  --------  --------  --------
Totals:   619D9000  47B62000  47AD4000  00000000  (in bytes)
           1599332   1174920   1174352         0  (in Kbytes)
          1561.848  1147.383  1146.829     0.000  (in Mbytes)
Number of objects = 1024.

Analysis of 'Free' areas:
There are 398 free blocks which total 1E5E7000 (497564K or 485.902M)
The largest 10 free areas are:
 address      size
8FC10000  1D016000 (475224K or 464.086M)

--- End code ---

4) System => General System => General System Information

--- Code: ---General System Information:

OS/2 version        = 2.45, revision = 0.
Os2krnl build level = 14.203

SYSLEVEL.OS2 information
OS/2 Component ID   = 5639A6101
CSD GA level        = XR04503
CSD Previous level  = XR0C006

SYSLEVEL.FPK information
OS/2 Component ID   = 566933010
CSD Current level   = XR0C006
CSD Previous level  = XR0C006

Theseus4 Version    = 4.001.00
Machine information: Model = 252 (0xFC),
                     Submodel = 1 (0x01),
                     Revision = 0 (0x00),
                     ABIOS = 0 (0x00).
BIOS date = 10/31/12.
RAM available to OS/2 = C7F0B000 bytes (3199.043M).
It appears that all of it is being used as 'paging space' by OS/2.
  (This is the 'proper' usage of the memory.)

Following are the values from DosQuerySysInfo:
 1. QSV_MAX_PATH_LENGTH      = 260.
 3. QSV_MAX_PM_SESSIONS      = 16.
 4. QSV_MAX_VDM_SESSIONS     = 128.
 5. QSV_BOOT_DRIVE           = 7.
 7. QSV_MAX_WAIT             = 1.
 8. QSV_MIN_SLICE            = 32.
 9. QSV_MAX_SLICE            = 32.
10. QSV_PAGE_SIZE            = 4096.
11. QSV_VERSION_MAJOR        = 20.
12. QSV_VERSION_MINOR        = 45.
14. QSV_MS_COUNT             = 263519 (0:04:23).
15. QSV_TIME_LOW             = 1637843077.
16. QSV_TIME_HIGH            = 0.
17. QSV_TOTPHYSMEM           = -940527616 (3275820K -> 3199.043M).
18. QSV_TOTRESMEM            = 1198608384 (1170516K -> 1143.082M).
19. QSV_TOTAVAILMEM          = 2082484224 (2033676K -> 1986.012M).
20. QSV_MAXPRMEM             = 370147328 (361472K -> 353.000M).
21. QSV_MAXSHMEM             = 313262080 (305920K -> 298.750M).
22. QSV_TIMER_INTERVAL       = 310.
23. QSV_MAX_COMP_LENGTH      = 255.
26. QSV_NUMPROCESSORS        = 6.
27. QSV_MAXHPRMEM            = 1409286144 (1376256K -> 1344.000M).
28. QSV_MAXHSHMEM            = 1404301312 (1371388K -> 1339.246M).
29. QSV_MAXPROCESSES         = 128.
System Anchor Segment (SAS) selector = 0070.
Size of PTDA                 = 0768 bytes.
Size of TCB                  = 0304 bytes.
Size of Alias        record  = 0008 bytes.
Size of Arena        record  = 0016 bytes.
Size of Object       record  = 0010 bytes.
Size of Context      record  = 0005 bytes.
Size of Page Frame   record  = 000C bytes.
Size of Virtual Page record  = 000A bytes.
Size of SFT entry            = 00A2 bytes.
har of System Arena Sentinel = 0004.
har of Shared Arena Sentinel = 0006.
har above 512m Shr Arena Sen = 0005.
har of Page Frame table      = 001C.
har of Virtual Page table    = 001F.
System Page Directory        @ FE48A000.
System Page Tables start     @ FE200000.
DLL code Page Tables start   @ FE1F8000.
Start address shared global  @ 1E000000.
Alias Record Table           @ FCB16020.

--- End code ---

Let me know if there is anything else you want me to catpure.

Dave Yeo:
For the Mozilla cache, I use the ramdisk, fairly easy to setup with SM. For Firefox need to use about:config to create preference something like this (H: is my ramdisk) "browser.cache.disk.parent_directory;H:\mozilla\firefox"

Dariusz Piatkowski:
Hi Dave!

--- Quote from: Dave Yeo on November 25, 2021, 08:05:44 pm ---For the Mozilla cache, I use the ramdisk, fairly easy to setup with SM. For Firefox need to use about:config to create preference something like this (H: is my ramdisk) "browser.cache.disk.parent_directory;H:\mozilla\firefox"

--- End quote ---

Hmm, from my perspective I'm not sure what the longer-term performance gain would be in terms of doing a FF ramdisk cache, after all I would want the cache to persist in order to speed up the load of the bigger sized web page elements the very next time they are requested. I suppose if you want to keep the profile clean (and smaller sized), and do not care to carry over the cache past any re-boot, that would do it.


...for the Thunderbird client that is absolutely what i'm going to take a look at. I see no point in keeping that cache persistent since emails will always change, and I would think that causes the normal disk cache to just fill up with stuff that rarely ever gets re-used, with the exception of very few standard emails: bills from the same companies, etc.

I do not have SM installed here so I'm not sure what you mean by "fairly easy to set up", but would that still hold true for TB? If so, can you point me in the right direction before I attempt to create that FF-like preference?

In the meantime I'm off to research the TB cache settings a bit...


Dave Yeo:
My ramdisk does persist over warm reboots (last computer, it even persisted over a couple of seconds of no power) and every time the browser crashes, which is too often lately, the whole cache gets invalidated.
As for Thunderbird, I'd think the cache is mostly used when displaying web pages, which it is quite capable of doing. I even have an add-on, thunderbrowse, which exposes displaying web pages.
For SM, you can point the cache in the Preferences under Advanced-->Cache. I assume that for TB, you have to open the Options-->Advanced-->Config Editor and create the same preference as for Firefox.


[0] Message Index

[#] Next page

[*] Previous page

Go to full version