Author Topic: shared memory value - Odin win32k or DosQuerySysInfo???  (Read 16249 times)

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1354
  • Karma: +26/-0
    • View Profile
shared memory value - Odin win32k or DosQuerySysInfo???
« on: September 26, 2023, 10:08:32 pm »
I'm making some adjustments to the PUMonitor utility to show the largest available shared memory area (low shared memory I believe) alongside the current 'available memory' estimate.

Specifically, the PUMonitor utility relies on win32k.h definitions and library calls:

Code: [Select]
/* $Id: win32k.h,v 1.11 2001/03/15 20:03:10 bird Exp $
 *
 * Top level make file for the Win32k library.
 * Contains library and 32-bit IOCtl definition.
 *
 * Copyright (c) 2000 knut st. osmundsen (knut.stange.osmundsen@pmsc.no)
...

That library has a K32SYSTEMMEMINFO record which appears to already collect the shared memory information:

Code: [Select]
...
    /* Virtual Memory manager info. */
    ULONG       ulAddressLimit;         /* VM: Current user virtual address limit - use this for high arena check. (VirtualAddressLimit / 0x20000000) */
    ULONG       ulVMArenaPrivMax;       /* VM: Current highest address in the private arena. (vmRecalcShrBound()) */
    ULONG       ulVMArenaSharedMin;     /* VM: Current lowest address in the shared arena. (ahvmShr) */
    ULONG       ulVMArenaSharedMax;     /* VM: Current highest address in the shared arena. (ahvmShr) */
    ULONG       ulVMArenaSystemMin;     /* VM: Current lowest address in the system arena. (ahvmhSys) */
    ULONG       ulVMArenaSystemMax;     /* VM: Current highest address in the system arena. (ahvmhSys) */
    ULONG       ulVMArenaHighPrivMax;   /* VM: Current highest address in the high private arena - aurora/smp only. (vmRecalcShrBound) */
    ULONG       ulVMArenaHighSharedMin; /* VM: Current lowest address in the high shared arena - aurora/smp only. (ahvmhShr) */
    ULONG       ulVMArenaHighSharedMax; /* VM: Current highest address in the high shared arena - aurora/smp only. (ahvmhShr) */
...

Just looking at that single header file I see the following functions being called out:

Code: [Select]
/*******************************************************************************
*   External Functions                                                         *
*******************************************************************************/
/* Win32k APIs */
APIRET APIENTRY  libWin32kInit(void);
APIRET APIENTRY  libWin32kTerm(void);
BOOL   APIENTRY  libWin32kInstalled(void);
APIRET APIENTRY  libWin32kQueryOptionsStatus(PK32OPTIONS pOptions, PK32STATUS pStatus);
APIRET APIENTRY  libWin32kSetOptions(PK32OPTIONS pOptions);

/* "Extra OS2 APIs" */
APIRET APIENTRY  DosAllocMemEx(PPVOID ppv, ULONG cb, ULONG flag);
APIRET APIENTRY  W32kQueryOTEs(HMODULE hMTE, PQOTEBUFFER pQOte, ULONG cbQOte);
APIRET APIENTRY  W32kQuerySystemMemInfo(PK32SYSTEMMEMINFO pMemInfo);
APIRET APIENTRY  W32kProcessReadWrite(PID pid, ULONG cb, PVOID pvSource, PVOID pvTarget, BOOL fRead);
APIRET APIENTRY  W32kHandleSystemEvent(ULONG ulEvent, HEV hev, BOOL fHandle);

/* Helper function */
USHORT APIENTRY  libHelperGetCS(void);
...

In particular PUMonitor uses the W32kQuerySystemMemInfo function to retrieve the existing physical available memory information (cbPhysAvail), and it would seem the following would provide what I'm looking for:

ULONG       ulVMArenaSharedMin;     /* VM: Current lowest address in the shared arena. (ahvmShr) */
ULONG       ulVMArenaSharedMax;     /* VM: Current highest address in the shared arena. (ahvmShr) */

However...I haven't been able to find any references and DEV explanations for these, so I'm curious if anyone has used these?

The DosQuerySysInfo is a known way to get this, but obviously I'd rather not re-invent the wheel if I can use the existing approach.

As always, appreciate the feedback!

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5088
  • Karma: +117/-1
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #1 on: September 26, 2023, 10:33:41 pm »
Wouldn't using win2k.h introduce a dependency in win2k.sys, which is now depreciated IIRC.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1354
  • Karma: +26/-0
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #2 on: September 26, 2023, 10:53:38 pm »
Hi Dave,

Wouldn't using win2k.h introduce a dependency in win2k.sys, which is now depreciated IIRC.

Somehow this must not be the case: I've had PUMonitor running on my machine for quite a few years now...I did actually make a few small changes to deal with my multi-core box back in 2020 and I do NOT run the win32k.sys driver on my machine.

Having said that, I do have the Odin runtime installed, so if anything I suspect that the LIBs included with the PUMonitor source tree are in fact making use of the ODIN DLLs...the only problem here is that 'DLL Tree' doesn't actually show anything like that.

Rich Walsh

  • Sr. Member
  • ****
  • Posts: 370
  • Karma: +24/-0
  • ONU! (OS/2 is NOT Unix!)
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #3 on: September 28, 2023, 09:24:00 pm »
Wouldn't using win2k.h introduce a dependency in win2k.sys, which is now depreciated IIRC.

Somehow this must not be the case: I've had PUMonitor running on my machine for quite a few years now. [...] and I do NOT run the win32k.sys driver on my machine.

XCenter's "Sentinel Memory Watcher" also attempts to use win32k. If it fails to open the driver, then it falls back to conventional means to generate it numbers. This 'PUMonitor' (which I'd never heard of before) almost certainly does the same. Without a device driver (or access to a backdoor into the kernel), the kernel data the util wants to use is simply not accessible to a Ring-3 app.

Rich Walsh

  • Sr. Member
  • ****
  • Posts: 370
  • Karma: +24/-0
  • ONU! (OS/2 is NOT Unix!)
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #4 on: September 28, 2023, 10:58:03 pm »
Dariusz, thanks for starting this thread. It motivated me to resurrect a long-stalled project: a memory meter that shows linear address usage the way OS/2 sees it.

All the mem monitors I've seen divide memory into Private and Shared. However, that "Shared" memory is actually two separate pieces: actual Shared memory, and the *Unassigned* memory that lies between private and shared. This address space can be assigned to either as needed. Because existing monitors lump the two together,  they will claim that your *shared* memory just decreased if any of that Unassigned memory gets reassigned for use as Private. This scenario happens constantly with the browsers and gives a mistaken impression of how they use all that memory.

Attached is 'memry.exe', a text-mode memory monitor that addresses the issue. It's a proof-of-concept that I'll eventually reimplement as an XCenter widget. Here's what it's output looks like:
Code: [Select]
| Private | Unassigned | Shared  [Used Free] |
|   150    |      207         |   155      [ 155      0  ] |

There are two display modes:
* static - enter the 'memry' command on the top line of an OS/2 window. The data line will update silently every 2.5 seconds.
* scrolling - enter the 'memry' command on the last line of an OS/2 window. The data line will update every 2.5 seconds and scroll down one line, giving you a listing of recent memory usage.

Edit: see the next posting for a newer version
« Last Edit: September 30, 2023, 10:39:01 am by Rich Walsh »

Rich Walsh

  • Sr. Member
  • ****
  • Posts: 370
  • Karma: +24/-0
  • ONU! (OS/2 is NOT Unix!)
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #5 on: September 30, 2023, 10:35:34 am »
Attached is an update to 'memry' that now displays both high and low memory usage. Start it before opening your browser, then watch its numbers change when you open it. Here's my before and after with Dooble.


Alex Taylor

  • Sr. Member
  • ****
  • Posts: 398
  • Karma: +6/-0
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #6 on: September 30, 2023, 01:38:08 pm »
It may be appropriate to point out my memory-reporting tool:
https://github.com/altsan/os2-mem
which also breaks down memory regions when the /v switch is used, although it lacks the ability to identify "unassigned" like Rich's.

One feature it does have, on ArcaOS systems (and possibly also on systems with QSINIT) is the ability to report the full amount of physical RAM, and any PAE-exclusive region available for use as a memory disk, even if OS/2 can't access it as system RAM.

(It's installed by default as mem.exe on all ArcaOS systems.)

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1354
  • Karma: +26/-0
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #7 on: October 18, 2023, 03:07:57 pm »
Hey Rich!

Attached is an update to 'memry' that now displays both high and low memory usage. Start it before opening your browser, then watch its numbers change when you open it. Here's my before and after with Dooble.

This is pretty neat stuff. I have been dreaming of building a GUI "browser" that basically visually maps the current memory utilization, goal being: make it human digestable!!!

So while tools like Theseus for example will expose a whole bunch of stuff, I often times wish that I could just see a quick snapshot that "lays it out" on the screen. Now, not knowing enough about this subject that may very well be a naive way of looking at it, but worth a try.

Do you mind sharing your source? I am very interested in the APIs you are using there, especially in light of the addition I'm trying to make to PUMonitor (to show the SHARED memory use).

In fact, running your monitor and comparing it's output to the likes of 'Free Shared Mem' util, or Alex's 'mem', I am not quite seeing the same numbers.

For example:

1) mem output
Quote
Total physical memory:      8,191 MB
Accessible to system:         3,199 MB
Additional (PAE) memory:  4,992 MB

Resident memory:            1,184 MB
Available virtual memory:    567 MB

Available process memory:
  Private low memory:         192 MB
  Private high memory:     1,344 MB
  Shared low memory:          84 MB
  Shared high memory:       564 MB

2) Free Shared Mem
88,408,064 bytes

3) Rich's memry
Quote
[Lo Priv | Unused | Shared  Used Free]  [Hi Priv | Unused | Shared  Used Free]
     109 |       84  |      318   291   27       1338 |     565  |   145      145   0

So focusing on SHARED memory:

mem shows: 84 MB
memry shows: Lo Priv Unused as 84

Are these two talking about the same memory area?...because your util Rich is certainly showing different numbers under the "Shared" label and neighter the Lo nor Hi match.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1354
  • Karma: +26/-0
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #8 on: October 18, 2023, 03:16:29 pm »
Gents!

Wouldn't using win2k.h introduce a dependency in win2k.sys, which is now depreciated IIRC.

Somehow this must not be the case: I've had PUMonitor running on my machine for quite a few years now. [...] and I do NOT run the win32k.sys driver on my machine.

XCenter's "Sentinel Memory Watcher" also attempts to use win32k. If it fails to open the driver, then it falls back to conventional means to generate it numbers. This 'PUMonitor' (which I'd never heard of before) almost certainly does the same. Without a device driver (or access to a backdoor into the kernel), the kernel data the util wants to use is simply not accessible to a Ring-3 app.

Rich is spot on here, as was Dave initially as well.

I went back to the code and sure enough, what I missed was a call to check the availability of the win32k stuff (libWin32kInstalled) and when not present the usage of Dos16MemAvail API instead.

Now, having said that, while I can find some on-line references to Dos16MemAvail, I cannot for the life of me find actual documentation for this. Is that some kind of a macro maybe?

Instead I find a lot of references to DosQuerySysInfo, where I could use:
QSV_TOTAVAILMEM - max num of bytes that can be allocated by a process in the system
QSV_MAXPRMEM - max num of bytes that can be allocated by a process in the PRIVATE arena
QSV_MAXSHMEM - max num of bytes that can be allocated by a process in the SHARED arena

So it would seem that perhaps I need to replace that single Dos16MemAvail call with a DosQuerySysInfo call using the three params listed above to get all the info I need?

Rich,
Any chance that's what you're using in your memry util?

Rich Walsh

  • Sr. Member
  • ****
  • Posts: 370
  • Karma: +24/-0
  • ONU! (OS/2 is NOT Unix!)
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #9 on: October 18, 2023, 07:10:41 pm »
running your monitor and comparing it's output to the likes of 'Free Shared Mem' util, or Alex's 'mem', I am not quite seeing the same numbers.

First off, my apologies. A dumb arithmetic error gave you 512mb more high memory than you actually have. The attached 'memry07.zip' fixes the error.

Quote
This is pretty neat stuff. I have been dreaming of building a GUI "browser" that basically visually maps the current memory utilization, goal being: make it human digestable!!!

That's been my goal as well which is why I've spent the last few days working on my "memap" XCenter widget. Below are some screenshots: the widget at its default size (nice and small), the widget expanded for greater visibility, and the widget showing its details window. Since you may want to monitor the details as you open and close stuff, I've made the details window movable so you move it elsewhere to keep it from getting covered. Note that this is a work-in-progress and subject to change (in particular, the details window).

BTW... the scaling for high and low memory differs. For low memory (bottom line) each box represents 64k. For high mem (top line) each box is total_high_mem / 8. Because I have 2gb of high mem, each box is 256mb; if you have 1gb, each box would be 128k.

Edit: added a screenshot of an alternate layout for the Details window. Which one is better?
« Last Edit: October 18, 2023, 08:41:24 pm by Rich Walsh »

Tom

  • Full Member
  • ***
  • Posts: 212
  • Karma: +6/-0
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #10 on: October 18, 2023, 11:54:11 pm »
I went back to the code and sure enough, what I missed was a call to check the availability of the win32k stuff (libWin32kInstalled) and when not present the usage of Dos16MemAvail API instead.

Now, having said that, while I can find some on-line references to Dos16MemAvail, I cannot for the life of me find actual documentation for this. Is that some kind of a macro maybe?

Some background at https://svn.netlabs.org/repos/xworkplace/branches/branch-1-0/001/xwphelp2/xcenter/xc_mem.html , which says:

"Internally, this widget uses the Dos16MemAvail API, an old 16-bit API that is no longer documented by IBM.

The documentation for OS/2 1.3 says that this reports "the size of the largest block of free memory". This is obviously not true any more, since all 32-bit OS/2 versions (since 2.0) no longer allocate memory in blocks, but with 4 KB page granularity instead.

From my experience, this API now does indeed show the amount of free physical memory in the system, that is, the amount of RAM that is currently not used by OS/2, either because it has never been used or swapped out or released again by a process. (From my testing, this API returns the same values as Theseus does for the free physical RAM.) This free RAM can be used by applications immediately without &os2; having to make room by swapping out other memory pages."

More info can be found at http://www.edm2.com/index.php/DosMemAvail which matches the information in the OS/2 1.3 "Control Program Programming Reference".
« Last Edit: October 19, 2023, 12:47:19 am by Tom »

Lars

  • Hero Member
  • *****
  • Posts: 1367
  • Karma: +70/-0
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #11 on: October 27, 2023, 05:00:01 pm »
running your monitor and comparing it's output to the likes of 'Free Shared Mem' util, or Alex's 'mem', I am not quite seeing the same numbers.

First off, my apologies. A dumb arithmetic error gave you 512mb more high memory than you actually have. The attached 'memry07.zip' fixes the error.

Quote
This is pretty neat stuff. I have been dreaming of building a GUI "browser" that basically visually maps the current memory utilization, goal being: make it human digestable!!!

That's been my goal as well which is why I've spent the last few days working on my "memap" XCenter widget. Below are some screenshots: the widget at its default size (nice and small), the widget expanded for greater visibility, and the widget showing its details window. Since you may want to monitor the details as you open and close stuff, I've made the details window movable so you move it elsewhere to keep it from getting covered. Note that this is a work-in-progress and subject to change (in particular, the details window).

BTW... the scaling for high and low memory differs. For low memory (bottom line) each box represents 64k. For high mem (top line) each box is total_high_mem / 8. Because I have 2gb of high mem, each box is 256mb; if you have 1gb, each box would be 128k.

Edit: added a screenshot of an alternate layout for the Details window. Which one is better?

From a graphical point of view I would think this would be more obvious:

Code: [Select]
Private:                  Low             High
   Used:                 xxx Bytes     xxx Bytes
   Unused:               xxx Bytes     xxx Bytes
Shared:
   Unused:               xxx Bytes     xxx Bytes
   Used:                 xxx Bytes     xxx Bytes

because that is effectively how the memory layout looks like.

Rich Walsh

  • Sr. Member
  • ****
  • Posts: 370
  • Karma: +24/-0
  • ONU! (OS/2 is NOT Unix!)
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #12 on: October 28, 2023, 01:52:18 am »
Here are screenshots of the final(?) product.

'mmap-v085.png' shows the widget and its details window which you can toggle on/off by clicking the widget.

 'mmap-v085-props.png' shows the Properties dialog which offers "live" updates (i.e. drop a color, the graph changes immediately). It also demonstrates that you can move the window elsewhere if needed - doing so will display an 'X'  button to make closing it easier.

This will probably show up in the XWP that follows the soon-to-be-released v1.0.16.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1354
  • Karma: +26/-0
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #13 on: November 08, 2023, 03:43:03 am »
Tom, everyone...

...while I can find some on-line references to Dos16MemAvail, I cannot for the life of me find actual documentation for this. Is that some kind of a macro maybe?

Some background at https://svn.netlabs.org/repos/xworkplace/branches/branch-1-0/001/xwphelp2/xcenter/xc_mem.html , which says:

"Internally, this widget uses the Dos16MemAvail API, an old 16-bit API that is no longer documented by IBM.

The documentation for OS/2 1.3 says that this reports "the size of the largest block of free memory". This is obviously not true any more, since all 32-bit OS/2 versions (since 2.0) no longer allocate memory in blocks, but with 4 KB page granularity instead.

From my experience, this API now does indeed show the amount of free physical memory in the system, that is, the amount of RAM that is currently not used by OS/2, either because it has never been used or swapped out or released again by a process. (From my testing, this API returns the same values as Theseus does for the free physical RAM.) This free RAM can be used by applications immediately without &os2; having to make room by swapping out other memory pages."...

Alright...so I've got the makefile fully rebuilt now to use NMAKE32 (just b/c that's what I wanted to use, and I needed a good lesson in makefile functionality), and so off to the "races" I went modifying the code to show the available shared memory.

However...while that seems to work just fine using DosQuerySysInfo and QSV_MAXSHMEM (the results between PUMon and other utils all match), I cannot for the life of me get a matching result when it comes to what Dos16MemAvail API produces.

Here is an example:

Dos16MemAvail shows me 544.8M

...and in the meantime the Theseus snapshot gives:

Quote
17. QSV_TOTPHYSMEM           = -940527616 (3275820K -> 3199.043M).
18. QSV_TOTRESMEM            = 1241985024 (1212876K -> 1184.449M).
19. QSV_TOTAVAILMEM          = 634712064 (619836K -> 605.309M).
20. QSV_MAXPRMEM             = 142082048 (138752K -> 135.500M).
21. QSV_MAXSHMEM             = 74907648 (73152K -> 71.438M).
27. QSV_MAXHPRMEM            = 1409286144 (1376256K -> 1344.000M).
28. QSV_MAXHSHMEM            = 429481984 (419416K -> 409.586M).

...none of which match the 544.8M that Dos16MemAvail shows.

Now you might say "..oh c'mon man, that's an obsolete API, we have better stuff, like DosQuerySysInfo...", sure, yet strangely enough 'Theseus=>Ram Usage by Process' still returns the correct 544.8M result that the old Dos16MemAvail API does.

What da???!!!

It's bad enough just trying to keep up with the logic flow of PUMon (although it does some nice & sophisticated work with the display cell definition logic), but I need some help in deciphering the above difference.

For what it's worth: I do NOT mind tossing in the gloves and settling in on using one of the most appropriate QSV values, but which one???

1) QSV_TOTAVAILMEM - seems like the best overall fit since it accounts for the virtual memory pool room that can be used to serve ALL processes

2) QSV_MAXPRMEM - seems like more appropriate fit since that really defines the most room a single process actually has left in the low private arena

In other words, while #1 is an overall SYSTEM-WIDE picture, it is really #2 that will dictate whether any particular process you wish to run can successfully allocate all the memory it needs (I believe this is ultimately due to memory segmentation that eventually happens).

Heck...I need to make the matching RC changes anyways (so these are all user selectable), so maybe that's a moot point...???

Thanks for all the feedback!

Rich Walsh

  • Sr. Member
  • ****
  • Posts: 370
  • Karma: +24/-0
  • ONU! (OS/2 is NOT Unix!)
    • View Profile
Re: shared memory value - Odin win32k or DosQuerySysInfo???
« Reply #14 on: November 09, 2023, 07:35:58 am »
1) QSV_TOTAVAILMEM

Apples...

Quote
2) QSV_MAXPRMEM

Oranges...

You are conflating two separate concepts: 'physical/virtual memory'  and 'linear address space'. Here's a simplistic analogy.

You have a store with inventory (RAM) and shelving to put it on (address space). Periodically, you'll face one of two problems:

A) you want to add more shelf space but you have nothing to put on it because your entire inventory is already on the other shelves. You could put empty boxes on the shelves (i.e. swap space) to make it look like you have more inventory than you really do but eventually you'll run out of that too. In the real world, you're out of 'physical/virtual memory'.

B) you have plenty of inventory but can't put it out because the shelves where it should go are already full. In the real world, you're out of 'linear address space'.

These days 'A' isn't the problem it used to be - most of us have far more than we can use. And for many of us, we can't even find a use for all the physical RAM we _could_ use, so we never run out and *never swap*. Those of us who know their needs are modest (don't build QT6, don't keep 40-50 browser tabs open, etc.) have no reason to give physical memory a second thought.

In contrast, 'B' (linear address space) will always be an issue because OS/2 will always be limited to 512mb of low/16-bit compatible memory. It isn't as bad as it used to be with the advent of high memory (i.e linear address space whose addresses start at 512mb). Still, it's worth monitoring - which is why I wrote a little widget to do so (shameless plug).

With the possible exception of PUMon when it uses 'Win32K.sys' (which I've never seen), every monitor that attempts to show available private and shared memory suffers from the same problem: GIGO. The values returned for QSV_MAXPRMEM and QSV_MAXSHMEM are misleading and substantially useless because the same stretch of memory is included in both. To illustrate this, I've copied the screen shot of my 'memmap' widget from an earlier posting. Take a look at it.

QSV_MAXPRMEM includes both the areas in red and those in gray. QSV_MAXSHMEM includes the areas in gray and in green. What good is that? I labelled the area in gray as "Free" due to space constraints but it's more correctly labelled "Unassigned". It's available for use as either private or shared memory, whichever claims it first.

When I took the screenshot, Dooble had been open and had claimed 46mb of the unassigned memory for private use (in addition to the 64mb of private memory every app is guaranteed). Even after closing Dooble, that 46mb would remain assigned to "private" until the system was forced to reclaim it. If you were to look at a typical mem monitor, you'd get the mistaken impression that Dooble had allocated but not freed 46mb of shared memory. But that wouldn't be correct because that memory was _never_ shared, it was "unassigned".

FYI... "real" shared memory starts at the lowest address that's actually being used. If that allocation is freed, it goes back into the unassigned pool. Any memory above that which gets freed is stuck behind the dividing line and will remain shared memory - whence the figure for "Shared - unused".

FYI2... unassigned memory claimed for private use, then freed, is not reclaimed until the system has run out of shared and unassigned mem. It will then survey all running apps to see how much private memory they're actually using. If <= 64mb, it will return the excess to the pool, then use what it needs to fill the request for shared mem that triggered the review.

Correction: high mem starts at 512mb, not 1gb as originally written
« Last Edit: November 10, 2023, 04:25:33 am by Rich Walsh »