PMMail and EARLYMEMINIT=TRUE problem?

[Dariusz Piatkowski]

So I figured I would try a few different options in order to improve the configuration of my system, but as I did I came up with this surprising (to me anyways) discovery:

Setting EARLYMEMINIT=TRUE in CONFIG.SYS causes my PMMail to crash the whole machine...not a hard crash, but CTRL-ALT-DEL is the only way out.

This is applicable to PMMail 3.22.00.2301 (32534).

Has anyone encountered this before? Any ideas what may be causing the symptom I'm seeing?

[Doug Bissett]

Has anyone encountered this before? Any ideas what may be causing the symptom I'm seeing?

I am amazed that you can run your machine without EARLYMEMINIT=TRUE, and I have never seen it cause a problem, when set. PMMail is probably just one of the programs that might have a problem. The most likely cause is running out of resources (most likely, lower shared memory space). If you haven't done it, yet, change VIRTUALADDRESSLIMIT to 2560. 3072 is max, but that often causes a different problem. Also, be sure that those DLLs, that can be marked to use high shared memory space, are marked to do so (Mozilla, OpenOffice). PMMail uses high shared memory space, but it also uses lower shared memory space. If lower shared memory space is almost full, the system may not be able to load something, and it will sit there waiting for the memory to become available. If it does not become available, you get what you describe. More often, something will crash, because a lot of programs simply assume hat they will get a memory allocation, and then try to use it, even when they don't get it.

EARLYMEMINIT is only used at boot time, to allow drivers etc. to access more memory, so it really shouldn't affect anything after the system is booted, unless you are right on the edge of running out of memory space, and it uses enough to push it over the cliff.

[Ian B Manners]

If memory serves..

The only time you should not have EARLYMEMINIT=TRUE in your config.sys file is if you are using a motherboard with ISA slots, possibly that includes laptops with PCMCIA 16bit cards as well. Otherwise it is as Doug states. I also use PMMail v3.11.47.2052

I have no problem with that statement along with 'VIRTUALADDRESSLIMIT=2560'.
I do turn my computer off every night but with Dave's build of Firefox left open all day along with PMMail, my PC gets  a lot of work using Open Office and PMView for university assignments so I like to start fresh every morning.

[Andreas Schnellbacher]

BTW: I have here a 14 years old mainboard Asus A7V880, that won't work with EARLYMEMINIT=TRUE. I've just checked the dmidecode output that it has no ISA connectors.

That's the only one, I know that has such problems. So in general, the ISA theory sounds reasonable.

[Sigurd Fastenrath]

If memory serves..

The only time you should not have EARLYMEMINIT=TRUE in your config.sys file is if you are using a motherboard with ISA slots, possibly that includes laptops with PCMCIA 16bit cards as well.

And if you are using HPFS386.

[Dariusz Piatkowski]

Sigurd's right on the money here...I used to run HPFS386 on all my drives, switched to JFS about a year ago (or so). With the HPFS386 cache sizes, even small ones, I was never able to use that setup.

Today...well, PMMail was the only app I noted failed consistently. There may be other ones as well, but given how important email is I never actually got beyond that testing criteria.

Here are some of the CONFIG.SYS entries:

Code: [Select]

...
VIRTUALADDRESSLIMIT=2560
REM EARLYMEMINIT=TRUE
REM DLLBASING=OFF
...

My JFS cache is set to 128Meg, and otherwise I do not think there is anything unusual about my setups.

Plenty of free shared memory (as reported by the 'Free Shared Mem' utility) over 150Meg.

Is there anything I can do to capture the crash I'm seeing? So far nothing in the LOG directory, nor in popuplog.os2.

[Doug Bissett]

Take out both REMs, and you probably won't have any more problems, for a while. REM DLLBASING=OFF causes shared memory to allocate on memory boundaries, which causes a lot of fragmentation. You may have 150 M left, but if the biggest chunk is 20 K, you have a major problem.

Older systems would be slowed by using DLLBASING=OFF, but any system after the 486SX won't have any problems with it.

Recently, I have abandoned using 128 M for JFS cache too (no setting, which means 64 M), but the jury is still out on that one. I am inclined to think that it is okay at 128 M (at least it doesn't use lower shared memory, like HPFS does).

Plenty of free shared memory (as reported by the 'Free Shared Mem' utility) over 150Meg.

Don't be fooled by that. I have seen lower shared memory drop from 180 M to about 5K, in seconds (another 6K, and the system would probably crash, or hang). The biggest "problem", that I have now, is trying to run VBox under OS/2, It uses huge amounts of lower shared memory, and I need to watch what I try to run at the same time.

Today, I also discovered that doing an update, with ANPM (RPM/YUM), can cause DLLs to lose their settings to load high. This can happen with DLLs that are used by Apache OpenOffice, and with Firefox (probably more). It is probably a good idea to use the HIGHMEM utility to redo those settings after each update. I use the AOO quick load, to keep those in memory, and set them to load both data and code, high. Firefox is probably better with only code loaded high, and use MOZTURBO to keep them loaded. Keeping the DLLs loaded helps to prevent shared memory fragmentation (as well as crashes, when they try to unload), which can cause problems, even when the system says it has lots of shared memory available.

[Neil Waldhauer]

If no POPUPLOG.OS2 entry is created, then are any .TRP files created? If so, these could contain a clue to what is going wrong.

[Dariusz Piatkowski]

Doug, everyone...

Take out both REMs, and you probably won't have any more problems, for a while. REM DLLBASING=OFF causes shared memory to allocate on memory boundaries, which causes a lot of fragmentation. You may have 150 M left, but if the biggest chunk is 20 K, you have a major problem.

Older systems would be slowed by using DLLBASING=OFF, but any system after the 486SX won't have any problems with it...

Yup, that did it.

So both REMs got removed, which means I am now running with the following config:

Code: [Select]

...
VIRTUALADDRESSLIMIT=2560
EARLYMEMINIT=TRUE
DLLBASING=OFF
...

PMMail comes up fine, re-started multiple times, no issues found. So far the other major apps are chugging away as well.

Great pointer Doug, appreciate the insight.

Now off I go stripping out some hardware (like the SCSI AdvanSys card that's my scanner connector) to see if that will have any impact on me trying to actually run in the full 6-core CPU configuration.

[xynixme]

Today, I also discovered that doing an update, with ANPM (RPM/YUM), can cause DLLs to lose their settings to load high. ... It is probably a good idea to use the HIGHMEM utility to redo those settings after each update.

Yes, and of course the distribution method nor utility matters. With the ZIP files and without a @unixroot you'd probably be more aware of the issue of lost customized settings, because then there's nothing which hides the "complicated installation stuff" for you. I did see some firefox-highmem-45_9_0-1_oc00 file, but I couldn't care less which marked DLLs this distribution includes. Only the FF ones, like XUL.DLL, or all.

The under-(re)construction-installer for MozTurbo won't check DLLs to notice a changed setting, but it can be used to mark all qualifying DLLs again. Per relevant product (FF/SM/TB). Or use e.g. an own CMD file with ABOVE512.EXE or HIGHMEM.EXE indeed, to cover all relevant products with one CMD file.

[Dariusz Piatkowski]

Hi Neil,

If no POPUPLOG.OS2 entry is created, then are any .TRP files created? If so, these could contain a clue to what is going wrong.

Nah, nuthin'...that's the funny (although maybe quite reasonable outcome given the possible scenario that Doug outlined). The app simply hung the system, in that soft-hang, which is why CTL-ALT-DEL still worked. But no TRP, no LOG, nothing else.

Well, now that this is behind me though, no sense spending any more time of this. Sometimes I honestly just thank the Almighty that things still keep on working...LOL!

[Doug Bissett]

Well, now that this is behind me though, no sense spending any more time of this. Sometimes I honestly just thank the Almighty that things still keep on working...LOL!

It will work, until they create enough DLLs to fill it up again. When that happens (when, not if), we have no other place to go, unless somebody knows about something else that will reduce the amount of lower shared memory that they require. One of the main problems, is that older programs need to be updated to use upper shared memory space, but, in many cases, we don't even have the source, so it isn't possible. Those who are porting new programs, or if they have the source for an older program, need to take steps to reduce the need for lower shared memory space. At least that will buy us some more time.

I will note, that VBox (running under OS/2) is using HUGE amounts of lower shared memory space. On the other side, CLAMD is automatically using HUGE amounts of upper shared memory space. Depending on the VIRTUALADDRESSLIMIT settings, there is a lot more upper shared memory space available, but it is not unlimited. There is now memory available in PAE memory. See the QSINIT documentation for more information. The "problem" is, that there is no easy way to actually use that memory in a program. So far, the only user, that I know about, is the PAE RAMDISK (see QSINIT). It seems to me, that CLAMD could use that memory, rather than filling up upper shared memory (but I don't have to do it).

[Lars]

I had a look at the QSINIT SDK code: there is a "mode" to start the HD4DISK.ADD with the /i parameter. In that case, no disk will show up but instead, the memory allocated for the RAM disk will be accessible as ordinary memory via IOCTL calls (you can read from and write to it).
However that will only allow one app to use that memory at a time, it can only read/write in blocks of 512 bytes and of course you will have no RAM disk. I guess that is not a good solution to the problem.

[Roderick Klein]

There is now memory available in PAE memory. See the QSINIT documentation for more information. The "problem" is, that there is no easy way to actually use that memory in a program. So far, the only user, that I know about, is the PAE RAMDISK (see QSINIT). It seems to me, that CLAMD could use that memory, rather than filling up upper shared memory (but I don't have to do it).

As it is right now OS/2 will *never* be able to use the memory above the 4 GB RAM for a program to allocate RAM.
And updating the kernel is very very time and labour intensive task. It would require massive patching to make this possible. You would need to change the memory routines that allocate memory to make this work.

The API in the QSINIT loader has no memory management. What I mean with this is that if you use that API the loader offers you can overwrite the memory the ram disc is using.

That said I did talk to Dmitry from BWW at Warpstock Europe in Cologone in 2016. And they do have an idea how OS/2 can use the memory the loader offers above the 4 GB boundary, without changing the kernel.
If hope I explain this correctly but OS/2 would kind of get the same idea you as you had with expanded memory.
You would get a memory buffer below the 4 GB of RAM and that would be used to push data to and from above the 4GB.

Note expanded memory on DOS does have somewhat of a negative image as it was not always easy to setup.
This idea would be transparant for the user. Todo would:

Something that would need manage the memory. As in keep track of which memory is used by who. If an application crashes or shutdown that the memory is freed! To get this to work this would require also an application be modified to use this mechanisme.  This idea would especially be usefull for programs requiring large amount of memory and especially usefull for LIBC applications (large ported applications).

Roderick

[Roderick Klein]

So I figured I would try a few different options in order to improve the configuration of my system, but as I did I came up with this surprising (to me anyways) discovery:

Setting EARLYMEMINIT=TRUE in CONFIG.SYS causes my PMMail to crash the whole machine...not a hard crash, but CTRL-ALT-DEL is the only way out.

This is applicable to PMMail 3.22.00.2301 (32534).

Has anyone encountered this before? Any ideas what may be causing the symptom I'm seeing?

While I read what your writing I have a hard time believing earlymeminit=true is the cause of this.
Unless you are using HPFS386 then earlymeminit=true is nono.
The usage of earlymeminit=true is simply just about  a requirement for OS/2 to even function.
Certain older *ISA* SCSI card do not work and HPFS386 file system. Other then that no documented defects
have been found.

eCS has been using this setting since just about day 1 and ArcaOS as well. The only thing it does is that it makes no longer limits the amount of memory OS/2 can use for loading device drivers. Without its 16 MB and in that space ACPI.PSD, the kernel IFS drivers etc all need to be loaded! If you go above the 16 MB without the system your system will crash somewhere during startup. Why because the kernel ran out of memory.

I have yet to year somewhat of a theory as to why earlymeminit=true would cause applications to crash.
As in drivers allocate RAM and the kernel knows which memory area's are allocated by drivers. That RAM can not be assigned to applications..

Roderick