OS/2, eCS & ArcaOS - Technical > Storage

PMSHELL

<< < (2/4) > >>

Remy:

--- Quote from: Doug Bissett on July 29, 2018, 10:54:07 pm ---I wouldn't call it a "storage" issue. It is a shortage of lower shared memory space. I find that it is only a matter of a few minutes, until my system crashes, when lower shared memory drops below 100 MB (it is probably already dead if it shows 10 MB). To avoid the problem, I add the line"

--- Code: ---VIRTUALADDRESSLIMIT=2560
--- End code ---
to CONFIG.SYS. Some systems will be able to use 3072, but, apparently, some other systems can't use that. It only increases the available upper shared memory anyway.

Doing that does not actually change the problem, without taking further steps to mark some things to actually use upper shared memory space (there are some fixes in the ArcaOS kernel for upper shared memory support too). The easiest thing to set to use upper shared memory space, is Apache Open Office. In the Extras folder, there is a tool to set it to use upper shared memory (some experimenting may be required). If you do that, you may find that there will be a LIBC problem (usually not obvious), if you close Open Office more than once. To avoid that, use the Quickstart thing, to prevent it from actually closing.

You should also realize, that shared memory may fragment. If the largest fragment is 5 MB, and some program needs 5.5 MB, you are dead in the water. If the program is properly written, to check if it actually got the requested memory, it will probably just crash the program. If it doesn't check (apparently, most programs don't), and tries to use it anyway, it usually causes what you describe.

--- End quote ---

It is already set to VIRTUALADDRESSLIMIT=2560 but this is usefull for high memory.

Remy:

--- Quote from: Roderick Klein on July 29, 2018, 11:32:39 pm ---
--- Quote from: Doug Bissett on July 29, 2018, 10:54:07 pm ---I wouldn't call it a "storage" issue. It is a shortage of lower shared memory space. I find that it is only a matter of a few minutes, until my system crashes, when lower shared memory drops below 100 MB (it is probably already dead if it shows 10 MB). To avoid the problem, I add the line"

--- Code: ---VIRTUALADDRESSLIMIT=2560
--- End code ---
to CONFIG.SYS. Some systems will be able to use 3072, but, apparently, some other systems can't use that. It only increases the available upper shared memory anyway.

Doing that does not actually change the problem, without taking further steps to mark some things to actually use upper shared memory space (there are some fixes in the ArcaOS kernel for upper shared memory support too). The easiest thing to set to use upper shared memory space, is Apache Open Office. In the Extras folder, there is a tool to set it to use upper shared memory (some experimenting may be required). If you do that, you may find that there will be a LIBC problem (usually not obvious), if you close Open Office more than once. To avoid that, use the Quickstart thing, to prevent it from actually closing.

You should also realize, that shared memory may fragment. If the largest fragment is 5 MB, and some program needs 5.5 MB, you are dead in the water. If the program is properly written, to check if it actually got the requested memory, it will probably just crash the program. If it doesn't check (apparently, most programs don't), and tries to use it anyway, it usually causes what you describe.

--- End quote ---

If I did not understand you correctly Doug. You typed:
"without taking further steps to mark some things to actually use upper shared memory space".
However GCC applications already store data (such as browser cache) into the high area.

For Remy:

The real killer is the size of the large DLL's that get loaded this can fixed by using a tool to mark the code segments to load high. Do this for Firefox DLL's, THunderbird AND Open Office.  Do not mark the datasegments high of the DLL.
Also make a backup of your DLL's before doing this.  There is a tool from Yuri called high512 that can mark the DLL's high.

You are most likely not the first and the last OS/2 user to expereince these type of lockups. It might be usefull to report this in a ticket to AN ?

Roderick

--- End quote ---

Arent't there set to load high by default?
How to know all dll's which aren't set to be loaded into high storage area?
Thanks

For now, I'm unable to boot ArcaOS after having got a TRAP0003 and lost of many JFS link entries... (made unrecoverable by JFS)
Trying to recover my system

I think it is no more possible to recover...
264 file(s) where scratched during the trap and a lot of uncoverable files... (including, exe, dll and drivers)  :( :o

Dave Yeo:
Hi Rémy. About high memory support in OS/2.
Understand by default OS/2 ver 2-4 only supported using the 1st GB of address space due to needing to be compatible with OS/2 ver 1.x along with the 16 bit code still in OS/2.
While IBM added support for high memory in Warp Server and then the desktop at ver 4.5 (fp#13), it was never finished. The kernel has bugs with it that have been slowly patched in the most recent kernels so it is best to not mark DLLs to load high before shipping them but rather leave it to the user to mark them.
Only some DLLs can be marked high and continue to function, do not mark system DLLs and mileage can vary depending on your setup. For Mozilla, I mostly recommend marking xul.dll to load code high though most all the DLLs can be marked. It is possible to use Theseus to see DLLs that are loaded high and I guess you could scan DLLs to see if the load high bit has been set.
About VIRTUALADDRESSLIMIT. while the CPU can access 4GBs of address, generally the top of memory is given over to hardware, PCI devices, video memory and such are mapped into the top of memory. How much is used is going to vary depending on your hardware, though generally about half a GB. Once again IBM never finished support so there i some experimentation involved in deciding the value to use. I've had hardware that worked fine with the maximum setting of 3072, this current hardware, I just reduced it to 2560, which fixed a few weird issues I was having and made the system more stable, previously it was locking up regularly. Once again need to experiment, best if the numbers are even when expressed in HEX, B00H, C00H, A00H etc.

I'm sure I'm leaving stuff out and this is just my understanding.

Doug Bissett:

--- Quote ---However GCC applications already store data (such as browser cache) into the high area.
--- End quote ---

Only when they are programmed to do so. It looks, to me, like very few actually do that by default. Of course, you do need the ArcaOS kernel patches to be able to use it successfully, so it would be silly for programs to automatically use high shared memory since not all users have upgraded.


--- Quote from: David McKenna on July 30, 2018, 01:19:22 pm --- I wonder if it would be possible in theory to create a 'memory defragmentation' daemon that monitors memory and warns when fragments get below a specified size (or even better, defrags when needed)?
--- End quote ---

Of course, it is "possible". The problem would be to find somebody who can do it. I doubt if anybody knows enough about how it works, to be able to do anything about it.


--- Quote ---but what is the value when you should be warned about to 'low' memory?
--- End quote ---

I use Above512 with a REXX script, that displays, and logs, the results every second. I have seen lower shared memory completely disappear from 100 MB (crash, before it logged 0 MB), and I have seen it run successfully with 1 MB left. If there was some way to intercept the request, and be sure that it would be successful, before returning to the program, a warning could be issued, and the user could be allowed to close other programs, then tell the interceptor to check again, to see if it would be successful, and return to the program if it is okay (the programs should do that themselves, but it seems that most do not, and that may not even be the programmers fault).


--- Quote ---Arent't there set to load high by default?
--- End quote ---

Not from what I have seen (but it has been a while since I looked). You can't do that, without having a separate set of DLLs for those systems than can use upper shared memory, and for those that can't use upper shared memory.


--- Quote ---How to know all dll's which aren't set to be loaded into high storage area?
--- End quote ---

HIGHMEM.EXE (it is included in Open Office) will tell you. It can also set DLLs to load data and/or code high, or reset them to load low. Be warned that simply setting all DLLs to load high will probably not work well, and some programs may work with individual DLLs loaded high, while others won't. I find that it is best to leave the common DLLs (in RPM/YUM) alone, and only play with the program specific DLLs (such as XUL.DLL in mozilla).

Some experimenting is required, because each user has a different set of programs that they use, in different ways.

Dave Yeo:

--- Quote from: Doug Bissett on July 30, 2018, 07:23:47 pm ---
--- Quote ---However GCC applications already store data (such as browser cache) into the high area.
--- End quote ---

Only when they are programmed to do so. It looks, to me, like very few actually do that by default. Of course, you do need the ArcaOS kernel patches to be able to use it successfully, so it would be silly for programs to automatically use high shared memory since not all users have upgraded.

--- End quote ---

You're confusing loading the DLL's high with allocating memory in the high arena, which seems to work with older kernels. We've been doing this with the Mozilla apps for years, both with the OS/2 API and by building with -Zmem-high. You are right that the builder does need to set the -Zmem-high (and include os2safe.h before os2.h) flag to use high memory. I think the latest libcx might automatically try to use high memory now.

--- Quote ---
--- Quote from: David McKenna on July 30, 2018, 01:19:22 pm --- I wonder if it would be possible in theory to create a 'memory defragmentation' daemon that monitors memory and warns when fragments get below a specified size (or even better, defrags when needed)?
--- End quote ---

Of course, it is "possible". The problem would be to find somebody who can do it. I doubt if anybody knows enough about how it works, to be able to do anything about it.

--- End quote ---

Actually I wouldn't be so sure its possible, even with all the kernel etc sources. It's just the way C works and defragging memory is hard with C. Other languages such as JAVA I believe can do it in their own space. Even when implemented, there is overhead. The old Apple II used to grind to a halt now and again as the Basic interpreter did garbage collection, though eventually it was optimized to be much better.

Navigation

[0] Message Index

[#] Next page

[*] Previous page

Go to full version