@Boris - Theseus during boot before it traps?
Whenever you have troubles with memory.
OTOH it just comes to my mind that I can not increase Virtualaddresslimit above 15xx as bootup stops with AFAIR 'out of memory' too. Another mystery maybe not related.
IFSes reside their cache in the system memory arena, i. e. above VIRTUALADDRESSLIMIT.
If you fail to boot if you increase VAL value, then, apparently, something fails to allocate enough memory in the system arena, cuz you've made it too small by increasing the user arena, and causes your system to stop operating.
JFS also caches a lot, not HPFS386 only.
After booting with HPFS386 I've plenty of memory free. But latest AOO eats a lot and only partially frees them when closing with dlls are loaded high --> no more than 5 to 15 open/close cycles possible (depending on other running software).
It doesn't seem to me that HPFS386 and your AOO4 memory issues are directly related. If AOO4 leaks memory with DLLs loaded high, first, check you marked the DLLs'
code section to load high, not the data one. If it still leaks, I'd file a bug if I were you, cuz it shouldn't.
The only way IMO how HPFS386 could be related is your VAL. If you were able to increase it, you'd have more open/close cycles available.
