It has been my experience, that AOO 4.1.7 works just fine, BUT, I do need to use the Quick Start feature, AND use Extras-> Enable High Mem code&data loading (after every update, to be sure that nothing got changed). I also need to raise the VIRTUALADDRESSLIMIT setting. 3072 is the max allowable, and 1536 is the default in ArcaOS or eCS (I think the OS/2 default is 512, or 1024, depending on what version you are using). I find that 1536 is not enough (ONE program, at a time, is usually okay, until shared memory fragments - see below), and some machines will not work with 3072, so I always use 2560 (so I don't need to figure out if the machine can use 3072). Depending on what else you use, 2560 may not be enough. Even if you are using the patched ArcaOS kernel, this is still likely to cause problems, when the program unloads. Other programs, like Firefox, can also eat large amounts of shared memory, and they work much better when set to use high shared memory (see other threads about this). The main thing, is to use upper shared memory, whenever possible, because there just isn't enough room in lower shared memory to run modern software, and modern software apparently doesn't always check to see if it got a memory allocation, before it tries to use it (very BAD programming, usually from the original, before porting).
Most of the time, when I have a problem like the one described, it is because LIBC cannot allocate shared memory, and that usually happens because there isn't enough shared memory left. One problem is that you can have 100,000,000 bytes left, and some program decides it needs 100,000,001 bytes. You are dead in the water. Another problem is that the free shared memory can fragment, and if the largest free space is 50,000,000 bytes, and a program wants 50,000,001, you are dead in the water, even if there are 20 other free spots with enough free shared memory to make up the difference. Preventing fragmentation is one of the main reasons to use a program, like AOO Quick Start, or Firefox Turbo (to keep it from unloading common DLLs, which will cause fragmentation of shared memory space).
Other things to seriously consider:
NEVER use the exp repository stuff, UNLESS you know exactly what you are doing, OR you are prepared to trouble shoot problems like you describe (certainly NEVER do a mass update from the exp repositories, unless you are really looking for trouble). Unfortunately, not enough people test what is in the exp repositories, so bad stuff gets moved to the release repositories, sometimes. If you want to test that stuff, use an alternate system, so you can still run your real system (and don't share the %UNIXROOT% when you do that). Sharing %UNIXROOT% between boot systems is possible, but you had better fully understand what that can do to you. Some things will be missed, on the "other" system, when updated, and you do need to run a Re-install of certain packages, on the "other" boot system to pick them up. Easier, and safer, to simply use a separate %UNIXROOT% (after all, disk space, and internet time, is rarely a problem any more).
ALWAYS use ANPM (RPM/YUM) to update the UNIX support. If you don't it is very easy to miss something (it can miss things, even if you do use it - one of the things that gets missed in testing). You could spend months trying to figure out that BC.EXE (for example) is at the wrong level. RPM/YUM is not just DLLs, it is PACKAGES. ANPM makes RPM/YUM usable for us mortals.
Inspect your system for stray duplicate DLLs (and other things). There should be exactly ONE copy of individual files in the Libpath (and PATH, but that isn't as critical), on any boot system, or you should know, exactly, why there is a copy somewhere other than in %UNIXROOT%, and you should understand exactly when that will be used, and what it will do to you. Also remember that you used the usr\local directory structure (if you do use it, which s NOT recommended, unless you know exactly what you are doing, and why), and manually update that, when you update the RPM/YUM stuff. Apparently, you need the full usr\local\* directory structure, or some things will get all upset. (If you don't know about that, DON'T use it).
Don't use LIBPATHSTRICT, unless you have no other option (and be sure that the program doesn't already do it for you - like later versions of Firefox). That causes even more shared memory to be used, and should not be required, unless you know exactly why it is needed.