OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Applications => Topic started by: Dariusz Piatkowski on January 09, 2022, 02:18:56 am

Title: LIBC panic errors anyone?
Post by: Dariusz Piatkowski on January 09, 2022, 02:18:56 am
I am running the following LIBC RPM packages:

1) libc.pentium4                            1:0.1.9-1.oc00
2) libc-devel.pentium4                      1:0.1.9-1.oc00
3) libcurl.pentium4                         7.75.0-2.oc00
4) libcx.pentium4                           0.7.2-1.oc00
5) libcx-devel.pentium4                     0.7.2-1.oc00

...and at some point in time (along the upgrade path) I started to notice the following "LIBC PANIC" errors in my trp directory:

Quote
LIBC PANIC!! _um_free_maybe_lock: Tried to free block twice - block=37013a00 lo
...
Call Stack
______________________________________________________________________
   EBP     Address    Module     Obj:Offset    Nearest Public Symbol
 --------  ---------  --------  -------------  -----------------------
 Trap  ->  1FE4B89E   LIBCN0    0001:000FB89E
 0013EAF8  1FE4C0A8   LIBCN0    0001:000FC0A8
 0013EB18  1FDE4B69   LIBCN0    0001:00094B69
 0013EB68  1FDC4D69   LIBCN0    0001:00074D69
 0013EB88  7B395268   XUL       0001:03565268  between js::gc::Arena::finalize + 228 and js::gc::ArenaLists::backgroundFinalize - 3678  (both in Unified_cpp_js_src23.cpp)
 34734D20  344B0538   *Unknown*
 Lost Stack chain - new EBP below previous
...

This one was actually generated by FF following my upgrade to the latest NSPR and NSS runtimes.

However, I've seen this error repeatedly with other applications, such as:

CRON/2:
Quote
LIBC PANIC!! _um_free_maybe_lock: Tried to free block twice - block=20030035 lo
...
 Call Stack
_____________________________________________________________________
   EBP     Address    Module     Obj:Offset    Nearest Public Symbol
 --------  ---------  --------  -------------  -----------------------
 Trap  ->  1FE4B89E   LIBCN0    0001:000FB89E
 0012FF14  1FE4C0A8   LIBCN0    0001:000FC0A8
 0012FF34  1FDE4B69   LIBCN0    0001:00094B69
 0012FF84  1FDC4D69   LIBCN0    0001:00074D69
 0012FFA4  000100E0   CRON2     0001:000000E0
 0012FFC4  00014238   CRON2     0001:00004238
 0012FFF4  1FFECE38   DOSCALL1  0004:0000CE38  DOS32R3EXITADDR
...

...or PMMail and AOO.

So I'm curious if anyone is seeing this too?

I can't quite peg this to a particular release, but it would have happened within the last couple of drops I think?
Title: Re: LIBC panic errors anyone?
Post by: Dave Yeo on January 09, 2022, 04:05:22 am
Yes, it became very common about the time that the latest NSPR4 and NSS were moved to netlabs-rel along with a bunch of other stuff including libc and libcx. Mostly I see them with SeaMonkey.
Title: Re: LIBC panic errors anyone?
Post by: David McKenna on January 09, 2022, 02:43:33 pm
 Dariusz,

  I also see these frequently with SeaMonkey, but so far not with any other software (I do use PMMail and AOO).

Regards,
Title: Re: LIBC panic errors anyone?
Post by: Jochen Schäfer on January 10, 2022, 09:14:26 am
I noticed that problem while using lsusb.
Title: Re: LIBC panic errors anyone?
Post by: Lars on January 10, 2022, 10:12:22 am
Yes, I have also seen this trap report:

Don't shoot the messenger. It looks like there is an actual improvement in libc to properly report if you call "free" on the very same pointer more than once (which is forbidden).

Consequently, that likely means that the affected applications actually have a problem managing their dynamic memory. These problems can go unnoticed if for example a memory page is not directly assigned to another allocation and therefore remains accessible for some time. That's why it is so important to report this error.

About lsusb: if you see them in LIBUSB, then that's the reason why I am still working on it :-)

Cheers,
Lars
Title: Re: LIBC panic errors anyone?
Post by: Dariusz Piatkowski on January 10, 2022, 04:40:55 pm
Lars,

...Don't shoot the messenger. It looks like there is an actual improvement in libc to properly report if you call "free" on the very same pointer more than once (which is forbidden)...

Thanks for that input...I started to review each trap report closer to see what might be the root cause. I have consistently seen what appear to be memory calls.

The good news is that CRON/2, which is the one most often showing these, had been released with source. That means I can attempt to debug (and probably learn a pile more while doing so).

Thank you, insight greatly appreciated!
Title: Re: LIBC panic errors anyone?
Post by: Dave Yeo on January 10, 2022, 07:00:36 pm
Yes, often the Mozilla trps seem to originate in the JavaScript engines garbage collection routines, I thought during allocation. Doing garbage collection in C++ is likely complicated. The last one here seems to have originated from doing a malloc,
Code: [Select]
0013F818  BB4F5160   XUL       0001:03315160  between _moz_xmalloc + 10 and _moz_xcalloc - 20  (both in Unified_cpp_memory_mozalloc0.cpp)
When getting Mozilla to compile and run with GCC 9.9.2 (actually GCC 6+), once it was compiled, with any optimization, it would crash, usually before finishing drawing the main window. Turned out it was the compiler helpfully optimizing away the memset() statements as they happened before constructors, something I gather is legal but bad practice. Luckily a cross-platform problem. The GCC folks suggested these CXXFLAGS, -flifetime-dse=1 -fno-delete-null-pointer-checks.
Leaves me wondering if other stuff is getting optimized away or even compiler bugs, which do often show up when optimizing.
Title: Re: LIBC panic errors anyone?
Post by: Dariusz Piatkowski on January 11, 2022, 09:29:41 pm
Well you guys...I really honestly wanted this to work...but man, a trap every 3 hrs. is just a little too much for me! (FF is what I'm talking about here)

So I'm trying different things now, I downgraded to LIBICU libicu-65.1-2.oc00.pentium4, which now of course kills the new AOO 4.1.11.

The work around for that is to run the 68 version of LIBICU for AOO (so separate set of DLLs) which I think spells trouble!

Does anyone know of a good AOO work-around for the latest LIBICU challenges?

Title: Re: LIBC panic errors anyone?
Post by: Dave Yeo on January 11, 2022, 11:12:25 pm
For Firefox, you might want to try downgrading NSS and NSPR4. Could probably put the DLLs in Firefox's program directory too, untested.
Title: Re: LIBC panic errors anyone?
Post by: Dariusz Piatkowski on January 12, 2022, 03:16:40 am
Dave, everyone...

For Firefox, you might want to try downgrading NSS and NSPR4. Could probably put the DLLs in Firefox's program directory too, untested.

Yup, that's the current "last straw" I'm on LOL.

I did downgrade to LIBICU libicu-65.1-2.oc00.pentium4 first, tried that, but crashes continued, at which point in time I re-installed the 68 release and moved the NSS stuff back to the previous level (3.23).

Since I'm doing all this anyways I thought I'd leave NSPR at 4.23 (per previous upgrade from 4.12) just to see if that is a contributing to the end results I'm seeing in any way.

More to follow as time progresses...thanks for the feedback everyone!
Title: Re: LIBC panic errors anyone?
Post by: Dave Yeo on January 12, 2022, 03:53:51 am
NSPR4 and NSS are usually a matched pair. At that to build standalone NSS, you use the NSPR4 build system.
Title: Re: LIBC panic errors anyone?
Post by: Dariusz Piatkowski on January 12, 2022, 04:06:21 am
NSPR4 and NSS are usually a matched pair. At that to build standalone NSS, you use the NSPR4 build system.

Oh Dave, I found that out pretty quickly...literally a HARD lock.

Yup, back to the previous versions of NSPR and NSS, and unfortunately had a single crash already...so that implies it's back to LIB's corner.

Man, what a shame though...looked like the updated AOO addressed a few issues too.
Title: Re: LIBC panic errors anyone?
Post by: Neil Waldhauer on January 12, 2022, 03:45:23 pm
I'm seeing no traps on Open Office 4.1.11 and Firefox 45. I was lazy and never upgraded my supporting libraries.

Libc 0.7.0-1
Libcx 0.7.0-1

nss 3.23.0-4
nspr 4.12.0-4
Title: Re: LIBC panic errors anyone?
Post by: Rene Hvidsoe on January 12, 2022, 04:38:23 pm

Hi,

I have no issues with traps with Libc.

Have Libc 0.1.9-1 i686 and Libcx 0.7.2-1 i686, nss 3.47.0-2 i686 and nspr 4.23.0-1 i686 installed.
Title: Re: LIBC panic errors anyone?
Post by: Dariusz Piatkowski on January 17, 2022, 03:40:18 pm
Quick update on what I found to be the best combo to assure FF stability:

1) libc.pentium4                            1:0.1.9-1.oc00
2) libcurl.pentium4                         7.75.0-2.oc00
3) libcx.pentium4                           0.7.2-1.oc00
4) libicu.pentium4                          56.1-2.oc00

5) nspr.pentium4                           4.12.0-4.oc00
6) nss                                           3.23.0-4.oc00
7) nss-util                                     3.23.0-4.oc00
8) nss-softokn                               3.23.0-4.oc00
9) nss-softokn-feebl                       3.23.0-4.oc00

This gets me back to rock-solid config for running FF, which means an occasional crash (rare) but a normal application re-start being required once FF gets over 1G of memory use.

The caveat here is that I went back to AOO 4.1.7 due to the libicu 68 and 65 issues as they apparently kept on impacting my stable FF setup (so the combo of NSPR and NSS packaged).
Title: Re: LIBC panic errors anyone?
Post by: Jochen Schäfer on January 29, 2022, 08:22:30 am
I switched libcx from the i686 to the pentium4 version, and the crashes on exit stopped.
This seems very strange to me, since running on an i5, this should make no difference.
Title: Re: LIBC panic errors anyone?
Post by: Lars on January 29, 2022, 12:54:10 pm
Maybe they are simply not equivalent. What I read somewhere is that BWW recommends to switch over to "Pentium4" architecture in general (even updating already installed packages if possible).
And as much as I can tell, they stopped producing "i686" versions of their RPMs for everything that is new.
Title: Re: LIBC panic errors anyone?
Post by: Roderick Klein on January 29, 2022, 03:58:08 pm
Maybe they are simply not equivalent. What I read somewhere is that BWW recommends to switch over to "Pentium4" architecture in general (even updating already installed packages if possible).
And as much as I can tell, they stopped producing "i686" versions of their RPMs for everything that is new.

That is correct.

Roderick
Title: Re: LIBC panic errors anyone?
Post by: Jochen Schäfer on January 30, 2022, 08:45:48 pm
Roderick, so there is a difference in functionality between LIBCX version apart from the emitted machine code?
Title: Re: LIBC panic errors anyone?
Post by: Lars on January 31, 2022, 08:01:24 am
I am not Roderick, but...

there is no intended functional difference (at least, that's not what I meant). But the optimization might not work correctly when you build for i686 but works ok when you build for pentium4. Or they fixed something in the compiler in the meantime. Or some such.
Title: Re: LIBC panic errors anyone?
Post by: Dave Yeo on January 31, 2022, 08:41:55 am
I rebuilt SeaMonkey with the intree NSPR4 and NSS, versions,
Code: [Select]
Expected minimum version Version in use
NSPR 4.12 4.12
NSS 3.21.4 Basic ECC 3.21.4 Basic ECC
NSSSMIME 3.21.4 Basic ECC 3.21.4 Basic ECC
NSSSSL 3.21.4 Basic ECC 3.21.4 Basic ECC
NSSUTIL 3.21.4 3.21.4

and SeaMonkey has been stable for 10 days, including leaving it running for 4 or 5 days steady. Today it finally crashed at the INT3 for out of memory conditions.
I still think there is something wrong. probably with NSPR4, which is basically a libc, could be NSS though. The difference the updated libcx and libc made was instead of vanishing without a trace, now there's those libc panics.

As for differences with the i686 and Pentium 4 builds of libc and libcx, it should mostly be instruction ordering unless they've changed the optimizations. I see QT5 uses various SSE[2] optimizations and likely has different code paths for SSE[2] or might have removed the non-SSE[2] code. It also uses -mfpmath=sse, which caused a subtle problem with Mozilla, without the screensaver installed, nsIdleServices took a different code path and after 5-10 minutes would have 100% on one core.
Generally, the more optimizations, the more chance of compiler bugs/features breaking things. Updating to GCC 9.2.0 caused memset()'s to get optimized away as they were before the constructors and I guess the compiler thought they weren't used. -flifetime-dse=1 fixed that