OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Setup & Installation => Topic started by: Ben on November 12, 2019, 03:58:51 pm

Title: Trying to solve server crash
Post by: Ben on November 12, 2019, 03:58:51 pm
My server has recently been having stability issues. It locks solid and leaves little indication as to what is going on.
In trying to track the problem down I noticed the following error repeated in my POPUP.LOG:

Code: [Select]
11-11-2019  23:23:20  SYS3175  PID 0012  TID 0001  Slot 0016
C:\OS2\SYSTEM\LOGDAEM.EXE
c0000005
15b112c9
P1=00000001  P2=001a2000  P3=XXXXXXXX  P4=XXXXXXXX
EAX=0005574c  EBX=ffffffe8  ECX=3ffff801  EDX=00242058
ESI=001a2000  EDI=0024403c
DS=0053  DSACC=d0f3  DSLIM=5fffffff
ES=0053  ESACC=d0f3  ESLIM=5fffffff
FS=150b  FSACC=00f3  FSLIM=00000030
GS=0000  GSACC=****  GSLIM=********
CS:EIP=005b:15b112c9  CSACC=d0df  CSLIM=5fffffff
SS:ESP=0053:00055554  SSACC=d0f3  SSLIM=5fffffff
EBP=00242058  FLG=00010206

ERRLOG.DLL 0001:000012c9

It has been a while since I looked at this stuff; any ideas?
Title: Re: Trying to solve server crash
Post by: ivan on November 13, 2019, 04:59:50 pm
Have you tested the server power supply voltages - if they fluctuate more than a few % the PS could need replacing.

The other thing is to run the full memtest tests from a bootable CD because bad memory is the biggest cause of instability and crashes.
Title: Re: Trying to solve server crash
Post by: Pete on November 13, 2019, 07:53:36 pm
Hi Ben

Unlike ivan I find the main cause of instability to be power supply or cooling related - I have yet to have a system fall over due to bad ram.

I suggest a visual examination of power supply and cooling system, clean or replace as necessary.

Also worth unplugging all leads inside the case - 1 at a time - and reinserting which will make sure connections are clean.

If the problem still persists then it is time to start checking ram and other bits of hardware...


All the above presumes there were no changes in software just before the server started showing problems.


Hope the above is of help.

Pete

Title: Re: Trying to solve server crash
Post by: Fahrvenugen on November 13, 2019, 09:19:33 pm
Also when you're looking inside the system, have a look at the capacitors on the motherboard.  If any are showing signs of bulging or leaking, this is a good indication for the cause of a crash.  If this is the case then spend some money on new capacitors and replace them (or find someone who's good with a soldering iron and see if they'll replace them for you).

Title: Re: Trying to solve server crash
Post by: Andreas Schnellbacher on November 13, 2019, 11:26:42 pm
have a look at the capacitors on the motherboard.
That's a good point. But in most cases and when the manufacturer has used capacitors of high quality, you won't see any deformation. You just can measure it. When capacity has decreased significantly, then it's time for soldering or ordering a new mainboard.
Title: Re: Trying to solve server crash
Post by: Ben on November 14, 2019, 01:56:25 am
Thanks people.
I'll give those suggestions a try over the weekend as that will take a while.

Does anyone have any idea why that entry is repeated in the POPUP.LOG?
Title: Re: Trying to solve server crash
Post by: Andi B. on November 14, 2019, 10:23:46 am
In contrary to the other speculations here in this thread I would guess if the problem is repentantly in Logdaem and errlog.dll then it's most probably not power supply related (includes capacitors).

Unfortunately I've no clue what logdamn and errlog do. No clue how to test if those program works as designed. I would check if errlog.dll is accessible and is the correct version and has the same CRC as a known working one from another system. Try to find someone who knows how to setup and test logdaem.

Of course all sort of problems are possible - failing harddisk, power supply, wrong LIBPATH, .... Only you know how old your system is and if you've used the cheapest parts or ones with better quality. F.i. if you're running a 2 years old Lenovo or HP standard system probability of failing power supply (capacitors) is much lower as when you're running a 15 years old server build up from the cheapest components available at ebay or alibaba.

For me until now there is far too less information in this thread to start speculating about HW issues.
Title: Re: Trying to solve server crash
Post by: Ben on November 14, 2019, 12:50:16 pm
At this point I'm trying to stick with what I know. I know that there is something popping up in the POPUP.LOG that shouldn't be there.
I can't say for certain if it is related to the crashes, but the frequency of it gives me pause for cause and it is the first place that I should look.

Either way, that issue has to be addressed. Once it is I will fix it and if my server instability perisists then I shall proceed onto the next most likely cause.

I have not seen that entry in the log before after using OS/2 for decades. It is also interesting to note that those entries started appearing on the 6th of November which is around the time that the server became unstable.
Title: Re: Trying to solve server crash
Post by: Ian Manners on November 14, 2019, 01:46:38 pm
Hi Ben,

The only reason I can think of why the log daemon would have a problem is if the log file cannot be written to, assuming there are no problems with your harddisks. It is close to 20 years since I've had OS/2 log or error daemons enabled in OS/2 (apart from syslog) so have no memory of where the logs maybe kept.
Do a search of your boot drive and maybe consider backing any log files up if you need then then delete them.
Title: Re: Trying to solve server crash
Post by: Fahrvenugen on November 15, 2019, 05:01:49 pm
Only you know how old your system is and if you've used the cheapest parts or ones with better quality. F.i. if you're running a 2 years old Lenovo or HP standard system probability of failing power supply (capacitors) is much lower as when you're running a 15 years old server build up from the cheapest components available at ebay or alibaba.

Hi,

I'm not trying to cause a big debate here, but just about 2 months ago I had to replace a bunch of failing capacitors on a 4 year old HP enterprise-grade sever.  I also just took some 5 year old HP desktop machines out of service which have capacitors showing signs of failure.

Poor quality capacitors are still a problem.
Title: Re: Trying to solve server crash
Post by: Ben on November 15, 2019, 06:53:27 pm
There is always the possibility that this is a hardware issue, however, as long as the entry in the log is consistently the same program, so that has to be eliminated first. If it were never the same thing or no logs had any recent errorsin them then I would focus more on the hardware.

It is a hard lock though, not a soft lock; everything locks solid without indication as to what is causing it not even the mouse or any monitor moves. Not even the 3-finger salut does anything. And it is completely inconsistant as to when it will happen, for example the server has been up for over seven hours right now without incident. And then it might lockup solid every few minutes. I have replaced some hardware, (video card), and shuffled around where my USB mouse connector is, but nothing had any effect; nothing made it worse or better.

So, back to the issue of the logdaem entry in the POPUP.LOG. What does that program even do? I looked for some information on it, but there is very little within OS/2 itself on it, nor in any book that I have. The obvious answer is that it controls the logging of certain, (or all), log files that OS/2 generates, but that's just a guess. It may not be related to OS/2 itself, but something else running under the OS.

An easy test is to remove it from the CONFIG.SYS and see if anything different manifest. I think I'll give that a go.
Title: Re: Trying to solve server crash
Post by: RickCHodgin on November 15, 2019, 09:33:25 pm
Hi,

I'm not trying to cause a big debate here, but just about 2 months ago I had to replace a bunch of failing capacitors on a 4 year old HP enterprise-grade sever.  I also just took some 5 year old HP desktop machines out of service which have capacitors showing signs of failure.

Poor quality capacitors are still a problem.

I'm surprised to hear that about HP.  I've purposefully chosen to buy HP equipment because the machines I've gotten from them have been rock solid stable.  I've had one laptop I used daily for 5+ years that my wife is still using daily.  It's almost 9 years old now.  I have another HP desktop machine that was purchased in 2007 that I use for development.  It is absolutely stable, and because of its design I was able to increase the memory to 8 GB and CPU to quad-core 2.4 GHz, and still have quite a decent performer out of it 12+ years later.

I would buy HP again in a heartbeat.  Never tried an enterprise-grade server, however. :-)
Title: Re: Trying to solve server crash
Post by: roberto on November 16, 2019, 10:44:00 am
I do not know if it could or not have something to do with your problem, but I in an HP laptop, very good team by the way. It happened to me that it was very unistable and finally removing the acpidaemon.exe works better than ever. You can kill this process and start it with the computer running and see differences.

Title: Re: Trying to solve server crash
Post by: Ben on November 16, 2019, 06:31:42 pm
This is not a new server setup, but a very long running one.

Nothing has really changed. I did install the latest XFolder a while back, but this latest problem has come in only recently... in the past week or so I shall say. Unless something can take a while to ferment, (a month or so), I cannot see any connection between the two. However, I probably should remove it and see if it makes any difference.
Title: Re: Trying to solve server crash
Post by: Fahrvenugen on November 18, 2019, 09:38:03 pm
Ben,

If it has been running for a while with XFolder without problems, then it is unlikely to be an issue there.  Over the years with OS/2 I've generally found that OS and software problems normally turn up fairly quickly, they don't normally just show up after a long time of stability.  But if that was the last major change made then certainly remove it and see if the problem fixes itself.

One thing to check though - make sure you've got free space on your drives, especially the one that the swap file is on.

Also - you mentioned this is running as a server.  Check log file sizes on any server software that you're running.  I recall once I had a situation where a MySQL server started crashing - eventually I figured out that the log that MySQL kept had become too big.  After deleting the log file (and I think I tweaked the configuration file to limit the log size or something) the stability came back.

You could also try running checkini on the system - just in case it is something wrong with the ini files.  Before you do this make a backup of your files just in case  something goes wrong.

And as previously mentioned, make sure that logdaem.exe and errlog.dll are not corrupted.

Just checking a few things, logdaem.exe is part of the first failure support technology diagnostic stuff, you could try REMing it out of your config.sys and see if the problem goes away.  I admit that after some googling I'm not finding a lot on logdaem.exe, but there is one article  suggesting it isn't really needed for everyday use and to try REMing it out.

I still am not willing to rule out power supply and / or hardware issues.  FFST could be picking up a hardware problem and throwing the error with logdaem catching it for display/logging.  I've also found that a lot of times when a system is completely locked up (3 finger salute, mouse, keyboard, all don't work) and the problem just starts happening after years of stability, those are the situations where I've often found a hardware issue to explain it.

Not sure if any of this will help.

Title: Re: Trying to solve server crash
Post by: Ben on November 19, 2019, 03:53:20 pm
I removed logdaem.exe from the CONFIG.SYS and it made no difference.

However, I did uninstall XFolder and the server has now been running for a day and half without a crash. I find that very odd since for as I said in a  previous post, it ran for about a month installed without incident.

I like the enhancements that XFolder adds and I'm reluctant to do without it, but it things are not stable with it then I may well have no choice, but to keep it off the system.

Grrrr...
Title: Re: Trying to solve server crash
Post by: Andi B. on November 19, 2019, 04:00:51 pm
Do you really use the ancient XFolder? Why not XWorkplace? Isn't XFolder the forerunner of XWP or do I mix up things?
Title: Re: Trying to solve server crash
Post by: Ben on November 19, 2019, 04:32:40 pm
XFolder / XWorkplace; it's the same thing.
Title: Re: Trying to solve server crash
Post by: Andi B. on November 20, 2019, 10:34:44 am
XFolder / XWorkplace; it's the same thing.
XFolder is about 15 years older than XWP. It's up to you if you think the changes/improvements/fixes are not related to your problem. But as you stated XFolder was the only thing you changed it's a valuable guess to think about this software IMO. I fear it's to much to ask which version of XFolder (or XWP?) you've installed.

Title: Re: Trying to solve server crash
Post by: Ben on November 20, 2019, 01:34:24 pm
As I said, (and as most people here know), XWorkplace is XFolder renamed; the two names are often used interchangeably.

The latest version of it is 1-0-13b1.

Normally, I do not use it on my servers because it does cause instability there where the machine is stressed.

However, I tried version *.12, (first time in a long while), when it was released in hope that it would not cause the same issues; it did, so I removed it.

When *.13b1 came out I gave that one a try. Oddly, it remained stable up until a week ago. I uninstalled it 2/3 days ago and I haven't had a lockup since, the server running continuously and smooth. Whether something I updated after installing version *.13b1 triggered it to misbehave or not I do not know. I do know that I would rather have it than not, but server stablity is of paramount importance and takes presidence over all else.

I do not have issues with stability while using it on any of my other OS/2 based machines, but none of them work so hard as the server and I have always had issues of stability while using it once installed, on every version of every server that I have ever had since the 90s. I test later versions every now and again to see if that has changed; so far is hasn't. Too bad.
Title: Re: Trying to solve server crash
Post by: Doug Bissett on November 20, 2019, 05:03:56 pm
Quote
As I said, (and as most people here know), XWorkplace is XFolder renamed; the two names are often used interchangeably.

Well, modern XWorkplace is certainly NOT the predecessor XFolder, and the names should NOT be used interchangeably. That is confusing, at best.

Quote
I test later versions every now and again to see if that has changed; so far is hasn't.

It has changed, for the better, but only parts of it have been updated (mostly XCenter). Other parts are still somewhat unstable, under some conditions. That is one of the main reasons to have the limited function (lite) versions (eWorkplace, for eCS, and the Arca Noae lite version). They omitted the most troublesome parts. It has been my experience, that just having XWP lite installed really doesn't cause instability. Using XWP lite, on my main machine, does cause problems, unless I clean my INI files (cleanini, and FPOS), every couple of weeks (which also includes a reboot). How often you need to do that, depends on exactly how you use the machine. A server, where you don't use the desktop much, is probably good for much longer than a machine where you use the desktop every day. Sometimes, I clean the INI files more often, if the machine starts to do strange things. It usually restores stability, and if not, I go looking for other things.

XWP is a wonderful add on, for OS/2. It adds a lot of desktop function that plain OS/2 does not have, However, as with all software, it always has ways to cause trouble. Overall, I find it worth the effort, for machines where I use the desktop. On a server, where you would rarely use the desktop, it may not be worthwhile.
Title: Re: Trying to solve server crash
Post by: roberto on November 20, 2019, 06:26:18 pm
I disagree that the problem is the Xwp 13. For me the problem is in acpidaemon.exe.
This does not mean that what you have done is the right thing, and I would have done the same.
From my point of view, acpidaemon.exe balances the processes so that some applications or others are better executed. And he does it wrong. In fact it has an option of possible configuration, acpid.cfg, which is a fool fools.
This acpid.cfg, only allows you to configure it to what is most important to you and that works for you.
I explain myself better;
When the processors work like this, type A
No. of CPUs: 8
CPU 1 - Busy: 99.6%, Interrupt: 0.4%, Idle: 0.0%
CPU 2 - Busy: 100.0%, Interrupt: 0.0%, Idle: 0.0%
CPU 3 - Busy: 100.0%, Interrupt: 0.0%, Idle: 0.0%
CPU 4 - Busy: 100.0%, Interrupt: 0.0%, Idle: 0.0%
CPU 5 - Busy: 100.0%, Interrupt: 0.0%, Idle: 0.0%
CPU 6 - Busy: 100.0%, Interrupt: 0.0%, Idle: 0.0%
CPU 7 - Busy: 100.0%, Interrupt: 0.0%, Idle: 0.0%
CPU 8 - Busy: 100.0%, Interrupt: 0.0%, Idle: 0.0%
Some programs work fine without problems. Truespectra, Manhjon approx 10 to 20%



When processors work like this, type B

 No. of CPUs: 8
CPU 1 - Busy: 0.0%, Interrupt: 0.4%, Idle: 99.6%
CPU 2 - Busy: 0.1%, Interrupt: 0.0%, Idle: 99.9%
CPU 3 - Busy: 12.7%, Interrupt: 0.0%, Idle: 87.3%
CPU 4 - Busy: 4.2%, Interrupt: 0.0%, Idle: 95.8%
CPU 5 - Busy: 0.0%, Interrupt: 0.0%, Idle: 100.0%
CPU 6 - Busy: 0.0%, Interrupt: 0.0%, Idle: 100.0%
CPU 7 - Busy: 29.7%, Interrupt: 0.0%, Idle: 70.3%
CPU 8 - Busy: 84.2%, Interrupt: 0.0%, Idle: 15.8%
Other programs work fine without problems, dbexpert, youtube, approximately 80%

What I mean is that with acpidaemon, I can make some type A programs work well, or those of type B work well, but never all at once. Although not all computers are the same, and the behavior on some computers might seem to work at both ends, but when you have programs at both ends that give problems, you realize that you don't.

So what I do is kill the acpidaemon, and I make sure that type B programs work well for me.
Losing those of type A.
Now I'm waiting to be able to execute those of type A.
What I do is create a virtual disk in ram like HPFS this is important in jfs it doesn't work.
I create a folder that contains a lot of files, from 10,000 to 90000 files. And I delete them with another program rm.exe -r g: \ aaa-testfiles000 as part of dtbackup.zip
While the files are deleted, programs of type A can be executed without problems, and most importantly without stressing the equipment, without acpidaemon, and with xwp13 everything perfect.
The greater the number of files I have more seconds for the tests.


https://www.tavi.co.uk/os2pages/utils/dtbackup.zip

The version [C: \] rm --version
rm (GNU coreutils) 8.26 that is in arcaos504 does not work.


Version 5.00 of the rm.exe if it works fine, I think it is slower.
I have the trash configured as delete everything.

All this is an idea that is very green, but I get the feeling that with certain applications running, they help others to behave in one way or another. As a catalyst.
Title: Re: Trying to solve server crash
Post by: Doug Bissett on November 20, 2019, 07:03:02 pm
Quote
No. of CPUs: 8
Does the machine actually have 8 processors, or is it 4 processors with Hyper Threading enabled? Hyper Threading is known to be very unfriendly to OS/2, and very often it will cause hangs, and other bad things, because it screws up the much more efficient OS/2 threading model.
Title: Re: Trying to solve server crash
Post by: roberto on November 20, 2019, 07:59:35 pm
Very good observation Doug. In the previous notebook that gave me a lot of war. But in this notebook, it is the same to have it marked than unchecked the HT. I have it marked ON. Thanks.
Title: Re: Trying to solve server crash
Post by: Doug Bissett on November 20, 2019, 08:11:31 pm
Quote
I have it marked ON.

I assume this means that you have Hyper Threading enabled. Disable it. It makes a slight improvement with windows. It causes nothing but trouble with OS/2, and most other operating systems.
Title: Re: Trying to solve server crash
Post by: Ben on November 21, 2019, 02:27:17 am
Quote
As I said, (and as most people here know), XWorkplace is XFolder renamed; the two names are often used interchangeably.

Well, modern XWorkplace is certainly NOT the predecessor XFolder, and the names should NOT be used interchangeably. That is confusing, at best.

Correct Doug; Xfolder is the predecessor to XWorkplace, (not the other way around), which was picked up after Ulrich moved away from OS/2.

Quote
I test later versions every now and again to see if that has changed; so far is hasn't.

It has changed, for the better, but only parts of it have been updated (mostly XCenter). Other parts are still somewhat unstable, under some conditions. That is one of the main reasons to have the limited function (lite) versions (eWorkplace, for eCS, and the Arca Noae lite version). They omitted the most troublesome parts. It has been my experience, that just having XWP lite installed really doesn't cause instability. Using XWP lite, on my main machine, does cause problems, unless I clean my INI files (cleanini, and FPOS), every couple of weeks (which also includes a reboot). How often you need to do that, depends on exactly how you use the machine. A server, where you don't use the desktop much, is probably good for much longer than a machine where you use the desktop every day. Sometimes, I clean the INI files more often, if the machine starts to do strange things. It usually restores stability, and if not, I go looking for other things.

XWP is a wonderful add on, for OS/2. It adds a lot of desktop function that plain OS/2 does not have, However, as with all software, it always has ways to cause trouble. Overall, I find it worth the effort, for machines where I use the desktop. On a server, where you would rarely use the desktop, it may not be worthwhile.

Is XWP Lite being updated still? If so, I wouldn't mind giving that a try.
Title: Re: Trying to solve server crash
Post by: Doug Bissett on November 21, 2019, 03:27:22 am
Quote
Is XWP Lite being updated still? If so, I wouldn't mind giving that a try

It is at HOBBES. v10.13 isn't there, yet, but v10.12 is: http://hobbes.nmsu.edu/download/pub/os2/apps/wps/xworkplace/xwp-1-0-12-lite-en.exe (http://hobbes.nmsu.edu/download/pub/os2/apps/wps/xworkplace/xwp-1-0-12-lite-en.exe). Other languages are also there.

AFAIK, only XCenter has been modified in XWP 10.13. It will probably be placed at HOBBES, when Rich Walsh gets around to doing it (he did the changes).
Title: Re: Trying to solve server crash
Post by: Ben on November 21, 2019, 01:13:23 pm
Excellent!

I shall down load it and give it a go.

Thanks for the heads up.
Title: Re: Trying to solve server crash
Post by: Andi B. on November 21, 2019, 02:51:37 pm
Quote
Correct Doug; Xfolder is the predecessor to XWorkplace, (not the other way around), which was picked up after Ulrich moved away from OS/2.
XFolder was renamed to XWorkplace in 1999. Yes it's more than 20 years ago now. In Nov. 1999 Ulrich released the first version of XWorkplace. He worked and improved XWP at least until 2004. After that others contributed a lot. Mostly bug fixes.

Quote
XFolder / XWorkplace; it's the same thing.
Quote
As I said, (and as most people here know), XWorkplace is XFolder renamed; the two names are often used interchangeably.
I can not speak for the 'most people here' but IMO these statements are very ignorant and misleading.
Title: Re: Trying to solve server crash
Post by: Ben on November 21, 2019, 03:50:37 pm
Quote
XFolder / XWorkplace; it's the same thing.
Quote
As I said, (and as most people here know), XWorkplace is XFolder renamed; the two names are often used interchangeably.
I can not speak for the 'most people here' but IMO these statements are very ignorant and misleading.

Do you now?

Well, I thought about suggesting that your remarks be moved to the forum category labeled "Rude Members Who Have No Tact, Social Graces or Friends"... but I have the grace to never do such a thing. 8)

So,it is fortunate that you have openly admitted that you do not speak for others here... others who in fact do have the tact to construct their responses to maintain social peace in this old and venerable forum. ;)

One can only hope that others have the ability to learn...


To Doug; I have installed the lite version of XWorkplace that you recommended and so far so good... now if only I can find out how to get rid of the trashcan since the XWorkplace setup object doesn't appear to have been created. I suspect that this is the norm for the "Lite" version.
Title: Re: Trying to solve server crash
Post by: Ben on November 21, 2019, 07:56:22 pm
Unfortunately, it no longer matters. The "Lite" version faired no better than the "Full" version; locked solid after a few hours.

I reloaded from a Unimaintenance backup, (easier than uninstalling).

Too damn bad.


Grrrr...
Title: Re: Trying to solve server crash
Post by: Martin Iturbide on November 22, 2019, 01:36:24 pm
Hi Again

Please remember the "Rules for the Forum - Code of Conduct (https://www.os2world.com/forum/index.php/topic,1659.0.html)" based on Wheaton's Law.

It is more useful to help without labeling people and sarcasm. Remember that there are skill level difference and cultural difference between the people that are interacting. I know that it boils the blood when we see a wrong statement, but it is better to focus on the why the statement is wrong without using unnecessary qualifiers and without targeting people.

Please remember that is not necessary to reply to everything, and sometimes is better to walk off from the thread instead of replying, because on "internet flame wars" there are no winners. 

Regards