• Welcome to OS2World OLD-STATIC-BACKUP Forum.
 

News:

This is an old OS2World backup forum for reference only. IT IS READ ONLY!!!

If you need help with OS/2 - eComStation visit http://www.os2world.com/forum

Main Menu

(Apache/PHP/MySQL related) Looking for info on PROCESSES= config.sys statement

Started by RobertM, 2010.04.12, 21:45:07

Previous topic - Next topic

RobertM

Hello all,

I am looking for information on the PROCESSES= statement for the config.sys file in regards to WSeB and eComStation. The info I have found so far is very vague.

This is for a server that is generally under high load running Apache. Any help would be greatly appreciated.

Best,
Robert


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


chennecke

Well, there simply is not much information available. All I ever found -- including that which Scott Garfinkle provided me with -- is in the XWorkplace documentation. Be careful with that setting since as far as I understood it, increasing the number of possible processes reduces the available shared memory.

That said, running high-traffic sites with Apache under OS/2 is pretty much a no-go, at least if also PHP and MySQL are involved. AFAIK the problem is that sockets/processes are opened/created but never properly closed/terminated. IIRC Apache 1.3.x and 2.x have slightly different problems. Increasing the number of processes could maybe postpone the point of failure but will not be able to actually solve the problem. You probably will have to reboot the machine regularly (which OS2.org does). What you CAN do is to reduce the risk of failures by using Squid to "shield" Apache so the problematic conditions will not occur.

Try contacting Roderick Klein for some advice. Mensys are probably those with the most experience in that area.

RobertM

Yup... and the problem seems to be Apache or PHP... killing it (Apache) seems to free the tied up resources.

It turns out I think I can write a script that does that in under one second... so... will try that for now till I can get some ideas as to possible fixes.

Currently I am running into the problems you noted at very low loads (compared to our main LDGW box).


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


RobertM

Here's something interesting I have noted:

When this system bombs, it's a crash in LIBC063.DLL and/or PHP5.dll that seems to cause it (and the related out of resources) even with a bunch of memory free.

I get a SYS3171, with a crash in httpd.exe caused by one or both of those DLLs. MySQL at that point becomes unusable as well until I kill it and restart it.

Or, inotherwords, once httpd.exe crashes via one of those modules, MySQL needs to be restarted in order to free the resources that are tied up.

On another machine, running the same version of MySQL which is only accessed via REXX and the MySQL.EXE CLI app, I have never had to restart due to lack of resources (even though it handles as many transactions).


Now, the differences between the systems are as follows:


System that "crashes"System that doesnt crash
Uses ApacheUses LDGW
Uses PHPUses REXX
2GB RAM4GB RAM
1536 VirtualAddressLimitVAL not set
Combination HPFS386/JFS (32MB/64MB)Combination HPFS386/JFS (32MB/64MB)
1600 Threads1600 Threads
Single CPUSMP (Quad Core)
UNI KernelSMP Kernel
Roughly 30,000 MySQL transactionsUpwards of 100,000 MySQL transaction
Roughly same number of web requestsRoughly same number of web requests
MySQL 5.0.51MySQL 5.0.51
Apache/2.2.8LDGW 4.6.2.8
PHP/5.2.5NO PHP
Apache uses ??? threads and processes???LDGW creates 1000 threads and manages them itself
LDGW presumably handles it's own processes for handling REXX scripts
No REXX used by Apache.

Dunno if that is helpful in determining the location or cause of the resource issues...


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


chennecke

I think you won't have much luck solving the issue here. IIRC there is an Apache for OS/2 mailing list. You could also try raising the issue on the netlabs.org IRC channel.

RobertM

Quote from: chennecke on 2010.04.20, 17:43:09
I think you won't have much luck solving the issue here. IIRC there is an Apache for OS/2 mailing list. You could also try raising the issue on the netlabs.org IRC channel.

I'm thinking of doing so... but once I (or we - ie: others in the community with more Apache/Warp experience than I) track down whether it's Apache, libc063, or PHP - or how MySQL interacts with them - or the UNI setup that this OS/2 machine uses)...

The interesting things I found are: whatever is causing this cannot be alleviated without killing MySQL and restarting it - even though in a non-Apache/PHP world, MySQL doesnt run into these issues.

So... is it LIBC063? Is it Apache? Is it PHP? Is it processes or connections left open by PHP? Or ones left open by Apache via PHP? or is it that PHP and/or Apache are making extensive use of LIBC063? And why doesnt use of REXX and RxSQL cause a similar problem?

One thing I did not note on my earlier post is that both systems use CREXX (though of course, the Apache system isn't actually using REXX to facilitate any requests).


(thus, these are my questions to anyone more familiar with this than I - before I post the issues elsewhere - to help determine where the "elsewhere" is that I should be posting these to).

So... before I go posting there, I was hoping to gain some more insight to ensure the Apache list is where I should be posting (as opposed to looking for a newer or fixed version of LIBC063, or posting in the PHP related groups, or so on).

Or perhaps there is simply something that gets handled differently with an SMP version of OS/2? In which case, it would seem that the UNI versions are experiencing the problems and not the SMP versions of OS/2? That of course may mean it's an OS/2 issue related to how the resources are handled on UNI versions.

Sadly, I dont have an Apache/PHP/MySQL setup on an SMP version of OS/2 yet... does anyone who has such a setup run into this problem even though it's on an SMP version of OS/2 on an SMP computer? If so, is this one of the issues that gets alleviated by running the SMP kernel and other code on a non SMP machine?

Best,
Robert


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


Fahrvenugen

Hi,

I had been running into exactly the same issue a while back, and there was a lot of discussion on the Apache/2 email list about the problem (others had it too).

I never was able to find a reliable solution.  My setup was Apache2, PHP, and MySQL.  I ran this on both SMP and non-SMP machines and kept running into the same issue - the whole thing would eventually (after a day or so) get excessively slow and then lock up (usually requiring a reboot).  As for specific versions, I tried multiple different versions of Apache, PHP, and MySQL (somewhere I have notes on all the combos I tried).  I never did find one to be more reliable then another.

As much as I hated to do so, I needed to get that server up and running, so I ended up going with another OS for that server.


Ben

Quote from: RobertM on 2010.04.20, 18:10:50

Or perhaps there is simply something that gets handled differently with an SMP version of OS/2? In which case, it would seem that the UNI versions are experiencing the problems and not the SMP versions of OS/2? That of course may mean it's an OS/2 issue related to how the resources are handled on UNI versions.

Sadly, I dont have an Apache/PHP/MySQL setup on an SMP version of OS/2 yet... does anyone who has such a setup run into this problem even though it's on an SMP version of OS/2 on an SMP computer? If so, is this one of the issues that gets alleviated by running the SMP kernel and other code on a non SMP machine?

I do not know if any of this will help you Robert, but I had similar problems with Apache/2 2.*, mySQL and PHP a year or two ago; I had upgraded from older versions.

At that point several others on the OS/2 ISP mailing list were having similar problems.

I can't tell you exactly what fixed the problem here, (which had become a major nuisance), but I'll tell you what I did and maybe it will give you a few more clues.


I switched over to the SMP kernel where I was only using one core in a dual core system before.

I backed up my conf files and SQL database, deleted and re-installed A/P/mS using the then latest version of AMP, (there have since been two or more releases since).

Though that's not very specific, I hope that some of this helps you.

Fahrvenugen

Ben,

If you have a stable system, could you possibly check to see which versions / builds of Apache, PHP and MySQL you're running?

Also, do you happen to know which build / version of the SMP kernel you're running? 

I had never considered that it might be a kernel issue, but all the machines I had the problem with were running 14.104a (one on SMP with 2 processors, the other with the Uni kernel and 1 processor).

I'd love to try switching that system back over to OS/2 (in fact, I have 2 other OS/2 servers running at the site where I had the issue that I'd initially intended to run that website on, only to give up and set up a 3rd computer - just to run a Wordpress website - because I couldn't have my other 2 OS/2 servers - which normally the only downtime they have is when I reboot them for some reason or another - locking up on me on a daily basis).


RobertM

Quote from: Fahrvenugen on 2010.04.21, 05:27:24
Hi,

I had been running into exactly the same issue a while back, and there was a lot of discussion on the Apache/2 email list about the problem (others had it too).

I never was able to find a reliable solution.  My setup was Apache2, PHP, and MySQL.  I ran this on both SMP and non-SMP machines and kept running into the same issue - the whole thing would eventually (after a day or so) get excessively slow and then lock up (usually requiring a reboot).  As for specific versions, I tried multiple different versions of Apache, PHP, and MySQL (somewhere I have notes on all the combos I tried).  I never did find one to be more reliable then another.

As much as I hated to do so, I needed to get that server up and running, so I ended up going with another OS for that server.



I've only run into one crash so far, and that crash was in JFS (which could be because when I got this system back up and running, I neglected to upgrade the JFS drivers to the latest versions - never had a JFS crash on my SMP JFS box).

I am suspecting that if the need arised for starting other processes from core subsystems, a crash is possible though - perhaps that is what happened with JFS on this box.

I have tried the following via a REXX script to try to resolve this (which is part of why I am sure it's not Apache itself - or rather not solely Apache).

  • I have a REXX script running in a separate process. It spawns Apache in it's own process.

  • Every six hours, the REXX script kills Apache (calls Apache/2's shutdown.cmd file - which simply calls KILL.EXE with the PID).

  • Waits a second upon completion and then spawns another copy of the REXX SCRIPT (not Apache).

  • The NEW copy of the REXX script then starts Apache in it's own process space.


What I have noted is that memory that Apache (and PHP?) had allocated does indeed get freed. But, still in a day or two (seems dependent on server usage) I still run into the same resource problem - which first results in slower connections, no responses to connections and then a crash in PHP and/or LIBC063 DLLs taking Apache out with it. Occassionally, the crash is in MySQL (but I'd say that's a one out of ten crashes scenario).

If I catch the box during the "inbetween time" I can kill Apache and MySQL and restart them - and everything is once again fine for about the same period of time. At this point, killing and restarting Apache itself is not sufficient to resolve the issue.

Running at 500+ requests per second, the LDGW/REXX/MySQL box does not run into those issues and happily chugs along... I've pushed that box to over 700 requests per second (at which time I run out of bandwidth - which eventually causes me to run out of TCPIP buffer space as WSeB tries buffering all the requests it cannot send upstream on the choked upstream pipe - but that's an obvious problem caused by not enough bandwidth and doesnt happen until we are at about 120% of our promised max).



|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


RobertM

Quote from: Ben on 2010.04.21, 14:34:28

Though that's not very specific, I hope that some of this helps you.[/color]

Hi Ben,

That may indeed be helpful. If you could check on one other thing for me... what version (or filesize or other version indicator) is the LIBC063 DLL on your system?

Since MySQL seems part and parcel to my problem on the Apache system - yet exhibits no problems on the SMP system... I think (assuming it's not the version of LIBC063 on the Apache system - or the way it acts on SMP vs UNI) that maybe you are on to something.

Any and all version numbers (Apache, PHP, MySQL, LIBC063 - or anything else you think relevant) would be greatly appreciated.

Best,
Robert


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


abwillis

Just over a week before this thread was started I had received a reply from Lewis Rosenthal that indicated this is certainly not an uncommon problem.  http://www.warpstock.org was essentially down and Lewis said it was probably Apache out of resources and that he was having to restart his own servers about every 5 days to keep the support calls from coming in that email was crawling. 
In the back of my mind I keep thinking there was a discussion about this several years ago (maybe about the time that Apache 2.0 was released) but I can't find anything nor do I recall if any type of solution other than rebooting the system was found.

Fahrvenugen

I will add to this, I think it is something to do with either PHP and / or MySQL, and how they interact with Apache.

For years I have been using Dink's Web/2 with the CGI PHP and MySQL, and have never run into problems.

I have found that if I'm serving static web pages using Apache2 doesn't seem to cause any problems, Apache can run for days, weeks, etc.

I've even found that when Apache is using PHP. but*not* trying to access MySQL it can run fine without problems (using the Apache PHP DLL module)

But when I put it all together where it is trying to access a MySQL database via PHP served through Apache, that's when I ran into problems.

What's even more interesting, when the system seems locked up, Apache will still occasionally be able to serve up static HTML files, which suggests that Apache isn't entirely frozen, the lockup is somewhere in PHP or MySQL.

I had thought potentially a MySQL - PHP issue, however MySQL - PHP runs fine when used with Dink's Web/2 / PHP / MySQL

As for a potential issue with the Apache PHP DLL, I tried running it under Apache with the PHP CGI (.EXE) - the same one / setup that I've never had issues with under Web/2, and I ran into the same issue under Apache.

So where's the problem?  I never found it and gave up looking.


RobertM

So... this time, MySQL crashes - but POPUPLOG.OS2 shows Apache crashed in LIBC063 - followed immediately by Apache crashing again (weird, huh?) in PHP5.DLL

Even more interesting is that Apache was still running - while MySQL was not (nor was there an entry in POPUPLOG for MySQL).

Now, I am even more baffled and beginning to understand why this is so difficult to track down.




Should I start looking for a shared, conflicting DLL? Does one have some LIBC063 code compiled into it, or it's own version of the DLL someplace?


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


Fahrvenugen

You can definitely try looking for DLL's, if you have multiple versions of the same DLL on your system this might be part of it. 

I do wish we had a PHP module (or a way to make PHP work reliably) with LDGW, and the ability to add some of the Apache features (such as the URL re-writing stuff) to LDGW.  LDGW is a great web server, unfortunately it never seemed to get the recognition it deserved.