• Welcome to OS2World OLD-STATIC-BACKUP Forum.
 

News:

This is an old OS2World backup forum for reference only. IT IS READ ONLY!!!

If you need help with OS/2 - eComStation visit http://www.os2world.com/forum

Main Menu

OS/2 Hardware: What's the maximum?

Started by RobertM, 2007.11.01, 02:26:13

Previous topic - Next topic

RobertM

Hey all,

We are upgrading one of our servers to an IBM x440 server. The machine supports OS/2 (IBM Cert Labs Certified), but brings up various questions (hardware and/or OS related):

- What happens to OS/2 on a machine with over 4GB of RAM? (or is that limit per CPU?) Has anyone tried? I have a Netfinity 7000 M10 with 4GB, and WSeB works fine with it with no fixes, but I dont have the secondary memory board to try bring the machine to it's 8GB hardware limit to see what happens).

- Is the OS/2 Thread limit a per CPU limit or a total limit for the OS?
(and if it's per CPU, is the MAXTHREADS line the system total, or the per CPU total?)

Thanks in advance,
Robert


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


The Blue Warper

As to the first question: the 4GB limit is imposed by the kernel (4 GB being the maximum amount of memory a 32 bit kernel can handle).  We're close to reach the physical limit of our OS: that's - among other things, of course - what pushed Netlabs to start a project like Voyager.  We should all hope they'll success in it.

As to your second question: don't really know.  Just guessing: I think it's a kernel limit too (as threads are handled at kernel level), so the number should be regarded as OS-global.

Regards

Robert Deed

Hate to contradict the other reply to this message, but the 4gb memory limit (depending on the hardware) is per processor.  In the case of your system, if you had 8gigs in it, one processor would see one 4gig bank while the other sees the other 4gig bank.. this would also lead to further application isolation.  I've tested this with a dual opteron and used a test application to allocate memory and I was actually capable of allocating 4gigs twice over with 0 growth of swapper.

The maxthreads are the maxthreads.. you can set the number higher with more processors but the setting is not per processor, it is the cumulative total of all processors. 

I cannot remember off hand what the total threads allowable by the kernel is (the smp and uni kernels have different limits i believe) but I think it is 32767.  However, in the real would unless you can using an app which scales dramatically with many threads then you should never need more then 1024 or so.

Quote from: RobertM on 2007.11.01, 02:26:13
Hey all,

We are upgrading one of our servers to an IBM x440 server. The machine supports OS/2 (IBM Cert Labs Certified), but brings up various questions (hardware and/or OS related):

- What happens to OS/2 on a machine with over 4GB of RAM? (or is that limit per CPU?) Has anyone tried? I have a Netfinity 7000 M10 with 4GB, and WSeB works fine with it with no fixes, but I dont have the secondary memory board to try bring the machine to it's 8GB hardware limit to see what happens).

- Is the OS/2 Thread limit a per CPU limit or a total limit for the OS?
(and if it's per CPU, is the MAXTHREADS line the system total, or the per CPU total?)

Thanks in advance,
Robert

RobertM

#3
Hi Robert...

Thanks for the reply - that would explain why the machines come with 8GB in the 2 CPU config.

As for the maxthreads setting, the help file says "The system supports a maximum of 4095 threads, which it allocates to itself and the applications running on it." - but is from the UNI model of Warp (which isnt what I am running). So I guess trial and error is going to be the only way I can find out what the maximum is. The one thing I did notice is that if I increase "THREADS=" to something over 2048, my HPFS386 cache drops considerably (even though I have the memory size hard-set in the INI file). I'm guessing the thread management section is grabbing some portion of shared memory that reduces available memory for the HPFS386 cache.

Currently I have 4 CPUs, 4GB RAM, 512MB HPFS386 cache, 2048 threads... over 2048 threads seems to reduce the system memory considerably.


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


RobertM

#4
Hmmm... actually, my last post is somewhat incorrect.

Setting the threads= to 4090 doesn't really decrease the amount of RAM... just what the HPFS386 system sees. During boot I get the following error:

Error HFS0012: The cache size specified is too large. 20% of available memory will be used.

Then the HPFS386 subsystem allocates just under 36MB to the HPFS386 cache (with 3.6GB of free memory)... so it obviously isnt using 20% (the rest of the missing RAM is being used by the JFS cache which is set to 256MB - with or without JFS loaded, the HPFS386 result is the same).

I'm going to try to manually set the HPFS386 cache to what it will think is 80% of available RAM and see how that goes... just don't know where it's getting that limit from (ie: why is it misreading available RAM when the number of threads is increased from 2048 to 4090). I've got the latest HPFS386 fixes installed (though the original WSeB CP2PF SMP kernel is installed - which is 14.089  --  maybe I should update the kernel?)

-Robert


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


RobertM

Well, that worked... till I tried running anything (not enough available memory errors, and then the system crashed - which is the first ever crash on this machine). I think I need to play with the memory settings... in the meantime (since this is a full time Internet server) I am going to put the settings back to something more conservative...


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


Robert Deed

#6
I think what you're experiencing is thread overhead.  I believe memory is allocated for all threads when the system is booted.

As well I believe the HPFS cache (hpfs386), threads, and drivers all occupy the region of memory which is limited below what the system memory totals.. (jfs is ok).

You may want to play around with hpfs386's ability to use high mem (or whatever it calls itself to load itself beyond the 16mb barrier). 

I'm guessing that all the thread specific memory allocation is being done in the first 16mb of memory which most of the drivers and kernel are loaded into.

Just out of curiosity, do you have a specific need for such a high number of threads? (or are you just interested in understanding it's limits?)

I would go for the newest smp kernel since there have been a great number of fixes in these kernels since wseb's release.  This may not fix your memory issue since I believe this is working as designed, but I cannot be certain of that.

Quote from: RobertM on 2007.11.01, 20:17:24
Hmmm... actually, my last post is somewhat incorrect.

Setting the threads= to 4090 doesn't really decrease the amount of RAM... just what the HPFS386 system sees. During boot I get the following error:

Error HFS0012: The cache size specified is too large. 20% of available memory will be used.

Then the HPFS386 subsystem allocates just under 36MB to the HPFS386 cache (with 3.6GB of free memory)... so it obviously isnt using 20% (the rest of the missing RAM is being used by the JFS cache which is set to 256MB - with or without JFS loaded, the HPFS386 result is the same).

I'm going to try to manually set the HPFS386 cache to what it will think is 80% of available RAM and see how that goes... just don't know where it's getting that limit from (ie: why is it misreading available RAM when the number of threads is increased from 2048 to 4090). I've got the latest HPFS386 fixes installed (though the original WSeB CP2PF SMP kernel is installed - which is 14.089  --  maybe I should update the kernel?)

-Robert

RobertM

Hi Robert,

USEALLMEM is set to access above 16MB (which works fine with less threads). I have managed to set the HPFS386 cache to 700MB, with the threads set to 1700.

I *may* need the thread space later (some intensive web serving) though not on this box. We're trying to get an x440 which we plan on putting 8 CPUs in, so the more threads the better when the web server is under load. Currently I have DominoGo allocating 1024 threads on start, but would like to push it to something closer to 3000 threads.

I was thinking that maybe I have to play with the virtualaddresslimit statement, but cant play with this machine much more today (perhaps when we get it's replacement in). As it is, because my cache idle times were set so high, it took a while to get the cache and thread settings back to something that worked (READ AS: "The first 3 attempts to change the file, which crashed the machine due to lack of memory resources didnt trigger a clue in my mind that the file never got written since it was being cached and the cache expire never occurred before the crash... so it took a while for me to realize I needed to boot to command prompt, change the files and reboot").

So, my stupidity cost us some downtime today... maybe late tonight I'll play with it some more. It's not that important (other than learning how) at this moment, since our new server wont be in for a week or so, and on intense SQL/dynamic page generations, the system only has enough CPU power to crank out about 100 completed simultaneous connections per second (and it lags 50-70 of them).


The only other thing I am trying to find is a reference to how OS/2 WSeB handles memory on multi-CPU units, and nothing I can find deals with that... only reference the 3GB per process limit, mentions SMP, but doesnt say anything about how SMP affects total limits. Not too big a deal as (other than disk cache) we are barely touching the 4GB in the machine we have now... just want to figure out what I can - or should max the new machine to once we manage to stick 8 CPUs in it.

Thanks,
Robert


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|