• Welcome to OS2World OLD-STATIC-BACKUP Forum.
 

News:

This is an old OS2World backup forum for reference only. IT IS READ ONLY!!!

If you need help with OS/2 - eComStation visit http://www.os2world.com/forum

Main Menu

OS2 & eCS 64 bit kernel? Do we need one? When? How?

Started by RobertM, 2008.08.24, 03:47:54

Previous topic - Next topic

RobertM

saborion2:

This (and others like it) is the thread to discuss kernel improvements.

For instance (to start this conversation/thread), it's not due to a lack of power or capabilities that OS/2 needs a 64bit kernel - it's because it would gain a massive increase in it's already monstrous capabilities.

So, here's an example. The host for www.startreknewvoyages.com got retarded yesterday and suspended the account (because another site on the server got infected). So, currently, I am hosting the entire web portion of the site on my ancient Netfinity Quad 550MHz. So far, since 12:28PM when the DNS change was submitted to the Internet domain name system, the server has handled over 80,000 requests... or roughly 12,000 an hour... or roughly 200 a minute... or roughly 3.3 a second...

Here's the key... the server doesnt even seem to notice it's doing something (see the attached screenshot and look at the CPU monitor - and keep in mind that the CPU levels are peaked a little because PMView was loaded and doing a screen capture).

OS/2 thus does not need a 64bit kernel to make it as capable as anything else (32bit or 64bit version of anything else for the Intel/AMD platform). It needs it to crush the others perhaps (well, it already does... it needs it to crush them even further). For instance, I am limited to 4095 threads on OS/2 and 4GB of memory... that means theoretically 3600 web requests at once (since Domino Go WebServer spawns a thread for each) - or less if SQL is called from the back end (since MySQL needs to be able to allocate threads).

With a 64bit kernel, more memory would be available for thread handles, and a larger shared memory pool would be available... but think about it. That's not necessary to make OS/2 competitive as a server platform for any Intel based server - it already outperforms them (think about that... on a piece of modern hardware with 2-3GHz CPUs, my CPU utilization would all be measured in fractions of a percent... then again, most of the time it is on at least 3 of the CPUs at 550MHz - in the meantime, the Linux box we were using for startreknewvoyages.com is usually over 40% CPU utilization - and is serving only one other site which gets 1/100th of the traffic of New Voyages - my server is serving over a dozen other sites, as well as a 100GB and growing, continuously accessed, FTP server).

If you want to discuss the merits of a 64bit kernel, this or other kernel related threads is the place... but you should truly understand the reasons we need one... and honestly, other than having more available threads to serve more concurrent connections, and more memory to cache the data... I dont see any. Even with the tremendous popularity of Star Trek New Voyages, my server sits here wondering why it has nothing to do...




So... now give me reasons why it DOES need a 64bit kernel... keep in mind, I will ignore any references to fighter jets, or software projects that dont exist or cars or any other vague and entirely unrelated reference you make.






Here's another place to discuss it:
http://www.os2world.com/component/option,com_smf/Itemid,63/topic,919.0/

And here (this thread should only be about bounty related aspects, such as what needs to be accomplished to meet the bounty, etc):
http://www.os2world.com/component/option,com_smf/Itemid,63/topic,83.0/


There ya go! Now that you have 3 threads (and there are more floating around), you can stop turning every question about OS/2 or eCS into a discussion about a 64bit kernel and fighter jets...



Screenshot below (CPU1: 0.8%, CPU2: 9.4%, CPU3: 6.3%, CPU4: 1.2%)


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


RobertM

So... here's a reason we DO need a new kernel (oddly, it's the same reason, from a different perspective):

I've noticed that my tests on my server are really mostly irrelevant to real world conditions. It doesn't matter how many connections I can spawn at once from my client machines for it to serve. It's a really nice number and all but...

My client stations are on the local network. That means latency is low (0-10ms), and bandwidth is at least 100Mb/s (only one client station has gigabit). I've managed to push 200Mb/s to the gigabit client computer, and maxed out the 100Mb client computers' connections.

What that means is (and why it's not real world testing), the rest of the world does not have a gigabit connection to the server. A web (ftp/sql/pop3/smtp/etc) connection that would take one of my clients a fraction of a second to be served by the server will take many many times longer for any client off the local network.

So, I can easily push 100 requests per second to a local client... let's say that is the site masthead graphic at startreknewvoyages.com (at 50K image size). *I* can do that 100 times in a second. (1) I would need a 50Mb/s upstream connection to serve that to the Internet... and even if I had that (2) the client requesting it would as well... instead, getting that image 100 times takes at best 100 times longer (at 500Kb/s). That's almost two minutes that 100 threads are in use - instead of 1 second that 100 threads are in use.

Thus, sadly, real world constraints mean in order to serve a high amount of traffic (at least with a server such as Domino Go WebServer, which spawns at least one thread per connection), I'd need a lot more threads available.

Or... to show the math... to do the equivalent of serving (completing) 100 requests per second, I would need to be able to spawn 10,000 threads (since each thread would be executing for 100 seconds).

Thus, for added thread handling alone, I see a new kernel as viable and useful.




Another added bonus is I cant seem to see much difference (errr... any... but havent done any scientific tests or anything) for OS/2 and Domino Go WebServer handling 10 threads/requests at once, or 100, or 200. Even when the requests are local clients at full network speed.

The beauty in that is it means OS/2 is quite capable of scheduling an extra few hundred threads at a time... or probably even an extra few thousand... thus, expanding it's thread pool wouldnt be moot (as in some OS's like the NT line, where performance degrades seriously with a lot of threads).




Of course, I dont know how that actually applies in other "real world" apps, like Apache, MySQL, or anything else for that matter. Why I say that is because from what I can see from LDGW's Trace Log, it actually manages (or helps manage) it's own thread pool/thread priority, It grabs as many as you tell it, and "sleeps" the ones it isnt using. Gets a request, assigns it to a thread and "activates" it. Pauses a thread when inactive. Pretty smart piece of software... and from looking at it, (speculation) it was designed for OS/2 (or AIX) and made to run on Windows. It's behavior (and OS/2's threading model - and presumably AIX's) would definitely explain why IBM's own tests show the Domino/DominoGo line running ten times faster on OS/2 than on NT.




At this particular second, I am cranking 300KB/s (3Mb/s), on 24 connections... but the bottleneck is still in the Internet/client side where they arent receiving the data at nearly that speed.

At peak traffic periods, I would need 1500 threads allocated to the web server (and as many as 1500 more for MySQL? to serve database requests needed by those threads - worst case scenario). That's 3,000... add a few hundred more for the OS, and I am nearly at the OS limit of 4095.




Here's reason two:

Instead of seeing my disk lights on solid or flashing all the time at times of high connections, I could have the data cached (either via the web server's caching routine, or via the file system caches)... but a usable/effective cache size would decrease available memory to the point that, even with tons of threads in it's thread pool, the web server wouldnt be able to do anything as it would keep running out of (memory) resources.





Here's a third reason:

There is some great CGI software out there for Linux (Blender)... if it was ported to OS/2, and had the advantage of up to 64 CPUs, and thousands of threads, it would fly, and outperform any other PC OS running it. (PS: Anyone wanna port Blender to OS/2???)

The problem being, while in theory it would be great to be able to run 3 or 4 or 8 renders at once, spawned across as many as 64 CPUs, I cant imagine that doing so would not cause an out of (memory) resources error. (Yeah, even so, running 1-2 renders at once wouldnt, and would still outperform Blender on other OS's... but I am just saying... think about it... it would make OS/2 very very viable in an important "niche" market).




One final reason:

This may not come to pass... but what happens when CPUs no longer support 32bit mode? Yeah, Intel tried that before and it didnt work... but things are slowly changing in the OS world... once other mainstream OS's have fully usable 64bit versions (by fully usable, I also am including apps and such or a virtual 32bit mode to operate 32bit apps). When that happens, OS/2 wont be the reason why hardware/CPU manufacturers continue to support 32bit mode. It happened with the 8/16bit 8088, and the 16bit 802086... one day it will happen - even if it's not soon.




So, even if a 64bit version of the kernel were not made, two enhancements that would be beneficial would be:
(1) Enabling/supporting the PAE memory addressing mode (64GB memory instead of 4GB)
(2) Increasing the size of the thread pool.

Of course, both of those could be handled via a 64bit kernel as well...




Anyone? C'mon. People seem to have many opinions that we do need a 64bit (or at least updated) kernel. Question is why?


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


cyberspittle

What would be nice is to have a 64-bit hypervisor. Then run multiple versions of OS/2 on it as virtual machines.

RobertM

Quote from: James on 2008.08.25, 06:11:42
What would be nice is to have a 64-bit hypervisor. Then run multiple versions of OS/2 on it as virtual machines.

The only problems I see with that are it doesnt really address the resource limits in OS/2 (unless each can share and update a mutual database and web data)... and secondly, I'd be surprised if a hypervisor written for Intel would have sufficiently good thread handling to handle OS/2's thread handling without dropping performance massively. It'd probably be great for a task or 3...

Well, maybe it would be good for a bunch of people... but I'd rather spawn 2000 threads to run.


Here's a question... can a hypervisor emulate an SMP machine at SMP performance? (that is where OS/2 really flies when under load).


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


El Vato

Here is my input, Robert:

What drives the 64-bit kernel architectural model (besides the manufacturers) are the applications --especially the business logic implementation and database related functionality.  Accordingly, your question could be framed as: what enterprise applications are there to run on the OS/2 that need 64-bit support?

Hence the truth table yields a false by default --if taken in strictly logical terms-- because there are essentially no enterprise grade applications for the OS/2, much less 64-bit based. Notwithstanding, analogous to the scholastics of the European "dark ages" who, realizing that their system of cosmology failed to correspond to objective analysis, you began adding equivalent "perfect" circles so as to bring the old system into an sustainable perspective, Robert.

The OS/2 was developed to address yesteryear's relatively inflexible enterprise client-server model of computing.  Notwithstanding, the operating systems' capabilities of today are basically addressing multi-tier architectures where the model, view, controller paradigm (MVC) is the norm that dictates (and demands) the decoupling and more efficient use of resources required for distributed (web or cloud) computing.

From the model, view, controller (MVC) of distributed systems, it is evident that your web serving example leaves out the business logic (model) aspect from the performance/stability metrics --and that is no small thing, needless to say.

Additionally, whereas OS/2 was tuned/optimized for Netfinity servers, it may not necessarily be the case with the GNU/Linux operating system that you are comparing it against.  For the comparison to be more adequate, the GNU/Linux kernel should be customized (rebuild) on its hardware so as to match equivalent optimizations of OS/2 on the Netfinity.

Furthermore, even if you could "pimp" a 32-bit operating system to attempt to take advantage of higher than 4Gigz of memory, the performance gain will not generally be linear but will even decrease in certain situations.  Theory will not necessarily correspond to actual experience --as enterprise class administrators have found out.

I will use another analogy (based on automobiles) that I once threw around and depends on how competitive the OS/2 is to be positioned and or regarded:



  • Passive
    - You do not need to push it to the limit.  As long as there is a supplier (OS/2 licensee X or Y) of ancient (though expensive) parts (device drivers, components) hacked to latch into a newer mechanism (user web applications and or WinXX-derived hacks), you can do without a 64-bit OS/2 --assuming the road (hardware) does not change much.
  • Active - You want to test the limits of the resources of the road infrastructure (64-bit enabled hardware) with your operating system? Then you want a hot-rod 64-bit kernel; for even if you will not use your operating system at full throttle all the time, it will provide important metrics that other x86 operating systems are already being exposed to (and gradually benefiting from).  Those measurements/observations will translate into better "fine tuning", performance and stability insight, application compatibility, and active and true evolution of the OS/2 kernel/subsystems --if the loop is subsequently closed by the developers-- for client and server OS/2 versions.

Although I am repeating myself, or it was not clear from previous discussions, I favour an OS/2 capable of supporting (distributed) enterprise applications: DB2, WebSphere application servers, Oracle DBs, etc., etc.  Accordingly, I lean towards a 64-bit kernel development with binary compatibility with GNU/Linux applications and hardware drivers.  Why ???

Compatibility with GNU/Linux is relatively open, as opposed to the guessing "games" played with the unstable WinXX-derived hacks.  Developing binary compatibility at the 64-bit kernel level will allow OS/2 at least a chance to be considered when hosting an enterprise class application for GNU/Linux --even if not officially supported by the vendor-- and other less critical (but useful) user-land level applications developed for GNU/Linux.

At the same time, it will alleviate the shortage of OS/2 developers for applications and device drivers.

On the other hand, as commented elsewhere and taking a clue from Albert Camus' "Exiled in the Kingdom",  I may be in the minority of OS/2ers who also use GNU/Linux.  I am surprised at how many OS/2ers refer to WinXX in derogatory terms in the forum; yet when the public real time feedJit module logs their operating system as they come to my site --referred from the OS/2 World forum-- those referrals predominantly are using a version of WinXX  :)!!!

Regarding the WinNT versus OS/2 comparison, I have an interesting anecdote.  3-4 years ago I attended an marketing event hosted by the parent OS/2 organization; I had just installed an version of WSEB on my recently acquired T40 laptop and carried it with me everywhere.  I began a conversation with a couple of apparently higher level programmers from the event hosting organization.  And I proudly mentioned that I had WSEB installed in my relatively new laptop.  I was surprised by their reply: since around 1999 or 2000 their unit had migrated to WinNT and they had not seen WSEB boot logo in a looong time!

Point:  The mere announcement of an open source alpha or beta 64-bit OS/2 kernel replacement would be enough to get a publicity boost for the OS/2, which is a necessary prerequisite for potential new users/contributors (in addition to cost and openness). The virtualized network grid fabric being stitched by MS, GNU/Linux sponsors, Solaris, and others' strategies will only allow OS/2 to participate if it effectively alters its core DNA to take advantage of open source technologies like Xen.  Hence, an 64-bit Xen hypervisor, embedded into a modified OS/2 kernel, could host/swap guest operating system (and/or application) images live with other operating systems participating in the "virtualized grid network is the computer" cloud paradigm. 

The standardization of the Apache, MySQL, and PHP-Perl-Python, open source stack (view and controller) components on heterogeneous operating systems will simply funnel the operating system capabilities hosting the applications' functionality (logic or model) componenent into the Web.

And yes, multiple virtual CPUs can be assigned under an implementation of Xen.  However, best practice advice is to limit those to the number of actual single CPUs and/or actual CPUs' cores.  And best practices will dictate that paravirtualized kernels will yield closer to hardware performance than fully virtualized alternatives.  Hence the requirement that an 32/64-bit OS/2 kernel be open to enable paravirtualization of the guest OS/2.


ddan

I don't see the 32-bit subsystem getting removed from new chips, any more than
the 8-bit 8086 instructions have been. At some point, every CPU MUST deal with
units of 8, 16, and 32 bits, whether data or instructions. An entirely new
instruction set would of course break everything existing; just won't be done.

Not to start on my usual rant against Crimosoft, but they're going to face
this problem too, if not already: that there must be some compelling reason to
change from "good enough" when a lot of time has been invested into learning.
The XP market is big enough to support third parties who'll keep XP going for
the foreseeable future. Crimosoft probably won't even be able to stop their
own support of it, let alone kill it off entirely; about the only ways to do
that are new incompatible hardware or an incompatible "Web 2.0".

Now, let's separate the market into servers, which you've already laid out the
potential advantages for, and game playing, and "normal" use, including office
and multimedia. I'll dispense with gaming by saying that the segment might
support development but that the potential benefits for anything resembling
useful work are next to zero.

That leaves "normal" use, which has rather low requirements. For instance, and
setting aside bandwidth limits, people have already noted that high resolution
video provides more detail than is wanted, distracting at best, so we're
already at the practical limits of any NEED. As for office use, no number of
bits will help me type or click any faster; human speed is the definite limit.

If there's no increase of function needed, or in most cases even possible
because of other fixed limits, then the end of the hardware road is in sight,
regardless how much raw speed can yet be obtained. Not only do MOST people
have no use for the raw speed, THERE IS NO USE FOR IT AT ALL, no one has even
come up with ways to squander it except with real time ray tracing (and that's
already done by specialized GPUs). This means ── brace yourself ── that the
world may have to rely on Crimosoft's ingenuity for better software that
actually increases ease and pleasure. ── PFFFT! They'll only cover up their
past crimes like killing OS/2 by continuing to suppress all new invention
because it would expose them as frauds.

Writing an operating system is not as difficult as Crimosoft makes it appear.
OS/2 requires only 8M (as a practical lower limit), as does Linux (in some
non-GUI versions). The Amiga had a full windowing GUI OS in only 512K of ROM
and 512K of RAM. The mystery is what the heck has bloated a thousand times
beyond that minimum, so that 64-bit processors are becoming a necessary
minimum. I don't believe that any application yet actually requires Vista's
incredible array of "operating system" services (DX10); even the latest games
COULD be made to run in OS/2, but programmers seem locked into the notion of
piling layer upon unnecessary layer based on Crimosoft libraries, the true
purpose of which is to lock up the market.

My conclusion is that we're doomed because Crimosoft is big enough to roll
along crushing all that's good and simple and squeezing out all common sense.
Windows 8 will be little more than eye-candy, endless pop-ups and menus, some
hidden, more like playing a video game in itself than a tool. ── And I don't
have much hope on the Linux front because of their uber-geek bias.

What's certain is that we're going to get SOMETHING labeled "new and improved"
64-bits. I expect to thoroughly despise it, not merely because a geezer
comfortable with what I know, but because that next stage WON'T ALLOW me to
even keep what I have. Total incompatibility marketed as a bold break with the
past is the only way to keep the Crimosoft money machine running. They're
already churning out propoganda on this line, just read the comments (the
favorable ones probably by Crimosoft trolls) at http://blogs.msdn.com/e7/

Hmm. Well, this isn't my USUAL rant against Crimosoft, but can't ignore them
when forecasting the future. As I've said before, the ONLY hope that looks at
all reliable for OS/2 is virtualization. So to return to where I started:
don't worry that the 32-bit subsystem will disappear from CPUs any time soon.


El Vato: you blithely ignore the 90 percent or whatever of the market that IS
the "client" side of the traditional server/client model, and that model isn't
going away even with buzzwords bloating up to three instead of two.

Topic has the word "need"; fact is WE don't NEED 64 bits for anything. WE
could get along with Windows 95, which runs BLAZING fast on modern hardware,
needs only a few applications to be updated. (OS/2 is incomparably preferable;
I'm just stating a minimum for what MOST people actually DO: email, pictures,
video, simple games such as the included Solitaire.)

"Cloud computing" is the centralized horror that the PERSONAL computer rescued
us from, temporarily it appears, now the high priests and bureaucrats have
built an even better TRAP. From a data security view, it's insane, giving your
data directly to unknown persons. As for streaming media, hmm, at the client
end, there's already this system called television, far simpler to operate,
requiring mostly a large display device. There is already a version of Linux
in ROM that boots within seconds enough to browse the web, needs only enough
more to pick a television channel.

Your comments illustrate what I see as the real problem: uber-geeks are going
to drive us all to where we don't NEED to go. I'm not saying STOP all
innovation, but that current trends are NOT innovation, merely a difference on
the surface for the sole sake of marketing. There comes a point in technology
where it's not only optimized, but perfected in a practical sense. Consider a
hammer, for instance, they come in a wide range for varied purposes, but
haven't been fundamentally improved in the last century. Software allows FAR
too much variation of interface to be tried upon a few fundamental tasks, but
doesn't change the nature of those fundamental tasks.

RobertM

Quote from: El Vato on 2008.08.25, 14:02:07
Here is my input, Robert:

What drives the 64-bit kernel architectural model (besides the manufacturers) are the applications --especially the business logic implementation and database related functionality.  Accordingly, your question could be framed as: what enterprise applications are there to run on the OS/2 that need 64-bit support?

Or maybe the question is which applications would benefit from (a) being 64bit, and (b) running on a 64bit OS/2. And would that make or keep OS/2 a viable Enterprise Server candidate?

Quote from: El Vato on 2008.08.25, 14:02:07
Additionally, whereas OS/2 was tuned/optimized for Netfinity servers, it may not necessarily be the case with the GNU/Linux operating system that you are comparing it against.  For the comparison to be more adequate, the GNU/Linux kernel should be customized (rebuild) on its hardware so as to match equivalent optimizations of OS/2 on the Netfinity.

Ah, but that isn't quite accurate. After all, it was IBM who did the Linux kernel, threading and other mods to get it running on them (especially neat boxes like the 64way x430).

So, the real questions are: (1) Did IBM put as much effort into optimizing it to the level they optimized OS/2 on those boxes? (2) Was IBM capable of being successful in such an effort (if they tried)? Inotherwords, without having to rewrite the entire kernel and thread manager, were modifications alone sufficient to achieve the same performance results?

And in that respect... keep in mind, it was yet again IBM who did the Windows Server modifications to run on those boxes as well. When Microsoft failed to deliver their 32way and 64way server versions of Windows, IBM took on the task of helping to get Windows server to run on those machines... and then started coming out with partitionable models so that what could/should be a 64way would be used as 4 16way units, then entirely dropped their 64way and 32way models (after all, even with their work, only OS/2 could make full use of the CPU hardware in those configuration). Of course, if what I have read online is correct, that's irrelevant as Windows' threading model is apparently quite abysmal (but, then again, you cant believe everything you read online).

Quote from: El Vato on 2008.08.25, 14:02:07
Furthermore, even if you could "pimp" a 32-bit operating system to attempt to take advantage of higher than 4Gigz of memory, the performance gain will not generally be linear but will even decrease in certain situations.  Theory will not necessarily correspond to actual experience --as enterprise class administrators have found out.

Very true... but the areas it could (or should) be, it would be a great boon for enterprise class daemons... such as being able to allocate a "reasonable" cache size for HPFS386 and JFS; or allowing the web server or sql server to cache it's own data - which for me, would optimally require 500MB-1GB; or being able to create a RAM Drive that can fit everything needed to run the stuff on - like the Linux box currently hosting startreknewvoyages.com does... (without using all of the shared memory).

And as far as I can tell, any memory access scheme and penalties related to using it, would still be far faster than 100-200 simultaneous disk access attempts to a physical, spinning, slow even at striped U320, hard drive(s).


Quote from: El Vato on 2008.08.25, 14:02:07
I will use another analogy (based on automobiles) that I once threw around and depends on how competitive the OS/2 is to be positioned and or regarded:



  • Passive
    - You do not need to push it to the limit.  As long as there is a supplier (OS/2 licensee X or Y) of ancient (though expensive) parts (device drivers, components) hacked to latch into a newer mechanism (user web applications and or WinXX-derived hacks), you can do without a 64-bit OS/2 --assuming the road (hardware) does not change much.
  • Active - You want to test the limits of the resources of the road infrastructure (64-bit enabled hardware) with your operating system? Then you want a hot-rod 64-bit kernel; for even if you will not use your operating system at full throttle all the time, it will provide important metrics that other x86 operating systems are already being exposed to (and gradually benefiting from).  Those measurements/observations will translate into better "fine tuning", performance and stability insight, application compatibility, and active and true evolution of the OS/2 kernel/subsystems --if the loop is subsequently closed by the developers-- for client and server OS/2 versions.

Very true on both points. :)

Quote from: El Vato on 2008.08.25, 14:02:07
Although I am repeating myself, or it was not clear from previous discussions, I favour an OS/2 capable of supporting (distributed) enterprise applications: DB2, WebSphere application servers, Oracle DBs, etc., etc.  Accordingly, I lean towards a 64-bit kernel development with binary compatibility with GNU/Linux applications and hardware drivers.  Why ???

Compatibility with GNU/Linux is relatively open, as opposed to the guessing "games" played with the unstable WinXX-derived hacks.  Developing binary compatibility at the 64-bit kernel level will allow OS/2 at least a chance to be considered when hosting an enterprise class application for GNU/Linux --even if not officially supported by the vendor-- and other less critical (but useful) user-land level applications developed for GNU/Linux.

Total agreement - in addition, my experience at the least, has shown that *nix ports seem to run faster, and more stably on OS/2 than a WinHack. This (your preference) would be an ideal method of creating a greater list of programs available for OS/2, while being able to use it's performance more fully.

Quote from: El Vato on 2008.08.25, 14:02:07
At the same time, it will alleviate the shortage of OS/2 developers for applications and device drivers.

On the other hand, as commented elsewhere and taking a clue from Albert Camus' "Exiled in the Kingdom",  I may be in the minority of OS/2ers who also use GNU/Linux.  I am surprised at how many OS/2ers refer to WinXX in derogatory terms in the forum; yet when the public real time feed Jit module logs their operating system as they come to my site --referred from the OS/2 World forum-- those referrals predominantly are using a version of WinXX  :)!!!

LoL... I've seen that as well... It's funny, since I admin the stnv site (and am thus on it all the time), if you look at their logs, it actually shows OS/2 Warp in the list of operating systems - at no small % of total usage. For me, it does what I need (though I admit, instead of firing up VPC or vBox to test web stuff in IE, I move over to the one XP box we have in the office... but that's simply lack of funds and/or laziness on my part for not putting together a workstation for myself that has a decent amount of memory and a decent CPU to run a virtual session without needing to shut down everything else running to make the virtual session usable).


Quote from: El Vato on 2008.08.25, 14:02:07
Regarding the WinNT versus OS/2 comparison, I have an interesting anecdote.  3-4 years ago I attended an marketing event hosted by the parent OS/2 organization; I had just installed an version of WSEB on my recently acquired T40 laptop and carried it with me everywhere.  I began a conversation with a couple of apparently higher level programmers from the event hosting organization.  And I proudly mentioned that I had WSEB installed in my relatively new laptop.  I was surprised by their reply: since around 1999 or 2000 their unit had migrated to WinNT and they had not seen WSEB boot logo in a looong time!

Point:  The mere announcement of an open source alpha or beta 64-bit OS/2 kernel replacement would be enough to get a publicity boost for the OS/2, which is a necessary prerequisite for potential new users/contributors (in addition to cost and openness). The virtualized network grid fabric being stitched by MS, GNU/Linux sponsors, Solaris, and others' strategies will only allow OS/2 to participate if it effectively alters its core DNA to take advantage of open source technologies like Xen.  Hence, an 64-bit Xen hypervisor, embedded into a modified OS/2 kernel, could host/swap guest operating system (and/or application) images live with other operating systems participating in the "virtualized grid network is the computer" cloud paradigm. 

Umm... wow. That isnt a hypervisor scenario I even thought of... that would open many possibilities.

Quote from: El Vato on 2008.08.25, 14:02:07
The standardization of the Apache, MySQL, and PHP-Perl-Python, open source stack (view and controller) components on heterogeneous operating systems will simply funnel the operating system capabilities hosting the applications' functionality (logic or model) componenent into the Web.

And yes, multiple virtual CPUs can be assigned under an implementation of Xen.  However, best practice advice is to limit those to the number of actual single CPUs and/or actual CPUs' cores.  And best practices will dictate that paravirtualized kernels will yield closer to hardware performance than fully virtualized alternatives.  Hence the requirement that an 32/64-bit OS/2 kernel be open to enable paravirtualization of the guest OS/2.

Nice, and thanks for that... not just a great explanation of what can be done, but how it could be used and beneficial!

Rob


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


obiwan

It might be useful to research what exactly constitutes 64-bit support. The technical implications of a 64-bit kernel are different whether you are talking about Windows or Linux, and OS/2 is neither. Much of this discussion to me seems very abstract, without a clear explanation of exactly what about the OS/2 kernel prevents a 64-bit binary from executing. That is what we are talking about, right? Ability to run 64-bit applications?

We're not talking about rewriting the OS/2 kernel so it does everything the kernel does using 64-bit instructions instead of 16 and 32, right? That seems completely pointless.

Else, are we talking about introducing a completely new kernel with a different abi, and a different name? Then what would we be introducing it to? That confuses me, because that would be a different OS. Not that there is anything wrong with using a different OS, if you want to. Just, if we're talking about a different OS, what does that have to do with OS/2? And why then talk about creating a new one, when others exist?

As far as I know, no-one has done any testcase to see what happens when you try to call a 64-bit instruction on OS/2. It guess it might trap, but why? How is a 64-bit instruction passed to the processor? Does the kernel really prevent it? Could the existing kernel be slightly modified to allow the instruction to pass? If so, that can probably be accomplished.

I really don't know the answers to these questions, and honestly I'm only mildly interested. It just seems to me that if anyone is seriously interested in executing 64-bit code on OS/2, the place to start is to read and experiment with exactly what prevents it, if anything. Not just jump to the conclusion that the kernel must be completely re-engineered for it, and make grandiose plans for this to come about somehow. Just because other OS's now have 64-bit versions of their kernels.

Maybe you all know something I don't know about this, and the reason you aren't saying it is because you assume everybody knows. However, since I don't know, and this piece is clearly missing from every thread, it would be useful to state it, with references. Otherwise I just don't get this.

saborion2

Re:

Quote from: obiwan on 2008.09.02, 06:07:36
It might be useful to research what exactly constitutes 64-bit support. The technical implications of a 64-bit kernel are different whether you are talking about Windows or Linux, and OS/2 is neither. Much of this discussion to me seems very abstract, without a clear explanation of exactly what about the OS/2 kernel prevents a 64-bit binary from executing. That is what we are talking about, right? Ability to run 64-bit applications?

We're not talking about rewriting the OS/2 kernel so it does everything the kernel does using 64-bit instructions instead of 16 and 32, right? That seems completely pointless.

Else, are we talking about introducing a completely new kernel with a different abi, and a different name? Then what would we be introducing it to? That confuses me, because that would be a different OS. Not that there is anything wrong with using a different OS, if you want to. Just, if we're talking about a different OS, what does that have to do with OS/2? And why then talk about creating a new one, when others exist?

As far as I know, no-one has done any testcase to see what happens when you try to call a 64-bit instruction on OS/2. It guess it might trap, but why? How is a 64-bit instruction passed to the processor? Does the kernel really prevent it? Could the existing kernel be slightly modified to allow the instruction to pass? If so, that can probably be accomplished.

I really don't know the answers to these questions, and honestly I'm only mildly interested. It just seems to me that if anyone is seriously interested in executing 64-bit code on OS/2, the place to start is to read and experiment with exactly what prevents it, if anything. Not just jump to the conclusion that the kernel must be completely re-engineered for it, and make grandiose plans for this to come about somehow. Just because other OS's now have 64-bit versions of their kernels.

Maybe you all know something I don't know about this, and the reason you aren't saying it is because you assume everybody knows. However, since I don't know, and this piece is clearly missing from every thread, it would be useful to state it, with references. Otherwise I just don't get this.

Hey obiwan,

I may tend to be in agreement with you that there is no need to re-invent the wheel - thus to repeat
QuoteElse, are we talking about introducing a completely new kernel with a different abi, and a different name? Then what would we be introducing it to? That confuses me, because that would be a different OS. Not that there is anything wrong with using a different OS, if you want to. Just, if we're talking about a different OS, what does that have to do with OS/2? And why then talk about creating a new one, when others exist?

Besides, 64-Bit addressing is nothing new; please see the attached:
QuoteThe PowerPC architecture supports both 64-bit and 32-bit addressing. Early implementations implemented the 32-bit subset and supported only 32-bit applications. Since 1995, 64-bit processors have implemented the complete architecture, allowing full binary application compatibility with existing 32-bit applications and support for new 64-bit applications. The first 64-bit CPUs were used in the AS/400 (now iSeries) platform. Today, 64-bit PowerPC chips can be found in iSeries, pSeries, and JS20 blade systems from IBM as well as G5 Power Macintosh from Apple.

http://penguinppc.org/ppc64/


Thus, from certain perspectives it may appear quite clearly  that these 64-Bit implementations can more or less be built upon when and where necessary since the "Kernel Trees" have been planted already.

Quote

A Word About Kernel Trees

http://penguinppc.org/kernel/#trees


I guess that you have heard the saying that - "Necessity Is The Mother Of All Inventions"!  8)

Best regards,

SAB

ddan

In brief, a 64-bit kernel would expand the ability of the processor to utilize physical memory beyond the absolute 4GB limit imposed by 32 address lines, quite similar to the move from 16 to 32 bits.

In practice, the CPU will still be dealing with the same bits, nibbles, bytes, words, long words, and other quantities (strings of arbitrary length) of data as at present; the 64-bit extensions pretty much only provide for the physical ability of more places to put data and instructions, plus overhead instructions to manage the system.

It'll be a while yet before anyone has a structure approaching the 64-bit limit that is held in RAM, because that'd be approaching the number of molecules on Earth. Even Crimosoft has yet to come up with ways to waste more than 4G, let alone utilize it in a personal system; we are at last reaching the range of "enough for anyone".

saborion2

#10
Re:

QuoteHey obiwan,

I may tend to be in agreement with you that there is no need to re-invent the wheel - thus to repeat
Quote
Else, are we talking about introducing a completely new kernel with a different abi, and a different name? Then what would we be introducing it to? That confuses me, because that would be a different OS. Not that there is anything wrong with using a different OS, if you want to. Just, if we're talking about a different OS, what does that have to do with OS/2? And why then talk about creating a new one, when others exist?

Besides, 64-Bit addressing is nothing new; please see the attached:
Quote
The PowerPC architecture supports both 64-bit and 32-bit addressing. Early implementations implemented the 32-bit subset and supported only 32-bit applications. Since 1995, 64-bit processors have implemented the complete architecture, allowing full binary application compatibility with existing 32-bit applications and support for new 64-bit applications. The first 64-bit CPUs were used in the AS/400 (now iSeries) platform. Today, 64-bit PowerPC chips can be found in iSeries, pSeries, and JS20 blade systems from IBM as well as G5 Power Macintosh from Apple.

http://penguinppc.org/ppc64/


Thus, from certain perspectives it may appear quite clearly  that these 64-Bit implementations can more or less be built upon when and where necessary since the "Kernel Trees" have been planted already.

Quote

A Word About Kernel Trees

http://penguinppc.org/kernel/#trees


I guess that you have heard the saying that - "Necessity Is The Mother Of All Inventions"!   8)

Not withstanding what was quoted above one has to also take into account that if the very "earth" (foundation, soil...) upon which the "Kernel Trees" were planted is not suitable then it (the entire foundation) may have to be completely uprooted (rewritten) if this is perceived; and, since you have stated the following
QuoteIt might be useful to research what exactly constitutes 64-bit support. The technical implications of a 64-bit kernel are different whether you are talking about Windows or Linux, and OS/2 is neither. Much of this discussion to me seems very abstract, without a clear explanation of exactly what about the OS/2 kernel prevents a 64-bit binary from executing. That is what we are talking about, right? Ability to run 64-bit applications?
then, perhaps one may find the following of reading interest and a very good idea of just where we are as far as the development of the OS/2 Kernel (whether 32-Bit or 64-Bit) is concerned

Quote

Inside the OS/2 Kernel

I. Introduction

In this article, I aim to take OS/2 users and developers on a figurative "journey to the center of the earth"-- an expedition into the little-seen but fundamental workings of the system kernel.

The main tool used in researching this article was the Kernel Debugger (KDB), helpfully provided by IBM Corporation, and the accompanying 4-volume "OS/2 Debugging Handbook". My previous article, "Adventures in Kernel Debugging" (EDM/2 Nov. 1996, http://www.edm2.com/0410/kdb.html), gave an introduction to the setup and use of KDB. To get the most out of the present article, you should have an understanding of that material as well as a working knowledge of the Intel 80x86 architecture and memory management features.

For a detailed look at the kernel, it is of course necessary to settle on a specific version of OS/2, since the kernel code has been modified and upgraded along with the other system components in the various versions and fixpack levels. I have chosen OS/2 2.1 for Windows, build 6.514, as the least common denominator of the systems that are likely to be still running. Since most users will be running newer versions, the memory locations, file sizes and so forth will be different from those shown here. But the major data structures and code components have not changed appreciably.
II. Components of the kernel

The main bulk of the kernel code is in the file OS2KRNL, which on the example system is about 730K (retail) or 1Mb (debug version). The supporting cast consists of several smaller files: OS2LDR (28K retail, 37K debug), and the base device drivers SCREEN01.SYS, PRINT01.SYS, KBD01.SYS, and CLOCK01.SYS (3-30K each). Since the source code for the base device drivers is provided on the IBM DDK Developer Connection CD-Roms, we will not delve into them here.

The system DLLs DOSCALL1.DLL, KBDCALLS.DLL, QUECALLS.DLL, SESMGR.DLL, OS2CHAR.DLL and so forth are not part of the Presentation Manager but are not strictly part of the kernel either. Much of this "code" is simply forwarder entries to the Dos* kernel APIs. The SESMGR code does deserve some comment, but I will postpone it to a future article in order to keep this one to a manageable length.

The file OS2KRNL is a standard "LX" format 32-bit segmented-executable file, and as such can be examined with an EXE-file formatting utility. But it is easier and more informative to use the KDB "lg" command, which reads the symbol files supplied by IBM. This reveals the following segments:.........

http://www.edm2.com/0607/kernel.html


Hey obiwan, there is something called - "Division of Labor"; and, just who will take on the job (undertake the "research" that you are talking about). 8)

Kindest regards,

SAB

obiwan


Quote from: saborion2 on 2008.09.02, 17:20:27
Hey obiwan, there is something called - "Division of Labor"; and, just who will take on the job (undertake the "research" that you are talking about). 8)

This very well confirms my point. It indeed takes work just to find this out, yet before we even get there (seemingly) we are already discussing aspirations on orders of magnitude greater scale.

I particularly like ddan's answer. Even if it is possible to access the 64-bit instruction sets while running the existing OS/2 kernel, it seems likely we would be limited to 4GB of memory. But, then, maybe that's still good enough? If not, seems conceivable to work around it, if this end is really desired.

saborion2

#12
Quote from: obiwan on 2008.09.02, 21:19:55

Quote from: saborion2 on 2008.09.02, 17:20:27
Hey obiwan, there is something called - "Division of Labor"; and, just who will take on the job (undertake the "research" that you are talking about). 8)

This very well confirms my point. It indeed takes work just to find this out, yet before we even get there (seemingly) we are already discussing aspirations on orders of magnitude greater scale.

I particularly like ddan's answer. Even if it is possible to access the 64-bit instruction sets while running the existing OS/2 kernel, it seems likely we would be limited to 4GB of memory. But, then, maybe that's still good enough? If not, seems conceivable to work around it, if this end is really desired.

Not quite sure what you are driving at; but, What will the scenario be when the 64-Bit Application that we are talking about borders on cases (figures) involving the U.S. National Debt. Do you think these can be handled quite comfortably by the limited "4GB" memory?  8)

Regards,

SAB

RobertM

Quote from: saborion2 on 2008.09.02, 23:49:34
Not quite sure what you are driving at; but, What will the scenario be when the 64-Bit Application than we are talking about borders on cases (figures) involving the U.S. National Debt. Do you think these can be handled quite comfortably by the limited "4GB" memory?  8)

Regards,

SAB

Why not? OS/2 does that already. If that limit was actually imposed in such a fashion, it wouldn't be able to show a hard drive size nowadays...

Heck, even REXX can handle numbers of a ridiculous size using the NUMERIC DIGITS # statement... I use 30 or 100 digits, with no impact on performance.

Robert


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|


saborion2

Re:

QuoteWhy not? OS/2 does that already. If that limit was actually imposed in such a fashion, it wouldn't be able to show a hard drive size nowadays...

Heck, even REXX can handle numbers of a ridiculous size using the NUMERIC DIGITS # statement... I use 30 or 100 digits, with no impact on performance.

I am curious to know of the particular implementations (the name(s) of the Government or Private Agencies...) where OS/2 performs the functions which you have stated. Another suspect area would be that of the challenges posed by applying complex models in order to forecast the needs in the Multi-Billion Dollars U.S. Health Care Industries and an U. S. Economic Recovery Program - let us say for the next 10 to 25 years. What were (would be) the Analytical Applications and Operating Systems used by the Government or Private Agencies... involved!!!   8)

Regards,

SAB