Public Discussions > General Discussion

Hybrid "64-Bit" eComStation

<< < (3/3)

Andreas Schnellbacher:

--- Quote from: Roderick Klein on April 23, 2013, 06:47:54 pm ---

--- Quote from: Daniel Caetano on February 18, 2013, 01:23:52 pm ---Hi there.
Yesterday I was thinking about 64 bits and eComStation. The biggest limitation eComStation has when working on 64 bit systems is the 4GiB barrier.

--- End quote ---

The real thing that will give it more breathing space s that we are working on fixing the loading of DLL's into high memory.
But thats not finished currently. Still in R&D stages.

--- End quote ---

Roderick, this thread is about 64 bit. Sooner or later 3 GB won't suffice anymore. I'm sure, the time isn't now, for non-server arrangements. E.g. DB apps (are there still any running with OS/2?) need more RAM for higher performance.

Daniel, the answer from Ivan was the only serious one, so far. No, we won't experience updates to the system to allow more than about 3 GB. IBM support is dead. We will rather manage to get the desktop ported to an OSS system. But also this is far away from reality.

eirik:
Dear all,

backwards compatibility is one issue, adapting to new hardware and computing is another.  Backwards compatibility means, as Neil remarks, the possibility of running older applications.  For many eCS customers this is an important issue.  But as older applications gradually are phased out, surviving on backwards compatibility is a dangerous and non-sustainable long-term strategy.  This does not imply eCS developers should disregard backwards compatibility, but that it should not carry as much weight as it appears to do today.

The future lies in meeting new computing needs on modern hardware.  ACPI is important here.  In the longer term, however, being able to use more than 4 GB of memory will be essential.  The current work that Roderick describes on being able to load more DLLs into high memory is a step in the right direction, but can hardly be called a long-term solution.  With the current speed in hardware development, in just a few years we will be looking at PCs with 10-20 GB in memory with the capability of processing large amounts of data fast.

In addition, connectivity is key.  For laptops (and in the future also stationary PCs)  this entails wireless access, an area where I unfortunately have seen little progress the last few years.  Getting the MultiMac project rolling again as GenMac has its clear limitations (and I know as I have tried hard to get wireless working on my T410), is important to attract new users and hold on to many current users.

User friendliness, flexibility to design the work environment, and low vulnerability towards viruses (high security) are the best selling points of eCS.  Unfortunately, this is not worth much in the longer term if we are unable to meet future computing needs.

Eirik

RobertMauro:

--- Quote from: Doug Bissett on February 18, 2013, 07:20:11 pm ---
--- Quote ---a) Store swapper.dat

--- End quote ---

I haven't seen any indication that swapper.dat ever gets used, when I have 1.5 GB, or more, installed. It may be possible to get it to be used, but I don't think I have enough programs installed to be able to do that, unless I run more than one virtual machine.

--- End quote ---

I have. But I have a few setups that will tax an entire 4GB real memory pool. At 1 and 1.5GB of actual RAM, I've grown my swap file a bit. BUT, I think this is dependent on how the memory pool is divided (such as VIRTUALADDRESSLIMIT settings) and used (such as what's using the HM pool).


--- Quote from: Eirik Romstad on April 25, 2013, 10:59:33 am ---Dear all,

backwards compatibility is one issue, adapting to new hardware and computing is another.  Backwards compatibility means, as Neil remarks, the possibility of running older applications. 

--- End quote ---

Actually, it is far worse than just the app layer. In 32 bit mode, a CPU can run 32 bit and 16 bit instructions. In 64 bit mode, it can run 64 bit and 32 bit instructions. Many of OS/2's lower level services still do 16 bit calls through an interesting but oddly very fast thunking layer (if people remember such implementations in earlier versions of Windows, they'll remember the big performance penalty). That ALSO includes numerous device interfaces.


--- Quote from: Conrad Callan on April 23, 2013, 12:41:31 am ---Just out of curiousity - was there a 64bit version of eComStation very early on in the development cycle? I have a cd here labelled as such but it  won't boot (quite old at this stage)....
Thanks,
Conrad.

--- End quote ---

I would love an ISO of that to play with...


The only option that seems even somewhat viable is to take the "Windows 32 bit server/Certain 32 bit client route" - namely using the buggy PAE stuff on certain CPUs. But that's often problematic.

The other option would be to fully utilize OS/2's virtual addressing mechanisms, which allows multiple 4GB pages. There may be a decent performance penalty involved, I suspect. It would also require a bunch of code changes that may not be possible. Theoretically, per this article, OS/2's "new" kernel can address up to 64 TERAbytes of virtual memory. The questions that begs though, are, (1) is the article accurate and (2) how much additional work (that IBM either never started or never finished) would be required to take advantage of that capability?

Dave Yeo:

--- Quote from: Robert Mauro on May 03, 2013, 07:15:45 pm ---
The other option would be to fully utilize OS/2's virtual addressing mechanisms, which allows multiple 4GB pages. There may be a decent performance penalty involved, I suspect. It would also require a bunch of code changes that may not be possible. Theoretically, per this article, OS/2's "new" kernel can address up to 64 TERAbytes of virtual memory. The questions that begs though, are, (1) is the article accurate and (2) how much additional work (that IBM either never started or never finished) would be required to take advantage of that capability?

--- End quote ---

While in theory the i386 and kernel can address 64 TB of virtual memory, that would be something like 16 billion processes using the full 4GB address space. You have to subtract the kernel address space as well as the PCI space reducing it to 48 TBs. The overhead of keeping track of that would be huge (tables of tables of tables) and the system is still limited to (usually) 3.5 GB real ram and I've discovered that if swapper.dat grows over 2 GB the system crashes with a swap file error so theory and practice are far apart. Using PAE might be able to extend that by quite a bit but everything has to written with PAE in mind including the kernel and device drivers (using a device driver to access PAE might be a workaround). That is the problem with Windows and PAE, one badly behaved driver and the whole system becomes unstable.
I'm not an expert and could be wrong about the above but it seems to me we're stuck in 32 bit land with it being a huge hassle to use the 3.5 GB of address limit and to have 2 programs using 3.5 GB is probably not doable.

RobertMauro:

--- Quote from: Dave Yeo on May 04, 2013, 05:05:39 am ---
--- Quote from: Robert Mauro on May 03, 2013, 07:15:45 pm ---
The other option would be to fully utilize OS/2's virtual addressing mechanisms, which allows multiple 4GB pages. There may be a decent performance penalty involved, I suspect. It would also require a bunch of code changes that may not be possible. Theoretically, per this article, OS/2's "new" kernel can address up to 64 TERAbytes of virtual memory. The questions that begs though, are, (1) is the article accurate and (2) how much additional work (that IBM either never started or never finished) would be required to take advantage of that capability?

--- End quote ---

While in theory the i386 and kernel can address 64 TB of virtual memory, that would be something like 16 billion processes using the full 4GB address space. You have to subtract the kernel address space as well as the PCI space reducing it to 48 TBs. The overhead of keeping track of that would be huge (tables of tables of tables) and the system is still limited to (usually) 3.5 GB real ram and I've discovered that if swapper.dat grows over 2 GB the system crashes with a swap file error so theory and practice are far apart. Using PAE might be able to extend that by quite a bit but everything has to written with PAE in mind including the kernel and device drivers (using a device driver to access PAE might be a workaround). That is the problem with Windows and PAE, one badly behaved driver and the whole system becomes unstable.
I'm not an expert and could be wrong about the above but it seems to me we're stuck in 32 bit land with it being a huge hassle to use the 3.5 GB of address limit and to have 2 programs using 3.5 GB is probably not doable.

--- End quote ---

Hi Dave,

I think there may be a few other things to it than that:

Swapper:
- The swapper limitations are due to the way it is created, written and maintained - it is not using any disk I/O functions that support greater than the original 2GB file size - hence, the system will crash as soon as it tries to allocate above the 2GB barrier. Some experimentation and tracing by others after moving it to a JFS partition helped diagnose this. The particular parts of the OS responsible don't use the new functions.

Memory pages and such:
Page allocations ("16 billion processes") don't need to be done in such a fashion. Keep in mind that the overhead to do such for one 4GB space is already allocated within that space. The only thing required would be a pointer to that address space, which (within that address space) would contain pointers for that space. In total, that's 16,000 pointers.

And of course, that space, as you noted, gets decreased by bus mapping - but it ALSO gets decreased by "fixed" memory mapping, such as the OS area and parts of the shared mapping. In total, all that really needs to happen is the "HMA" could be paged, each "memory set" maintaining its own mappings within its memory area. That allows 4GB pages to hold the HMA and its own pointers.

Other issues:
This would require a number of other changes, to allow seeing and virtually utilizing the 64TB address space.

As for the swap file, I disable it on all systems with 4GB - it's just a waste of disk space and allocation time.

Navigation

[0] Message Index

[*] Previous page

Go to full version