OS/2 - ArcaOS 64Bits Kernel Discussion

[Martin Iturbide]

And yes, in my dreams I still think about a 64 bit recompiled OS/2 kernel

I would love to theorize about that in another forum thread. I know it is too hard, but it will be interesting to brainstorm with people with more knowledge on this subject. I will look on other past threads about this topic.

Regards

[Joop]

As far as I know is that the memory is a little less than 4Gb. But this is for the whole OS, no matter if you have 1 kernel or 4 or 8 or... What could be done is each kernel its own little less than 4GB. So with a 4 kernel system you could install 16Gb memory. And 64-bit is a hoax. I have a W10 64-bit from work, everything is bigger including what you make with the applications, but no improvement on the 32-bit applications in my OS/2 system. We complain sometimes about this and that, with Windows 10 its worse. Each day I encounter problems with USB. The Wacom is sometimes working and sometimes not. Now this can easy be fixed, but its not okay. Office from Microsoft is a mess, really, no application does have the same file interface, parts are deaf for the cloud, some programs are even worse then found in the shareware corner. And you don't want to know what the company has to pay for this package.

[Lars]

Physical addresses under OS/2 are also limited to 32-bit. Which means your approach of "4 GB of physical memory per core" cannot work.
You cannot address beyond the physical 4 GB boundary with the current OS/2. You would have to completely reimplement OS/2 memory management to support 64-bit physical addresses (and linear addresses).
Which becomes a massive problem because many new systems "waste" physical address space below the 4GB boundary by simply leaving it unused or by placing large regions of "non system-memory", for example, PCI address space.
And that's why there are machines that have 4 GB of system memory but can only use 2.5 GB or so under OS/2. That's because the other 1.5 GB of system memory are physically addressable beyond the 4 GB physical address boundary.

[Jochen Schäfer]

And 64-bit is a hoax. I have a W10 64-bit from work, everything is bigger [..]

That's not a hoax, it is to be expected. If you want to address 64bit instead of 32bit, you need a bigger pointer. Most C/C++ compilers will make their types bigger, e.g. int going from 32bit to 64bit, so the data grows.  Everything grows, if you switch word size (32bit vs. 64bit). It was like that with the translation from 8bit to 16bit, then from 16bit to 32bit.
So, in a nutshell: Making your word size bigger, makes your executables and your data bigger. That's the downside for getting a so much bigger address space.

[Jochen Schäfer]

Physical addresses under OS/2 are also limited to 32-bit. Which means your approach of "4 GB of physical memory per core" cannot work.

I remember, reading that the 64bit Darwin kernel always presents 32bit software with a 4 GB address space of its own. It does this via virtual addressing. This could be a way to go for OS/2, I think.

[Lars]

Ok,that refers to the logical/linear address (the one the CPU translates to the physical address via the paging mechanism). That could remain 32-bit.
Still, your memory management has to support 64-bit physical addresses. The memory is a global resource, a physical address is unique, in contrast to a logical one.
And this means a rewrite of the memory management.

[Martin Iturbide]

Hi

I slip the topic, since it is a very interesting discussion.

Regards

[Martin Iturbide]

Hi

Since the OS/2 kernel is not open source, I always thought that some alternative for OS/2 kernel evolution issue will be to replace completely the OS/2 kernel for a different 64bit kernel and interpret the OS/2 personality with the IBM binaries over it. People will complain that the kernel is the OS, but I'm starting to believe that OS/2 is more about the experience of the desktop and being able to run the OS/2 applications software that we like and use.

Maybe like Project 2ine tried to do, but not re-implementing all the OS/2 API (like PM) in one shot, but trying to use first the IBM Binaries tricking them to think they are running on the OS/2 kernel while not. 2ine tried to do a Wine for OS/2 under Linux, maybe we should bring the OS/2 components over a different kernel instead. Of course, all OS/2 drivers and application that uses a driver (like some utilities) will turn useless.

I just trying to think how something like that should look from a architectural high level point view.

Regards

[Roderick Klein]

Physical addresses under OS/2 are also limited to 32-bit. Which means your approach of "4 GB of physical memory per core" cannot work.
You cannot address beyond the physical 4 GB boundary with the current OS/2. You would have to completely reimplement OS/2 memory management to support 64-bit physical addresses (and linear addresses).
Which becomes a massive problem because many new systems "waste" physical address space below the 4GB boundary by simply leaving it unused or by placing large regions of "non system-memory", for example, PCI address space.
And that's why there are machines that have 4 GB of system memory but can only use 2.5 GB or so under OS/2. That's because the other 1.5 GB of system memory are physically addressable beyond the 4 GB physical address boundary.

It might be possible to use the ArcaOS loader to use memory in large programs above the 4 GB memory border. But these are rough idea's. It would make more sense to see if you can modify large GCC applications to use such scheme. Do not modify the kernel, modify the application. Hopefully next year we can look further at this and see if simple proof of concept can be delivered.  To be clear again this is an idea.

Roderick Klein

[Andreas Schnellbacher]

How I understand it, its mainly a question of adding PAE to OS/2. 64 bit wouldn't be the next step, and will never be. With PAE there would exist 4 GB address space per application. 32 bit variants of many OS have that implemented.

[Doug Bissett]

How I understand it, its mainly a question of adding PAE to OS/2. 64 bit wouldn't be the next step, and will never be. With PAE there would exist 4 GB address space per application. 32 bit variants of many OS have that implemented.

PAE already exists for OS/2. QSINIT uses it with the RAMDISK, and there is a program interface to use it in other ways (pretty limited, I think). The problem is, that the RAMDISK is the only application that uses it, Nobody else has stepped up to expand on it.

I agree that OS/2 will never be 64 bit. Somebody may think they can do it, but it would be a lifetime project. I doubt if it would be possible to even identify all of the things that would need to be done to make the change.

[Lars]

I don't know how PAE actually works.
But what I know is that the support that QSINIT / the AOS loader offer to access memory > 4 GB physical address is effectively based on offering 512-byte sectors to use by a requester.
This is not surprising (and also pretty clever :-)), as the QSINIT supplied RAM disk driver uses that memory as file data sectors (in the real sense of the word).
However, that also means that they offer their own sector addressing API for use by other applications but there can only be ONE user at any one time which means you will then have to give up on the RAM disk support and you cannot have multiple applications using different parts of that memory at the same time.
In short, this extension is not generic byte addressable support of memory as the CPU understands it. It's not memory management as we know it and therefore applications have to be written to use it.
Rather think of it as if you were accessing external video memory on an adapter card. A special purpose for a special user.

[Andreas Schnellbacher]

PAE already exists for OS/2.

That is wrong. The question is how to make it available for the system, not for a RAM disk. Other OS do this.

What would you gain with a RAM disk, if the usual memory exhausts? Don't mix up memory space with hard disk (or RAM disk) space.

BTW: I can't comment on Lars' post.

[Doug Bissett]

That is wrong. The question is how to make it available for the system, not for a RAM disk. Other OS do this.

It is not wrong. PAE exists. The problem is, that there is no real API to use it. Some programs could use it, as it is (raw memory space), for some things, but that could be expanded, if somebody knows how to do it. The one program, that comes to mind, is CLAMD.EXE, which loads huge amounts of data into upper shared memory space (at least it is UPPER shared space). That could live in PAE memory, with a bit of work. I am sure there are other things that could use it, without having to write a whole API. Another possibility is to use PAE memory for disk cache. At least the source code for JFS, and FAT32, is available.

Perhaps some current API could be redirected to use PAE memory, rather than upper shared memory (for instance). There are possibilities, if someone knows how to do it, and is willing to try. Part of the problem is, that not all computers have more than 4 GB of physical memory, but most new computers do have more, and it could be used. Another part of the problem is that many OS/2 programmers have lost the ability to do much more than port programs from other platforms (which is actually what is causing the problem, in the first place, but is probably necessary), and they just don't know how to do things like this any more. Those who do have the knowledge are far to busy doing more important things.

Any part of the current memory space, that can be eliminated, leaves more space for other things to use. The RAMDISK works very well, but there is a limit to how much space you can actually use as a disk drive that needs to be loaded at boot time, and unloaded at shut down, if you want to use it with data that must to be saved over a boot (in some cases, the RAMDISK can still contain the contents, but there is no guarantee about that).

Don't say "you can't do that". Ask yourself "how can i do that". That sort of thinking produced the QSINIT package, and the QSINIT package has enabled more than just the RAMDISK. It also enables using fragmented memory, that is often found in new computers. It is essential to the UEFI support that is being developed (which includes better support for DOS/WinOS2), and it will probably be used for many more things as the need arises. Small steps lead to greater things.

[Andreas Schnellbacher]

Doug, just try to understand what Lars wrote.