Compatibility with OS/2

[RickCHodgin]

I'm mapping out the features that will be supported with ES/2.  I would like community feedback on how important some features of OS/2 are, such as 16-bit support?  16-bit and 32-bit Win-OS/2 support?  16-bit and 32-bit DOS support?

In addition, what are some must-have's in moving forward?  64-bit is a given.  But what else?  What is OS/2 lacking that would be needed in moving forward?

[ivan]

Rick, I don't think there is a definitive answer because any answer depends on usage.

For example, the manufacturing plants that my old company looks after are using OS/2 to run the machine tool controllers, all of which are 32bit programs and physical PCI cards.  Almost all of the computers now use 64bit processors (the remaining few should be upgraded by the end of the year).

In that instance, as long as a 64bit OS/2 clone will still run 32bit OS/2 programs and act in the same way as OS/2 does at the moment (high stability, up running for several months at a time and operators don't need retraining) I think the company could sell them on the upgrade with little or no certification problems.

[RickCHodgin]

Rick, I don't think there is a definitive answer because any answer depends on usage.

My thinking is this:  For full compatibility we have avenues like ArcaOS or eCM which are moving forward to support modern hardware with full binary compatibility on all DOS + OS/2 + Win-OS/2 code.  But in moving forward with a fully open source kernel, drivers, and app stack, I am wondering if we need full compatibility on the new product?

Would it be enough to allow existing OS/2 code to recompile from source code in ES/2 and then run in that environment the same way?  In that way OS/2 apps can be recompiled and run natively in ES/2 as they have, but will not be binary compatible.

My personal goal is to allow most all existing OS/2 code to recompile with no to very little refactoring.  I want to keep everything pretty much standard with the existing OS/2 design, but there will be changes here and there.  The extent to which I change things will be influenced by this poll.

[Doug Bissett]

You know that there is another approach to this.

If you look at the whole situation, the only reason why we think we need a 64 bit OS, is to run the bloated crap that is ported from other platforms (Mozilla, Office clones etc.). We have lost the ability to create compact, efficient, software for OS/2.. We are also very close to losing the ability to create drivers for modern hardware. Unfortunately, part of that is that the manufacturers won't reveal their secrets, but Linux seems to be able to get the required information, so it isn't impossible.

The OS/2 kernel is still very powerful, even without 64 bit. All it needs are some new programs to replace the crap that comes from elsewhere, and new drivers to run new hardware. A new, 64 bit, clone of OS/2 sounds impressive (and it would be very impressive, IF it can be done), but the software to run is old, and it is that old software that actually keeps OS/2 going, so complete compatibility is required, or there is no point in doing it.

Currently, there has been a break though, by using the technique to use memory above 4 GB, as pioneered by the QSINIT package. That is only the beginning of what could be done to use the expanded memory space. An API to be able to use that would solve a lot of the memory constraints, and the kernel doesn't need to get involved. Many more things can be done.

[RickCHodgin]

You know that there is another approach to this.

If you look at the whole situation, the only reason why we think we need a 64 bit OS, is to run the bloated crap that is ported from other platforms (Mozilla, Office clones etc.). We have lost the ability to create compact, efficient, software for OS/2.. We are also very close to losing the ability to create drivers for modern hardware. Unfortunately, part of that is that the manufacturers won't reveal their secrets, but Linux seems to be able to get the required information, so it isn't impossible.

The OS/2 kernel is still very powerful, even without 64 bit. All it needs are some new programs to replace the crap that comes from elsewhere, and new drivers to run new hardware. A new, 64 bit, clone of OS/2 sounds impressive (and it would be very impressive, IF it can be done), but the software to run is old, and it is that old software that actually keeps OS/2 going, so complete compatibility is required, or there is no point in doing it.

Currently, there has been a break though, by using the technique to use memory above 4 GB, as pioneered by the QSINIT package. That is only the beginning of what could be done to use the expanded memory space. An API to be able to use that would solve a lot of the memory constraints, and the kernel doesn't need to get involved. Many more things can be done.

As I understand it, there is a legal barrier in what we're able to do with the existing OS/2 kernel and drivers.  We are not allowed legally to reverse engineer anything, or to perform binary patches.  Arca Noae was able to get a license to do that for their new release of OS/2, but for the rest of us it's out of bounds.

Can we move forward legally extending the OS/2 kernel?

I've been looking at the OS/2 API and there are parts of it that are daunting, but I know if I can get the base framework developed and begin coding the functions, then other low-level OS/2 developers will come on board and help with completing all of the functions.  I'm also hoping that many original OS/2 developers from IBM will see this project and, in their retirement, come forward and offer their assistance, knowledge, and insight.

I really want to move us forward with a new kernel.  And if full compatibility is a must, then that will be my target ... but I honestly don't know if I can do that because of the legal barrier.  I would have to have intimate knowledge of how things work, and without reverse engineering, decompiling, and examining how the binaries work, I don't think it's attainable legally.  It's why my goal has been to reproduce the entire environment, allowing for recompiles of existing source code with little or no change, but that the OS/2 binaries would not work on ES/2 systems, or vice-versa, except in simple app cases where everything is runtime linked.  But for drivers, and more complex apps it would require the recompile.

[Dave Yeo]

I'm mapping out the features that will be supported with ES/2.  I would like community feedback on how important some features of OS/2 are, such as 16-bit support?  16-bit and 32-bit Win-OS/2 support?  16-bit and 32-bit DOS support?

In addition, what are some must-have's in moving forward?  64-bit is a given.  But what else?  What is OS/2 lacking that would be needed in moving forward?

With a 64bit kernel, 16 bit code simply will not run, the CPU design is such that in 64 bit mode, we lose access to 16 bit code running directly on the CPU. It would be possible to use DOSBOX or even VirtualBox to run 16 bit code in a virtualized environment, but it's not the same and if people want to do that, they may as well run OS/2 in one virtual machine as well.
Also our device drivers are mostly 16 bit, so there is no way to migrate to your new operating system and the truth is, without device drivers of some type to interface with computers, it wouldn't be possible to use it.

[Valery Sedletski]

2dryeo:

This is not true. This is a common misunderstanding. 16-bit code works in 64-bit mode. Dixie, the author of QSINIT (aka Tetris) said that he safely uses 16-bit IDT/GDT entries together with 64-bit and 32-bit ones. This is EFI version of QSINIT. EFI is 64-bit here, and QSINIT uses some 16-bit routines. So, 16-bit code can coexist with 32-bit and 64-bit ones. Indeed, under "16-bit", many people mean 16-bit real mode code, not 16-bit protected mode code, as 16-bit OS/2 and Win16 are. So, VM86 processor mode won't work in 64-bit "long" mode indeed, but 16-bit protected mode code will work.

[RickCHodgin]

With a 64bit kernel, 16 bit code simply will not run, the CPU design is such that in 64 bit mode, we lose access to 16 bit code running directly on the CPU...

ES/2 won't only be a 64-bit kernel.  It will be able to boot into both 32-bit mode and 64-bit mode depending on settings, or boot-time overrides.

Also our device drivers are mostly 16 bit, so there is no way to migrate to your new operating system and the truth is, without device drivers of some type to interface with computers, it wouldn't be possible to use it.

I intend to write a full driver suite, and to create tools to help others port drivers to support as much hardware as is possible from published sources.

My goals for ES/2 are comprehensive.  I'm not trying to hack together something, but to build a base that will be our operating system from here on out.  It is a real effort with real purpose and focus.

[Doug Bissett]

ES/2 won't only be a 64-bit kernel.  It will be able to boot into both 32-bit mode and 64-bit mode depending on settings, or boot-time overrides.

That really doesn't make sense. If a user needs to reboot to use older software, why would you need a new kernel? There are many options to do that already, and there are probably very few OS/2 programs that could be converted to 64 bit programs. In a lot of cases, the source is not available, so it would require a complete rewrite to make those programs work. Programs like OpenOffice, and Firefox, already run better in other operating systems.

I intend to write a full driver suite, and to create tools to help others port drivers to support as much hardware as is possible from published sources.

Why don't you start with drivers, and make it possible to convert to 64 bit, when/if that feature becomes available. Today, we need a lot of new drivers. If we don't get those drivers, there won't be any need for a new kernel because the platform will be long dead (unusable), before the kernel becomes available. There are very few people, spending long hours, trying to keep OS/2 viable, and that doesn't include even dreaming about a new kernel. Today, we desperately need USB 3. eMMC,  GPT, WiFi, (although this one is under way), and more.

As I understand it, there is a legal barrier in what we're able to do with the existing OS/2 kernel and drivers.  We are not allowed legally to reverse engineer anything, or to perform binary patches.  Arca Noae was able to get a license to do that for their new release of OS/2, but for the rest of us it's out of bounds.

It is true, that you cannot reverse engineer the kernel (legally), however OS/2 is designed to be enhanced by simply replacing parts with new parts (Object oriented). Doing so is not going to cause legal problems, as long as you don't copy what somebody else did. Patches are a gray area. From what I understand, you are not allowed to patch something, then distribute the patched file (Arca Noae has an exemption, for OS/2 itself, but not for other things). It seems to be okay, if you distribute a patch that a user can apply to the original file.

As i understand it, one of the biggest problems with drivers, is that they need the device driver development package, from IBM (the DDK), and that is no longer available. Some people do have that package, and they can develop drivers, using it. If that package could be replaced, drivers would be easier to develop. In addition, I expect that there are more than a few DDK licenses, sitting idle, and those could be put to good use, if those who have them would donate them to somebody who can/will use them.

There isn't any point in putting a modern gas/electric hybrid engine in a model T Ford, if the tires are going to wear out next month, and they cannot be replaced. Better to find a way to reliably replace the tires, then worry about replacing the engine.

[RickCHodgin]

Why don't you start with drivers...

GNU did that back in the mid-80s.  They wrote replacement after replacement for Unix-based drivers + programs.  Fast forward to 2017 and they still don't have their own kernel ready for a production environment.  The GNU HURD is unstable, buggy, and not recommended for any type of use.

ES/2 will start with the kernel, then drivers, then apps.  It's a purposeful goal based off the truly excellent core design of OS/2.  It is something I believe in and have wanted for over 20 years.  The time is now.

[Andi B.]

Code: [Select]

...Some people do have that package, and they can develop drivers, using it. If that package could be replaced, drivers would be easier to develop. In addition, I expect that there are more than a few DDK licenses, sitting idle, and those could be put to good use, if those who have them would donate them to somebody who can/will use them....I got the impression some people think the DDK is mystical well hidden secret code no one had access to without paying a truck load of gold in advance. It isn't. It was free to download. At least at the time I got my copy. Of course registration was required and you have to accept the usual lawyers bullshit speech before which most of it I forgot the second I pressed the accept button. Though I'm pretty sure it isn't allowed to share the DDK sources in the public. But it is/was definitely allowed to me -

1) to freely download
2) to use for education and work on OS/2 device drivers
3) to release drivers based on that code

IMHO you hardly can argue giving DDK sources to a friend of mine would be covered by the 'do not share in the public' clause. So friends of mine who accept to not release it into the public can have a copy from me.

[ak120]

I got the impression some people think the DDK is mystical well hidden secret code no one had access to without paying a truck load of gold in advance. It isn't. It was free to download. At least at the time I got my copy. Of course registration was required and you have to accept the usual lawyers bullshit speech before which most of it I forgot the second I pressed the accept button. Though I'm pretty sure it isn't allowed to share the DDK sources in the public.

I agree, this has been true for the times of the OS/2 Developer Toolbox program. The Entry level without additional payment was enough. And even before it was neither ultra expensive nor black magic at all.
The so-called "legal stuff" was only about the third-party components that were included. They should (in some cases) be used for device driver development only. But not all OS/2 device drivers were released in DDK source form and vice versa.

And just a picture for the real mode compatibility...

[xynixme]

I would like community feedback on how important some features of OS/2 are, such as 16-bit support?  16-bit and 32-bit Win-OS/2 support?  16-bit and 32-bit DOS support?

</silence>You'll have the freedom to not support whatever component of an OS/2 install one may be using indeed, in my case (almost) all of the above, but for one it's yet another method to reduce the size of the user base. At the moment I do use MS Office 4.3 more frequently than the number of times I'd really need and want 64-bit support, and on average our collection of software (and investements, if anything of time) will be older than software for more common OSes. If one has to get rid of old software, then one may as well consider using any other OS.<silence>

[RickCHodgin]

</silence>You'll have the freedom to not support whatever component of an OS/2 install one may be using indeed, in my case (almost) all of the above, but for one it's yet another method to reduce the size of the user base. At the moment I do use MS Office 4.3 more frequently than the number of times I'd really need and want 64-bit support, and on average our collection of software (and investements, if anything of time) will be older than software for more common OSes. If one has to get rid of old software, then one may as well consider using any other OS.<silence>

I would like to include everything OS/2 has today.  Every API, every subtlety, every nuance.  If (more) people come on board and help me it can all be done, but I think people will need to see some real progress before they're willing to invest time and labor in an unproven product like this, by a potential crackpot like me no less.  hehe

When I see your posts, your </silence>....<silence> tags crack me up. :-)

[xynixme]

</silence>
FWIW: DOSBox may seem to be a possible answer to reduce a theoretical workload of 64-bit developers, but it isn't. If you're going to suggest a 64-bit DOSBox to play my collection of legacy DOS games, which may be too slow anyway, then it's very likely that I'll discover a faster 64-bit DOSBox for Windows and never switch to a 64-bit DOSBox for OS/2.

Recently AN did mention adding DOS sound support, which is a better niche market strategy than telling potential new customers that there's such a thing as DOSBox if they want to hear their legacy sounds.

Besides of that, mainly a browser may require a 64-bit environment in the future. Without any 16-bit and 32-bit legacy support I'll have to stop using all of my apps, to browse with a new 64-bit browser, so then I may as well switch to another 64-bit OS and renew all of my apps too.
<silence>