ES/2, the open source OS/2

[RickCHodgin]

I've been going over the OS/2 API and there is so much in there that can be removed or combined to be made simpler overall.

I really think the OS/2 designers made things much more difficult than it needed to be, and probably some of that was for legacy purposes.  But to create a new OS/2, one with a simple, elegant, yet completely robust and powerful API, to be created for a very powerful operating system ... I think a from-scratch effort would desire to pare down significantly the API, and provide a much better tool for development, debugging, maintenance, and expansion.

There are so many aspects of design I'd like to keep in moving forward, so the open source ES/2 looks and feels very much like OS/2 even down to many of the same settings levels, but I have come to realize that about half of the API can be dropped (at least for the initial release), and then anything that is deemed essential can be added back in later.

There are also new features that need to be added, like full 64-bit support, NUMA awareness, plug-in abilities for expansions available on certain hardware architectures (like Intel's AVX).  And there needs to be support for a modern bootable version on newer BIOSes.  I would also like to create a translation layer between Windows and ES/2, so that a person can #include <es2_windows.h> instead of #include <windows.h> and have it automatically compile and run their 32- or 64-bit Windows apps in ES/2 without changing anything.  I began working on one of these translation layers for Linux to Windows in 2014, and got it nearly 100% designed for my project's needs, and about 40% coded.  I'd like to do the same for Windows, so that we can have access to all software that would run on Windows 7 or Windows XP or earlier.

I'm looking to make ES/2 be a modern OS, but one that is designed in the OS/2 tradition and likeness.  I am looking to do this because we already have OS/2, eCM, and ArcaOS for true OS/2 implementations, and these can be run in a virtual machine if true OS/2 is needed.  But a new thing needs to be addressing current and future hardware trends, and that's where I want to position ES/2.

I am still planning ES/1 as an interim release for simple testing of device drivers that will become part of the full ES/2.  And I would also like to make ES/2 be a non-monolithic kernel, but one made up of discrete servers that all interact with one another, linked together dynamically at bootup, able to have individual modules unplugged, then plugged back in so that updates can occur without rebooting.

I would appreciate any feedback (positive, neutral, or negative).  I'd like to hear what you think.

[Martin Iturbide]

Hi Rick

My opinion is that it will be better to start with something smaller and that can allow you to show progress in a positive way. I would lean more on trying to replicate/clone some of the OS/2 API working over OS/2 and ArcaOS. For example: Trying to create a CPI clone that works over OS/2 and that can little by little replace DOSCALL1.DLL. Like creating a fake "DOSCALL1.DLL" with some of the functions replacements and rename IBM's DOSCALL1.DLL to "DOSCALLX.DLL" and what can not be cloned yet forward it to the IBM's DLL until with time and more development a full "DOSCALL1.DLL"  clone can be achieved. The first version does not has to be perfect, the idea will be to have a starting point of something that can grow with time, and that is also the importance of using an open source license.

Or it can be with some other PM DLLs (PMMMERGE.DLL?) or any other component. So, instead of trying to go directly from scratch and from zero, you can do something over OS/2 that can be growing and showing progress at the same time. It would be interesting to show some accomplishment in a component that can be used under OS/2 and that is open source so it can keep growing.

The idea is not to take you away of the path of you ES/2 goal, the idea is to make something that can work on the current OS/2, show accomplishment, show benefits for the OS/2 user/community, and that can also be part of the bigger ES/2 when you get more time and support to accomplish a full OS.

Like I said before, this is just my opinion.

Regards

[RickCHodgin]

In the next several months I'm going to present a full API outlining the functions I believe should be kept, which kernel modules or DLLs they belong to, along with the functions that are able to be deleted with the reason(s) why.

There really is a lot of unnecessary baggage tied up in the API.  A much cleaner / smaller / more elegant API can be created, one which will allow us to develop powerful applications for everyone to use, and without losing any functionality, and in some cases gaining some new functionality more modern OSes have brought into existence.

[Martin Iturbide]

Hi

The other crazy idea will be to replace the kernel with something Haiku OS, Zircon (Fuchsia OS) or any other open source Microkernel kernel. There is a very initial try to run OS/2 apps on Linux with 2ine but it requires a lot more work to be usable, and also, we are microkernel guys

Changing the kernel and making OS/2 IBM's binaries running over a different kernel, like tricking PM, SOM, WPS and OS/2 apps to run over a non OS/2 kernel, will be an scandal for the community. It is logical since in technical terms developers thinks that the kernel is the OS. On the other side the regular user does not interact directly with the kernel and does not know what a kernel is, while the user have the same apps and same GUI experience he think he is using the same OS he always used.

Sharing a kernel with another community can be a good thing. If we set a crazy example, that we make OS/2 components and apps to run over Zircon it will benefit both communities. Zircon (Fuchsia OS) will finally start to have a user base, while the OS/2 users can benefit from the drivers created for that platform (...and all the money that Google is putting on it ).

A crazy idea (and scandal) like this will also benefit the dream of an open source OS/2 clone and ES/2, since all the effort for cloning CPI required to run on a different kernel can be also reused for anybody that wants to create a OS/2 based open source kernel.

Regards

[RickCHodgin]

The other crazy idea will be to replace the kernel with something Haiku OS, Zircon (Fuchsia OS) or any other open source Microkernel kernel. There is a very initial try to run OS/2 apps on Linux with 2ine but it requires a lot more work to be usable, and also, we are microkernel guys

[snip]

Sharing a kernel with another community can be a good thing. If we set a crazy example, that we make OS/2 components and apps to run over Zircon it will benefit both communities. Zircon (Fuchsia OS) will finally start to have a user base, while the OS/2 users can benefit from the drivers created for that platform (...and all the money that Google is putting on it ).

A crazy idea (and scandal) like this will also benefit the dream of an open source OS/2 clone and ES/2, since all the effort for cloning CPI required to run on a different kernel can be also reused for anybody that wants to create a OS/2 based open source kernel.

One thing to remember about my efforts is I'm not just doing them to do them.  I have a purpose.  I have agreed not to discuss that purpose here on this forum, but it is my #1 driving force.

Remember also that the reason I started the Liberty Software Foundation under the flag and banner I did in July 2012 was following my desire to go to GNU and finish their HURD kernel for them.  I was going to give GNU a viable non-Linux alternative.  In researching that kernel, more about GNU, about the FSF and even Richard Stallman himself, I came across some information that made me abandon that effort and begin anew from a fresh ground-up creation of a new kernel.

In 2012 I had planned to create an open source replacement for Visual FoxPro, which had been cancelled shortly before.  I had expected many developers to come on board and help me complete that project because it was a very popular tool used by many 10s of thousands of developers, and I was looking to release it under a basic clone of the GPLv3 at that time (later changed to more of a Public Domain style license).  However, because of the flag and banner I operated the project under, I literally only had two other developers step forward and agree to help and actually write any code toward the project.  As such, I spent the next 2.75 years of my life working on Visual FreePro effectively by myself, until I made myself sick in early 2015.  I went to the doctors and was examined and they couldn't find anything physically wrong with me.  People told me I had exhausted myself and maybe I did.  I don't know.  I know that since then I've had a more difficult time working and concentrating as deeply as before.  But I've stepped up my exercise, changed my eating habits, stopped drinking soda / colas / caffeinated beverages, and am mostly drinking water now, etc.  I feel better, and here in 2018 for the first time I've truly sort of come back to my own and am able to work almost as before.

I am older now.  When I began in 2012 I was 42, almost 43.  I'm now 49.  I have looked at those 6+ years of development and where I am compared to where I wanted to be.  The flag and banner I operate under has been a great hindrance to me in getting people to work on my projects, but slowly and steadily I am making real progress toward my goals.  I have about 200,000 lines of code written by now in various projects, and am moving forward atop those building blocks so much of the later code I write (in terms of line count) actually does considerably more than the early code.

My goals are to create a new OS/2 that is not OS/2.  I want the existing code bases that were written for OS/2 to be re-compilable under ES/2, but I am not going to design it the same way, and I am not going to support all of the existing API features.  I can add back in any that people need, but initially it's literally going to be about half the overall API, and if applications are written to use those features, no user will be able to tell a difference between running in OS/2 and running in ES/2, save the more modern color scheme, etc.

With ES/2, I am creating an offering for the world.  It's not just another project.  It's a project explicitly given under that flag and banner I'm in pursuit of.  I have the skills and talents to accomplish this, and I am working with great diligence to get to the place where I'm able to devote my full time to these projects, and to even be able to hire other developers.

It is a long-term goal, and I am committed.  If some people could look past the flag and banner, and see the raw me at work with such passion, they would be moved by what I'm doing, and seeking to do.  And while I don't do it for people primarily, they are my secondary goal.  My first goal is to honor that which I refer to when I say I operate under a particular flag and banner.

I am resolute in moving forward.  I have to go through steps to get there so that the foundation is built before the upper structures.  But keep watching.  Provided nothing happens to me, and if it is willed to be so, I will continue on this project and in time will give the world something new they can use that is distinctly different from other things, because of the flag and banner I operate under.

It's going to be a regular OS, very similar in overall design and feel to OS/2.  I will have regular apps.  Be completely usable by people to do their daily work.  It will connect to the Internet and there will be web browsers, email, all manner of socket connections, office apps, databases, compilers, form developer tools, etc.  It will be a complete system.  All of the steps I'm working on now are heading toward that end.  And if I could find developers willing to come on board and operate under that same flag and banner I do/am, then it would be done that much faster.  But to date ... nobody finds enough value in that flag and banner to put in the time.

It will come.  When the time is right it will all break loose very quickly because each of the pieces are close.  If I can get the situation worked out so I can devote more time to these projects, and even hire other developers to help me, then we'll really see some things moving forward.

In time.  Patience.  Persistence.  Vision.  Hard work.  Dreams of the end goal.  These are all required, and they are all in effect.

[David Graser]

Hi Rick

What has made OS/2 so appealing to a lot of us users is the Workplace Shell.  Windows and Linux have nothing like it.  It is one of the reasons I have remained with versions of OS/2 all these years.  So long as WPS works on top of your kernal, I would be happy.  I look forward to seeing what you can accomplish.  Be sure and do away with the SIQ in your version.  Thanks again.

David

[RickCHodgin]

What has made OS/2 so appealing to a lot of us users is the Workplace Shell.  Windows and Linux have nothing like it.  It is one of the reasons I have remained with versions of OS/2 all these years.  So long as WPS works on top of your kernal, I would be happy.  I look forward to seeing what you can accomplish.

Outwardly, it will be identical.  Under the hood, it will be open source.  On the whole, my goal is to compete with Linux, Windows, and Mac OS ... and yes, I am serious.  I want to produce the best product this world has seen, and I want people to come and help me do it.

Be sure and do away with the SIQ in your version.  Thanks again.

Yes.  It's only natural.  Done. :-)

[roberto]

little by little replace DOSCALL1.DLL...

With reference to the doscall1.dll it occurs to me that it could be interesting simply and without modifying anything, recompiling it in a mono processor computer and see if it works better than the current one or not. It would not be a lot of work and the difference could be considerable, or a waste of time.
I have many sys3170, from doscall1.dll in my popuplog.os2, when the browser is closed without warning.
saludos

[Dave Yeo]

little by little replace DOSCALL1.DLL...

With reference to the doscall1.dll it occurs to me that it could be interesting simply and without modifying anything, recompiling it in a mono processor computer and see if it works better than the current one or not. It would not be a lot of work and the difference could be considerable, or a waste of time.
I have many sys3170, from doscall1.dll in my popuplog.os2, when the browser is closed without warning.
saludos

There are 2 versions of doscall1.dll, an SMP one and a UNI one. See x:\os2\install\smp and x:\os2\install\W4. These need to match the type of kernel installed, hopefully you haven't got the wrong one installed   I don't seem to have any sys3170's in DOSCALL1 caused by the browser.

[Mathias]

Dear Rick,

in general I DO like opensource and also the way everybody can contribute/participate in creating a cewl product.
On the other hand though, I see many many forks are being created. Developer power is being "wasted" to do the same thing, just in another project, to "reinvent the bicycle" again.

This is not too much of a big deal in the Linux world, as there are enough functional distributions around. But for OS/2, there is only ONE product left, that is being maintained and looked after, which is ArcaOS. In my opinion all developers with the ability to help this project should concentrate their work force and knowledge to that one product.

By forking OS/2 again, lots of developer power is needed for the other project as well, two or more people working on the same problem in a different project. This is quite inefficient, especially when you see that OS/2 is a dinosaur that has already died out almost, and now comes back to life slowly.

You asked for our opinions, so I will tell you mine: If I had the power and knowledge to actively develop for the core operating system and its features, I would rather go and help ArcaNoae out, but start my own thing. Not only that ArcaNoae already has the knowledge and sources, so developers can jump right into work, they also have the IBM/Microsoft licenses, lawyers and all the necessary stuff, that could get in your way later, when developing "your own thing".
There always is a gap between what's allowed and what could be done.. - So developing stuff is not the only thing, but also to legally create something. Else you might run into lawyers that could take your thing away later on.

Another aspect are drivers and support for hard- & software. Will your forked product be 100% compatible to OS/2? Also in later stages, people might develop software for your forked version, that might maybe not run under native OS/2. The users might end up with two OS/2ish operating systems, where only one of these can run their favourite software... and therefore they would need both of them to be able to run all their stuff.

This does not only sound complicated, it actually is. In the end, users might come to think "Nah.. OS/2 is too complicated and nothing works, always need two versions of them", which might hurt the big scope dream about an up2date OS/2 being liked by its users.
Then this would require even more developer power to write wrappers or to generally make the two OS/2ish operating systems compatible... .... a never ending story.

Meanwhile all developers, that are able to produce nice OS/2 applications, could also have concentrated their time and efforts onto one operating system, that already exists, to make that one even better than it ever was..

Just my thoughts..

[RickCHodgin]

Hi Mathias.  I appreciate your input.  I will consider your guidance.

Some background:  I contacted Arca Noae in 2016 or 17.  They were unreceptive.

And I personally do not believe in a closed source model for OS files of any sort.  Too much riding on what an operating system does to follow after the "please trust me" model.  It needs to be open.

[Martin Iturbide]

Hi Mathias

I disagree in just a little part of what you said, and it is just my opinion.

By forking OS/2 again, lots of developer power is needed for the other project as well, two or more people working on the same problem in a different project. This is quite inefficient, especially when you see that OS/2 is a dinosaur that has already died out almost, and now comes back to life slowly.

You asked for our opinions, so I will tell you mine: If I had the power and knowledge to actively develop for the core operating system and its features, I would rather go and help ArcaNoae out, but start my own thing. Not only that ArcaNoae already has the knowledge and sources, so developers can jump right into work, they also have the IBM/Microsoft licenses, lawyers and all the necessary stuff, that could get in your way later, when developing "your own thing".
There always is a gap between what's allowed and what could be done.. - So developing stuff is not the only thing, but also to legally create something. Else you might run into lawyers that could take your thing away later on.

Arca Noae is doing a great job keeping OS/2 alive by producing bug fixing, new installer and newer/updated drivers. But Arca Noae goals is not to liberate, free or open source OS/2, their goal is to keep the platform running in modern hardware. So, people with a higher dream of an open source OS/2 should not be limited only to help Arca Noae, because AN dream/goal is not to open source OS/2 (yet?).

While some guys dream on more OS/2 drivers that will need to be replaced every 5 to 10 years, there are other people with a longer team dream of finally liberating our self from IBM copyrights and allow the platform to be open and available for all the public as a first step for the platform to adopt new users.

Helping Arca Noae is good, but do not look for leadership there because they have a different goal at this point. If you are going to do any software development for Arca Noae, do it under an open source license that can not be close sourced in the future, since we need things that adds up in time (and snow ball), and not things that the source code is secret like we did on the 80's and 90's with IBM's OS/2. As I always said, close source software is future abandonware.

Don't be afraid of forking OS/2. Experimentation may lead to disaster or to discover new things, and this is just software not human cloning, please experiment !!!!

Regards

[RickCHodgin]

As I always said, close source software is future abandonware.

History is proving this to be completely accurate ... save big money operations like Microsoft, Oracle, Apple, etc.  But I do believe that even big money operations will eventually become abandonware because people will want the free and open source models that support them, and are "good enough" to get the daily jobs done.

[Mathias]

*g* Don't get me wrong guys; I'm not against open source. - Not at all infact. - It's just developer power and knowledge in projects with "low global interest" should imho be concentrated and not spreaded wide, as the progress in whole could be faster witih more devs working on one project. May it be open or closed : )
I doubt though, that ArcaOS will ever be opensource, since it relies on a closed source base, that is even licensed by thirdparties.

Years ago I always wanted the Warp 3 feeling on a linux desktop, but could never find a Window Manager, that could deliver the look & feel. In the end this might be the outcome of an open source project.. which itself would be great already! - But then again.. there is so much more around that, what OS/2 is all about. That would take years and years of developer power to rebuild, if it can be made at all. Developer power is needed. It's all fine, when you have enough people at hand, that work together nicely. But many open source projects start out nicely with flags flying, and 5, 10, 20 devs at hand... but as time goes by, people become interested in other things, develop a real life, gets occupied there, and it's hard to find replacement in such a niche product, that OS/2 is. Infact the number of people who know OS/2 is getting smaller and smaller, people that could develop software for it ... even worse.

In the end, the main attraction for devs to get known to OS/2, and actually produce something usable and user friendly, is interest at first. Then, over years, interest might wander off, and what's left is money to keep the majority of people going on. That's the point where closed software was in an advantage until now, but today we have platforms like patreon and co, where opensource does not necessarily mean, developers do everything for free..

Anyway... I'd be happy to see such a project, but as a developer I can somewhat guess the ammount of work that is necessary to come to that point of functionality that we have today in OS/2.
That's another reason, why I suggested to rather invest time and efforts into already existing projects. "where everybody else is", to take on the big task to bring OS/2 onto new computers, while still keeping backward compatibility for older machines..

[RickCHodgin]

Something else I'm going to do with ES/2 is provide multiple pathways to achieve the same thing, meaning more than one technology solution to accomplish something that works well in one particular technology, allowing a way around catastrophic bugs or errors or security issues in a particular technology.

This is prompted by Java's recent changes by Oracle, such that the way Java's been for 20+ years will no longer be that way due to Oracle choosing to change their model.

In this example, ES/2 may have supported Java, but it will also allow other technologies to exist besides Java so they can be supported natively as well, allowing users to leave Java easily if something comes along they do not like.

In short:  I will not be married to any particular hardware or software, save the tools that I create myself for development, all of which will be open source and able to be inspected and repaired by all at any time.