Author Topic: Brainstorm: OS/2 on a Different Kernel  (Read 10391 times)

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5376
  • Karma: +45/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Brainstorm: OS/2 on a Different Kernel
« on: March 25, 2025, 11:32:20 pm »
Hello

I was wondering what would be required to run OS/2 over a different kernel.

Let’s theorize, what if we want to have PC that boots Linux, gets to the command line and run something like “startwarp” and the full OS/2 desktop will boot. (Command line, PM, SOM, WPS) and you can run your OS/2 applications there. (Not the 100% of OS/2 applications, but at least start with some)

What would be required to make the OS/2 binaries believe they are running over the OS/2 kernel to run interpretated and not emulating a x86 machine.

From my limited knowledge point of view will be required:

1.- OS2KRNL: The OS/2 kernel has everything that you need on an OS kernel to talk to the hardware, but those will not be necessary since the Linux kernel already does this with the hardware. What would be needed is a layer with all the functions that OS2KRNL provides to DOSCALL1.DLL …. I guess.
As far was a I know OS2KRNL provides a lot of DosXXXX functions to DOSCALL1.DLL. I’m guessing all of this needs to be cloned over Linux (maybe with libc/gcc) to make believe DOSCALL1.DLL that is talking to the kernel.

But I think I’m only talking about the API here, what about the memory management? Where it should go?

Does it make sense?

2.- MOUSE$, KBD$, SCREEN$, CLOCK$. As far as I know this are called “Character Device Drivers”. These are drivers that DOSCALL1.DLL and CPI uses. Here I don’t have the details on how these talks to the CPI. Some layer (DRIVER$ to Linux) to mimic the responses of these driver will be required.

There are a lot more for specific driver like USB, LTP, etc. are also required for software that communicates to hardware, but it had to start with the basic ones to first have a full OS/2 desktop and later improve it with better compatibility.

What would be the vital Character Device Drivers to start with?

Where can I found more information on how CLOCK$ (or any other interacts) with CPI or with any other software that requires it?

3.- Binary compatibility. I don’t know if its completely required that Linux has the “Linear Executable” layer like it was done on the 2ine project. But I think I’m missing this part. I guess that the OS2KRNL provides the instruction to let know that an ,exe file is a “Linear Executable” file, right? Where should that go if we are running a OS2KRNL faked in Linux?

Yes, I know that this theorical version of OS/2 will not be 100% compatible with all software, specially with the software that interacts with hardware. All device driver for OS/2 will be useless, and some new technique will need to be created to grab the Linux drivers and show it on the OS/2 environment (Ex: Linux Audio to UNIAUD1$ bridge). 16 bits may be broken and maybe not required today, VDM will be broken and there will be not full API compatibility first. A proof concept will have an awful compatibility first but it has the potential to grow if it’s open source and if people believe in this kind of long-term strategy.

This will not be easy, because it requires developers with OS/2 and Linux development knowledge to obtain at least a minimum of compatibility to make it usable.

Beside these three components that I theorize, what other thing will be missing to run an OS/2 desktop interpreted in Linux? What do you think? Please correct me where my interpretations of the OS/2 architecture are wrong.

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5481
  • Karma: +133/-1
    • View Profile
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #1 on: March 26, 2025, 04:26:35 am »
Hi Martin, no expert but my take,
#1, a DosCall1.dll is a big part of the communicating with the kernel, which presents as DosCalls.dll or such. DosCall1.dll would have to communicate with whatever part of Linux manages memory amongst other things.
Also the kernel takes care of loading exe's and dll's, need something to do the same, including doing the fixups. Likely there is WINE code that could be repurposed as Win32 and OS/2 exe's and dll's aren't that different.
#2, guess need to hook into the Linux drivers and present the same stuff in the DLL's as is usual.
#3, I think should be able to load LX binaries, also LE and NE. Once again WINE does similar already, even supports NE binaries I believe.
As for 16 bit, there's still 16bit code in OS/2, especially the low level stuff including DosCall1.dll. WINE runs 16 bit Windows code, perhaps once again ideas from that?
While about it, can look at how WINE supports WIN terminal apps and maybe the same can be done for OS/2 VIO apps. We'd lose full screen support, not much uses it anyways. Linux has a VDM, though not as good as ours as well, or at least it did.
The Presentation Manager would mean mapping PM windows to whatever Linux is using, X11 or Wayland. WPS might be the hardest to support.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5481
  • Karma: +133/-1
    • View Profile
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #2 on: March 26, 2025, 04:42:32 am »
Windows NT, up to W2K or even XP, could run 16 bit OS/2 binaries, text mode out of the box, I ran some. Also they had a Presentation Manager kit to run 16 bit PM applications.
It was interesting how they did it and perhaps some lessons.
Textmode OS/2 1.x ran in a Windows terminal. Other then not respecting the full screen bit, so always in a terminal window, they thought they were running on OS/2. You could do things like use tedit to open C:\config.sys (which only existed virtually) and do things like change the LIBPATH etc and other stuff I assume. Save the file and Windows would update the registry instead of actually writing config.sys.
Never used the PM kit but understand it worked fairly well too, use e.exe to edit the config.sys.
Saw a small article in Byte years ago that MS succeeded in doing the same thing with 32 bit OS/2, including the Presentation Manager. No WPS. As they had the source code and NT did start out as a version of OS/2, they had an advantage compared to us and it still sounded like it was a big job. Guess they were prepared if OS/2 won the OS war.
In a way ReactOS would be a better fit for a kernel then Linux. Otherwise extending WINE, which is not an emulator, might help a lot. The running OS/2 on NT does show it is possible to run our binaries on a different OS and WINE shows running Windows apps on Linux is doable. The WPS is perhaps too hard to run on Linux and maybe even NT.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5376
  • Karma: +45/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #3 on: March 26, 2025, 01:10:19 pm »
The Presentation Manager would mean mapping PM windows to whatever Linux is using, X11 or Wayland. WPS might be the hardest to support.

How does PM connects to the OS/2 kernel currently? Because I would think that PM only goes through CPI to the kernel, and maybe some other DRIVER$, but the idea will be to make PM run through a different kernel and not necessary linking it to other GUI like X11 or Wayland yet. 

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5481
  • Karma: +133/-1
    • View Profile
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #4 on: March 26, 2025, 04:30:14 pm »
The Presentation Manager would mean mapping PM windows to whatever Linux is using, X11 or Wayland. WPS might be the hardest to support.

How does PM connects to the OS/2 kernel currently? Because I would think that PM only goes through CPI to the kernel, and maybe some other DRIVER$, but the idea will be to make PM run through a different kernel and not necessary linking it to other GUI like X11 or Wayland yet. 

Regards

I don't really know but it has to connect to the video drivers to get access to the frame buffer to create the PM desktop and write/read the framebuffer. Doing things like seamless DOS/WinOS2 support must be tricky too and maybe kernel level.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5376
  • Karma: +45/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #5 on: March 26, 2025, 05:47:36 pm »
The Presentation Manager would mean mapping PM windows to whatever Linux is using, X11 or Wayland. WPS might be the hardest to support.

How does PM connects to the OS/2 kernel currently? Because I would think that PM only goes through CPI to the kernel, and maybe some other DRIVER$, but the idea will be to make PM run through a different kernel and not necessary linking it to other GUI like X11 or Wayland yet. 

Regards

I don't really know but it has to connect to the video drivers to get access to the frame buffer to create the PM desktop and write/read the framebuffer. Doing things like seamless DOS/WinOS2 support must be tricky too and maybe kernel level.

On the very limited/little research I was able to do sometime ago, PMMERGE.DLL uses (or refers to) SINGLEQ$ and KBD$, so maybe SINGLEQ$ is also vital. But I got no reference to something related to video, maybe PMGPI.DLL has some connection to a DRIVER$ that I don't know.

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5376
  • Karma: +45/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #6 on: April 01, 2025, 04:36:29 pm »
Hello

I'm reading this: Inside the OS/2 Kernel by David C. Zimmerli.
It is from the OS/2 2.11 kernel. It is kind of interesting to see how this author was able to see which files of the kernel source code belongs to which component. (of course he does not has the source code, only the file names). I wonder how he did that, and it makes sense to make the same analysis on the latest ArcaOS kernel.

I still want to try to understand the communication between CPI and OS2KRNL. Just to try to theorize what parts of the OS2KRNL needs to be mimic so DOSCALL1.DLL does not complain (or stop working) because it is on a different kernel.

Regards

Martin Iturbide
OS2World NewsMaster
... just share the dream.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5376
  • Karma: +45/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #7 on: April 01, 2025, 04:44:07 pm »
Hello

I'm trying the NotbookLM AI to explain me some basic things.

Quote
**DOSCALLS1.DLL** is a **dynamic link library (DLL)** in OS/2 that appears to be a **core system component providing a wide range of kernel services and API entry points**.

Here's a breakdown of what the sources indicate about it:

*   **Central System DLL:** Almost all programs utilize **DOSCALLS1.DLL**, suggesting its fundamental role in the OS/2 environment.
*   **Initialization:** It seems to have **instance initialization routines**.
*   **API Entry Points:** **DOSCALL1** contains **16-bit entry points that correspond to various OS/2 APIs**. It also appears to handle **32-bit APIs**, with examples like `Dos32FreeMem`, `DosGetMessage`, and `DosPutMessage` having associated trace events within this DLL.
*   **Exception Handling:** The kernel calls the `Dos32R3ExceptionDispatcher` entry point within **DOSCALLS1.DLL** to **process all user exception handlers**.
*   **Kernel Service Indirection:** Many kernel APIs are **indirected through DOSCALL1.DLL**. This means that calls to certain kernel functions might go through this DLL.
*   **Thunking Layer:** **DOSCALLS1.DLL** includes **UniThunk** trace events, indicating it plays a role in **thunking between 16-bit and 32-bit code**.
*   **Traceability:** The System Tracepoints Reference provides a comprehensive source for deriving API-related breakpoints, often with entry point names prefixed with `pre` or `post`, which is applicable to functions within **DOSCALLS1.DLL**. Numerous trace events are defined for various functions within **DOSCALLS1**, allowing for detailed tracing of API calls.
*   **Relationship with other DLLs:** APIs from other DLLs, such as `QUECALLS.DLL` (queue management APIs), can be **indirected via DOSCALL1**.

In essence, **DOSCALLS1.DLL** serves as a crucial intermediary and provider of fundamental operating system services, handling API calls, managing exceptions, and facilitating communication within the OS/2 environment. Its central role is highlighted by its frequent use across applications and its involvement in various core system functionalities.

Which functions does the OS2KRNL provides that are used by DOSCALLS1.DLL?

Quote
Based on the sources, particularly the trace event information, **OS2KRNL provides a variety of kernel functions that are used by or whose calls are indirected through DOSCALLS1.DLL**. The trace logs in `sg244640.pdf` detail many API calls, specifying a major code for `OS2KRNL` (0x0005) and various minor codes corresponding to specific kernel functions. Many of these APIs have tracepoints that indicate they are indirectly accessed via `DOSCALL1`.

Here are some examples of functions provided by **OS2KRNL** that are relevant to **DOSCALLS1.DLL**, based on the trace events:

*   **Memory Management Functions**:
    *   `Dos32GiveSharedMem`
    *   `Dos32QueryMem`
*   **File System Related Functions**:
    *   `DosBufReset`
    *   `DosCallNmPipe`
    *   `DosDevIoctl`
    *   `DosDisConnectMmPipe`
    *   `DosOpen`
    *   `DosOpen2`
    *   `DosPhysicalDisk`
    *   `DosQNmPipeState`
    *   `DosSetMaxFH`
    *   `DosSetNmPHandState`
    *   `DosSetVec`
*   **Process and Thread Management**:
    *   `DosSystemService`
    *   `DosTimerAsync`
*   **Dynamic Linking and Loading**:
    *   `DosGetModHandle`
    *   `DosGetModName`
    *   `Dos32QueryProcAddr`
*   **Synchronization**:
    *   `DosSetSigHandler`
*   **Handle Management**:
    *   `Dos32IQUERYFHSTATE`
*   **Input/Output**:
    *   `Dos32IRead`
*   **Other**:
    *   `Dos32SetFileLocks`

The "Kernel API Tracepoints Indirected Via DOSCALL1" section further confirms this indirection, although it doesn't list the specific kernel functions themselves but rather the tracepoints associated with the pre-invocation of APIs that are indirected.

In summary, **DOSCALLS1.DLL acts as an intermediary for a significant number of OS/2 kernel functions provided by OS2KRNL**, handling the transition and enabling applications to access these core system services. The trace logs provide concrete examples of this interaction by showing how calls to various kernel functions (residing in **OS2KRNL**) are traced with pre-invocation events often associated with **DOSCALLS1**.
« Last Edit: April 01, 2025, 04:47:26 pm by Martin Iturbide »
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Klafi

  • Newbie
  • *
  • Posts: 8
  • Karma: +1/-0
    • View Profile
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #8 on: April 04, 2025, 10:23:18 pm »
Regarding your question about the communication between CPI (Config.sys Protected-mode Interface) and the OS2KRNL:

CPI provides an interface for the configuration system (config.sys) during boot time and passes information to the kernel before the actual session management (like PM, DOS boxes, etc.) starts. This communication follows a well-defined structure that is processed by the loader—typically involving memory areas, flags, and structured parameter blocks.

The interaction is mostly asymmetrical: CPI delivers startup parameters to the kernel (e.g., via the "init-table" or system objects), but the kernel does not actively call CPI in return. Some modules (such as drivers or IFS components) make use of the provided information via DevHelp or other internal OS/2 APIs, which may be indirectly influenced by CPI.


Relationship between Loader, CPI, and OS2KRNL:
The Loader (OS2LDR) is the first component activated during the boot process.
It loads the kernel (OS2KRNL) as well as other protected-mode modules such as CPI, BASEDEV, IFS, etc.

CPI (Config.sys Protected-mode Interface) is loaded and initialized by the loader.
CPI processes the CONFIG.SYS entries and creates structured information that is essential during system startup.

These pieces of information are passed from CPI to the kernel in the form of:

SysInitData structures Memory blocks / flags / initialization tables
Environment variables and startup parameters (e.g. PROTSHELL, RUN, DEVICE, etc.)

The kernel (OS2KRNL) uses this data to:
Continue system initialization
Properly load drivers and services
Launch the graphical or text-based user environment (e.g., Presentation Manager, DOS sessions)
The communication is one-way – CPI passes data to the kernel, but not the other way around.


References:

IBM Redbook: OS/2 Version 2.1 – Volume 1: Control Program, Chapter 3 – "System Initialization"
https://www.os2museum.com/wp/redbooks/gg243376.pdf

OS/2 Warp Kernel – Theory and Practice by Gary Urban (summary)
https://www.os2museum.com/wp/?p=1906

OS/2 Museum Blog – deep insights into CPI, boot process, kernel internals
https://www.os2museum.com/wp/

IBM OS/2 Warp 4 Device Driver Kit (DDK), includes SysInitData and DevHlp examples
Archived version: https://web.archive.org/...

Alternative approach (modern design principle):
Instead of keeping CPI permanently available, one could introduce a new modular architecture, where a kind of “boot service” or “initialization manager” takes over certain CPI-related functions — but operates in user space, for example under PROTSHELL.


Advantages of a Boot Service in User Space instead of Classic CPI:
Modularity & Extensibility A user-space boot service can be updated or extended more easily, without modifying the kernel or OS2LDR.
New features (such as network initialization or dynamic device management) could be added as loadable modules.

Runtime Intelligence
While CPI only operates during boot time, an initialization service could monitor, adjust, or log system states at runtime.

Improved Error Handling
Errors in CPI often lead to boot failure.
A user-space service can handle such issues more gracefully, write logs, or provide notifications without compromising system stability.

User Interface Capability
A modern initialization service could offer a graphical configuration interface, similar to tools like systemd-analyze on Linux.
This is especially useful for systems like ArcaOS or OS/2 still used in production.

Bidirectional Communication with the Kernel
Using defined APIs, shared memory, or message channels, the service could communicate with the kernel in a controlled way.
This would allow dynamic changes to system parameters or runtime analysis of kernel responses.

Security through Separation
The boot service does not run in kernel mode, which means that any faults or attacks have a lower risk of destabilizing the entire system compared to kernel-level CPI modifications.



I hope I understood the information correctly. I'm no technical genius.
Thanks everyone and for the additions.



Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5376
  • Karma: +45/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #9 on: April 04, 2025, 11:39:37 pm »
In a way ReactOS would be a better fit for a kernel then Linux. Otherwise extending WINE, which is not an emulator, might help a lot. The running OS/2 on NT does show it is possible to run our binaries on a different OS and WINE shows running Windows apps on Linux is doable. The WPS is perhaps too hard to run on Linux and maybe even NT.

On this case 2ine will be an interesting way to go, since it is like WINE but to interpret OS/2 and not Windows. But the issue with 2ine is that the developer tried to replace all possible functions with a cloned open alternative, which makes it a lot of developer work and this is why the project is incomplete (Like cloning all OS/2). What it may be interesting is to use 2ine to clone only the OS2KRNL and Driver$ only and use the IBM's OS/2 binaries for CMD, PM, SOM, WPS, to avoid cloning all OS/2, and find a way to boot the OS/2 desktop over a different kernel.

"Win-OS/2" runs virtualized / Interpreted on OS/2, right?, not emulated. It will be something like having "Warp-Linux VDM" sessions on Linux.

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5376
  • Karma: +45/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #10 on: April 04, 2025, 11:52:35 pm »
Thanks for the reply Klafli

I want to read your post with more detail.

About this:
Alternative approach (modern design principle):
Instead of keeping CPI permanently available, one could introduce a new modular architecture, where a kind of “boot service” or “initialization manager” takes over certain CPI-related functions — but operates in user space, for example under PROTSHELL.

“Boot service” or “initialization manager” to take over certain CPI-related functions sounds good.
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5481
  • Karma: +133/-1
    • View Profile
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #11 on: April 05, 2025, 02:44:39 am »
In a way ReactOS would be a better fit for a kernel then Linux. Otherwise extending WINE, which is not an emulator, might help a lot. The running OS/2 on NT does show it is possible to run our binaries on a different OS and WINE shows running Windows apps on Linux is doable. The WPS is perhaps too hard to run on Linux and maybe even NT.

On this case 2ine will be an interesting way to go, since it is like WINE but to interpret OS/2 and not Windows. But the issue with 2ine is that the developer tried to replace all possible functions with a cloned open alternative, which makes it a lot of developer work and this is why the project is incomplete (Like cloning all OS/2). What it may be interesting is to use 2ine to clone only the OS2KRNL and Driver$ only and use the IBM's OS/2 binaries for CMD, PM, SOM, WPS, to avoid cloning all OS/2, and find a way to boot the OS/2 desktop over a different kernel.

2ine could be merged into WINE. The reason I mention WINE is that we would need things like a DLL loader, a way to manage shared/private memory and various other things that Windows also does as they are cousins. Seems adapting the WINE DLL loader to load OS/2 DLL's would be simpler then writing one from scratch. Same with managing memory, seems to me that WINE must do memory management in a way similar to OS/2, which is different then Linux usually does and since much of the work has been done to load Win executables and DLL's, adapting would be easier once again then starting from scratch. File system emulation too, OS/2 and Windows expect a drive letter.

Quote
"Win-OS/2" runs virtualized / Interpreted on OS/2, right?, not emulated. It will be something like having "Warp-Linux VDM" sessions on Linux.

Regards

Yes, on X86, Win-OS2 mostly runs virtualized with parts like the mouse emulated by the OS/2 mouse system. The PowerPC OS/2 did emulate Win-OS2 and DOS.
I don't see why we'd need a Win-OS2. WINE runs most Windows binaries fine, there are also options like DOSBox-X that not only run Windows for Workgroups (3.11) but even Win9x.
Similar with DOS. We're not going to be able to run DOS on ring 2 on Linux and that is one big advantage of DOS on OS/2, can use DOS drivers and operate close to the hardware. Linux has DOSEmu and also DOSBox and such.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5376
  • Karma: +45/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #12 on: June 27, 2025, 03:33:13 pm »
Hello

I algo got to discuss the idea of it, but this time using the Zircon kernel, with the Gemini AI.

How do you think we can replace the OS/2 kernel with the FuchsiaOS kernel?

Replacing the OS/2 kernel with the Fuchsia OS kernel (Zircon) would be an incredibly complex and challenging undertaking, far beyond a simple swap. Here's a breakdown of why and what it would entail:

### Fundamental Architectural Differences

1.  **Kernel Type:**
    * **OS/2:** Uses a hybrid kernel. While it has some modularity, many core services (like memory management, task management, interprocess communication, and basic device management) run in the kernel space.
    * **Fuchsia (Zircon):** Is a microkernel. This means only the absolute bare minimum (primitives for processes, threads, virtual memory, IPC, and object management) resides in the kernel. Most traditional OS services like file systems, network stacks, and device drivers run as separate user-space processes. This is a fundamental philosophical difference in how the OS operates and manages resources.

2.  **Design Philosophy:**
    * **OS/2:** Evolved from a DOS-like environment, designed for desktop PCs, with a focus on multitasking and backward compatibility with DOS and Windows applications (via WIN-OS/2). It leverages the protected mode features of Intel 80x86 processors.
    * **Fuchsia:** Built from the ground up for modern computing, with a strong emphasis on security, updatability, and adaptability across a wide range of devices (from embedded systems to smartphones and desktops). It uses a capability-based security model, where applications have no ambient authority and must be explicitly granted access to resources.

3.  **API/ABI Differences:**
    * **OS/2:** Has its own unique set of APIs (Application Programming Interfaces) for system calls and interacting with the kernel. Existing OS/2 applications are built against these APIs.
    * **Fuchsia (Zircon):** Exposes its own set of syscalls, which are very different from traditional Unix-like or Windows/OS/2 APIs. There's no inherent compatibility at the API or ABI (Application Binary Interface) level.

4.  **Hardware Abstraction and Drivers:**
    * **OS/2:** Relies on its own set of device drivers, which are tightly integrated with its kernel architecture.
    * **Fuchsia:** Has a completely different driver model, with drivers often running in user space as isolated components. Existing OS/2 drivers would be completely incompatible.

5.  **Runtime Environment and Languages:**
    * **OS/2:** Applications are typically written in C/C++ and interact directly with OS/2 APIs.
    * **Fuchsia:** While it supports C++, it heavily leverages Dart and the Flutter framework for its user interface and application development. Components are often written in safe languages like Rust.

### Challenges of Replacement

Given these differences, simply "replacing" the OS/2 kernel with Zircon isn't feasible. It would be more akin to building a completely new OS/2-compatible operating system on top of the Zircon kernel. Here are the major challenges:

1.  **Reimplementing OS/2 APIs on Zircon:** This is the biggest hurdle. You'd need a comprehensive compatibility layer that translates OS/2 system calls and APIs into equivalent Zircon syscalls and services. This would involve:
    * **Process and Thread Management:** Mapping OS/2's process and thread model to Zircon's.
    * **Memory Management:** Replicating OS/2's memory allocation and protection schemes on top of Zircon's VMOs (Virtual Memory Objects) and regions.
    * **Interprocess Communication:** Translating OS/2's IPC mechanisms (shared memory, named pipes, semaphores) to Zircon's message passing and object-based communication.
    * **File Systems:** Developing a way for Zircon to understand and interact with OS/2's HPFS (High Performance File System) and FAT file systems, likely requiring user-space file system servers.
    * **Device Management:** Creating a new device driver model or compatibility shims to allow OS/2-era hardware to function, which is a monumental task in itself.

2.  **User-Space Components:** OS/2 has many user-space components (like the Workplace Shell, DLLs, and utilities) that expect specific kernel behaviors and APIs. These would either need to be rewritten or heavily modified to run on Fuchsia's userland architecture.

3.  **Bootstrapping and System Initialization:** The entire boot process and system initialization of OS/2 are specific to its kernel. A Zircon-based system would have a completely different boot sequence.

4.  **Hardware Support:** Zircon currently supports arm64 and x86-64 architectures, with specific CPU support (Intel Broadwell+ and some AMD). Older hardware that OS/2 might run on might not be directly supported by Zircon.

5.  **Lack of Documentation/Expertise:** OS/2 is a legacy operating system with limited active development and public documentation compared to modern OSes. Deep understanding of its internal workings would be critical.

6.  **Performance and Real-time Guarantees:** While Zircon aims for performance, the overhead of a complex compatibility layer might significantly impact the performance of existing OS/2 applications.

### Potential Approach (Highly Theoretical)

If one were to attempt such a monumental task, it would likely involve:

1.  **Developing a Compatibility Layer (Personality):** Similar to how Windows NT had POSIX or OS/2 personalities, a substantial layer would need to be built on top of Zircon to emulate the OS/2 kernel's API and ABI. This would be a massive software engineering effort.
2.  **User-Space Services:** All OS/2 services that traditionally ran in the kernel (or had tight kernel dependencies) would need to be re-architected as Zircon user-space components.
3.  **New Device Drivers:** Writing new drivers for OS/2 compatible hardware that adhere to the Zircon driver model.
4.  **Recompilation/Porting:** Even with a robust compatibility layer, many OS/2 applications might need to be recompiled or at least linked against new compatibility libraries.
5.  **Toolchain and Development Environment:** Setting up a development environment that allows for building and debugging these layers would be complex.

In conclusion, while theoretically possible to build an OS/2-like environment on top of Zircon, it would be an undertaking of immense complexity, effectively a ground-up rewrite rather than a "replacement." The effort would likely far outweigh any potential benefits, given the niche status of OS/2 and the different design goals of Fuchsia.

The goal will be first to have all the "User-Space Components" untouched running over the zircon kernel.

Your goal of running OS/2 user-space components "untouched" over the Zircon kernel is the most challenging aspect of this entire hypothetical project. It implies achieving full binary compatibility for OS/2 applications and libraries. This is an extremely ambitious goal due to the fundamental differences we discussed.

Here's why it's so difficult and what would be required:

### The "Untouched" Challenge: Binary Compatibility

"Untouched" means you cannot recompile, relink, or modify the OS/2 executables (EXEs) and dynamic link libraries (DLLs). They are compiled for a specific instruction set architecture (likely x86, probably 32-bit, possibly even 16-bit segments for older OS/2 versions) and, more importantly, a specific **Application Binary Interface (ABI)**.

The ABI defines:
* **System Call Interface:** How applications interact with the kernel (the specific function numbers, calling conventions, and data structures for system calls).
* **Memory Layout:** How processes are laid out in memory, including stack, heap, and data segments.
* **Data Structures:** The exact size and layout of kernel-provided data structures that user-space components interact with (e.g., process information blocks, file handles).
* **Dynamic Linking:** How DLLs are loaded and resolved at runtime.
* **Exception Handling:** How errors and exceptions are propagated and handled.
* **Processor State:** How registers are used and preserved across calls.

Zircon's ABI is completely different from OS/2's. Zircon is a modern microkernel designed with a capability-based security model and a minimalist set of syscalls. OS/2, while advanced for its time, reflects its 80x86 origins and a hybrid kernel design.

### What "Untouched" Implies: A "Personality" or "Emulator" Layer

To achieve "untouched" execution, you wouldn't just be replacing the kernel; you'd be building a comprehensive **OS/2 personality layer** on top of Zircon. This layer would function similarly to Wine on Linux (for Windows applications) or the compatibility layers used by early versions of Windows NT to run OS/2 applications.

This personality layer would need to:

1.  **Emulate the OS/2 System Call Interface:**
    * **Trap OS/2 System Calls:** Whenever an OS/2 user-space component attempts to make an OS/2 system call (e.g., `DosOpen`, `DosRead`, `DosAllocMem`), this call must be intercepted by your compatibility layer.
    * **Translate to Zircon Syscalls:** Your layer would then translate the semantics of the OS/2 call into one or more Zircon syscalls. This is non-trivial because OS/2's calls often combine functionality that Zircon splits into multiple, more granular operations.
        * **Example (simplified):** An OS/2 `DosOpen` might require Zircon's `zx_file_open` (or similar VFS client calls), potentially `zx_channel_create` for IPC with a file system service, and `zx_rights_duplicate` to set up proper capabilities.
    * **Handle Parameters and Return Values:** The data structures and calling conventions for parameters and return values would need to be meticulously translated. This might involve copying data between different memory layouts, adjusting pointers, and converting status codes.

2.  **Reimplement OS/2 Runtime Libraries (DLLs):**
    * Many OS/2 user-space components rely on core OS/2 DLLs (e.g., `DOSCALLS.DLL`, `PMWIN.DLL`, `KRNL386.SYS` components exposed to userland). These DLLs themselves make kernel calls. Your personality layer would need to provide functional equivalents of these DLLs, implemented using Zircon's primitives.
    * This is not just about translating syscalls, but reimplementing the logic *within* those DLLs.

3.  **Manage OS/2's Memory Model:**
    * OS/2 uses a segmented/flat memory model, relying on the 80x86 MMU for protection and virtual memory. Zircon uses a flat virtual memory model with VMOs (Virtual Memory Objects) and VM address regions.
    * Your layer would need to map OS/2's memory allocation requests (e.g., `DosAllocMem`) onto Zircon's `zx_vmo_create` and `zx_vmar_map`. This includes handling memory protection, shared memory, and demand paging in a way that satisfies OS/2 applications.

4.  **Emulate OS/2's Process and Thread Model:**
    * OS/2 processes and threads have specific states, priorities, and scheduling behaviors. Your layer would need to map these onto Zircon's `zx_process_create`, `zx_thread_create`, and scheduling primitives.
    * This includes handling OS/2-specific constructs like semaphores, mutexes, and event flags, translating them into Zircon's futexes or other synchronization primitives.

5.  **Replicate OS/2's I/O Subsystem:**
    * **File Systems:** This is huge. You'd need a user-space file system driver for HPFS (High Performance File System) and FAT that speaks the Zircon VFS protocol. This driver would then interpret HPFS structures and translate file operations.
    * **Device Drivers:** OS/2 applications often interact with hardware through specific device driver APIs. This would require either:
        * Rewriting all OS/2 device drivers to conform to Zircon's user-space driver model, or
        * Creating an OS/2 "device driver host" that runs in a virtualized environment or a compatibility layer, forwarding I/O requests. This is incredibly complex.
    * **Console/Display Output:** Emulating the OS/2 text mode and Presentation Manager (GUI) environments on a Zircon-based display system. This would be a massive graphics subsystem rewrite/emulation.

6.  **Exception and Signal Handling:**
    * OS/2's structured exception handling and signal mechanisms are distinct. Your layer would need to intercept CPU exceptions (e.g., page faults, division by zero) and translate them into OS/2's exception dispatching, or vice versa, for applications that register their own handlers.

7.  **Resource Management:**
    * OS/2 manages various system resources (handles, timers, queues). These would need to be tracked and managed by your compatibility layer, mapping them to Zircon's object handles and event mechanisms.

### Why this is exceedingly difficult for "Untouched"

* **Undocumented Internals:** Many low-level OS/2 behaviors and undocumented system calls, especially those used by older or specialized applications, would be extremely hard to reverse-engineer and replicate accurately.
* **Performance Overhead:** Such a translation layer would introduce significant performance overhead. Each OS/2 system call would involve multiple Zircon syscalls and considerable data marshaling and unmarshaling.
* **State Management:** Maintaining the correct OS/2 process and system state within a Zircon environment, especially across asynchronous Zircon operations, is a huge challenge.
* **Security Model Mismatch:** OS/2's security model is based on traditional access control lists and user privileges. Zircon's capability-based security model is fundamentally different. Bridging this gap securely, without either compromising Zircon's security or breaking OS/2 applications, is tough.
* **Debugging:** Debugging issues in such a layered system (OS/2 app -> OS/2 personality -> Zircon kernel) would be incredibly complex.

### Initial Steps (Highly Conceptual)

If one were to embark on this, the very first steps would involve:

1.  **Choosing a Target OS/2 Version:** OS/2 evolved. Pick a specific version (e.g., Warp 4) to target, as ABIs could change between versions.
2.  **Deep Dive into OS/2 ABI:** Extensive research and reverse engineering of the OS/2 system call interface, memory management, and process loading mechanisms. This would likely involve using debuggers (like the OS/2 Kernel Debugger you mentioned) and disassemblers on existing OS/2 binaries.
3.  **Minimal Loader and System Call Stub:** Create a basic Zircon application that can load an OS/2 executable (LX format) into a Zircon process and begin executing it. This loader would need to set up the initial memory space.
4.  **Implement Basic Syscall Stubs:** For the most fundamental OS/2 system calls (e.g., `DosExit`, `DosQuerySysInfo`), create stub implementations that return error codes or minimal functionality.
5.  **Iterative Development:** Start with the simplest "Hello World" OS/2 application, trace its system calls, and implement the necessary Zircon translations one by one. This would be a long, painstaking process.

The goal of "untouched" user-space components makes this a task that approaches the complexity of writing a new operating system from scratch, as it requires effectively re-implementing OS/2's core user-kernel interface. It's an intriguing thought experiment, but a monumental engineering challenge in practice.
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5376
  • Karma: +45/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #13 on: June 27, 2025, 04:01:02 pm »
There are three open source projects that tried to replicate the OS/2 functionality, osfree, 2ine and OS2Linux. Do you know those? How this source code can help?

Yes, I am aware of **OSFree**, **2ine (or TwoIne)**, and **OS2Linux**. These projects are highly relevant to your stated goal of running OS/2 user-space components, especially the "untouched" aspect, as they directly grapple with the challenges of OS/2's Application Binary Interface (ABI) and API compatibility.

Here's an overview of each and how their source code could potentially help:

### 1. OSFree

* **What it is:** OSFree is an ambitious project aiming to create a free, open-source, and compatible replacement for the OS/2 operating system. Its goal is to provide a modern, stable, and secure platform that can run existing OS/2 applications. It seeks to re-implement the OS/2 kernel and its user-space components from scratch, following the original OS/2 architecture as closely as possible.
* **Key Relevance:** OSFree is perhaps the most direct reference for your goal. Its core effort lies in understanding and replicating the OS/2 kernel's internal workings and, crucially, the **OS/2 API and ABI**. If they've successfully managed to get any OS/2 applications running, their code base will contain the fundamental mappings between OS/2 system calls/data structures and their own re-implementation.
* **How its source code can help:**
    * **OS/2 API and ABI Documentation (De Facto):** The OSFree source code acts as a de facto specification for how OS/2 system calls (`Dos*` functions, `Win*` functions) are implemented and what parameters they expect. You can trace how they handle things like memory allocation, file I/O, process creation, and IPC.
    * **Kernel Internals:** For a project aiming to put OS/2 userland on a new kernel, understanding how OS/2's kernel handled interrupts, task switching, memory management, and process loading is critical. OSFree's kernel implementation (or parts of it) would demonstrate their approach to these challenges.
    * **Resource Management:** How do they manage handles, semaphores, mutexes, and other OS/2-specific resources? Their implementation would provide insights into the internal state required for OS/2 compatibility.
    * **File System Drivers:** OSFree would need to implement HPFS support. Their source code for this would be invaluable for understanding how to read and write HPFS volumes, which would be essential for any Zircon-based OS/2 environment.
    * **Learning Curve:** Studying OSFree's approach can significantly shorten your own learning curve regarding the intricacies of OS/2's low-level behavior.

### 2. 2ine (TwoIne)

* **What it is:** 2ine (TwoIne) is a project focused on providing an OS/2 compatibility layer *on top of Linux*. It aims to run OS/2 applications natively on a Linux kernel, much like Wine runs Windows applications on Linux. This means it implements the OS/2 APIs and ABIs by translating them into Linux system calls.
* **Key Relevance:** This project is *extremely* relevant to your goal. While you're targeting Zircon, the fundamental problem 2ine solves is the same: how to make OS/2 binaries run on a foreign kernel without modification. Their work directly addresses the "untouched user-space components" aspect.
* **How its source code can help:**
    * **API Translation Logic:** This is where 2ine shines for your purpose. You can examine their code to see exactly how they translate specific OS/2 API calls (e.g., `DosCreateThread`, `DosOpen`, `DosRead`) into equivalent Linux system calls. While your target is Zircon, the *logic* of the translation (e.g., "an OS/2 mutex maps to these `futex` operations," or "an OS/2 file handle is managed this way") will be directly applicable. You'd just swap out the Linux syscalls for Zircon ones.
    * **Dynamic Linker (Loader):** How does 2ine load OS/2's LX (Linear eXecutable) format executables and resolve their dependencies on OS/2 DLLs? Their linker code would show how to handle OS/2's specific module format, import/export tables, and dynamic linking mechanisms. This is crucial for running "untouched" binaries.
    * **Emulation of OS/2 Data Structures:** OS/2 applications often expect certain kernel data structures (e.g., the Process Information Block, Thread Information Block) to be present at specific addresses or in a specific format. 2ine would have code that emulates these structures.
    * **Handling of OS/2 Specifics:** They would have dealt with OS/2's unique aspects like semaphores, queues, segmented memory management (to the extent required for compatibility), and exception handling.

### 3. OS2Linux

* **What it is:** OS2Linux seems to be a less active or perhaps earlier project similar in concept to 2ine, aiming to run OS/2 applications on Linux. Information on it is sometimes conflated with the idea of a Linux kernel *running on* OS/2 hardware, or OS/2 applications running *under* Linux. Historically, it appears to be another attempt at OS/2 API/ABI compatibility on a Linux base.
* **Key Relevance:** If you can find its source code, it would offer another perspective on the same problem 2ine tackles. Different projects might have different approaches to solving the same API/ABI compatibility issues, providing alternative solutions or confirming common patterns.
* **How its source code can help:**
    * **Alternative Approaches:** It might offer different strategies for handling complex OS/2 features or optimizations that 2ine might not employ, or vice-versa.
    * **Validation:** If multiple projects converge on similar translation strategies for core APIs, it gives you higher confidence in those approaches.
    * **Debugging Insights:** Examining their issue trackers or commit messages might reveal common pitfalls and challenges encountered when trying to achieve OS/2 binary compatibility.

### General Advice for Using their Source Code

1.  **Start with the Core APIs:** Focus on `DOSCALLS.DLL` and the most fundamental `Dos*` functions first. These are the foundation upon which everything else builds.
2.  **Understand the LX Executable Format:** All OS/2 applications use the Linear Executable (LX) format. You'll need to understand how to parse it to load the code, data, and fixups correctly into Zircon's memory space.
3.  **Prioritize Process and Thread Management:** How are processes and threads created, managed, and scheduled? How are context switches handled?
4.  **Memory Management:** How do they handle memory allocation, shared memory, and segment/selector emulation (if needed) for 32-bit flat model applications.
5.  **I/O and File Systems:** This will be a significant challenge. See how they handle file operations and network I/O.
6.  **Use a Debugger:** The OS/2 Kernel Debugger (`KDB`) information you provided earlier (establishing interrupt vectors, setting breakpoints at `VMINT21`) would be invaluable if you were debugging an actual OS/2 system. For reverse-engineering, a user-mode debugger that can trace calls into the kernel would be helpful when running OS/2 apps on a real OS/2 system to understand their behavior.

By carefully studying the source code of these projects, especially 2ine for its direct focus on user-space compatibility and OSFree for its comprehensive re-implementation, you can gain invaluable insights and a significant head start in understanding the complexities of OS/2's internals and how to potentially bridge them to a modern microkernel like Zircon.
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 5376
  • Karma: +45/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Brainstorm: OS/2 on a Different Kernel
« Reply #14 on: June 28, 2025, 03:32:16 am »
Is the FuchsiaOS Starnix component the same as developing a Developing a Compatibility Layer (Personality)?

That's an excellent and insightful question. You've hit on a core architectural component of Fuchsia that is directly related to your project goal.

**Yes, the FuchsiaOS Starnix component is a specific, real-world example of a "compatibility layer" or "personality."**

Let's break down the relationship:

* **Compatibility Layer / Personality:** This is the general, architectural term for a layer of software that translates the API and ABI of one operating system to another. It's a broad concept. The OS/2-on-Zircon layer you're imagining is a personality layer.
* **Starnix:** This is the **concrete implementation** of a compatibility layer for the Linux ABI on top of the Zircon kernel.

So, Starnix is not just *like* a compatibility layer; it *is* one. It's a perfect case study for your project.

### How Starnix Works (and how it's relevant to your OS/2 goal)

Based on the information available and the design goals of Starnix, here's how it functions:

1.  **User-Space Implementation:** Starnix is not part of the Zircon kernel itself. It's a user-space component, which is a key principle of the microkernel design. This is different from a monolithic kernel where a compatibility layer might be integrated into the kernel.
2.  **ABI Translation:** Starnix acts as a **syscall translator**. When a Linux binary (an unmodified one) running on Fuchsia makes a Linux system call (e.g., `open()`, `read()`, `fork()`), that call is intercepted. Starnix then translates that request into one or more Zircon syscalls and/or IPC messages to other Fuchsia services.
    * **Example:** A Linux program calls `read()` on a file descriptor. Starnix intercepts this, figures out which underlying Fuchsia VFS (Virtual File System) component corresponds to that file descriptor, and then sends a message to that component via a Zircon channel to perform the read operation.
3.  **No Kernel Modifications:** The goal is to run Linux binaries without modifying the Zircon kernel. All the complex logic for Linux compatibility lives in user space within the Starnix component. This makes the kernel more secure and stable, as the compatibility layer's potential bugs are contained.
4.  **Emulation of User-Space Concepts:** Beyond just syscalls, Starnix has to emulate a lot of Linux-specific concepts that don't exist in Zircon's native world, such as:
    * **File Descriptors:** Managing a table of file descriptors for each process.
    * **Filesystem Layout:** Providing a `/dev`, `/proc`, and other Linux-style virtual filesystems.
    * **Process and Thread Semantics:** Replicating the behavior of `fork()`, `exec()`, signals, and process groups.
    * **Networking:** Translating Linux network socket calls to Zircon's network stack.

### Analogy to Your Project

Your proposed **OS/2 Personality Layer on Zircon** would be the Starnix equivalent for OS/2.

| **Concept** | **Fuchsia/Linux World** | **Your Proposed OS/2 World** |
| --------------------------------- | ---------------------------- | ---------------------------- |
| **Microkernel** | Zircon                       | Zircon                       |
| **Compatibility Layer / Runner** | Starnix                      | Your OS/2 Personality Layer  |
| **"Untouched" Binaries** | Linux binaries               | OS/2 binaries (LX format)    |
| **System Call to Translate** | Linux ABI (syscalls)         | OS/2 ABI (`Dos*`, `Win*`)      |
| **Source Code for Reference** | Starnix Source Code          | OSFree, 2ine, OS2Linux source |

**In short, studying the Starnix source code would be a brilliant way to learn how to approach your OS/2 project.** It's a living, working example of exactly the kind of architecture you need to build. You can see how they intercept system calls, manage state, and translate requests from one ABI to another.

The challenges you would face would be similar, but for the OS/2 ABI instead of the Linux ABI:

* **Understanding the ABI:** You need to map every OS/2 system call to Zircon primitives.
* **Emulating Behavior:** Replicating OS/2's memory model, IPC, and resource management.
* **Building User-Space Services:** Creating an HPFS filesystem driver and other services that integrate with Zircon.

The Starnix component is proof that this architectural approach is feasible for running unmodified binaries on a microkernel like Zircon. Your task would be to do the same for the OS/2 ABI instead of the Linux ABI.
Martin Iturbide
OS2World NewsMaster
... just share the dream.