Author Topic: native mainframe mini-clone  (Read 10691 times)

Rich Walsh

  • Sr. Member
  • ****
  • Posts: 418
  • Karma: +30/-0
  • ONU! (OS/2 is NOT Unix!)
    • View Profile
Re: native mainframe mini-clone
« Reply #15 on: September 13, 2025, 01:15:54 am »
can I have a normal executable (.exe), running at the OS/2 fullscreen prompt, that has the privilege required to disable interrupts and switch from PM32 to LM64?

Can a Ring 3 executable execute a Ring 0 priveleged instruction that could blow up the entire system? Hell no!

Quote
If so, what do I need to do to get such privilege on OS/2?

You would need a device driver which operates at Ring 0.  In the UEFI version of ArcaOS, the bootloader stub which remains after the OS is running will switch to long-mode to read the machine's NVRAM variables, then will switch back. While doing so, all of OS/2 must stop because it is 100% *incompatible* with long-mode. Why? Because the kernel and device drivers are all 16-bit code, and the flag that signals "16-bit code" on an x86 signals "64-bit code" when long-mode is enabled.  You can have 16/32 code or 32/64 code but not 16/32/64.

kerravon

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +3/-0
    • View Profile
Re: native mainframe mini-clone
« Reply #16 on: September 13, 2025, 01:00:02 pm »
You would need a device driver which operates at Ring 0.
Ok, in that case, I guess I am after the minimum device driver that's sole duty is to switch from ring 3 to ring 0 - callable from any ordinary app.

That would be enough to prove the concept of allowing me to create my own mini-Win64 that purely uses memory above 4 GiB.

Note that I am only interested in executing non-buggy programs, so I don't mind OS/2 crashing the same way that MSDOS crashed if you run a buggy program.

I simply debug the application.

Running everything privileged has worked fine for me for decades. I'm not trying to make a commercial quality product. Fleshing that out is left as an exercise for the reader or whoever sees a commercial opportunity for something more robust.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5585
  • Karma: +141/-1
    • View Profile
Re: native mainframe mini-clone
« Reply #17 on: September 13, 2025, 06:02:46 pm »
There's some sample drivers at http://hobbesarchive.com/?path=%2fpub%2fos2%2fdev%2fsamples%2fdrivers Perhaps the fastio driver, http://hobbesarchive.com/Home/Download?path=/Hobbes/pub/os2/dev/samples/drivers/FastIO_A1-0.zip can be modified. From the blurb,
Code: [Select]
FastioA$ Device Driver for direct port I/O. Based on Holger Veits EDM/2 fastio driver. Modified for use with watcom. Includes watcom project files and example program. (FastIOA1.0.zip)

kerravon

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +3/-0
    • View Profile
Re: native mainframe mini-clone
« Reply #18 on: September 14, 2025, 05:47:22 am »
Perhaps the fastio driver
Thanks - looking at that, there is something I didn't think of.

It does a far call into the I/O driver, which gives an opportunity for the cs to change, and I'm guessing that while in the driver, it has a privileged cs, but on return to the caller, the original cs is restored to an unprivileged cs. ie it's not that easy to simply say "go privileged".

I guess I can modify the GDT table to make the original cs privileged though. Not sure if ds needs a change too.

Lars

  • Hero Member
  • *****
  • Posts: 1463
  • Karma: +73/-0
    • View Profile
Re: native mainframe mini-clone
« Reply #19 on: September 14, 2025, 11:25:10 am »
You might want to have a look at:

https://hobbesarchive.com/Home/Download?path=/Hobbes/pub/os2/system/drivers/misc/SysCall_3-0.wpi

That is a package that I have written to run code in Ring 0.
It comes with full source code. Which would allow you to adapt/extend it to your needs.

The idea was to have an interfacing DLL to be used by an application. That DLL will return to the caller an entry point that will allow the caller to execute user functions in Ring 0.

It uses the syscall/sysret instructions as the fastest way to enter Ring 0.
« Last Edit: September 14, 2025, 11:26:52 am by Lars »

kerravon

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +3/-0
    • View Profile
Re: native mainframe mini-clone
« Reply #20 on: September 15, 2025, 06:59:48 am »
You might want to have a look at:

https://hobbesarchive.com/Home/Download?path=/Hobbes/pub/os2/system/drivers/misc/SysCall_3-0.wpi

That is a package that I have written to run code in Ring 0.
It comes with full source code. Which would allow you to adapt/extend it to your needs.

The idea was to have an interfacing DLL to be used by an application. That DLL will return to the caller an entry point that will allow the caller to execute user functions in Ring 0.

It uses the syscall/sysret instructions as the fastest way to enter Ring 0.

Looks exactly what I need - thanks!

There are some competing priorities for what I should do next though - a strategy for this being the most interesting at the moment:

https://gcc.gnu.org/pipermail/gcc/2025-September/246666.html

(I need a solution different from the x32 solution which I now know is 0x67 override)

Lars

  • Hero Member
  • *****
  • Posts: 1463
  • Karma: +73/-0
    • View Profile
Re: native mainframe mini-clone
« Reply #21 on: September 15, 2025, 09:43:01 am »
Correction: it is sysenter/sysexit that is used and not syscall/sysret.

For exactly this reason: only sysenter/sysexit are available in x32 for both, Intel and AMD. The interfacing DLL/the Syscall device driver makes no attempt whatsoever to switch from x32 to x64. Everything remains in x32 mode.

For syscall/sysret it is more complicated. I think for Intel it only works for x64 (long mode) (or was it AMD ? I cannot remember ...)
« Last Edit: September 15, 2025, 09:50:39 am by Lars »

kerravon

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +3/-0
    • View Profile
Re: native mainframe mini-clone
« Reply #22 on: September 21, 2025, 09:45:53 am »
Correction: it is sysenter/sysexit that is used and not syscall/sysret.

For exactly this reason: only sysenter/sysexit are available in x32 for both, Intel and AMD. The interfacing DLL/the Syscall device driver makes no attempt whatsoever to switch from x32 to x64. Everything remains in x32 mode.

For syscall/sysret it is more complicated. I think for Intel it only works for x64 (long mode) (or was it AMD ? I cannot remember ...)

Hi Lars.

I'm not sure if you've used the wrong terminology or I'm just confused.

x32 is a target of gcc. I'm not very familiar with it, but I believe it uses x64 instructions, but 32-bit addresses. And it does the latter (as I just found out) by using 0x67 override (which isn't appropriate if you were really running in x86 mode).

So this is very different from x86, and that's why the different target exists. ie it isn't i386.

What you described above sounds like you are describing x86 (aka i386 aka 80386).

Lars

  • Hero Member
  • *****
  • Posts: 1463
  • Karma: +73/-0
    • View Profile
Re: native mainframe mini-clone
« Reply #23 on: September 22, 2025, 05:32:21 pm »
Correction: it is sysenter/sysexit that is used and not syscall/sysret.

For exactly this reason: only sysenter/sysexit are available in x32 for both, Intel and AMD. The interfacing DLL/the Syscall device driver makes no attempt whatsoever to switch from x32 to x64. Everything remains in x32 mode.

For syscall/sysret it is more complicated. I think for Intel it only works for x64 (long mode) (or was it AMD ? I cannot remember ...)

Hi Lars.

I'm not sure if you've used the wrong terminology or I'm just confused.

x32 is a target of gcc. I'm not very familiar with it, but I believe it uses x64 instructions, but 32-bit addresses. And it does the latter (as I just found out) by using 0x67 override (which isn't appropriate if you were really running in x86 mode).

So this is very different from x86, and that's why the different target exists. ie it isn't i386.

What you described above sounds like you are describing x86 (aka i386 aka 80386).

Correct. The system stays in "protected 32-bit" mode, aka x86. It does not change to long mode.

kerravon

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +3/-0
    • View Profile
Re: native mainframe mini-clone
« Reply #24 on: October 13, 2025, 03:08:25 pm »
You would need a device driver which operates at Ring 0.  In the UEFI version of ArcaOS, the bootloader stub which remains after the OS is running will switch to long-mode to read the machine's NVRAM variables, then will switch back. While doing so, all of OS/2 must stop because it is 100% *incompatible* with long-mode. Why? Because the kernel and device drivers are all 16-bit code, and the flag that signals "16-bit code" on an x86 signals "64-bit code" when long-mode is enabled.  You can have 16/32 code or 32/64 code but not 16/32/64.

I think you might have been misinterpreted, and you are in fact correct, but could you please answer the below?

https://bttr-software.de/forum/board_entry.php?id=22728

That’s what I thought, but it is incorrect. Long mode can deal with all three code segment sizes at the same time, as indicated by the very first table in the system programming volume of AMD’s architecture manual (“Operating Modes”). What you cannot have is VM86 mode. I verified PM16 on my current Linux installations last week (from a 32-bit task). One significant piece of functionality is missing, although this is just a limitation of Linux, not the machine architecture: having your own SIGSEGV handler intercept general protection faults from that 16-bit code segment. Unfortunately, the kernel does not understand this kind of code and will kill the faulting process directly instead of invoking its signal handler.


Thanks. Paul.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 5585
  • Karma: +141/-1
    • View Profile
Re: native mainframe mini-clone
« Reply #25 on: October 13, 2025, 07:06:38 pm »
Couple of things. There is a 64 bit LX format that NASM supports. Bird came up with the patch for VirtualBox.
Currently OS/2 can use PAE to a limited degree, mostly used as a ram disk. Things are setup by os2ldr before the kernel launches. Current os2ldr is based on QSINIT, from the latest blurb,
Code: [Select]
QSINIT - 2025-08-10

QSINIT had been updated (2025-08-10):

    Binary: ftp://212.12.30.18/public/QS/QS_LDR.ZIP
    Source Code and SDK:  ftp://212.12.30.18/public/QS/QS_SDK.ZIP

QSINIT is a small 32-bit something (you may call it an operating system, or a "small DOS"), that you can write applications for, and that may act as an OS/2 kernel bootloader.

The author of this software, Dmitry Zavalskov, has reserved the copyright to himself. The binary is released as freeware, the source code as freeware for non-commercial use.

Much of the kernel is actually 16 bit, with 16 bit code limited to 1 GB address space. OS/2 2.x before Warp V4 FP13 (Warp Server also supported high memory earlier) only supported 1 GB of address space with the kernel getting the upper half, so you'd have to work from the lower 512MBs (less as much is used by DLL's) and there are still API's that underneath are 16 bit and have to reside in low memory.

kerravon

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +3/-0
    • View Profile
Re: native mainframe mini-clone
« Reply #26 on: October 13, 2025, 07:27:51 pm »
Much of the kernel is actually 16 bit, with 16 bit code limited to 1 GB address space. OS/2 2.x before Warp V4 FP13 (Warp Server also supported high memory earlier) only supported 1 GB of address space with the kernel getting the upper half, so you'd have to work from the lower 512MBs (less as much is used by DLL's) and there are still API's that underneath are 16 bit and have to reside in low memory.

I'm not sure if we're talking cross-purposes. In a DOS extender, the entire DOS nominally resides in the lower 640k too. That doesn't prevent you from getting access to 4 GiB of memory for your 32-bit program.

Ditto any 64-bit OS/2 extender I created would simply reside above 4 GiB to keep things simple. And the rest would reside wherever OS/2 loaded my executable - lower 512 MiB or whatever.

Note that there is work afoot to stand up a (better) public domain x64 compiler over here:

https://github.com/sal55/langs

(mcc.asm mainly, but also older mcc.c)

There is no public domain notice on either at the moment though. Still fixing bugs that affect my toolchain.

Rich Walsh

  • Sr. Member
  • ****
  • Posts: 418
  • Karma: +30/-0
  • ONU! (OS/2 is NOT Unix!)
    • View Profile
Re: native mainframe mini-clone
« Reply #27 on: October 15, 2025, 08:11:42 am »
the kernel and device drivers are all 16-bit code, and the flag that signals "16-bit code" on an x86 signals "64-bit code" when long-mode is enabled.  You can have 16/32 code or 32/64 code but not 16/32/64.

I think you might have been misinterpreted, and you are in fact correct, but could you please answer the below?

https://bttr-software.de/forum/board_entry.php?id=22728

That’s what I thought, but it is incorrect. Long mode can deal with all three code segment sizes at the same time, as indicated by the very first table in the system programming volume of AMD’s architecture manual (“Operating Modes”).


The author of that posting missed an important point: to have code run in AMD64 compatibilty mode, the host OS must be 64-bit. Given that OS/2 is 32-bit, the CPU can only run in one of the legacy modes (i.e. 16 or 32-bit).




kerravon

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +3/-0
    • View Profile
Re: native mainframe mini-clone
« Reply #28 on: October 15, 2025, 07:13:25 pm »
the kernel and device drivers are all 16-bit code, and the flag that signals "16-bit code" on an x86 signals "64-bit code" when long-mode is enabled.  You can have 16/32 code or 32/64 code but not 16/32/64.

I think you might have been misinterpreted, and you are in fact correct, but could you please answer the below?

https://bttr-software.de/forum/board_entry.php?id=22728

That’s what I thought, but it is incorrect. Long mode can deal with all three code segment sizes at the same time, as indicated by the very first table in the system programming volume of AMD’s architecture manual (“Operating Modes”).


The author of that posting missed an important point: to have code run in AMD64 compatibilty mode, the host OS must be 64-bit. Given that OS/2 is 32-bit, the CPU can only run in one of the legacy modes (i.e. 16 or 32-bit).

Hi Rich. But in what situation is 32/64 allowed (as per your original post), but not 16/32/64?

Perhaps you were considering Windows 10, where you need to choose between 16/32 or 32/64?

I think Windows 10 could have supported 16-bit PM programs (CM16), and CM32 and LM64 all at the same time, but since V8086 wasn't available, they just abandoned 16-bit completely.

But they didn't necessarily have to do that - they could have spent the effort to support Win16 PM16 programs.

Presumably that is not cost-justified though. But I'm just interested in the technical possibility.

Lars

  • Hero Member
  • *****
  • Posts: 1463
  • Karma: +73/-0
    • View Profile
Re: native mainframe mini-clone
« Reply #29 on: October 17, 2025, 11:00:34 am »
I think Rich said it all and I am quoting the Intel Spec (where Intel Term "IA-32e mode" =  "AMD64 compatiblity mode"):

Chapter 6.2.1 "Code-Segment Descriptor in 64-bit Mode"

IA-32e mode uses a previously unused bit in the CS descriptor. Bit 53 is defined as the 64-bit (L) flag and is
used to select between 64-bit mode and compatibility mode when IA-32e mode is active (IA32_EFER.LMA = 1).
See Figure 6-2.
— If CS.L = 0 and IA-32e mode is active, the processor is running in compatibility mode. In this case, CS.D
selects the default size for data and addresses. If CS.D = 0, the default data and address size is 16 bits. If
CS.D = 1, the default data and address size is 32 bits.
— If CS.L = 1 and IA-32e mode is active, the only valid setting is CS.D = 0. This setting indicates a default
operand size of 32 bits and a default address size of 64 bits. The CS.L = 1 and CS.D = 1 bit combination is
reserved for future use and a #GP fault will be generated on an attempt to use a code segment with these
bits set in IA-32e mode.

That said, the Processor will only recognize the "CS.L" flag and treat it as such when it has already been placed into the "IA-32e mode".
Only then will "CS.L" make the difference (CS.L = 0 -> CS.D selects between 16-bit or 32-bit as the default data/address size, CS.L = 1 -> CS.D is ignored and even expected to be 0 and default data size of 32-bit and default address size is 64-bit)

As far as I understand, if you run the Processor in IA-32e mode you will have a code segment descriptor with "CS.L = 1" (in the global descriptor table). This descriptor is used for 64-bit protected mode programs by loading it into CS, it will never change, implicitely cover all of the address range and effectively disables segment protection (which is never used in IA-32e "native" mode).

In order to run 32-bit or 16-bit protected mode programs, you will add additional code segment descriptor with "CS.L = 0" (in the global descriptor table) which when CS is loaded with this segment selector will allow the distinction between 16-bit and 32-bit default data/address size with the Processor looking at the "CS.D" bit.

You should be able to verify this under "64-bit" Windows: if you run a 64-bit program under a debugger, it should have a different CS selector then if you run a 32-bit program under that debugger.
« Last Edit: October 17, 2025, 11:05:29 am by Lars »