Author Topic: ES/2 open source OS/2 kernel  (Read 10614 times)

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: Exodus/2 (ES/2)
« Reply #30 on: August 11, 2016, 02:02:58 pm »
I explicitly intend to write a complete replacement for OS/2 which is fully API compatible, and UI-identical, to the OS/2 PM, down to the full object model.

It is always good to validate it :)

Absolutely.  When I get to that point, I intend to stress test it.  I want to create a complete system that is server quality, ready to go into the most intense stressful 24/7/365 service.

Best regards,
Rick C. Hodgin

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: Exodus/2 (ES/2)
« Reply #31 on: August 23, 2016, 02:44:28 pm »
I will be offering a class online teaching C to xbase developers.

The approach comes from that knowledge background as I am trying to get those interested in having Visual FreePro, Jr. (a free open source Visual FoxPro follow-on) completed to come on board and help out with development.  There are 172 members in my Visual FreePro Facebook group, and about 20 of them have expressed interest in wanting to help me complete development, but not having strong enough C/C++ development skills.

I am hoping that from the class one, two or three people will gain enough development skills to help out.  It's why I'm devoting some of my time toward this goal, because if one or more can help me out, then together we'll be able to do far more than I can do alone:

Intro2C Tutorial

I began the class in 2013, but very few people were interested.  I mapped out and prepared several more lessons, but none of them were ever published.

Best regards,
Rick C. Hodgin

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: Exodus/2 (ES/2)
« Reply #32 on: September 21, 2016, 06:04:32 pm »
I've had a requirement come up at work which has taken my development time the past few weeks.  When it is completed I intend to return to my assembler development.

I made the decision to modify some of the support libraries I had previously developed for use in the assembler.  I had previously designed things to operate around a line + component model, such that "x = a + b" would be broken out into parts:
Code: [Select]
line ---> [x][whitespace][=][whitespace][a][whitespace][+][whitespace][b]

After simplification:
line ---> [x][=][a][+][b]

This model would probably work fine for an assembler, because most things in assembly are single-line syntax forms.  However, my next goal is my lsc C compiler and it needs syntax constructs which are entirely outside of line references.  As such, I've migrated the line+comp model to now use a comp-only model, which has an internal line reference, but is largely unimportant.

Code: [Select]
[[x] ---> line][[=] ---> line]...
By using this model, I can inject and delete components as needed, allowing for a far more robust and free form translation from C code to assembly code.

It may not seem like I'm making any progress, but I am.  It's just a lot of foundation to lay properly so that everything built atop is correct.

Best regards,
Rick C. Hodgin

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: Exodus/2 (ES/2)
« Reply #33 on: June 04, 2017, 01:49:44 am »
I may pick this effort back up (to create a drop-in replacement kernel for OS/2).  It's a big job, but I truly believe it's the right place to start.  We need a completely flexible base to build everything atop, which can also be tweaked and extended as needed.  We also need a full kernel debugger.

I'm going to give it some thought over the next week.

Thank you,
Rick C. Hodgin
« Last Edit: June 04, 2017, 02:00:02 am by Rick C. Hodgin »

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: ES/2 open source OS/2 kernel
« Reply #34 on: June 04, 2017, 11:00:33 pm »
I have discovered the OS/2 kernel API is not published anywhere, except through illegally leaked IBM original kernel source code, which I've been told is what the Phoenix OS/4 project is based on.

UPDATE:  I've been told by developers on #os2russian on EFnet that the Phoenix OS/4 kernel is not based on leaked IBM kernel sources.  So, my apologies for having written that.  However, they have also told me that the kernel is written in "poetic form" and is to simply be accepted "as-is" ... so it has me in a wondering state to be honest. :)

As such, I'm going to focus on a full ground-up new effort, similar to what Linux did for UNIX.  This will be the ES/2 project, and it will be a complete mimic / duplication of everything in OS/2, but it will be made from all new sources.  Every main exe, dll, file, file format, etc., will be replicated.  Except for the fact that it won't be binary compatible, I believe every piece of code written for OS/2 that has sources, will be able to be re-compiled in the new ES/2 environment and work without change.

This is going to be a very large effort.  And I'm going to need help.  I'm going to start trying to recruit developers.  If any of you are interested in trying to help out, I'm willing to teach a class on C/C++ programming and even assembly programming on x86-based CPUs so you learn enough to start port user apps, including even low-level things like cmd.exe, a text-based version of e.exe, etc.

Thank you,
Rick C. Hodgin
« Last Edit: June 05, 2017, 06:04:43 pm by Rick C. Hodgin »

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: ES/2 open source OS/2 kernel
« Reply #35 on: June 05, 2017, 06:09:46 pm »
I began looking at my assembler last night, and when I stopped regular development last September, and worked only sporadically on the project since then (see blog), it was up to a point where the assembler was loading source code, expanding macros, and parsing top-down.  The parsing algorithms are partially complete, and development will continue from that point forward.  I've also since migrated my source code off of GitHub to my own Git server.  You can see the source code here.

My goals once again are to have the kernel up and running by the end of this year, first part of next year, and to be able to begin replicating the OS/2 command prompt on boot, as we add more and more abilities on top.

My first goals are to support the FAT32 and HPFS file systems, and a small range of video hardware devices, and to use BIOS for standard disk access.  This will allow more people to use the system in stages as it's being developed.  It will initially only be 16-bit and 32-bit, but eventually will be extended to be a true hybrid 32-bit / 64-bit kernel, capable of launching both from the single kernel.

My second goal is to have armies of developers come on board and begin helping write the lower-level drivers and devices, and then to begin porting apps from OS/2 to ES/2, as the API and compiler environment should be the same, but it will not be binary compatible any longer.

Here's to success!

Thank you, and God bless,
Rick C. Hodgin
« Last Edit: June 05, 2017, 06:23:20 pm by Rick C. Hodgin »

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: ES/2 open source OS/2 kernel
« Reply #36 on: November 02, 2017, 08:14:59 pm »
I am migrating my ES/2 developer environment to a full OS/2 environment (previously it was on Windows).  It will be developed from this point forward in an OS/2 4.5 developer environment running on native hardware (not a VM), with a second native hardware machine used for testing the code.

My goals are to use the OS/2 developer environment to get my kernel compiled, to create a stable serial interface between the OS/2 machine and my ES/2 machine so I can rapidly send updates to my kernel while it's running, to do remote debugging, and so on.

The code I've previously written in Windows will be ported to OS/2 and the Presentation Manager.  I've written it generically all along as I originally was targeting Windows, Linux, and FreeBSD, so there is nothing Windows-specific to it that shouldn't be easily adapted to integrate well with OS/2.

----
Again, if you have the skills, I would like to invite you to come and help out on this project.  It's to be an offering of the skills and abilities God first gave us, and the opportunities He's given us since then to harness and hone them, to return them back to Him directly, and not for money, not for fame, not for personal glory, but because we have these interests, and we recognize all of our abilities come from Him.

Consider:
If someone gave you $10 million to build a company and some large campus, you would begin work and invite that person to be a part of all of your project, opening ceremonies, you'd give speeches which include parts like, "Without so-and-so, we wouldn't be here today."  You would name a building after that person, or the whole campus even.  There would be plaques commemorating that individual, with their picture here and there all around.  The "About us..." documentation within the company would include references to that individual's generosity.  And so on.

All I'm trying to do here with these projects of mine is give God His proper place in my life.  He made the Earth, Sun, air, trees, animals, plants, everything, including me and you.  It is desirous, and proper, to give Him all that He is rightly due with our lives.

ES/2 is to be a ground-up open-source OS/2-like operating system.  The compilers I'm writing are ground-up open-source creations using some familiar languages as a starting point (assembly, C, Java, Julia, and others), but then extending them forward.  And the same holds true for apps that will be developed which are similar to other apps.

All of these are ground-up, sanctified, purposeful efforts given over to Him acknowledging His rightful place in our lives.

Please ... look up to God.  Look within your heart.  And come and be a part of this project.  Let His gifts to you shine outwardly to others in this endeavor, so that you can receive from Him the reward of your efforts given over to Him here in this world.

--
Rick C. Hodgin

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: ES/2 open source OS/2 kernel
« Reply #37 on: November 04, 2017, 12:23:22 am »
On the machine I had to use (VIA-based chipset + AMD CPU), I couldn't get native IBM OS/2 Warp 4 or 4.52 to boot.  It got to the end of the third diskette and crashed.  I also had to reduce the RAM to 1 GB and enable an OS/2 option in BIOS to avoid SYS2025 and SYS2027 errors at Installation Diskette boot.

Ultimately, I was able to get my copy of ArcaOS to boot and install.  Everything worked there.  I was able to get my original kernel code copied over, and OS/2 versions of the assembler and C compiler installed.  I still have to migrate my DOS .BAT files to REXX scripts, and then it should be compiling.

I am also pleased to announce nearly everything I've written in my LibSF library for Windows, Linux, and FreeBSD will migrate with little modification to OS/2 and the Presentation Manager based on what I've researched thus far.

OS/2 programming with the PM is easier than Windows programming with the GDI.  However, I have greater knowledge of Windows so it will be a learning curve.

I'm looking forward to getting Visual FreePro, Jr. (VJr) running in OS/2.  It will serve as my primary developer base as it is well-developed for these purposes (meaning the facilities created to support VJr are generic, and can be used for other purposes).

It took me one day to make the migration, and it will take probably a week to get VJr all compiling, running, and ES/2 installing and booting natively on my second machine.  And a while to get the serial driver working.

I did verify the version of my kernel I had on that hard drive boots from floppy disk with the compiled code that's there, but I think the source code base there is about 10% behind my current version, so I will be switching to the newer one.

Am very pleased with how smooth the migration has gone thus far.
« Last Edit: November 04, 2017, 12:35:23 am by Rick C. Hodgin »

ivan

  • Hero Member
  • *****
  • Thank You
  • -Given: 0
  • -Receive: 24
  • Posts: 651
    • View Profile
Re: ES/2 open source OS/2 kernel
« Reply #38 on: November 04, 2017, 01:25:04 am »
Keep up the good work.

When you get to the testing stage I have several new mini itx motherboard computers with AMD 2 and 4 core APUs as well as a couple of AMD multi processor, multi core HP server boxes that I can make available if that would help.

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: ES/2 open source OS/2 kernel
« Reply #39 on: November 04, 2017, 01:48:03 am »
Keep up the good work.

When you get to the testing stage I have several new mini itx motherboard computers with AMD 2 and 4 core APUs as well as a couple of AMD multi processor, multi core HP server boxes that I can make available if that would help.

Will do.  Still have a long way to go.  One of my primary goals is to try and find developer recruits.  I will be pursuing that avenue with vigor as I want this project to succeed.  A fully open-source modern OS/2 running in 64-bits on modern hardware would be a thing of beauty.

ivan

  • Hero Member
  • *****
  • Thank You
  • -Given: 0
  • -Receive: 24
  • Posts: 651
    • View Profile
Re: ES/2 open source OS/2 kernel
« Reply #40 on: November 04, 2017, 10:28:17 am »
As you say 'A fully open-source modern OS/2 running in 64-bits on modern hardware would be a thing of beauty.' I couldn't agree more.

Unfortunately I am a hardware engineer (now retired) so there is not much that I can do to help with the development other than testing whatever you produce.

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: ES/2 open source OS/2 kernel
« Reply #41 on: November 04, 2017, 11:12:12 am »
As you say 'A fully open-source modern OS/2 running in 64-bits on modern hardware would be a thing of beauty.' I couldn't agree more.

Unfortunately I am a hardware engineer (now retired) so there is not much that I can do to help with the development other than testing whatever you produce.

You might find my L1 cache design of interest then:
 L1 cache cell

And my Arxoda CPU register model:
 40-bit registers, windowed access

I have long-term goals to design my own 40-bit 80386-like core, part of creating a full new hardware and software stack.  Removing legacy limitations, and making everything completely open source.  I am prepared to use older methodologies to work around patents, and to integrate / roll out newer features over time.

 Basic Arxoda CPU core design

It uses a pivot, which allows each core to come over and help the other core perform work on single-threaded apps, allowing serial performance gains.  I call it "love-threading" because it's how two people who love each other help the other out when one has a temporary need, "Honey, can you come here and help me a moment?"

 Pivoting CPU State "Love Threading"

I want to empower people.  Make them owners, not renters.
« Last Edit: November 04, 2017, 11:21:28 am by Rick C. Hodgin »

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: ES/2 open source OS/2 kernel
« Reply #42 on: November 07, 2017, 04:00:00 am »
I was looking for a test machine for ES/2 development.  I think I'm going to start with this VIA-based machine.  They have published their VX900 Chipset System Programming Manual online.  I've emailed them to find out about any example / template source code released for Linux drivers.  I could use that information to drawn from to create ES/2 drivers.

I am so excited about this project.

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: ES/2 open source OS/2 kernel
« Reply #43 on: November 07, 2017, 02:10:23 pm »
I was looking for a test machine for ES/2 development.  I think I'm going to start with this VIA-based machine.  They have published their VX900 Chipset System Programming Manual online.  I've emailed them to find out about any example / template source code released for Linux drivers.  I could use that information to drawn from to create ES/2 drivers.

I received an email response from VIA.  The source code for their Linux graphics driver is available.  If you choose "Ubuntu 12.04.5" from the list, and VX900, the driver shows up.  And their System Programming manual is comprehensive.  Every port.  Every value.  It's even written as a manual should be. :-)

These resources should provide all VIA-specific information I need, and with input from other sources it should all come together so this little x86-64 VIA Eden-powered dual-core EPIA-M910 machine will be the vehicle which brings ES/2 to life.  A new kernel, new drivers, and apps.

UPDATE:  If you download the source code and unzip it, you'll see in the TTM\bios\ folder information about the motherboard.  It includes everything related to video timing.  The cbios_cea_timing.h/.c files contain absolutely straight-forward information about the register timings involved to enter each video mode.  And the entire source code structure contains well-written code.
« Last Edit: November 07, 2017, 03:32:18 pm by Rick C. Hodgin »

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 13
  • -Receive: 2
  • Posts: 182
    • View Profile
    • Liberty Software Foundation
Re: ES/2 open source OS/2 kernel
« Reply #44 on: November 07, 2017, 05:09:33 pm »
After getting IBM PC DOS 7 installed the other day, I've decided to move to DOS for development of ES/2.  I'll use BIGREAL mode, which allows access to all of memory through FS: and GS: segments, but CS:, DS:, ES:, and SS: are all in real mode in the normal 0..1MB range.  I'll use VESA BIOS Extensions to give DOS some graphics, as I've already written graphics algorithms for Visual FreePro and Visual FreePro, Jr.

So, except for answering questions, I'm going into stealth mode for a while to focus on development.

You'll be able to keep track of my work on LibSF's Git server for ES/2.  If you'd like to follow along, you can download a .ZIP of the ES2 portion of the repository (useful in DOS where you can't use git), buy the same VIA machine I'm using, keep up with source code, run the buildme script, and generally speaking follow the BUILDME.TXT on how to keep up with development each day.

I would welcome your assistance on this project.  Please contact me via email.  Use rick.c.hodgin@gmail.com and "ES/2" in the subject.

If you are a reasonably skilled C/C++ developer you can be of use.  You don't have to have great skills or hardware system knowledge, but just decent skills in C/C++ development, and mostly C, and a desire to learn and help out.  If you don't have enough skills, I'll be happy to teach you, or to help sharpen the skills you're missing.