Author Topic: Open Source / 2  (Read 34531 times)

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 4710
  • Karma: +41/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Open Source / 2
« Reply #15 on: March 17, 2016, 01:52:56 pm »
Hi

There are also some open source projects that I consider that should be taken notice when trying to produce OS/2 clone alternatives.
If I'm based on the OS/2 components architecture I think it is good to see the source code of this projects and check what can it be re-used.

- OS/2 Kernel:
* OS2Ldr - OS/2 Loader, there are some articles and samples on how to create an clone loader.

- CPI - Control Program Interface.
* OS2Linux - has some functions available
* OSFree - OSFree Project generated some of the forwarder's DLLs of this component.

- Presentation Manager
* OSFree - OSFree grabbed things from FreePM.

- SOM
* somFree - It was not created or ported to OS/2.
* NOM - Was a SOM replacement project created by Netlabs.

- Workplace Shell
* XWorkplace - Some WPS classes replacement and improvements.
* Several open source WPS Classes.

There are some other projects that are not open source, but has the source code available:
- OS/2 Kernel:
* QSInit - OS/2 Loader - Source code available - Free for Non commercial use.
* SCREEN01.SYS, PRINT01.SYS, KBDBASE.SYS, CLOCK01.SYS, RESOURCE.SYS - Source code is on the IBM DDK, but the source code can not be shared (only privately improved). Binaries can be freeware for any use.

But remember that some of this are only "half-baked" projects and was not completed.  This are the ones that came to my mind right now, there my be other out there.

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

RickCHodgin

  • Guest
Re: Open Source / 2
« Reply #16 on: March 17, 2016, 04:05:03 pm »
I'm not a developer so I have limited skills on this area. My way of thinking about the continuity of OS/2 in the long term future is to try to grab little things from OS/2 and clone them as open source and test that it keep working on the OS/2 platform (trying not to break compatibility). I think that sometimes planning super big projects (without having high budgets) produce too much stress and people just dump out the idea because the lack of short term deliverable. So, I prefer when people do little things, even if the first version is not stable enough. But that little things can be added up to other things to produce a bigger things, and with time even bad open source code can be improved.

I wish you luck with your project and I'm here to help on whatever is on my hands.

I appreciate that.  I've had high hopes on the last two projects I've worked on that some developers would come and help out.  It didn't happen on the first project, but on the second I had two developers contribute source code:  http://www.visual-freepro.org/wiki/index.php/VXB#Contributions

That project is an XBASE tool called Visual FreePro, Jr. (VJr), that is designed to be a nearly 100% compatible open alternative to Visual FoxPro.  I'm to the point in development on it where I need to complete the compiler.  I had previously planned to work on it this year and complete it, but there just isn't interest in the project right now, and no help.  I had planned to complete VJr by mid-2017 and resume development on my OS then.  However, I've revised that plan to get my kernel up and running and then complete development of VJr inside my kernel to give it a nice developer base for making business apps.

The kernel has been a long-time goal for me.  I began it when I was about 25 years old.  I'm now 46 years old and am resuming full-time (after my normal day job that is) development on it.  I'll try to get my developer toolsets completed this year, and have the kernel able to boot and launch and run apps using the basic video drivers in the emulation environment.

I may not post much here once I get deep into development.  Compiler development is particularly tricky, but I won't be worrying about optimization in the first release, so that will make it somewhat easier.  And, because it is my own kernel design, I will have free reign over what gets added and where.

I'd like to expose a fully compatible API with OS/2's kernel, allowing it ultimately to be a literal drop-in replacement, but unless I get some documentation that won't be possible because I refuse to reverse-engineer the kernel design since it's illegal to do so.

I've written an email to Scott Garfinkle at IBM.  No reply yet.  We'll see if anything comes from there.

As for now, my plans are to create a very OS/2-like system, with many of the same features, but it will be a new kernel, and a new system, new drivers, etc., and atop that I will expose an identical API to the published API, along with some extensions, allowing for existing code that relies upon many OS/2 features to be recompiled and work in ES/2 and AS/2.

And, I'm hoping other developers will come on board later this year when the toolset is maturing, or at least next year when some real progress is being shown.  These are my plans, but I keep them within the constraint of James 4:15 (Lord willing).  :)

Best regards,
Rick C. Hodgin

RickCHodgin

  • Guest
Re: Open Source / 2
« Reply #17 on: March 17, 2016, 04:07:48 pm »
Hi

There are also some open source projects that I consider that should be taken notice when trying to produce OS/2 clone alternatives.
If I'm based on the OS/2 components architecture I think it is good to see the source code of this projects and check what can it be re-used.

- OS/2 Kernel:
* OS2Ldr - OS/2 Loader, there are some articles and samples on how to create an clone loader.

- CPI - Control Program Interface.
* OS2Linux - has some functions available
* OSFree - OSFree Project generated some of the forwarder's DLLs of this component.

- Presentation Manager
* OSFree - OSFree grabbed things from FreePM.

- SOM
* somFree - It was not created or ported to OS/2.
* NOM - Was a SOM replacement project created by Netlabs.

- Workplace Shell
* XWorkplace - Some WPS classes replacement and improvements.
* Several open source WPS Classes.

There are some other projects that are not open source, but has the source code available:
- OS/2 Kernel:
* QSInit - OS/2 Loader - Source code available - Free for Non commercial use.
* SCREEN01.SYS, PRINT01.SYS, KBDBASE.SYS, CLOCK01.SYS, RESOURCE.SYS - Source code is on the IBM DDK, but the source code can not be shared (only privately improved). Binaries can be freeware for any use.

But remember that some of this are only "half-baked" projects and was not completed.  This are the ones that came to my mind right now, there my be other out there.

Excellent.  Thank you, Martin.  I was able to locate a copy of the IBM DDK from 2004.  I've had a preliminary look and it is an amazing treasure trove of information.  It should provide a solid map to the API.

Best regards,
Rick C. Hodgin
« Last Edit: March 17, 2016, 06:22:27 pm by Rick C. Hodgin »

RickCHodgin

  • Guest
Re: Open Source / 2
« Reply #18 on: April 01, 2016, 02:03:48 am »
With Martin's guidance, I have been able to find everything necessary to handle the kernel development, and API replication, so that the system I'm creating is as compatible with OS/2 as I'll be able to make it.

OS/2 4.5 Toolkit Documents

Best regards,
Rick C. Hodgin

RickCHodgin

  • Guest
Re: Open Source / 2
« Reply #19 on: April 01, 2016, 03:09:15 pm »
With Martin's guidance, I have been able to find everything necessary to handle the kernel development, and API replication, so that the system I'm creating is as compatible with OS/2 as I'll be able to make it.

OS/2 4.5 Toolkit Documents

In reviewing this material, I have come to realize that there is a large portion of the system design that really isn't relevant any longer in 2016 and beyond.  In addition, there are some disruptive technologies due out in the middle of this year (namely Intel's 3D XPoint memory, which is a non-volatile memory 1000x faster than NAND, 10x more dense than DRAM, and has the ability to change the way operating systems are designed, and operate).

As such, I will be migrating my efforts to what I'm going to call Exodus OS/3, which will be basically OS/2 with the DOS and Windows portions stripped out, and some new extensions added which allow use of 3D graphics devices to provide a type of node-based connection mechanism between drivers and applications, rather than through config file settings (see Blender 3D software and specifically Blender's node editor for what I'm thinking about -- it won't be quite like that, but similar in that you hook things up using the lines/noodles which go between (see a YouTube video example here), so imagine being able to route your digital sound to audio devices by simply plugging them in, or to be able to direct your printer driver to the network so it now exposes it as a device available on an OS/3 network, etc.).

I am also open to hearing input on what should be done.  Eventually I would like for Exodus OS/3 to support a virtual machine allowing for guest OSes as well, including a full version of OS/2 4.52 as it exists today.

Best regards,
Rick C. Hodgin

aramaswa

  • Guest
Re: Open Source / 2
« Reply #20 on: July 31, 2018, 08:44:27 am »
Hello all,

I came across this thread accidentally and am pleasantly surprised to see OS/2 world back in action. I was part of the Presentation Manager development team and very happy to be part of any initiative around OS/2.  I know this is an old thread but if there are any means to contact you and know more about this, I would be the happiest.

cheers,
Anitha

OS4User

  • Sr. Member
  • ****
  • Posts: 406
  • Karma: +10/-0
    • View Profile
Re: Open Source / 2
« Reply #21 on: July 31, 2018, 09:19:45 am »
I was part of the Presentation Manager development team and very happy to be part of any initiative around OS/2

Hi,

It is exciting  to meet somebody from PM dev team. Do you remember  any details about PMMERGE.DLL ?

 

Rich Walsh

  • Sr. Member
  • ****
  • Posts: 331
  • Karma: +23/-0
  • ONU! (OS/2 is NOT Unix!)
    • View Profile
Re: Open Source / 2
« Reply #22 on: July 31, 2018, 10:29:34 am »
I came across this thread accidentally and am pleasantly surprised to see OS/2 world back in action.

OS/2 development has never ceased though it certainly has slowed a lot. This particular thread may not be the most representative since the O.P.'s grandiose plans were largely a mixture of fantasy and self-deception that never amounted to anything.

Quote
I was part of the Presentation Manager development team and very happy to be part of any initiative around OS/2.

We would certainly all benefit if someone with your experience cared to do some original native programming (most development today seems to center around porting Linux apps).

As you may have seen, the current incarnation of OS/2 is "ArcaOS", a product of Arca Noae, LLC. Their goal is to enable OS/2 to run on modern hardware, so they have developed their own drivers for ACPI, AHCI, and the other things needed nowadays. Since their contract with IBM allows them to reverse-engineer the kernel, they've made some significant improvements there as well. (Full disclosure: I do development work for them so I may be biased, but I'm impressed nonetheless with what they've achieved.)

If you'd like more info, send me a private message or just open a new thread - you'll probably get an avalanche of opinions on what's needed.

Valery Sedletski

  • Sr. Member
  • ****
  • Posts: 368
  • Karma: +2/-0
    • View Profile
Re: Open Source / 2
« Reply #23 on: July 31, 2018, 04:19:56 pm »
Yes, ArcaOS does good job with supporting OS/2 on modern hardware. Though, it is closed. It is even more closed than eComStation. They do copy protection by branding some binaries with user's own name. Neither IBM OS/2, nor eCS did not do such bad things. Newer drivers like DANIS506 or AHCI are linked with closed-source libraries (DRV16, DRV32 etc.), so that they are not fully open source while having been published under GNU GPL. And they now port drivers from FreeBSD, because BSD license allows them to not publish sources.
These are bad things, though I appreciate their successes with porting drivers from UNIX systems plus their very clean and updated OS/2 distribution called ArcaOS/BlueLion.
But except for that, there is open source OS/2 cloning project called osFree (http://osfree.org). And I'm participating in it. We are collecting existing open source OS/2 code. We have cloned some OS/2 command line utilities, plus borrowed existing open source code, including FreeDOS command line utilities (for both OS/2 CLI and VDM CLI), Regina REXX, a number of open source REXX DLL's. Also, an ongoing project is REXX.DLL and REXXAPI.DLL for Regina REXX, aiming to create a replacement for IBM classic REXX. Also, we have osFree PM project (based on FreePM project by Evgen Kotsuba). It is aiming to create a PM replacement. It is intended to support a remote network-transparent execution of PM programs. It is in early stage, yet, but we did proper structuring of it. PM programs can execute on a remote host, communicating with a "PM server" via a communication backend. So far, a "pmpipe" backend exist, which allows to communicate via a named pipe (local or remote.). PM server should draw windows on a screen via a PM presentation driver. For the first time, there is a "pmwindow" DLL (though, it is a stub for now, doing no real work, except for a drawing picture on a desktop, as a background). PMWINDOW.DLL draws a FreePM screen in a "host" PM window. So, it uses a PM of the host OS/2 system to draw a desktop. But now the missing part is PMGRE/PMGPI, so no real drawing, for now. A client program links with usual DLL's, like PMWIN, PMDRAG, PMGRE etc., most of which forward to PMMERGE.DLL. PMMERGE.DLL connects to PMSRV.DLL (a PM server), PM server is loaded by a first PM application started, like pmshell.exe (which is cloned by me). So far, only simplest PM applications are run, executing a message loop, having anchor block, message queue etc. So, unmodified PM application is run both on PM and FreePM. Though, as GRE/GPI is now missing, it does not draw anything on th screen. FreePM is an interesting development track, but we have currently too little manpower, so we focus on other tasks now (like L4-microkernel-based set of servers, supporting the OS/2 base API, called osFree OS/2 personality). You can join osFree development, if you interested. (It would be an area of your expertise).

Also, the most important sub-project within osFree project is OS/2 personality. We have a prototype based on L4Env (L4 Environment, a set of libraries on top of L4/Fiasco microkernel. It is now abandoned in favor of newer L4Re, L4 Runtime environment, based on Fiasco.OC, the new generation microkernel). Except for L4Env, there is also L4Re, and Genode (http://genode.org). Genode OS framework is most promising from the three mentioned OS frameworks. It supports creation of component-based OS'es from a number of small components, having a small number (about 15) orthogonal interfaces between them. So, they can be combined with special config files written in XML, in orbitrary ways. But it's ideology is best described on their site. Genode allows running the same components on top of about 8 microkernels of L4 family (Like Fiasco, Fiasco.OC, NOVA microhypervisor, L4Ka::Pistachio, OKL4, SeL4, HW ("execution on bare hardware" -- a special custom kernel, combined with Genode root component, called core)), and on top of Linux kernel. Genode has a number of DDE's (Device driver environments), for Linux kernel (dde_linux), which supports wrappers for Linux drivers (running unprivileged in ring3), e.g., Linux TCP-IP stack, Linux wireless drivers, Linux Intel framebuffer drivers. Also, there is dde_bsd, which ports Audio drivers from OpenBSD, dde_ipxe, porting NIC drivers from iPXE project, and dde_rump, porting NetBSD rump kernel to Genode. Rump kernel is a special usermode server/library, allowing to execute unmodified NetBSD drivers (like TCP/IP stack, file systems or device drivers) as a simple usermode libraries. So far, some file systems from NetBSD are running under dde_rump.

All these features make Genode a very attractive alternative to L4Env/L4Re, so we're now trying to port our OS/2 personality prototype to Genode.

So far, we have implemented a number of servers on top of L4Env, implementing the OS/2 API. 1) There is "os2exec" server, which loads sections of executable files into its own memory, and it implements a shared memory arena. All OS/2 DLL's load into shared memory. os2exec server loads support of different executable formats, implemented as L4Env shared libraries. These libraries implement support for executable formats, transparently for the rest of the system. They are called Installable eXecutable Formats (IXF, by analogy with IFS). 2) There is (currently a stub for)  a "os2fs" server, which implements the file system API's. In the future, it should run OS/2-like IFS drivers (LX binaries, loaded by "os2exec" into "os2fs" address space). So, we plan to implement an "os2fs" server as a "filesystem router" in which IFS drivers are plug-in as LX DLL's. These LX DLL's should support "FS32_*" entry points, as it's done by JFS.IFS, without the need to support the 16-bit FS_* entry points. The first driver ported will be "ramfs.ifs" ramdisk driver. Then we plan to port FAT32.IFS and JFS.IFS (there is OpenJFS project based on Linux JFS sources). 3) And there is the main "os2srv" (OS/2 Server) server, which parses CONFIG.SYS, processes the system initialization and starts other servers and programs 4) There is also "os2app" L4Env application, which is a "container" of an OS/2 application. It calls OS/2 personality servers to load the LX executable and all required DLL's, then jumps to its entry point. There is a number of CPI (Control program Interface, the OS/2 kernel base API) DLL's, which are implemented, to some extent. As in case with PM, there are a number of forwarder DLL's linked to DOSCALLS.DLL. API's like SESMGR, MOUCALLS, VIOCALLS, KBDCALLS, etc. are implemented in DOSCALLS, and are forwarded to DOSCALLS from DLL's like VIOCALLS.DLL/MOUCALLS.DLL/etc. Also, there is a virtual KAL.DLL (which stands for Kernel Abstraction Layer), which is a core of OS/2 API, implemented in OS/2 personality and L4Env Servers. DOSCALLS functions are built on top of them. The border between KAL and DOSCALLS separates "OS/2 world" and "L4 world". KAL is implemented as a native L4env shared library, DOSCALLS is a native LX OS/2 DLL. These two worlds are built with different toolchains. "OS/2 world" is built with OS/2-specific OpenWatcom tools, "L4 world" uses GNU toolchain, specific to the host environment (L4Env/L4Re/Genode).

So far, we have implemented support for pure 32-bit CLI OS/2 applications. There is about 70 API's implemented (with different level of readiness). VIO/MOU/KBD should be pure 32-bit in the future, but so far, they are not implemented. Now programs, based on stdin/stdout/stderr are working, based on Libc/VFS/terminal support in L4env. There is a small minishell is started, which can run external programs, like zip.exe, sed.exe, awk.exe, wasm.exe, wmake.exe (OpenWatcom assembler and make utility). Also, xmakeini.exe (a makeini clone by Ulrich Moeller) is run in the minishell. In the future, we will incrementally increase the functionality. We will first try to run 4OS/2 shell instead of a minishell. Then move towards VIO/KBD/MOU and try running FC/2

So far, we're a bit stick porting it to Genode. But moving on slowly. Unfortunately, there are only three more-less active developers, from which one became a big IT boss, which is very busy, another graduated from the university, so, he became very busy too. Only me is trying to do something currently. Though, I'm currently jobless this time, searching for a job, and could be overloaded by work soon too.

Anybody willing to help is welcome. The former PM developer could help a lot with advancing the FreePM project.

WBR,
valery.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 4710
  • Karma: +41/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Open Source / 2
« Reply #24 on: August 02, 2018, 01:40:33 am »
Hi Anitha

Presentation Manager is considered an important part of OS/2 and it is one of its vital components. One of the greatest goals of the OS/2 community will be to finally clone it and offer it as open source software so it can keep evolving,  but we lack skill and hands for that tasks.

There are projects that can be the starting point for a Presentation Manager clone like:
- FreePM (part of the OSFree project) : http://osfree.org/
- 2ine: Which has some incomplete cloned components of PM.  https://hg.icculus.org/icculus/2ine/

It would be really great if someone can help on that task to improve FreePM and move it forward.

Regards

Martin Iturbide
OS2World NewsMaster
... just share the dream.

RickCHodgin

  • Guest
Re: Open Source / 2
« Reply #25 on: September 20, 2018, 03:57:54 am »
Presentation Manager is considered an important part of OS/2 and it is one of its vital components. One of the greatest goals of the OS/2 community will be to finally clone it and offer it as open source software so it can keep evolving,  but we lack skill and hands for that tasks.

I would be willing to write an open source PM.  I'll look into the API and see what's required.

UPDATE:  I found documentation here:  http://www.os2museum.com/wp/os2-history/os2-library/os2-1-x-sdk/ ... Does anybody have a link to something newer?  Or a copy of this book: https://books.google.com/books/about/OS_2_Warp_Presentation_Manager_API.html?id=nnIhAQAAIAAJ
« Last Edit: September 20, 2018, 04:16:40 am by Rick C. Hodgin »

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Posts: 4710
  • Karma: +41/-1
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: Open Source / 2
« Reply #26 on: September 20, 2018, 04:32:12 am »
Hi Rick

It can be interesting if you can focus in a single component first to create a clone of it. PM is big and requires a lot of effort, here are some links.

For the latest Presentation Manager API documentation check the "Presentation Manager Programming Guide and Reference" from IBM on this place: http://www.edm2.com/index.php/IBM_OS/2_Warp_Development_Manuals
It is referred also as the PM1.INF, PM2.INF, PM3.INF, PM4.INF, PM5.INF books.

There are some other PM books available here:
http://www.edm2.com/index.php/List_of_OS/2_Software_Development_Related_Books
I was not able to find the one you are looking for.

You may also get some ideas/inspiration from the FreePM project (incomplete) of OSFree.
https://github.com/osfree-project/osfree

Regards

Martin Iturbide
OS2World NewsMaster
... just share the dream.

RickCHodgin

  • Guest
Re: Open Source / 2
« Reply #27 on: September 20, 2018, 04:56:50 am »
It can be interesting if you can focus in a single component first to create a clone of it. PM is big and requires a lot of effort, here are some links.

That API is huge, probably 50x more complex than it needs to be.  I'll definitely pare it down.

I'll work on a creating the skeleton of an open PM on Windows using Visual Studio.  Once the base algorithms are developed and it's able to do basic things, it can be ported to OS/2 and expanded by somebody.
« Last Edit: September 20, 2018, 10:14:28 am by Rick C. Hodgin »

RickCHodgin

  • Guest
Re: Open Source / 2
« Reply #28 on: September 20, 2018, 02:36:13 pm »
It would be really great if someone can help on that task to improve FreePM and move it forward.

Would anyone be interested in a PM-lite version?  Something that is very PM in nature and function, but doesn't include the full API?

Andi B.

  • Hero Member
  • *****
  • Posts: 811
  • Karma: +11/-2
    • View Profile
Re: Open Source / 2
« Reply #29 on: September 20, 2018, 04:01:16 pm »
Quote
That API is huge, probably 50x more complex than it needs to be.
Needs to be for what? I'm pretty sure the complexity isn't there for it's own sake. Maybe you only didn't understand fully for what it's good for?

Quote
Something that is very PM in nature and function, but doesn't include the full API?
IMHO then it's no PM. It's another graphic engine with limited functionality. Zero applications and no GUI. Or do you think WPS and SOM runs on your lightweight version? What is the benefit compared to our working PM. I fore myself hardly see any need for such thing.