OS2 World Community Forum

Public Discussions => General Discussion => Topic started by: A. Demetrious Sharpe, Sr. on December 06, 2017, 03:23:29 am

Title: OS/2 application developers
Post by: A. Demetrious Sharpe, Sr. on December 06, 2017, 03:23:29 am
Hello all,

Are there many application developers still active in the OS/2 community? If so, what API features would you like to see present if there were a 64-bit implementation of the OS/2 API? What features would you like to see removed from the OS/2 API?
Title: Re: OS/2 application developers
Post by: Andi B. on December 06, 2017, 12:54:23 pm
I know you will not like my answer but you've asked -
Quote
If so, what API features would you like to see present if there were a 64-bit implementation of the OS/2 API?
All.

Quote
What features would you like to see removed from the OS/2 API?
None.

:-)
Title: Re: OS/2 application developers
Post by: Martin Iturbide on December 06, 2017, 01:02:27 pm
Hi Demetrius

That is a difficult question because a developer will request to have the same experience on a clone implementation, so you will get the "all" answer. Implementing or cloning all is a very difficult task. I would recommend you to stick on the main APIs of OS/2 (CPI, PM). Maybe the first step will be try to replicate the CPI API  maybe trying to reuse some things from the OS2Linux project (https://github.com/OS2World/LINUX-SYSTEM-OS2Linux).

After you got some of the CPI API working it may lead to continue with the rest that it is on top, like PM.

Regards
Title: Re: OS/2 application developers
Post by: A. Demetrious Sharpe, Sr. on December 06, 2017, 02:04:49 pm
I know you will not like my answer but you've asked -
Quote
If so, what API features would you like to see present if there were a 64-bit implementation of the OS/2 API?
All.

Quote
What features would you like to see removed from the OS/2 API?
None.

:-)

Fair enough lol
Title: Re: OS/2 application developers
Post by: A. Demetrious Sharpe, Sr. on December 06, 2017, 02:09:03 pm
Hi Demetrius

That is a difficult question because a developer will request to have the same experience on a clone implementation, so you will get the "all" answer. Implementing or cloning all is a very difficult task. I would recommend you to stick on the main APIs of OS/2 (CPI, PM). Maybe the first step will be try to replicate the CPI API  maybe trying to reuse some things from the OS2Linux project (https://github.com/OS2World/LINUX-SYSTEM-OS2Linux).

After you got some of the CPI API working it may lead to continue with the rest that it is on top, like PM.

Regards

Most likely, there would be additional changes as the API becomes 64-bit. Sure, compatibility for a lot of older programs may be achievable, but I'm not really focused on the older programs -I'm looking at 64-bit OS/2 the way Microsoft looked at 64-bit Windows.
Title: Re: OS/2 application developers
Post by: xynixme on December 08, 2017, 05:28:16 am
I'm not really focused on the older programs -I'm looking at 64-bit OS/2 the way Microsoft looked at 64-bit Windows.

</silence>
We may need 64-bit for a 64-bit browser. That's it. And maybe for a 64-bit compiler for at least one user, to compile such a browser. If you'd expect that people want to throw away 20+ years of investements, if anything of time, to browse, then they may as well throw away their software now and start using a 64-bit browser for your 64-bit Windows. I don't really care what your private focus is, but then I'd suggest to throw away "the older programs", i.e. everything for 16- and 32-bit OS/2, and start using whatever already existing 64-bit OS with a 64-bit browser software. So it's not the best point of view ever.

I don't need nor mind 64-bit APIs, but I'd prefer an inclusive strategy instead of yet another lazy it-works-for-me strategy to reduce the size a small community. If someone likes your approach, then delete all "older programs" you've got now, start using any 64-bit OS, and pretend to be happy while browsing with a 64-bit browser.

Regarding 64-bit-APIs, I have no huge files myself, is that it would be nice if it was easier to always support huge files with "older programs", without having to write onesr own LONGLONG-code. Now 2 GiB often is the implied limit, just because it's too hard to support DosOpenL() for a few files of a few users. But we don't have to discuss this here, because "old programs", including but not limited to some older compilers, is not your focus.
<silence>
Title: Re: OS/2 application developers
Post by: A. Demetrious Sharpe, Sr. on December 08, 2017, 04:37:16 pm
We may need 64-bit for a 64-bit browser. That's it. And maybe for a 64-bit compiler for at least one user, to compile such a browser.

That's already covered. Though, in the future, those things will have to be converted to the 64-bit implementation of OS/2's API in order to be native.

Quote
If you'd expect that people want to throw away 20+ years of investements, if anything of time, to browse, then they may as well throw away their software now and start using a 64-bit browser for your 64-bit Windows.

Who said anything about throwing away anything?

Quote
I don't really care what your private focus is, but then I'd suggest to throw away "the older programs", i.e. everything for 16- and 32-bit OS/2, and start using whatever already existing 64-bit OS with a 64-bit browser software. So it's not the best point of view ever.

Ok, so now YOU want to throw away 20+ years of work? No, that's not the best point of view ever -which is why it's not my point of view.

Quote
I don't need nor mind 64-bit APIs, but I'd prefer an inclusive strategy instead of yet another lazy it-works-for-me strategy to reduce the size a small community. If someone likes your approach, then delete all "older programs" you've got now, start using any 64-bit OS, and pretend to be happy while browsing with a 64-bit browser.

Your own assumptions are causing you to argue against something that I've never said nor implied. At this point, your arguing with yourself -which is why you seem to be having a hard time simply answering the question as it was asked.

Quote
Regarding 64-bit-APIs, I have no huge files myself, is that it would be nice if it was easier to always support huge files with "older programs", without having to write onesr own LONGLONG-code. Now 2 GiB often is the implied limit, just because it's too hard to support DosOpenL() for a few files of a few users. But we don't have to discuss this here, because "old programs", including but not limited to some older compilers, is not your focus.

Look, at the end of the day, those old programs will NEVER support huge files. Those programs were written based on certain assumptions that were perfectly valid for the OS/2 API of that timeframe. Now, that time has passed. Obviously, support for running those programs is something that's deliverable -albeit in a very limited way. However, this post isn't about programs that were written in the past, it's about programs that will be written for future implementations of the OS/2 API. And that's precisely why I posed the question to APPLICATION PROGRAMMERS. If you are not such a programmer, or if you're someone who has no idea of what goes on behind the APIs, then this post obviously was NOT for you. Your temper tantrums have no place here.
Title: Re: OS/2 application developers
Post by: A. Demetrious Sharpe, Sr. on December 08, 2017, 04:39:16 pm
The point is to push the community forward, not to stay mired in the past. And if we MUST go forward (& yes, we MUST), then we might as well do it with an updated API.
Title: Re: OS/2 application developers
Post by: Olafur Gunnlaugsson on December 08, 2017, 05:15:16 pm
I'm looking at 64-bit OS/2 the way Microsoft looked at 64-bit Windows.

Why?

Microsoft implemented a full software emulation of 32 bit x86 Windows  (WOW64) because they developed the 64bit version on Compaq Alpha servers that had no virtualisation support of any kind and similarly were not x86 compatible in any way. It was not until the AMD64 architecture (Which Intel licences from AMD as well) that they released it and then utilised the x86 hardware emulation available in the AMD64 architecture (they had been using Windows Server Alpha 64 in-house since 2000).

This made perfect sense 20 years ago but today we have AMD64 compatible chips with shedloads of cores, fairly reasonable virtualisation features and memory is cheap to boot. Why not implement a container based mechanism where a real instance of OS/2 lives that can communicate with other containers, and develop a 64 bit OS/2 workalike in another container. This would alleviate the need for developing any sort of OS/2 compatibility or x86 emulation in the beginning allowing developers to focus on the new OS or new shell on top of an existing OS. Already existing open source virtualisation software may actually be up to the task with some modifications.

The plus would also be the possibility of running other OS's in containers and of developing and debugging the system on the same PC using different containers.
Title: Re: OS/2 application developers
Post by: A. Demetrious Sharpe, Sr. on December 08, 2017, 05:54:30 pm
Why?

Microsoft implemented a full software emulation of 32 bit x86 Windows  (WOW64) because they developed the 64bit version on Compaq Alpha servers that had no virtualisation support of any kind and similarly were not x86 compatible in any way. It was not until the AMD64 architecture (Which Intel licences from AMD as well) that they released it and then utilised the x86 hardware emulation available in the AMD64 architecture (they had been using Windows Server Alpha 64 in-house since 2000).

This made perfect sense 20 years ago but today we have AMD64 compatible chips with shedloads of cores, fairly reasonable virtualisation features and memory is cheap to boot. Why not implement a container based mechanism where a real instance of OS/2 lives that can communicate with other containers, and develop a 64 bit OS/2 workalike in another container. This would alleviate the need for developing any sort of OS/2 compatibility or x86 emulation in the beginning allowing developers to focus on the new OS or new shell on top of an existing OS. Already existing open source virtualisation software may actually be up to the task with some modifications.

The plus would also be the possibility of running other OS's in containers and of developing and debugging the system on the same PC using different containers.

Because being satisfied with the virtualization of a platform that's no longer being developed only ensures that the platform continues to NOT be developed. It's time to move forward. It's time for OS/2 environments to exist as first class citizens & run natively on current hardware. It's also time for the pool of programmers that're familiar with the platform to expand. We're not going to get new developers to write software for platforms that they consider to be dead. It's time to breath new life into this community. Otherwise, it dies with it's current users -as they eventually die.
Title: Re: OS/2 application developers
Post by: Andreas Schnellbacher on December 08, 2017, 06:19:27 pm
Strange idea. Is your plan to take a recent compiler (gcc I guess) and get it to produce 64-bit OS/2 code, you'll also have to create the specifications for? Do you plan after that, only some decades later, to revive an OS from the 80s with a compatibility mode for 32-bit and also for 16-bit applications? Who would be interested in that?

The main problem is: Who should do that? What do you have to deal with to gather enough money to hire your divisions of developers?

You are 10 ... 20 years late. But without having the source code from IBM, it makes no sense to start with something like this when OSS possibilities exist.
Title: Re: OS/2 application developers
Post by: Olafur Gunnlaugsson on December 08, 2017, 08:09:05 pm
Because being satisfied with the virtualization of a platform that's no longer being developed only ensures that the platform continues to NOT be developed. It's time to move forward. It's time for OS/2 environments to exist as first class citizens & run natively on current hardware. It's also time for the pool of programmers that're familiar with the platform to expand. We're not going to get new developers to write software for platforms that they consider to be dead. It's time to breath new life into this community. Otherwise, it dies with it's current users -as they eventually die.

There must be some sort of a misunderstanding here, OS/2 will not be developed in the future and has not been developed now for more than 20 years. I was giving a rational alternative for those who want to move forward while still retaining their own software for the time being, rather than take on the mammoth task of emulating the (humongous and outdated) OS/2 API from the outset. And you are contradicting yourself on technical terms, your proposal was emulation, I proposed virtualisation instead, which is a completely different technology.
Title: Re: OS/2 application developers
Post by: Neil Waldhauer on December 08, 2017, 09:08:01 pm
To answer the original poster's question, I want 64-bit OS/2 API to permit the implementation of libc and the rest of the netlabs RPM repository that would permit 64-bit compilation of newly ported open-source programs from other 64-bit platforms. Browsers, Office Suites, e-mail clients, image and movie editors all use too much 32-bit resources and need to be extended.

For most of the OS/2 32-bit API, I'd just want it to still work or be accessible through a 64-32 bit translation layer so existing code would recompile more easily.
Title: Re: OS/2 application developers
Post by: A. Demetrious Sharpe, Sr. on December 08, 2017, 09:21:55 pm
Strange idea. Is your plan to take a recent compiler (gcc I guess) and get it to produce 64-bit OS/2 code, you'll also have to create the specifications for? Do you plan after that, only some decades later, to revive an OS from the 80s with a compatibility mode for 32-bit and also for 16-bit applications? Who would be interested in that?

At this point, the plan is immaterial -which is why I've never mentioned what the plan is. I'm here asking application developers what they'd like to see in a potential 64 bit implementation of the OS/2 API. That's all.

Quote
The main problem is: Who should do that? What do you have to deal with to gather enough money to hire your divisions of developers?

This is also immaterial.

Quote
You are 10 ... 20 years late. But without having the source code from IBM, it makes no sense to start with something like this when OSS possibilities exist.

No, I'm not late. I'm right on time. However, other members of the community could have worked on a replacement for OS/2 long before things got to this point, but they didn't. So, they're late. As for source code from IBM, it's completely unnecessary. There's nothing that can be used from IBM's code that couldn't be better off made from scratch, when it comes to modern hardware. And if OSS possibilities were a solution, there'd already be a fully functional OS/2 replacement by now.
Title: Re: OS/2 application developers
Post by: A. Demetrious Sharpe, Sr. on December 08, 2017, 09:33:20 pm
There must be some sort of a misunderstanding here, OS/2 will not be developed in the future and has not been developed now for more than 20 years.

I've never said otherwise. My question was about OS/2's API potentially being implemented in 64-bit mode -I never said anything about OS/2 itself, nor what would actually run the API.

Quote
I was giving a rational alternative for those who want to move forward while still retaining their own software for the time being, rather than take on the mammoth task of emulating the (humongous and outdated) OS/2 API from the outset.

My question had absolutely nothing to do with emulation. In fact, my question was for application programmers, so obviously my attention was on future programs that will need to be written -this does not require emulation. In fact, I never brought up emulation.

Quote
And you are contradicting yourself on technical terms, your proposal was emulation, I proposed virtualisation instead, which is a completely different technology.

Your assumptions have caused you to read something into my original post that simply isn't there. If you're not a developer, then you probably have absolutely no idea of what I actually have in mind -and I'm not going to share it. For now, you'll have to just wait and see what (if anything) comes out of any of this.

Going forward, stop jumping to conclusions. It only serves to distract from the actual topic.
Title: Re: OS/2 application developers
Post by: A. Demetrious Sharpe, Sr. on December 08, 2017, 09:34:38 pm
To answer the original poster's question, I want 64-bit OS/2 API to permit the implementation of libc and the rest of the netlabs RPM repository that would permit 64-bit compilation of newly ported open-source programs from other 64-bit platforms. Browsers, Office Suites, e-mail clients, image and movie editors all use too much 32-bit resources and need to be extended.

For most of the OS/2 32-bit API, I'd just want it to still work or be accessible through a 64-32 bit translation layer so existing code would recompile more easily.

Fair enough.

This is the type of feedback that I'm looking for!
Title: Re: OS/2 application developers
Post by: Dave Yeo on December 08, 2017, 10:09:54 pm
Related to Neils post, it would be nice to have an up to date Posix layer including working full mmap, copy on write etc and better shared library support, no 8.3 limit, the capability of symlinking shared libraries and even versioned symbols. Using ELF would probably supply all this.
Title: Re: OS/2 application developers
Post by: A. Demetrious Sharpe, Sr. on December 08, 2017, 11:02:12 pm
Related to Neils post, it would be nice to have an up to date Posix layer including working full mmap, copy on write etc and better shared library support, no 8.3 limit, the capability of symlinking shared libraries and even versioned symbols. Using ELF would probably supply all this.

Thank you.
Title: Re: OS/2 application developers
Post by: Alfredo Fernández Díaz on December 16, 2017, 03:43:15 pm
Not a "serious" application developer in OS/2 land, I've coded some here and there.

The OS/2 API seems more than capable for everything I have wanted to do, I missed nothing -- I don't do 3D multimedia animation, or anything that flourished after the API was fleshed out, where I suspects most real lacks (if any) must lie. Sometimes the API forces doing things in a way that seems strange to me, but that may be because I'm not familiar enough with it. However, as the large file API has been mentioned, I'll say I'd like to see a 'clean' approach in some areas, *if* it's possible, and I'll use that as an example:

When file sizes were limited to 2GB, file position pointers, etc. could be expressed by a four-byte natural number (2^32 = 4GB), and that's the size of most file handling API arguments at the time. Support for anything beyond that "naturally" required new functions with would allow wider arguments, hence we have a parallel file API for large files.

From my point of view as a non-specialist in low-level implementation of such APIs, I'd like to have one working set of file APIs that would do its thing according to argument size (> vs <= 32 bits for starters), and any necessary redirecting APIs to address legacy applications. All APIs should be 'open' regarding that (argument size, or even number) if I am getting the point across (am I?). The FTP protocol comes to mind to further illustrate the point: at some point we needed newer clients and servers that could do 'huge' files; no protocol changes were necessary because that was handled in a more future-proof way (IIRC file size is transmitted as a decimal string, so any limitations of that are WAY into the future).
Title: Re: OS/2 application developers
Post by: Lars on December 18, 2017, 02:13:24 pm
Today, every application should use the "L" type of functions: DosOpenL, DosSetFileSizeL, DosSetFilePtrL, etc.
DosOpenL has a special parameter "OPEN_SHARE_DENYLEGACY" that will allow application A to open a file > 2 GB (if the filesystem supports that) while preventing application B to open shared that very same file via DosOpen (and therefore restricting itself right from the outset). Of course, the intention is to assure that BOTH applications can access the whole file.
Of course, the "L" variants will ensure that the underlying filesystems' capabilities (to support files > 2 GB) are completely transparent to the application. If the underlying filesystem does not support files > 2GB you will get an error if you attempt to create a file > 2 GB or try to access a fileoffset > 2 GB.

That's about as good as it gets. I was talking about the native OS/2 API. If you need proper RTL support (fstat, fstat64 etc.) then you would need to invest more effort.

In the year 2017, applications should be able to assume they are running on a kernel supporting the large file API and they should never ask for less.

1) Use "L" APIs, in particular DosOpenL
2) use "OPEN_SHARE_DENYLEGACY" on the DosOpenL call
Title: Re: OS/2 application developers
Post by: Alfredo Fernández Díaz on December 18, 2017, 02:54:14 pm
What Lars said, 'only' everywhere.

In the year 2017, this being a mostly theoretical question anyway, applications should expect APIs with no duplicate functions except maybe for legacy application support, and that will hit no practical limits in the foreseeable future in a stupid way: if sizes of anything being handled (memory size? file name length? what have you) would overflow this and that, then do it some intelligent way: assume it can be presented as a series of chunks, or however, to applications that are not too stupid (or old). At a minimum, any such things we have seen in the past, we shouldn't see in the future.

Arithmetic precision in REXX is arbitrary at the expense of lengthier/slower operation. I know there is a difference between interpreted languages and implementing a (presumably) low-level API, yadda yadda,  but the basic principle I allude to is the same one again: handle things (especially things that are likely to change in predictable ways, like continually skyrocketing sizes of everything) intelligently.

Over and out :)
Title: Re: OS/2 application developers
Post by: xynixme on December 18, 2017, 03:41:33 pm
Today, every application should use the "L" type of functions: DosOpenL, DosSetFileSizeL, DosSetFilePtrL, etc.

</silence>
Should. Not every development environment of every application is aware of a LONGLONG, and implementing your own LONGLONG variable is slightly harder than appending a reasonable L. If a development environment supports LONGLONG by default, then it resulted in a broken app (VAC3.08 -> VAC4FP2, IIRC).

I wish it was as easy as appending an L, and implementing my own LONGLONG is a bit over the top for an app which typically won't be used with huge files. Nevertheless I've looked at appending it, but it resulted in VAC4's fatal new errors.
<silence>
Title: Re: OS/2 application developers
Post by: RickCHodgin on December 18, 2017, 05:27:32 pm
FWIW, ES/2 goals are to use:

#include <os2.h> for all legacy apps as they were with IBM's OS/2 Warp 4.x.
#include <os2_32.h> for any newly added 16-bit and 32-bit app extensions.
#include <os2_64.h> for all new apps (be they 16-bit, 32-bit, or 64-bit).

    UPDATE:
    I have also considered adding direct support for ES/2 using new programming extensions:
    #include <es2.h> for all new apps.
    Note:  This add-on extension will likely only appear in ES/2's own custom compilers.


os2_32.h will auto-include all new functionality added to 16-bit and 32-bit code, including any extensions added to existing functions.  These will be exposed under new names where appropriate, or extended from existing names where appropriate.

os2_64.h will auto-migrate each function to its 32-bit or 64-bit counterpart by default, while allowing legacy functions to be accessed under a different name.  It will also introduce any new functionality that will be exposed through extensions to the 16-bit and 32-bit areas, as with os2_32.h.

-----
I think the goal has to be to maintain full backward compatibility so that everything which previously worked still works as it did with zero changes in the new kernel.  And any new development can work with new features using familiar function names without special prefixes, but just adjusted up to their 64-bit counterparts where appropriate.

That's the goal of ES/2 at least.
Title: Re: OS/2 application developers
Post by: Andi B. on December 18, 2017, 05:40:47 pm
Quote
</silence>
Should. Not every development environment of every application is aware of a LONGLONG, and implementing your own LONGLONG variable is slightly harder than appending a reasonable L. If a development environment supports LONGLONG by default, then it resulted in a broken app (VAC3.08 -> VAC4FP2, IIRC).

I wish it was as easy as appending an L, and implementing my own LONGLONG is a bit over the top for an app which typically won't be used with huge files. Nevertheless I've looked at appending it, but it resulted in VAC4's fatal new errors.
<silence>
Your a masochist if you're still developing new >2GB aware applications with VAC3.08. No serious application developer will do this. Did you forget the thread is about dreaming of an 64 bit OS/2? And you're talking about problems with a compiler from the last century.

Even for current OS/2 application development use VAC3.65 or VAC4 or even better OpenWatcom and gcc with os2tk45. And simply do -
Code: [Select]
#define INCL_LONGLONG
Title: Re: OS/2 application developers
Post by: xynixme on December 21, 2017, 02:50:22 pm
</silence>

Phase -1: implement your own LONGLONG
Phase 0: add a wrapper, DosOpenL isn't always available

Easy phases 1 and beyond: append a L to DosOpen or replace DosOpen()

VAC4's fatal new errors

FWIW: calculated output was different, compared to C and REXX. It may be a VAC3.65/VAC4 bug, it may be some change of the C programming language. Since I'm not being paid by the hour I stopped trying to append the L, mainly because of phase -1 and the errors.

<silence>
Title: Re: OS/2 application developers
Post by: xynixme on December 21, 2017, 03:18:55 pm
At a minimum, any such things we have seen in the past, we shouldn't see in the future.

Arithmetic precision in REXX is arbitrary at the expense of lengthier/slower operation.

</silence>

If one wants to promote DosOpenL as some new default, why not, then one has to make it easier to replace DosOpen/fopen/... by DosOpenL, regardless of the development environment (Cs, Pascals, BASICs, ...). Unfortunately it's harder than just appending the L.

I'm not sure there's such a thing as arithmetic precision in REXX, even if we ignore rounding due to NUMERIC DIGITS settings. No numbers, no arithmetic precision. With any programming language with 8-bit numbers you can compile a string which represents a result and stop when the length of the compiled string is N characters.

As such a proper design of DosOpenL isn't required. It's possible to use the best available API, which is e.g. DosOpen XOR DosOpenL. A wrapper takes care of that.

<silence>