Author Topic: The OS/2 API Project at EDM/2  (Read 6356 times)

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 7
  • -Receive: 87
  • Posts: 1098
    • View Profile
Re: The OS/2 API Project at EDM/2
« Reply #15 on: June 24, 2016, 02:20:55 am »
It might exist in a DLL and only referenced by ordinal, and without knowing the ordinal. There are undocumented functions hiding behind ordinal numbers, eg a comment in Cairo/src/cairo-os2-private.h says
Quote
/**
 * Unpublished API:
 *   GpiEnableYInversion = PMGPI.723
 *   GpiQueryYInversion = PMGPI.726
 **/
It also exists on the DevCon3 CD according to this article, http://www.verycomputer.com/3_93ea71e0279eddf3_1.htm and probably in the PPC version of OS/2

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 12
  • -Receive: 2
  • Posts: 130
    • View Profile
    • Liberty Software Foundation
Re: The OS/2 API Project at EDM/2
« Reply #16 on: June 24, 2016, 05:56:15 am »
Quote
On the latest cp1.inf I can only find "VioScrollDown" and not "VioScrollDn". I can only find "VioScrollDn" as legacy on prcp.inf.

Me too. Apparently 32 bit versions of the 16 bit legacy functions have been documented. Unfortunately these are not supported by the header and library files. Only the 16 bit legacy functions are in there. Therefore these are the only ones available to an application programmer.

You may be able to still access those functions if you know their name and which dll they reside in.  You can use a function equivalent of LoadLibrary() and GetProcAddress(), which will open the dll and locate the entry point within the dll to access the function.  If the 32-bit equivalents / wrappers to the 16-bit versions exist, they may be exposed only in that way.

It is possible, for example, that they do exist internally, but are not exposed through the normal header files because the IBM authors did not want people to use them, but only those people who really really had a need and therefore knew what they were doing by accessing the functions directly.

Here is some code in my Visual FreePro project whereby I access functions explicitly manually within a DLL by name:

See lines 226 and 230
Note:  OSS stands for operating system specific.

It's also possible they do not exist.  A enumeration of the functions actually contained in each DLL would be desirable to then compare against the names exposed through documented source code APIs.

Best regards,
Rick C. Hodgin
« Last Edit: June 24, 2016, 05:58:29 am by Rick C. Hodgin »

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 7
  • -Receive: 87
  • Posts: 1098
    • View Profile
Re: The OS/2 API Project at EDM/2
« Reply #17 on: June 24, 2016, 06:53:57 am »
The problem is that most OS/2 DLLs export functions by ordinal rather then by name, so you need to know the ordinal number of a function to directly call it. eg the Doscalls entry points (the loader does some magic here and can resolve by name sometimes and also will fallback to doscall1 if not loaded from doscalls or the version in doscalls is ring 0 code and a user needs ring3 as I understand it) http://home.clara.net/orac/os2/doscalls.htm, who knows if anything is between 1117 and 9004 or what 9009, 9012-9018 and 9019+ reference.

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 12
  • -Receive: 2
  • Posts: 130
    • View Profile
    • Liberty Software Foundation
Re: The OS/2 API Project at EDM/2
« Reply #18 on: June 24, 2016, 07:26:30 am »
The problem is that most OS/2 DLLs export functions by ordinal rather then by name, so you need to know the ordinal number of a function to directly call it. eg the Doscalls entry points (the loader does some magic here and can resolve by name sometimes and also will fallback to doscall1 if not loaded from doscalls or the version in doscalls is ring 0 code and a user needs ring3 as I understand it) http://home.clara.net/orac/os2/doscalls.htm, who knows if anything is between 1117 and 9004 or what 9009, 9012-9018 and 9019+ reference.

Has anyone tried requesting ordinals between 1117 and 9004 to see if it returns something valid?  If so, we could disassemble the code within and see if it's a wrapper to any other function.

Here's a simple disassembler I wrote for 32-bit code (no direct 16-bit support without explicit overrides).  It compiles under Windows using Visual Studio (and will probably still compile under GCC as I originally wrote it in Eran Ifrah's CodeLite using the GCC+GDB toolchain in Linux, though it might need a few tweaks).

Best regards,
Rick C. Hodgin
« Last Edit: June 24, 2016, 07:37:58 am by Rick C. Hodgin »

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 7
  • -Receive: 87
  • Posts: 1098
    • View Profile
Re: The OS/2 API Project at EDM/2
« Reply #19 on: June 24, 2016, 08:05:58 am »
Has anyone tried requesting ordinals between 1117 and 9004 to see if it returns something valid?  If so, we could disassemble the code within and see if it's a wrapper to any other function.

I don't know if anyone has requested the ordinals and requesting from user land might fail anyways if they're ring 0 functions. (some functions come in both ring0 and ring3 versions)
As for disassembling, understand that doscalls is split between the kernel and doscall1.dll though I guess the OS4 guys may well have as they've re-implemented both the kernel and doscall1.dll.

I'm right at the edge of my knowledge of this stuff and perhaps beyond.

Wim Brul

  • Full Member
  • ***
  • Thank You
  • -Given: 3
  • -Receive: 28
  • Posts: 130
    • View Profile
    • Wim's home page
Re: The OS/2 API Project at EDM/2
« Reply #20 on: June 24, 2016, 11:57:05 am »
I want to check the headers of the OS/2 Toolkit 4.5 and see if I can understand those and try to make a quick list (on other page of the wiki) on which functions may be available on the CP1 documentation, but missing on the headers.

Since I see that all the headers on toolkit are on /H directory and I only want to focus on the CPI components for the moment, which are the .H files that I should be looking at? Does this question makes sense ?

The main include is OS2.H which leads you to BSE.H which in turn leads you to BSEDOS.H (for Dos functions) and BSESUB.H (for Kbd, Mou and Vio functions).

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Thank You
  • -Given: 244
  • -Receive: 46
  • Posts: 1642
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: The OS/2 API Project at EDM/2
« Reply #21 on: June 24, 2016, 04:33:01 pm »
Thanks Wim. I will be checking those headers from the OS2TK45.
I'm going to mark as "* Not Implemented" the ones that I can found.

For the moment I got:
- VioScrollDown
- VioScrollLeft
- VioScrollRight

Anybody that know any other functions, please post it so I can also validate it and mark those.

As for disassembling, understand that doscalls is split between the kernel and doscall1.dll....

This is also interesting. For what I know DOSCALLS.DLL is from older OS/2 version and the latest ones has DOSCALL1.DLL. But I never was able to understand or list which functions are now on the kernel, I guess that is the OS2KRNL file. I also  want to update the kernel part the OS2 API project, but I want to focus on CPI first. Maybe I will ask this later.

Regards

Martin Iturbide
OS2World NewsMaster
... just share the dream.

Wim Brul

  • Full Member
  • ***
  • Thank You
  • -Given: 3
  • -Receive: 28
  • Posts: 130
    • View Profile
    • Wim's home page
Re: The OS/2 API Project at EDM/2
« Reply #22 on: June 24, 2016, 05:47:26 pm »
Hi Martin,

When I look into \TOOLKIT\OS2TK45\H\BSESUB.H then I see that e.g. VioScrollUp is in reality supported by VIO16SCROLLUP and that the argument list described in cp1.inf is in fact incorrect since the argument list described in prcp.inf applies. I checked several other Kbd, Mou and Vio functions and these all appear to be supported the same way i.e. their argument list in cp1.inf is incorrect. Consequently all Kbd, Mou and Vio functions need to be marked as ** Not implemented *.

Regards


Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Thank You
  • -Given: 244
  • -Receive: 46
  • Posts: 1642
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: The OS/2 API Project at EDM/2
« Reply #23 on: June 24, 2016, 07:01:39 pm »
Hi.

So let me get the issues.

1) There are some functions that are listed on CP1 but not implemented on the headers of OS2TK45.

I searched the OS2TK45 *.h files and found out this ones that does not seems to have a definition on the headers:

Code: [Select]
VioScrollDown  - not found
VioScrollLeft  - not found
VioScrollRight   - not found
VioGetOrigin  - not found
VioSetOrigin   - not found
KbdGetConsole  - not found
KbdGetLayout  - not found
KbdGetLayoutUni  - not found
KbdSetLayout  - not found
DosSuppressPopUps  -  Looks strange. incomplete? No APIRET no Ordinal
DosDumpProcess  -  Looks strange. incomplete? No APIRET no Ordinal
DosForceSystemDump  - not found

Can someone validate these since my method is not fail-proof ?

In this case I plan to put some warning inside the functions. Something like "Warning: This function has not been found or implemented on the headers of OS2TK45".  Is it ok? any better wording?

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Thank You
  • -Given: 244
  • -Receive: 46
  • Posts: 1642
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: The OS/2 API Project at EDM/2
« Reply #24 on: June 24, 2016, 07:13:18 pm »
Hi. 

I didn't liked the other issue.

Issue 2) The second issue is that there functions on CP1.INF that are also on the legacy (prcp.INF), but the CP1 function is not specified correctly. Which it also means that those CP1 functions have not been implemented, and only the legacy documentation should apply. right?

For example:

VioScrollUp from CP1.INF says:

Code: [Select]
ULONG     TopRow;     /*  Top row to be scrolled. */
ULONG     LeftCol;    /*  Left column to be scrolled. */
ULONG     BotRow;     /*  Bottom row to be scrolled. */
ULONG     RightCol;   /*  Right column to be scrolled. */
ULONG     Lines;      /*  Number of lines. */
PBYTE     Cell;       /*  Cell to be written. */
HVIO      VioHandle;  /*  VIO presentation-space handle. */
APIRET    rc;         /*  Return code. */

rc = VioScrollUp(TopRow, LeftCol, BotRow,
       RightCol, Lines, Cell, VioHandle);

VioScrollUp from prcp.inf (legacy) says:

Quote
VioScrollUp   (TopRow, LeftCol, BotRow, RightCol, Lines, Cell, VioHandle)

TopRow (USHORT) - input
LeftCol (USHORT) - input
BotRow (USHORT) - input
RightCol (USHORT) - input
Lines (USHORT) - input
Cell (PBYTE) - input
VioHandle (HVIO) - input

OS2TK45 has:

bseord.h:
Code: [Select]
/* VIOCALLS */
#define ORD_VIOSCROLLUP          7


bsesub.h
Code: [Select]
#define VioScrollUp             VIO16SCROLLUP
...
#define VR_VIOSCROLLUP             0x00040000L
...   
   APIRET16 APIENTRY16  VioScrollUp (USHORT usTopRow,
                                     USHORT usLeftCol,
                                     USHORT usBotRow,
                                     USHORT usRightCol,
                                     USHORT cbLines,
                                     PBYTE pCell,
                                     HVIO hvio);


If you call VIOSCROLLUP but sending ULONG parameters (like CP1 says) will it not work?

Is there a way to confirm that CP1's VIOSCROLLUP (the new one) does not exists at all? Or what I posted is enough evidence? I'm not sure about it.

If it does not work, that would mean to me that I also need to document the legacy functions, and also put some warning on it that even that there is a "non-legacy function" you still need to use the legacy one because the newer one does not work. Does it makes sense?

I will need to think about it which will be a good way to document this on the CPI API Project.

Regards
« Last Edit: June 24, 2016, 07:21:22 pm by Martin Iturbide »
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Rick C. Hodgin

  • Full Member
  • ***
  • Thank You
  • -Given: 12
  • -Receive: 2
  • Posts: 130
    • View Profile
    • Liberty Software Foundation
Re: The OS/2 API Project at EDM/2
« Reply #25 on: June 24, 2016, 07:23:09 pm »
Hi. 

I didn't liked the other issue.

Issue 2) The second issue is that there functions on CP1.INF that are also on the legacy (prcp.INF), but the CP1 function is not specified correctly. Which it also means that those CP1 functions have not been implemented, and only the legacy documentation should apply. right?

I think the difference you're seeing is 16-bit and 32-bit.  The 16-bit functions use USHORT, which is a 2-byte (16-bit) quantity on x86 CPUs, whereas the ULONG is a 4-byte (32-bit) quantity.  The "U" means unsigned, and the SHORT is 16-bit, with LONG being 32-bit.

Variable size info.  Things are typically signed, but you must then prefix with "u" to make it unsigned, so "short" becomes "ushort", and "long" becomes "ulong", etc.

You're also seeing an evolution there over time of a preferred method of data exchange between functions.  The newer ULONG function includes an extra parameter which is the return value, rather than passing it externally.  This allows the destination of the return value to be specified as an input parameter, rather than relying on the calling convention.

Quote
For example:

VioScrollUp from CP1.INF says:

Code: [Select]
ULONG     TopRow;     /*  Top row to be scrolled. */
ULONG     LeftCol;    /*  Left column to be scrolled. */
ULONG     BotRow;     /*  Bottom row to be scrolled. */
ULONG     RightCol;   /*  Right column to be scrolled. */
ULONG     Lines;      /*  Number of lines. */
PBYTE     Cell;       /*  Cell to be written. */
HVIO      VioHandle;  /*  VIO presentation-space handle. */
APIRET    rc;         /*  Return code. */

rc = VioScrollUp(TopRow, LeftCol, BotRow,
       RightCol, Lines, Cell, VioHandle);

I don't think this is correct.  I think it should be like this with the return value passed in as a parameter:

Code: [Select]
VioScrollUp(TopRow, LeftCol, BotRow, RightCol, Lines, Cell, VioHandle, &rc);

In the alternative, it should be like this:
Code: [Select]
Input:
ULONG     TopRow;     /*  Top row to be scrolled. */
ULONG     LeftCol;    /*  Left column to be scrolled. */
ULONG     BotRow;     /*  Bottom row to be scrolled. */
ULONG     RightCol;   /*  Right column to be scrolled. */
ULONG     Lines;      /*  Number of lines. */
PBYTE     Cell;       /*  Cell to be written. */
HVIO      VioHandle;  /*  VIO presentation-space handle. */
Output:
APIRET    rc;         /*  Return code. */

rc = VioScrollUp(TopRow, LeftCol, BotRow,
       RightCol, Lines, Cell, VioHandle);

Quote
VioScrollUp from prcp.inf (legacy) says:

Quote
VioScrollUp   (TopRow, LeftCol, BotRow, RightCol, Lines, Cell, VioHandle)

TopRow (USHORT) - input
LeftCol (USHORT) - input
BotRow (USHORT) - input
RightCol (USHORT) - input
Lines (USHORT) - input
Cell (PBYTE) - input
VioHandle (HVIO) - input

OS2TK45 has:

bseord.h:
Code: [Select]
/* VIOCALLS */
#define ORD_VIOSCROLLUP          7


bsesub.h
Code: [Select]
#define VioScrollUp             VIO16SCROLLUP
...
#define VR_VIOSCROLLUP             0x00040000L
...   
   APIRET16 APIENTRY16  VioScrollUp (USHORT usTopRow,
                                     USHORT usLeftCol,
                                     USHORT usBotRow,
                                     USHORT usRightCol,
                                     USHORT cbLines,
                                     PBYTE pCell,
                                     HVIO hvio);

Is there a way to confirm that CP1's VIOSCROLLUP (the new one) does not exists at all? Or what I posted is enough evidence? I'm not sure about it.

Is there some source code that can be compiled and examined to see what it links to internally?  You'll be able to tell by the calling convention in the assembly output, if it pushes 16-bit or 32-bit quantities.

Quote
That means to me that I also need to document the legacy functions, and also put some warning on it that even that there is a "non-legacy function" you still need to use the legacy one because the newer one does not work. Does it makes sense?

I will need to think about it which will be a good way to document this on the CPI API Project.

Regards

Once I get my C/C++ compiler parser far enough along, maybe we can automate some of this and regenerate from the ground-up based on a system-wide examination of the DLLs and the header files and libraries, the entire system API.  By parsing it into this type of tool, we can then directly wield it and generate documentation, along with wrappers for other purposes.

I won't have that completed until later this year (Lord willing).  It's a fairly substantial task, and it's not slated to begin until sometime around September.

Best regards,
Rick C. Hodgin
« Last Edit: June 24, 2016, 07:51:26 pm by Rick C. Hodgin »

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Thank You
  • -Given: 244
  • -Receive: 46
  • Posts: 1642
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: The OS/2 API Project at EDM/2
« Reply #26 on: June 26, 2016, 07:19:08 pm »
Hi.

For the moment I will organize CPI API like I set it up on the page.

The latest documentation (CP1 and Addendum) is going to be on the top. 
The Legacy (PRCP) is going to be after that on the next block. On the legacy functions I'm going to name the pages of the functions:  "OS2 API:CPI:LEGACY:xxxxxxx" so they can not be confused with the CP1 ones. 

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Wim Brul

  • Full Member
  • ***
  • Thank You
  • -Given: 3
  • -Receive: 28
  • Posts: 130
    • View Profile
    • Wim's home page
Re: The OS/2 API Project at EDM/2
« Reply #27 on: July 03, 2016, 12:07:32 pm »
Quote
Issue 2) The second issue is that there functions on CP1.INF that are also on the legacy (prcp.INF), but the CP1 function is not specified correctly. Which it also means that those CP1 functions have not been implemented, and only the legacy documentation should apply. right?
Yes. CP1.INF contains tentative documentation of 32 bit functions intended to replace those16 bit legacy functions.
Quote
If you call VIOSCROLLUP but sending ULONG parameters (like CP1 says) will it not work?
That will not work.
Quote
Is there a way to confirm that CP1's VIOSCROLLUP (the new one) does not exists at all? Or what I posted is enough evidence? I'm not sure about it.
For me that is enough evidence. I also inspected os2386.lib and those tentative functions are absent  whereas those legacy functions are present.
Quote
If it does not work, that would mean to me that I also need to document the legacy functions, and also put some warning on it that even that there is a "non-legacy function" you still need to use the legacy one because the newer one does not work. Does it makes sense?
Only the 16 bit legacy functions need to be documented. I would omit that tentative documentation completely because it would only lead to confusion.

Martin Iturbide

  • OS2World NewsMaster
  • Global Moderator
  • Hero Member
  • *****
  • Thank You
  • -Given: 244
  • -Receive: 46
  • Posts: 1642
  • Your Friend Wil Declares...
    • View Profile
    • Martin's Personal Blog
Re: The OS/2 API Project at EDM/2
« Reply #28 on: July 03, 2016, 04:12:34 pm »
Thanks for the feedback.

Wim, can you let me know how to inspect the .LIB files?  Any tools? I really do now know anything about those files. How do you see it's internals?

Regards
Martin Iturbide
OS2World NewsMaster
... just share the dream.

Wim Brul

  • Full Member
  • ***
  • Thank You
  • -Given: 3
  • -Receive: 28
  • Posts: 130
    • View Profile
    • Wim's home page
Re: The OS/2 API Project at EDM/2
« Reply #29 on: July 03, 2016, 10:44:22 pm »
Wim, can you let me know how to inspect the .LIB files?  Any tools? I really do now know anything about those files. How do you see it's internals?

Hi Martin,

With the IBM Library Manager you can a.o. create a cross-reference listing file from a library file.

A cross-reference listing file contains two lists. The first is an alphabetic listing of all public symbols in the library. The name of the module, the symbol name refers to, comes after that symbol name.

The second list is an alphabetic list of the modules in the library. Under each module name is an alphabetic listing of the public symbols the module refers to.

Code: [Select]
lib.exe os2386.lib,os2386.lst;

You may find lib.exe in the ddk. It is not present in os2tk45. An alternative is the Microsoft Library Manager contained in c7os2.zip on hobbes.

Regards