OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Programming => Topic started by: Martin Iturbide on December 27, 2017, 03:12:33 pm

Title: Compiling an OS/2 Driver
Post by: Martin Iturbide on December 27, 2017, 03:12:33 pm
Hi

I want to make some little experiment to learn to compile a simple driver on OS/2, and if something works write some little/quick thing for EDM/2.

I have zero experience on this area (just like when I compiled some PM samples (http://www.edm2.com/index.php/Compiling_%22OS/2_Presentation_Manager_Programming%22_Samples_on_2016)). I'm just interested on the process of compiling the driver, since developing a driver is a lot more complex.

Do you recommend any drivers that is very straight forward to compile, just to try to understand the procedure ?

I have the IBM Device Driver Kit (Dated on 2004), so I can also use the tools and samples from it. And I also have some driver source code samples on the OS2World github (https://github.com/OS2World?utf8=%E2%9C%93&q=DRV-&type=&language=).

Regards
Title: Re: Compiling an OS/2 Driver
Post by: Eike on December 27, 2017, 05:48:45 pm
Hi Martin,

you know this?
http://www.edm2.com/index.php/List_of_Driver_Articles (http://www.edm2.com/index.php/List_of_Driver_Articles)

MfG
Wilfried
Title: Re: Compiling an OS/2 Driver
Post by: Martin Iturbide on December 27, 2017, 07:06:29 pm
Hi Eike.

I noticed that articles but are more focused on the development of the driver instead on just the compiling process. I'm seeking advice on which can be an easy sample to compile a driver for newbies.

Regards
Title: Re: Compiling an OS/2 Driver
Post by: Eike on December 27, 2017, 08:07:02 pm
Handouts for a session from warpstock europe in bochum 1999.
HTML and sources.

Have fun. ;)
Title: Re: Compiling an OS/2 Driver
Post by: Martin Iturbide on December 27, 2017, 09:22:16 pm
Thank you Eike.

I didn't known that material existed. I will check it out and see if I can compile that sample.

Regards
Title: Re: Compiling an OS/2 Driver
Post by: Andreas Kohl on December 27, 2017, 11:41:37 pm
It depends on which drivers from DDK should be built. For first timers I would recommend to set up a virtual machine with Warp 4. Two 2 GB partitions with FAT file system is a good start for the system installation. Then simply install C/2 or MS C 6.0. Now you can follow the route from The Developer Connection Device Driver Kit for OS/2 CD-ROM. The file USEDDK.INF from directory \DDKX86\BOOK includes a short overview and step-by-step instructions for each samples.

For drivers that only use 16-bit tools it can be easier to use Windows 2000 or DOS - at least sometimes.
Title: Re: Compiling an OS/2 Driver
Post by: Roderick Klein on December 29, 2017, 11:54:34 pm
https://www.youtube.com/watch?v=t9Oct3xUpLw

http://www.warpstock.eu/images/2015/presentations/David_Azarewicz-WSE2015_Writing_Device_Drivers.pdf

Does this help you Martin ?

Roderick
Title: Re: Compiling an OS/2 Driver
Post by: Andy Willis on December 30, 2017, 04:19:57 am
You put my HLT driver up on github... I had never written a driver so I used the HelloWorld driver to build it.
http://www.edm2.com/index.php/OS/2_Device_Drivers_for_Dummies_-_The_Beginning
Title: Re: Compiling an OS/2 Driver
Post by: Lars on December 30, 2017, 04:57:35 pm
Hi Martin,

my gut feeling is that you actually want to do more than just compiling a driver. My feeling is that you want to enable others to actually write a device driver, maybe you are thinking about people capable of writing a device driver for Windows but who have no clue about OS/2.

That said, I'd go with Roderick's hint. Have a look at David's driver library (for 16-bit and also for 32-bit).
Don't use the DDK for these purposes:
1) the DDK is a mess. It is unstructured, things are difficult to find.
2) the DDK mostly relies on the old 16-bit Microsoft C compiler and/or masm.exe, an extremly old Microsoft assembler that can only deal with nothing better than an 386 CPU.
2) I do not recommend to take any of the DDK code as being correct or suitable for SMP systems. Fact is, many of the DDK drivers are VERY old. You cannot rely on that code to work on any modern system, SMP systems in particular.

The only thing that is really necessary from the DDK are the header files as they basically define all of the internal interfaces/structures (as far as things have been made public to begin with).

Unfortunately, even with this info there are things about OS/2 that are really special:
1) a 16-bit driver is, well, 16-bit. For example, if you want to read a 32-bit HW register there is usually a HW requirement to read it as full 32-bits. That will require a minimum of assembler code because you will need to use memory instructions that will move a full 32-bit of data. A 16-bit compiler won't generate these instructions. There are more of these issues.

2) You will need to fumble around with virtual addresses: a selector and an offset. That is what most programmers nowadays will have forgotten about. Then there is the additional difficulty to hop back and forth between virtual addresses, linear addresses (a 32-bit offset in a flat 4 GB address space) and physical addresses. The reason for this most likely will be that virtual addresses can only address a "chunk" of memory <= 64 kBytes. More often than not this is not sufficient. For example, USBEHCD.SYS needs to address 128 kBytes of internal management data structures to make isochronous transfers work (something that you need to make USBAUDIO.SYS work, for example).


Lars
Title: Re: Compiling an OS/2 Driver
Post by: Martin Iturbide on December 31, 2017, 04:57:51 pm
Thanks Lars. Your feeling is right, but I will need to try it to see if I can accomplish something.

I found your post on the other thread interesting.
Here you go with 2 drivers that I have written:

http://hobbes.nmsu.edu/download/pub/os2/system/drivers/misc/mchnchk2.wpi
http://hobbes.nmsu.edu/download/pub/os2/system/drivers/misc/syscall.wpi

The WPIs contain all the source code. While they might not serve a practical purpose for most people, they are a live example of what can be done in a device driver.

And here is another one (including source code).
http://hobbes.nmsu.edu/download/pub/os2/system/drivers/misc/os2pcat3.wpi

The latter is a PSD. Hopefully the source code contains enough comments for people to understand :-)
I will check on that too.

Let me see how my testing goes so I can elaborate more specific question and the problems that may arise.

Thanks for help.

Regards
Title: Re: Compiling an OS/2 Driver
Post by: Roderick Klein on December 31, 2017, 06:27:32 pm
Also forgot about the Microsoft C compiler. Its better to use the Open Watcom compiler. It may not be suitable to compile Mozilla with as that requires GCC. The Open Watcom compiler seems to be much stricter in grabbing any errors in your code then Visual Age or the Microsoft 6 C compiler that is included in the DDK. Also the Open Watcom code I understands generates better optimized code.

If you do want to write an article recommending how to write drivers.

* Include some decent comment in the code.

* Any kind of source control system is your friend. Having worked at Mensys the fact  drivers where *frequently* checked in made it *easier* to find why a certain version of a driver would suddenly blow up. Its then easier to locate the code that was modified.

As I have said so often, finding the cause of a kernel dump can be extremely time consuming.  That is why the above conditions may cost some time when you work on you project. But you will be gladd you did it when you run into a problem.

Roderick
Title: Re: Compiling an OS/2 Driver
Post by: Lars on January 01, 2018, 12:53:19 pm
Hm,

unfortunately, Watcom produces debug info that no IBM supplied debugger can use, including ICSDEBUG.EXE (the debugger that comes with Visual Age) and ICAT (the remote source code debugger that can debug either via serial port or via UDP). This is also true if Watcom creates debug info in CODEVIEW format (looks like the CODEVIEW variant that Watcom generates is not understood by the IBM debuggers).

Just for that reason it can occasionally be better to still use Microsoft C 6.0 compiler as that one produces a CODEVIEW format that the IBM debuggers can understand.
If you don't care about source code debugging then of course this is not relevant.

Where that is possible I tend to create 2 makefiles: one for compiling/linking via Watcom and one for compiling/linking via VAC. Then I can develop using VAC and if everything works ok I can switch over to Watcom for the final build.

Lars

Title: Re: Compiling an OS/2 Driver
Post by: Martin Iturbide on January 03, 2018, 03:36:37 am
Hi.

I'm starting a little bit slow to compile the driver. I'm starting with the driver1.zip example. It is a little bit old but I wanted to try it.
I installed "os2tk45" with all the packages so "os2.h" and "os2def.h" are from there.

I'm getting this error when I run "nmake".

If I have the "startup.obj" already generated I get his message:

Code: [Select]
Microsoft (R) Program Maintenance Utility   Version 1.13
Copyright (c) Microsoft Corp 1988-91. All rights reserved.

        masm -Mx -e -t -l -N startup;
 Assembling: startup.asm
        cl -c -Asnw -Gs -G2 -Zl -Zp -Ox -Fadriver1.cod driver1.c
Microsoft (R) C Optimizing Compiler Version 6.00A.04
Copyright (c) Microsoft Corp 1984-1990. All rights reserved.

driver1.c
C:/usr/include/os2tk45\os2def.h(99) : error C2054: expected '(' to follow '_Seg1
6'
C:/usr/include/os2tk45\os2def.h(99) : error C2061: syntax error : identifier 'PU
CHAR16'
NMAKE : fatal error U1077: 'C:\OS2\CMD.EXE' : return code '2'
Stop.


Title: Re: Compiling an OS/2 Driver
Post by: Dave Yeo on January 03, 2018, 04:42:39 am
Using too new of a toolkit? Looking at my DDK\base\h directory, that line is guarded, eg
Code: [Select]
   #ifdef __IBMC__
      typedef UCHAR     * _Seg16 PUCHAR16;
      typedef CHAR      * _Seg16 PCHAR16;
   #endif
along with a lot of other stuff that is also guarded. I'd guess you should be using the DDK headers. Hopefully someone more knowledgeable can chime in.
Edit:
Looking at env.cmd, it looks like I'm correct, you need the toolkit included with the DDK.
Title: Re: Compiling an OS/2 Driver
Post by: Lars on January 03, 2018, 10:01:22 am
If you compile with a 16-bit compiler (which cl.exe is), make sure you include the DDK header file directories before you include the TOOLKIT directories.
In fact, you will very likely not need to include the TOOLKIT header files at all if you include the DDK header files.

Be absolutely sure that you reset/delete the INCLUDE and LIB environment variables before you run the makefile and that you properly define the includes and libraries via the proper compiler and linker switches/directives. I have been caught by the problem of including the wrong header and library files more than once.

Lars
Title: Re: Compiling an OS/2 Driver
Post by: Andreas Kohl on January 03, 2018, 11:19:27 am
The setting of environment seems to be incomplete. The output of SET INCLUDE or SET LIB would be more helpful. Additionally PATH should be adjusted properly.

I would check the installation of the C PDS. Which runtime libraries were created by the installation procedure? There's no matching entry inside INCLUDE btw. Simply read the product documentation. In "Microsoft C Advanced Programming Techniques" starting on page 343 (Part 4 OS/2 Support), for the compiler errors "Microsoft C Reference" or later MSDN information.
Title: Re: Compiling an OS/2 Driver
Post by: Martin Iturbide on January 03, 2018, 05:20:52 pm
Hi.

Yes, the .h files path was wrong. I notices I missed a lot from DDK.

It seems it compiles, a driver.sys file was generated.

Quote
   masm -Mx -e -t -l -N startup;
 Assembling: startup.asm
   cl -c -Asnw -Gs -G2 -Zl -Zp -Ox -Fadriver1.cod driver1.c
driver1.c
   link /nod /map /far startup.obj+driver1.obj,driver1.sys,driver1.map,dhcalls.lib+slibcep.lib+os2386.lib,driver1.def

Microsoft (R) Segmented-Executable Linker  Version 5.13
Copyright (C) Microsoft Corp 1984-1991.  All rights reserved.

   mapsym driver1.map


Operating System/2 Symbol File Generator

Version 4.00.000 Oct  4 2001

Copyright (C) IBM Corporation 1988-2001

Copyright (C) Microsoft Corp. 1988-1991.

All rights reserved.


No entry point, assume 0000:0100

It even worked at boot showing the sample text.



Title: Re: Compiling an OS/2 Driver
Post by: Martin Iturbide on January 03, 2018, 10:52:33 pm
Hi again.

As my next exercise on compiling drivers I would try like to try to compile the same driver with open/current tools.

Checking the make file I would like to hear your opinions on:
- Replacing NMAKE.EXE for GNU MAKE.EXE?
- Replacing link.exe for GNU Link.exe ?
- mapsym.exe? Is there a replacement for it?
- masm.exe? Would a different alternative compiler will produce a working driver?
- cl.exe? Would a different alternative compiler will produce a working driver?

I would check some GNU Make files to see if I can adapt the make file of this sample.

Regards
Title: Re: Compiling an OS/2 Driver
Post by: Dave Yeo on January 03, 2018, 11:58:34 pm
I believe you will have to go with the OpenWatcom tool chain.
Wmake is close to nmake and IIRC, it has a compatibility mode.
Wlink can link OS/2 drivers, including mixing 16 bit and 32 bit I believe. GNU ld can't.
I believe there are Perl scripts that can do the same as mapsym and they are needed for creating sym files for OW..
Same with OW's compiler and assembler, they can produce 16 bit OS/2 code and are closer to the MS tools.

GCC and the rest of the toolchain just aren't capable of building OS/2 drivers or even 1.x programs and even for 32 bit binaries, really need to be used with an IBM linker or the OpenWatcom linker.

Note also that OpenWatcom comes with sample device driver code, namely a timer.sys and example code to test by the looks of it.
Title: Re: Compiling an OS/2 Driver
Post by: Martin Iturbide on January 04, 2018, 12:24:48 pm
Thanks for the suggestion Dave. I will look that way and try to compile the openwatcom sample.

Regards
Title: Re: Compiling an OS/2 Driver
Post by: Dave Yeo on January 04, 2018, 04:33:21 pm
Steven's site has map2symw.pl for creating sym files from OW map files, http://home.earthlink.net/~steve53/os2diags/mapsymw-0.3-20120319.zip (http://home.earthlink.net/~steve53/os2diags/mapsymw-0.3-20120319.zip)
Title: Re: Compiling an OS/2 Driver
Post by: Martin Iturbide on January 04, 2018, 10:04:32 pm
Hi

Thanks for the support.
I had wrote down my experience on the EDM/2 wiki: Compiling a Driver Sample with IBM DDK on ArcaOS 5.0.1 (http://www.edm2.com/index.php/Compiling_a_Driver_Sample_with_IBM_DDK_on_ArcaOS_5.0.1). (Spellcheck is always welcome)

Now I had installed OpenWatcom and see if I can play with the make file to change the compilation tools on the same sample.

Regards
Title: Re: Compiling an OS/2 Driver
Post by: Dave Yeo on January 05, 2018, 01:53:29 am
Looks good Martin.
I did some minor editing.
Title: Re: Compiling an OS/2 Driver
Post by: Martin Iturbide on January 05, 2018, 12:58:49 pm
Thanks Dave. I look at your changes and I really needed those (sometimes I skip words while writing).

I'm have a different question now. I had installed OpenWatcom (Version 1.90 Installer for C/C++) and I noticed it sets several include files on the path. Those files conflict with the OS/2 Toolkit .h files (like os2.h and several others). What are the recommended one to use? Or if I'm going to use the OpenWatcom compilers I should use the ones bundled with it?

Regards
Title: Re: Compiling an OS/2 Driver
Post by: Dave Yeo on January 05, 2018, 06:15:39 pm
I've never done driver development, but for 32 bit applications, while in config.sys I have the toolkit stuff in front of the OW stuff, I have a setenv.cmd that puts the OW stuff in front of the toolkit. It is easy to experiment using a setenv.cmd.
For building Doodles Screen Saver I ended up with,
setenv.cmd,
Code: [Select]
call owsetenv
call tkenv
call start-sdk

owsetenv.cmd,
Code: [Select]
@echo off
echo Open Watcom Build Environment
SET PATH=g:\WATCOM\BINP;g:\WATCOM\BINW;%PATH%
SET BEGINLIBPATH=g:\WATCOM\BINP\DLL;%BEGINLIBPATH%
SET INCLUDE=g:\WATCOM\H;g:\WATCOM\H\OS2;%INCLUDE%
SET WATCOM=g:\WATCOM
SET EDPATH=g:\WATCOM\EDDAT
SET WIPFC=g:\WATCOM\WIPFC

tkenv.cmd
Code: [Select]
ECHO OFF
set os2tk=G:\OS2TK45
set BEGINLIBPATH=%OS2TK%\DLL;%BEGINLIBPATH%
set PATH=%OS2TK%\BIN;%PATH%
SET DPATH=%OS2TK%\MSG;%DPATH%
SET HELP=%OS2TK%\HELP;%HELP%
SET BOOKSHELF=%OS2TK%\BOOK;%BOOKSHELF%
SET NLSPATH=%OS2TK%\MSG\%N;%NLSPATH%
SET IPFC=%OS2TK%\IPFC;%IPFC%

For the SNAP sdk, only needed for DSS,
start-sdk.cmd
Code: [Select]
@echo off
rem Batch file to create the initial setup for using the SciTech MGL
rem Graphics Library, using the selected default compiler.

rem Setup the default path environment variable

set OLDPATH=%PATH%
set DEFPATH=C:\OS2;C:\OS2\MDOS
PATH %DEFPATH%

rem Initialise the default compile time environment

call C:\SCITECH\bin-os2\set-vars.cmd
set DEFPATH=%SCITECH%\redist\release;%DEFPATH%
rem call C:\SCITECH\bin-os2\wc11-o32.cmd
call C:\SCITECH\bin-os2\ow10-o32.cmd

rem Change to the directory containing all the MGL sample code

cd %SCITECH%\EXAMPLES\SNAP\GRAPHICS

rem restore old path
set PATH=%PATH%;%OLDPATH%
Title: Re: Compiling an OS/2 Driver
Post by: Andreas Kohl on January 05, 2018, 06:38:56 pm
Watcom uses different includes for 16-bit OS/2. That should be mentioned in the README or in more detail Tools Guide. For physical device driver information simply read chapter 39 Developing an OS/2 Physical Device Driver from the Watcom C/C++ Programmer's Guide. At least everything works as usual under Open Watcom 1.8. The later 1.9 may have some issues.