OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Programming => Topic started by: Marcoapc on May 07, 2018, 10:09:08 am

Title: Alternatives to compiling Rust in Firefox port OS/2
Post by: Marcoapc on May 07, 2018, 10:09:08 am
There are alternatives to llvm, using gcc, but need to be ported to OS/2. 8)

https://www.phoronix.com/scan.php?page=news_item&px=MTUzMzM Mozilla's Rust Language Gets A GCC Compiler Front-End - Phoronix

https://gcc.gnu.org/wiki/RustFrontEnd

https://github.com/redbrain/gccrs

https://dragonegg.llvm.org/  Using LLVM as a GCC backend

https://github.com/thepowersgang/mrustc It is not gcc, but it may have port to OS/2

Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 08, 2018, 06:33:18 am
The problems are that RUST is a fast moving target with each new version of Firefox needing the latest version and it seems that it takes a lot of address space to link mixed RUST and C++ code to the point that it likely not linkable on a 32 bit OS.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Marcoapc on May 10, 2018, 09:38:19 am
This is one of the problems with rust, but is need to have more languages for OS/2!
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Carlos de Luna Saenz on January 07, 2021, 05:08:36 pm
I must agree and i think Rust can be a good news not just if someone wants to por Mozilla but for being a good framework where even the compiler checks some good  coding practices .
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on January 07, 2021, 06:12:54 pm
I agree, the other day I looked at librsvg, now needs Rust to compile. Has a bunch of GTK dependencies too, which I doubt are needed for what I want it for.
The problem is skilled developers and to port something like Rust will take special skills. Before that we really need to switch GCC from outputting AOUT object files to ELF which means updating the tool chain to produce OMF from ELF and once again no one with the skills has the time and motivation.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Rob on May 07, 2021, 07:30:56 am
What all is required to produce OMF from ELF? Is there a doc to read?
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: OS4User on May 07, 2021, 08:05:11 am
to produce OMF from ELF

What all is required to produce OMF from ELF? Is there a doc to read?

From my point of view - there is no need to produce OMF from ELF as WLINK, can generate DLL/EXE from ELF (however, it may be necessary to improve WLINK)
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 07, 2021, 08:39:38 am
What all is required to produce OMF from ELF? Is there a doc to read?

Here's a program that can do it, https://www.agner.org/optimize/#objconv (https://www.agner.org/optimize/#objconv) along with the documentation and source, it builds easy on OS/2, there's a copy on Hobbes.
I'm sure there is also lots of documentation out there on the OMF and the ELF formats.
As OS4User points out, another possibility is just staying with ELF until the linking stage and having a linker that can link ELF objects into LX executables and DLL's. Wlink can handle both file formats, whether it can currently build a LX executable or a DLL out of ELF obj files needs investigation. It's ELF support isn't the highest as it can't even create ELF shared libraries.
Either route needs someone who knows this stuff.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Lars on May 07, 2021, 09:14:44 am
Does EMXOMF only convert from AOUT to OMF but not from ELF to OMF ? I thought that if you called gcc with the -Zomf switch that then, OMF objects would he produced ?
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 07, 2021, 04:43:26 pm
Does EMXOMF only convert from AOUT to OMF but not from ELF to OMF ? I thought that if you called gcc with the -Zomf switch that then, OMF objects would he produced ?

Yes, only AOUT to OMF, with the conversion happening at compile or linking time depending on whether -Zomf is in CFLAGS as well as LDFLAGS. GCC outputs AOUT and then can call emxomf to convert the object files to OMF and we can use the native linker, native debugger etc. There's also tools like emxomfar for creating static OMF libs.
It is also possible to build executables and DLLs directly out of AOUT objects.
The problems with AOUT range from officially unsupported by GCC to just plain old too simple. Doesn't allow much in alignment options for example.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 07, 2021, 11:47:42 pm
Hey Dave,
The problems with AOUT range from officially unsupported by GCC to just plain old too simple. Doesn't allow much in alignment options for example.

Changing GCC to elf probably wouldn't be that hard - getting the necessary tools updated...... more difficult....
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 08, 2021, 02:04:58 am
Changing GCC to elf probably wouldn't be that hard - getting the necessary tools updated...... more difficult....

I have gcc-elf binaries from 2.91 built with EMX, they were needed to build XFree/2 as we loaded the ELF modules, could even just copy over a Linux video module.
Code: [Select]
[E:\emx\bin]dir *_elf*

The volume label in drive E is WARP_V4.
The Volume Serial Number is 6C5C:6414.
Directory of E:\emx\bin

11-07-99  2:24p       417,796    124 a---  ar_elf.exe
11-07-99  4:03p       450,564    124 a---  as_elf.exe
11-07-99  7:27p     2,711,556    124 a---  cc1plus_elf.exe
11-07-99  7:18p     2,297,746    124 a---  cc1_elf.exe
11-07-99  7:11p       194,534    124 a---  cpp_elf.exe
11-07-99  1:46p       135,172    124 a---  gasp_elf.exe
11-08-99  9:08p       145,075    124 a---  gcc_elf.exe
 2-18-97 12:37a       145,075    124 a---  GCC_ELF1.EXE
11-08-99  7:45p       413,700    124 a---  ld_elf.exe
11-07-99  2:24p       409,604    124 a---  nm_elf.exe
11-08-99  8:18p       417,796    124 a---  objcopy_elf.exe
11-07-99  2:23p       450,564    124 a---  objdump_elf.exe
11-07-99  2:23p       417,796    124 a---  ranlib_elf.exe
11-07-99  2:23p       405,508    124 a---  size_elf.exe
11-08-99  8:18p       417,796    124 a---  strip_elf.exe
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 08, 2021, 08:19:35 am
Hey Dave,

Changing GCC to elf probably wouldn't be that hard - getting the necessary tools updated...... more difficult....

I have gcc-elf binaries from 2.91 built with EMX, they were needed to build XFree/2 as we loaded the ELF modules, could even just copy over a Linux video module.
Code: [Select]
[E:\emx\bin]dir *_elf*

The volume label in drive E is WARP_V4.
The Volume Serial Number is 6C5C:6414.
Directory of E:\emx\bin

11-07-99  2:24p       417,796    124 a---  ar_elf.exe
11-07-99  4:03p       450,564    124 a---  as_elf.exe
11-07-99  7:27p     2,711,556    124 a---  cc1plus_elf.exe
11-07-99  7:18p     2,297,746    124 a---  cc1_elf.exe
11-07-99  7:11p       194,534    124 a---  cpp_elf.exe
11-07-99  1:46p       135,172    124 a---  gasp_elf.exe
11-08-99  9:08p       145,075    124 a---  gcc_elf.exe
 2-18-97 12:37a       145,075    124 a---  GCC_ELF1.EXE
11-08-99  7:45p       413,700    124 a---  ld_elf.exe
11-07-99  2:24p       409,604    124 a---  nm_elf.exe
11-08-99  8:18p       417,796    124 a---  objcopy_elf.exe
11-07-99  2:23p       450,564    124 a---  objdump_elf.exe
11-07-99  2:23p       417,796    124 a---  ranlib_elf.exe
11-07-99  2:23p       405,508    124 a---  size_elf.exe
11-08-99  8:18p       417,796    124 a---  strip_elf.exe

Interesting - got a link?

Cheers,

Paul
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 08, 2021, 08:49:52 am
It was on Holger's old pages which are long gone. Frank Giessler also offered to send me the package (60MB). Not sure what Frank is up to now a days. My original archive is also long gone it seems
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 08, 2021, 09:04:43 am
Here's the binaries, they don't seem to need anything besides the emx dlls.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 08, 2021, 09:57:06 am
Here's the binaries, they don't seem to need anything besides the emx dlls.
Thanks, but I'd prefer source so we can update where required....
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 08, 2021, 06:39:52 pm
Yes thought I'd post them anyways in case anyone else wants to play with them.
I'd imagine it is mostly configure options to build ELF binaries from your existing source. Testing is another thing, which is where those binaries might help.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 08, 2021, 11:33:49 pm
Hey Dave,,

Yes thought I'd post them anyways in case anyone else wants to play with them.
I'd imagine it is mostly configure options to build ELF binaries from your existing source. Testing is another thing, which is where those binaries might help.

Agree - except for ld-elf.exe - as we're not able to build a working a.out ld.exe from binutils source....
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 09, 2021, 01:15:24 am
Well, it may be possible to do a emxelfld that calls wlink.exe and outputs a LX binary. DLLs I'm not sure about, at that building something that needs imports from DLLs might be tricky, likely no 16 bit support at all as well.
Be interesting to see the SDK for the PowerPC version of OS/2 as that was ELF based.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: OS4User on May 09, 2021, 08:55:32 am
Well, it may be possible to do a emxelfld that calls wlink.exe and outputs a LX binary. DLLs I'm not sure about, at that building something that needs imports from DLLs might be tricky, likely no 16 bit support at all as well.
Be interesting to see the SDK for the PowerPC version of OS/2 as that was ELF based.

True. Wlink has some limitation for 16 bit DLL. But is it really nessasery for porting software ?
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 10, 2021, 05:04:54 am
Hey Dave,

Does EMXOMF only convert from AOUT to OMF but not from ELF to OMF ? I thought that if you called gcc with the -Zomf switch that then, OMF objects would he produced ?

Yes, only AOUT to OMF, with the conversion happening at compile or linking time depending on whether -Zomf is in CFLAGS as well as LDFLAGS. GCC outputs AOUT and then can call emxomf to convert the object files to OMF and we can use the native linker, native debugger etc. There's also tools like emxomfar for creating static OMF libs.
It is also possible to build executables and DLLs directly out of AOUT objects.
The problems with AOUT range from officially unsupported by GCC to just plain old too simple. Doesn't allow much in alignment options for example.

I did some more thinking about this over the weekend.... I'm not the statement GCC output AOUT is 100% correct..... Isn't it the assembler (as.exe) that produces the AOUT output? So by dropping in an as.exe built to use ELF - then GCC would produce ELF output? Linking is a whole another story, that would almost certainly require gcc changes - as ld.exe won't work, and whilst -Zomf would invoke wlink - it would also try and use emxomf to convert aout to OMF which would fail with ELF output.

I think I'll try a build of binutils with ELF output, then mess around with GCC manually to see what's possible. ie try and compile a helloworld.o with ELF output (using rebuilt as.exe); then see if an executable can be linked with wlink (manually from the command line initially).

Cheers,

Paul.

PS this is kinda getting off the topic - perhaps GCC and ELF output should be split out into a separate topic?
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 10, 2021, 06:33:26 am
Yes, it would be AS actually outputting the ELF objects, I didn't realize it was hard coded what AS outputs, thought more like NASM etc where you specify the output.
GCC would still need to pass the correct source (.S) code to AS. For example, -pic doesn't make sense on AOUT but does on ELF for shared libraries as shared libraries don't have fixups like DLLs do. So just guessing, both GCC and binutils would need to target ELF.
Then it is just the question of whether wlink can convert from ELF to a LX binary. Otherwise it would mean converting from ELF to OMF then linking. I know objconv can do it, though by default it has a 15 char symbol limit and doesn't want to add an underline without forcing it to with a parameter.
For C++, there is also a question of symbol mangling, no idea if it varies.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: OS4User on May 10, 2021, 09:43:00 am
Then it is just the question of whether wlink can convert from ELF to a LX binary.

Once  I linked  ELF produced by NASM into LX  using  Wlink.  It was just test.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 10, 2021, 11:02:04 am
Anyone know how to use wlink to create an executable from the command line?

Using:
wlink option quiet name helloworld.exe file helloworld.obj library libc_dll.lib

gives:
Code: [Select]
Error! E2028: ___main is an undefined reference
Error! E2028: _printf is an undefined reference
Warning! W1014: stack segment not found
Warning! W1023: no starting address found, using 0001:00000000
file helloworld.obj(helloworld.obj): undefined symbol ___main
file helloworld.obj(helloworld.obj): undefined symbol _printf

Cheers,

Paul
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Lars on May 10, 2021, 11:27:01 am
Anyone know how to use wlink to create an executable from the command line?

Using:
wlink option quiet name helloworld.exe file helloworld.obj library libc_dll.lib

gives:
Code: [Select]
Error! E2028: ___main is an undefined reference
Error! E2028: _printf is an undefined reference
Warning! W1014: stack segment not found
Warning! W1023: no starting address found, using 0001:00000000
file helloworld.obj(helloworld.obj): undefined symbol ___main
file helloworld.obj(helloworld.obj): undefined symbol _printf

Cheers,

Paul

Looks like you are not linking to the correct library/libraries. I suspect that libc_dll.lib is for creating a DLL rather than an exectuable (which then defines "dllmain" instead of "main" etc.). I think you should try the "system" keyword on the linker commandline. It should allow to easily specify what exactly you are intending to build and will hopefully then pull in the correct libraries.
See the linker help.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: OS4User on May 10, 2021, 11:50:42 am
Anyone know how to use wlink to create an executable from the command line?
Using:
wlink option quiet name helloworld.exe file helloworld.obj library libc_dll.lib


Try

wlink system os2v2 option map name helloworld.exe file helloworld.obj library libc.lib
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 10, 2021, 04:37:52 pm
Anyone know how to use wlink to create an executable from the command line?
Using:
wlink option quiet name helloworld.exe file helloworld.obj library libc_dll.lib


Try

wlink system os2v2 option map name helloworld.exe file helloworld.obj library libc.lib

What about crt0.obj? Or one of the other crt0*.obj files?
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Lars on May 10, 2021, 04:48:06 pm
I would hope that the system keyword will pull in correct startup code and libraries. It should never be necessary to explicitly link the correct startup code (that's just too involved and error prone for the average developer).
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 10, 2021, 05:06:00 pm
Wouldn't the system keyword bring in Watcom startup code and libraries?
I was for a while linking xul.dll from the command line, using emxomfld, needed due to GCC producing recursive response files when 9.9.2 was first released.
Can look at a map file to see the libs linked, for helloworld.cpp I see,
Code: [Select]
W:\usr\lib\gcc\i686-pc-os2-emx\9\libgcc_so_d.lib
W:\usr\lib\libc_alias.lib
W:\usr\lib\libc_dll.lib
W:\usr\lib\libend.lib
Also
Code: [Select]
Module: W:\usr\lib\crt0.obj(D:\Temp\ccD416Lu.s)

They'll need the full pathname or the equivalent of -L, LIBPATH.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 10, 2021, 07:01:07 pm
Perhaps something like,
Code: [Select]
emxomfld -Zno-autoconv -o hello.exe hello.o -l W:\usr\lib\gcc\i686-pc-os2-emx\9\libgcc_so_d.lib -l W:\usr\lib\libc_alias.lib -l W:\usr\lib\libc_dll.lib -l W:\usr\lib\libend.lib W:\usr\lib\crt0.obj

The -Zno-autoconv should stop emxomfld from trying to change the object to OMF and emxomfld otherwise should massage the output for wlink.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 10, 2021, 09:13:59 pm
Well, that fails with a weak prelinker fail.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 10, 2021, 09:44:01 pm
This might work, though it fails using the old gcc_elf like this,
Code: [Select]
H:\tmp>wl  @hello.lnk
** EXPERIMENTAL (HLL) ** Open Watcom Linker Version 2.0beta1 Limited Availability
Portions Copyright (c) 1985-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
loading object files
searching libraries
Error! E2028: printf is an undefined reference
Error! E2028: _main is an undefined reference
creating map file
Error! E2026: redefinition of reserved symbol _edata
Error! E2026: redefinition of reserved symbol __edata
Error! E2026: redefinition of reserved symbol _end
Error! E2026: redefinition of reserved symbol __end
Perhaps just the calling convention, with an OMF produced by GCC 9.2.0
Code: [Select]
H:\tmp>wl  @hello.lnk
** EXPERIMENTAL (HLL) ** Open Watcom Linker Version 2.0beta1 Limited Availabilit
y
Portions Copyright (c) 1985-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
loading object files
searching libraries
creating map file
creating an OS/2 and ArcaOS executable
And a working hello.exe

hello.lnk
Code: [Select]
FORMAT OS2 LX
DEBUG ALL
file Hello.o
file W:\usr\lib\crt0.obj
OPTION MAP=Hello
OPTION DESCRIPTION 'Elf test'
OPTION ELIMINATE
OPTION MANYAUTODATA
OPTION OSNAME='OS/2 and ArcaOS'
OPTION SHOWDEAD
LIBRARY W:\usr\lib\gcc\i686-pc-os2-emx\9\libgcc_so_d.lib
LIBRARY W:\usr\lib\libc_alias.lib
LIBRARY W:\usr\lib\libc_dll.lib
LIBRARY W:\usr\lib\libend.lib

Obviously adjust the libraries paths. It's a start anyways. Invoked as wl.exe @hello.lnk.  Does work given an OMF hello.o
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 11, 2021, 11:15:20 am
Thanks everyone... what works here is:

Code: [Select]
gcc -S helloworld.c
as_elf helloworld.s -o helloworld.o
wl @hello.lnk
** EXPERIMENTAL (HLL) ** Open Watcom Linker Version 2.0beta1 Limited Availability by SHL on Jul  2 2020
Portions Copyright (c) 1985-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
loading object files
searching libraries
creating map file
creating an OS/2 and ArcaOS executable

For interest - https://smedley.id.au/tmp/helloworld.zip

I have a day off on Friday, I'll try and think about how to modify GCC to use this stuff....

Cheers,

Paul
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 12, 2021, 04:20:19 am
I still think that emxomfld could be stripped down by not converting and bypassing the weak linker to call wl.exe. This may also result in needing the minimum changes to GCC.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 12, 2021, 05:01:46 am
Hey Dave,

I still think that emxomfld could be stripped down by not converting and bypassing the weak linker to call wl.exe. This may also result in needing the minimum changes to GCC.

I'm still thinking about how best to handle it. We may still WANT to convert to OMF in some cases in order to use the 'classic' debuggers.

Of course, switching to ELF may make gdb easier to compile/work.

Just thinking out load, I was considering introducing -Zelf to tell GCC to invoke as_elf.exe rather than as.exe - that also opens up the use of -Zelf -Zomf to use ELF initially, but convert to OMF at link time (if we have a suitable ELF -> OMF converter that makes the exercise worthwhile).

I'll probably hack something together short term and make sure that 'stuff' works, then think about how best to support a.out, elf & OMF object formats in a more flexible manner...

Cheers,

Paul.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 12, 2021, 06:54:27 am
It's the ELF to OMF converter that is hard. It's a shame that the source for elfomf is lost, I have a binary but being GPL...
There is also objconv, https://www.agner.org/optimize/#objconv (https://www.agner.org/optimize/#objconv) which I noticed didn't like our OMF when I tried to convert to ELF.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: OS4User on May 12, 2021, 07:05:04 am
if we have a suitable ELF -> OMF converter that makes the exercise worthwhile

"ELF -> OMF converter"   is not  a good way  from point of view of alignment.

AFAIK   OMF support  2,4,16,4K  only  and thus  data   cannot be aligned  to 64 and 128
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 12, 2021, 08:02:15 am
@os4user I'm not disagreeing with you, but if we don't convert to omf, how do we debug on OS/2?
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 12, 2021, 08:20:45 am
"ELF -> OMF converter"   is not  a good way  from point of view of alignment.

AFAIK   OMF support  2,8,16,4K  only  and thus  data   cannot be aligned  to 64 and 128

From the NASM docs, https://www.nasm.us/xdoc/2.15.05/html/nasmdoc8.html#section-8.4.1 (https://www.nasm.us/xdoc/2.15.05/html/nasmdoc8.html#section-8.4.1)
Quote
ALIGN is used, as shown above, to specify how many low bits of the segment start address must be forced to zero. The alignment value given may be any power of two from 1 to 4096; in reality, the only values supported are 1, 2, 4, 16, 256 and 4096, so if 8 is specified it will be rounded up to 16, and 32, 64 and 128 will all be rounded up to 256, and so on. Note that alignment to 4096-byte boundaries is a PharLap extension to the format and may not be supported by all linkers.

Now what wlink supports is another question.


Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: OS4User on May 12, 2021, 02:45:24 pm
From the NASM docs, https://www.nasm.us/xdoc/2.15.05/html/nasmdoc8.html#section-8.4.1 (https://www.nasm.us/xdoc/2.15.05/html/nasmdoc8.html#section-8.4.1)

from OMF spec:
...
A  Alignment
A 3-bit field that specifies the alignment required when this program segment is placed within a
logical segment. Its values are:
   0 Absolute segment.
   1 Relocatable, byte aligned.
   2 Relocatable, word (2-byte, 16-bit) aligned.
   3 Relocatable, paragraph (16-byte) aligned.
   4 Relocatable, aligned on a page boundary. (The original Intel 8086 specification
      defines a page to be 256 bytes. The IBM implementation of OMF uses a 4096-byte or 4K  page size).
   5 Relocatable, aligned on a double word (4-byte) boundary.
   6 Not supported.
   7 Not defined.

So for OMF we have 1,2,4,16,4K  only, with no relation what is supported by NASM.

Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 13, 2021, 10:39:58 am
Hi All,

I still think that emxomfld could be stripped down by not converting and bypassing the weak linker to call wl.exe. This may also result in needing the minimum changes to GCC.

I'm still thinking about how best to handle it. We may still WANT to convert to OMF in some cases in order to use the 'classic' debuggers.

Of course, switching to ELF may make gdb easier to compile/work.

Just thinking out load, I was considering introducing -Zelf to tell GCC to invoke as_elf.exe rather than as.exe - that also opens up the use of -Zelf -Zomf to use ELF initially, but convert to OMF at link time (if we have a suitable ELF -> OMF converter that makes the exercise worthwhile).

I'll probably hack something together short term and make sure that 'stuff' works, then think about how best to support a.out, elf & OMF object formats in a more flexible manner...

Thinking some more.... I think I'll fork emxomfld.c and create an emxelfld.c which will do the following:
- use wlink by default
- use noautocnv by default to ensure we don't try and convert elf to OMF

This should make it easy to add -Zelf to gcc which would use emxelfld.exe as the linker

The bit I haven't worked out yet is how to tell gcc to use as_elf.exe not as.exe

It may be that the gcc elf needs to be a completely separate install to gcc aout/omf
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: OS4User on May 13, 2021, 11:21:27 am
@os4user I'm not disagreeing with you, but if we don't convert to omf, how do we debug on OS/2?

I am not familiar with this matter and have no idea why object files are needed for debugging.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 13, 2021, 05:38:32 pm
@os4user I'm not disagreeing with you, but if we don't convert to omf, how do we debug on OS/2?

I am not familiar with this matter and have no idea why object files are needed for debugging.

The real question is the amount of support for our stabs (dwarf?) debugging data in elf object files and whether the linker can put that data in the executable.
Currently, my understanding is that aout supports stabs and emxomf transfers the stabs to the omf intermediate objs and then the linker transfers them to the executable or a dbg file. Not sure if our debuggers support dbg files, exceptq does.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 13, 2021, 05:53:45 pm
Hi Paul, really first it needs to be checked out whether OMF is needed for debugging or whether ELF can handle it along with wl.exe putting it into the executable. Currently debugging works fine with our aout-->omf-->linker-->exe/dll.
I'd suggest testing and possibly it might work or wl.exe can be altered for it to work.
There's also the weak linker to consider. Reading up on it, it is probably seldom used and can be ignored for now. https://en.wikipedia.org/wiki/Weak_symbol (https://en.wikipedia.org/wiki/Weak_symbol)
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: OS4User on May 13, 2021, 06:15:36 pm
The real question is the amount of support for our stabs (dwarf?) debugging data in elf object files and whether the linker can put that data in the executable.
Currently, my understanding is that aout supports stabs and emxomf transfers the stabs to the omf intermediate objs and then the linker transfers them to the executable or a dbg file. Not sure if our debuggers support dbg files, exceptq does.

Good point...

AFAIK wl supports  dwarf.  Is only question if  as_elf   can generate it.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 13, 2021, 11:53:18 pm
Status:

Minor tweaks to GCC made to always invoke emxelfld as the linker.

- emxelfld.c forked from emxomfld.c (copy at https://smedley.id.au/tmp/emxelfld.c)
- emxomfld checkomf function modified to also check for elf header and not attempt to convert it to OMF - we want the autoconvert code as we may be trying to link against an a.out library or object file (eg crt0.o)
- emxomfld - don't call weak_prelink() (as weakld doesn't understand elf)


Currently failing to link an executable with:
gcc -Zexe -Zomf -o hello.exe helloworld.c
Error! E2026: redefinition of reserved symbol _edata
Error! E2026: redefinition of reserved symbol __edata
Error! E2026: redefinition of reserved symbol _end
Error! E2026: redefinition of reserved symbol __end

This is similar to the error Dave was getting earlier with wl.exe - more investigation required....

TODO: (other than fix linked an exe)
- Update as_elf to something more modern than binutils 2.9.1
- work out how to workaround the lack of elf support in emximp - in order to create import libraries for DLL's - worst case is to use OMF libs
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 14, 2021, 12:25:53 am
Removing libend.lib from the linking step allows an executable to be created, but running it gives:
Killed by SIGILL
pid=0xf202 ppid=0x776e tid=0x0001 slot=0x009d pri=0x0200 mc=0x0001 ps=0x0010
U:\DEV\T\HELLOWORLD.EXE
HELLOWOR 0:00000024
cs:eip=005b:00010024      ss:esp=0053:00027f6c      ebp=00027f88
 ds=0053      es=0053      fs=150b      gs=0000     efl=00012206
eax=00000000 ebx=00027f90 ecx=1c8dc060 edx=1c8dc068 edi=00000000 esi=00000000
Process dumping was disabled, use DUMPPROC / PROCDUMP to enable it.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 14, 2021, 03:06:11 am
If I'm understanding https://sourceware.org/gdb/onlinedocs/stabs.html (https://sourceware.org/gdb/onlinedocs/stabs.html) correctly, elf can put the stabs in its own section much like aout does. So it should be possible and with luck might work out of the box. Hopefully it'll fix those warnings about too big stabs that are generated while compiling.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 14, 2021, 03:26:43 am
- work out how to workaround the lack of elf support in emximp - in order to create import libraries for DLL's - worst case is to use OMF libs
There's the IMPLIB option for wlink to create an implib during linking. Hmm, it actually calls wlib to create the import library.
Wlib also has the option ie to create elf import records and an option to list all symbols found in a library file, including static.
There's other tools available, some from binutils some from OW which could be tweaked to output exports from object files.
Wlink has a variety of way to create a DLL, with def files hardly used though they work. With the screensaver's cairo clock, I imported the symbols directly from cairo2.dll and the makefile has,
Code: [Select]
$(dllname).dll: $(object_files)
    @wlink system 386 lx dll INITINSTANCE TERMINSTANCE name $(dllname) &
    f CaClock f MSGX op MAP=$(dllname) op DESC 'Cairo Clock - ScreenSaver module
' &
    op ELIMINATE op MANYAUTODATA op OSNAME='OS/2 and eComStation' op SHOWDEAD &
    LIBPATH %os2tk%\lib LIBRARY libuls.lib LIBRARY libconv.lib @Imports.lnk @des
cription

!ifdef has_resource_file
    rc $(rcname) $(dllname).dll
!endif
!ifdef create_packed_dll
    lxlite $(dllname).dll
!endif
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 14, 2021, 05:37:48 am
Thanks everyone... what works here is:

Code: [Select]
gcc -S helloworld.c
as_elf helloworld.s -o helloworld.o
wl @hello.lnk
** EXPERIMENTAL (HLL) ** Open Watcom Linker Version 2.0beta1 Limited Availability by SHL on Jul  2 2020
Portions Copyright (c) 1985-2002 Sybase, Inc. All Rights Reserved.
Source code is available under the Sybase Open Watcom Public License.
See http://www.openwatcom.org/ for details.
loading object files
searching libraries
creating map file
creating an OS/2 and ArcaOS executable

For interest - https://smedley.id.au/tmp/helloworld.zip

I have a day off on Friday, I'll try and think about how to modify GCC to use this stuff....

Cheers,

Paul

'tis weird, this worked the other day, today I'm getting the
Code: [Select]
Error! E2026: redefinition of reserved symbol _edata
Error! E2026: redefinition of reserved symbol __edata
Error! E2026: redefinition of reserved symbol _end
Error! E2026: redefinition of reserved symbol __end

Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 14, 2021, 06:13:45 am
'tis weird, this worked the other day, today I'm getting the
Code: [Select]
Error! E2026: redefinition of reserved symbol _edata
Error! E2026: redefinition of reserved symbol __edata
Error! E2026: redefinition of reserved symbol _end
Error! E2026: redefinition of reserved symbol __end

Removing libend 'fixes' the linking - but I get a SIGILL trying to run the resulting executable...
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 14, 2021, 06:43:16 am
Perhaps rebuild libend with those symbols commented out?
I tried but I seem to be missing the ml assembler and building all fails due to missing demangle.h
Looking, it seems they're reserved symbols when the option DOSSEG is specified. Where it is getting specified, I'm not sure unless it is automatic when making an executable. It is possible to override the segment ordering, only recommended for embedded applications without an OS. The segments probably should be ordered in a specific order. Search for DOSSEG in the Open Watcom Linker guide.
Looking at the emxomfld source, I can't quite figure out the whole list of options being passed and why linking OMF doesn't cause the same error.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 14, 2021, 07:06:54 am
Hey Dave,

Perhaps rebuild libend with those symbols commented out?
I tried but I seem to be missing the ml assembler and building all fails due to missing demangle.h
Looking, it seems they're reserved symbols when the option DOSSEG is specified. Where it is getting specified, I'm not sure unless it is automatic when making an executable. It is possible to override the segment ordering, only recommended for embedded applications without an OS. The segments probably should be ordered in a specific order. Search for DOSSEG in the Open Watcom Linker guide.
Looking at the emxomfld source, I can't quite figure out the whole list of options being passed and why linking OMF doesn't cause the same error.

Yeah I tried rebuilding libend with those symbols commented out - it now links but I get the same SIGILL as if I just remove libend.

I think I was confused when I thought it was working the other day - my hello.lnk had helloworld.obj in it - which was an OMF version of the object file :(
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 14, 2021, 08:04:37 am
Yeah I tried rebuilding libend with those symbols commented out - it now links but I get the same SIGILL as if I just remove libend.

Might be a problem with the section ordering

Quote
I think I was confused when I thought it was working the other day - my hello.lnk had helloworld.obj in it - which was an OMF version of the object file :(

I must have made the same mistake :) Today I fixed your hello.lnk as I had lost my original test after a reboot (ram disk) and then got the error.
Need a compiler expert who understands all these segments and stuff.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 15, 2021, 02:08:49 am
Hey Dave,

Need a compiler expert who understands all these segments and stuff.

That rules me out then :P

Cheers,

Paul
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Dave Yeo on May 15, 2021, 04:10:31 am
Yet, you probably know more then most of us, if only from hacking on the GCC source. There are people who seem to instinctively understand this stuff though, AZ who did the GCC 2.9x ports was one. IBM chased him down and payed him to do the first kLIBC GCC port, 3.22 or so for Mozilla. There was someone else who ported GCC back in the day for fun, a different port then EMX. So much software lost.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 20, 2021, 04:47:09 am
Hi All,

Yet, you probably know more then most of us, if only from hacking on the GCC source. There are people who seem to instinctively understand this stuff though, AZ who did the GCC 2.9x ports was one. IBM chased him down and payed him to do the first kLIBC GCC port, 3.22 or so for Mozilla. There was someone else who ported GCC back in the day for fun, a different port then EMX. So much software lost.

Just an update on this - the issue with OS/2 executables and elf format object files appears to be a wl (Watcom Linker) bug. I built as.exe with coff object format and a simple hello world executable runs ok.

I'll use this to continue with some tweaks to gcc and emxelfld.exe as time permits, to see what's possible - ie can we still build a DLL with the tools and whatever other changes may be required.

Cheers,

Paul.
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 20, 2021, 12:13:27 pm
FWIW the example helloworld.o in coff format, as as-coff.exe is at https://smedley.id.au/tmp/helloworld-coff.zip
Title: Re: Alternatives to compiling Rust in Firefox port OS/2
Post by: Paul Smedley on May 22, 2021, 08:21:11 am
Quick update, with https://smedley.id.au/tmp/gcc-11.1.0-os2-coff-20210522.zip - I can compile and link *C* executables with assembly produced in the COFF format. (For ELF, we have a bug with wl.exe preventing a valid executable being created).

Note c++ executables can't currently be linked. This is because weakld.c (part of emxomfld) only understand OMF, so I have currently commented out weak_prelink() in emxelfld, so we don't have weak symbols support.

For weakld,  https://github.com/bitwiseworks/libc/blob/master/src/emx/src/emxomf/weakld.c in WLDAddObject() will need to be enhanced so it understands coff (and eventually ELF).