Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - Lars

Pages: [1] 2 3 ... 23
1
Applications / Re: OS/4 (technical details only)
« on: March 11, 2019, 10:15:02 pm »
No. The FIRST variable in a section is aligned. But not necessarily the ones that follow.

2
Applications / Re: OS/4 (technical details only)
« on: March 11, 2019, 03:40:58 pm »
Only the FIRST variable in the section is aligned to the desired alignment.The others need to be aligned manually.

Lars,

ff_cos_32  defined in C,  what is defined in asm  right now is aligned ok

Really ? I could see various variables that were only aligned to 16 bytes (typically it was a sequence of 4 * 4 bytes). If these are only accessed by SSE code then everything is fine but if you pass these to AVX code then you will experience a trap.

3
Applications / Re: OS/4 (technical details only)
« on: March 11, 2019, 07:36:10 am »
Only the FIRST variable in the section is aligned to the desired alignment.The others need to be aligned manually.

4
Applications / Re: OS/4 (technical details only)
« on: March 10, 2019, 10:09:32 am »
You Need to define all segments as public. Only then will they combine.

They are declared as public. The weird thing is the various types of text segments. From the map file,
Code: [Select]
Segment                Class          Group          Address         Size
=======                =====          =====          =======         ====

TEXT32                 CODE           CGROUP         0001:00000000   009fe590
________TEXT           CODE           CGROUP         0001:009fe590   00000000
text                   CODE           AUTO           0002:00000000   000ca2cb
DATA32                 DATA           DGROUP         0003:00000000   0001d990

Aout gives,
Code: [Select]
TEXT32                 CODE           CGROUP         0001:00000000   00bd1560
________TEXT           CODE           CGROUP         0001:00bd1560   00000000
DATA32                 DATA           DGROUP         0002:00000000   00015a20

Note that the DGROUP ends up in segment 0003.
The other question is, does it matter?

It does not seem to matter. At least the AVCODE65 that I use has only 2 segments: a CODE segment and a DATA segment.
You can check with THESEUS: System->Modules->AVCODE65->double click one MTE @
The AVCODE65 that I used was installed via RPM from the Netlabs repo.

But you can also enforce that differents segments go into the same group:
CGROUP GROUP ________TEXT,TEXT32,text
put that in an assembler file that is linked, preferably the one for the first obejct file to be linked.
You might get warnings like the one you posted but if it is the FIRST object file to be linked it will take precedence over everything that follows :-)

By the way: I very much doubt that our OS/2 tools can do anything else but the OMF and the AOUT format. I used nasm without the -f switch and that worked right out of the box. Which means that in this case it produces OMF format.

5
Applications / Re: OS/4 (technical details only)
« on: March 09, 2019, 10:59:46 pm »
Same error.
Anyways there are bigger problems, still using 64kb segments, lots of missing symbols for some reason, perhaps lost when the code is bigger then the segment, inconsistent underline prefix (I just hacked the code to fix for us, changing an ifdef to ifndef). Too many segments.
Here's a map file.
You Need to define all segments as public. Only then will they combine.

6
Applications / Re: OS/4 (technical details only)
« on: March 08, 2019, 08:08:05 am »
1) there is no problem in telling nasm to align a segment/section to a 16-byte or 32-byte boundary using the SECTION / SEGMENT directive. Also if it outputs OMF format, I am using that already. But of course you need to take care that all variables but the first (which is aligned to the requested alignment) in the segment are also aligned to 16-byte / 32-byte. And you have to do that by hand for all subsequent variables, for example by injecting zero bytes ...
nasm also supports the ALIGN directive in the code but then you would precede every variable with that directive ...

2) yes, nm.exe cannot handle OMF format. I just found that out ...

3) yes, you definitely can tell nasm to output 32-bit flat OMF format (FLAT is the keyword to use in the segment declaration). That's what I have been doing. And it linked perfectly well with VAC and WATCOM compilers' generated code. Since GCC also accepts OMF input, it should also link with GCC (or the linkers that it calls).

4) as OS4user suggested, you might also want to move the SECTION_RODATA to the data segment. Then it is not "read only" anymore but who cares ...

5) I guess you meant to say "syscall" instead of "stdcall". GCC for OS/2 also understands the _System keyword (instead of specifying via "attribute") for the syscall calling convention.

7
Applications / Re: OS/4 (technical details only)
« on: March 08, 2019, 06:52:43 am »
Found this GCC switch

-fleading-underscore
-fno-leading-underscore

https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html

and NASM similar:
--prefix
--postfix

NM has switches to control the mangling:
--demangle[=STYLE]
--no-demangle

Maybe taking all this together helps solving the problem.



8
Applications / Re: OS/4 (technical details only)
« on: March 08, 2019, 06:49:38 am »
1) I used
SEGMENT _TEXT CLASS=CODE PUBLIC USE32 ALIGN=% FLAT
for the segment definition (SEGMENT is equivalent to SECTION). When linking with either the VAC or the WATCOM linker it appears that those linkers will only honour (or prefer) the CLASS on combining segments. The actual segment name was irrelevant. In both cases the segment was combined with the default code segments that the VAC and WATCOM compilers generated.

2) what error are you getting on running nm ? If it is about a missing underscore or excessive underscore, maybe GCC has a switch to control generation of the underscore before a symbol or maybe it has a switch to choose the default calling convention (which will impact the generated symbol names). Fortunately, CDECL and SYSCALL seem to be interchangeable enough to use them either way (and only have an impact on the generated symbol name).
I hope there is some switch documentation for the OS/2 port of GCC ...

9
Applications / Re: OS/4 (technical details only)
« on: March 07, 2019, 11:12:21 pm »
OK, we have a workaround, namely building with --disable-AVX.

Now we're just left with the question of why our toolchain is not aligning these AVX instructions. It only seems to be a problem in libavcodec/x86/fft.asm.
One possibility is a weakness in the AOUT format. I vaguely remember problems getting things to work as can be seen by this from libavutil/x86/x86inc.asm
Code: [Select]
; aout does not support align=
; NOTE: This section is out of sync with x264, in order to
; keep supporting OS/2.
%macro SECTION_RODATA 0-1 16
    %ifidn __OUTPUT_FORMAT__,aout
        SECTION .text
    %elifidn __OUTPUT_FORMAT__,coff
        SECTION .text
    %elifidn __OUTPUT_FORMAT__,win32
        SECTION .rdata align=%1
    %elif WIN64
        SECTION .rdata align=%1
    %else
        SECTION .rodata align=%1
    %endif
%endmacro

IIRC, one problem is that aout only supports text segments and is quite primitive compared to newer object formats.
Perhaps I should port the whole build system to using obj (OMF32) format.

Background, aout is an ancient object format that was once common.
Our GCC was forked a long time back and still outputs aout object code. Generally we then use emxomf at link time to convert to native omf and use the system linker to create executables.
Currently in FFmpeg, nasm outputs aout, which is also converted to omf at link time.
We can also convert GCC object files at build time, which would also allow nasm to output omf or obj as nasm calls it.
Obj also needs the FLAT and USE32 parameters to create 32 bit code as it was originally a 16 bit format.

Problem is testing as I don't have AVX hardware.

Well, should that not be:
%macro SECTION_RODATA 0-1 32

(I know little about the nasm syntax).

I think we are ALREADY using OMF and not AOUT format. At least that's what the OS/2 port of nasm should do (and I used nasm myself and linked to code produced by the Watcom and VAC compilers).


10
Applications / Re: OS/4 (technical details only)
« on: March 07, 2019, 01:10:59 pm »
After reading this:

https://software.intel.com/en-us/forums/intel-isa-extensions/topic/279587
(in particular the last comment of the Intel guru)

I can only say that the BEST way to go about all this is to NOT use the VMOVAPS/VMOVAPD/VMOVDQA instructions but instead to use the VMOVUPS/VMOVUPD/VMOVDQU instructions:

1) these will allow to load from unaligned addresses without trapping
2) IF the data is aligned nonetheless, the performance will be as good as with the aligned variants of these instructions

What is still true is that the best memory access performance will be achieved with aligned data (32-bytes for AVX, 64-bytes for AFX-512).


This is all so ridiculous ...

11
Applications / Re: OS/4 (technical details only)
« on: March 06, 2019, 04:39:19 pm »
Hi Dave (Yeo),

after looking at your LIBAV repo and at the trap info, it appears to me that the IMDCT ("Inverse Modified Discrete Cosine Transform") implementation only contains SSE support but not AVX support. Unfortunately, the SSE implementation of the IMDCT calls the AVX implementation of the FFT ("Fast Fourier Transfomration").
I guess this is where some constant data (say, the FFT coefficient vectors) becomes unaligned.


Lars

12
Applications / Re: OS/4 (technical details only)
« on: March 05, 2019, 08:09:47 am »
Hi David, that crash looks like one I get now and again, I believe font related.
Have you tested with the Firefox build that OS4USER mentioned? Perhaps with the AVX enabled kernel? You can get a list of CPU capabilities with this program, http://hobbes.nmsu.edu/download/pub/os2/util/misc/cpuinfo.zip
If needed, I could build FFmpeg with AVX disabled. Be nice to know which function it is crashing in too.

If you sent me the MAP file for AVCODE65.DLL it would be a piece of cake to find the failing routine. We have the trap location ...

As I said, the "something" not aligned correctly seems to be a 32-byte (AVX) constant (read only) directly encoded in the code segment.
Where do I find the code for AVCODE65.DLL ?
I looked at:
https://github.com/FFmpeg/FFmpeg/tree/master/libavcodec
but there is no assembler code there, only plain "C" code. Which makes me wonder why we do not use plain "C" code and let GCC do all the gory stuff like data alignment (once GCC aligns SSE and AVX data the way it should ...). I would expect a good "C" compiler to automatically parallelize data accesses (and therefore making good use of the SIMD capabilities of the CPU). At least the Intel compiler does that ...


By the way: since the OS/2 kernel does not support AVX, wouldn't it be better to NOT build with AVX capability in general ?
I don't have AVX (only SSE) and therefore I am not affected but we can already see that with more modern systems that do support AVX or even AVX2 (with 512-bit registers), we now run into a whole lot of problems with AVX (data not properly aligned, no OS support for saving away AVX registers on a thread switch, with exception of OS/4 kernel).
Note: OS/2 does have support to save and restore SSE registers on a thread swith.

For me, OS/2 is falling more and more apart.


Lars

13
Applications / Re: OS/4 (technical details only)
« on: March 04, 2019, 05:00:46 pm »

Thanks for all the responses.

I just installed the noAVX kernel and my traps have disappeared.  This is just preliminary but I have no problem going to that web site and others so far.  So far, so good.

My avcode56.dll is the same size and date on both my ArcaOS 5.02 and ArcaOS 5.03 partitions.

Attached a pic of my type of CPU info.  While modifying this message, I had another trap which I have attached.  I just remembered that this is another instance when a trap occurred. However, so far, this one trap is all I have encountered so far with the noAVX.

Yes, the "new" trap is not AVX related at all. It is something completely different. Looks like EBX is supposed to contain a full address but it seems to contain only an address offset (or an offset summed up on top of a NULL pointer which is more likely ...)

Too bad, your screen shot does not show the feature flags. That would have been more interesting ...

Lars

14
Applications / Re: OS/4 (technical details only)
« on: March 04, 2019, 09:18:01 am »
Looking at it again:

If it is true that all SSE/AVX code in AVCODE56.DLL is handwritten in assembler then I suspect either one of 2 possibilities:

1) some dynamic memory is passed to AVCODE56.DLL which is not aligned: but from the used registers, EAX and EDI point to memory allocated by LIBCN0 which aligns to 32-bytes. That would be enough for both, SSE and also AVX.

2) the data used in AVCODE56.DLL itself is not properly aligned (even if coded by hand). Maybe someone forgot that AVX code needs data to be 32-byte aligned when some code was upgraded from SSE to AVX. The incorrectly decoded code seems to imply that some constant seems to be taken directly from the code segment (address: 0x0cd22e50). That is, this constant is directly embedded in the code. And this constant is only 16-byte aligned where AVX code requires 32-byte alignment. If somebody had the AVCODE56 code, it would be easy to check.

If I had to guess I would think that the incorrectly decoded instruction is a VMOVDQA or VMOVAPD, VMOVAPS instruction:
VMOVAPD ymm0,[0x0cd22e50). And according to Intel manual, this instruction requires 32-byte data alignment.

Lars

15
Applications / Re: OS/4 (technical details only)
« on: March 04, 2019, 07:23:12 am »
1) looks like you have an out of date EXCEPTQ package. It does not properly decode the real instruction.

2) the failing instruction starts with a VEX prefix (and is incorrectly decoded in the following). It is likely an SSE or AVX instruction that fails due to unaligned memory (memory accesses need to be 16-byte aligned for SSE (128 bit) registers and 32-byte aligned for AVX (256 bit) registers).
That said I think there is a problem wtih libcn0.dll that needs a rebuild with a new compiler so that data used with SSE/AVX instructions is properly aligned (for all stack based references and also all static data in the data segment). You might want to try a newer test build of the new "libc" package (which contains libcn0).

Question is why does it only appear with the OS/4 kernel ? I can only guess: the code takes a different code path with the OS/2 kernel and the OS/4 kernel. Maybe because OS/4 kernel signals AVX support (if of course the CPU supports it) and the OS/2 kernel does not.

Lars

Pages: [1] 2 3 ... 23