Author Topic: OS/4 (technical details only)  (Read 156595 times)

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: OS/4 (technical details only)
« Reply #75 on: March 06, 2019, 01:47:39 am »
Hi David.
Ok, it wasn't the previous browser. Go to https://bitbucket.org/dryeo/mozilla-os2/downloads/ and download the firefox-sandybridge-xqs package and install it in your sandybridge firefox directory. Seems I forgot to package the xqs files :)
Then download the ffmpeg-3.2.13-libs.zip package and put the DLLs in @unixroot\usr\local\lib, which should be ahead of @unixroot\usr\lib in your LIBPATH. Restart the browser and test playing a video. If it traps, look in the trp file to make sure it was using and trapped in the new avcode57 DLL and post the trp file (I have the source for this one). If it still used avcode56, you might need to uninstall the ffmpeg libs package.
We'll see if using a different compiler helps. If not, I'll build an AVX disabled version for you.
Thanks

David Graser

  • Hero Member
  • *****
  • Posts: 870
  • Karma: +84/-0
    • View Profile
Re: OS/4 (technical details only)
« Reply #76 on: March 06, 2019, 02:38:23 am »
Still traps at the youtube page and still traps at avcode56, Attached trap file.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: OS/4 (technical details only)
« Reply #77 on: March 06, 2019, 02:48:17 am »
Try moving avcode56.dll and avutil55.dll out of the way and putting the new DLLs in their place

David Graser

  • Hero Member
  • *****
  • Posts: 870
  • Karma: +84/-0
    • View Profile
Re: OS/4 (technical details only)
« Reply #78 on: March 06, 2019, 01:44:10 pm »
Try moving avcode56.dll and avutil55.dll out of the way and putting the new DLLs in their place

OK Dave, I moved the 2 dlls  out of the libpath, installed the sandybride  FF and the AVX OS/4 kernel and the traps have disaapeared.  I can access youtube without it trapping.  Will test more today. 

Note, I had gone back to the noAVX OS/4 kernel  with the FF perntium version and this has been the most stable so far.  I could not get it to crash. 

I will test he sandybridge version with the AVX OS/4 kernel with the two dlls removed more.  So far, it has not crashed at the the two web pages I have had problems with.  It looks it may be just as stable  as the noAVX version..  Time will tell.

The Sandybridge version is now just as stable as the pentium version with those two dlls removed.  I did find one site that it is unable to play the video whereas the noAVX version was able to.

https://www.trunews.com/stream/diplomacy-sabotage-unpatriotic-democrats-torpedoed-us-north-korea-peace-summit
« Last Edit: March 06, 2019, 03:57:00 pm by David Graser »

Lars

  • Hero Member
  • *****
  • Posts: 1271
  • Karma: +65/-0
    • View Profile
Re: OS/4 (technical details only)
« Reply #79 on: March 06, 2019, 04:39:19 pm »
Hi Dave (Yeo),

after looking at your LIBAV repo and at the trap info, it appears to me that the IMDCT ("Inverse Modified Discrete Cosine Transform") implementation only contains SSE support but not AVX support. Unfortunately, the SSE implementation of the IMDCT calls the AVX implementation of the FFT ("Fast Fourier Transfomration").
I guess this is where some constant data (say, the FFT coefficient vectors) becomes unaligned.


Lars

OS4User

  • Sr. Member
  • ****
  • Posts: 406
  • Karma: +10/-0
    • View Profile
Re: OS/4 (technical details only)
« Reply #80 on: March 06, 2019, 06:33:23 pm »
Hi Dave

It seams I am missing your point with replacement of ffmpeg libs.
I understand you offer to replace avcode56.dll/avutil54.dll with avcode57.dll/avutil55.dll.
But how FF will come to know to use this new pair ?
For a sake of testing I did such replacement on my comp and youtube disappeared.
I am not sure but looks like FF has to be rebuilt against this new pair of libs.

(I do not have an AVX equipment to do a proper test.)

At the same time thanks a lot for building AVX  binaries - it's obvious that this technology has to be available for our community too.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: OS/4 (technical details only)
« Reply #81 on: March 06, 2019, 07:52:33 pm »
Hi Dave (Yeo),

after looking at your LIBAV repo and at the trap info, it appears to me that the IMDCT ("Inverse Modified Discrete Cosine Transform") implementation only contains SSE support but not AVX support. Unfortunately, the SSE implementation of the IMDCT calls the AVX implementation of the FFT ("Fast Fourier Transfomration").
I guess this is where some constant data (say, the FFT coefficient vectors) becomes unaligned.

Hi Lars, that was the impression I had. Note that the libav code is fairly old and a fork of FFmpeg so this may have changed. Unluckily this old browser doesn't want to load the right branch at FFmpeg's git repository. Current libs from the 3.2 branch. I just did a minor patch to find a header.
Have to continue testing if the GCC 5.5.0 build does better. It helps with Firefox.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: OS/4 (technical details only)
« Reply #82 on: March 06, 2019, 08:01:46 pm »
Hi Dave

It seams I am missing your point with replacement of ffmpeg libs.
I understand you offer to replace avcode56.dll/avutil54.dll with avcode57.dll/avutil55.dll.
But how FF will come to know to use this new pair ?
For a sake of testing I did such replacement on my comp and youtube disappeared.
I am not sure but looks like FF has to be rebuilt against this new pair of libs.

(I do not have an AVX equipment to do a proper test.)

At the same time thanks a lot for building AVX  binaries - it's obvious that this technology has to be available for our community too.

Yes, it seems I missed a dependency. Should have checked the map file.
Code: [Select]
K:\work\ffmpeg\src\ffmpeg.obj\libswresample\libswresample_dll.lib
K:\work\ffmpeg\src\ffmpeg.obj\libavutil\libavutil_dll.lib
H:\tmp\ldconv_libcx_a_99775c7f48361effd8.lib
H:\tmp\ldconv_libvpx_dll_a_99775c7f48361effd8.lib
H:\tmp\ldconv_lzma_dll_a_99775c7f48371eb5a0.lib
H:\tmp\ldconv_libz_dll_a_99775c7f48371eb5a0.lib
K:\usr\local550\lib\gcc\i686-pc-os2-emx\5.5.0\libgcc_so_d.lib
K:\usr\lib\libc_alias.lib
K:\usr\lib\libc_dll.lib
K:\usr\lib\libos2.lib
K:\usr\lib\libend.lib

So need swresa2.dll as well as lzma, z and libvpx installed.

Firefox tries to dynamically load the DLLs in this order, from FFmpegRuntimeLinker.cpp
Code: [Select]
#if defined(XP_OS2)
// On OS/2, we must load avutil directly to access its symbols
static const char* sLibs[][2] = {
  { "avcode57.dll", "avutil55.dll" },
  { "avcode56.dll", "avutil54.dll" },
  { "avcode55.dll", "avutil54.dll" },
  { "avcode53.dll", "avutil51.dll" },
};

So anyways need to redownload the libs package.
Can test whether its working at https://www.youtube.com/html5. Should show H.264 and VP8+VP9 working.
David will also need to redo the libraries.

OS4User

  • Sr. Member
  • ****
  • Posts: 406
  • Karma: +10/-0
    • View Profile
Re: OS/4 (technical details only)
« Reply #83 on: March 06, 2019, 08:56:26 pm »
Dave:

So anyways need to redownload the libs package.

After redownload https://bitbucket.org/dryeo/mozilla-os2/downloads/ffmpeg-3.2.13-libs.zip and installation of  ALL the libs from it  youtube came back.

Theseus also confirms that  avcode57 is in use and avcode56 is no longer in use.

So we need to test new ffmpeg on AVX equipment.


David:

Looking forward to your test of updated  ffmpeg.


David Graser

  • Hero Member
  • *****
  • Posts: 870
  • Karma: +84/-0
    • View Profile
Re: OS/4 (technical details only)
« Reply #84 on: March 06, 2019, 10:08:38 pm »
Using
ArcaOS 5.02
Firefox Sandbridge 45.9.0
Latest OS/4 test kernel SVN5449
Latest libs downloaded.

It crashed periodically starting with the following web page.  Then the OS/2world site starting crashing.

https://www.trunews.com/stream/diplomacy-sabotage-unpatriotic-democrats-torpedoed-us-north-korea-peace-summit

FF traps with the following trap messages.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: OS/4 (technical details only)
« Reply #85 on: March 07, 2019, 12:00:46 am »
OK, I uploaded a version with AVX disabled if you'd like to test.

David Graser

  • Hero Member
  • *****
  • Posts: 870
  • Karma: +84/-0
    • View Profile
Re: OS/4 (technical details only)
« Reply #86 on: March 07, 2019, 12:46:48 am »
OK, I uploaded a version with AVX disabled if you'd like to test.

I downloaded it, but may not get to test it to totomorrow. I am at work right now.

Do I use these new libs with the sandybridge or the pentium FF version or does it matter?  If I am right, I need to use the AVX version of OS/4 which is the latest test kernel. 

Dave, the noAVX OS/4 kernel along with the FF pentium 45.9.0 version is stable and opens and plays everything I throw at it.  It is the most stable FF I have run while using ArcaOS.
« Last Edit: March 07, 2019, 12:50:31 am by David Graser »

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: OS/4 (technical details only)
« Reply #87 on: March 07, 2019, 01:08:05 am »
I downloaded it, but may not get to test it to totomorrow. I am at work right now.

That's fine.
Quote
Do I use these new libs with the sandybridge or the pentium FF version or does it matter?  If I am right, I need to use the AVX version of OS/4 which is the latest test kernel.

They'll work with any CPU newer then a Pentium Pro. The code detects the CPU and OS support and enables various code paths depending on what it is running on. So should be fine on any build of FF 45.9 and the corresponding SeaMonkey. Won't work with FF38 though as it'll only use avcode56.dll or older.

Quote
Dave, the noAVX OS/4 kernel along with the FF pentium 45.9.0 version is stable and opens and plays everything I throw at it.  It is the most stable FF I have run while using ArcaOS.

That's good to hear though Bitwise did the work, I just changed the build environment a bit along with some very minor patches to make the newer compiler happy. Not sure if it is the newer compiler or enabling SSE uses more tested code paths. SeaMonkey has been very stable here as well with the very occasional trap like the other one you posted, perhaps once a month.
The AVX kernel should be fine as well with the Pentium M build, at least I haven't had any reports of problems besides yours.

OS4User

  • Sr. Member
  • ****
  • Posts: 406
  • Karma: +10/-0
    • View Profile
Re: OS/4 (technical details only)
« Reply #88 on: March 07, 2019, 11:15:25 am »
Hi Dave,

The fallen instruction is (function fft8_avx):
Code: [Select]
mova       m4,  [ps_cos16_1] ; vmovaps

static variable ps_cos16_1 is 16 bit aligned only (0001:009AE730,  yes it is in code seg)

ref.: https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/HEAD:/libavcodec/x86/fft.asm line 346

AFAIK  the following variables have to be 32 bit aligned too:
Code: [Select]
ps_cos16_2
ps_root2
ps_root2mppm
ps_p1p1m1p1
perm1
perm2
ps_p1p1m1p1root2
ps_m1m1p1m1p1m1m1m1
ps_m1p1
« Last Edit: March 07, 2019, 01:34:56 pm by OS4User »

Lars

  • Hero Member
  • *****
  • Posts: 1271
  • Karma: +65/-0
    • View Profile
Re: OS/4 (technical details only)
« Reply #89 on: March 07, 2019, 01:10:59 pm »
After reading this:

https://software.intel.com/en-us/forums/intel-isa-extensions/topic/279587
(in particular the last comment of the Intel guru)

I can only say that the BEST way to go about all this is to NOT use the VMOVAPS/VMOVAPD/VMOVDQA instructions but instead to use the VMOVUPS/VMOVUPD/VMOVDQU instructions:

1) these will allow to load from unaligned addresses without trapping
2) IF the data is aligned nonetheless, the performance will be as good as with the aligned variants of these instructions

What is still true is that the best memory access performance will be achieved with aligned data (32-bytes for AVX, 64-bytes for AFX-512).


This is all so ridiculous ...
« Last Edit: March 07, 2019, 01:16:24 pm by Lars »