OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Applications => Topic started by: OS4User on April 16, 2018, 08:09:59 pm

Title: OS/4 (technical details only)
Post by: OS4User on April 16, 2018, 08:09:59 pm
Patched DIVE.DLL can increase video speed on OS/4 system.

At the same time the patched DIVE.DLL is not compatible with original OS/2 -  to avoid missboot do the following:

1) Create C:\OS4 dir
2) Put the patched DIVE.DLL into C:\OS4
3) Insert C:\OS4; exactly after LIBPATH=  into your CONFIG.OS4 so you will have LIBPATH=C:\OS4;.;.......

This way the patched DIVE.DLL will be loaded instead of original DIVE.DLL and  original DIVE.DLL will remain for booting with CONFIG.SYS (for ex. original kernel).

http://gus.biysk.ru/os4/test/patched_dll/patched_dive.zip
Title: Re: OS/4 (technical details only)
Post by: Martin Iturbide on April 17, 2018, 01:12:02 am
Hi OS4User

Can it be possible for the author to explain a little bit what was patched on this DIVE.DLL ? like which functions of the OS4 kernel is now using to increase the video speed?

Regards
Title: Re: OS/4 (technical details only)
Post by: OS4User on April 17, 2018, 11:20:08 am
Can it be possible for the author to explain a little bit what was patched on this DIVE.DLL ?

Hi Martin

some details is here https://www.os2world.com/wiki/index.php/Phoenix_OS/4#Video_bus_bandwidth_.28VBB.29 (https://www.os2world.com/wiki/index.php/Phoenix_OS/4#Video_bus_bandwidth_.28VBB.29)
Title: Re: OS/4 (technical details only)
Post by: Dariusz Piatkowski on April 17, 2018, 05:10:44 pm
OK, OS4User, time for me to get off of my butt and get the debug info to troubleshoot the hard trap right after PMSHELL comes up. With FreeRDP making progress and VBox supporting extra features of the OS4KRNL I have no reason to delay further.

I re-read the Wiki you posted above and will get the debug kernel setup and ready to go. Beyond this though, is there anything specific I need to do in order to capture the "moment" the system traps?

My assumption is that maybe through the debug kernel I have some way to communicate with the OS/2 box, but even if that is the case, once the trap occurs, how do I know what additional information to log? Can you provide any 'working instructions' for this? Even high-level stuff might help...

Thanks!
Title: Re: OS/4 (technical details only)
Post by: OS4User on April 17, 2018, 06:12:54 pm
I re-read the Wiki you posted above and will get the debug kernel setup and ready to go. Beyond this though, is there anything specific I need to do in order to capture the "moment" the system traps?

My assumption is that maybe through the debug kernel I have some way to communicate with the OS/2 box, but even if that is the case, once the trap occurs, how do I know what additional information to log? Can you provide any 'working instructions' for this? Even high-level stuff might help...

Welcome on board :)

You do not need anything specific, you will see trap info in your comport log.
Additional info always depends on trap info, working instruction is quite big,  it is called
"The OS/2 Debugging Handbook"  for ex. https://openlibrary.org/works/OL8414318W/The_OS_2_Debugging_Handbook

Title: Re: OS/4 (technical details only)
Post by: David Graser on April 18, 2018, 12:05:29 am
Lenovo Thinkpad x220

Dual boot to either Windows 7 or ArcaOS 5.02

lSVN5094_unoff.zip

I really like this kernel.  The system is more responsive, less CPU trashing, and my CPU operates cooler.  My system even appears to operate in a more stable condition.  It even shows one how bad  Firefox is with all its jumbled code.  The CPU maxes out, but Firefox still remains responsive and the CPU continues to remain cool.

I have encountered a problem and it deals with RAM drives.  I use the ArcaOS ram drive in HPFS.  If I reboot, everything is fine.  However, when I power down and then power up to boot to ArcaOS, the RAM drive disappears and I cannot get it back.  Playing with the RAM drive settings does no good.  The only way to get it back is to install an archive.  I can reboot nothing changes.  Only when my computer is powered down and then turned back on that the RAM drive problem surfaces.  I like my RAM drive, however, if I have to, I can do without it to continue to use this kernel.
Title: Re: OS/4 (technical details only)
Post by: ivan on April 18, 2018, 01:05:04 am
Hi David,

What you are seeing with power down and the ramdisk is what you should expect.  What is on a ram drive is lost when the system is powered down - Ramdisks are only there as long as there is power to the ram and the refresh mechanism is working - no power, even for a very short time, results in no ramdrive.
Title: Re: OS/4 (technical details only)
Post by: David Graser on April 18, 2018, 02:10:09 am
Hi David,

What you are seeing with power down and the ramdisk is what you should expect.  What is on a ram drive is lost when the system is powered down - Ramdisks are only there as long as there is power to the ram and the refresh mechanism is working - no power, even for a very short time, results in no ramdrive.

Hi Ivan

Yes, but should not the RAM disk be rebuilt when the system is booted back up.  The RAM drive is no longer there.  The driver is listed in the config.os4 file.  Now I can boot using the IBM loader and kernel and the RAM disk is there after power up and boot up. The IBM kernel had no problem with the disappearing RAM drive.  Once the RAM disk was selected in the RAM disk setting, it is always present whether one rebooted or powered down and powered  back up.  Yes, the information should be lost when powering down, but the whole RAM should not disappear when booted back up.
Title: Re: OS/4 (technical details only)
Post by: Valery Sedletski on April 18, 2018, 03:27:39 am
2David Graser: I don't understand you. The ram disk disappears when the system is powered down, because power is switched off, and RAM is not refreshed, as Ivan said. This is what happens. Or, do you mean that the .ZIP file does not unzip to ramdisk? BTW, do you use QSINIT, or OS/4 os2ldr? The ram disk works only with QSINIT (or Arca Noae os2ldr, which is a crippled version of QSINIT). OS/4 os2ldr does not create the ram disk, possibly, that's because you lose your ramdisk. So, did you tried QSINIT, and does it change something?

PS: Note that you should disable the OS/4 logo, by adding "NOLOGO" to kernel parameters, or QSINIT will cause system to hang on VESA mode detect. This is unsolved problem, yet.

PPS: The ram disk is in fact created by QSINIT. hd4disk.add only allows this disk to be mounted under OS/2. So, hd4disk.add without QSINIT will not work. Arca Noae loader should work too, instead of QSINIT, theoretically, if you'll boot OS/4 kernel with it. But AFAIK, it does not support choosing multiple kernels from the menu. So, you'd better install QSINIT, instead.
Title: Re: OS/4 (technical details only)
Post by: Valery Sedletski on April 18, 2018, 04:02:02 am
Also, I should note that I don't know, how does Arca Noae loader restores the ram disk. For QSINIT, it requires creating the qssetup.cmd file in the root directory, with the following lines:

Quote
ramdisk z:
unzip /boot /q qsinit.ldi c:\

-- This will create the single FAT/FAT32 z: disk under OS/2 (drive letter c: under QSINIT) and unzip the qsinit.ldi file to the ramdisk.

To get it formatted with HPFS, you should use qssetup.cmd with the following contents:

Quote
ramdisk z:
format c: /fs:hpfs
unzip /boot /q qsinit.ldi c:\

QSINIT supports multiple ramdisks with different FS-es, like this:

Quote
ramdisk z: q: div=565
format c: /fs:exfat /a:1k /q /wipe
mount d: hd3 1
unzip /boot /q qsinit.ldi c:\

-- This will create disk z: (c: or 0: under QSINIT) formatted as exFAT and extract qsinit.ldi to it, and disk q: (d: or 1: under QSINIT), 565 MB in size, formatted with FAT16 (empty). For more details, refer to QSINIT documentation. exFAT ramdisk can be mounted via fat32.ifs.
You can use exFAT or FAT32, or FAT16, or HPFS for your ram disks the same way.
Title: Re: OS/4 (technical details only)
Post by: David Graser on April 18, 2018, 04:09:43 am
2David Graser: I don't understand you. The ram disk disappears when the system is powered down, because power is switched off, and RAM is not refreshed, as Ivan said. This is what happens. Or, do you mean that the .ZIP file does not unzip to ramdisk? BTW, do you use QSINIT, or OS/4 os2ldr? The ram disk works only with QSINIT (or Arca Noae os2ldr, which is a crippled version of QSINIT). OS/4 os2ldr does not create the ram disk, possibly, that's because you lose your ramdisk. So, did you tried QSINIT, and does it change something?

PS: Note that you should disable the OS/4 logo, by adding "NOLOGO" to kernel parameters, or QSINIT will cause system to hang on VESA mode detect. This is unsolved problem, yet.

PPS: The ram disk is in fact created by QSINIT. hd4disk.add only allows this disk to be mounted under OS/2. So, hd4disk.add without QSINIT will not work. Arca Noae loader should work too, instead of QSINIT, theoretically, if you'll boot OS/4 kernel with it. But AFAIK, it does not support choosing multiple kernels from the menu. So, you'd better install QSINIT, instead.

Hi Valery

I believe I understand now. 
Although I have Ramdisk enabled in ArcaOS, it will not work with OS4 kernel unless QSINIT is installed. 
Whereas, with the IBM kernel, with RAMDISK enabled in ArcaOS, QSINIT is not needed for the RAMDISK to mount.
I am presently not using QSINIT.  I am using the kernel loader that comes with OS4.

I guess what is confusing to me is that the both use hd4disk.add to mount the RAM disk.  ArcaOS with the IBM kernel and loader loads the RAM disk just fine.  However, if I understand what has been said, OS4 kernel and loader will fail to mount the RAMDISK if the computer has been turned off.  The OS4 loader must be replaced with QSINIT.

Since I am not using QSINIT at the moment, I can use the OS4 loader to select option 2 which switches to the OS2 loader and kernel to boot to ArcaOS.  The RAM drive is mounted.  So long as I do not turn my computer off and I reboot and select option 3 to boot using OS4, the RAMDISK is mounted and shown in ArcaOS.
Title: Re: OS/4 (technical details only)
Post by: Valery Sedletski on April 18, 2018, 04:21:33 am
> ArcaOS with the IBM kernel and loader loads the RAM disk just fine.

ArcaOS does not use IBM os2ldr. It uses a "stripped-down" variant of QSINIT, with some features ripped-off (like the boot menu and some QSINIT apps which usually present on the ram disk). You see the ramdisk with OS/4 kernel and loader after warm reboot because the ramdisk remains in memory as RAM is refreshed. If you turn it off, and back on, the ram disk gets lost. So, it requires QSINIT, or Arca Noae loader for the ram disk to be created again. OS/4 os2ldr knows nothing about ram disks, the same way original IBM os2ldr does.

So, you have two oprions:

1) use QSINIT, which is more flexible and allows to boot multiple kernels from the menu, and which is compatible with OS/4 loader "os2ldr.ini" config file.
2) boot Arca Noae loader from OS/4 os2ldr with "restart" option, like this:

os2ldr.arc = Arca Noae loader, restart

-- where os2ldr.arc is your renamed version of Arca Noae loader. But it will boot kernel with fixed "os2krnl" name, only.
Title: Re: OS/4 (technical details only)
Post by: David Graser on April 18, 2018, 05:17:22 am
Valery

A big thank you!  I am using QSINI now.

I made my QSSETUP.CMD

ramdisk x: hpfs
format x: /fs/hpfs
unzip /boot /q qsinit.ldi d:\

I left my original OS2LDR as OS2LDR.ORY

I am using the OS2LDR.INI I created for OS4.

Copied the QSINIT.LDI file to the root of my D drive.

I also disabled the RAMDISK option found in the COMPUTER\SYSTEM SETUP\RAM DISK

Powered down and started back up an everything works great.  With your help and the docs supplied it so not hard setting up.  Thanks again.

David
Title: Re: OS/4 (technical details only)
Post by: David Graser on April 18, 2018, 06:15:14 am
One last comment.  I am a user of eCo essentials.  I like the eComstation Device Manager, the PCI Dock, and the Removable Drive Format Tool.  When using the OS4 kernel, the Removable Drive Format Tool won't come up.  It gives the following error message:

Cannot start D:\sys\system\refomat.exe

Make sure the path for the program is correct and the file is an os/2 for Dos program.  There might not be enough memory to run the program etc....

Now I can boot using the ArcaOS kernel and the program work great.  This is the only program so far that does not work with the OS4 kernel that I have come across.
Title: Re: OS/4 (technical details only)
Post by: Doug Clark on April 18, 2018, 11:09:58 pm
I have downloaded os2krnlscn5096_unoff.zip

The HOWTO file says this only works with acpi-3.23.0-1

I have acpi-3.32.02.zip, and later versions. I also have an eCS and ArcaOS subscription. Where can I get the version of acpi-3.23.01 that works with OS4?
Title: Re: OS/4 (technical details only)
Post by: David Graser on April 19, 2018, 12:22:27 am
I have downloaded os2krnlscn5096_unoff.zip

The HOWTO file says this only works with acpi-3.23.0-1

I have acpi-3.32.02.zip, and later versions. I also have an eCS and ArcaOS subscription. Where can I get the version of acpi-3.23.01 that works with OS4?

I think this is what you may be looking for. 

http://www.gus.biysk.ru/os4/Patches/AcpiPsdPatches.rar
Title: Re: OS/4 (technical details only)
Post by: Valery Sedletski on April 19, 2018, 02:46:47 am
Just tried to run reformat.exe on my Asus laptop with eCS 2.2 and OS/4 kernel -- it starts. But on this machine (MCP2 with OS/4 kernel) it does not start, though I put ecolange.dll and ecomedia.dll in program dir. So, something is indeed missing in this system. But it works fine on the previous system.

PS: Now tried to run reformat.exe on this machine, and it starts here too. Weird, as previously, it did not started. (I put the required DLL's: ecolange.dll, ecomedia.dll and progress.dll in the same dir as the program)

PPS: BTW, this program is not really needed, because there is native FORMAT from fat32.ifs which allows to format to FAT12/FAT16/FAT32/exFAT file systems. Required flash media preparations can be done with DFSee, without the need to reformat media. You usually only need to add the LVM info, which can be done non-destructive, without the need to repartition and reformat.

PPPS: Tried to press the "Format" button, it says "Your eCo software subscription has expired" :)))) Useless, indeed.
Title: Re: OS/4 (technical details only)
Post by: Valery Sedletski on April 19, 2018, 02:49:23 am
2Doug Clark: Does not Arca Noae supply the  build archive? I still have 3.23.01 from my previous eCS installation. These: http://www.gus.biysk.ru/os4/Patches/AcpiPsdPatches.rar are patches, but not the actual PSD.
Title: Re: OS/4 (technical details only)
Post by: Doug Clark on April 19, 2018, 08:45:32 pm
My subscription for ArcaOS and my subscription for ECS only allows access to the latest version. I will send a note to Lewis and ask for 3.23.01 - or better yet, access to all previous versions. I have occasionally downloaded whatever was the latest on the subscription site and saved those copies, but since I didn't do that all that often I am missing some versions of all the packages.

I just checked my eCS 2.2 beta CD and it has ACPI_3_21. I noticed that the RAR has patches for a number of previous versions, but I am reluctant go back too many versions on ACPI since it seems there are some inter-dependencies between ACPI and other stuff.
Title: Re: OS/4 (technical details only)
Post by: OS4User on April 20, 2018, 11:43:50 am
but I am reluctant go back too many versions on ACPI since it seems there are some inter-dependencies between ACPI and other stuff.

AFAIK   ACPI32.DLL,  ACPI.PSD, APM.ADD and ACPIDAEMON.EXE have to be from the same version.  You can rename ACPI.PSD, APM.ADD and ACPIDAEMON.EXE  to ACPI_1.PSD, APM_1.ADD and ACPIDAEMON_1.EXE  accordingly and make appropriate changes in you config.os4

For ACPI32.DLL   it is a bit more complicated   -  but you can do the same as with patched DIVE.DLL
Title: Re: OS/4 (technical details only)
Post by: Doug Clark on April 20, 2018, 03:23:56 pm
certainly DIVE, but I was thinking more along the lines of uniaud32 and any of the other stuff that depends on some level ACPI.  Just finding out what has dependencies let alone what versions are dependent on other versions is painful.
Title: Re: OS/4 (technical details only)
Post by: OS4User on April 20, 2018, 03:32:46 pm
certainly DIVE, but I was thinking more along the lines of uniaud32 and any of the other stuff that depends on some level ACPI.  Just finding out what has dependencies let alone what versions are dependent on other versions is painful.

so far no other dependencies are discovered, except, ACPI32.DLL,  ACPI.PSD, APM.ADD and ACPIDAEMON.EXE have to be from the same version
Title: Re: OS/4 (technical details only)
Post by: David Graser on April 23, 2018, 02:54:03 am
One last comment.  I am a user of eCo essentials.  I like the eComstation Device Manager, the PCI Dock, and the Removable Drive Format Tool.  When using the OS4 kernel, the Removable Drive Format Tool won't come up.  It gives the following error message:

Cannot start D:\sys\system\refomat.exe

Make sure the path for the program is correct and the file is an os/2 for Dos program.  There might not be enough memory to run the program etc....

Now I can boot using the ArcaOS kernel and the program work great.  This is the only program so far that does not work with the OS4 kernel that I have come across.

I found another program that does not work with the OS4 kernel.  This one is "styler.exe" and it has been around for a long time and has been in ecomstation for years.

Using PMDLL, it states that neither Reformat.exe or Styler.exe can not be started using the OS4 kernel.  When one scrolls down from the exe for either of them, LVM is highlighted red saying it has a problem.  Just starting the program object indicates that the exes are not OS/2 programs.  When I reboot under the Arca 5.02 kernel using PMDLL, both exes and LVM.dll show no problems and both programs work as they should.

Using FM/2 to start the programs, it gives the following attached error message.  Other than these two programs not starting and no acpi, the OS4 kernel has worked great for me.  I believe ArcaOS 5.02 comes with the .8 version of the ACPI and there are presently no patches for it.

Title: Re: OS/4 (technical details only)
Post by: OS4User on April 23, 2018, 03:14:56 pm

Cannot start D:\sys\system\refomat.exe


I found another program that does not work with the OS4 kernel.  This one is "styler.exe" and it has been around for a long time and has been in ecomstation for years.


I have just check on my system  - reformat.exe starts ok. I do not have styler.exe to check.
Title: Re: OS/4 (technical details only)
Post by: David Graser on April 23, 2018, 08:52:58 pm


I have just check on my system  - reformat.exe starts ok. I do not have styler.exe to check.
[/quote]

It was called eStyler in eComstation.  In the ArcaOS releases, it was renamed to Styler.  I am using ArcaOS 5.02.  I can boot to the ArcaOS kernel and everything works fine.  When I reboot using OS4 kernel, the two programs do not work.

Other than these two programs not working, I have found the OS4 kernel to be more stable and its performance better.  Your modified dive.dll speed up my graphics.  I don't plan on going back to the IBM kernel if I can help it unless I need to boot to it to use one of the two programs.  That is what is good about the kernel loader.  I can boot to either kernel.
Title: Re: OS/4 (technical details only)
Post by: Valery Sedletski on April 24, 2018, 02:28:10 am
Styler/2 worked fine on my machine with OS/4 kernel, as well as REFORMAT. So, probably, you have problems with paths. Probably, LVM.DLL is not on LIBPATH.

PS: Do you have different CONFIG.SYS for OS/4 and IBM kernel? If so, you should compare them for any differences.
Title: Re: OS/4 (technical details only)
Post by: David Graser on April 29, 2018, 08:13:50 pm
Styler/2 worked fine on my machine with OS/4 kernel, as well as REFORMAT. So, probably, you have problems with paths. Probably, LVM.DLL is not on LIBPATH.

PS: Do you have different CONFIG.SYS for OS/4 and IBM kernel? If so, you should compare them for any differences.

To test the config.sys, I used Doug Bissett's latest config.sys sorter to sort both config.sys and config.os4 (Renamed it config.sys for the sort and then renamed it back to config.os4)  Then booted to each.  The problem still exist when using the OS4 kernel.  The problem disappears with the IBM kernel.  I can see no difference in the libpath statement for the way things are listed.  However, this is something I can live with because I have the option to boot using either kernel which I like.

Does anyone know if there are plans for the new OS4 kernel to do away with the SIQ (single Input Que)?
Title: Re: OS/4 (technical details only)
Post by: OS4User on April 29, 2018, 10:44:36 pm
LVM is highlighted red saying it has a problem

I can confirm that with LVM.DLL rev 1.04.2 it's true. At the same time rev 1.03 of this DLL works ok. After compress/decompress LVM.DLL rev 1.04.2 by LXLITE, it does not create problems any more. What you need is:   LXLITE LVM.DLL  and after that LXLITE -x LVM.DLL
Title: Re: OS/4 (technical details only)
Post by: David Graser on April 30, 2018, 05:21:07 pm
LVM is highlighted red saying it has a problem

I can confirm that with LVM.DLL rev 1.04.2 it's true. At the same time rev 1.03 of this DLL works ok. After compress/decompress LVM.DLL rev 1.04.2 by LXLITE, it does not create problems any more. What you need is:   LXLITE LVM.DLL  and after that LXLITE -x LVM.DLL

Thank you OS4User!!!  Did as you suggested and everything works as it should.  Hopefully other Arca 5.02 users will fix their lvm.dll also.

David
Title: Re: OS/4 (technical details only)
Post by: David Graser on May 08, 2018, 04:40:28 am
I just checked you latest OS4 kernel on my Lenovo Thinkpad T430s and it works great. Picture of some of its features.  This is with ArcaOS 5.02. 
Title: Re: OS/4 (technical details only)
Post by: Mikhail on July 01, 2018, 09:31:36 pm
Hello everyone!
I'm a little confused. Is it possible to install the OS.4 kernel on the current version of ArcaOS (5.02)?
I still have the old version of the ACPI installation package (3.23.01), but after its installation the system stops loading - apparently the kernel already works only with a fresh ACPI driver.
How bad is the idea of ​​installing OS / 4 without a patch on ACPI?
My ACPI.pcd (from the package for ArcaOS) fails to verify the patch with the correct ACPI3_23_01.PAT file
Title: Re: OS/4 (technical details only)
Post by: Roderick Klein on July 01, 2018, 10:48:31 pm
No. to the best of my understanding you can not install the OS/4 kernel with the current versions of ACPI.

Best regards,

Roderick Klein

Title: Re: OS/4 (technical details only)
Post by: Lars on July 01, 2018, 11:10:23 pm
OS/4 comes with OS4APIC.PSD which you have to use with the /APIC Switch to Use multiple CPUs.
Title: Re: OS/4 (technical details only)
Post by: Mikhail on July 01, 2018, 11:14:25 pm
Soooo ... i.e. there is no need for a patch ACPI when using APIC?
Title: Re: OS/4 (technical details only)
Post by: Roderick Klein on July 01, 2018, 11:57:26 pm
Well that driver as far as I can tell does not provide all the support that the ACPI.PSD provices.
Title: Re: OS/4 (technical details only)
Post by: Mikhail on July 02, 2018, 01:27:50 am
I took a risk, but something went wrong. The current release and the previous one that I ran on the previous motherboard did not boot. On booting I see only the logo. If I delete it then I see the inscription, something like - Loading: oemhlp, and it lasts forever

The current motherboard is Supermicro X10SBA, before that OS4 tested on different platforms, Andrey Vasilkin was very helpful, then there were no problems.
Title: Re: OS/4 (technical details only)
Post by: Lars on July 02, 2018, 09:16:46 am
OS4APIC.PSD supports multiple processors and in particular, the use of the APIC instead of the old PIC (something that is necessary for a multi-core / multi-processor system ,the PIC does not support a multi-processor configuration).
It does not support ACPI as such. However there is a free driver ACPI4.BPD (or some such, on the web page where you also find the OS4 kernel) that adds ACPI support. The current state of ACPI4.BPD is experimental. You just drop the file into the \OS2\BOOT directory where it is loaded automagically.
Apart from OS4KRNLR / OS4KRNL you will also need to dump all the other stuff of the OS4 kernel package to the right place. There are various drivers that need to go into the OS2\BOOT directory, including OEMHLP.BPD which is the OS4 kernel guys' way of supporting the OEMHLP driver interface. Again, this driver is loaded automagically (as every driver under OS4 which has the .BPD file name extension).
Last but not least you have an updated DOSCALL1.DLL in the OS4 kernel package. I have placed that into a subdir \OS4\DLL so that I can create myself an alternate config.os4 (which you need to do anyway so that you can load different drivers for the OS4 kernel) where I put \OS4\DLL before \OS2\DLL in LIBPATH.
I can only speak for myself but on my system (an 8-core AMD board from ASRock) the OS4 kernel works just as well as the standard OS2 kernel.

The only current change that the OS4 kernel guys have decided to do is to make the RM (Resource Management) interface to the drivers a 32-bit/flat interface which brings some changes to device driver writers. But nothing the users is directly affected with.

If you do all of the above then yes, you won't need ACPI.PSD if you run the OS4 kernel.
Title: Re: OS/4 (technical details only)
Post by: OS4User on July 02, 2018, 03:11:01 pm
I'm a little confused. Is it possible to install the OS.4 kernel on the current version of ArcaOS (5.02)?

Somebody told me that he successfully installed OS/4 into latest ArcaOS.

I still have the old version of the ACPI installation package (3.23.01), but after its installation the system stops loading - apparently the kernel already works only with a fresh ACPI driver.

First of all, I hope that you have performed the install exactly as it is written in the HowTo file supplied with kernel. OS/4 does not support the latest version of ACPI.psd so you need an older version.

How bad is the idea of ​​installing OS / 4 without a patch on ACPI?

Unpatched ACPI.psd spoils OS/4 kernel during the system boot that's why it has to be patched and /ST=0 has to be put into PSD line in Config.sys

My ACPI.pcd (from the package for ArcaOS) fails to verify the patch with the correct ACPI3_23_01.PAT file

ACPI.psd from Arca package  may be needs to be unpacked by lxlite utility before applying the patch.

Title: Re: OS/4 (technical details only)
Post by: Lars on July 02, 2018, 05:05:30 pm
So what does ACPI.PSD add ? If I use ACPI4.BPD I can do the things that I expect: for example, execute "shutdown" with the system powering off at the end (much like APM). For CPU temperature control, I don't know in how far ACPI4.BPD supports that.

The one thing that OS4APIC.PSD/ACPI4.BPD does not support is MSI interrupts (or am I wrong ?) which the latest versions of ACPI.PSD do. But as far as I understand you need 2 things for MSI to work:
1) MSI support in ACPI.PSD (or OS4APIC.PSD/ ACPI4.BPD)
2) MSI support in the kernel itself (AN comes with an updated kernel that goes together with the latest versions of ACPI.PSD)


That's one of the reasons why I said repeatedly that the OS4 kernel needs better documentation ...
Title: Re: OS/4 (technical details only)
Post by: Mikhail on July 02, 2018, 05:19:12 pm
If you do all of the above then yes, you won't need ACPI.PSD if you run the OS4 kernel.

For the ArcaOS 5.02 I installed, I did all these things. I'll try on the older 5.01, but I think in this case it's a hardware compatibility issue.

Well, we will build a UPNP audio renderer on the stock os/2 core.
Title: Re: OS/4 (technical details only)
Post by: David Graser on July 02, 2018, 06:17:39 pm
[D:\OS2\BOOT]bldlevel acpi.psd
Build Level Display Facility Version 6.12.675 Sep 25 2001
(C) Copyright IBM Corporation 1993-2001
Signature:       @#Arca Noae LLC:3.23.09#@##1## 13 May 2018 19:5
   ::::09::@@ACPI based PSD for OS/2 (c) 2018 Arca Noae LLC
Vendor:          Arca Noae LLC
Revision:        3.23.09
Date/Time:       13 May 2018 19:55:35
Build Machine:   DAZAR1
File Version:    3.23.9
Description:     ACPI based PSD for OS/2 (c) 2018 Arca Noae LLC

This is with ArcaOS 5.02.

Correct me if I am wrong, but I believe the patches only work for versions 3.23.6 and earlier.
Title: Re: OS/4 (technical details only)
Post by: OS4User on July 02, 2018, 06:22:27 pm
Correct me if I am wrong, but I believe the patches only work for versions 3.23.6 and earlier.

Patches are available for:
3.22.03
3.22.06
3.22.06 dbg
3.23.01
3.23.01 dbg
Title: Re: OS/4 (technical details only)
Post by: Mikhail on July 02, 2018, 06:23:02 pm
That's one of the reasons why I said repeatedly that the OS4 kernel needs better documentation ...

I confirm these words!

As I said earlier, without help from Andrey Vasilkin, I would hardly understand how to make the correct installation, it was in the build SVN5067, then the manual was much worse than the current one. Now the folder structure in the archive is ready and there is a clear manual, but still a lot of conventions that are not obvious.

For example, I,as an ArcaOS user, do not understand which version (5.01, 5.02) OS/4 installation is preferred. Is it possible to apply the patch to the versions of the ACPI emerging in packages subscription to ArcaOS, and generally whether it is if there is an alternative in the face of ACPI4? Do I need to register ACPI4.Bpd in config? Or just put it in the OS2/BOOT folder

Even that would just find APIC4.BPD I've wasted a lot of time.

But all those guys including the ones from Russia who are engaged in this product - Уважуха!

P.S. - On the SuperMicro X10SBA motherboard with the Intel® Celeron J1900 (4cores) processor, I was unable to run the OS/4 project with ArcaOS 5.01 or 5.02, as ACPI was using ACPI4.BPD and PSD = os4apic.psd / APIC
But the loading stop, it seems to me, took place long before the ACPI check. The last inscription on the screen without logo - loading oemhlp
 those. the stop occurs immediately after you try to load the selected item from the os2ldr menu.

Sadly, I wanted to evaluate the effect of this core on the sound quality in the system with this motherboard.
Title: Re: OS/4 (technical details only)
Post by: OS4User on July 03, 2018, 12:59:28 am
but still a lot of conventions that are not obvious.
Please  advise what wasn't obvious and it will be added to HowTo file

For example, I,as an ArcaOS user, do not understand which version (5.01, 5.02) OS/4 installation is preferred.
To tell the truth, the version of ArcaOS doesn't matter, that's why this info is not included in HowTo file.

Is it possible to apply the patch to the versions of the ACPI emerging in packages subscription to ArcaOS, and generally whether it is if there is an alternative in the face of ACPI4? Do I need to register ACPI4.Bpd in config? Or just put it in the OS2/BOOT folder

Even that would just find APIC4.BPD I've wasted a lot of time.

The available patches are listed in the README.TXT which is together with patches.
ACPI.BPD is beta now and is not distributed widely. When one person receives ACPI.BPD he also receives instructions how to apply it to the system. ACPI.BPD is an optional driver.

P.S. - On the SuperMicro X10SBA motherboard with the Intel® Celeron J1900 (4cores) processor, I was unable to run the OS/4 project with ArcaOS 5.01 or 5.02, as ACPI was using ACPI4.BPD and PSD = os4apic.psd / APIC
But the loading stop, it seems to me, took place long before the ACPI check. The last inscription on the screen without logo - loading oemhlp those. the stop occurs immediately after you try to load the selected item from the os2ldr menu.

It would be great if you could get a comport log, meanwhile in the OS/4 hardware tested list there is one mobo with J1900, so there is a chance that your mobo may also start working.
Title: Re: OS/4 (technical details only)
Post by: Mikhail on July 03, 2018, 12:30:15 pm
It would be great if you could get a comport log, meanwhile in the OS/4 hardware tested list there is one mobo with J1900, so there is a chance that your mobo may also start working.

Please tell me how to get such a log. I need a Com-null modem cable and some software on the other side (Windows OS & with logging software, perhaps)? This type of testing, I have not yet conducted.
Title: Re: OS/4 (technical details only)
Post by: OS4User on July 03, 2018, 01:21:44 pm
Please tell me how to get such a log. I need a Com-null modem cable and some software on the other side (Windows OS & with logging software, perhaps)? This type of testing, I have not yet conducted.

Here is some info how to get com port log https://www.os2world.com/wiki/index.php/Phoenix_OS/4
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 04, 2019, 06:44:50 am
I use OS/4 kernel on one of my partitions.

Presently, I have been using test build SVN5447.

i noticed that when I try to use FF 45.9.0, it always traps at this web page.   It occurs with FFTurbo enabled or disabled.

https://beforeitsnews.com/v3/

If I boot using the OS/2 loader and kernel, the traps disappear.

Attached are a couple of trap logs.

The traps occur on earlier test builds of the OS/4 kernel also.
Title: Re: OS/4 (technical details only)
Post by: Lars on March 04, 2019, 07:23:12 am
1) looks like you have an out of date EXCEPTQ package. It does not properly decode the real instruction.

2) the failing instruction starts with a VEX prefix (and is incorrectly decoded in the following). It is likely an SSE or AVX instruction that fails due to unaligned memory (memory accesses need to be 16-byte aligned for SSE (128 bit) registers and 32-byte aligned for AVX (256 bit) registers).
That said I think there is a problem wtih libcn0.dll that needs a rebuild with a new compiler so that data used with SSE/AVX instructions is properly aligned (for all stack based references and also all static data in the data segment). You might want to try a newer test build of the new "libc" package (which contains libcn0).

Question is why does it only appear with the OS/4 kernel ? I can only guess: the code takes a different code path with the OS/2 kernel and the OS/4 kernel. Maybe because OS/4 kernel signals AVX support (if of course the CPU supports it) and the OS/2 kernel does not.

Lars
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 04, 2019, 08:00:16 am
Hi David, can you install the FFmpeg debug package (ffmpeg-debug) as well as the libc debug package and produce another trp?
I take it you were playing a video or other media when it crashed?
Perhaps also try playing some videos with FFplay and see if it crashes. I don't think the Netlabs build has exceptq support so it'll just print some exception info.
Lars may well be right about libcn, be interesting if the same crash occurs with kLIBC.
The SSE and VEX code in libavcodec is all handwritten assembler, so unless there is a bug in our nasm port, libcn seems like a candidate for the failure.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 04, 2019, 08:31:49 am
https://beforeitsnews.com/v3/

Hi David

I have just checked this site on my os/4 based system  - it works ok, but my avcode56.dll has different size  (6 053 546) inspite of the same date. May be it is because of deferent platform.
And I still use libc066-38(oc00).


What CPU do you use in your comp? And which platform your avcode56 is compiled for?

Title: Re: OS/4 (technical details only)
Post by: Lars on March 04, 2019, 09:18:01 am
Looking at it again:

If it is true that all SSE/AVX code in AVCODE56.DLL is handwritten in assembler then I suspect either one of 2 possibilities:

1) some dynamic memory is passed to AVCODE56.DLL which is not aligned: but from the used registers, EAX and EDI point to memory allocated by LIBCN0 which aligns to 32-bytes. That would be enough for both, SSE and also AVX.

2) the data used in AVCODE56.DLL itself is not properly aligned (even if coded by hand). Maybe someone forgot that AVX code needs data to be 32-byte aligned when some code was upgraded from SSE to AVX. The incorrectly decoded code seems to imply that some constant seems to be taken directly from the code segment (address: 0x0cd22e50). That is, this constant is directly embedded in the code. And this constant is only 16-byte aligned where AVX code requires 32-byte alignment. If somebody had the AVCODE56 code, it would be easy to check.

If I had to guess I would think that the incorrectly decoded instruction is a VMOVDQA or VMOVAPD, VMOVAPS instruction:
VMOVAPD ymm0,[0x0cd22e50). And according to Intel manual, this instruction requires 32-byte data alignment.

Lars
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 04, 2019, 10:09:23 am
Looking at it again:
If it is true that all SSE/AVX code in AVCODE56.DLL is handwritten in assembler then I suspect either one of 2 possibilities:
....

To Lars:

What you saying is quite possible. One more way to check it -  is test OS/4 kernel without AVX support.

To David:

Pls, open  your site   on this test kernel http://gus.biysk.ru/os4/os4krnl_noAVX.zip
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 04, 2019, 12:37:27 pm
Looking at it again:
If it is true that all SSE/AVX code in AVCODE56.DLL is handwritten in assembler then I suspect either one of 2 possibilities:
....

To Lars:

What you saying is quite possible. One more way to check it -  is test OS/4 kernel without AVX support.

To David:

Pls, open  your site   on this test kernel http://gus.biysk.ru/os4/os4krnl_noAVX.zip

Thanks for all the responses.

I just installed the noAVX kernel and my traps have disappeared.  This is just preliminary but I have no problem going to that web site and others so far.  So far, so good.

My avcode56.dll is the same size and date on both my ArcaOS 5.02 and ArcaOS 5.03 partitions.

Attached a pic of my type of CPU info.  While modifying this message, I had another trap which I have attached.  I just remembered that this is another instance when a trap occurred. However, so far, this one trap is all I have encountered so far with the noAVX.

By the way Dave, I just installed the debug versions you recommended after this last trap.  So far though, Firefox is much more stable with the noAVX kernel version.  It feels good to be able to use FF again without it continually crashing when using the OS/4 kernel. 
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 04, 2019, 02:39:35 pm
I just installed the noAVX kernel and my traps have disappeared.

It looks like Lars was correct and the issue is related to AVX instruction. Meanwhile, the bug is not in kernel, but probably in AVCODE56.DLL or in the code which calls it.

If OS/4 kernel detects AVX hardware it installs support for AVX  and sets proper bit in CR4.

The software may detects this bit later on and engages own AVX code branch. (There are a few progs which successfully use AVX.  For ex. https://bitbucket.org/dryeo/dry-comm-esr31/downloads/firefox-45.9.0.en-US.os2.SandyBridge.zip)

P.S.
It was suggested by BWW to remove libc-debug 0.6.6-40  and install libc-debuginfo instead of it.
Title: Re: OS/4 (technical details only)
Post by: Lars on March 04, 2019, 05:00:46 pm

Thanks for all the responses.

I just installed the noAVX kernel and my traps have disappeared.  This is just preliminary but I have no problem going to that web site and others so far.  So far, so good.

My avcode56.dll is the same size and date on both my ArcaOS 5.02 and ArcaOS 5.03 partitions.

Attached a pic of my type of CPU info.  While modifying this message, I had another trap which I have attached.  I just remembered that this is another instance when a trap occurred. However, so far, this one trap is all I have encountered so far with the noAVX.

Yes, the "new" trap is not AVX related at all. It is something completely different. Looks like EBX is supposed to contain a full address but it seems to contain only an address offset (or an offset summed up on top of a NULL pointer which is more likely ...)

Too bad, your screen shot does not show the feature flags. That would have been more interesting ...

Lars
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 04, 2019, 06:18:02 pm
Hi Lars

I have attached a picture of the CPU flags.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 04, 2019, 06:24:40 pm
I just installed the noAVX kernel and my traps have disappeared.

P.S.
It was suggested by BWW to remove libc-debug 0.6.6-40  and install libc-debuginfo instead of it.

Done.  Thanks OS4User.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 05, 2019, 01:00:30 am
Hi David, that crash looks like one I get now and again, I believe font related.
Have you tested with the Firefox build that OS4USER mentioned? Perhaps with the AVX enabled kernel? You can get a list of CPU capabilities with this program, http://hobbes.nmsu.edu/download/pub/os2/util/misc/cpuinfo.zip (http://hobbes.nmsu.edu/download/pub/os2/util/misc/cpuinfo.zip)
If needed, I could build FFmpeg with AVX disabled. Be nice to know which function it is crashing in too.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 05, 2019, 01:13:29 am
Hi Lars, FFmpeg is pretty well tested with the fate suite of tests, probably a couple of thousand test now. See (currently not loading) http://fate.ffmpeg.org/ (http://fate.ffmpeg.org/)
I used to run it and pass all the tests. Unluckily our tool chain is just too buggy now. Problems with GCC, problems with libc, problems with git, even problems with the PMSHELL at least.
To pass I was using an early build of GCC 4.4.6, libc065 on Warp v4.
Problems included. At one point using the official nasm built with OpenWatcom, I had four H264 conformance tests failing, there were quite a few at the time. This was fixed by recompiling nasm with GCC. Weird subtle failure.
Upgrading GCC caused various transcoding failures as well filter errors.
Upgrading to eCS caused floating point crashes without patching to restore the FPU control word.
Upgrading to libc066 fixed the floating point errors but caused errors transcoding to MP2 and failures with other tests that depended on the MP2 file.
Currently running the testsuite causes git to fail and I believe output its help, at least it is actually less that hangs waiting for input. IIRC, this is a test to make sure all code has correct licensing and cats the output of git through grep or similar.
There are other tests where the ffmpeg binary hangs, killing it allows the test suite to continue.
Most platforms usually pass all the tests.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 05, 2019, 01:43:05 am
Hi David, that crash looks like one I get now and again, I believe font related.
Have you tested with the Firefox build that OS4USER mentioned?

Dave, I tried the SandyBridge port of FF and the program would not start when using the OS/4 kernel and loader. This is when I decided to post here to see what could be done. I did not try it with the OS/2 kernel.  The noAVX version is working great with the OS/4 kernel. The lastest build of your FF 45.9.0 pentium is the best yet.  I am having no problems watching videos and accessing web pages.  I no longer have to boot to Windows to get reliable internet.  Again, thanks to all who continue to donate their time and effort to make using ArcaOS and other versions more enjoyable.

PS- I posted another graphic of the CPU cabilities (feature flags) above at reply #55.

David
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 05, 2019, 02:48:38 am
PS- I posted another graphic of the CPU cabilities (feature flags) above at reply #55.

Yes, it seems to be incomplete, only showing up to SSE2.
Hmm, looking at Intel's site, it does seem to support AVX, which is what I was wondering.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 05, 2019, 03:00:24 am
PS- I posted another graphic of the CPU cabilities (feature flags) above at reply #55.

Yes, it seems to be incomplete, only showing up to SSE2.
Hmm, looking at Intel's site, it does seem to support AVX, which is what I was wondering.

I can verify it  does support AVX.  Attached pic
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 05, 2019, 04:18:49 am
Thanks
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 05, 2019, 04:44:11 am
Should I open a ArcaOS ticket concerning AVX?
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 05, 2019, 06:21:22 am
There's already one open about looking into the feasibility of adding AVX support to the IBM kernel, which you probably can't access. Not much there besides a description of the needed feature.
The problem is that during a context switch, the AVX registers need to be preserved. Those registers can be up to 512 bits wide. Think of what happens if two programs are using AVX when the OS switches between them. I doubt that we'll see AVX support in the official kernels anytime soon.
You may have hit on a bug in the OS4 kernel, at least OS4USER is aware of your crash.
Would still be interesting to know what function was crashing in libavcodec though I wouldn't be surprised if something just wasn't aligned correctly. Interesting that the AVX enabled Firefox didn't run for you as well. Did it generate a trp?
Exceptq could also use an update to correctly disassemble AVX opcodes. Luckily Lars can spot them.
Title: Re: OS/4 (technical details only)
Post by: Lars on March 05, 2019, 08:09:47 am
Hi David, that crash looks like one I get now and again, I believe font related.
Have you tested with the Firefox build that OS4USER mentioned? Perhaps with the AVX enabled kernel? You can get a list of CPU capabilities with this program, http://hobbes.nmsu.edu/download/pub/os2/util/misc/cpuinfo.zip (http://hobbes.nmsu.edu/download/pub/os2/util/misc/cpuinfo.zip)
If needed, I could build FFmpeg with AVX disabled. Be nice to know which function it is crashing in too.

If you sent me the MAP file for AVCODE65.DLL it would be a piece of cake to find the failing routine. We have the trap location ...

As I said, the "something" not aligned correctly seems to be a 32-byte (AVX) constant (read only) directly encoded in the code segment.
Where do I find the code for AVCODE65.DLL ?
I looked at:
https://github.com/FFmpeg/FFmpeg/tree/master/libavcodec
but there is no assembler code there, only plain "C" code. Which makes me wonder why we do not use plain "C" code and let GCC do all the gory stuff like data alignment (once GCC aligns SSE and AVX data the way it should ...). I would expect a good "C" compiler to automatically parallelize data accesses (and therefore making good use of the SIMD capabilities of the CPU). At least the Intel compiler does that ...


By the way: since the OS/2 kernel does not support AVX, wouldn't it be better to NOT build with AVX capability in general ?
I don't have AVX (only SSE) and therefore I am not affected but we can already see that with more modern systems that do support AVX or even AVX2 (with 512-bit registers), we now run into a whole lot of problems with AVX (data not properly aligned, no OS support for saving away AVX registers on a thread switch, with exception of OS/4 kernel).
Note: OS/2 does have support to save and restore SSE registers on a thread swith.

For me, OS/2 is falling more and more apart.


Lars
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 05, 2019, 12:08:19 pm
There's already one open about looking into the feasibility of adding AVX support to the IBM kernel, which you probably can't access. Not much there besides a description of the needed feature.
The problem is that during a context switch, the AVX registers need to be preserved. Those registers can be up to 512 bits wide. Think of what happens if two programs are using AVX when the OS switches between them. I doubt that we'll see AVX support in the official kernels anytime soon.
You may have hit on a bug in the OS4 kernel, at least OS4USER is aware of your crash.
Would still be interesting to know what function was crashing in libavcodec though I wouldn't be surprised if something just wasn't aligned correctly. Interesting that the AVX enabled Firefox didn't run for you as well. Did it generate a trp?
Exceptq could also use an update to correctly disassemble AVX opcodes. Luckily Lars can spot them.

Dave, I just tried the SandyBridge version with the lastest OS/4 kernel and FF works now.  Don't know why it did not start before.  Cannot test further since I am at work.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 05, 2019, 04:45:39 pm
I just tried the SandyBridge version with the lastest OS/4 kernel and FF works now

try this http://hobbes.nmsu.edu/download/pub/incoming/freerdp-wpi-20190301.zip
it also requires AVX
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 05, 2019, 06:09:11 pm
Hi David, that crash looks like one I get now and again, I believe font related.
Have you tested with the Firefox build that OS4USER mentioned? Perhaps with the AVX enabled kernel? You can get a list of CPU capabilities with this program, http://hobbes.nmsu.edu/download/pub/os2/util/misc/cpuinfo.zip (http://hobbes.nmsu.edu/download/pub/os2/util/misc/cpuinfo.zip)
If needed, I could build FFmpeg with AVX disabled. Be nice to know which function it is crashing in too.

If you sent me the MAP file for AVCODE65.DLL it would be a piece of cake to find the failing routine. We have the trap location ...

I don't have the map files for the Netlabs build of FFmpeg. I can build one after work if David is interested in testing.

Quote
As I said, the "something" not aligned correctly seems to be a 32-byte (AVX) constant (read only) directly encoded in the code segment.
Where do I find the code for AVCODE65.DLL ?
I looked at:
https://github.com/FFmpeg/FFmpeg/tree/master/libavcodec
but there is no assembler code there, only plain "C" code. Which makes me wonder why we do not use plain "C" code and let GCC do all the gory stuff like data alignment (once GCC aligns SSE and AVX data the way it should ...). I would expect a good "C" compiler to automatically parallelize data accesses (and therefore making good use of the SIMD capabilities of the CPU). At least the Intel compiler does that ...

Start here for the assembler code, https://git.ffmpeg.org/gitweb/ffmpeg.git/tree/HEAD:/libavcodec/x86 (https://git.ffmpeg.org/gitweb/ffmpeg.git/tree/HEAD:/libavcodec/x86) there's also some in libavutil.
As for using plain C, building FFmpeg with --disable-assembler results in code that is so slow it is unusable. GCC is quite crappy at optimizing compared to hand written assembly by experts.
Also need assembler to dynamically optimize based on CPU, which simd instructions it supports, how many registers are available, even things like whether using CMOV is a good idea or if it is a really slow implementation.
GCC is still calling the assembly functions so I guess that is where alignment comes in.

Quote
By the way: since the OS/2 kernel does not support AVX, wouldn't it be better to NOT build with AVX capability in general ?
I don't have AVX (only SSE) and therefore I am not affected but we can already see that with more modern systems that do support AVX or even AVX2 (with 512-bit registers), we now run into a whole lot of problems with AVX (data not properly aligned, no OS support for saving away AVX registers on a thread switch, with exception of OS/4 kernel).
Note: OS/2 does have support to save and restore SSE registers on a thread swith.

Lots of code uses AVX assembler and dynamically loads it if the OS and CPU support it. Firefox's JIT, libvpx, libx264, even flac just of the top of my head. Most I don't have control over.
It does seem to help using a newer GCC. Hopefully when Bitwise moves to GCC 8 (soon), things will improve.
Quote
For me, OS/2 is falling more and more apart.


Lars
:)
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 05, 2019, 06:12:39 pm
Dave, I just tried the SandyBridge version with the lastest OS/4 kernel and FF works now.  Don't know why it did not start before.  Cannot test further since I am at work.

I take it you are using the AVX enabled build of the OS4 kernel. Is that bad web page now working for you with avcod56 and the Sandybridge Firefox?
If not I can build some test builds of FFmpeg this evening.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 05, 2019, 06:43:32 pm
Dave, I won't know until I get home from work to test it and yes it is the AVX enabled OS/4 latest kernel.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 05, 2019, 11:51:09 pm
Dave, I just tried the SandyBridge version with the lastest OS/4 kernel and FF works now.  Don't know why it did not start before.  Cannot test further since I am at work.

I take it you are using the AVX enabled build of the OS4 kernel. Is that bad web page now working for you with avcod56 and the Sandybridge Firefox?
If not I can build some test builds of FFmpeg this evening.

That SandyBridge version with the latest AVX OS/4 kernel is working fine.   That bad web page is working fine.  Everything loads fine.  No crashes.  This may be the best so far.  Only time will tell.

I spoke too soon.  It appears to crash every time I go to youtube videos and it tries to load a video.  Attached are the trap messages.  ffturbo does not work with this version. It won't start.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 06, 2019, 12:00:22 am
I just tried the SandyBridge version with the lastest OS/4 kernel and FF works now

try this http://hobbes.nmsu.edu/download/pub/incoming/freerdp-wpi-20190301.zip
it also requires AVX

OS2user, I am not that savvy with programs like this.  I installed it, but don't know how to set it up or what I would use it for.
Title: Re: OS/4 (technical details only)
Post by: Dariusz Piatkowski on March 06, 2019, 12:29:44 am
David,

OS2user, I am not that savvy with programs like this.  I installed it, but don't know how to set it up or what I would use it for.

That is a RDP (Remote DeskTop) client, which allows you to connect to a remote Desktop / PC somewhere over the network. Typically that might be a WINx machine that you need to connect to in order to do some action/work on that remote box.

I am using this client on my LAN to ease the maintenance of all the PCs in the house. I occasionally use the remote WINx boxes to do stuff that my local OS/2 box does not allow me to do (lack of apps with specific support, etc.).

Do you have a WINx box on your LAN/network that you could try connecting to?
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 06, 2019, 12:37:53 am
David,

OS2user, I am not that savvy with programs like this.  I installed it, but don't know how to set it up or what I would use it for.

Do you have a WINx box on your LAN/network that you could try connecting to?

Hi Dariusz

I have no network set up.  I essentially use just one machine.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 06, 2019, 01:47:39 am
Hi David.
Ok, it wasn't the previous browser. Go to https://bitbucket.org/dryeo/mozilla-os2/downloads/ (https://bitbucket.org/dryeo/mozilla-os2/downloads/) and download the firefox-sandybridge-xqs package and install it in your sandybridge firefox directory. Seems I forgot to package the xqs files :)
Then download the ffmpeg-3.2.13-libs.zip package and put the DLLs in @unixroot\usr\local\lib, which should be ahead of @unixroot\usr\lib in your LIBPATH. Restart the browser and test playing a video. If it traps, look in the trp file to make sure it was using and trapped in the new avcode57 DLL and post the trp file (I have the source for this one). If it still used avcode56, you might need to uninstall the ffmpeg libs package.
We'll see if using a different compiler helps. If not, I'll build an AVX disabled version for you.
Thanks
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 06, 2019, 02:38:23 am
Still traps at the youtube page and still traps at avcode56, Attached trap file.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 06, 2019, 02:48:17 am
Try moving avcode56.dll and avutil55.dll out of the way and putting the new DLLs in their place
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 06, 2019, 01:44:10 pm
Try moving avcode56.dll and avutil55.dll out of the way and putting the new DLLs in their place

OK Dave, I moved the 2 dlls  out of the libpath, installed the sandybride  FF and the AVX OS/4 kernel and the traps have disaapeared.  I can access youtube without it trapping.  Will test more today. 

Note, I had gone back to the noAVX OS/4 kernel  with the FF perntium version and this has been the most stable so far.  I could not get it to crash. 

I will test he sandybridge version with the AVX OS/4 kernel with the two dlls removed more.  So far, it has not crashed at the the two web pages I have had problems with.  It looks it may be just as stable  as the noAVX version..  Time will tell.

The Sandybridge version is now just as stable as the pentium version with those two dlls removed.  I did find one site that it is unable to play the video whereas the noAVX version was able to.

https://www.trunews.com/stream/diplomacy-sabotage-unpatriotic-democrats-torpedoed-us-north-korea-peace-summit
Title: Re: OS/4 (technical details only)
Post by: Lars on March 06, 2019, 04:39:19 pm
Hi Dave (Yeo),

after looking at your LIBAV repo and at the trap info, it appears to me that the IMDCT ("Inverse Modified Discrete Cosine Transform") implementation only contains SSE support but not AVX support. Unfortunately, the SSE implementation of the IMDCT calls the AVX implementation of the FFT ("Fast Fourier Transfomration").
I guess this is where some constant data (say, the FFT coefficient vectors) becomes unaligned.


Lars
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 06, 2019, 06:33:23 pm
Hi Dave

It seams I am missing your point with replacement of ffmpeg libs.
I understand you offer to replace avcode56.dll/avutil54.dll with avcode57.dll/avutil55.dll.
But how FF will come to know to use this new pair ?
For a sake of testing I did such replacement on my comp and youtube disappeared.
I am not sure but looks like FF has to be rebuilt against this new pair of libs.

(I do not have an AVX equipment to do a proper test.)

At the same time thanks a lot for building AVX  binaries - it's obvious that this technology has to be available for our community too.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 06, 2019, 07:52:33 pm
Hi Dave (Yeo),

after looking at your LIBAV repo and at the trap info, it appears to me that the IMDCT ("Inverse Modified Discrete Cosine Transform") implementation only contains SSE support but not AVX support. Unfortunately, the SSE implementation of the IMDCT calls the AVX implementation of the FFT ("Fast Fourier Transfomration").
I guess this is where some constant data (say, the FFT coefficient vectors) becomes unaligned.

Hi Lars, that was the impression I had. Note that the libav code is fairly old and a fork of FFmpeg so this may have changed. Unluckily this old browser doesn't want to load the right branch at FFmpeg's git repository. Current libs from the 3.2 branch. I just did a minor patch to find a header.
Have to continue testing if the GCC 5.5.0 build does better. It helps with Firefox.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 06, 2019, 08:01:46 pm
Hi Dave

It seams I am missing your point with replacement of ffmpeg libs.
I understand you offer to replace avcode56.dll/avutil54.dll with avcode57.dll/avutil55.dll.
But how FF will come to know to use this new pair ?
For a sake of testing I did such replacement on my comp and youtube disappeared.
I am not sure but looks like FF has to be rebuilt against this new pair of libs.

(I do not have an AVX equipment to do a proper test.)

At the same time thanks a lot for building AVX  binaries - it's obvious that this technology has to be available for our community too.

Yes, it seems I missed a dependency. Should have checked the map file.
Code: [Select]
K:\work\ffmpeg\src\ffmpeg.obj\libswresample\libswresample_dll.lib
K:\work\ffmpeg\src\ffmpeg.obj\libavutil\libavutil_dll.lib
H:\tmp\ldconv_libcx_a_99775c7f48361effd8.lib
H:\tmp\ldconv_libvpx_dll_a_99775c7f48361effd8.lib
H:\tmp\ldconv_lzma_dll_a_99775c7f48371eb5a0.lib
H:\tmp\ldconv_libz_dll_a_99775c7f48371eb5a0.lib
K:\usr\local550\lib\gcc\i686-pc-os2-emx\5.5.0\libgcc_so_d.lib
K:\usr\lib\libc_alias.lib
K:\usr\lib\libc_dll.lib
K:\usr\lib\libos2.lib
K:\usr\lib\libend.lib

So need swresa2.dll as well as lzma, z and libvpx installed.

Firefox tries to dynamically load the DLLs in this order, from FFmpegRuntimeLinker.cpp
Code: [Select]
#if defined(XP_OS2)
// On OS/2, we must load avutil directly to access its symbols
static const char* sLibs[][2] = {
  { "avcode57.dll", "avutil55.dll" },
  { "avcode56.dll", "avutil54.dll" },
  { "avcode55.dll", "avutil54.dll" },
  { "avcode53.dll", "avutil51.dll" },
};

So anyways need to redownload the libs package.
Can test whether its working at https://www.youtube.com/html5 (https://www.youtube.com/html5). Should show H.264 and VP8+VP9 working.
David will also need to redo the libraries.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 06, 2019, 08:56:26 pm
Dave:

So anyways need to redownload the libs package.

After redownload https://bitbucket.org/dryeo/mozilla-os2/downloads/ffmpeg-3.2.13-libs.zip and installation of  ALL the libs from it  youtube came back.

Theseus also confirms that  avcode57 is in use and avcode56 is no longer in use.

So we need to test new ffmpeg on AVX equipment.


David:

Looking forward to your test of updated  ffmpeg.

Title: Re: OS/4 (technical details only)
Post by: David Graser on March 06, 2019, 10:08:38 pm
Using
ArcaOS 5.02
Firefox Sandbridge 45.9.0
Latest OS/4 test kernel SVN5449
Latest libs downloaded.

It crashed periodically starting with the following web page.  Then the OS/2world site starting crashing.

https://www.trunews.com/stream/diplomacy-sabotage-unpatriotic-democrats-torpedoed-us-north-korea-peace-summit

FF traps with the following trap messages.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 07, 2019, 12:00:46 am
OK, I uploaded a version with AVX disabled if you'd like to test.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 07, 2019, 12:46:48 am
OK, I uploaded a version with AVX disabled if you'd like to test.

I downloaded it, but may not get to test it to totomorrow. I am at work right now.

Do I use these new libs with the sandybridge or the pentium FF version or does it matter?  If I am right, I need to use the AVX version of OS/4 which is the latest test kernel. 

Dave, the noAVX OS/4 kernel along with the FF pentium 45.9.0 version is stable and opens and plays everything I throw at it.  It is the most stable FF I have run while using ArcaOS.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 07, 2019, 01:08:05 am
I downloaded it, but may not get to test it to totomorrow. I am at work right now.

That's fine.
Quote
Do I use these new libs with the sandybridge or the pentium FF version or does it matter?  If I am right, I need to use the AVX version of OS/4 which is the latest test kernel.

They'll work with any CPU newer then a Pentium Pro. The code detects the CPU and OS support and enables various code paths depending on what it is running on. So should be fine on any build of FF 45.9 and the corresponding SeaMonkey. Won't work with FF38 though as it'll only use avcode56.dll or older.

Quote
Dave, the noAVX OS/4 kernel along with the FF pentium 45.9.0 version is stable and opens and plays everything I throw at it.  It is the most stable FF I have run while using ArcaOS.

That's good to hear though Bitwise did the work, I just changed the build environment a bit along with some very minor patches to make the newer compiler happy. Not sure if it is the newer compiler or enabling SSE uses more tested code paths. SeaMonkey has been very stable here as well with the very occasional trap like the other one you posted, perhaps once a month.
The AVX kernel should be fine as well with the Pentium M build, at least I haven't had any reports of problems besides yours.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 07, 2019, 11:15:25 am
Hi Dave,

The fallen instruction is (function fft8_avx):
Code: [Select]
mova       m4,  [ps_cos16_1] ; vmovaps

static variable ps_cos16_1 is 16 bit aligned only (0001:009AE730,  yes it is in code seg)

ref.: https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/HEAD:/libavcodec/x86/fft.asm line 346

AFAIK  the following variables have to be 32 bit aligned too:
Code: [Select]
ps_cos16_2
ps_root2
ps_root2mppm
ps_p1p1m1p1
perm1
perm2
ps_p1p1m1p1root2
ps_m1m1p1m1p1m1m1m1
ps_m1p1
Title: Re: OS/4 (technical details only)
Post by: Lars on March 07, 2019, 01:10:59 pm
After reading this:

https://software.intel.com/en-us/forums/intel-isa-extensions/topic/279587
(in particular the last comment of the Intel guru)

I can only say that the BEST way to go about all this is to NOT use the VMOVAPS/VMOVAPD/VMOVDQA instructions but instead to use the VMOVUPS/VMOVUPD/VMOVDQU instructions:

1) these will allow to load from unaligned addresses without trapping
2) IF the data is aligned nonetheless, the performance will be as good as with the aligned variants of these instructions

What is still true is that the best memory access performance will be achieved with aligned data (32-bytes for AVX, 64-bytes for AFX-512).


This is all so ridiculous ...
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 07, 2019, 06:23:13 pm
OK, I uploaded a version with AVX disabled if you'd like to test.

OK just tested and this version works good.  Went to all the problem web sites and they loaded, worked, and played just fine.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 07, 2019, 07:04:11 pm
OK, we have a workaround, namely building with --disable-AVX.

Now we're just left with the question of why our toolchain is not aligning these AVX instructions. It only seems to be a problem in libavcodec/x86/fft.asm.
One possibility is a weakness in the AOUT format. I vaguely remember problems getting things to work as can be seen by this from libavutil/x86/x86inc.asm
Code: [Select]
; aout does not support align=
; NOTE: This section is out of sync with x264, in order to
; keep supporting OS/2.
%macro SECTION_RODATA 0-1 16
    %ifidn __OUTPUT_FORMAT__,aout
        SECTION .text
    %elifidn __OUTPUT_FORMAT__,coff
        SECTION .text
    %elifidn __OUTPUT_FORMAT__,win32
        SECTION .rdata align=%1
    %elif WIN64
        SECTION .rdata align=%1
    %else
        SECTION .rodata align=%1
    %endif
%endmacro

IIRC, one problem is that aout only supports text segments and is quite primitive compared to newer object formats.
Perhaps I should port the whole build system to using obj (OMF32) format.

Background, aout is an ancient object format that was once common.
Our GCC was forked a long time back and still outputs aout object code. Generally we then use emxomf at link time to convert to native omf and use the system linker to create executables.
Currently in FFmpeg, nasm outputs aout, which is also converted to omf at link time.
We can also convert GCC object files at build time, which would also allow nasm to output omf or obj as nasm calls it.
Obj also needs the FLAT and USE32 parameters to create 32 bit code as it was originally a 16 bit format.

Problem is testing as I don't have AVX hardware.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 07, 2019, 07:34:56 pm
Problem is testing as I don't have AVX hardware.

As indirect way you can describe ps_cos16_1  as Public  and always check its address and segment in map file. And, of course, we can ask David to test AVX ver with proper alignment (at least I hope). 

For some reason  ps_cos16_1 comes to CODE seg (.text ?)  but not to .rdata with proper alignment.
 
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 07, 2019, 07:40:19 pm
Problem is testing as I don't have AVX hardware.

As indirect way you can describe ps_cos16_1  as Public  and always check its address and segment in map file. And, of course, we can ask David to test AVX ver with proper alignment (at least I hope). 

For some reason  ps_cos16_1 comes to CODE seg (.text ?)  but not to .rdata with proper alignment.

I will be glad to help anyway I can.
Title: Re: OS/4 (technical details only)
Post by: Lars on March 07, 2019, 11:12:21 pm
OK, we have a workaround, namely building with --disable-AVX.

Now we're just left with the question of why our toolchain is not aligning these AVX instructions. It only seems to be a problem in libavcodec/x86/fft.asm.
One possibility is a weakness in the AOUT format. I vaguely remember problems getting things to work as can be seen by this from libavutil/x86/x86inc.asm
Code: [Select]
; aout does not support align=
; NOTE: This section is out of sync with x264, in order to
; keep supporting OS/2.
%macro SECTION_RODATA 0-1 16
    %ifidn __OUTPUT_FORMAT__,aout
        SECTION .text
    %elifidn __OUTPUT_FORMAT__,coff
        SECTION .text
    %elifidn __OUTPUT_FORMAT__,win32
        SECTION .rdata align=%1
    %elif WIN64
        SECTION .rdata align=%1
    %else
        SECTION .rodata align=%1
    %endif
%endmacro

IIRC, one problem is that aout only supports text segments and is quite primitive compared to newer object formats.
Perhaps I should port the whole build system to using obj (OMF32) format.

Background, aout is an ancient object format that was once common.
Our GCC was forked a long time back and still outputs aout object code. Generally we then use emxomf at link time to convert to native omf and use the system linker to create executables.
Currently in FFmpeg, nasm outputs aout, which is also converted to omf at link time.
We can also convert GCC object files at build time, which would also allow nasm to output omf or obj as nasm calls it.
Obj also needs the FLAT and USE32 parameters to create 32 bit code as it was originally a 16 bit format.

Problem is testing as I don't have AVX hardware.

Well, should that not be:
%macro SECTION_RODATA 0-1 32

(I know little about the nasm syntax).

I think we are ALREADY using OMF and not AOUT format. At least that's what the OS/2 port of nasm should do (and I used nasm myself and linked to code produced by the Watcom and VAC compilers).

Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 08, 2019, 02:44:05 am
Well, as I said, GCC actually puts out aout. Currently the nasm output is getting converted at link time to OMF as fewer changes were needed, so more likely to be accepted upstream, as well as static libs that could be used in more places. Plus at the time I was having problems with the right syntax. I've continued to learn.
Anyways, I'll change configure and x86inc.asm. I think the correct change is to something like
Code: [Select]
section .TEXT32 align=%1 use32 class=CODE
group CGROUP TEXT32
Have to look at the map files to make sure we don't end up with too many segments as well.
edit, used to be part of a kludge for macho (Mac) which seems to have gone away. x86inc.asm is actually from X264, who refused to support OS/2. I should double check on KOMH's latest patches.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 08, 2019, 05:46:00 am
OK, one problem with going with CFLAGS=-Zomf (needed if nasm emits obj format) is configure uses nm https://en.wikipedia.org/wiki/Nm_%28Unix%29 (https://en.wikipedia.org/wiki/Nm_%28Unix%29), https://sourceware.org/binutils/docs/binutils/nm.html (https://sourceware.org/binutils/docs/binutils/nm.html) in various places including in choosing how to mangle symbols (whether to add an underline prefix).
The configure snippet is
Code: [Select]
check_cc <<EOF || die "Symbol mangling check failed."
int ff_extern;
EOF
sym=$($nm $TMPO | awk '/ff_extern/{ print substr($0, match($0, /[^ \t]*ff_extern/)) }')
extern_prefix=${sym%%ff_extern*}
Ideas?
Title: Re: OS/4 (technical details only)
Post by: Lars on March 08, 2019, 06:49:38 am
1) I used
SEGMENT _TEXT CLASS=CODE PUBLIC USE32 ALIGN=% FLAT
for the segment definition (SEGMENT is equivalent to SECTION). When linking with either the VAC or the WATCOM linker it appears that those linkers will only honour (or prefer) the CLASS on combining segments. The actual segment name was irrelevant. In both cases the segment was combined with the default code segments that the VAC and WATCOM compilers generated.

2) what error are you getting on running nm ? If it is about a missing underscore or excessive underscore, maybe GCC has a switch to control generation of the underscore before a symbol or maybe it has a switch to choose the default calling convention (which will impact the generated symbol names). Fortunately, CDECL and SYSCALL seem to be interchangeable enough to use them either way (and only have an impact on the generated symbol name).
I hope there is some switch documentation for the OS/2 port of GCC ...
Title: Re: OS/4 (technical details only)
Post by: Lars on March 08, 2019, 06:52:43 am
Found this GCC switch

-fleading-underscore
-fno-leading-underscore

https://gcc.gnu.org/onlinedocs/gcc/Option-Summary.html

and NASM similar:
--prefix
--postfix

NM has switches to control the mangling:
--demangle[=STYLE]
--no-demangle

Maybe taking all this together helps solving the problem.


Title: Re: OS/4 (technical details only)
Post by: OS4User on March 08, 2019, 07:48:41 am
Well, as I said, GCC actually puts out aout. Currently the nasm output is getting converted at link time to OMF as fewer changes were needed

Dave, AFAIK  OMF does not support 32 bit alignment.

For test purpose you can replace:

Code: [Select]
SECTION_RODATA 32     in   fft.asm line 54
with
Code: [Select]
SECTION .rdata align=32 public use32 FLAT class=DATA

Try to get from nasm   coff or win32  format,  but I am not sure if linker will accept this.
(What linker do you use ?)   -   may be wl  will be required.

Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 08, 2019, 07:53:46 am
Hi Lars,
Objconv should work as an nm, https://www.agner.org/optimize/#objconv (https://www.agner.org/optimize/#objconv) also has a good disassembler that can output in nasm format and supports AVX. Compiles and works fine with
Code: [Select]
g++ -o objconv -O2 -Zomf *cpp
Don't really like using something that needs extra install step but fine for experimenting.
Can use ATTRIBUTE to switch between cdecl and stdcall but without the underlines, Firefox would probably not be able to use the DLLs.
The error with nm was of course, "unrecognized file format" or such. No OMF support.
It's bedtime here, so after work tomorrow I'll see if I can get nasm to output 32 bit flat obj format code. Ideally with patches that will be accepted upstream and doesn't screw up other projects that use FFmpeg.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 08, 2019, 07:58:08 am
Well, as I said, GCC actually puts out aout. Currently the nasm output is getting converted at link time to OMF as fewer changes were needed

Dave, AFAIK  OMF does not support 32 bit alignment.

For test purpose you can replace:

Code: [Select]
SECTION_RODATA 32     in   fft.asm line 54
with
Code: [Select]
SECTION .rdata align=32 public use32 FLAT class=DATA

Try to get from nasm   coff or win32  format,  but I am not sure if linker will accept this.
(What linker do you use ?)   -   may be wl  will be required.

Interesting, I'm using wl which may accept coff along with omf (documentation says it will, but whether on OS/2?). Still have the problem of mixing GCC and NASM code so need CPPFLAGS=-Zomf. ideally need to support static libs as well.
More tomorrow.
Title: Re: OS/4 (technical details only)
Post by: Lars on March 08, 2019, 08:08:05 am
1) there is no problem in telling nasm to align a segment/section to a 16-byte or 32-byte boundary using the SECTION / SEGMENT directive. Also if it outputs OMF format, I am using that already. But of course you need to take care that all variables but the first (which is aligned to the requested alignment) in the segment are also aligned to 16-byte / 32-byte. And you have to do that by hand for all subsequent variables, for example by injecting zero bytes ...
nasm also supports the ALIGN directive in the code but then you would precede every variable with that directive ...

2) yes, nm.exe cannot handle OMF format. I just found that out ...

3) yes, you definitely can tell nasm to output 32-bit flat OMF format (FLAT is the keyword to use in the segment declaration). That's what I have been doing. And it linked perfectly well with VAC and WATCOM compilers' generated code. Since GCC also accepts OMF input, it should also link with GCC (or the linkers that it calls).

4) as OS4user suggested, you might also want to move the SECTION_RODATA to the data segment. Then it is not "read only" anymore but who cares ...

5) I guess you meant to say "syscall" instead of "stdcall". GCC for OS/2 also understands the _System keyword (instead of specifying via "attribute") for the syscall calling convention.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 08, 2019, 08:50:34 am
Dave

I have done some testing for -f flag for nasm command line:

1) coff: does not support align=:
"test1.nasm:1: error: standard COFF does not support section alignment specification"

2) win32:  seams does not support class=  and my test var comes  to code seg
but align=32 works ok

3) aout:  does not support align=  and other directives

4) obj:  what is most surprising OBJ  supports align=256 !!! and var comes to proper data seg

wlink and wl work with 1) 2) 4)

Variant 4 for me most preferable.

So macro should be like this:

Code: [Select]
; aout does not support align=
; NOTE: This section is out of sync with x264, in order to
; keep supporting OS/2.
%macro SECTION_RODATA 0-1 16
    %ifidn __OUTPUT_FORMAT__,aout
        SECTION .text
    %elifidn __OUTPUT_FORMAT__,coff
        SECTION .text align=%1
    %elifidn __OUTPUT_FORMAT__,obj
        SECTION .DATA32 align=%1 public use32 FLAT class=DATA
    %elifidn __OUTPUT_FORMAT__,win32
        SECTION .rdata align=%1
    %elif WIN64
        SECTION .rdata align=%1
    %else
        SECTION .rodata align=%1
    %endif
%endmacro

add to nasm command line:  -f obj  (instead of -f aout)

SECTION_RODATA 256    in   fft.asm line 54 ( if you do not like warning  "OBJ format does not support alignment of 32: rounding up to 256")

This way you will  get proper  fft.obj  which can be directly given to wl without any conversion.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 08, 2019, 06:00:16 pm
Using
Code: [Select]
%elif __OUTPUT_FORMAT__,obj
        SECTION .DATA32 align=%1 public use32 FLAT class=DATA

Build dies with
Code: [Select]
Error! E2021: size of segment text exceeds 64k by 808181 bytes
make: *** [libavcodec/avcode57.dll] Error 1

Will investigate this evening. Might have to replace all the text segments with a macro.
edit: There's 81 instances of SECTION .text just in libavcodec which I assume aren't getting the use32 FLAT directive.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 08, 2019, 06:32:17 pm
There's 81 instances of SECTION .text just in libavcodec which I assume aren't getting the use32 FLAT directive.

You may try the following modification:

Code: [Select]
; aout does not support align=
; NOTE: This section is out of sync with x264, in order to
; keep supporting OS/2.
%macro SECTION_RODATA 0-1 16
    %ifidn __OUTPUT_FORMAT__,aout
        SECTION .text
    %elifidn __OUTPUT_FORMAT__,coff
        SECTION .text
    %elifidn __OUTPUT_FORMAT__,obj
        if %1 = 16
            SECTION .rdata align=16  public use32 FLAT class=DATA
        %elif
            SECTION .rdata align=256 public use32 FLAT class=DATA
        %endif
    %elifidn __OUTPUT_FORMAT__,win32
        SECTION .rdata align=%1
    %elif WIN64
        SECTION .rdata align=%1
    %else
        SECTION .rodata align=%1
    %endif
%endmacro

after definition this macro in x86inc.asm  add:

Code: [Select]
%ifidn __OUTPUT_FORMAT__,obj
    SECTION .text align=4 public use32 FLAT class=CODE
%endif

AFAIK  no other changes  are necessary.

To check compilation and linking  add: 
Code: [Select]
global ps_cos16_1
before
Code: [Select]
ps_cos16_1: dd 1.0, M_COS_PI_1_8, M_SQRT1_2, M_COS_PI_3_8, 1.0, M_COS_PI_1_8, M_SQRT1_2, M_COS_PI_3_8
this way we could come to know if ps_cos16_1 is in proper segment with proper alignment

After compilation I would like to see avcode57.map too  :)
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 08, 2019, 10:25:51 pm
Doesn't like the
Code: [Select]
if %1 = 16 part,
Code: [Select]
src/libavfilter/x86/af_volume.asm:24: error: parser: instruction expected
src//libavutil/x86/x86inc.asm:92: ... from macro `SECTION_RODATA' defined here
make: *** [libavfilter/x86/af_volume.o] Error 1

Title: Re: OS/4 (technical details only)
Post by: OS4User on March 08, 2019, 10:33:56 pm
Doesn't like the
Code: [Select]
if %1 = 16 part,

try
Code: [Select]
   if %1 == 16
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 08, 2019, 11:43:59 pm
Same error.
Anyways there are bigger problems, still using 64kb segments, lots of missing symbols for some reason, perhaps lost when the code is bigger then the segment, inconsistent underline prefix (I just hacked the code to fix for us, changing an ifdef to ifndef). Too many segments.
Here's a map file.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 09, 2019, 12:05:21 am
Same error.

I am not very good in nasm macro  so for test  you can use  this variant:

Code: [Select]
%macro SECTION_RODATA 0-1 16
        SECTION .rdata align=256 public use32 FLAT class=DATA
%endmacro

hope you give to nasm  -f obj

Anyways there are bigger problems, still using 64kb segments

Did you add

Code: [Select]
%ifidn __OUTPUT_FORMAT__,obj
    SECTION .text align=4 public use32 FLAT class=CODE
%endif

as it was suggested ?

It looks like did not   - that is why you have:

"text                                  AUTO"

and "still using 64kb segments"

---------------

Error! E2028: ff_h264_cabac_tables is an undefined reference  but you have
0001:0010af60  _ff_h264_cabac_tables

it is another story  - may be suggestion from Lars will help https://www.os2world.com/forum/index.php/topic,1735.msg20825.html#msg20825
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 09, 2019, 03:29:30 am
Ok, with the text section added and configure hacked a bit to set the correct prefix (some inline assembler was missing the correct prefix causing the unresolved symbols) compilation finishes.
Still have too many segments though, TEXT32, text and DATA32.
Untested.
Patch for x86inc.asm and avcodec57.map attached
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 09, 2019, 07:41:48 am
Here's the disassembly of fft.asm in nasm format. If I understand it correctly, the constants are at the bottom of the file in the rodata32 section and aligned to 32 bytes.
It appears the align directive aligns the whole section.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 09, 2019, 07:45:37 am
Still have too many segments though, TEXT32, text and DATA32.

Wrong segments are:
text                   CODE           AUTO
rodata32               DATA           AUTO

These segments were generated from .asm  (by nasm)  -  I do not know how to tell to nasm that    text  has to be in CGROUP  and  rodata32 has to be in DGROUP

We can try to do this way:

after
Code: [Select]
SECTION .rodata32 align=256 public use32 FLAT class=DATA
add
Code: [Select]
group DGROUP rodata32

after
Code: [Select]
SECTION .text align=4 public use32 FLAT class=CODE
add
Code: [Select]
group CGROUP text

by the way, if we are going to change macro SECTION_RODATA  what  __OUTPUT_FORMAT__  it should  be ready for ?
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 09, 2019, 07:53:29 am
Here's the disassembly of fft.asm in nasm format. If I understand it correctly, the constants are at the bottom of the file in the rodata32 section and aligned to 32 bytes.
It appears the align directive aligns the whole section.

Better to see in  avcode57.map   at  0004:00000700  ps_cos16_1   - it is pretty good aligned 256 as expected

(I sent you private message - pls see)

Upd:

It  looks  like there is  more straight way  -  select  win32 format for nasm output:  -f win32
No changes  to sources  will be necessary.
Title: Re: OS/4 (technical details only)
Post by: Lars on March 09, 2019, 10:59:46 pm
Same error.
Anyways there are bigger problems, still using 64kb segments, lots of missing symbols for some reason, perhaps lost when the code is bigger then the segment, inconsistent underline prefix (I just hacked the code to fix for us, changing an ifdef to ifndef). Too many segments.
Here's a map file.
You Need to define all segments as public. Only then will they combine.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 10, 2019, 04:28:14 am
It  looks  like there is  more straight way  -  select  win32 format for nasm output:  -f win32
No changes  to sources  will be necessary.

How to get the symbols to put in the def file? The assembler and compiler object files are mixed together. Currently use emxexp, which pukes on the win32 object files.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 10, 2019, 04:31:44 am
You Need to define all segments as public. Only then will they combine.

They are declared as public. The weird thing is the various types of text segments. From the map file,
Code: [Select]
Segment                Class          Group          Address         Size
=======                =====          =====          =======         ====

TEXT32                 CODE           CGROUP         0001:00000000   009fe590
________TEXT           CODE           CGROUP         0001:009fe590   00000000
text                   CODE           AUTO           0002:00000000   000ca2cb
DATA32                 DATA           DGROUP         0003:00000000   0001d990

Aout gives,
Code: [Select]
TEXT32                 CODE           CGROUP         0001:00000000   00bd1560
________TEXT           CODE           CGROUP         0001:00bd1560   00000000
DATA32                 DATA           DGROUP         0002:00000000   00015a20

Note that the DGROUP ends up in segment 0003.
The other question is, does it matter?
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 10, 2019, 08:01:07 am
It  looks  like there is  more straight way  -  select  win32 format for nasm output:  -f win32
No changes  to sources  will be necessary.

How to get the symbols to put in the def file? The assembler and compiler object files are mixed together. Currently use emxexp, which pukes on the win32 object files.

What .def ?  Do you need it for some purpose ?

I would like to see avcode57.map  if you give to nasm  -f win32 with this definition of macro:
Code: [Select]
%macro SECTION_RODATA 0-1 16
        SECTION .rodata align=%1
%endmacro

I did not understand if you try suggestion from https://www.os2world.com/forum/index.php/topic,1735.msg20846.html#msg20846  -  pls clarify
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 10, 2019, 09:46:09 am
It  looks  like there is  more straight way  -  select  win32 format for nasm output:  -f win32
No changes  to sources  will be necessary.

How to get the symbols to put in the def file? The assembler and compiler object files are mixed together. Currently use emxexp, which pukes on the win32 object files.

What .def ?  Do you need it for some purpose ?{/quote]
Need a def file to link a DLL
Quote
I would like to see avcode57.map  if you give to nasm  -f win32 with this definition of macro:
Code: [Select]
%macro SECTION_RODATA 0-1 16
        SECTION .rodata align=%1
%endmacro
No map file as the build dies creating the def file
Quote
I did not understand if you try suggestion from https://www.os2world.com/forum/index.php/topic,1735.msg20846.html#msg20846  -  pls clarify

Code: [Select]
YASM    libavcodec/x86/fmtconvert.o
src//libavutil/x86/x86inc.asm:99: warning: segment `text' is already part of a group: first one takes precedence
YASM    libavcodec/x86/flacdsp.o
src//libavutil/x86/x86inc.asm:99: warning: segment `text' is already part of a group: first one takes precedence
src/libavcodec/x86/dirac_dwt.asm:24: warning: segment `rdata32' is already part
of a group: first one takes precedence
src//libavutil/x86/x86inc.asm:91: ... from macro `SECTION_RODATA' defined here
Title: Re: OS/4 (technical details only)
Post by: Lars on March 10, 2019, 10:09:32 am
You Need to define all segments as public. Only then will they combine.

They are declared as public. The weird thing is the various types of text segments. From the map file,
Code: [Select]
Segment                Class          Group          Address         Size
=======                =====          =====          =======         ====

TEXT32                 CODE           CGROUP         0001:00000000   009fe590
________TEXT           CODE           CGROUP         0001:009fe590   00000000
text                   CODE           AUTO           0002:00000000   000ca2cb
DATA32                 DATA           DGROUP         0003:00000000   0001d990

Aout gives,
Code: [Select]
TEXT32                 CODE           CGROUP         0001:00000000   00bd1560
________TEXT           CODE           CGROUP         0001:00bd1560   00000000
DATA32                 DATA           DGROUP         0002:00000000   00015a20

Note that the DGROUP ends up in segment 0003.
The other question is, does it matter?

It does not seem to matter. At least the AVCODE65 that I use has only 2 segments: a CODE segment and a DATA segment.
You can check with THESEUS: System->Modules->AVCODE65->double click one MTE @
The AVCODE65 that I used was installed via RPM from the Netlabs repo.

But you can also enforce that differents segments go into the same group:
CGROUP GROUP ________TEXT,TEXT32,text
put that in an assembler file that is linked, preferably the one for the first obejct file to be linked.
You might get warnings like the one you posted but if it is the FIRST object file to be linked it will take precedence over everything that follows :-)

By the way: I very much doubt that our OS/2 tools can do anything else but the OMF and the AOUT format. I used nasm without the -f switch and that worked right out of the box. Which means that in this case it produces OMF format.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 10, 2019, 11:01:55 am
I'll test tomorrow, it's bedtime here.
Wlink claims it can link win32 and OMF object code. Whether it'll work for OS/2 binaries remains to be seen.
Xfree86 actually used ELF code and included an ELF loader to use Linux modules. Needed GCC-ELF etc to build it.
Did NASM produce 32 bit OMF without -f? According to the docs, it will use the native object format but I'd expect 16 bit.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 10, 2019, 11:20:22 am
Code: [Select]
Segment                Class          Group          Address         Size
=======                =====          =====          =======         ====

TEXT32                 CODE           CGROUP         0001:00000000   009fe590
________TEXT           CODE           CGROUP         0001:009fe590   00000000
text                   CODE           AUTO           0002:00000000   000ca2cb
DATA32                 DATA           DGROUP         0003:00000000   0001d990

try this addition  to  x86inc.asm:
Code: [Select]
; NOTE: This section is out of sync with x264, in order to keep supporting OS/2
%ifidn __OUTPUT_FORMAT__,obj
    %define .text TEXT32
    SECTION .text align=4 public use32 FLAT class=CODE
%endif
it should avoid altering 118 .asm files and remove "text                   CODE           AUTO"
Better to keep this after  definition of SECTION_RODATA macro
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 11, 2019, 01:51:45 am
try this addition  to  x86inc.asm:
Code: [Select]
; NOTE: This section is out of sync with x264, in order to keep supporting OS/2
%ifidn __OUTPUT_FORMAT__,obj
    %define .text TEXT32
    SECTION .text align=4 public use32 FLAT class=CODE
%endif
it should avoid altering 118 .asm files and remove "text                   CODE           AUTO"
Better to keep this after  definition of SECTION_RODATA macro

Ok, that was what was needed. New zip uploaded to Bitbucket, replacing old upload. Map files in zip. David, please test.
I'll attach the latest diffs. Still have to solve the mangle prefix, perhaps by adding an option to configure.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 11, 2019, 02:27:36 am
Dave, I am still getting traps.  This is with the FF sandybridge, OS/4 kernel SVN5452 and your latest libs
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 11, 2019, 07:07:13 am
Dave, I am still getting traps.  This is with the FF sandybridge, OS/4 kernel SVN5452 and your latest libs

Dave,

It is because of wrong alignment  of   _ff_cos_32:
Code: [Select]
vmovaps ymm0, yword [_ff_cos_32]
fft.asm      Line 182
Title: Re: OS/4 (technical details only)
Post by: Lars on March 11, 2019, 07:36:10 am
Only the FIRST variable in the section is aligned to the desired alignment.The others need to be aligned manually.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 11, 2019, 12:18:41 pm
Only the FIRST variable in the section is aligned to the desired alignment.The others need to be aligned manually.

Lars,

ff_cos_32  defined in C,  what is defined in asm  right now is aligned ok

Dave,

 What version of GCC do you use ?
Title: Re: OS/4 (technical details only)
Post by: Lars on March 11, 2019, 03:40:58 pm
Only the FIRST variable in the section is aligned to the desired alignment.The others need to be aligned manually.

Lars,

ff_cos_32  defined in C,  what is defined in asm  right now is aligned ok

Really ? I could see various variables that were only aligned to 16 bytes (typically it was a sequence of 4 * 4 bytes). If these are only accessed by SSE code then everything is fine but if you pass these to AVX code then you will experience a trap.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 11, 2019, 03:57:47 pm
Really ? I could see various variables that were only aligned to 16 bytes (typically it was a sequence of 4 * 4 bytes). If these are only accessed by SSE code then everything is fine but if you pass these to AVX code then you will experience a trap.

Interesting to see what do you mean ? (var names)
Vars in asm files are defined  in 3 ways:
SECTION_RODATA        (equ align 16)
SECTION_RODATA 32
SECTION_RODATA 64
so, not all of them have to be aligned 32 or 64.
However,  now  all asm sections are aligned 256.
Title: Re: OS/4 (technical details only)
Post by: Lars on March 11, 2019, 10:15:02 pm
No. The FIRST variable in a section is aligned. But not necessarily the ones that follow.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 11, 2019, 10:35:30 pm
No. The FIRST variable in a section is aligned. But not necessarily the ones that follow.

True.  But  authors of the code wrote it  with this idea in mind  -  if it was necessary they might have added a direct align 32 directive.

Let us see fft.asm  -  first var  ps_cos16_1 is aligned  because of section alignment, second var ps_cos16_2 also aligned 32  because of the size of ps_cos16_1 is 32 bit  and so on and so on.

Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 11, 2019, 11:12:37 pm
The whole thing is frustrating. This code has been around and works fine on other platforms, so it has to be problem with our platform, rather nasm or GCC. BTW, I'm currently using GCC 5.5.0 as Paul did some more alignment fixes compared to 4.9.2. I've also switched to building Mozilla for the same reason.
One possibility I guess is that GCC is screwing up with -O3.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 11, 2019, 11:23:46 pm
One possibility I guess is that GCC is screwing up with -O3.

We'll try a version built with -O2. David could you test the build with -O2 in the zip name? Thanks
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 11, 2019, 11:53:19 pm
One possibility I guess is that GCC is screwing up with -O3.

We'll try a version built with -O2. David could you test the build with -O2 in the zip name? Thanks
Dave

I tried it and it still traps. 

Right now the ArcaOS kernel does not appear to support AVX, so there is no crash when watching video  The OS/4 standard kernel does support AVX and that is when the problem appears.  The noAVX version of the OS/4 kernel does not crash when playing videos either.

Here are the new trap messages.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 12, 2019, 01:00:33 am
OK, this is frustrating, especially with my not having AVX hardware. For now just disabling AVX seems the simplest.
Thanks for testing
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 12, 2019, 06:48:16 am
I tried it and it still traps. 

Dave

It is again  wrongly aligned   ff_cos_32 (fft_template.c  Line 127):
Code: [Select]
vmovaps ymm0, yword [_ff_cos_32]
fft.asm      Line 182

Problem is definitely in GCC  -   no questions to nasm so far.

You can try  to change compilation options to GCC  and check alignment of  _ff_cos_32  in map file to understand if  it is aligned 32.


Upd:

What is object file format  when you compile  .c files  by GCC ?   
Title: Re: OS/4 (technical details only)
Post by: Paul Smedley on March 12, 2019, 09:59:59 am
Hey Dave,

OK, this is frustrating, especially with my not having AVX hardware. For now just disabling AVX seems the simplest.
Thanks for testing

http://www.voidcn.com/article/p-vtipwtsk-cu.html might be worth a read....
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 12, 2019, 05:18:51 pm
Hey Dave,

OK, this is frustrating, especially with my not having AVX hardware. For now just disabling AVX seems the simplest.
Thanks for testing

http://www.voidcn.com/article/p-vtipwtsk-cu.html might be worth a read....

Hi Dave,

Pls add -fno-common  to gcc command line  and show   avcode57.map
On simple test  this option shows good result  -  all vars in obj aligned ok.
Alignment between  obj files  is still a question mark.
 
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 12, 2019, 06:05:51 pm
OK, -fno-common seems to have fixed the alignment, from the map file,
Code: [Select]
Module: K:\work\ffmpeg\src\ffmpeg.obj\libavcodec\fft_fixed.o(src\libavcodec\fft_fixed.c)
0001:001fbc83  _ff_init_ff_cos_tabs_fixed
0001:001fc050  _ff_cos_tabs_fixed
0001:001fbd3c  _ff_fft_init_fixed
0001:001fbf90  _ff_fft_end_fixed
0002:001cc900  _ff_cos_131072_fixed
0002:0020c8e0  _ff_cos_16_fixed
0002:0020c8c0  _ff_cos_32_fixed
0002:0020c880  _ff_cos_64_fixed
0002:0020c800  _ff_cos_128_fixed
0002:0020c700  _ff_cos_256_fixed
0002:0020c500  _ff_cos_512_fixed
0002:0020c100  _ff_cos_1024_fixed
0002:0020b900  _ff_cos_2048_fixed
0002:0020a900  _ff_cos_4096_fixed
0002:00208900  _ff_cos_8192_fixed
0002:00204900  _ff_cos_16384_fixed
0002:001fc900  _ff_cos_32768_fixed
0002:001ec900  _ff_cos_65536_fixed
...
Code: [Select]
Module: K:\work\ffmpeg\src\ffmpeg.obj\libavcodec\fft_float.o(src\libavcodec\fft_float.c)
0001:002114e0  _ff_init_ff_cos_tabs
0001:00211870  _ff_cos_tabs
0001:00211561  _ff_fft_init
0001:002117bc  _ff_fft_end
0002:0020c900  _ff_cos_131072
0002:0028c8c0  _ff_cos_16
0002:0028c880  _ff_cos_32
0002:0028c800  _ff_cos_64
...

Edit, BTW, GCC outputs aout
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 12, 2019, 06:27:48 pm
New lib package uploaded. It also contains the map files. David, can you test? Thanks
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 12, 2019, 08:58:51 pm
New lib package uploaded. It also contains the map files. David, can you test? Thanks

Success!!!!!!!!!! So far no crashes at the problem sites.  I will continue to test, but so far everything looks good. 
Title: Re: OS/4 (technical details only)
Post by: Paul Smedley on March 13, 2019, 02:57:58 am
New lib package uploaded. It also contains the map files. David, can you test? Thanks

Success!!!!!!!!!! So far no crashes at the problem sites.  I will continue to test, but so far everything looks good.

Cool, this might help me find how to fix this more correctly within GCC. From what I've read, it was fixed for mingw back in GCC 4.4.0 timeframe - so I should hopefully be able to track down what they changed.

Cheers,

Paul
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 13, 2019, 05:47:24 am
I ran the test suite (FATE) and seemed to have the same errors as previously, though that was with static libs. I do note that the article says this,
Quote
but it can not work in the sharing mode of FFMPEG in the current official release.
Though compiling with --enable-shared worked fine here.

I've also created a ticket at netlabs, http://trac.netlabs.org/ports/ticket/206 (http://trac.netlabs.org/ports/ticket/206)
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 14, 2019, 06:12:14 pm
I ran the test suite (FATE) and seemed to have the same errors as previously, though that was with static libs. I do note that the article says this,
Quote
but it can not work in the sharing mode of FFMPEG in the current official release.
Though compiling with --enable-shared worked fine here.

I've also created a ticket at netlabs, http://trac.netlabs.org/ports/ticket/206 (http://trac.netlabs.org/ports/ticket/206)

Dave, I have been using now for a few days.  Not one crash.  Working great.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 14, 2019, 06:26:41 pm
Dave, I have been using now for a few days.  Not one crash.  Working great.

David,

It is because you have a good karma  :)   We cannot expect  always  positive result with current  GCC.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 14, 2019, 07:55:46 pm
I think it is limitations of the aout format. At one point there were plans to switch GCC to outputting ELF and converting that to OMF. Unluckily I can't see that ever happening with current manpower.
Another option would be switching to Win32 output, probably be a lot easier, especially if wlink can take win32 object files and output OS/2 executables, but even that'll likely never happen.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 14, 2019, 08:05:06 pm
I think it is limitations of the aout format. At one point there were plans to switch GCC to outputting ELF and converting that to OMF

AFAIK no need to convert ELF  to OMF    -  wl can link ELF to  LX
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 14, 2019, 08:20:00 pm
Dave

could you compile to asm  using gcc  5.x.x  http://gus.biysk.ru/os4/test_align.c

1) without -fno-common
2) with  -fno-common

And show results as attachment ?
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 14, 2019, 08:29:49 pm
I think it is limitations of the aout format. At one point there were plans to switch GCC to outputting ELF and converting that to OMF

AFAIK no need to convert ELF  to OMF    -  wl can link ELF to  LX

I've wondered at that possibility. Still would be a lot more to overcome compared to Win32 such as shared libs vs DLLs.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 14, 2019, 08:53:53 pm
Dave, I have been using now for a few days.  Not one crash.  Working great.

David,

It is because you have a good karma  :)   We cannot expect  always  positive result with current  GCC.

I  did the ultimate challenge.  My wife has a facebook page.  She needed me to check something for her.  I logged on and went to her page.  Although the sandybrigde browser did not trap, it slowed down to a crawl making the pages almost unusable.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 14, 2019, 09:01:39 pm
Dave, I have been using now for a few days.  Not one crash.  Working great.

David,

It is because you have a good karma  :)   We cannot expect  always  positive result with current  GCC.

I  did the ultimate challenge.  My wife has a facebook page.  She needed me to check something for her.  I logged on and went to her page.  Although the sandybrigde browser did not trap, it slowed down to a crawl making the pages almost unusable.

Pls do some testing:

1)  reg OS/4 kernel,  P4 FF, latest FFMPEG from Dave
2) nonAVX OS/4 kernel, P4 FF, latest FFMPEG from Dave

And share you opinion about speed  and CPU load on video, especially the sites  where  you got traps.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 14, 2019, 09:32:43 pm
I  did the ultimate challenge.  My wife has a facebook page.  She needed me to check something for her.  I logged on and went to her page.  Although the sandybrigde browser did not trap, it slowed down to a crawl making the pages almost unusable.

Yea, our JavaScript engine has speed issues, which is probably the issue with Facebook and quite a few other sites. Generally if you wait, things recover but it depends.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 14, 2019, 09:58:58 pm
Dave

could you compile to asm  using gcc  5.x.x  http://gus.biysk.ru/os4/test_align.c

1) without -fno-common
2) with  -fno-common

And show results as attachment ?

Missed this at first.
Happened to be trying to create an elf object with [H:\tmp\test]gcc_elf --version
3.0.3, which along with regular GCC 3.0.3 gives this output
Code: [Select]
[H:\tmp\test]gcc_elf -c align.c -o align.o
align.c:17:1: warning: no newline at end of file
align.c:8: warning: alignment of `Aligned32_Var1' is greater than maximum object
 file alignment. Using 16.
align.c:10: warning: alignment of `Aligned32_Var2' is greater than maximum object file alignment. Using 16.
align.c:12: warning: alignment of `Aligned32_Var3' is greater than maximum object file alignment. Using 16.

Which I guess is the basic problem.


Title: Re: OS/4 (technical details only)
Post by: David Graser on March 14, 2019, 10:09:40 pm
I  did the ultimate challenge.  My wife has a facebook page.  She needed me to check something for her.  I logged on and went to her page.  Although the sandybrigde browser did not trap, it slowed down to a crawl making the pages almost unusable.

Yea, our JavaScript engine has speed issues, which is probably the issue with Facebook and quite a few other sites. Generally if you wait, things recover but it depends.

What is weird is that this was the first time I had ever accessed my wife's facebook page with my laptop.  It was extremely slow.  However, since that first time, access time has sped up.  It is almost like it is gathering info about my system the very first time.

OS/2 kernel using FFPentium
OS/4 kernel using FFPentium
OS/4 kernel no AVX using FFPentium

The speed results were around the same.  Everything appears to load in blocks.

The OS/4 kernel with the FFSandybridge loads in blocks also.  Probably the big difference between it and the above is that the Facebook search engine was much slower bringing up the search result.  Tested this several times.
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 14, 2019, 10:28:21 pm
Happened to be trying to create an elf object with [H:\tmp\test]gcc_elf --version
3.0.3, which along with regular GCC 3.0.3 gives this output
Code: [Select]
[H:\tmp\test]gcc_elf -c align.c -o align.o
align.c:17:1: warning: no newline at end of file
align.c:8: warning: alignment of `Aligned32_Var1' is greater than maximum object
 file alignment. Using 16.
align.c:10: warning: alignment of `Aligned32_Var2' is greater than maximum object file alignment. Using 16.
align.c:12: warning: alignment of `Aligned32_Var3' is greater than maximum object file alignment. Using 16.

Which I guess is the basic problem.

Dave,

Tnx for .asm files.  One without key  does not look ok
GCC_elf 3.0.3  could be  to old to generate  proper alignment.

David,

We need sites with video but without  JavaScript  to understand the difference.
Pls pay attention to CPUs load.

may be  http://www.quirksmode.org/html5/videos/big_buck_bunny.mp4
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 14, 2019, 11:41:29 pm
OS2User

I found one problem which although did not trap FFsandybridge.  Not sure when I did it.   it did have an impact on Facebook search.  I was loading the FFturbo for the FF pentium.  With the correct FFturbo running, FFSandybrige is is handling the Facebook search snappy now.  Also, although loading is still blocky, it is loading faster.  It appears that it may be even faster than the noAVX version of kernels and browsers.  On the video sites I go to, very seldom does any of the four CPUs reach 50%  The 4 CPUs are almost drawing straight lines while playing video.

You want to know the worst site.  This site as I write this text.  the CPUs are jumping from 0 to 100% steadily.

The second video shows the CPU video playing a youtube video.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 15, 2019, 12:17:57 am
Quote
You want to know the worst site.  This site as I write this text.  the CPUs are jumping from 0 to 100% steadily.

Strange, it uses almost no CPU here, perhaps as I have scripting turned off. I'll turn it on.
OK, that increased CPU usage. A couple of points, some animated gifs can cause a lot of CPU usage, some bug in our port I believe, they show up here if I turn on JavaScript. Typing doesn't seem to make it much worse.
I also notice heavy CPU usage on the main forum page if there are any new comments, this goes away if no new comments. Once again likely GIF related.
We had a similar problem at Mantis, after I tracked it down Lewis changed all the GIFs to PNGs, which fixed it.
Edit, I'm using SeaMonkey, which isn't much different code wise as most code is shared. There is some differences such as Chatzilla, which causes high CPU usage always if enabled, whether actually being used or not. Same XPI is fine on Firefox and it is the same as was in the 38ESR tree.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 15, 2019, 12:52:12 am
The ArcaOS kernel appears to be more CPU intensive while using FFPentium.  Attached is a pic.

On the Youtube pages, it appears the OS/4 kernel with the FFSandybridge does a somewhat better job with all 4 CPUs. 

I think the OS/4 kernel with the FFSandybridge is a good choice for using the internet.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 15, 2019, 07:43:19 am
The OS4 kernel with the FFsandybridge and AVX libs is the way to go compared to the OS/2 kernel with the FFPentium.  Trunews is the most demanding video site I have encountered so far.  The last 1/3 portion of the first pic is the OS/4 kernel being used for this web site.  With the OS/2 kernel, the longer the video ran, the slower and more choppy the video became with the CPU oscillation getting bigger.

https://www.trunews.com/stream/harvard-entrance-scandal-did-jared-kushner-receive-special-treatment

Attached are the 2 CPU comparisons.

Also note that the IRQ red line is being more utilized in the OS/4 kernel than the OS/2 kernel.

The third pic is OS/4 noAVX kernel.  Not much difference between it and the regulator OS/4 kernel except it appears to have a higher IRQ load (red line) and a little less CPU.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 17, 2019, 12:40:35 pm
This is with the OS/4 kernel, FFSandyBridge, and AVX libs.

While playing video, the CPU usage oscillates little.  However, when I stop play, the CPU starts ocsilatting all over the place.  Start play again oscillates little.

Attached is a pic. The high swings occurs when the video is stopped.  Need to check with the OS/2 kernel next.

The OS/2 kernel does the same thing as seen in the second pic.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 18, 2019, 01:54:06 am
One more observation about FF.  When I minimizze FF, the CPUs goes crazy.  The low usage is the OS2World home page.  When minimized, the CPU usuage goes crazy.  I have not tested with the OS/2 kernel, but I am sure it is the same way.  Pic attached.

Note:  While testing this, I had 2 traps.  The first I have had in days.  Attached the two traps.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 18, 2019, 02:51:39 am
You need to install https://bitbucket.org/dryeo/mozilla-os2/downloads/firefox-sandybridge-xqs.zip (https://bitbucket.org/dryeo/mozilla-os2/downloads/firefox-sandybridge-xqs.zip) as I forgot to package the xqs files.

The high CPU load when minimized is a known issue related to the theme somehow. Try installing the attached theme by dropping it on about:addons, enable it after restarting the browser, and then minimize.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 18, 2019, 03:33:12 am
You need to install https://bitbucket.org/dryeo/mozilla-os2/downloads/firefox-sandybridge-xqs.zip (https://bitbucket.org/dryeo/mozilla-os2/downloads/firefox-sandybridge-xqs.zip) as I forgot to package the xqs files.

The high CPU load when minimized is a known issue related to the theme somehow. Try installing the attached theme by dropping it on about:addons, enable it after restarting the browser, and then minimize.

Hi Dave,

That was it.  Switching themes corrected the minimize CPU usage.  Thanks.  Also installed the xqs package.
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 23, 2019, 04:08:10 pm
You need to install https://bitbucket.org/dryeo/mozilla-os2/downloads/firefox-sandybridge-xqs.zip (https://bitbucket.org/dryeo/mozilla-os2/downloads/firefox-sandybridge-xqs.zip) as I forgot to package the xqs files.

The high CPU load when minimized is a known issue related to the theme somehow. Try installing the attached theme by dropping it on about:addons, enable it after restarting the browser, and then minimize.

Hi Dave,

That was it.  Switching themes corrected the minimize CPU usage.  Thanks.  Also installed the xqs package.

Just an update.  Everything is working great with the latest OS/4 test kernels, the FFSandbridge, the AVX libs,  the xqs files installed, and the lite theme installed.  Not one trap in over a week.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on March 23, 2019, 04:46:44 pm
Of interest, a patch was posted for pixman by Intel to use AVX2. Pixman takes care of the drawing primitives for Cairo and Cairo handles the browsers canvas (basically all the drawing excepting things like the frame). Hasn't been accepted as the maintainers want to see benchmarks for AMD but gives some idea of possible speed improvements.
Code: [Select]
Performance benefits benchmarked on a Intel(R) Core(TM) i9-7900 CPU @
3.30GHz. These patches improve performance of both high level and low level
benchmarks.

|-----------------------+----------+------------+----------+------------+----------|
| cairo-perf-trace      | AVX2 Avg | AVX2 stdev | SSE2 Avg | SSE2 stdev | % change |
|-----------------------+----------+------------+----------+------------+----------|
| poppler               | 1.125s   | 0.30%      | 1.284s   | 0.19%      | +14.13%  |
| firefox-canvas-scroll | 2.503s   | 0.21%      | 2.853s   | 0.22%      | +13.98%  |
|-----------------------+----------+------------+----------+------------+----------|

|--------------------+---------+---------+----------|
| lowlevel-blt-bench | AVX2    | SSE2    | % change |
|--------------------+---------+---------+----------|
| OVER_8888_8888 L1  | 2118.06 | 1250.50 | +69.37%  |
| OVER_8888_8888 L2  | 1967.59 | 1245.87 | +57.90%  |
| OVER_8888_8888 M   | 1694.10 | 1183.65 | +43.12%  |
| OVER_8888_8888 HT  | 562.82  | 556.45  | +01.11%  |
| OVER_8888_8888 VT  | 411.19  | 349.78  | +17.56%  |
| OVER_8888_8888 R   | 369.46  | 332.09  | +11.25%  |
| OVER_8888_8888 RT  | 172.36  | 151.37  | +13.86%  |
|--------------------+---------+---------+----------|
Title: Re: OS/4 (technical details only)
Post by: OS4User on March 23, 2019, 09:41:13 pm
Of interest, a patch was posted for pixman by Intel to use AVX2

OS/4 does not support AVX2
Title: Re: OS/4 (technical details only)
Post by: David Graser on March 31, 2019, 12:25:45 am
Pulled out an older laptop that has AVX to check the OS/4 kernel, the FFSandyBridge, and the AVX libs on it and no FF traps.  The AVX libs are working great.
Title: Re: OS/4 (technical details only)
Post by: Moritz on April 01, 2019, 04:06:49 pm
Pulled out an older laptop that has AVX to check the OS/4 kernel, the FFSandyBridge, and the AVX libs on it and no FF traps.  The AVX libs are working great.

Hello David,

Are you using the AN Acpi or the OS2Apic shipped with the OS4 Kernel?

Thanks

M

Title: Re: OS/4 (technical details only)
Post by: David Graser on April 04, 2019, 01:24:57 am
Pulled out an older laptop that has AVX to check the OS/4 kernel, the FFSandyBridge, and the AVX libs on it and no FF traps.  The AVX libs are working great.

Hello David,

Are you using the AN Acpi or the OS2Apic shipped with the OS4 Kernel?

Thanks

M

There is no patch presently for the ACPI included in ArcaOS 5.02 to use with the OS/4 kernel.  As a result, I have the statement remmed out in the conf.OS4.  The OS4 kernel supports AVX while the OS/2 kernel does not.
Title: Re: OS/4 (technical details only)
Post by: Lars on April 04, 2019, 11:58:17 am
For what it is worth:
if you ask the OS/4 guys, they are going to provide you with an ACPI4.BPD driver plus the OS4APIC.PSD driver that goes along with it. This combination is a full replacement for ACPI.PSD as provided by AN.
The "BPD" driver variant is an OS4 extension to the driver model (they are special, will load without being listed in CONFIG.SYS and load early in the boot process).

However, the ACPI4.BPD driver is beta.
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on April 04, 2019, 05:04:37 pm
BTW,  I'm moving the Firefox builds, including the AVX one, to https://bitbucket.org/dryeo/mozilla-os2/downloads/ (https://bitbucket.org/dryeo/mozilla-os2/downloads/)
Title: Re: OS/4 (technical details only)
Post by: Andi B. on April 05, 2019, 09:50:37 am
Dave do you have the matching SM builds too? Are the improvements you've made the last months relevant for SM too? I think I'm using your latest SM but it's some time ago since I updated. Maybe you'll provide a link to the best one here again to assure we SM users are using the latest greatest. Thanks..
Title: Re: OS/4 (technical details only)
Post by: Dariusz Piatkowski on April 05, 2019, 03:54:03 pm
Lars, everyone...

For what it is worth:
if you ask the OS/4 guys, they are going to provide you with an ACPI4.BPD driver plus the OS4APIC.PSD driver that goes along with it. This combination is a full replacement for ACPI.PSD as provided by AN...

...like you said, "for what its worth", it would make the most sense that by default the ACPI4.BPD driver is bundled with the regular OS4 package drops, does it not? I think most of us are probably looking to try out the equivalent level of functionality (and more) to what's currently officially available and supported.

Does anyone know what specifically is (and why) in the ACPI4.BPD as opposed to OS4APIC?
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on April 05, 2019, 04:41:04 pm
Dave do you have the matching SM builds too? Are the improvements you've made the last months relevant for SM too? I think I'm using your latest SM but it's some time ago since I updated. Maybe you'll provide a link to the best one here again to assure we SM users are using the latest greatest. Thanks..

Haven't had anyone ask for matching SM builds. I don't really know how much the extra SIMD instructions make, was hoping someone would benchmark. I did build an i686 version of FF for someone and in testing, I didn't notice much difference from the Pentium M build. (Newer ones don't work on this CPU)
The latest SM is https://bitbucket.org/dryeo/dry-comm-esr31/downloads/seamonkey-2.42.9esr_r3.en-US.os2-Pentium-M.zip (https://bitbucket.org/dryeo/dry-comm-esr31/downloads/seamonkey-2.42.9esr_r3.en-US.os2-Pentium-M.zip) which is basically what I'm using.
Title: Re: OS/4 (technical details only)
Post by: OS4User on April 06, 2019, 09:52:51 am
Does anyone know what specifically is (and why) in the ACPI4.BPD as opposed to OS4APIC?

ACPI4.BPD is a kind of BASEDEV  which is loading before any PSD. It contains Intel ACPICA lib and interface part  to be recognized as PDD by any other system components which need ACPI service.

OS4APIC.PSD  needs info about HW configuration. There are two sources for this info: MPtable and ACPI system.  OS4APIC.PSD is able to get info from MPtable itself, and on the contrary, to get info from ACPI it engages ACPI4.BPD.

It happens so that some mobos and notebooks  do not have valid MPtable and ACPI is the only way for OS4APIC.PSD to get the required info.

One more consumer for ACPI4.BPD service is CLOCK03c.SYS.  CLOCK03c uses ACPI  to get config info about HPET.


Title: Re: OS/4 (technical details only)
Post by: Lars on April 06, 2019, 02:06:09 pm
That still does not answer the question why ACPI4.BPD is not bundled with the OS4 kernel package. The kernel package only contains OS4APIC.PSD but not the BPD.
And by now, ACPI4.BPD cannot be found on the OS4 web page any more.
I do have ACPI4.BPD but I don't even remember where I got it from ...
Title: Re: OS/4 (technical details only)
Post by: Dariusz Piatkowski on April 08, 2019, 12:19:12 am
Hey OS4User,

ACPI4.BPD is a kind of BASEDEV  which is loading before any PSD. It contains Intel ACPICA lib and interface part to be recognized as PDD by any other system components which need ACPI service.

So is it a required part, or an extended functionality type of a thing?

...OS4APIC.PSD  needs info about HW configuration. There are two sources for this info: MPtable and ACPI system.  OS4APIC.PSD is able to get info from MPtable itself, and on the contrary, to get info from ACPI it engages ACPI4.BPD.

It happens so that some mobos and notebooks  do not have valid MPtable and ACPI is the only way for OS4APIC.PSD to get the required info.

Alright, so from that perspective it might explain why on my machine running OS4APIC with /APIC switch causes a HARD-HANG...as in: power off/on is the only way out. I tried this with mulitple combinations and nothing got me past this brick-wall. However, removing /APIC does allow the machine to boot up successfully. Yet the output of MPTable clearly shows an active MPS 1.4 setup.

...One more consumer for ACPI4.BPD service is CLOCK03c.SYS.  CLOCK03c uses ACPI  to get config info about HPET.

I've attached outputs of the MPTable as well as KERNLOG$ from a successfull boot, is this enough to troubleshoot the /APIC hang? Is there a better (more appropriate) place to submit such results?

In particular, going back to the original question re: ACPI4.BPD, it would seem that in my case in particular running this driver might help??? So where can I actually obtain the ACPI4.BPD driver to debug further?

Thanks!
-Dariusz
Title: Re: OS/4 (technical details only)
Post by: Lars on April 08, 2019, 09:40:14 am
You should have read the OS/4 kernel readme. It clearly states that in order to use OS4APIC.PSD with the /APIC switch you have to also use ACPI4.BPD. The only alternative to this set up is a patched ACPI.PSD as a replacement for OS4APIC.PSD+ACPI4.BPD (which will never happen as the OS/4 guys need to patch each and every new version of ACPI.PSD in order to make it work with the OS/4 kernel).

In short: if you don't have ACPI4.BPD then you cannot specify the /APIC switch. But if you have working MPS 1.4 tables then you also do not necessarily need ACPI support. One drawback will be that you cannot power off via ACPI (obviously) if you don't have ACPI support.
Title: Re: OS/4 (technical details only)
Post by: OS4User on April 08, 2019, 02:37:37 pm
It clearly states that in order to use OS4APIC.PSD with the /APIC switch you have to also use ACPI4.BPD.

Lars,

OS4APIC.PSD works ok with /APIC switch  if MPtable is correct. One needs acpi4.bpd  only if MPtable is incorrect or missing. 

I admit that os4apic.txt was not clear enough - it is already altered.


Dariusz,

How is your testing of acpi4.bpd doing  (hope you've managed to get it) ?
Title: Re: OS/4 (technical details only)
Post by: Dariusz Piatkowski on April 09, 2019, 12:08:30 am
Lars,

You should have read the OS/4 kernel readme. It clearly states that in order to use OS4APIC.PSD with the /APIC switch you have to also use ACPI4.BPD...

Ohhh??? Well, having read all these docs multiple times I have to say I completely missed it...so yeah, I'd love to see you quote the specific section, b/c as best as I can tell this is not spelled out.
Title: Re: OS/4 (technical details only)
Post by: Dariusz Piatkowski on April 09, 2019, 04:02:38 am
OS4User!

...How is your testing of acpi4.bpd doing  (hope you've managed to get it) ?

Very well indeed. The test version allows me to use the /APIC switch. I've attached the matching LOG for this run.

The icing on the cake would be if I could finally use the full 6 cores of my Phenom II X6 cpu...however as long as I'm using SNAP that will most likely not happen - I'm hitting the dreaded Blue Screen of death (right before WPS starts up).

Anyways, thank you...I'll put some runtime on this combo and report back!
Title: Re: OS/4 (technical details only)
Post by: OS4User on April 09, 2019, 06:57:54 am
Hi

tnx for testing

your system reports about 5 enabled and one disabled CPUs, I do not knot why one is disabled.

however as long as I'm using SNAP that will most likely not happen - I'm hitting the dreaded Blue Screen of death (right before WPS starts up).

you can try http://gus.biysk.ru/os4/test/sddhelp/sddhelp_v01a.RAR and
http://gus.biysk.ru/os4/test/gradd/gradd.sys
Title: Re: OS/4 (technical details only)
Post by: David Graser on April 09, 2019, 09:02:27 pm
Finally am starting to see some traps with FFSandyBridge. I have attached. trap files.
Title: Re: OS/4 (technical details only)
Post by: OS4User on April 09, 2019, 09:23:16 pm
Finally am starting to see some traps with FFSandyBridge. I have attached. trap files.

It looks like you ran FFSandyBridge on noAVX kernel.

By the way, why do you set your VAL so low ?
Title: Re: OS/4 (technical details only)
Post by: Dariusz Piatkowski on April 10, 2019, 01:18:02 am
hey!

...your system reports about 5 enabled and one disabled CPUs, I do not knot why one is disabled...

I have disabled the 6th core because leaving it in 'enabled' state and not using any APIC parameters to limit the # of cores being used always causes the WPS blue-screen hang, ie: they all get used and that causes some type fo an error. Subsequently I just found it easier to simply disable the core that doesn't actually get used otherwise.

Anyways, alright...I've seen the additional drivers you listed before, wasn't quite sure what to make of 'em before...I'll give these a try and report back.

Thanks again, input is much appreciated!
Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on April 10, 2019, 02:03:46 am
Finally am starting to see some traps with FFSandyBridge. I have attached. trap files.

How long had your system been up when these happened?
OS4user makes a good point about your VIRTUALADDRESSLIMIT. I'd suggest raising it, perhaps to 2560

Edit, actually these trps are the same as I get trying to run the Sandybridge build. I guess due to not using a AVX aware kernel.
Title: Re: OS/4 (technical details only)
Post by: David Graser on April 10, 2019, 03:36:14 am
Finally am starting to see some traps with FFSandyBridge. I have attached. trap files.

How long had your system been up when these happened?
OS4user makes a good point about your VIRTUALADDRESSLIMIT. I'd suggest raising it, perhaps to 2560

Edit, actually these trps are the same as I get trying to run the Sandybridge build. I guess due to not using a AVX aware kernel.

I had the VIRTUALADDRESSLIMIT remmed out in the config.os4 statement.  Put it back in.

The howto doc stated to rem it out.

CHANGES TO CONFIG.OS4:

    1. REM the following lines out:
        PSD=ACPI.PSD                  (see note about ACPI.PSD usage below)
        BASEDEV=APM.ADD               (it is not compatible with OS/4)
        BASEDEV=TIMER0.SYS            (it is not used with OS/4)
        DEVICE=C:\OS2\BOOT\PmDD.SYS   (PmDDk.SYS will be used instead)
        VIRTUALADDRESSLIMIT=xxxx      (is set in C:\OS2LDR.INI)
        SET NSPR_OS2_NO_HIRES_TIMER=1 (OS/4 provides bugless DosTmrQueryTime)

I must have had some kind of corruption.  FF would come up but not access the internet.  Back leveled kernel and it was working normally.  Tested many other kernels  with the latest test kernel and everything is now working as it should.  What ever happened must have been causing the traps.  Will continue to monitor.  Everything is back to normal.
Title: Re: OS/4 (technical details only)
Post by: Dariusz Piatkowski on April 10, 2019, 04:05:28 am
OS4User,

...Anyways, alright...I've seen the additional drivers you listed before, wasn't quite sure what to make of 'em before...I'll give these a try and report back....

Nah, no luck.

The replacement drivers result in the same symptom as my normal OS2APIC boot - WPS hang.

I actually suspect this may have something to do with how memory is being allocated during boot, and maybe it's a result of the known SNAP SMP instability (although various parties have said numerous changes were made to the drivers to fix some of this).

Title: Re: OS/4 (technical details only)
Post by: Dave Yeo on April 10, 2019, 05:19:02 am

I had the VIRTUALADDRESSLIMIT remmed out in the config.os4 statement.  Put it back in.

The howto doc stated to rem it out.

CHANGES TO CONFIG.OS4:

    1. REM the following lines out:
        PSD=ACPI.PSD                  (see note about ACPI.PSD usage below)
        BASEDEV=APM.ADD               (it is not compatible with OS/4)
        BASEDEV=TIMER0.SYS            (it is not used with OS/4)
        DEVICE=C:\OS2\BOOT\PmDD.SYS   (PmDDk.SYS will be used instead)
        VIRTUALADDRESSLIMIT=xxxx      (is set in C:\OS2LDR.INI)
        SET NSPR_OS2_NO_HIRES_TIMER=1 (OS/4 provides bugless DosTmrQueryTime)

I must have had some kind of corruption.  FF would come up but not access the internet.  Back leveled kernel and it was working normally.  Tested many other kernels  with the latest test kernel and everything is now working as it should.  What ever happened must have been causing the traps.  Will continue to monitor.  Everything is back to normal.

The Howto (mine is from Dec) says, including typos,
Quote
***Note***

    OS/4 kernel sets VirtualAddressLimit (VAL) according to settings in os2lsr.ini
    Supplied os2ldr.ini.template contains setting for VAL as 3072 which is
    optimal for many cases. But if your application/system requires some other
    value of VAL just edit your OS2LDR.INI to set VAL to desirable value.
Title: Re: OS/4 (technical details only)
Post by: David Graser on April 10, 2019, 08:30:50 am
Thanks everyone for the info! I made a few changes to my os2ldr.ini file.
Title: Re: OS/4 (technical details only)
Post by: CalY on April 21, 2019, 05:32:11 pm
Installed OS4 on P4 2.6ghz single core 2gb ram with ECS 2.1. Tried kernel versions 5511 and 5403. When starting Kon editor from program object immediate trap 000d every time. After testing found if loading Kon through email client or file manger, editor loads and functions normal.
Ideas on cause of behavior appreciated.
Title: Re: OS/4 (technical details only)
Post by: OS4User on April 21, 2019, 09:43:29 pm
Tried kernel versions 5511 and 5403. When starting Kon editor from program object immediate trap 000d every time

try 5514
Title: Re: OS/4 (technical details only)
Post by: CalY on April 23, 2019, 10:51:52 pm
Kon loaded normal. with 5514 kernel. Now got trap 000e errors in Postroad Mail and Firefox 31
Title: Re: OS/4 (technical details only)
Post by: OS4User on April 24, 2019, 04:27:57 pm
Kon loaded normal. with 5514 kernel. Now got trap 000e errors in Postroad Mail and Firefox 31

pls  give more info:   trap screen or comport log,  where to get Postroad Mail  and how do you run Postroad Mail  to get this trap, etc.
Title: Re: OS/4 (technical details only)
Post by: CalY on April 26, 2019, 03:18:58 pm
Will leave this for now until I get another machine available for testing. All the traps are disturbing hard drive on main machine. Is ACPI necessary with OS4? This computer did not have it enabled.
Title: Re: OS/4 (technical details only)
Post by: OS4User on April 29, 2019, 10:13:07 am
Is ACPI necessary with OS4?

ACPI is optional in many cases. But some systems with broken or missing MP table need it to run in SMP mode.

Title: Re: OS/4 (technical details only)
Post by: David Graser on July 09, 2019, 05:27:49 pm

Update:  I have been using the OS4 kernel with the sandybridge AVX FireFox for some time now.  I have another partition with the OS/2 kernel and the no VAX FF.  I can honestly say I prefer the FF AVX vesrion over the OS/2 kernel no AVX version.  The Firefox AVX version with the OS/4 kernel appears to be to be far more robust.
Title: Re: OS/4 (technical details only)
Post by: ivan on July 09, 2019, 09:25:41 pm
I see many references to Intel processors but I have a question.  Will OS/4 work with a Ryzen 3 2200G with built in graphics and if so, what do I need to change on my OS/2 4.52 CP image to make it work (has to be from the image file as there are no USB 3 drivers and the installer doesn't work - not enough memory with 4GB for some reason).

I ask because I have a spare computer now I have upgraded my main computer to a Ryzen 5 2400G processor unit and would like to test OS/4 on the old unit.
Title: Re: OS/4 (technical details only)
Post by: OS4User on July 10, 2019, 07:59:40 am
Will OS/4 work with a Ryzen 3 2200G with built in graphics (?)

Some HW with Ryzen is reported as working:
https://www.os2world.com/wiki/index.php/Phoenix_OS/4#OS.2F4_Tested_Hardware_List

and if so, what do I need to change on my OS/2 4.52 CP image to make it work (has to be from the image file as there are no USB 3 drivers and the installer doesn't work - not enough memory with 4GB for some reason).

http://gus.biysk.ru/os4/os2krnlSVN5537_unoff.zip  includes the required binaries and HowTo file with an instruction.

Pls, let me know if something is unclear. I am also curious about the results of installation.
Title: Re: OS/4 (technical details only)
Post by: Sigurd Fastenrath on July 10, 2019, 08:10:23 am
Are you interested in Results using OS/4 kernel with OS4ACPI on Thinkpad 25 with Intel Core i7 7500U CPU?

I have got it up and running, OS/4 kernel starts in log mode, so I can see a lot of messages during start.

So far I discovered the following traps wich do not occure once I use the 14.201SMP Kernel with the same OS/2 4.52 based system (not ArcaOS). I have the latest QSINIT installed and from that I am using that Bootmenue where I can switch between different config.sys files. So I am able to boot what I want to use.

Traps so far (again: does not happen with kernel 14.201SMP):

- starting DOSBOX freeze WPS
- sometimes opening Drives (Icon view) freeze WPS (50%)

What can I provide for you?

I am not using ANPM, Open Office is already running, have to modify for latest Firefox/Thunderbird to get them to run, once this works I will test the AVX capability with the AVX Firefox. During start OS/4 kernel reports "AVX capability" of the system (or some kind of similar message, do not have it at hand now)

By the way: I plan to build a Thinkpad 25 "Frankenpad" later this year that will be based on a Thinkpad 25 but will have a Intel Core i7 - 8550U CPU (8th Generation in contrast to the usual 7th Generation), so it will double kernels and provide even more speed. In addition it will have another Display as the one in the Thinkpad 25 is unfortunately of very poor qualitiy (low brightness, low colours). Will test my customized Warp 4.52 including OS/4 kernel with it as well then. The first parts did already arrive  :) :) Including the "Legendary" keyboard of the Thinkpad 25, the best I have ever used, so far. Really hard to find and get. I plan to call "my" Thinkpad 25 the "Thinkpad 25-W" Edition, where the W is a synonym for "Watson" and "Warp" ;)

If interested in you can follow the Thinkpad 25-W Edition Thread here:

https://thinkpad-forum.de/threads/220152-Thinkpad-25-W-der-Thinkpad-25-Frankenpad-Umbau-Versuchsbericht?p=2198168#post2198168 (https://thinkpad-forum.de/threads/220152-Thinkpad-25-W-der-Thinkpad-25-Frankenpad-Umbau-Versuchsbericht?p=2198168#post2198168)
Title: Re: OS/4 (technical details only)
Post by: David Graser on July 10, 2019, 10:36:24 am
Are you interested in Results using OS/4 kernel with OS4ACPI on Thinkpad 25 with Intel Core i7 7500U CPU?

I have got it up and running, OS/4 kernel starts in log mode, so I can see a lot of messages during start.

So far I discovered the following traps wich do not occure once I use the 14.201SMP Kernel with the same OS/2 4.52 based system (not ArcaOS). I have the latest QSINIT installed and from that I am using that Bootmenue where I can switch between different config.sys files. So I am able to boot what I want to use.


Are you using the latest test OS4 kernel located at

http://gus.biysk.ru/os4/os2krnlSVN5537_unoff.zip

David
Title: Re: OS/4 (technical details only)
Post by: OS4User on July 10, 2019, 10:18:43 pm
Are you interested in Results using OS/4 kernel with OS4ACPI on Thinkpad 25 with Intel Core i7 7500U CPU?

Yes, it is interesting.

I have got it up and running, OS/4 kernel starts in log mode, so I can see a lot of messages during start.

So far I discovered the following traps wich do not occure once I use the 14.201SMP Kernel with the same OS/2 4.52 based system (not ArcaOS). I have the latest QSINIT installed and from that I am using that Bootmenue where I can switch between different config.sys files. So I am able to boot what I want to use.

Traps so far (again: does not happen with kernel 14.201SMP):

- starting DOSBOX freeze WPS
- sometimes opening Drives (Icon view) freeze WPS (50%)

I have never head about this behavior.  Need more testing.
Title: Re: OS/4 (technical details only)
Post by: Sigurd Fastenrath on July 12, 2019, 11:02:12 am
Are you interested in Results using OS/4 kernel with OS4ACPI on Thinkpad 25 with Intel Core i7 7500U CPU?

Yes, it is interesting.

I have got it up and running, OS/4 kernel starts in log mode, so I can see a lot of messages during start.

So far I discovered the following traps wich do not occure once I use the 14.201SMP Kernel with the same OS/2 4.52 based system (not ArcaOS). I have the latest QSINIT installed and from that I am using that Bootmenue where I can switch between different config.sys files. So I am able to boot what I want to use.

Traps so far (again: does not happen with kernel 14.201SMP):

- starting DOSBOX freeze WPS
- sometimes opening Drives (Icon view) freeze WPS (50%)

I have never head about this behavior.  Need more testing.

Hi, I will respond to your e-mails later this weekend. I do use lastest unofficial kernel as suggested by David Graser.

During booting OS/4 kernel I have the message: "Can not load PMDD.SYS" and to confirm it, may be I have not exactly followed the instructions (did not have this errormessage on other Hardware though). I will have a closer look.