Qt6 Development

[Paul Smedley]

FYI - I just raised https://github.com/bitwiseworks/python-os2/issues/9 around an issue with python 3.x and building QtWebEngine 6.3.x (the build system in 6.2.x used python2, they now use python3). Fix is in the Issue..

Got another breakage later on, so have made a further change to subprocess.py - will see if that resolves things.

Currently at: [1519/20019] - started with ~25000 items to build so in reality 6500/25000 (ish) or 26%

[Paul Smedley]

Stuck for the time being with 6.3.2 at ~40% built (against the 6.2.4 base libraries)

Code: [Select]

[31/15740] ACTION //third_party/devtools-frontend/src/inspector_overlay:build_inspector_overlay(//build/toolchain/os2:x86)
FAILED: resources/inspector_overlay/main.js
C:/USR/BIN/python3 ../../../../../src/3rdparty/chromium/third_party/devtools-frontend/src/scripts/build/build_inspector_overlay.py main.js 67000 --input_path gen/third_party/devtools-frontend/src/inspector_overlay --output_path resources/inspector_overlay --rollup_plugin ../../../../../src/3rdparty/chromium/third_party/devtools-frontend/src/inspector_overlay/loadCSS.rollup.js
Usage: ../../../../../src/3rdparty/chromium/third_party/devtools-frontend/src/scripts/build/build_inspector_overlay.py filename_1 max_size_1 filename_2 max_size_2 ... filename_N max_size_N --input_path <input_path> --output_path <output_path>
Traceback (most recent call last):
  File "U:/DEV/qt6-webengine-os2-6.3.x/build/src/core/Release/i386/../../../../../src/3rdparty/chromium/third_party/devtools-frontend/src/scripts/build/build_inspector_overlay.py", line 92, in <module>
    sys.exit(main(sys.argv))
  File "U:/DEV/qt6-webengine-os2-6.3.x/build/src/core/Release/i386/../../../../../src/3rdparty/chromium/third_party/devtools-frontend/src/scripts/build/build_inspector_overlay.py", line 77, in main
    rollup(input_path, output_path, filename, max_size,
  File "U:/DEV/qt6-webengine-os2-6.3.x/build/src/core/Release/i386/../../../../../src/3rdparty/chromium/third_party/devtools-frontend/src/scripts/build/build_inspector_overlay.py", line 52, in rollup
    raise Exception("rollup failed: " + error)
Exception: rollup failed:
gen/third_party/devtools-frontend/src/inspector_overlay/main.js \u2192 stdout...


[Paul Smedley]

Looks to be a problem calling python's popen.... adding some print() statements to work out the parameters to node.exe - the command runs to completion. Will have a play in the morning and try work out what's broken....

[Paul Smedley]

OK the issue is that node.exe isn't sending output to stdout, so it thinks that popen has failed....

Edit:
Interesting, the command line that is run is:

Code: [Select]

node.exe U:/DEV/qt6-webengine-os2-6.3.x/src/3rdparty/chromium/third_party/devtools-frontend/src/node_modules/rollup/dist/bin/rollup --format iife -n InspectorOverlay --input gen/third_party/devtools-frontend/src/inspector_overlay/main.js --plugin ../../../../../src/3rdparty/chromium/third_party/devtools-frontend/src/inspector_overlay/loadCSS.rollup.js --plugin terser

Which produces no output. However, if I run:

Code: [Select]

node.exe U:/DEV/qt6-webengine-os2-6.3.x/src/3rdparty/chromium/third_party/devtools-frontend/src/node_modules/rollup/dist/bin/rollup --format iife -n InspectorOverlay --input gen/third_party/devtools-frontend/src/inspector_overlay/main.js --plugin ../../../../../src/3rdparty/chromium/third_party/devtools-frontend/src/inspector_overlay/loadCSS.rollup.js
Then I get output...

If I then run terser on the generated js - then this works.... so it's something about running multiple plugins that's failing...

Edit2: hacked around this for now, by not running terser, and not checking the size of the generated js (terser is there to reduce the file size of the generated js)

[Paul Smedley]

Got an interesting build error this morning...

Code: [Select]

In file included from gen/third_party/blink/renderer/modules/canvas/canvas_jumbo_1.cc:11:
./../../../../../src/3rdparty/chromium/third_party/blink/renderer/modules/canvas/canvas2d/canvas_image_source_util.cc:7:10: fatal error: ./../../../../../src/3rdparty/chromium/third_party/blink/renderer/modules/canvas/canvas2d/third_party/blink/renderer/bindings/modules/v8/v8_union_cssimagevalue_htmlcanvaselement_htmli
mageelement_htmlvideoelement_imagebitmap_offscreencanvas_svgimageelement_videoframe.h: Result too large
    7 | #include "third_party/blink/renderer/bindings/modules/v8/v8_union_cssimagevalue_htmlcanvaselement_htmlimageelement_htmlvideoelement_imagebitmap_offscreencanvas_svgimageelement_videoframe.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

I'm assuming 'result too large' means the path is too long?

[Mentore]

Hi Paul, not sure about this but the path showing the problem is long 268 characters. If I remember correctly, there is a 256 character limit in OS/2 CMD for paths, but I may be sorely mistaken...

[Lars]

What's more, there is a systemwide 260 character limit (CCHMAXPATH) on any file path.

   /*************************************************************************\
   * CCHMAXPATH is the maximum fully qualified path name length including  *
   * the drive letter, colon, backslashes and terminating NULL.            *
   \*************************************************************************/
   #define CCHMAXPATH         260

[Paul Smedley]

Thanks, glad my memory was correct - hacked around this for now by shortening the filename and changing the .cc file to use the shortened name. One day I'll work out how to automate this...

[Paul Smedley]

I do wonder if I should add -mnoavx to https://github.com/psmedley/qt6-webengine-os2/blob/main/src/3rdparty/chromium/build/config/os2/BUILD.gn#L15 ?

[Paul Smedley]

Also, just noticed https://github.com/psmedley/qt6-webengine-os2/blob/main/src/3rdparty/chromium/third_party/libyuv/include/libyuv/row.h#L47

GCC_HAS_AVX2 was definitely being set for OS/2

Edit: also in scale_row.h - and there are also 3 copies of these files - third_party/libaom, third_party/libvpx (not used right now) and third_party/libyux

[Dave Yeo]

I do wonder if I should add -mnoavx to https://github.com/psmedley/qt6-webengine-os2/blob/main/src/3rdparty/chromium/build/config/os2/BUILD.gn#L15 ?

Probably the more places that it is set, the better. If our kernel ever gets AVX support, it can be undone though GCC will still have alignment issues.

[Paul Smedley]

I'll do a build with the libyuv changes first, then change the os2/build.gn - that will trigger a full rebuild which will take all day....

[Dave Yeo]

Isn't libyuv for raw video? In which case it is seldom used.

libyuv is an open source project that includes YUV scaling and conversion functionality. Scale YUV to prepare content for compression, with point, bilinear or box filter. Convert to YUV from webcam formats for compression. Convert to RGB formats for rendering/effects. Rotate by 90/180/270 degrees to adjust for mobile devices in portrait mode.

Worth trying

[Paul Smedley]

I'll do a build with the libyuv changes first, then change the os2/build.gn - that will trigger a full rebuild which will take all day....

rebuilding with -mno-avx now... [371/18295]

[Paul Smedley]

Didn't get too much further - but confirmation we look to be using avx with libaom...

Code: [Select]

FAILED: obj/third_party/libaom/libaom_intrinsics_avx2/jnt_convolve_avx2.o
gcc -MMD -MF obj/third_party/libaom/libaom_intrinsics_avx2/jnt_convolve_avx2.o.d -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DOFFICIAL_BUILD -DTOOLKIT_QT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNO_UNWIND_TABLES -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -Igen -I../../../../../src/3rdparty/chromium -I../../../../../src/3rdparty/chromium/third_party/libaom/source/config -I../../../../../src/3rdparty/chromium/third_party/libaom/source/config/linux/ia32 -I../../../../../src/3rdparty/chromium/third_party/libaom/source/libaom -mavx2 -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -Wno-unknown-pragmas -Wno-parentheses -Wno-sign-compare -Wno-stringop-overflow -Wno-stringop-overread -Wno-psabi -Wno-multichar -Wno-format-zero-length -fno-unwind-tables -fno-asynchronous-unwind-tables -Zomf -mno-avx -m32 -mfpmath=sse -msse3 -O2 -fomit-frame-pointer -g0 -Wno-unused-local-typedefs -Wno-maybe-uninitialized -Wno-deprecated-declarations -fno-delete-null-pointer-checks -Wno-comments -Wno-packed-not-aligned -Wno-dangling-else -Wno-missing-field-initializers -Wno-unused-parameter -std=gnu11 -c ../../../../../src/3rdparty/chromium/third_party/libaom/source/libaom/av1/common/x86/jnt_convolve_avx2.c -o obj/third_party/libaom/libaom_intrinsics_avx2/jnt_convolve_avx2.o
In file included from C:/usr/lib/gcc/i686-pc-os2-emx/9/include/immintrin.h:53,
                 from ../../../../../src/3rdparty/chromium/third_party/libaom/source/libaom/av1/common/x86/jnt_convolve_avx2.c:12:
../../../../../src/3rdparty/chromium/third_party/libaom/source/libaom/aom_dsp/x86/convolve_avx2.h: In function 'prepare_coeffs_lowbd':
C:/usr/lib/gcc/i686-pc-os2-emx/9/include/avx2intrin.h:588:1: error: inlining failed in call to always_inline '_mm256_shuffle_epi8': target specific option mismatch
  588 | _mm256_shuffle_epi8 (__m256i __X, __m256i __Y)

Seems avx is enabled for all x86 processors...
https://github.com/psmedley/qt6-webengine-os2/blob/main/src/3rdparty/chromium/third_party/libaom/BUILD.gn#L232