Author Topic: AVX enabled Firefox  (Read 7258 times)

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
AVX enabled Firefox
« on: February 19, 2021, 11:19:53 pm »
Finally got around to trying to build Firefox with -march=sandybridge. While the build works fine with -march=i686 or pentium-m, it dies with -march=sandybridge, doesn't seem to like the GCC basic_string<char>. Error log attached if anyone has any ideas.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: AVX enabled Firefox
« Reply #1 on: February 21, 2021, 06:02:28 am »
OTOH, building a Phenom II (-march=amdfam10) succeeds. Previously when building the AVX Firefox, packaging failed while creating startup.cache. I thought due to a sigill but never actually checked. Well, the AMD build's packaging also failed, after re-enabling the startup.cache, so I figured swap in a working xpcshell.exe. Still failed. Seems to actually run out of memory.
Code: [Select]
make[3]: Entering directory `Y:/work/cc45-git/mozilla/obj-fb/browser/installer'
OMNIJAR_NAME=omni.ja \
NO_PKG_FILES="core bsdecho js js-config jscpucfg nsinstall viewer TestGtkEmbed elf-dynstr-gc mangle* maptsv* mfc* msdump* msmap* nm2tsv* nsinstall* res/samples res/throbber shlibsign* certutil* pk12util* BadCertServer* OCSPStaplingServer* GenerateOCSPResponse* chrome/chrome.rdf chrome/app-chrome.manifest chrome/overlayinfo components/compreg.dat components/xpti.dat content_unit_tests necko_unit_tests *.dSYM " \
Y:/work/cc45-git/mozilla/obj-fb/_virtualenv/bin/python Y:/work/cc45-git/mozilla/toolkit/mozapps/installer/packager.py -DMOZ_APP_NAME=firefox -DPREF_DIR=defaults/preferences -DMOZ_NATIVE_NSPR=1 -DMOZ_NATIVE_NSS=1 -DJAREXT= -DMOZ_CHILD_PROCESS_NAME=plugin-container.exe -DDLL_PREFIX= -DDLL_SUFFIX=.dll -DBIN_SUFFIX=.exe -DDIR_MACOS= -DDIR_RESOURCES= -DBINPATH=bin -DRESPATH=bin -DLPROJ_ROOT=en -DMOZ_ICU_VERSION=56 -DMOZ_NATIVE_ICU -DMOZ_SHARED_ICU -DMOZ_ICU_DBG_SUFFIX= -DX_DISPLAY_MISSING=1 -DMOZ_ENABLE_PROFILER_SPS=1 -DMOZILLA_VERSION='"45.9.0"' -DMOZILLA_VERSION_U=45.9.0 -DMOZILLA_UAVERSION='"45.0"' -DOS2=1 -DXP_OS2=1 -DOS2EMX_PLAIN_CHAR=1 -DMOZ_OS2_HIGH_MEMORY=1 -DD_INO=d_ino -DMOZ_DEBUG_SYMBOLS=1 -DSTDC_HEADERS=1 -DHAVE_DIRENT_H=1 -DHAVE_GETOPT_H=1 -DHAVE_SYS_BITYPES_H=1 -DHAVE_MEMORY_H=1 -DHAVE_UNISTD_H=1 -DHAVE_NL_TYPES_H=1 -DHAVE_MALLOC_H=1 -DHAVE_IO_H=1 -DHAVE_CPUID_H=1 -DHAVE_SYS_QUEUE_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_NETINET_IN_H=1 -DHAVE_SIN_LEN=1 -DHAVE_SCONN_LEN=1 -DHAVE_SA_LEN=1 -DHAVE_LIBCX=1 -DNO_X11=1 -DHAVE_PTHREAD_H=1 -DHAVE_GMTIME_R=1 -DHAVE_LOCALTIME_R=1 -DHAVE_ARC4RANDOM=1 -DHAVE_CLOCK_MONOTONIC=1 -DHAVE_LANGINFO_CODESET=1 -DVA_COPY=va_copy -DHAVE_VA_COPY=1 -DHAVE_THREAD_TLS_KEYWORD=1 -DHAVE_I18N_LC_MESSAGES=1 -DHAVE_LOCALECONV=1 -DMALLOC_H='<malloc.h>' -DHAVE_ALLOCA_H=1 -DHAVE_STRNDUP=1 -DHAVE_POSIX_MEMALIGN=1 -DHAVE_MEMALIGN=1 -DHAVE_MALLOC_H=1 -DMALLOC_USABLE_SIZE_CONST_PTR=const -DHAVE_VALLOC=1 -DHAVE_VALLOC=1 -DTARGET_XPCOM_ABI='"x86-gcc3"' -DRELEASE_BUILD=1 -DMOZ_UPDATE_CHANNEL=default -DMOZ_PHOENIX=1 -DMOZ_BUILD_APP=browser -DMOZ_PDF_PRINTING=1 -DNS_PRINTING=1 -DNS_PRINT_PREVIEW=1 -DMOZ_OFFICIAL_BRANDING=1 -DMOZ_DISTRIBUTION_ID='"org.mozilla"' -DACCESSIBILITY=1 -DMOZ_SAMPLE_TYPE_FLOAT32=1 -DMOZ_WEBSPEECH=1 -DMOZ_WEBSPEECH_TEST_BACKEND=1 -DATTRIBUTE_ALIGNED_MAX=16 -DMOZ_WEBM=1 -DMOZ_FFMPEG=1 -DMOZ_FMP4=1 -DMOZ_EME=1 -DMOZ_MEDIA_NAVIGATOR=1 -DMOZ_LIBKAI=1 -DMOZ_VPX=1 -DHAVE_LIBVPX=1 -DMOZ_VPX_NO_MEM_REPORTING=1 -DMOZ_WAVE=1 -DMOZ_VORBIS=1 -DMOZ_WEBM_ENCODER=1 -DMOZ_PERMISSIONS=1 -DENABLE_SYSTEM_EXTENSION_DIRS=1 -DMOZ_WEBGL_CONFORMANT=1 -DMOZ_CRASHREPORTER_ENABLE_PERCENT=100 -DMOZ_ENABLE_SIGNMAR=1 -DMOZ_FEEDS=1 -DMOZ_SAFE_BROWSING=1 -DMOZ_URL_CLASSIFIER=1 -DGL_PROVIDER_=1 -DMOZ_STACKWALKING=1 -DMOZ_LOGGING=1 -DFORCE_PR_LOG=1 -DMOZ_PAY=1 -DMOZ_ACTIVITIES=1 -DMOZ_SECUREELEMENT=1 -DHAVE___CXA_DEMANGLE=1 -DHAVE__UNWIND_BACKTRACE=1 -DJS_DEFAULT_JITREPORT_GRANULARITY=3 -DMOZ_OMNIJAR=1 -DMOZ_USER_DIR='"Mozilla"' -DHAVE_FT_BITMAP_SIZE_Y_PPEM=1 -DHAVE_FT_GLYPHSLOT_EMBOLDEN=1 -DHAVE_FT_LOAD_SFNT_TABLE=1 -DHAVE_FONTCONFIG_FCFREETYPE_H=1 -DHAVE_STDINT_H=1 -DHAVE_INTTYPES_H=1 -DMOZ_XUL=1 -DMOZ_PROFILELOCKING=1 -DENABLE_MARIONETTE=1 -DBUILD_CTYPES=1 -DMOZ_PLACES=1 -DMOZ_SOCIAL=1 -DMOZ_SERVICES_COMMON=1 -DMOZ_SERVICES_CRYPTO=1 -DMOZ_SERVICES_HEALTHREPORT=1 -DMOZ_SERVICES_METRICS=1 -DMOZ_SERVICES_SYNC=1 -DMOZ_SERVICES_CLOUDSYNC=1 -DMOZ_JSDOWNLOADS=1 -DMOZ_MACBUNDLE_ID=org.mozilla.firefox -DMOZ_B2G_VERSION='"1.0.0"' -DMOZ_B2G_OS_NAME='""' -DMOZ_APP_UA_NAME='""' -DMOZ_APP_UA_VERSION='"45.9.0"' -DFIREFOX_VERSION=45.9.0 -DMOZILLA_OFFICIAL=1 -DMOZ_TELEMETRY_DISPLAY_REV=2 -DMOZ_DATA_REPORTING=1 -DMOZ_DLL_SUFFIX='".dll"' -DA11Y_LOG=1 -DEXPOSE_INTL_API=1 -DENABLE_INTL_API=1 -DMOZ_STATIC_JS=1 -DNO_NSPR_10_SUPPORT=1 -DAB_CD=en-US \
--format omni \
--removals Y:/work/cc45-git/mozilla/browser/installer/removed-files.in \
\
\
\
\
--optimizejars \
\
Y:/work/cc45-git/mozilla/browser/installer/package-manifest.in ../../dist ../../dist/firefox \

Executing Y:/work/cc45-git/mozilla/obj-fb/dist/bin/xpcshell.exe -g Y:/work/cc45-git/mozilla/obj-fb/dist/bin/ -a Y:/work/cc45-git/mozilla/obj-fb/dist/bin/ -f Y:/work/cc45-git/mozilla/toolkit/mozapps/installer/precompile_cache.js -e precompile_startupcache("resource://gre/");
out of memory: 0x0000000020000000 bytes requested
Creating 00EC_01.TRP

With the trp showing a INT 3 triggered by the OOM handler.
I'll rerun the build with the startup.cache disabled, then, since I seemed to have forgot to renable it, try a Pentium M build and see if it does the OOM thing. SM has always had the startup.cache enabled.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
Re: AVX enabled Firefox
« Reply #2 on: February 21, 2021, 06:36:04 pm »
Dave!

OTOH, building a Phenom II (-march=amdfam10) succeeds...

I'm here to test that...just let me know where to pick it up from...  ;)

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: AVX enabled Firefox
« Reply #3 on: February 21, 2021, 07:13:07 pm »
I'm going to have to rebuild it with the startup cache disabled. I think the OOM error may be misleading and xpcshell is failing due to xul.dll being built for the Phenom.
If I run the Firefox binary. it dies pretty quick here, as expected.
Then figure out the best place to upload.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: AVX enabled Firefox
« Reply #4 on: February 22, 2021, 07:04:07 pm »

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
Re: AVX enabled Firefox
« Reply #5 on: February 22, 2021, 08:14:35 pm »
Hi Dave!

Hi Dariusz, this should be it, https://drive.google.com/file/d/1dV1S9yYyyV6LAfOTqNcCDlu42kDzkpgG/view?usp=drivesdk
Let me know it it works.

Alright, so this report is regarding running this version of FF on OS/2 kernel (202 SMP).

Things came up fine, hit a bunch of sites, including YouTube, no issues. I will run this version for an extended period of time to see how it holds up to normal "wear'n'tear".

In the meantime, tying this back to the AVX enabled ffmpeg conversation: does this mean that the newly enabled AVX capability in FF really does NOT come into play unless I'm running it on OS/4 kernel?

Thanks for the effort and time you always put into these things Dave!

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: AVX enabled Firefox
« Reply #6 on: February 22, 2021, 08:23:47 pm »
This Firefox is not AVX enabled. See my first message, for some reason targeting sandybridge resulted in some weird errors, so out of curiosity I targeted your CPU to see if it would compile and it did, so once built, why not share.
I was also curious what would happen if I tried to run the AVX enabled Firefox here on my I5 with the OS/2 kernel. I think it would have caused the system to become funky as I doubt that it would have checked for the OS having AVX capabilities, unlike programs that are written to run on all CPU's.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: AVX enabled Firefox
« Reply #7 on: February 22, 2021, 08:30:10 pm »
I should also mention that this build of Firefox won't run on too many processors, even newer AMD CPU's will likely fail due to it possibly using some 3DNOW code which I believe supprt for was removed at some point.

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
Re: AVX enabled Firefox
« Reply #8 on: February 22, 2021, 09:05:34 pm »
Dave,

This Firefox is not AVX enabled. See my first message, for some reason targeting sandybridge resulted in some weird errors, so out of curiosity I targeted your CPU to see if it would compile and it did, so once built, why not share.

Ahh...OK, sorry I focused on the "...OTOH, building a Phenom II (-march=amdfam10) succeeds...". Either way, I am good with this build because I think this does a bit of a catch-up as far as some security patches you had applied to the non-AMD builds...so I'm glad to have that deployed!

...I was also curious what would happen if I tried to run the AVX enabled Firefox here on my I5 with the OS/2 kernel. I think it would have caused the system to become funky as I doubt that it would have checked for the OS having AVX capabilities, unlike programs that are written to run on all CPU's.

For what it's worth there is a AVX Instruction set recognition bug (https://bugzilla.mozilla.org/show_bug.cgi?id=1110570) out there, I just started to read through it as it came up as I was doing a bit of a deeper-dive into "why is AVX good for me??? LOL".

In that bug they specifically point out the following:

1) AVX instructions should be emitted only if XSAVE is enabled by the OS
2) check the extended control register for OS support

The bug itself is from 6 years ago...

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: AVX enabled Firefox
« Reply #9 on: February 22, 2021, 09:13:22 pm »
That is about the JS JIT compiler not respecting whether the OS supports AVX instructions. Doesn't seem to apply to us as there hasn't been any reports of crashes that could be tied to it.
It does bring up the point that on an OS4 kernel, the JIT compiler likely produces AVX instructions.

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: AVX enabled Firefox
« Reply #10 on: February 23, 2021, 08:33:36 am »
Tried building with -march=native, with native being an I5-2400, a Sandybridge processor. This worked. It also seems to work fine using the OS/2 kernel, presumably as there are not many AVX instructions. Seems to me as long as only one thread uses AVX, the registers won't get scrambled during a context switch. I doubt that it is checking the OS capabilities as it is targeted at this machine.
https://drive.google.com/file/d/1dk49wLSOgqQ0BWYzAqNOSOpS9Ixr9FbP/view?usp=drivesdk

Dariusz Piatkowski

  • Hero Member
  • *****
  • Posts: 1317
  • Karma: +26/-0
    • View Profile
Re: AVX enabled Firefox
« Reply #11 on: February 23, 2021, 03:28:33 pm »
Hey Dave!

That is about the JS JIT compiler not respecting whether the OS supports AVX instructions. Doesn't seem to apply to us as there hasn't been any reports of crashes that could be tied to it.
It does bring up the point that on an OS4 kernel, the JIT compiler likely produces AVX instructions.

Yeah, I caught that too but figured for what it's worth it seemed to have spelled out how FF attempts to deal with AVX hardware capabilities. I have not looked into the bigger 'FF and AVX' any further though.

You know, the JIT use-case may be a reasonable excercise, after all we have seen some performance hits to FF on our platform due to JIT, have we not? I'm thinking various JS issues, CPU cores going 100% and all that...if there is a chance of that being even slightly improved that may be a neat little "added bonus" to getting FF AVX enabled.

Not quite sure though how JIT would benefit from AVX as the purpose of that instruction is to apply same execution to a much larger data set, stuff you'd normally see in media decoding such as YouTube playback, etc.
« Last Edit: February 24, 2021, 04:22:48 am by Dariusz Piatkowski »

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4787
  • Karma: +99/-1
    • View Profile
Re: AVX enabled Firefox
« Reply #12 on: February 24, 2021, 02:21:20 am »
Well it seems the build using -march=native does not use AVX, presumably due to GCC being smart enough to check the operating system support before enabling it for "native"
Adding -mavx to force AVX support results in the same error as posted earlier and Googling, I can't find any hints as to why.