Author Topic: Curious what the problem is making new firefox versions.  (Read 15110 times)

Dave Yeo

  • Hero Member
  • *****
  • Posts: 4786
  • Karma: +99/-1
    • View Profile
Re: Curious what the problem is making new firefox versions.
« Reply #15 on: December 25, 2020, 03:17:08 am »
There's a couple of steps in building an executable, including a DLL. First the compiler turns source code into object files. For a large project there are lots of source files, arranged in subdirectories for various functionality. Second the compiler calls a linker to link all those object files into a program or DLL. Sometimes there are other steps involved but we'll ignore them for now.
Another consideration is debug data, which is handy for debugging a program. The debug data can be big as it keeps a bunch of data in the source files and program so that when run in a debugger, you can see the original text code as well as variables etc.
First problem is that sometimes compiling those source files with debug data can use upwards of 2GB's of memory, the workaround there is to compile one at a time, which is wasteful on a multi-core machine and adds time. Firefox here on a 3.1 GHz I5 takes over a hundred minutes to build when compiling up to 5 files at a time, perhaps twice as long or longer when doing one at a time. So by trading off time we can get around this bottleneck.
The next step is feeding all those files into a linker. The linker has to keep track of a lot of stuff as it combines them into one file and can get pretty memory hungry. Firefox's xul.dll is 75 MB's at this point and the QT5 web one much bigger. Even with Mozilla, we had the IBM linker start to fail back at Firefox 4 and had to switch to the OpenWatcom (OW) linker, which we do have the source for and Steven knows how it works and has commit access to put fixes back upstream.
The OW linker also had problems and has been adjusted a few times, it can use what it calls a spill file (think swap) and this is what Roderick is talking about when he talks about modifying wlink the OW linker. So basically we're, or rather some developers, are doing what you suggest. GCC is an open source cross platform current compiler that we have the source for and can adjust and wlink is a very good linker, which once again we have the source for and can adjust. It all takes time from an expert though and gets harder when and if you have to juggle parts into our limited address space.
Then there's the memory that a program uses when running, and a browser can use a lot.

David Kiley

  • Full Member
  • ***
  • Posts: 131
  • Karma: +2/-0
    • View Profile
Re: Curious what the problem is making new firefox versions.
« Reply #16 on: January 29, 2021, 10:41:54 am »
It all takes time from an expert though and gets harder when and if you have to juggle parts into our limited address space.
Thanks for the explanation - I had no idea how involved it was :).
And thanks for the work of all the experts making it happen.