Author Topic: mozturbo  (Read 11828 times)

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 13
  • -Receive: 226
  • Posts: 3151
  • Karma: +59/-0
    • View Profile
mozturbo
« on: December 16, 2018, 06:16:19 pm »
I've updated the old mozturbo using the code from SM-1.19 for Firefox, SeaMonkey and Thunderbird, 3 binaries, one package. https://bitbucket.org/dryeo/mozturbo/downloads/
These will preload the DLLs used by the various packages. The main reason for this is that our kernel has a bug, see https://mantis.arcanoae.com/view.php?id=2194. Basically when using high memory and closing an app, the memory that the DLLs allocated is not unallocated, leaving zombie holes in memory and eventually running out of high memory. Even without using high memory, this may help memory fragmentation.
Install with the browser or email client closed. Simplest is to put it in the working directory where the DLLs are located though it has an option to pass the location to it, create a program object, put in your startup folder with the -l (L) parameter and start it or better, reboot. Use programs as usual. You can use Theseus to keep track.

I've had good results using it with SM and TB, haven't tested too much with FF. Others have reported some problems with FF, low memory usage or not fixing slowness on restart.
Anyways shouldn't hurt, it has a command to unload the DLLs if needed. Run without parameters to see options.

Should work with most versions of Mozilla, targeted at 45ESR and 38ESR

Pete

  • Hero Member
  • *****
  • Thank You
  • -Given: 1
  • -Receive: 51
  • Posts: 1098
  • Karma: +4/-0
    • View Profile
Re: mozturbo
« Reply #1 on: December 16, 2018, 09:39:09 pm »
Hi Dave

... The main reason for this is that our kernel has a bug, see https://mantis.arcanoae.com/view.php?id=2194.


I can only get an Access Denied page when trying the above url - even when logged in. Is anyone else having problems accessing this url?


Regards

Pete

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 13
  • -Receive: 226
  • Posts: 3151
  • Karma: +59/-0
    • View Profile
Re: mozturbo
« Reply #2 on: December 16, 2018, 10:00:34 pm »
Quite possible you don't have permission. It's just Rich's description of the problem with some logs and screenshots from Theseus showing the problem.
You can load Theseus, highlight a process and choose under the process menu, Shared Object Summary and after opening and closing and reopening an app with DLLs marked to load high, see the results, basically chunks of allocated memory with no owner but the same size as the DLL currently loaded. For example, though the zombie memory was from SM 2.35, first area after the first Free area was probably xul.dll
Code: [Select]
  Object Allocated Committed   Present   Swapped
 address    memory    memory    memory    memory  Description
8BF31000  0000F000                                 Free
8BF40000  03E1F000  00000000  00000000  00000000 
8FD5F000  00001000                                 Free
8FD60000  000A0000  00000000  00000000  00000000 
8FE00000  00704000  00000000  00000000  00000000 
90504000  0000C000                                 Free
90510000  00080000  00000000  00000000  00000000 
90590000  00018000  00000000  00000000  00000000 
905A8000  00008000                                 Free
905B0000  000D1000  00000000  00000000  00000000 
90681000  0000F000                                 Free
90690000  00002000  00000000  00000000  00000000 
90692000  0000E000                                 Free
906A0000  00005000  00000000  00000000  00000000 
906A5000  0000B000                                 Free
906B0000  00001000  00000000  00000000  00000000 
906B1000  0000F000                                 Free
906C0000  00037000  00000000  00000000  00000000 
906F7000  00009000                                 Free
90700000  00001000  00000000  00000000  00000000 
  Object Allocated Committed   Present   Swapped
 address    memory    memory    memory    memory  Description
90701000  0000F000                                 Free
90710000  03E1F000  00000000  00000000  00000000 
9452F000  00001000                                 Free
94530000  000A0000  00000000  00000000  00000000 
945D0000  006BC000  006BC000  006BC000  00000000  SUITE    #0001 (shared code)
94C8C000  00004000                                 Free
94C90000  0006A000  00000000  00000000  00000000  LGPLLIBS #0001 (shared code)
94CFA000  00006000                                 Free
94D00000  000A7000  00000000  00000000  00000000  MOZSQLT3 #0001 (shared code)
94DA7000  00009000                                 Free
94DB0000  039A0000  00000000  00000000  00000000  XUL      #0001 (shared code)

OS4User

  • Sr. Member
  • ****
  • Thank You
  • -Given: 97
  • -Receive: 16
  • Posts: 306
  • Karma: +9/-0
    • View Profile
Re: mozturbo
« Reply #3 on: December 16, 2018, 10:10:58 pm »
Basically when using high memory and closing an app, the memory that the DLLs allocated is not unallocated

VMM releases only physic mapping  but address space remains in use. (This bug is fixed in OS/4 long back).
« Last Edit: December 17, 2018, 06:47:14 pm by OS4User »

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 13
  • -Receive: 226
  • Posts: 3151
  • Karma: +59/-0
    • View Profile
Re: mozturbo
« Reply #4 on: December 17, 2018, 06:12:54 am »
Note there is a bug in ffturbo, missing DLL. I'll upload a new build later. The others should be fine.

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 13
  • -Receive: 226
  • Posts: 3151
  • Karma: +59/-0
    • View Profile
Re: mozturbo
« Reply #5 on: December 17, 2018, 04:20:57 pm »
New version uploaded, only change is for Firefox users, namely adding a missing DLL in ffturbo.

André Heldoorn

  • Hero Member
  • *****
  • Thank You
  • -Given: 119
  • -Receive: 19
  • Posts: 787
  • Karma: +0/-0
    • View Profile
Re: mozturbo
« Reply #6 on: December 19, 2018, 10:48:56 am »
Simplest is to put it in the working directory where the DLLs are located though it has an option to pass the location to it

I presume you mean the non-LIBPATH home directory of XUL.DLL.

Is it recommended to use high memory in general, because all qualifying FF/TB or SM DLLs will always be loaded, even when you're not going to use FF/SM/TB while working with some large database?

I'll try to find the minimum memory requirements of just using *TURBO.EXE, without actually using the related full product. I can restrict installs to >= 1 GiB of RAM, but if it just loads DLLs then 256 MiB may be enough too. To simplify installs, one tends to not actually use FF/SM/TB that frequently with 256 MiB of RAM.

Later you could consider to ship *TURBO.EXEs with SM/TB updates, if any, since the Mozilla browser installed by the OS has an own MOZTURBO.EXE in its own main directory too. Mine's x:\PROGRAMS\MOZILLA\MOZTURBO.EXE. Just to promote consistency, and using it will remain optional.

André Heldoorn

  • Hero Member
  • *****
  • Thank You
  • -Given: 119
  • -Receive: 19
  • Posts: 787
  • Karma: +0/-0
    • View Profile
Re: mozturbo
« Reply #7 on: December 21, 2018, 05:10:42 pm »
Is it recommended to use high memory in general, because all qualifying FF/TB or SM DLLs will always be loaded, even when you're not going to use FF/SM/TB while working with some large database?

Can it be compiled as a PM app, without the FCF_TASKLIST flag? You may be able to use Van Wijk's PMTPOSD2 (stealth mode) as sample code.

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 13
  • -Receive: 226
  • Posts: 3151
  • Karma: +59/-0
    • View Profile
Re: mozturbo
« Reply #8 on: December 23, 2018, 05:06:22 am »
Hi André,
Sorry for not responding sooner, no power for a few days due to big wind.
Yes, you should never have any of the Mozilla apps on LIBPATH, so home directory of xul.dll.
Loading the DLLs into high memory is a users choice. Generally a good idea due to the shortage of address space in low memory. Note that you need a recent kernel ideally due to bugs in older kernels. Some or most of these bugs may happen when unloading the DLLs, so mozturbo maybe a workaround for that as well.
If you do load into high memory, when unloading, the kernel will not unallocate the memory, so you lose memory anyways, even if working on large databases etc.
It does jsut load the DLLs and I have an update to push that uses less memory but doesn't work as well as a turbo launcher, there's a switch to bring back the current way it works. I'll probably upload tomorrow after considering whether to make it a PM app. I'll look at Van Wijk's PMTPOSD2 later.
BTW, with mozturbo running and doing various tests which involved unloading and reloading, I managed 8 days of uptime, usually the system traps or locks up after 2-3 days. Been years since I've had much more then 3 days uptime.

Rich Walsh

  • Full Member
  • ***
  • Thank You
  • -Given: 1
  • -Receive: 14
  • Posts: 147
  • Karma: +2/-0
  • ONU! (OS/2 is NOT Unix!)
    • View Profile
Re: mozturbo
« Reply #9 on: December 23, 2018, 04:36:35 pm »
[...] after considering whether to make it a PM app.

Please don't! Currently it can be launched as a background daemon (e.g. "detach ffturbo") which uses almost no memory on its own. I can't see how making it a PM app adds anything useful.

Creating a couple of program objects to launch and terminate it makes more sense to me:
Path and file name: 'cmd.exe'
Parameters: '/c detach ffturbo -l'  [..or..]   '/c ffturbo -u'
Working directory: [your FF directory]

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 13
  • -Receive: 226
  • Posts: 3151
  • Karma: +59/-0
    • View Profile
Re: mozturbo
« Reply #10 on: December 25, 2018, 05:07:08 am »
Updated to v0.5. Default is now not to force load the modules, saving quite a bit of memory at the cost of turbo functionality. Thanks to Rich for pointing this out. Added the -t parameter for the old behaviour.
I don't anticipate any more binary changes to the programs.
https://bitbucket.org/dryeo/mozturbo/downloads/mozturbo-v0.5.zip

André Heldoorn

  • Hero Member
  • *****
  • Thank You
  • -Given: 119
  • -Receive: 19
  • Posts: 787
  • Karma: +0/-0
    • View Profile
Re: mozturbo
« Reply #11 on: December 27, 2018, 12:49:26 pm »
Please don't! Currently it can be launched as a background daemon (e.g. "detach ffturbo") which uses almost no memory on its own. I can't see how making it a PM app adds anything useful.

Creating a couple of program objects to launch and terminate it makes more sense to me:
No protests obviously, as long as results are the same: an user doesn't see it in any default tasklist and can shutdown the system without an extra dialog. Without PM all *TURBO.EXEs will work the same way too, AFAICT. Including the original one.

André Heldoorn

  • Hero Member
  • *****
  • Thank You
  • -Given: 119
  • -Receive: 19
  • Posts: 787
  • Karma: +0/-0
    • View Profile
Re: mozturbo
« Reply #12 on: December 27, 2018, 01:04:33 pm »
I'll stick to the default settings w.r.t. high or low memory.

BTW, with mozturbo running and doing various tests which involved unloading and reloading, I managed 8 days of uptime, usually the system traps or locks up after 2-3 days. Been years since I've had much more then 3 days uptime.

Probably no surprise, but according to Theseus it seems okay to use it with systems with <= 512 MiB of RAM too. One tends to not use e.g. SM that often then, but one also tends to not use other hungry apps. Which allows me to use the *TURBOs everywhere.

PMTPOSD2 would be hardly more than a skeleton app with too many bones, but it has a matching stealth mode (no tasklist, no extra shutdown dialog). I'll try DETACH instead, if that fully hides it as well.

Thanks for the smart solutions!

Dariusz Piatkowski

  • Hero Member
  • *****
  • Thank You
  • -Given: 15
  • -Receive: 32
  • Posts: 990
  • Karma: +21/-0
    • View Profile
Re: mozturbo
« Reply #13 on: December 27, 2018, 04:40:12 pm »
Hi Dave!

Updated to v0.5. Default is now not to force load the modules, saving quite a bit of memory at the cost of turbo functionality. Thanks to Rich for pointing this out...

Alright...so what is the benefit / functionality being delivered with this new default setting? I mean after all, isn't the whole point to pre-load the DLLs and prevent their un-loading during program shut-down?

Sorry, I simply missed it and/or do not understand what the utility would do otherwise if it is not actualy doing the DLL load.

Also, as a follow-up request for clarification, what happens to other DLLs that do not reside in the main directory? For FF I am specifically thinking:

Code: [Select]
The volume label in drive G is OS2.
The Volume Serial Number is 45B6:75C3.

Directory of G:\apps\tcpip\firefox\browser\components

12-23-18   6:36p    369383         124  brwsrcmp.dll
        1 file(s)     369383 bytes used

Directory of G:\apps\tcpip\firefox\gmp-clearkey\0.1

12-23-18   6:20p    306136         124  clearkey.dll
        1 file(s)     306136 bytes used


Are these left-out?

Thanks!

Dave Yeo

  • Hero Member
  • *****
  • Thank You
  • -Given: 13
  • -Receive: 226
  • Posts: 3151
  • Karma: +59/-0
    • View Profile
Re: mozturbo
« Reply #14 on: December 27, 2018, 06:34:03 pm »
Hi André, using the FCF_TASKLIST flag means a full PM app, I believe it applies to the frame, something that would be overkill and I'm not interested in. The source is there if you decide to do it yourself, it would be fairly easy as far as I can see.
Being detached, the process won't show up in the window list but may still need to be shutdown on exit, I haven't tested. Most of us are running some version of Xworkplace as it is included in ArcaOS and at least later versions of eCS as well as running fine on older versions of Warp, or at least Warp v4. It can ignore processes on shutdown so a one time setting and forget about it. At the worst, you can create a program object to unload the daemon and click it before shutting down.