OS2 World Community Forum

OS/2, eCS & ArcaOS - Technical => Networking => Topic started by: Paul Smedley on January 18, 2022, 09:46:31 am

Title: Squid Cache
Post by: Paul Smedley on January 18, 2022, 09:46:31 am
Hi All,

Lewis asked for an update to squid cache (http://www.squid-cache.org/),  so  I built v5.3  this morning.

Note this has only been very lightly tested.

https://smedley.id.au/tmp/squid-5.3-os2-20220118.zip

Feedback appreciated :)

Cheers,

Paul.
Title: Re: Squid Cache
Post by: Harald Kamm on January 21, 2022, 07:00:30 am
Hi Paul, thank you very much for having ported Squid 5.3! I would have liked to test it: "SYS1804: The system cannot find the file GSSAPI." After having installed all the Heimdal / Kerbors stuff Squid traps:
______________________________________________________________________

 Exception Report - created 2022/01/21 06:56:32
______________________________________________________________________

 LIBC: Killed by SIGILL

 Hostname:         hal
 OS2/eCS Version:  2.45
 # of Processors:  1
 Physical Memory:  2046 mb
 Virt Addr Limit:  1536 mb
 Exceptq Version:  7.11.5-shl BETA8 (Jun  1 2020 18:37:02)

______________________________________________________________________

 Exception C000001C - Illegal Instruction
______________________________________________________________________

 Process:  L:\SQUID\SBIN\SQUID.EXE (01/18/2022 06:20:10 7,628,584)
 PID:      4A74 (19060)
 TID:      01 (1)
 Priority: 200

 Filename: L:\SQUID\SBIN\SQUID.EXE (01/18/2022 06:20:10 7,628,584)
 Address:  005B:000D7A99 (0001:000C7A99)

So may be some hint on the requirements would be useful :-)

Best, Harald
Title: Re: Squid Cache
Post by: Dave Yeo on January 21, 2022, 08:13:09 am
Hi Harald, SIGILL is a bad instruction, usually from targeting a newer processor then the one you're using. What CPU do you have? And what is the failing opcode, be the section "Failing instruction" in your trp report just below the part you quoted.
Title: Re: Squid Cache
Post by: David McKenna on January 22, 2022, 02:53:15 am
Hi Paul,

 Well, I tried, but just could not get this thing to work. It seems to run OK, except for some errors in the log file (attached), but doesn't add anything to the access log. SeaMonkey just hangs forever if I set it up to use squid, but confess I may not understand how to set it up properly - instructions on the net are all for Windows and Linux and do not seem even remotely related to this port. Another thing I found is it only works if the 'squid' directory is copied to the root of the 'UNIXROOT' drive. Anywhere else and it says it can't find the squid.conf file (is the location hardcoded?) - not even %ETC%, or %UNIXROOT%\etc. I'm stumped...

Regards,
Title: Re: Squid Cache
Post by: David McKenna on January 22, 2022, 03:04:25 am
 Forgot to mention - when first running squid it would give an error that it could not determine the hostname of the computer (even though I have SET HOSTNAME=NETMACHINE in CONFIG.SYS) and close, so I added 'visible_hostname=NETMACHINE' to squid.conf to take care of that...

Regards,
Title: Re: Squid Cache
Post by: Dave Yeo on January 22, 2022, 03:59:23 am
Hi David, setting it up in SeaMonkey should be a matter of going to Preferences-->Advanced-->Proxies and choosing Manual Proxy Configuration and entering your address, 127.0.0.1 was what I used for squid along with the port number, 3128? It's set in squid.conf.
Might need to add your hostname to squid.conf, I had,
Code: [Select]
#Default:
# Automatically detect the system host name
visible_hostname bluelion-40654
During the bluelion days.
Title: Re: Squid Cache
Post by: Dariusz Piatkowski on January 22, 2022, 05:20:53 am
Guys...I'm curious: what's the purspose of setting up Squid on a client machine?

If I understood it's intended purpose correctly, it's meant to be a caching proxy primarily for server environments where the clients which are connecting to the server may fetch the very same data, and when that's coming from the caching proxy instead, it makes the response that much quicker, thus providing scalability, etc.

So when deployed on a client I'm thinking there is a very limitted benefit, no?

I haven't tried it, so no actual metrics to toss into the YEY or NAY side.

Thanks!
Title: Re: Squid Cache
Post by: Dave Yeo on January 22, 2022, 06:08:11 am
In theory, on a client, especially one with a bad/slow connection, using the cache speeds things up as many responses come from the cache instead of over the internet. It was why I used squid when I was on dial up. In practice I found less and less benefit as things went to HTTPS as it is hard to cache encrypted content and stopped using it.
Title: Re: Squid Cache
Post by: Harald Kamm on January 22, 2022, 10:15:38 am
Hi Dave,

thank you very much.

Hi Harald, SIGILL is a bad instruction, usually from targeting a newer processor then the one you're using. What CPU do you have? And what is the failing opcode, be the section "Failing instruction" in your trp report just below the part you quoted.

It's quite an outdated cpu, an Intel Xeon 2 GHz.

 Failing Instruction
______________________________________________________________________

 000D7A91  FCOMI  ST, ST(2)          (dbf2)
 000D7A93  FSTP   ST(2)              (ddda)
 000D7A95  JA     0xd7b10            (77 79)
 000D7A97  FMULP                     (dec9)
 000D7A99 >FISTTP QWORD [EBP-0x100]  (dd8d 00ffffff)
 000D7A9F  MOV    EAX, [EBP-0x100]   (8b85 00ffffff)
 000D7AA5  MOV    EDX, [EBP-0xfc]    (8b95 04ffffff)
 000D7AAB  ADD    ESP, 0x10c         (81c4 0c010000)

Best, Harald
Title: Re: Squid Cache
Post by: David McKenna on January 22, 2022, 02:50:15 pm
 @Dave,

  I tried 127.0.0.1 and port 3128 (also 0.0.0.0, localhost and 192.168.21.3) but nothing seems to work. I also tried running curl from the command line, that too did not work. Could very well be user error though...

 @Dariusz,

 I agree it probably is not very useful for a client, but I wanted to try it out as potentially useful on my server, both for clients and possibly a reverse proxy for a website (supposedly squid can do that). But I gotta figure out how to use it first :-)

Regards,
Title: Re: Squid Cache
Post by: Dave Yeo on January 22, 2022, 05:12:14 pm
Hi Harald, unluckily given Intel's crappy naming, Xeon doesn't say much besides being the high end of whatever Intel was making. Seems "000D7A99 >FISTTP QWORD " is SSE3 and I guess your CPU doesn't support it and is pre-Prescott if I understand it right.
Attached is cpuinfo.exe, that'll list your CPU characteristics if you want to post them.
Paul can comment on whether this was assembly or C code.
Title: Re: Squid Cache
Post by: Dave Yeo on January 22, 2022, 05:45:13 pm
Tried it, seems to just sit there forever trying to load a page. In the log I see,
Code: [Select]
2022/01/22 08:29:29 kid1| ERROR: loading file '/squid/share/errors/templates/ERR_PROTOCOL_UNKNOWN': (2) No such file or directory
2022/01/22 08:29:29 kid1| WARNING: failed to find or read error text file ERR_PROTOCOL_UNKNOWN

Which is missing. Have to try an older browser in case it is SeaMonkey/Firefox (tried both) that is broken.

Edit: seems to be stuck listening at 0.0.0.0, which won't work, there's other invalid arguments even using  my old working squid.conf (almost the same), attaching log.
Title: Re: Squid Cache
Post by: David McKenna on January 22, 2022, 06:24:13 pm
Hi Dave,

  Yes, I got that file missing error too, and was able to get rid of it by replacing the 'errors' directory under \squid\share with the one from the squid 5.3 tarball. A packaging mistake I presume...

  Also, the 'ipcacheAddEntryFromHosts: Bad IP address 'fe80::1%lo0' I got rid of by changing 'fe80::1%lo0' in my Hosts file to 'fe80::1' - probably not needed at all anyway.

 Everything else I see in my log is identical to yours, and I agree that  'DNS Socket created at 0.0.0.0, FD 9' doesn't smell right. Also don't like the lines that say 'invalid argument' and 'Socket operation on non-Socket'...

Regards,
Title: Re: Squid Cache
Post by: Dave Yeo on January 22, 2022, 06:44:09 pm
Tried my old squid, which is working fine as I'm posting this using it. The log also has :DNS Socket created at 0.0.0.0, FD 9" but works. Here's the log for comparison, no invalid arguments, the cache was dirty after 4 years :)
 
Title: Re: Squid Cache
Post by: Harald Kamm on January 23, 2022, 08:42:18 am
Hi Harald, unluckily given Intel's crappy naming, Xeon doesn't say much besides being the high end of whatever Intel was making.

Hi Dave,

yes, of course. Here is the output of cpuinfo:

vendor_id   : GenuineIntel
cpu family   : 15
model      : 2
stepping   : 4
flags      : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
cpu MHz      : 1991.393
model name   : Intel(R) XEON(TM) CPU 2.00GHz

Best, Harald
Title: Re: Squid Cache
Post by: Dave Yeo on January 23, 2022, 09:44:19 am
Hi Harald, basically a Pentium 4 Northwood and as you can see, no SSE3 support. Paul can explain why squid is using SSE3. The trp report is accurate and this build of squid will not run on your processor.
Gotta say that Intel has crappy CPU naming, especially with Xenon's
Thanks.
Title: Re: Squid Cache
Post by: Digi on January 27, 2022, 06:12:38 am
What package is missing on my system to run squid?

C:\Programs\squid\sbin>chkdll32 squid.exe
Loading DLL 'libcn0' --> D:\UNIXROOT\USR\LIB\LIBCN0.DLL.
Loading DLL 'libcx0' --> D:\UNIXROOT\USR\LIB\LIBCX0.DLL.
Loading DLL 'gtls30' --> D:\UNIXROOT\USR\LIB\GTLS30.DLL.
Loading DLL 'tcpip32' --> C:\MPTN\DLL\TCPIP32.DLL.
Loading DLL 'gssapi' --> NOT loaded!
Unable to load DLL 'gssapi'.  DosLoadModule returned: 2
OS/2 reports 'GSSAPI' contributed to the failure.
Loading DLL 'krb5' --> NOT loaded!
Unable to load DLL 'krb5'.  DosLoadModule returned: 2
OS/2 reports 'KRB5' contributed to the failure.
Loading DLL 'com_err' --> NOT loaded!
Unable to load DLL 'com_err'.  DosLoadModule returned: 2
OS/2 reports 'COM_ERR' contributed to the failure.
Loading DLL 'doscalls' --> loaded.
Loading DLL 'pthr01' --> D:\UNIXROOT\USR\LIB\PTHR01.DLL.
Title: Re: Squid Cache
Post by: tisa on January 27, 2022, 08:13:44 am
Hi Paul! Thanks for your job to porting Squid. I tried version 5.3, in the log file:

ERROR: loading file '/squid/share/errors/templates/ERR_PROTOCOL_UNKNOWN': (2) No such file or directory
WARNING: failed to find or read error text file ERR_PROTOCOL_UNKNOWN
This is solved by copying not filed files from a distribution for Linux, for example ... After that, Squid starts, but in cache.log there are more curses, and Squid does not respond. Also does not work "Squid -k shutdown" for example ...

**************************************************************************************
Initializing IP Cache...
commSetNonBlocking FD 7: (22) Invalid argument
...
Logfile: opening log daemon:/squid/var/logs/access.log
Logfile Daemon: opening log /squid/var/logs/access.log
commSetNonBlocking FD 10: (22) Invalid argument
commSetNonBlocking FD 10: (22) Invalid argument
commSetNonBlocking FD 12: (38) Socket operation on non-socket
commSetNonBlocking FD 15: (38) Socket operation on non-socket
commUnsetNonBlocking FD 15: (38) Socket operation on non-socket
...
Finished loading MIME types and icons.
commSetNonBlocking FD 18: (22) Invalid argument
...
**************************************************************************************

On the same configuration of Squid 3.5.23 works perfectly - uptime is more than 400 days about 50 users.
Thanks again for your excellent work.
Title: Re: Squid Cache
Post by: Dave Yeo on January 27, 2022, 08:31:57 am
Hi Digi,
Code: [Select]
Y:\>yum provides gssapi.dll
heimdal-libs-1.5.3-1.oc00.i386 : Heimdal kerberos shared libraries
Repo        : arcanoae-rel
Matched from:
Other       : gssapi.dll



heimdal-libs-1.5.3-2.oc00.i386 : Heimdal kerberos shared libraries
Repo        : arcanoae-rel
Matched from:
Other       : gssapi.dll



heimdal-libs-7.7.0-4.oc00.i686 : Heimdal kerberos shared libraries
Repo        : installed
Matched from:
Other       : gssapi.dll

Basically the same response from yum provides krb5.dll and com_err.dll
Title: Re: Squid Cache
Post by: Digi on January 28, 2022, 12:54:11 pm
Hi Digi,
Code: [Select]
Y:\>yum provides gssapi.dll

Thanks! For some reason anpm didn't show me this.
Title: Re: Squid Cache
Post by: Paul Smedley on January 28, 2022, 10:16:43 pm
Hi all, sorry for my absence from this thread. I've been extremely busy in my personal and work life, and haven't had much time for anything OS/2 related.

A couple of comments:
1) ERR_PROTOCOL_UNKNOWN missing - I have no idea why 'make install'  didn't install that file  - I guess it didn't show in my testing as I installed over an existing squid  installation
2) SIGILL for Harald -I am targetting -mtune=pentium4 - so perhaps this triggers the instructions that aren't supported by older CPUs
3)  "Squid -k shutdown" not working - known issue - not sure why - note that using 'go -ka pid' doesn't kill it either
4) dependencies on kerberos, et al - this wasn't intended - for the next build, I'll see if we can stop needing them - I don't see any advantage in the typical usages of squid of OS/2 for these
5) logs about 'commSetNonBlocking FD 10: (22) Invalid argument' & 'commSetNonBlocking FD 12: (38) Socket operation on non-socket' I see these too - again, not sure why yet
Title: Re: Squid Cache
Post by: Dave Yeo on January 29, 2022, 12:07:58 am
Hi all, sorry for my absence from this thread. I've been extremely busy in my personal and work life, and haven't had much time for anything OS/2 related.

A couple of comments:
1) ERR_PROTOCOL_UNKNOWN missing - I have no idea why 'make install'  didn't install that file  - I guess it didn't show in my testing as I installed over an existing squid  installation

My old install was missing it as well

Quote
2) SIGILL for Harald -I am targetting -mtune=pentium4 - so perhaps this triggers the instructions that aren't supported by older CPUs

What is your march? Personally I've been doing march=pentium4 -mtune=generic which should work better on most CPU's. Is there any assembler involved?

Quote
3)  "Squid -k shutdown" not working - known issue - not sure why - note that using 'go -ka pid' doesn't kill it either
4) dependencies on kerberos, et al - this wasn't intended - for the next build, I'll see if we can stop needing them - I don't see any advantage in the typical usages of squid of OS/2 for these
5) logs about 'commSetNonBlocking FD 10: (22) Invalid argument' & 'commSetNonBlocking FD 12: (38) Socket operation on non-socket' I see these too - again, not sure why yet

Does it work for you?
And thanks for the work.
Title: Re: Squid Cache
Post by: Paul Smedley on September 24, 2022, 11:08:47 am
Sorry for the long delay here.... I found the cause of the SIGILL - I believe this should be fixed in https://smedley.id.au/tmp/squid-5.3-os2-20220924.zip

Simply put, the configure script for squid tries to use 'native' cpu optimisation - which overrode my normal defaults.

PS the kerberos dependency should be gone too.
Title: Re: Squid Cache
Post by: Paul Smedley on September 25, 2022, 12:18:23 am
https://smedley.id.au/tmp/squid-5.3-os2-20220925.zip should fix the commSetNonBlocking FD 10: (22) Invalid argument'
Title: Re: Squid Cache
Post by: Paul Smedley on September 25, 2022, 03:46:32 am
Updated to latest source (v5.7) - https://smedley.id.au/tmp/squid-5.7-os2-20220925.zip
Title: Re: Squid Cache
Post by: Paul Smedley on September 26, 2022, 11:07:20 am
https://smedley.id.au/tmp/squid-5.7-os2-20220926.zip should fix the issues with 'squid -k shutdown' not working