• Welcome to OS2World OLD-STATIC-BACKUP Forum.
 

News:

This is an old OS2World backup forum for reference only. IT IS READ ONLY!!!

If you need help with OS/2 - eComStation visit http://www.os2world.com/forum

Main Menu

Need help setting up proxy server

Started by icemantc6, 2007.07.03, 01:22:18

Previous topic - Next topic

mobybrick

Unless we are careful, we could end up confusing and bewildering the original poster, or anyone else reading this thread...

You said:

Quote from: obiwan on 2007.07.20, 20:30:04
This is very good information mobybrick, interesting and useful. I have a minor correction for you though.

Quote from: mobybrick on 2007.07.20, 11:38:41
5. SQUID's FTP support is NOT a true FTP proxy. You cannot use an FTP client through SQUID. You can, however, use FTP from a web browser (as this is FTP over HTTP).

This is not strictly true, at least of the version of Squid I use.

And then you said you "That seems consistent with my experience. A more careful reading of my previous post would find no indication that I actually disbelieve the technicalities of your word on the matter" when you do seem to take a different view to mine, www.squid-cache.orgs, or linuxsecurity.com.

SQUID only talks HTTP to a client - so any Windoze software must do HTTP transfers to do this (or use the SSL 'Connect' method). The 'average' Windows program, requiring direct FTP access to the web, will NOT work with Squid.

So, for the average user - as we are not in a position to test all of their requirements - they should NOT plan on their FTP access working with Squid - and just be pleasantly suprised if it does!

Additionally, Squid can support either passive or non-passive transfers - but not both at once....

Just trying to be pragmatic and work on worst case scenario - as well as trying to explain what Squid does under the covers...

Regards,
Moby.

obiwan

#16
Quote from: mobybrick on 2007.07.23, 17:10:36
Unless we are careful, we could end up confusing and bewildering the original poster, or anyone else reading this thread...

I could not agree with you more. I think that is the trouble with getting too technical in a thread about a usage question.

Do you understand the difference between something being "not strictly true" and "disbelieving the technicalities" of something?

For something to be not strictly true, it needs only to have a single case where something about it does not hold true. It does not mean that everything about it is false, so the use of the phrase "not strictly true" implies that there is only a minor defect; and if that is missed, the phrase "minor correction" found in the opening line of my original post should help in catching the intent. The example I give of my usage is not to challenge you to a lengthy technical argument, but to offer a real option to interested parties which is otherwise excluded by your advice. That only means that the statement "You cannot use an FTP client through SQUID" is in certain cases not true, nothing more.

I follow up the example with language such as "then again," "it could be," and "I really don't know," attached to each mention of the underlying technicalities, which strongly implies that I have no certainty about them; which thus carries the implication that I have no basis to disbelieve what you say about them. What is more, nothing I said about them contradicts what you had said about them.

Notice here I refer to technicalities, which is distinguished from usage. Usage is what people use a thing for, while technicalities are the technological details which make it possible. For example, "FTP over HTTP" is a technicality, whereas "use the NFTP client with Squid" is a usage.

When I said, "That seems consistent with my experience," it was not a contradiction to my previous statement, but a confirmation that certain FTP clients can be used with Squid to upload and download files, not delete or rename them, as you quoted from linuxsecurity.com. Notably, this is consistent with my original statement that Squid may have some problems or limitations with FTP, and that some FTP clients can use it.

Quoteyou do seem to take a different view to mine, www.squid-cache.orgs, or linuxsecurity.com.

I do not take a view at all, so that is a mischaracterization. I have a different usage from you, and as I said my experience is consistent with your quote of linuxsecurity.com, so I do not differ with it. My usage is the same as icemantc6's original stated intent, to "record visited web pages and block ones I don't want accessed," a purpose for which Squid works excellently for me with select FTP clients.

QuoteThe 'average' Windows program, requiring direct FTP access to the web, will NOT work with Squid.

Or 'average' FTP program for OS/2 or any platform. I specifically said in my original post that only wget, NFTP, and web browsers will work.

What exactly am I mistaken about? Are you arguing that I am mistaken in how I am using Squid, or that I am using it incorrectly, or that I am imprudent to suggest to others that it is possible to use it the same way I do? Or simply that you know more about Squid than I do? I am happy to agree with you on that last point.

Quotetrying to explain what Squid does under the covers...

An approach which seems a bit too technical even for me, though the information is duly noted and appreciated.


icemantc6

ok so my proxy is up and running squid turned out to be amazing one problem though, one of the reasons this server is needed is to block web sites i tried doing that using the acl tags using the urlregex urlpath regex and none seem to work, i have to block multiple web sites but for now i want to block myspace to get a feel of how to do it

mobybrick

Quote from: icemantc6 on 2007.07.23, 21:42:45
ok so my proxy is up and running squid turned out to be amazing one problem though, one of the reasons this server is needed is to block web sites i tried doing that using the acl tags using the urlregex urlpath regex and none seem to work, i have to block multiple web sites but for now i want to block myspace to get a feel of how to do it

Great that you have made some progress!

Try the following. Remember that the allow/deny statements are processed in order, so if you had something like 'http access allow all' at the beginning, any deny statements would never be processed....


acl test_deny url_regex [-i] ^http://www.myspace.com


#Recommended minimum configuration:
#
# Only allow cachemgr access from localhost
http_access allow manager localhost
http_access deny manager

http_access deny test_deny
# insert any other allow statements here

http_access allow localhost
http_access allow our_networks

# And finally deny all other access to this proxy
http_access deny all


It's a good idea to always finish with the http_access deny all statement.

Let me know how things go!

Regards,
Moby.

mobybrick

Quote from: obiwan on 2007.07.23, 19:52:43
Quote from: mobybrick on 2007.07.23, 17:10:36
Unless we are careful, we could end up confusing and bewildering the original poster, or anyone else reading this thread...

I could not agree with you more. I think that is the trouble with getting too technical in a thread about a usage question.

Do you understand the difference between something being "not strictly true" and "disbelieving the technicalities" of something?

Oh dear. Now your uncareful use of language could easily be taken to imply that I'm stupid. I avoided the personal comments. You really have to have the last word don't you. Actually, I think I've been quite patient. A quick Google search produces lots of information that contradicts pretty much everything you say...

Quote from: obiwan on 2007.07.23, 19:52:43
For something to be not strictly true, it needs only to have a single case where something about it does not hold true. It does not mean that everything about it is false, so the use of the phrase "not strictly true" implies that there is only a minor defect; and if that is missed, the phrase "minor correction" found in the opening line of my original post should help in catching the intent. The example I give of my usage is not to challenge you to a lengthy technical argument, but to offer a real option to interested parties which is otherwise excluded by your advice. That only means that the statement "You cannot use an FTP client through SQUID" is in certain cases not true, nothing more.


Anything which uses anything additional to the FTP protocol, is, by definition **not** a plain FTP client. We've already established that SQUID only talks HTTP. Therefore, the statement 'You cannot use an FTP client through SQUID' is correct. Only timewasters would go and find something that is not *just* an FTP client (i.e. it talks something else as well) and say it works....

Quote from: obiwan on 2007.07.23, 19:52:43
I follow up the example with language such as "then again," "it could be," and "I really don't know," attached to each
If you really don't know, why waste everyones time trying to make out that you do and then confuse everything?

Quote from: obiwan on 2007.07.23, 19:52:43
Or 'average' FTP program for OS/2 or any platform. I specifically said in my original post that only wget, NFTP, and web browsers will work.

Funning, 'coz according to the NFTP documentation, SQUID is listed as being an 'HTTP' proxy with severe limitations. They even recommend patching the SQUID code so that transfers are reliable.

See http://www.ayukov.com/nftp/firewall.html

5. HTTP proxy (Squid)
Currently, only Squid and Netscape SuiteSpot server are supported. You have to specify `firewall-host' and port (typically 3128 for Squid). Both Squid 1.x and 2.x are supported; my tests were done on 1.1.22 and 2.1-RELEASE. For best results it is recommended (but of course not necessary) to apply a patch to Squid sources before compiling it. The patch is available from ftp://ftp.ayukov.com/pub/nftp; instructions are inside. It will force Squid to report file size in bytes for NFTP instead of kilobytes; this makes file sizes in NFTP precise instead of rounded. Restarting transfers through Squid is not yet supported, and some features are not available via Squid; these include making directories, renaming and deleting files. Uploading and authentication are supported since NFTP version 1.60. All transfers are made in binary mode. With Squid, there's no such thing as 'permanent connection to server', and you can't verify connection aliveness or send verbatim commands to server

Yup, as I guessed, NFTP is working exactly as I said it would be....

Quote from: obiwan on 2007.07.23, 19:52:43
An approach which seems a bit too technical even for me, though the information is duly noted and appreciated.

I wasn't the one on an ego trip here. I only came to this board to help people. Unfortunately, most people's queries here are of a technical nature. Don't you think it requires a technical approach to solve a technical problem?


*************************** SUMMARY FOR ALL THOSE BORED ***************************

SQUID may support your FTP client, it may not. It is not intended as a real FTP proxy. This means that no-one can guarantee anything but a web-browser to work. Also, the developers routinely make changes to the code and only test these against web browsers. If your FTP client does work, great. If it does not, then you will need to find another way. Unfortunately, SQUID is only a HTTP proxy (and SSL tunnel) and HTTP caching/proxying is the primary design goal of the developers - FTP support is only there for web browser *download* usage. So, even if it does work, only extremely old browsers will allow you to upload via FTP through SQUID.

So, if anyone says that they can't get their FTP client to work with SQUID - but hear that others may have got theirs to work in a blue moon - look to see if an option for 'HTTP proxy' support is available. If there is not, then probably little can be done. Others may find that only FTP sites that accept anonymous credentials will work - another possible limitation of the SQUID design.

Lastly, there are *real* FTP proxy solutions available, and these are free. If you need full FTP support (i.e. not just for web browser downloads) then it may be a good idea to install one of these. Alternatively, other solutions such as the DELEGATE proxy, or a NAT solution may suit you.

Best regards to everyone,
Moby.

obiwan

Quote from: mobybrick on 2007.07.24, 00:03:21
Oh dear. Now your uncareful use of language could easily be taken to imply that I'm stupid. I avoided the personal comments. You really have to have the last word don't you. Actually, I think I've been quite patient.

All personal and irrelevant. I'm not going there.

QuoteAnything which uses anything additional to the FTP protocol, is, by definition **not** a plain FTP client. We've already established that SQUID only talks HTTP.

My use is for FTP requests to FTP servers, not HTTP requests to web servers.

QuoteTherefore, the statement 'You cannot use an FTP client through SQUID' is correct.

It is correct, but not when taken in a strict sense. That is why the word strict exists.

QuoteOnly timewasters would go and find something that is not *just* an FTP client (i.e. it talks something else as well) and say it works....

If that is the case, I use Squid in a way that wastes time; but it is how I do it. I thought the information might be useful, but apparently it is controversial. I wish you had simply said it is a poor recommendation if that is what you meant. Since the entire content of what I tried to contribute is what I have done, it cannot be contradicted by Google searches.

QuoteIf you really don't know, why waste everyones time trying to make out that you do and then confuse everything?

I did not say or "make out" that I know about the technicalities involved. As was just discussed I implied otherwise. I said that I use Squid in a way that is in some contexts described as an FTP proxy, because I thought it might be helpful. I am sorry that turned out to be confusing.

QuoteYup, as I guessed, NFTP is working exactly as I said it would be....

I am glad reading the documentation satisfies you that the way I use it is consistent with your technical understanding. I don't see where that or anything you are quoting contradicts what I said.

QuoteUnfortunately, most people's queries here are of a technical nature. Don't you think it requires a technical approach to solve a technical problem?

Absolutely, but what I understand you to be doing is arguing with me and telling me I am wrong about technical details I did not really make a statement about. It is interesting information, but doesn't really make any difference to how I use Squid. I did thank you for explaining about what a true FTP proxy is. You did help me to understand better, but I do not plan to replace Squid with one because it works so well for me. I do have some limitations, but I do not have a problem with it. As I acknowleged it was limited, I don't think my suggesting it to others with that caveat was out of line.

But judging by the amount of text and emotion that resulted, maybe it turned out to be a mistake.

obiwan

Quote from: mobybrick on 2007.07.24, 00:03:21
If you really don't know, why waste everyones time trying to make out that you do and then confuse everything?

There is, of course, a better way to answer this. That is that if one (such as myself in this case) posts something to a forum with a tenor of ignorance, it is usually with the invitation to be set straight on the matter.

You did that when you pointed me to the ftpproxy site. The documentation there was very cursory in explaining how it worked so it took me a few minutes to understand. That is why I came back, thanked you, and posted more clearly and succinctly than its own documentation why it might be a better choice in some cases.

There is of course no requirement that anyone set me straight because this is a friendly community forum where we help out of sheer kind volunteerism. But if one is to do it, it should be to correct what I actually said, or tell me where I am wrong or misunderstanding. That is why when you told me I am mistaken about things I did not say, and did not accept the thing I really did say, I followed up with more explanation of my previous text. That certainly can end up wasting time, which is why it is generally good practice to read and write the text carefully. I thought I had done that, and there seems to be evidence for that since I am able to point to the original text to support every clarification, but apparently it was not clear enough; so I am sorry I injected myself into this thread.

I think your "For the Bored" text sums up the facts well enough, so I'll exit here. If you feel any desire to tell me more about your observations of my mistakenness and uncarefulness and other flaws I heartily welcome you to send them to me privately.

mobybrick

Quote from: obiwan on 2007.07.24, 01:13:11
Quote from: mobybrick on 2007.07.24, 00:03:21
Oh dear. Now your uncareful use of language could easily be taken to imply that I'm stupid. I avoided the personal comments. You really have to have the last word don't you. Actually, I think I've been quite patient.

All personal and irrelevant. I'm not going there.
[/quote]

You started it... But I am sorry if we wound each other up.

Quote from: obiwan on 2007.07.24, 01:13:11
Quote from: mobybrick on 2007.07.24, 00:03:21
Anything which uses anything additional to the FTP protocol, is, by definition **not** a plain FTP client. We've already established that SQUID only talks HTTP.

My use is for FTP requests to FTP servers, not HTTP requests to web servers.
[/quote]
Actually, we weren't here to talk just about your requests. My responses were to help the original poster. We've already established that NFTP is *not* just a plain FTP client (because it also can use HTTP GET) and neither is wget.

Quote from: obiwan on 2007.07.24, 01:13:11
Quote from: mobybrick on 2007.07.24, 00:03:21
Therefore, the statement 'You cannot use an FTP client through SQUID' is correct.

It is correct, but not when taken in a strict sense. That is why the word strict exists.
[/quote]

But I never used the word 'strict.'

Quote from: obiwan on 2007.07.24, 01:13:11
Quote from: mobybrick on 2007.07.24, 00:03:21
Only timewasters would go and find something that is not *just* an FTP client (i.e. it talks something else as well) and say it works....

If that is the case, I use Squid in a way that wastes time; but it is how I do it. I thought the information might be useful, but apparently it is controversial. I wish you had simply said it is a poor recommendation if that is what you meant. Since the entire content of what I tried to contribute is what I have done, it cannot be contradicted by Google searches.
[/quote]

Wouldn't it be better to say that there is a possibility that a person's FTP client may support more than just FTP and that it may work? Rather than saying that it works for you and therefore must work for everyone? I'd still wager that most FTP clients will NOT work through SQUID.

I do find that sometimes an assumption as to how things work can cause great problems; I run SQUID on OS/2 with thousands of clients, and people 'promising' things that cannot be done is a cause of great frustration. Taking a bit of time to fully understand how things are working and therefore what is available prevents frustration for everyone...

Quote from: obiwan on 2007.07.24, 01:13:11
plan to replace Squid with one because it works so well for me. I do have some limitations, but I do not have a problem with it. As I acknowleged it was limited, I don't think my suggesting it to others with that caveat was out of line.

What limitations? Anything we can help with?

obiwan

Quote from: mobybrick on 2007.07.24, 11:16:28
What limitations? Anything we can help with?

Thank you, no. Just the limitations we described. I can only use NFTP for uploads, and I'm not crazy about NFTP. Trying to delete produces weird behavior. It is acceptable to me for my purposes.

obiwan

#24
I did intend to exit this, believe it or not.

Quote from: mobybrick on 2007.07.24, 11:16:28
You started it...

I haven't said anything about you or myself, other than what I have done with regard to the topic and what we have said here. My question about your understanding of my words was in response to the way you were representing them. I cannot imply things that I cannot know, and I do not know you.

QuoteActually, we weren't here to talk just about your requests.

My purpose of mentioning my use is to clarify that Squid will talk to FTP servers, and in this context it is in a sense an FTP proxy, even if not a "true FTP proxy."

QuoteWe've already established that NFTP is *not* just a plain FTP client (because it also can use HTTP GET) and neither is wget.

From the NFTP documentation:

wget can also download via HTTP though while NFTP cannot

NFTP is not an ordinary FTP client, but it is not for purposes other than FTP.

QuoteBut I never used the word 'strict.'

I did. I said the statement is not strictly true, and you said I am mistaken, which means that what I said is untrue. Saying that a statement is untrue is logically equivalent to stating the opposite of the statement, which would be, "It is strictly true."

Quote
Wouldn't it be better to say that there is a possibility that a person's FTP client may support more than just FTP and that it may work? Rather than saying that it works for you and therefore must work for everyone?

That would be acceptable, but what I said was more precise, that NFTP, wget, and web browsers work, and I have found nothing else that does. That makes it explicit what works and what doesn't. These clients work not because I am the one using them but because they are compatible, so they will work for anyone. In contrast, the wording you give seems to hint that it is some function other than FTP that works. So no, I don't think that would be better.

Quote
I'd still wager that most FTP clients will NOT work through SQUID.

That has been made clear by both of us several times now already. I never said this was not the case.

QuoteI do find that sometimes an assumption as to how things work can cause great problems; I run SQUID on OS/2 with thousands of clients, and people 'promising' things that cannot be done is a cause of great frustration. Taking a bit of time to fully understand how things are working and therefore what is available prevents frustration for everyone...

Agreed. I appreciate your explaining how it is that NFTP works when others do not. I don't think, though, that avoiding frustration is so paramount as to make necessary the exclusion of all options that have a possibility of it. As long as we are clear about the limitations, it is legitimate to consider even such unconventional options as OS/2 and eComStation.

mobybrick

Dear reader,

C'mon, this has got to be a wind-up. I'm half expecting Dave Letterman to appear from behind somewhere! Lighten up!

I'm sorry Obiwong, but you seem to be wound up on arguing about language semantics, implying people (myself) are 'stupid' because they don't understand the differences in simple bits of language, and picking holes in choice of words rather than having a real technical discussion based on fact...

SQUID supports HTTP only. This means that web browsers can download through SQUID, as this is what SQUID is designed to provide. Only extended FTP clients that support HTTP methods (i.e. a web-browser) will work. Most FTP (only) clients will not work. None of the FTP-only tools supplied with OS/2 will work through SQUID. Period. Tools like NFTP and WGET use SQUID as an HTTP proxy, and functionality is limited, although WGET works better for downloads and it is is designed to use HTTP properly.

If you are far more worried about whether this is strictly true (which it is), or whether this is 'Saying that a statement is untrue is logically equivalent to stating the opposite of the statement, which would be, "It is strictly true."' then I'd suggest that you have bigger worries than FTP to deal with.

Incidentally, it's worth reading http://www.squid-cache.org/mail-archive/squid-users/200312/0122.html to see the progress thus far on adding delete support to SQUID. I don't think it has yet been implemented, although you could try adding DELETE to the HTTP extension_methods in the SQUID.CONF.

Now, come on out Dave!

Regards,
Moby.


obiwan

That entire response was ad hominem and personal and not worth a reply, except the tip about adding delete to Squid, so I'll watch for that, thanks.

mobybrick

#27
Hi,

Ad hominem and personal obviously mean pretty much the same thing, but I won't beat you up about repeating yourself...

I actually was trying to be a bit light-hearted and do appreciate the ideas behind some of the points you have made. But I've also spent a lot of time in the last few months trying to support SQUID and am rather fed up of people making assumptions about what it will do, rather than reading the (extensive) documentation first. Nothing I've stated is factually incorrect and all of it was designed to be nothing but helpful...

See you around...

Regards,
Moby.

Quote from: obiwan on 2007.07.24, 21:00:51
That entire response was ad hominem and personal and not worth a reply, except the tip about adding delete to Squid, so I'll watch for that, thanks.

RobertM

Quote from: obiwan on 2007.07.24, 19:49:43
As long as we are clear about the limitations, it is legitimate to consider even such unconventional options as OS/2 and eComStation.


Though they may seem like unconventional options (eCS/Warp), we are finding that none of our clients care what OS is running on their server as long as

  • They dont have to touch it, look at it, reboot it, scan it for viruses, upgrade it on a continuous cycle
  • They dont need to pay an IT guy to come in to do the above on a monthly basis

For that, we find eCS an optimal solution. Linux may do as well, though personally I have found the current threading model in eCS to be better... while that is supposed to be changing with the kernel threading changes in Linux, those changes are still kinda experimental. In addition, installing eCS seems a little easier. Configuring it to manage all levels of hosting and serving other than MS Exchange and similar tools (which none of our clients use) is also easy. Makes a great firewall, file server, ftp server, mail server, web server, proxy server, DDNS server, DHCP server, print server, the list goes on... and all of which are easily REXX scriptable to one extent or another (through callable REXX scripts called directly through the app/daemon, or like in the case of the Firewall, by having a REXX script rewrite rules and configuration info, and telling the firewall to reload it's configuration and rules).

eCS is easy to secure, all aspects of serving are well documented, and updates are rarely if ever needed. On today's hardware it flies - even when running multiple server daemons - unlike Windows. And using highly threaded apps (like Lotus Domino GoWebserver) it outperforms any PC based OS I have ever tested such things on. And each new license we purchase helps fund further development for eCS.

As for the rest of where this thread is going (the FTP debate), please guys, lets keep it civil. Tom (icemantc6) wasn't that concerned with FTP as it is being blocked (by the firewall) anyway. Letting him know that the support is non-standard is helpful in the event (which is possible) that it needs to be unblocked and supported later. You both seem to already agree as to Squid's FTP functionality and/or lack thereof and seem to be just debating semantics right now.

That can lead to this thread becoming an endless debate on semantical issues - where instead - this thread can be very informative and helpful (thanks to both of your wonderful input) to anyone else who in the future wishes to use Squid as a proxy server on eCS.


|
|
Kirk's 5 Year Mission Continues at:
Star Trek New Voyages
|
|