Welcome to the IPFire Wiki

This wiki is a community-maintained resource about everything there is to know about IPFire. Join us and help us improving it!

Looking for something?

Use the search and find answers to everything about IPFire. If you cannot find what you are looking for, join our community and talk to fellow IPFire users, developers and everybody else involved in the project.

IPFire Community

Differences in Revisions: Squid Web-Proxy

Deleted LDAP Bug solution
# Advanced Web Proxy
 
A web proxy can be used for many reasons, but the most common use is to minimize Internet traffic in a LAN with multiple computers. On a network without a proxy, every time a client PC visits a website, it must download the content and images from the website. If a second PC goes to the same website, this identical content is downloaded again. When using a proxy, it saves the downloaded content locally, and each successive access to this page it will be loaded from the proxy instead of the Internet.
 
## Configuration
 
 
The configuration page has several categories, they are: Proxy Settings, Cache Management, Port and Client Settings, Traffic and Authentication Settings.
 
Here is an example config of my home with 4 clients.
 
![](/en/configuration/network/configuration_network_proxy_settings1.png)
 
 
The Proxy is enabled if:
 
* Enabled is set on the **<color green>green</color>** interface.
* Transparent means, a configuration of the clients isn´t required to use the Proxy.
* The number of processes is set to 20, that means the Proxy is able to scan more pages than in the default setting.
* To each filter the recommended number of processes is shown, the amount should be the number of processes for the Proxy, as example at the beginning 0, +8 processes if you want to use Squidclamav, +6 for Urlfilter and +5 for Updx, for all 3 that is 19 processes, without the virus scanner 11 processes ( the recommendation is calculated on the numbers of known clients )
* Logging is disabled (**<color red>**Warning privacy!!!** Each visited webpage, using port 80, will be logged if enabled</color>**)
* The [URL-Filter](/en/configuration/network/url-filter) is activated to block unwanted pages
* The [Update-Accelerator](/en/configuration/network/update-booster) is also enabled and caches updates for operating system and virus scanner
 
 
### Cache-Management
 
![](/en/configuration/network/configuration_network_proxy_caching.png)
 
The Cache-Management decides the working speed of the Proxy. A wrong configuration can easily make the Proxy unusable. The correct settings are dependent on the workspace of the IPFire.
 
 
Configured is:
 
* 8 MB Memory
* 128 MB Hard Disk space - The Proxy benefits from more
* the minimum filesize that should be cached, here 0kB, the maximum is set to 4 MB (a size of 0 kB keeps every object in cache regardless of how small it is and can result in slower performance, so it is recommended to set it to 1kB on a faster internet connection)
* the number of level1-subdirectories is limited to 16, you should be careful when experimenting. More is not necessarily better.
* the memory replacement policy is set here to "heap GDSF". The best option depends on the configuration and the workspace of the IPFire.
* the Cache replacement policy has also been set to "heap GDSF". The same rules apply here as with the memory replacement policy.
* domains which shouldn´t be cached are filled in here, but they are only exempted from the Proxy, not from the URL-Filter
* the offline mode isn´t activated here, if enabled, the check of the cached objects is disabled. This allows access to more cached information.
 
<note>Since IPFire 2.3 the proxy can be used without hard disk space. That is for example for USB or CF-installations useful. The proxy acts more as an access control and URL-Filter.</note>
 
More information on the options and their effects at [advproxy.net](http://www.advproxy.net)
 
 
### Port- and Client-Settings
 
In this section acceptable ports and subnets and particular IP´s or MAC-Adresses will be allowed or blocked. The Advanced Proxy [DOKU](http://www.advproxy.net/documentation/ipcop-advproxy-en.pdf) is very informative at point 4.5. It is suggested to read this [DOKU](http://www.advproxy.net/documentation/ipcop-advproxy-en.pdf), as everything is explained very clearly.
 
* The option "Target port" is preconfigured and needs only in exceptional case to be changed.
* If you want to exclude clients or subnets, fill them in here.
* If an internal webserver is running ( intranet pages and so on) it is recommended to enable the option "Disable Proxy to green from other subnets".
* Extension for classrooms is an option to control management settings for proxy users - more on this at [DOKU](http://www.advproxy.net/documentation/ipcop-advproxy-en.pdf)
* More options are time limitations, traffic volume, download reduction and MIME-Type filter. Further information can be found in the Advanced-Web-Proxy-[DOKU](http://www.advproxy.net/documentation/ipcop-advproxy-en.pdf).
 
 
### Authentication-Settings
 
Here you can enable proxy authentication for each client. If this option is enabled, only authenticated clients are allowed to use the proxy, all other requests will be dropped. The browser selection appears after activating and saving.
 
![](/en/configuration/network/configuration_network_proxy_auth.png)
 
 
<note>The browser based access isn´t used for the following clients</note>
 
* Unrestricted IP-Adresses
* Unrestricted MAC-Adresses
* Members of the group "Advanced" if the proxy uses "Local Authentication"
 
 
### Authentication on a Windows Server 2003
 
If you want to use your IPFire in combination with a Windows Sever, you have to install the package "cyrus-sasl":
 
`pakfire install -y cyrus-sasl`
 
More information here [Entry in our Forum (German)](http://forum.ipfire.org/index.php?topic=1057.msg11298;topicseen#msg11298)
### Proxy in transparent mode
 
The transparent mode allows the proxy to be used without any client configuration. An iptables rule will be generated that forwards any traffic from port 80 to the proxy. The client is not aware of this which is why it is called transparent proxy.
 
In some cases this mode can cause problems if clients do not follow the http standard or the proxy receives packages that aren´t destined for the proxy. Or a client sends a packet to a special webserver, through TCP the packet finds it's the way to the host (the client leaves the hostfield in the http header empty and the relative URI provides the required result), but if a proxy is between them this won´t work anymore. This sometimes leads to [Forwarding loops](http://www.comfsm.fm/computing/squid/FAQ-11.html#ss11.31). The client sends a package for example with a missing http target host (the TCP destination isn´t important at this moment), the proxy associates it with a local webserver and forwards it to its own host (the packet is sent of course to the proxy again->transparent mode, everything is cached), this loop can repeat 1000 times, so that at the end the proxy either stops with "Running out of File Descriptors" or becomes very slow. This problem can be bypassed, when the field Forward proxy address is activated, the proxy adds some extra information to the http packet (proxyname and port), in the case of a loop it recognizes the packet is from itself and stops the forwarding. The same could happen if there are multiple proxies which are referencing each other and the packet starts playing PingPong. Here the expansion of the http header also solves the problem. But the target server also recognizes that the client is behind a proxy, that means he gets the IP address and the port.
 
You can even try adding "redirector_access deny localhost" to your squid.conf, this may prevent some loops caused by the redirectors.
 
 
### BUG in authenticating via LDAP on Active Directory
 
Affects all IPFire versions with Advanced Proxy Squid version 3.1 .
 
If we use the LDAP authentication (Active Directory) on a Windows Server DC (Tested: 2003 x64), then there are problems with special characters and umlauts for the password or if applicable also the username.
 
Background:
LDAP on MS servers is only capable of UTF-8 (Whether it is rfc compliant?).
The HTTP Authentication is transferred (at least in Europe) by ISO 8859-1 (Western Europe), then Squid passes the information to the external tool squid_ldap_auth pass, which checks then this information to the persistent connection to LDAP.
 
Problem solution (temporarily):
Adding the line manually
`auth_param basic utf8 on`
in /etc/squid/squid.conf .
 
<box 90% round green|Datei: /etc/squid/squid.conf>
```
# Do not modify '/var/ipfire/proxy/squid.conf' directly since any changes
# you make will be overwritten whenever you resave proxy settings using the
# web interface!
#
# Instead, modify the file '/var/ipfire/proxy/advanced/acls/include.acl' and
# then restart the proxy service using the web interface. Changes made to the
# 'include.acl' file will propagate to the 'squid.conf' file at that time.
 
shutdown_lifetime 5 seconds
icp_port 0
 
http_port 192.168.40.254:800
http_port 192.168.41.254:800 transparent
 
 
cache_effective_user squid
cache_effective_group squid
umask 022
auth_param basic utf8 on
 
pid_filename /var/run/squid.pid
 
cache_mem 50 MB
cache_dir aufs /var/log/cache 50 16 256
 
error_directory /usr/lib/squid/errors/de
 
memory_replacement_policy heap GDSF
cache_replacement_policy heap GDSF
 
access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log none
 
log_mime_hdrs off
forwarded_for off
via off
 
acl within_timeframe time MTWHFAS 00:00-24:00
 
#acl all src all
acl localhost src 127.0.0.1/32
acl SSL_ports port 443 # https
acl SSL_ports port 444 # https
acl SSL_ports port 563 # snews
acl SSL_ports port 631 # grub
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 444 # https
acl Safe_ports port 563 # snews
acl Safe_ports port 631 # grub
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 800 # Squids port (for icons)
acl Safe_ports port 1002 # Applejuice
acl Safe_ports port 1003 # Scanner
 
acl IPFire_http port 81
acl IPFire_https port 444
acl IPFire_ips dst 192.168.40.254
acl IPFire_networks src "/var/ipfire/proxy/advanced/acls/src_subnets.acl"
acl IPFire_servers dst "/var/ipfire/proxy/advanced/acls/src_subnets.acl"
acl IPFire_green_network src 192.168.40.0/24
acl IPFire_green_servers dst 192.168.40.0/24
acl IPFire_blue_network src 192.168.41.0/24
acl IPFire_blue_servers dst 192.168.41.0/24
acl IPFire_unrestricted_ips src "/var/ipfire/proxy/advanced/acls/src_unrestricted_ip.acl"
acl CONNECT method CONNECT
 
#Start of custom includes
 
http_port 127.0.0.1:800 transparent
acl to_localhost dst 127.0.0.0/8
acl purge method PURGE
http_access deny to_localhost
http_access allow localhost
http_access allow purge localhost
http_access deny purge
url_rewrite_access deny localhost
 
#End of custom includes
 
#Access to squid:
#local machine, no restriction
http_access allow localhost
 
#GUI admin if local machine connects
http_access allow IPFire_ips IPFire_networks IPFire_http
http_access allow CONNECT IPFire_ips IPFire_networks IPFire_https
 
#Deny not web services
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
 
#Set custom configured ACLs
http_access allow IPFire_unrestricted_ips
http_access allow IPFire_networks within_timeframe
http_access deny all
 
#Strip HTTP Header
request_header_access X-Forwarded-For deny all
reply_header_access X-Forwarded-For deny all
request_header_access Via deny all
reply_header_access Via deny all
 
maximum_object_size 2048 KB
minimum_object_size 2 KB
 
request_body_max_size 0 KB
visible_hostname eXciter-IPFire
 
cache_mgr sr@tne.de
cachemgr_passwd ipfire all
 
max_filedescriptors 8192
 
url_rewrite_program /usr/sbin/redirect_wrapper
url_rewrite_children 15
```
</box>
 
This adjustment, is only available since Squid version 3.1
 
Then you can either use
`/etc/init.d/squid stop`
followed by a
`/etc/init.d/squid start`
to start Squid, or you try a
`squid -k reconfigure`
to load the modified squid.conf .
 
Thus, the external authentication program*squid_ldap_auth * will be fed with UTF-8 compliant data.
 
Test:
`- /usr/lib/squid/squid_ldap_auth with appropriate options, but start without the -P option (see squid.conf)`
`- Now enter the username<blank>password.`
`- If a OK returns, everything works.`
 
Unfortunately it works only as long until Squid gets written a new squid.conf over the web interface. Under normal conditions such a system should not be frequently tackled after the configuration any longer, so that this manual modification should go by order over the command line.
 
Problem solution (permanently):
This line (and only this line) with "utf8" needs to be added into **/svr/web/ipfire/cgi-bin/proxy.cgi**.
Note: The line must/should be approximately in line 3150 (may vary depending on the adv. proxy version)
 
```
`if ($proxysettings{'AUTH_METHOD'} eq 'ldap')`
`{`
` print FILE "auth_param basic utf8 on\n";`
` print FILE "auth_param basic program $authdir/squid_ldap_auth -b \"$proxysettings{'LDAP_BASEDN'}\"";`
```
By editing the files, be careful and always create a backup copy before.
 
# Automatic distribution of the proxy settings
 
## Introduction
 
My thanks for the friendly and competent support goes to [Arne.F](http://forum.ipfire.org/index.php?action=profile;u=92), [Maniacicarus](http://forum.ipfire.org/index.php?action=profile;u=50) and of course to all who were and are involved in the development of IPFire.
 
There are basically two ways to incorporate a Proxy server. For one thing, you can enter it manually in the browser or system and on the other hand it can be operated transparently. Both options have advantages and disadvantages which should not list in here. However the serious difference of the two versions concerns to me in here. If the Proxy operates in transparent mode there is naturally no need to make the adjustments on the clients and the http traffic goes automatically over the Proxy. This also means that you can configure no exceptions. Unfortunately, there are some sites when they are invoked through a proxy, they does not work cleanly. This is exactly why you need sometimes exceptions. So we want that the Proxy does not operate transparently, but we also does not want to touch every client to make the manual entries for the proxy configuration, and the exceptions. But everything should be configured at a central point with a automatic distribution to all clients.
 
## Distribution via DHCP-option
 
A proxy configuration script provides IPFire by default. Which is findable under
`http://IPFireIP:81/proxy.pac`
 
Thus a system also use this script, of course there is the need to distribute
 
![](/en/configuration/network/proxy_config_automatic_distribution_dhcp_options_en.jpg)
 
it.
In addition there are two possibilities which we arrange in the course of this guidance.
The first possibility is the distribution over the DHCP-option. Therefore we put the following option under the*"network"* tab in the*"DHCP Server"* configuration.
 
`wpad code 252=text`
`wpad "http://IPFireIP:81/proxy.pac"`
 
## Let the browser automatically detect the Proxy configuration
 
In some cases, the supplies of the DHCP server could be ignored. However, mostly browsers will be able to automatically detect the proxy configuration. In such a case, you have to apply another http-vhost on port 80, which have in his Webroot only the proxy.pac and wpad.dat .
 
Therefor we initially create the directory*/srv/web/ipfire/wpad*
 
`mkdir /srv/web/ipfire/wpad`
 
and subsequently the file*/etc/httpd/conf/vhosts.d/wpad.conf*
 
`touch /etc/httpd/conf/vhosts.d/wpad.conf`
 
with the following content:
 
<box 90% round green|File: /etc/httpd/conf/vhosts.d/wpad.conf>
```
Listen 80
<VirtualHost *:80>
DocumentRoot "/srv/web/ipfire/wpad"
ServerName wpad.[localdomain]
Alias /wpad.dat /srv/web/ipfire/html/proxy.pac
Alias /proxy.pac /srv/web/ipfire/html/proxy.pac
</VirtualHost>
```
</box>
 
<note important>Please note that **[localdomain]** needs to be replaced by your own domain name!</note>
 
Thus, the directory also have the desired content, we create two links to the already existing proxy.pac / wpad.dat.
 
`ln -s /srv/web/ipfire/html/proxy.pac /srv/web/ipfire/html/proxy.pac`
 
`ln -s /srv/web/ipfire/html/wpad.dat /srv/web/ipfire/html/wpad.dat`
 
Afterwards we restart the Apache2 with:
 
`/etc/init.d/httpd restart`
 
Now there must be still created a additional entry under "Edit Hosts" (fundable under the tab*"network"* ->*"Edit Hosts"*) named **wpad** with the IPFire-IP, so that you get
 
![](/en/configuration/network/proxy_config_automatic_distribution_addhost_eng.jpg)
 
under
 
http://wpad.[localdomain]/proxy.pac and http://wpad.[localdomain]/wpad.dat
 
also the script.
 
Thus, both ways to distribute a proxy configuration automatically are established.
 
## Configuring exceptions
 
To configure centralized exceptions, we can create a file named*"src_no_proxy.acl"*.
 
`touch /var/ipfire/proxy/advanced/acls/src_no_proxy.acl`
 
In this we write in plain text, **only one per line**, the URLs or domain that should be called directly.
 
<box 90% round green|File: etc/httpd/conf/vhosts.d/wpad.conf>
```
*.ipfire.org
http://wiki.ipfire.org/de/configuration/network/proxy
```
</box>
 
<note warning>Please note here that each registered URL/domain are no longer runs through the proxy and thus the [URL-Filter](/de/configuration/network/url-filter) and [ClamAV](en/addons/clamav) does NOT check the pages.</note>
 
Then the following lines was integrated to the file*/var/web/ipfire/cgi-bin/proxy.cgi* .
 
<note warning>!!! Please make a backup copy of this file before you start because a built-in error leads to the problem that the complete "network" tab on the IPFire WUI can no longer be invoked. !!!</note>
 
To create a variable that we use later, we add at the beginning, where all the other variables are defined this line.
 
my $acl_src_noproxy = "$acldir/src_noproxy.acl";
 
<box 90% round green|File: /srv/web/ipfire/cgi-bin/proxy.cgi>
```
#!/usr/bin/perl
###############################################################################
# #
# IPFire.org - A linux based firewall #
# Copyright (C) 2007-2011 IPFire Team <info@ipfire.org> #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
# #
###############################################################################
#
# (c) 2004-2009 marco.s - http://www.advproxy.net
#
# This code is distributed under the terms of the GPL
#
# $Id: advproxy.cgi,v 3.0.2 2009/02/04 00:00:00 marco.s Exp $
#
 
use strict;
 
# enable only the following on debugging purpose
#use warnings;
#use CGI::Carp 'fatalsToBrowser';
 
require '/var/ipfire/general-functions.pl';
require "${General::swroot}/lang.pl";
require "${General::swroot}/header.pl";
 
my $acl_src_noproxy = "$acldir/src_noproxy.acl";
my @squidversion = `/usr/sbin/squid -v`;
my $http_port='81';
my $https_port='444';
```
</box>
 
Further down in the file we find the section "sub writepacfile". There we add the following lines after the first END;
 
<box 90% round green|Datei: around line 2915 /srv/web/ipfire/cgi-bin/proxy.cgi>
```
` if ($proxysettings{'ENABLE'} eq 'on')`
` {`
` print FILE "if (\n";`
 
` undef @templist;`
` if (-e "$acl_src_noproxy") {`
` open(NOPROXY,"$acl_src_noproxy");`
` @templist = <NOPROXY>;`
` close(NOPROXY);`
` chomp (@templist);`
` }`
 
` foreach (@templist)`
` {`
print FILE " (shExpMatch(url, \"*$_*\")) ||\n";
` }`
` print FILE <<END`
(shExpMatch(url, \"*ipfire.org*\"))\n
` )`
` return "DIRECT";`
 
`else`
 
END
;
 
```
</box>
 
When we added the rows successfully and saved the file once, we have to restart the proxy via the web interface.
(Make a click under the tab "network" -> "Webproxy" -> "Save and Restart")
 
At each restart, we initiate in such a way, a new **proxy.pac** will be created.
So if we change the file **/var/ipfire/proxy/advanced/acls/src_no_proxy.acl***, we need to trigger a restart of the proxy server.
 
The proxy.pac looks almost like this:
 
<box 90% round green|File: /srv/web/ipfire/html/proxy.pac>
```
function FindProxyForURL(url, host)
{
if (
` (isPlainHostName(host)) ||`
` (dnsDomainIs(host, ".lan.ipfire")) ||`
` (isInNet(host, "10.0.0.0", "255.0.0.0")) ||`
` (isInNet(host, "172.16.0.0", "255.240.0.0")) ||`
` (isInNet(host, "192.168.0.0", "255.255.0.0")) ||`
` (isInNet(host, "169.254.0.0", "255.255.0.0"))`
` )`
` return "DIRECT";`
 
else
 
if (
(shExpMatch(url, "*testdomain1.de*")) ||
(shExpMatch(url, "*testdomain2.de*")) ||
(shExpMatch(url, \"*ipfire.org*\"))
` )`
` return "DIRECT";`
 
else
 
if (
` (isInNet(myIpAddress(), "192.168.6.0", "255.255.255.0")) ||`
` (isInNet(myIpAddress(), "10.66.78.0", "24"))`
 
 
` )`
` return "PROXY 192.168.6.1:800";`
 
else
 
if (
` (isInNet(myIpAddress(), "192.168.61.0", "255.255.255.0"))`
` )`
` return "PROXY 192.168.61.1:800";`
}
```
</box>
 
<note important>!!! Please pay attention to the permissions of the files !!!</note>
 
`/var/ipfire/proxy/advanced/acls/src_no_proxy.acl`
`(nobody:nobody 644)`
 
`/srv/web/ipfire/cgi-bin/proxy.cgi`
`(root:root 755)`
 
So that was it already, now we can make also exceptions in the automatic Proxy configuration.
 
The IPFire team would like to say thanks for this Wiki to [WhyTea](http://forum.ipfire.org/index.php?action=profile;u=3565)