inComum

BASIC

1. place the inComum file in /usr/lib/squid

     cp. /inComum/usr/lib/squid/

2. make it executable

     chmod + x /usr/lib/squid/inComum

3. change the owner for the same squid (inComum  proxy.proxy):

     proxy.proxy chown /usr/lib/squid/inComum

4. Install script in squid.conf

5. configurations redirector that makes the equivalence of URL

storeurl_rewrite_program /​​usr/lib/squid/inComum
storeurl_rewrite_children 10 // depends on your load. Normally use 5-100.

6. acl to just send the URL redirector modifiable pro

acl store_rewrite_list dstdomain. youtube.com. googlevideo.com orkut.com. avast.com. catalog.video.msn.com. blip.tv
storeurl_access allow store_rewrite_list
storeurl_access deny all

7. remove a disservice Squid
8. COMMENT these lines (to disable them)

# acl QUERY urlpath_regex cgi-bin \?
# cache deny QUERY

9. put this down below

cache allow all

10. not expire youtube videos - currently (Feb/2010), expire under 2 hours.

ATTENTION: BEFORE putting this line the other lines refresh_pattern

refresh_pattern (get_video\?|videoplayback\?|videodownload\?) 5259487 999% 5259487 override-expire reload-into-ims stale-while-revalidate=2592000
refresh_pattern ^http:\/\/av\.vimeo\.com\/.\.(mp4|flv) 5259487 999% 5259487 override-expire reload-into-ims stale-while-revalidate=2592000
refresh_pattern ^http:\/\/.{0,2}\.video[0-9]\.blip\.tv.\.(flv|mp4|wmv|rm|ram|mov|avi|m4v|mp3) 5259487 999% 5259487 override-expire reload-into-ims stale-while-revalidate=2592000
refresh_pattern ^http://flashvideo.globo.com/*mp4 5259487 999% 5259487 override-expire reload-into-ims stale-while-revalidate=2592000
refresh_pattern (catalog.video.msn.com) 5259487 999% 5259487 override-expire reload-into-ims stale-while-revalidate=2592000
11. Remember to increase the size of your cache of objects.

     between 30 and 100MB is usually a good value

maximum_object_size 30 MB


to Read first

1. Prepare enviroment
Install needed files to compile.
From apt-get's man: "build-dep causes apt-get to install/remove packages in an attempt to satisfy the build dependencies for a source package."
# apt-get build-dep squid
2. Download Squid's sources
System download and apply patches.
# apt-get source squid
3. Edit compilations config on file ./debian/rules
# vim <squid source directory>/debian/rules
3.1. Find following lines inside 'rules' file:
# Configure the package.
config.status:
In my case (squid-2.7.STABLE9), they were on lines 132 e 133.
3.2. Add HTTP violations
On that point, you will find a serie of options. Add violations on it's middle.
My section became (notice line 166):
132 # Configure the package.
133 config.status:
134
135 rm cfgaux/config.sub cfgaux/config.guess
136 ln -s /usr/share/misc/config.sub cfgaux/
137 ln -s /usr/share/misc/config.guess cfgaux/
138
139 $(opt_ac_cv_func_setresuid) \
140 ./configure \
141 --prefix=/usr \
142 --exec_prefix=/usr \
143 --bindir=/usr/sbin --sbindir=/usr/sbin \
144 --libexecdir=/usr/lib/squid \
145 --sysconfdir=$(sysconfdir) \
146 --localstatedir=/var/spool/squid \
147 --datadir=/usr/share/squid \
148 $(with_pthreads) \
149 $(with_netfilter) \
150 $(with_arp_acl) \
151 $(with_epoll) \
152 --enable-removal-policies=lru,heap \
153 --enable-snmp \
154 --enable-delay-pools \
155 --enable-htcp \
156 --enable-cache-digests \
157 --enable-underscores \
158 --enable-referer-log \
159 --enable-useragent-log \
160 --enable-auth="basic,digest,ntlm,negotiate" \
161 --enable-negotiate-auth-helpers=squid_kerb_auth \
162 --enable-carp \
163 --enable-follow-x-forwarded-for \
164 --with-large-files \
165 --with-maxfd=65536 \
166 --enable-http-violations \
167 $(DEB_HOST_ARCH_CPU)-debian-$(DEB_HOST_ARCH_OS)
4. Make .deb files.
You must be on <squid source> directory. If you are still on <squid source>/debian, go up one level.
# dpkg-buildpackage
It will be generated a squid_<version>.deb and a squid-common_<version>.deb inside directory above <squid_source>.
5. Install generated debs.
# dpkg -i squid_2.7.STABLE9-2_i386.deb squid-common_2.7.STABLE9-2_all.deb
That's it. You will have now a much more powerfull Squid and you may finally use all the power of refresh pattern.

IMPORTANT
=========
Don't put on production before read this!
There is a possibility for your Squid to be imcompatible with this script.
When you initialize Squid, a process-father has the basic function to initialize process-children.
These shall receive and treat the requests.
In some versions, each child process initialize some instancy from sctipt called in storeurl_rewrite_program.
These children processes stay active until the parent process decides to remove them from memory.
In some versions of Squid, when children process are removed from memory, they do not remove the external program, letting it â??orphanâ?.
In younger versions, that does not happen.
You have to check if your version of Squid produces this unexpected result before putting it in production.
This can be done verifying how many processes run inComum:
$ ps ax|grep inComum
It is normal that the number is a little bit bigger or smaller than the number shown in storeurl_rewrite_children. It is a problem if this number keeps growing (it will take all your memory with ghosts processes).
Observe the system at the first day and check out its behavior.
If there are not large changes in quantity, your system is ready to have the script implanted.

INSTALL
=======
PRE-REQUISITES
--------------
=> Squid2 >= v2.7 (yet not ported for Squid3)
For better results, Squid must be compiled with --enable-http-violations directive.
Some versions may present problems. Read important.txt before install.
INSTALL
----------
1. Put file inComum in /usr/lib/squid and inComum.conf in /etc/squid/
# cp ./inComum /usr/lib/squid/
# cp ./inComum.conf /etc/squid/
2. Make it runnable
# chmod +x /usr/lib/squid/inComum
3. Change owner to the same squid runner (usually proxy.proxy):
# chown proxy.proxy /usr/lib/squid/inComum
# chown proxy.proxy /etc/squid/inComum.conf
4. Configure inComum.conf
For now, you may change the following options:
-storeurl_rewrite_children
-maximum_object_size
5. Install helper on squid.conf
To install add following line (without #) on squid.conf:
include /etc/squid/inComum.conf
IMPORTANT: above line MUST be inserted BEFORE any other config.
(Not exactly: must be before acl QUERY or any other refresh_pattern).