Coder Social home page Coder Social logo

squid_proxy's Introduction

Squid Proxy

Sample squid proxy and Dockerfile demonstrating various config modes.

The Dockerfile and git image compiles squid with ssl_crtd enabled which allows for SSL intercept and rewrite.

The corresponding docker image is on dockerhub:

The image has no entrypoint set to allow you to test and run different modes.

To run the image, simply invoke a shell in the container and start squid in the background for the mode you are interested in:

This image should be used for debugging and testing so i've left a lot of stuff in the dockerfile (eg, gcc, python, etc).

If you really want to use this in prod, modify the image to and minimize the footprint (use multistage builds, alpine, distroless images, etc)

docker run  -p 3128:3128 -ti docker.io/salrashid123/squidproxy /bin/bash

please note that the root CA's have been updated (on 1/9/22. You can find the docker image with the original certs as salrashid123/squidproxy:1 (or you can regenerate your own image from a prior commit))

The CA's provided currently are chained (root-ca.crt -> tls-ca.crt -> server_crt.pem. With the combined root and subordinate as tls-ca-chain.pem)

changelog

  • 11/20/22: Upgrade to debian-11, squid-5.7:

    • docker.io/salrashid123/squidproxy@sha256:latest
    • docker.io/salrashid123/squidproxy@sha256:
  • 1/10/22: Upgrade built in CA, squid-3.5.27:

    • docker.io/salrashid123/squidproxy@sha256:b46d3648443d675bb3ac020248495d5d7af1b7f3b683c3068e45c0f040aa5d9c

Also see

FORWARD

Explicit forward proxy mode intercepts HTTP traffic and uses CONNECT for https.

Launch:

$ /apps/squid/sbin/squid -NsY -f /apps/squid.conf.forward &

then in a new window run both http and https calls:

curl -v -x localhost:3128 -L http://httpbin.org/get
curl -v -x localhost:3128 -L https://httpbin.org/get

you should see a GET and CONNECT logs within the container

$ cat /apps/squid/var/logs/access.log
1668952181.370    112 192.168.9.1 TCP_MISS/301 1560 GET http://www.bbc.com/ - HIER_DIRECT/151.101.0.81 text/html
1668952181.517    146 192.168.9.1 TCP_TUNNEL/200 237935 CONNECT www.bbc.com:443 - HIER_DIRECT/151.101.0.81 -

You can also setup allow/deny rules for the domain:

If you want to use https_port, use squid.conf.https_port. For https_port see curl options like this:

curl -v --proxy-cacert tls-ca.crt  --resolve  squid.yourdomain.com:3128:127.0.0.1  -x https://squid.yourdomain.com:3128  https://httpbin.org/get

HTTPS INTERCEPT

In this mode, an HTTPS connection actually terminates the SSL connection on the proxy, then proceeds to download the certificate for the server you intended to visit. The proxy server then issues a new certificate with the same specifications of the site you wanted to visit and sends that down.

Essentially, the squid proxy is acting as man-in-the-middle. Ofcourse, you client needs to trust the certificate for the proxy but if not, you will see a certificate warning.

Here is the relevant squid conf setting to allow this:

squid.conf.intercept:

# Squid normally listens to port 3128
visible_hostname squid.yourdomain.com

http_port 3128 ssl-bump generate-host-certificates=on cert=/apps/tls-ca.crt key=/apps/tls-ca.key

always_direct allow all

acl excluded_sites ssl::server_name .wellsfargo.com
ssl_bump splice excluded_sites
ssl_bump bump all

sslproxy_cert_error deny all
sslcrtd_program /apps/squid/libexec/security_file_certgen -s /apps/squid/var/lib/ssl_db -M 4MB sslcrtd_children 8 startup=1 idle=1

The configuration above will insepct all SSL traffic but only splice traffic to wellsfargo.com to view its intended SNI (server_name). You can use the splice capability to apply ACL rules against without inspecting.

Launch

$ docker run  -p 3128:3128 -ti docker.io/salrashid123/squidproxy /apps/squid/sbin/squid -NsY -f /apps/squid.conf.intercept

then in a new window, try to access a secure site

$ wget https://raw.githubusercontent.com/salrashid123/squid_proxy/master/tls-ca.crt

$ curl -v --proxy-cacert tls-ca.crt --cacert tls-ca.crt -x localhost:3128  https://www.httpbin.org/get

you should see the proxy intercept and recreate httpbin's public certificate:

* Server certificate:
*  subject: CN=www.httpbin.org
*  start date: Jan  9 22:05:43 2022 GMT
*  expire date: Jan  9 22:05:43 2032 GMT
*  subjectAltName: host "www.httpbin.org" matched cert's "www.httpbin.org"
*  issuer: C=US; O=Google; OU=Enterprise; CN=Enterprise Subordinate CA       <<<<<<<<<
*  SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /get HTTP/1.1
> Host: www.httpbin.org
> User-Agent: curl/7.85.0
> Accept: */*

note the issuer is the proxy's server certificate (tls-ca.crt), NOT httpbin's official public cert

Now try to access www.wellsfargo.com. The configuration above simply views the SNI information without snooping on the data

$ curl -vvvv --proxy-cacert tls-ca.crt --cacert tls-ca.crt -x localhost:3128  https://www.wellsfargo.com

* Server certificate:
*  subject: jurisdictionC=US; jurisdictionST=Delaware; businessCategory=Private Organization; serialNumber=251212; C=US; ST=California; L=San Francisco; O=Wells Fargo & Company; CN=www.wellsfargo.com
*  start date: Aug  3 00:00:00 2022 GMT
*  expire date: Aug  3 23:59:59 2023 GMT
*  subjectAltName: host "www.wellsfargo.com" matched cert's "www.wellsfargo.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert EV RSA CA G2
*  SSL certificate verify ok.

Content Adaptation

content_adaptation/ allows you to not just intercept SSL traffic, but to actually rewrite the content both ways.

CACHE

Has cache enabled for HTTP traffic

Launch


$ /apps/squid/sbin/squid -NsY -f /apps/squid.conf.cache

Run two requests

curl -s -x localhost:3128 -L http://www.bbc.com/robots.txt
curl -s -x localhost:3128 -L http://www.bbc.com/robots.txt

First request is a TCP_MISS, the second is TCP_MEM_HIT

$ cat /apps/squid/var/logs/access.log
1669042557.206     75 192.168.9.1 TCP_MISS/200 20927 GET http://www.bbc.com/robots.txt - HIER_DIRECT/151.101.0.81 text/plain
1669042569.313      0 192.168.9.1 TCP_MEM_HIT/200 20935 GET http://www.bbc.com/robots.txt - HIER_NONE/- text/plain

Basic Auth

Enables squid proxy in default mode but requires a username password for the proxy

  • user: user1
  • password:user1

Launch:

$ /apps/squid/sbin/squid -NsY -f /apps/squid.conf.basicauth &
$ curl -v -x localhost:3128 --proxy-user user1:user1 -L http://httpbin.org/get

THe specific config for this mode:

squid.conf.basicaith

#user1:user1
#/apps/squid/squid_passwd:  user1:aje5nXwboMxWY
auth_param basic program /apps/squid/libexec/basic_ncsa_auth /apps/squid_passwd
acl authenticated proxy_auth REQUIRED
http_access allow authenticated
http_access deny all

Logs would show

1669042602.565     37 192.168.9.1 TCP_MISS/200 606 GET http://httpbin.org/get user1 HIER_DIRECT/34.203.186.29 application/json

Dockerfile

FROM debian:11 AS build
RUN apt-get -y update

RUN apt-get install -y curl supervisor git openssl  build-essential libssl-dev wget vim curl git python3 python3-pip procps
RUN mkdir -p /var/log/supervisor
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
WORKDIR /apps/
RUN wget -O - http://www.squid-cache.org/Versions/v5/squid-5.7.tar.gz | tar zxfv - \
    && CPU=$(( `nproc --all`-1 )) \
    && cd /apps/squid-5.7/ \
    && ./configure --prefix=/apps/squid --enable-icap-client --enable-ssl --with-openssl --enable-ssl-crtd --enable-auth --enable-basic-auth-helpers="NCSA" \
    && make -j$CPU \
    && make install \
    && cd /apps \
    && rm -rf /apps/squid-5.7
ADD . /apps/

RUN chown -R nobody:nogroup /apps/
RUN mkdir -p  /apps/squid/var/lib/
RUN /apps/squid/libexec/security_file_certgen -c -s /apps/squid/var/lib/ssl_db -M 4MB
RUN /apps/squid/sbin/squid -N -f /apps/squid.conf.cache -z
RUN chown -R nobody:nogroup /apps/
RUN chgrp -R 0 /apps && chmod -R g=u /apps
RUN ln -s /usr/bin/python3 /usr/bin/python

EXPOSE 3128
#CMD ["/usr/bin/supervisord"]

like i said, its a bit of a large, bloated image.

Generating new CA

THis repo and image comes with a built-in CA (root-ca.crt is the true parent CA that signed a subordinate ca tls-ca.crt (yes, i know, its confusing but i used that subca with that name)). You are free to generate and volume mount your own CA.

squid_proxy's People

Contributors

gilesbradshaw avatar mike-ensor avatar salrashid123 avatar stevesbrain avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

squid_proxy's Issues

Unable to run on OpenShift

Hi there,

I am unable to run this image as-is on OpenShift due to security related limitations. By default, OpenShift assigns its containers a random user ID at runtime. Since the files under /apps are owned by nobody (uid=65534) and are not globally writable, the container dies quickly after start up due to not being able to write to logs and cache locations. There are ways around it by configuring the pod to run with elevated permissions (covered in detail here), but this is frowned upon in production for security reasons.

This can be tested with docker by running docker run --user=1000 <remaining args..>. I was able to fix this issue for myself by writing the following simple Dockerfile:

FROM docker.io/salrashid123/squidproxy

RUN chmod -R 777 /apps/squid/var

Is this project interested in making this change directly to the original Dockerfile?

Using python's httpx module with intercept configuration

Running the docker container with /apps/squid/sbin/squid -NsY -f /apps/squid.conf.intercept and then running the following:

import httpx


proxy_url = "http://localhost:3128"
target_url = "https://www.httpbin.org"
cert_path = 'CA_crt.pem'


def run_request():
    with httpx.Client(proxies=proxy_url,verify=cert_path) as client:
        r = client.get(target_url)
        print(r.text)


if __name__ == '__main__':
    run_request()

I get httpx.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get issuer certificate (_ssl.c:1131) I suppose there is something wrong with the certificate retrieved with wget https://raw.githubusercontent.com/salrashid123/squid_proxy/master/CA_crt.pem.

A workaround would be to with httpx.Client(proxies=proxy_url,verify=False) as client:, however this is not recommended.
Any help fixing this would be much appreciated!

Cache for HTTPS

Could the last example be made to work for HTTPS as well?

I tried making some changes on my own (I hoped refresh_pattern ^https: would do it) but always get cache misses on Github downloads. I tested with a MS PowerShell release (65MB), which gets redirected to https://github-production-release-asset-2e65be.s3.amazonaws.com:443/*.deb file.

cache_mem 500 MB
cache_dir ufs /apps/squid/var/cache/squid 500 16 256

Question Docker Cache

Sorry for the stupid question.

I've ran as follows:

docker run  -p 3128:3128 -ti docker.io/salrashid123/squidproxy /bin/bash

Then

/apps/squid/sbin/squid -NsY -f /apps/squid.conf.cache

I have setup my docker proxy config as follows:

Web Server(HTTP)
http://localhost:3128

Secure Web Server(HTTPS)
http://localhost:3128

Bypass proxy settings for these hosts & domains
localhost,127.0.0.1,.local,192.168.

When I run

20:47 $ docker pull hello-world
Using default tag: latest
Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: TLS handshake timeout
✘-1 ~

My setup

20:51 $ docker version
Client: Docker Engine - Community
 Cloud integration  0.1.18
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 16:58:31 2020
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:07:04 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
✔ ~
20:51 $

Any ideas why I get the TLS handshake timeout?

My internet performance is good.

20:53 $ curl -s https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py | python -
Retrieving speedtest.net configuration...
Testing from BT (xxx.xxx.xxx.xxx)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by Sure Telecom (Douglas) [140.05 km]: 35.817 ms
Testing download speed................................................................................
Download: 92.33 Mbit/s
Testing upload speed......................................................................................................
Upload: 34.70 Mbit/s

provided CA_cert.pem untrusted - missing signing CA cert?

Hi team,

I am having troubles with testing the SSL intercept option with the provided CA_cert.pem file.

Is there a parent signing authority chain that I need reference in conjunction with this cert - what CA was used to sign it?

If I revert to self-signed CA cert, everything works fine. So I suspect I may be missing some intermediate/root ca certs if I am to leverage the supplied CA_cert.pem.

Thank you very much,
-igor

SQUID PROXY TRANSPARENT SOCKS

Hello, could you help me with an installation of Squid for transparent proxy that supports up to 60000 ports and SOCKS protocol?

stupid question about squid and apache2 ^^

H i @salrashid123 ,

Hope you are all well !

I am sorry about this question but I still do not get how to cache html results from a website hosted by an apache2 server.

Do you mind 1-2 minutes of your precious time to share some insights or inputs on that ?

Thanks in advance as I am a little bit lost on this one.

Cheers,
Luc Michalski

Squid cache directories not initialized

TODO: understand why squid cache directories are not initalized on startup:

eg

$ docker run  -p 3128:3128 -ti docker.io/salrashid123/squidproxy /bin/bash
root@39bad019df55:/apps# /apps/squid/sbin/squid -NsY -f /apps/squid.conf.cache

just terminates

squid[6]: Set Current Directory to /apps/squid/var/cache

however, if you init the cache folder inside the container, it works fine

$ docker run  -p 3128:3128 -ti docker.io/salrashid123/squidproxy /bin/bash
root@3616b779278b:/apps# /apps/squid/sbin/squid -z -f /apps/squid.conf.cache
2018/07/08 17:15:25 kid1| Set Current Directory to /apps/squid/var/cache
2018/07/08 17:15:25 kid1| Creating missing swap directories
2018/07/08 17:15:25 kid1| /apps/squid/var/cache/squid exists
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/00
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/01
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/02
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/03
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/04
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/05
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/06
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/07
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/08
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/09
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/0A
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/0B
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/0C
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/0D
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/0E
2018/07/08 17:15:25 kid1| Making directories in /apps/squid/var/cache/squid/0F
squid[8]: Squid Parent: (squid-1) process 10 exited with status 0

I've added in

 /apps/squid/sbin/squid -z -f /apps/squid.conf.cache

to the dockerfile but that didn't make a difference

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.