Endless login loop about traefik-forward-auth

[romracer]

I was able to get this working with Kubernetes. I run traefik-forward-auth in the same pod as my traefik ingress controller (I then run several of these in a deployment). Hence the usage of "localhost" in my forward URL. I also did not need the auth-trust-headers annotation but maybe your case is different.

The key is applying the auth-* annotations to the auth ingress as it's the auth forwarder module in Traefik which provides the X-Forwarded-Uri. Without the annotations, the auth ingress misses the forwarded URL and ends up in a loop.

You'll notice there is no traefik.toml. Everything is accomplished with annotations, command line arguments and secrets/environment variables.

My auth host ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  labels:
    app: traefik-ingress-lb
  name: traefik-auth-ingress
  namespace: traefik-ingress
  annotations:
    kubernetes.io/ingress.class: "traefik"
    ingress.kubernetes.io/ssl-redirect: "true"
    ingress.kubernetes.io/auth-type: "forward"
    ingress.kubernetes.io/auth-url: "http://localhost:4181"
    ingress.kubernetes.io/auth-response-headers: "X-Forwarded-User"
spec:
  rules:
  - host: auth.mydomain.com
    http:
      paths:
      - path: /auth/_oauth
        backend:
          serviceName: traefik
          servicePort: 4181
  tls:
  - hosts:
    - auth.mydomain.com
    secretName: traefik-auth-cert

A protected service example (Traefik's dashboard)

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  labels:
    app: traefik-ingress-lb
  name: traefik-dashboard-ingress
  namespace: traefik-ingress
  annotations:
    kubernetes.io/ingress.class: "traefik"
    ingress.kubernetes.io/ssl-redirect: "true"
    ingress.kubernetes.io/auth-type: "forward"
    ingress.kubernetes.io/auth-url: "http://localhost:4181"
    ingress.kubernetes.io/auth-response-headers: "X-Forwarded-User"
spec:
  rules:
  - host: traefik.mydomain.com
    http:
      paths:
      - backend:
          serviceName: traefik
          servicePort: 8080
  tls:
  - hosts:
    - traefik.mydomain.com
    secretName: traefik-cert

Here's a very stripped down deployment example:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: traefik-ingress-lb
  name: traefik-ingress-controller
  namespace: traefik-ingress
spec:
  selector:
    matchLabels:
      app: traefik-ingress-lb
  replicas: 2
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 50%
      maxUnavailable: 50%
  template:
    metadata:
      labels:
        app: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - traefik-ingress-lb
            topologyKey: "kubernetes.io/hostname"
      containers:
        - name: traefik-ingress-lb
          image: traefik:v1.7.8
          args:
            - --defaultentrypoints=http,https
            - --entrypoints=Name:http Address::80 Compress:true Redirect.EntryPoint:https
            - --entrypoints=Name:https Address::443 TLS:/ssl/tls.crt,/ssl/tls.key TLS.MinVersion:VersionTLS12 TLS.DefaultCertificate.Cert:/ssl/tls.crt TLS.DefaultCertificate.Key:/ssl/tls.key
            - --entrypoints=Name:api Address::8080
            - --kubernetes
            - --kubernetes.ingressclass=traefik
            - --api
            - --api.entrypoint=api
            - --api.dashboard=true
          ports:
            - name: http
              containerPort: 80
            - name: https
              containerPort: 443
            - name: dashboard
              containerPort: 8080
          volumeMounts:
            - name: ssl
              mountPath: /ssl
        - name: traefik-forward-auth
          image: thomseddon/traefik-forward-auth:latest
          imagePullPolicy: Always
          env:
            - name: CLIENT_ID
              valueFrom:
                secretKeyRef:
                  name: traefik-env
                  key: oauth-client-id
            - name: CLIENT_SECRET
              valueFrom:
                secretKeyRef:
                  name: traefik-env
                  key: oauth-client-secret
            - name: SECRET
              valueFrom:
                secretKeyRef:
                  name: traefik-env
                  key: oauth-secret
            - name: LOG_LEVEL
              value: info
            - name: AUTH_HOST
              value: auth.mydomain.com
            - name: URL_PATH
              value: auth/_oauth
            - name: COOKIE_DOMAINS
              value: mydomain.com
            - name: WHITELIST
              value: [email protected]
          ports:
            - name: http-auth
              containerPort: 4181
      volumes:
        - name: ssl
          secret:
            secretName: traefik-default-cert

And finally a service definition to power our ingress backend services:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: traefik-ingress-lb
  annotations:
    traefik.backend.loadbalancer.stickiness: "true"
  name: traefik
  namespace: traefik-ingress
spec:
  ports:
    - name: dashboard
      port: 8080
    - name: http-auth
      port: 4181
  selector:
    app: traefik-ingress-lb

The only allowed redirect URL in my Google Developer console is: https://auth.mydomain.com/auth/_oauth

It took quite a bit of fiddling and research to get this working; feel free to include it as an example somewhere if it's useful to others.

[maximede]

I was also having an authentication loop issue using AUTH_HOST with docker-compose. I solved it and thought I'd post my solution here since this is one of the top post when you search for this issue on Google :) .
In my case, the problem was due to the fact that I didn't enabled traefik-forward-auth.middlewares using traefik-forward-auth globally for my https entrypoint, I was under the assumption that I could make it work by only adding it to the service I wanted authenticated, and that was a mistake.
The traefik-forward-auth.middlewares also needs to be set on the traefik-forward-auth service.

something like this :

  traefik-forward-auth:
    image: thomseddon/traefik-forward-auth:2
    container_name: traefik-forward-auth
    environment:
      - PROVIDERS_GOOGLE_CLIENT_ID=clientid
      - PROVIDERS_GOOGLE_CLIENT_SECRET=secret      
      - SECRET=secret      
      - COOKIE_DOMAIN=$DOMAIN      
      - AUTH_HOST=auth.$DOMAIN
      - LOG_LEVEL=debug
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik-forward-auth.rule=Host(`auth.$DOMAIN`)"
      - "traefik.http.routers.traefik-forward-auth.entrypoints=websecure"
      - "traefik.http.routers.traefik-forward-auth.tls=true"
      - "traefik.http.routers.traefik-forward-auth.middlewares=traefik-forward-auth"
      - "traefik.http.middlewares.traefik-forward-auth.forwardauth.address=https://auth.$DOMAIN"
      - "traefik.http.middlewares.traefik-forward-auth.forwardauth.address=http://traefik-forward-auth:4181"
      - "traefik.http.middlewares.traefik-forward-auth.forwardauth.authResponseHeaders=X-Forwarded-User"

  whoami:
    image: containous/whoami
    container_name: whoami
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.whoami.rule=Host(`whoami-$DOMAIN`)"
      - "traefik.http.routers.whoami.entrypoints=websecure"
      - "traefik.http.routers.whoami.tls=true"
      - "traefik.http.routers.whoami.middlewares=traefik-forward-auth"

Maybe pointing in the docs that the forwardauth middleware needs to be applied to the traefik-forward-auth service too would be good ?

[HedgeShot]

@thomseddon It is not self explanatory as I am trying to setup AUTH_HOST. It indeed works if I add hydra.DOMAIN.COM/_oauthin Google but that is not what I want. It is a bit annoying to add every subdomain to Google but if that is what it takes I can live with it.

[thomseddon]

There doesn't seem to be a X-Forwarded-Uri header when the request hits ssoauth.mydomain.com, so the server doesn't know to call the auth callback.

Can you post full toml config?

I'm haven't looked into the way these variables are parsed, but it might be worth trying:

ingress.kubernetes.io/auth-trust-headers: true

as opposed to:

ingress.kubernetes.io/auth-trust-headers: "true"

**also, could you post the annotations for the prometheus.mydomain.com container as this correctly passes X-Forwarded-Uri

[devster31]

The traefik-forward-auth.middlewares also needs to be set on the traefik-forward-auth service.

This is indeed a solution that worked for me too, however I suspect this masks something deeper.
Looking at how traefik represents traffic:

it seems that adding the middleware is allowing a successful redirect that otherwise isn't working.
I initially thought the "callback URL" on GitHub had to be https://auth.domain.com/_oauth with the explicit path, but it seems to work without.
Any thoughts @thomseddon ?

Here the log from my side:

traefik-forward-auth_1  | time="2021-08-08T23:23:56Z" level=debug msg="Authenticating request" cookies="other_cookies; _forward_auth_csrf=398c6ddafd150f4267dc8455639eb26c]" handler=Auth host=whoami.domain.com method=GET proto=https rule=default source_ip=150.150.150.150 uri=/
traefik-forward-auth_1  | time="2021-08-08T23:23:56Z" level=debug msg="Set CSRF cookie and redirected to provider login url" csrf_cookie="_forward_auth_csrf=c12a92bbfde54ae8acb99278455357ed; Path=/; Domain=domain.com; Expires=Mon, 09 Aug 2021 11:23:56 GMT; HttpOnly; Secure" handler=Auth host=whoami.domain.com login_url="https://github.com/login/oauth/authorize?client_id=a4f9be1831eccc6e2cce&redirect_uri=https%3A%2F%2Fauth.domain.com%2F_oauth&response_type=code&scope=profile+email&state=c12a92bbfde54ae8acb99278455357ed%3Ageneric-oauth%3Ahttps%3A%2F%2Fwhoami.domain.com%2F" method=GET proto=https rule=default source_ip=150.150.150.150 uri=/
traefik-forward-auth_1  | time="2021-08-08T23:23:57Z" level=debug msg="Handling callback" cookies="other_cookies; _forward_auth_csrf=c12a92bbfde54ae8acb99278455357ed]" handler=AuthCallback host=auth.domain.com method=GET proto=https rule=default source_ip=150.150.150.150 uri="/_oauth?code=2b3aeb15259366fe073e&state=c12a92bbfde54ae8acb99278455357ed%3Ageneric-oauth%3Ahttps%3A%2F%2Fwhoami.domain.com%2F"
traefik-forward-auth_1  | time="2021-08-08T23:23:58Z" level=info msg="Successfully generated auth cookie, redirecting user." handler=AuthCallback host=auth.domain.com method=GET proto=https provider=generic-oauth redirect="https://whoami.domain.com/" rule=default source_ip=150.150.150.150 uri="/_oauth?code=2b3aeb15259366fe073e&state=c12a92bbfde54ae8acb99278455357ed%3Ageneric-oauth%3Ahttps%3A%2F%2Fwhoami.domain.com%2F" [email protected]
traefik-forward-auth_1  | time="2021-08-08T23:23:58Z" level=debug msg="Authenticating request" cookies="other_cookies; _forward_auth=source_ip= B_DUovzYcDqcWZ5PSDRoJic3RCxbEzaZTilCUP=|1628508238|[email protected]]" handler=Auth host=whoami.domain.com method=GET proto=https rule=default source_ip=150.150.150.150 uri=/
traefik-forward-auth_1  | time="2021-08-08T23:23:58Z" level=debug msg="Allowing valid request" handler=Auth host=whoami.domain.com method=GET proto=https rule=default source_ip=150.150.150.150 uri=/
traefik-forward-auth_1  | time="2021-08-08T23:23:59Z" level=debug msg="Authenticating request" cookies="other_cookies; _forward_auth=source_ip= B_DUovzYcDqcWZ5PSDRoJic3RCxbEzaZTilCUP=|1628508238|[email protected]]" handler=Auth host=whoami.domain.com method=GET proto=https rule=default source_ip=150.150.150.150 uri=/favicon.ico
traefik-forward-auth_1  | time="2021-08-08T23:23:59Z" level=debug msg="Allowing valid request" handler=Auth host=whoami.domain.com method=GET proto=https rule=default source_ip=150.150.150.150 uri=/favicon.ico

[thomseddon]

Can you send your compose setup and config file (if you have one)

[HedgeShot]

Sure, here you go:

version: "3.6"
services:

######### FRONTENDS ##########

# Traefik Reverse Proxy
  traefik:
    hostname: traefik
    image: traefik:latest
    container_name: traefik
    restart: unless-stopped
    domainname: ${DOMAINNAME}
    networks:
      - default
      - traefik_proxy
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    environment:
      - NAMECOM_USERNAME=${NAMECOM_USERNAME}
      - NAMECOM_API_TOKEN=${NAMECOM_API_TOKEN}
      - NAMECOM_SERVER=${NAMECOM_SERVER}
    labels:
      - "traefik.enable=true"
      - "traefik.backend=traefik"
      - "traefik.frontend.rule=Host:traefik.${DOMAINNAME}"
      - "traefik.port=8080"
      - "traefik.docker.network=traefik_proxy"
      - "traefik.frontend.headers.SSLRedirect=true"
      - "traefik.frontend.headers.STSSeconds=315360000"
      - "traefik.frontend.headers.browserXSSFilter=true"
      - "traefik.frontend.headers.contentTypeNosniff=true"
      - "traefik.frontend.headers.forceSTSHeader=true"
      - "traefik.frontend.headers.SSLHost=DOMAIN.COM"
      - "traefik.frontend.headers.STSIncludeSubdomains=true"
      - "traefik.frontend.headers.STSPreload=true"
      - "traefik.frontend.headers.frameDeny=true"
      - "traefik.frontend.auth.basic.usersFile=/shared/.htpasswd"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ${USERDIR}/docker/traefik:/etc/traefik
      - ${USERDIR}/docker/shared:/shared

# NZBHydra – NZB Meta Search
  hydra:
    image: "linuxserver/hydra"
    container_name: "hydra"
    volumes:
      - ${USERDIR}/docker/hydra:/config
      - /nfs/Multimedia/Downloads:/downloads
      - ${USERDIR}/docker/shared:/shared
    restart: unless-stopped
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
    networks:
      - traefik_proxy
    labels:
      - "traefik.enable=true"
      - "traefik.backend=hydra"
      - "traefik.frontend.rule=Host:hydra.${DOMAINNAME}, hydra.localhost"
      - "traefik.port=5075"
      - "traefik.docker.network=traefik_proxy"
      - "traefik.frontend.headers.SSLRedirect=true"
      - "traefik.frontend.headers.STSSeconds=315360000"
      - "traefik.frontend.headers.browserXSSFilter=true"
      - "traefik.frontend.headers.contentTypeNosniff=true"
      - "traefik.frontend.headers.forceSTSHeader=true"
      - "traefik.frontend.headers.SSLHost=DOMAIN.COM"
      - "traefik.frontend.headers.STSIncludeSubdomains=true"
      - "traefik.frontend.headers.STSPreload=true"
      - "traefik.frontend.headers.frameDeny=false"
      #- "traefik.frontend.auth.basic.usersFile=/shared/.htpasswd"
      - "traefik.frontend.auth.forward.address=http://forward-oauth:4181"

# Simple oauth (with google)
  forward-oauth:
    image: thomseddon/traefik-forward-auth
    container_name: "forward-oauth"
    environment:
      - CLIENT_ID=${GOOGCLIENTID}
      - CLIENT_SECRET=${GOOGCLIENTSEC}
      - SECRET=${COOKIESECRET}
      - COOKIE_SECURE=false
      - COOKIE_DOMAINS=DOMAIN.COM
      - DOMAIN=gmail.com
      - AUTH_HOST=auth.DOMAIN.COM
    networks:
      - traefik_proxy
    labels:
      - "traefik.enable=true"
      - "traefik.backend=forward-oauth"
      - "traefik.frontend.rule=Host:auth.DOMAIN.COM"
      - "traefik.port=4181"
      - "traefik.docker.network=traefik_proxy"
      - "traefik.frontend.headers.SSLRedirect=true"
      - "traefik.frontend.headers.STSSeconds=315360000"
      - "traefik.frontend.headers.browserXSSFilter=true"
      - "traefik.frontend.headers.contentTypeNosniff=true"
      - "traefik.frontend.headers.forceSTSHeader=true"
      - "traefik.frontend.headers.SSLHost=DOMAIN.COM"
      - "traefik.frontend.headers.STSIncludeSubdomains=true"
      - "traefik.frontend.headers.STSPreload=true"
      - "traefik.frontend.headers.frameDeny=false"
######### UTILITIES ##########

networks:
  traefik_proxy:
    external:
      name: traefik_proxy
  default:
    driver: bridge

Traefik.toml:

# WEB interface of Traefik - it will show web page with overview of frontend and backend configurations
#[web]
#address = ":8080"
#  [web.auth.basic]
#  usersFile = "/shared/.htpasswd"

[api]
  entryPoint = "traefik"
  dashboard = true
  address = ":8080"
  usersFile = "/shared/.htpasswd"

# Force HTTPS
[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
  #  [entryPoints.https.auth.basic]
  #  usersFile = "/shared/.htpasswd"
    [entryPoints.https.tls]

[file]
# directory = "/etc/traefik/rules/"
  watch = true
  filename = "/etc/traefik/rules.toml"

# Let's encrypt configuration
[acme]
email = "[email protected]"
storage="/etc/traefik/acme/acme.json"
entryPoint = "https"
acmeLogging=true
onDemand = false #create certificate when container is created
#onHostRule = true #disable for wildcard domains
# Use a HTTP-01 acme challenge rather than TLS-SNI-01 challenge
#[acme.httpChallenge] #does not work with wildcard domains, use dns challenge
#  entryPoint = "http" #does not work with wildcard domains
[acme.dnsChallenge]
  provider = "namedotcom"
  delayBeforeCheck = 120
[[acme.domains]]
   main = "DOMAIN.COM"


# Connection to docker host system (docker.sock)
[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "DOMAIN.COM"
watch = true
# This will hide all docker containers that don't have explicitly
# set label to "enable"
exposedbydefault = false

[thomseddon]

Try removing the AUTH_HOST variable to check it works without?

[HedgeShot]

If I do that, I am getting this error in the browser (from Google):

Error: redirect_uri_mismatch

The redirect URI in the request, https://hydra.DOMAIN.COM/_oauth, does not match the ones authorized for the OAuth client. To update the authorized redirect URIs, visit: 

[mliudvikas]

Hi

since you're using forward auth per frontend, you need to add the

• "traefik.frontend.auth.forward.address=http://forward-oauth:4181"

label to your forward-oauth container as well.

[thomseddon]

Fortunately, that error is quite self explanatory, please follow the instructions in the README and allow https://hydra.DOMAIN.COM/_oauth via the google console.

Once this is done I think it should work.

We also added a new feature this week so you can limit login to specific users via the whitelist option , this should be preferable as your current config would allow any gmail.com user.

[thomseddon]

Closing as this doesn't look's like a setup issue as opposed to a bug, happy to continue to help whilst closed

[thomseddon]

Okie dokie, can you re-read the auth host setup: https://github.com/thomseddon/traefik-forward-auth#auth-host

If it it all looks correct, please capture the response from auth.DOMAIN.com and check what domain the _forward_auth cookie is being set on? It should be DOMAIN.com

[billimek]

I think I'm seeing this behavior too when attempting to use the AUTH_HOST approach.

In the _forward_auth_csrf cookie, I see that the Domain is set to .auth.mydomain.com. I run the docker image with -e COOKIE_DOMAINS="mydomain.com" & -e AUTH_HOST="auth.mydomain.com" where auth.mydomain.com is the traefik-forward-auth service.

[materemias]

@thomseddon I face the very same issue with a kubernetes setup

SECRET | something-random-1456454
DOMAIN | mydomain.com
COOKIE_SECURE | true
COOKIE_DOMAINS | myotherdomain.com
CLIENT_SECRET | dnhYvMstjustrandomthings
CLIENT_ID | 123456789-yxcvasdfqwer.apps.googleusercontent.com
AUTH_HOST | ssoauth.myotherdomain.com

since I would like to cover many hosts auth_host mode would be desired. could you please give me a hint?

[materemias]

it looks to me like there were no X-Forwarded-Uri coming back
this is what _oauth gets:

GET https://ssoauth.mydomain.com/_oauth?state=609fcb7c8f965456985a1520d:https://myorigin.domain.com/&code=4/zgDrIGDFVAv21SvkY-ZY-6csrwVFmOatalYzrsCrblEXAcDDPbrFj4Mt34Lmw6dqFIAreGFv0CTHtN1M&scope=email%20profile%20https://www.googleapis.com/auth/userinfo.email%20https://www.googleapis.com/auth/userinfo.profile
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Referer: https://accounts.google.com/signin/oauth/oauthchooseaccount?client_id=109257456456456aopb.apps.googleusercontent.com&as=7dFny456stination=https%3A%2F%2Fssoauth.mydomain.com&approval_state=!ChRiUGp3QllxdUFjNGFHbEpTRU9MNxIfSS1zN0pBbHpENzRWOERFgfGFDG1GMnZtVmd4WQ%E2%88%99APNbktkAAAAAXDj3SniSgJFDuhWoDFPUrrN3d7McK25t&oauthgdpr=1&xsrfsig=ChkAeAh8T8WPgfh6mg9MTlXQ6q9sSX02Eg5hcHByb3ZhbF9zdGF0ZRILZGVzdGluYXRpb24SBXNvYWN1Eg9vYXV0aHJpc2t5c2NvcGU&flowName=GeneralOAuthFlow
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9,hu-HU;q=0.8,hu;q=0.7,en-US;q=0.6
Cookie: __cfduid=d7743c12ea58bb8cd44bbf65f808ec4194845886740; _forward_auth_csrf=609fcb7c8f4484645c195321b985a1520d

[thomseddon]

Can you send me a copy of the responses headers for the initial request (should be forwarding to Google) then each subsequent response before you end up back at Google? i.e. the entire request loop.

If you could then also send a copy of your config where you've masked the domains the same way is they're masked in the requests?

I think it may actually be a documentation issue, but once I can see where it's going wrong I should be able to provide a quick fix :)

[materemias]

@thomseddon here is the complete log of the requests

these are the ENV vars I use (with same mask):

AUTH_HOST | ssoauth.mydomain.com
CLIENT_ID | 1092577527532-123456789yxvasdfqwlrhs00u932aopb.apps.googleusercontent.com
CLIENT_SECRET | dnhYvMstjustrandomthings
COOKIE_DOMAINS | mydomain.com    
COOKIE_NAME | _cc_auth
COOKIE_SECURE | true
DOMAIN | myotherdomain.com      //domain of allowed email addresses, different from mydomain.com
SECRET | something-random-1456454

[materemias]

@thomseddon could you please have a look on this?

[thomseddon]

@materemias can you post your traefik config? It looks like it's not recognising that it's hit the callback path, which may be because trustForwardHeader may be disabled?

[materemias]

@thomseddon it is a kubernetes install, for the given service I use these annotations:

    ingress.kubernetes.io/auth-response-headers: X-Auth-User, X-Secret
    ingress.kubernetes.io/auth-tls-insecure: "true"
    ingress.kubernetes.io/auth-trust-headers: "true"
    ingress.kubernetes.io/auth-type: forward
    ingress.kubernetes.io/auth-url: https://ssoauth.mydomain.com/_oauth
    kubernetes.io/ingress.class: traefik

[thomseddon]

Can you add:

ingress.kubernetes.io/auth-trust-headers: true

It looks like traefik will only add the X-Forwarded-Uri header if this option is enabled: https://github.com/containous/traefik/blob/master/middlewares/auth/forward.go#L198

I'm not sure if this is the intended consequence of this option though, I might raise it with traefik

[materemias]

I already have that!

[thomseddon]

Ah, so you do, my bad :)

I'm going to fix #18 so we can see what's going on inside

[materemias]

makes sense, thanks for the info!

[thomseddon]

@materemias I've just pushed a version with refactored logging, hopefully this will show us what we need! If you could run with log-level debug and let's see what we get!

[materemias]

great news, this is a redirect loop:

time="2019-01-22T13:53:55Z" level=debug msg="Set CSRF cookie and redirecting to google login" RemoteAddr="10.0.0.1:63628"
time="2019-01-22T13:53:55Z" level=debug msg="Handling request" Headers="map[User-Agent:[GoogleHC/1.0] Connection:[Keep-alive]]" RemoteAddr="10.0.0.1:47874"
time="2019-01-22T13:53:55Z" level=debug msg="Set CSRF cookie and redirecting to google login" RemoteAddr="10.0.0.1:47874"
time="2019-01-22T13:53:56Z" level=debug msg="Handling request" Headers="map[User-Agent:[GoogleHC/1.0] Connection:[Keep-alive]]" RemoteAddr="10.0.0.1:35476"
time="2019-01-22T13:53:56Z" level=debug msg="Set CSRF cookie and redirecting to google login" RemoteAddr="10.0.0.1:35476"
time="2019-01-22T13:54:02Z" level=debug msg="Handling request" Headers="map[Cf-Ipcountry:[US] X-Forwarded-For:[84.174.249.70, 10.0.0.1,32.211.152.141, 10.0.0.1] X-Forwarded-Uri:[/] User-Agent:[Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36] Cf-Visitor:[{\"scheme\":\"https\"}] Cookie:[__cfduid=d188c3779f2c63b26460221f476ead15e1546593046; _forward_auth_csrf=bf6b407c035e386234fc22647339dfd2] X-Real-Ip:[10.0.0.1] Accept-Encoding:[gzip] Accept-Language:[en-GB,en;q=0.9,hu-HU;q=0.8,hu;q=0.7,en-US;q=0.6] Cf-Connecting-Ip:[32.211.152.141] X-Forwarded-Host:[prometheus.mydomain.com] X-Forwarded-Port:[443] X-Forwarded-Proto:[https] X-Forwarded-Server:[traefik-bf9f79b5d-gfdd9] Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8] Cdn-Loop:[cloudflare; subreqs=1] Cf-Ray:[49d2829e3b229762-FRA] Upgrade-Insecure-Requests:[1] X-Forwarded-Method:[GET]]" RemoteAddr="10.0.0.77:54640"
time="2019-01-22T13:54:02Z" level=debug msg="Set CSRF cookie and redirecting to google login" RemoteAddr="10.0.0.77:54640"
time="2019-01-22T13:54:06Z" level=debug msg="Handling request" Headers="map[X-Forwarded-Server:[traefik-bf9f79b5d-gfdd9] Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8] Accept-Encoding:[gzip] Cdn-Loop:[cloudflare] Cf-Ipcountry:[HU] Referer:[https://accounts.google.com/signin/oauth/oauthchooseaccount?client_id=1092577486532-ht6e3ug7jklt8cqup7lrhs00u932aopb.apps.googleusercontent.com&as=JqYm0MLa_PnJk7uhNAm29g&destination=https%3A%2F%2Fssoauth.mydomain.com&approval_state=!ChRWOVpJMV9DRC1lZ0l4ZjNXQlRZdRIfay1RaURsS2s5allVOERFdWhZOThQYzhWLUcxZGh4WQ%E2%88%99APNbktkAAAAAXEhx-p59fQXmHOtUepEPDlBVNBtLGveZ&oauthgdpr=1&xsrfsig=ChkAeAh8T6nTvumaRl_8trcE6fincuGDMAcVEg5hcHByb3ZhbF9zdGF0ZRILZGVzdGluYXRpb24SBXNvYWN1Eg9vYXV0aHJpc2t5c2NvcGU&flowName=GeneralOAuthFlow] X-Forwarded-Host:[ssoauth.mydomain.com] Upgrade-Insecure-Requests:[1] X-Forwarded-For:[84.174.249.70, 10.0.0.1] X-Real-Ip:[10.0.0.1] Cf-Connecting-Ip:[84.174.249.70] Cf-Ray:[49d282b5cbab8c88-VIE] Cf-Visitor:[{\"scheme\":\"https\"}] Cookie:[__cfduid=d188c3779f2c63b26460221f476ead15e1546593046; _forward_auth_csrf=07d91868cbf2f07f7042af430912bcda] User-Agent:[Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36] Accept-Language:[en-GB,en;q=0.9,hu-HU;q=0.8,hu;q=0.7,en-US;q=0.6] X-Forwarded-Port:[443] X-Forwarded-Proto:[https]]" RemoteAddr="10.0.0.77:54640"
time="2019-01-22T13:54:06Z" level=debug msg="Set CSRF cookie and redirecting to google login" RemoteAddr="10.0.0.77:54640"
time="2019-01-22T13:54:10Z" level=debug msg="Handling request" Headers="map[User-Agent:[GoogleHC/1.0] Connection:[Keep-alive]]" RemoteAddr="10.156.0.15:35634"
time="2019-01-22T13:54:10Z" level=debug msg="Set CSRF cookie and redirecting to google login" RemoteAddr="10.156.0.15:35634"
time="2019-01-22T13:54:10Z" level=debug msg="Handling request" Headers="map[User-Agent:[GoogleHC/1.0] Connection:[Keep-alive]]" RemoteAddr="10.156.0.15:62502"
time="2019-01-22T13:54:10Z" level=debug msg="Set CSRF cookie and redirecting to google login" RemoteAddr="10.156.0.15:62502"
time="2019-01-22T13:54:10Z" level=debug msg="Handling request" Headers="map[User-Agent:[GoogleHC/1.0] Connection:[Keep-alive]]" RemoteAddr="10.156.0.15:65276"
time="2019-01-22T13:54:10Z" level=debug msg="Set CSRF cookie and redirecting to google login" RemoteAddr="10.156.0.15:65276"

[materemias]

these are the annotations I am trying to get this working with:

    ingress.kubernetes.io/auth-response-headers: X-Auth-User, X-Secret
    ingress.kubernetes.io/auth-tls-insecure: "true"
    ingress.kubernetes.io/auth-trust-headers: "true"
    ingress.kubernetes.io/auth-type: forward
    ingress.kubernetes.io/auth-url: https://ssoauth.mydomain.com/_oauth
    kubernetes.io/ingress.class: traefik

true needs to be passed as string.

traefik.toml:

# traefik.toml
logLevel = "INFO"
defaultEntryPoints = ["http","https"]
[entryPoints]
[entryPoints.http]
address = ":80"
compress = true
[entryPoints.http.forwardedHeaders]
trustedIPs = ["1.2.3.4, "127.0.0.1/32", "10.0.0.1/32"]
[entryPoints.http.redirect]
regex = "^http://(.*)"
replacement = "https://$1"
[entryPoints.https]
address = ":443"
compress = true
[entryPoints.https.forwardedHeaders]
trustedIPs = ["1.2.3.4", "127.0.0.1/32", "10.0.0.1/32"]
[entryPoints.https.tls]
[[entryPoints.https.tls.certificates]]
CertFile = "/ssl/tls.crt"
KeyFile = "/ssl/tls.key"
[entryPoints.traefik]
address = ":8080"
[kubernetes]
ingressClass = "traefik"
[kubernetes.ingressEndpoint]
publishedService = "traefik/traefik"
[traefikLog]
format = "json"
[accessLog]
format = "common"
[accessLog.fields]
defaultMode = "keep"
[accessLog.fields.names]
[accessLog.fields.headers]
defaultMode = "keep"
[accessLog.fields.headers.names]
[acme]
email = "[email protected]"
storage = "/acme/acme.json"
entryPoint = "https"
onHostRule = true
[acme.dnsChallenge]
provider = "cloudflare"
[api]
entryPoint = "traefik"
dashboard = true
[metrics]
[metrics.prometheus]
entryPoint = "traefik"

[materemias]

and here are the available annotations

[materemias]

@thomseddon do you have an idea how to proceed with this?

[thomseddon]

I've flagged this as one to look at when we improve the kubernetes support/documentation!

Please let me know if the above does not work!

[dalanmiller]

Getting this same loop using Traefik v2.0:
traefik.toml

[global]
  Debug = true
  SendAnonymousUsage = false

[accessLog]

[api]

[providers]
  [providers.file]
    watch=true
    
  [providers.docker]
    endpoint = "unix:///var/run/docker.sock"
    watch = true
    exposedByDefault = false

[entrypoints]
  [entrypoints.http]
    address = ":80"
  [entrypoints.https]
    address = ":443"

[http.routers]
  [http.routers.synology]
      entryPoints = ["https"]
      middlewares = ["forward-auth"]
      rule = "Host(`synology.sub.domain.com`)"
      service = "synology"
      [http.routers.synology.tls]

[http.middlewares]
  [http.middlewares.forward-auth.forwardauth]
        address = "http://traefik-forward-auth:4181"
        authResponseHeaders = ["X-Forwarded-User"]
        trustForwardHeader = true

docker-compose.yml

traefik-forward-auth:
    container_name: traefik-forward-auth
    image: thomseddon/traefik-forward-auth
    environment:
      - CLIENT_ID=client_id

      - CLIENT_SECRET=client_secret
      - SECRET=secret

      - AUTH_HOST=auth.sub.domain.com
      - COOKIE_DOMAIN=sub.domain.com

      - INSECURE_COOKIE=true

      - [email protected]

      - LOG_LEVEL=debug

    # When using an auth host, adding it here prompts traefik to generate certs
    labels:
      - traefik.enable=true

      - "traefik.http.routers.traefik-forward-auth.rule=Host(`auth.sub.domain.com`)"
      - "traefik.http.routers.traefik-forward-auth.service=traefik-forward-auth"
      - "traefik.http.routers.traefik-forward-auth.entryPoints=http"
      - "traefik.http.routers.traefik-forward-auth.tls=false"

      - "traefik.http.routers.traefik-forward-auth-https.rule=Host(`auth.sub.domain.com`)"
      - "traefik.http.routers.traefik-forward-auth-https.service=traefik-forward-auth"
      - "traefik.http.routers.traefik-forward-auth-https.entryPoints=https"
      - "traefik.http.routers.traefik-forward-auth-https.tls"

      - "traefik.http.services.traefik-forward-auth.loadbalancer.server.port=4181"

In the end I get a list of 302 and 307 redirects between auth.sub.domain.com and accounts.google.com. I'm also not seeing a X-Forwarded-URI anywhere in the logs.

[dalanmiller]

#45 Not sure if related to v2.0

[gheibia]

I also see this when I deploy the service to Service Fabric. With some added logging I came to the same conclusion that the X-Forwarded-Uri isn't being passed from Traefik (v1.7.14).

I use this to secure Traefik's API endpoint (port 8080). Traefik's Forward Auth URL is http://localhost:4181. Redirect URL (coming back from Identity Service) however, goes through Traefik's Http Endpoint (port 80).

The way I got around it was to remove these lines from RootHandler

r.Method = r.Header.Get("X-Forwarded-Method")
r.Host = r.Header.Get("X-Forwarded-Host")
r.URL, _ = url.Parse(r.Header.Get("X-Forwarded-Uri"))

This allows the service to handle the incoming requests as they are. In fact, I don't understand why they're needed.