Not Authorized error about traefik-forward-auth

I don't think this should be closed. I run into this problem frequently. It may be because we need a traefik header

[sharkoz]

I can confirm I have the same error even after deleting my cookies...

[726a67]

I experience the same issue intermittently in auth host mode, via both desktop and mobile browsers.

time="2019-03-22T13:48:49Z" level=warning msg="Error validating csrf cookie: CSRF cookie does not match state" RemoteAddr="x.x.x.x:xxxx" csrf=<xyz> state="<zyx>:https://<domain>/"

As @robertbaker stated, this seems to happen most frequently when re-selecting an account which has been authorized (in my case, via WHITELIST) after the session lifetime has expired. Perhaps a token refresh issue?

[thomseddon]

@bekoeppel I have created an issue to track that specific pattern as I think there could be a better way of handling it.

I believe a lot of people here will have run into #103 whereby Google will not offer up the account selection by default on subsequent requests, so if you select the wrong account you have to wait for that selection to expire on Google's side before having another go (which is why some people have said you just need to wait and clearing cookies is not enough).

Beyond the two instances above, I think the most common cause will be a misconfiguration. I have opened #114 to improve debug logging to help track these down

@frankforpresident I will confirm when #114 is merged, if you could send your debug logs again following that, it would be greatly appreciated.

[zetas]

FWIW I've been struggling with this issue for months with no good workaround except to logout of all google accounts, delete all google related cookies and cache, and close and re-open the browser. It happens every month or so and it's incredibly annoying.

The incognito mode fix does work for me though so I think it's safe to say it's something to do with caching and tokens. When I get the "not authorized" message I also get the CSRF error log mentioned by @726a67. I hope we come up with a fix for this soon.

[Queuecumber]

Getting the same error intermittently, completely closing chrome didnt fix it

[Queuecumber]

Just tried it, still getting the error. My log level is set to debug, let me know what to look for

[Queuecumber]

Can you double check you're running the image with the tag: 3e6ccc8

I wasn't able to come up with a good way to check this. The only docker tag on that is latest and there apparently isn't a shell in the container I could use to check the source. I restarted a couple of times, so it should have pulled the image.

Do you get any logs with the 401?

I actually do not have any 401 errors in the log

I have

logger.Warn("Missing csrf cookie") 

I was grepping on "401" before, I didn't realize the logs dont actually include the http status code.

[frankforpresident]

@thomseddon, I did my best to gather some information for you. My setup was based on: https://www.smarthomebeginner.com/google-oauth-with-traefik-docker/

Description:
Open chrome in incognite mode (or normal). Visit the configured url and login with configured email address.

Expected: Proper redirection after the authentication. Content is visible.

Actual: After login the webpage redirects and returns with 401 Not authorized.

Note: I did changed some personal data and revoked the 0auth cert just after this run.

Logs:

time="2020-01-20T22:20:42Z" level=debug msg="Starting with options: {\"LogLevel\":\"debug\",\"LogFormat\":\"text\",\"AuthHost\":\"oauth.fakedomainforissue31.com\",\"CookieDomains\":[{\"Domain\":\"fakedomainforissue31.com\",\"DomainLen\":16,\"SubDomain\":\".fakedomainforissue31.com\",\"SubDomainLen\":17}],\"InsecureCookie\":true,\"CookieName\":\"_forward_auth\",\"CSRFCookieName\":\"_forward_auth_csrf\",\"DefaultAction\":\"auth\",\"Domains\":null,\"LifetimeString\":0,\"Path\":\"/_oauth\",\"Whitelist\":[\"[email protected]\"],\"Providers\":{\"Google\":{\"ClientId\":\"xxxxxxxxxxxxx.apps.googleusercontent.com\",\"Scope\":\"https://www.googleapis.com/auth/userinfo.profile https://www.googleapis.com/auth/userinfo.email\",\"Prompt\":\"\",\"LoginURL\":{\"Scheme\":\"https\",\"Opaque\":\"\",\"User\":null,\"Host\":\"accounts.google.com\",\"Path\":\"/o/oauth2/auth\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"TokenURL\":{\"Scheme\":\"https\",\"Opaque\":\"\",\"User\":null,\"Host\":\"www.googleapis.com\",\"Path\":\"/oauth2/v3/token\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"UserURL\":{\"Scheme\":\"https\",\"Opaque\":\"\",\"User\":null,\"Host\":\"www.googleapis.com\",\"Path\":\"/oauth2/v2/userinfo\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"}}},\"Rules\":{},\"Lifetime\":0,\"CookieDomainsLegacy\":null,\"CookieSecureLegacy\":\"\",\"ClientIdLegacy\":\"\",\"PromptLegacy\":\"\"}"
time="2020-01-20T22:20:42Z" level=info msg="Listening on :4181"
time="2020-01-20T22:21:10Z" level=debug msg="Authenticating request" headers="map[Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9] Accept-Encoding:[gzip, deflate, br] Accept-Language:[en-US,en;q=0.9,nl;q=0.8,af;q=0.7] Sec-Fetch-Mode:[navigate] Sec-Fetch-Site:[none] Sec-Fetch-User:[?1] Upgrade-Insecure-Requests:[1] User-Agent:[Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36] X-Forwarded-For:[192.168.0.1] X-Forwarded-Host:[debug.fakedomainforissue31.com] X-Forwarded-Method:[GET] X-Forwarded-Proto:[https] X-Forwarded-Uri:[/]]" rule=default source_ip=192.168.0.1
time="2020-01-20T22:21:10Z" level=debug msg="Set CSRF cookie and redirecting to google login" source_ip=192.168.0.1
time="2020-01-20T22:21:10Z" level=debug msg=Done source_ip=192.168.0.1
time="2020-01-20T22:21:13Z" level=debug msg="Handling callback" headers="map[Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9] Accept-Encoding:[gzip, deflate, br] Accept-Language:[en-US,en;q=0.9,nl;q=0.8,af;q=0.7] Referer:[https://accounts.google.com/signin/oauth/oauthchooseaccount?client_id=xxxxxxxxxxxxxxxxxxx.apps.googleusercontent.com&as=siqf3xKp8E6XuK8hiWhYSw&destination=https%3A%2F%2Foauth.fakedomainforissue31.com&approval_state=!ChRjUnBHenZ0aWlkMlJOWjB1UmZBZRIfODFnTTFFUjNYR1VkY0t0V0xtY192ZHBvNWJsUV9CWQ%E2%88%99AJDr988AAAAAXid5VqkCKO41SE29VN3yjfxvWPdXqezJ&oauthgdpr=1&xsrfsig=ChkAeAh8TzAGW13wZT6mNVUkgf1VI3Un__LKEg5hcHByb3ZhbF9zdGF0ZRILZGVzdGluYXRpb24SBXNvYWN1Eg9vYXV0aHJpc2t5c2NvcGU&flowName=GeneralOAuthFlow] Sec-Fetch-Mode:[navigate] Sec-Fetch-Site:[cross-site] Sec-Fetch-User:[?1] Upgrade-Insecure-Requests:[1] User-Agent:[Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36] X-Forwarded-For:[192.168.0.1] X-Forwarded-Host:[oauth.fakedomainforissue31.com] X-Forwarded-Method:[GET] X-Forwarded-Proto:[https] X-Forwarded-Uri:[/_oauth?state=f480a65f3c0c40bc1ec395dd5884d862%3Ahttps%3A%2F%2Fdebug.fakedomainforissue31.com%2F&code=4%2FvgFS7t4mz4NGZPiFu_WqGI4IM-rwRbZpSLkhC8lsjakaQ3wZcRe1VJIuXxL_8bzruh7dP5LfKUGJraM3UyPUQc4&scope=email+profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+openid&authuser=0&prompt=consent&session_state=efcdb271432c17ec7c1ac1b246b908fbc2dd43eb..413a]]" rule=default source_ip=192.168.0.1
time="2020-01-20T22:21:13Z" level=warning msg="Missing csrf cookie" source_ip=192.168.0.1
time="2020-01-20T22:21:13Z" level=debug msg="Authenticating request" headers="map[Accept:[image/webp,image/apng,image/*,*/*;q=0.8] Accept-Encoding:[gzip, deflate, br] Accept-Language:[en-US,en;q=0.9,nl;q=0.8,af;q=0.7] Referer:[https://oauth.fakedomainforissue31.com/_oauth?state=f480a65f3c0c40bc1ec395dd5884d862%3Ahttps%3A%2F%2Fdebug.fakedomainforissue31.com%2F&code=4%2FvgFS7t4mz4NGZPiFu_WqGI4IM-rwRbZpSLkhC8lsjakaQ3wZcRe1VJIuXxL_8bzruh7dP5LfKUGJraM3UyPUQc4&scope=email+profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+openid&authuser=0&prompt=consent&session_state=efcdb271432c17ec7c1ac1b246b908fbc2dd43eb..413a] Sec-Fetch-Mode:[no-cors] Sec-Fetch-Site:[same-origin] User-Agent:[Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36] X-Forwarded-For:[192.168.0.1] X-Forwarded-Host:[oauth.fakedomainforissue31.com] X-Forwarded-Method:[GET] X-Forwarded-Proto:[https] X-Forwarded-Uri:[/favicon.ico]]" rule=default source_ip=192.168.0.1
time="2020-01-20T22:21:13Z" level=debug msg="Set CSRF cookie and redirecting to google login" source_ip=192.168.0.1
time="2020-01-20T22:21:13Z" level=debug msg=Done source_ip=192.168.0.1

docker-compose.yaml

whoami:
    container_name: debug
    hostname: debug
    image: containous/whoami
    labels:
      - traefik.frontend.rule=Host:debug.fakedomainforissue31.com
      - traefik.port=80
      - traefik.backend=whoami
      - traefik-docker.network=internal
      - traefik.frontend.auth.forward.address=http://oauth:4181
      - traefik.frontend.auth.forward.authResponseHeaders=X-Forwarded-User
      - traefik.frontend.auth.forward.trustForwardHeader=true
      - traefik.redirectorservice.frontend.entryPoints=http
      - traefik.redirectorservice.frontend.redirect.entryPoint=https
      - traefik.webservice.frontend.entryPoints=https
	  
oauth:
    image: thomseddon/traefik-forward-auth:latest
    container_name: oauth
    hostname: oauth
    restart: always
    env_file:
      - oauth.env
    labels:
      - traefik.enable=true
      - traefik.port=4181
      - traefik.backend=oauth
      - traefik.docker.network=internal
      - traefik.frontend.rule=Host:oauth.fakedomainforissue31.com
      - traefik.frontend.headers.SSLHost=oauth.fakedomainforissue31.com
      - traefik.frontend.auth.forward.address=http://oauth:4181
      - traefik.frontend.auth.forward.authResponseHeaders=X-Forwarded-User
      - traefik.frontend.auth.forward.trustForwardHeader=true
      - traefik.frontend.headers.SSLRedirect=true
      - traefik.frontend.headers.browserXSSFilter=true
      - traefik.frontend.headers.contentTypeNosniff=true
      - traefik.frontend.headers.forceSTSHeader=true
      - traefik.frontend.headers.STSSeconds=315360000
      - traefik.frontend.headers.STSIncludeSubdomains=true
      - traefik.frontend.headers.STSPreload=true
      - traefik.frontend.headers.frameDeny=true
      - traefik.frontend.passHostHeader=true
      - traefik.frontend.headers.SSLForceHost=true
      - traefik.frontend.headers.customResponseHeaders=X-Robots-Tag:noindex,nofollow,nosnippet,noarchive,notranslate,noimageindex
      - traefik.redirectorservice.frontend.entryPoints=http
      - traefik.redirectorservice.frontend.redirect.entryPoint=https
      - traefik.webservice.frontend.entryPoints=https	  

oauth.env

PROVIDERS_GOOGLE_CLIENT_ID=XXXXXXXXXXXXXXXXXX.apps.googleusercontent.com
PROVIDERS_GOOGLE_CLIENT_SECRET=XXXXXXXXXXXXXX
SECRET=XXXXXXXXXXXXXXX
COOKIE_DOMAIN=fakedomainforissue31.com
INSECURE_COOKIE=true
AUTH_HOST=oauth.fakedomainforissue31.com
URL_PATH=/_oauth
[email protected]
LOG_LEVEL=debug
LIFETIME=2592000 # 30 days
OIDC_ISSUER=https://accounts.google.com
CSRF_COOKIE_NAME=_forward_auth_csrf

Note: I also tried INSECURE_COOKIE=false, changing _forward_auth_csrf to a different key and a different email address

How do you want me to collect the request logs? or is this enough to get started?

I did saw this message in the console after landing on the 401 page.

/_oauth?state=014308…49a7a48f6c..576a#:1 A cookie associated with a cross-site resource at https://accounts.google.com/ was set without the `SameSite` attribute. A future release of Chrome will only deliver cookies with cross-site requests if they are set with `SameSite=None` and `Secure`. You can review cookies in developer tools under Application>Storage>Cookies and see more details at https://www.chromestatus.com/feature/5088147346030592 and https://www.chromestatus.com/feature/5633521622188032.
/_oauth?state=014308…49a7a48f6c..576a#:1 A cookie associated with a cross-site resource at http://google.com/ was set without the `SameSite` attribute. A future release of Chrome will only deliver cookies with cross-site requests if they are set with `SameSite=None` and `Secure`. You can review cookies in developer tools under Application>Storage>Cookies and see more details at https://www.chromestatus.com/feature/5088147346030592 and https://www.chromestatus.com/feature/5633521622188032.
/_oauth?state=014308…49a7a48f6c..576a#:1 A cookie associated with a cross-site resource at https://google.com/ was set without the `SameSite` attribute. A future release of Chrome will only deliver cookies with cross-site requests if they are set with `SameSite=None` and `Secure`. You can review cookies in developer tools under Application>Storage>Cookies and see more details at https://www.chromestatus.com/feature/5088147346030592 and https://www.chromestatus.com/feature/5633521622188032.
Cross-Origin Read Blocking (CORB) blocked cross-origin response https://accounts.google.com/o/oauth2/auth?client_id=110929115210-iebt17j9me…b0168a5…%3Ahttps%3A%2F%2Foauth.computerheld.net%2Ffavicon.ico with MIME type text/html. See https://www.chromestatus.com/feature/5629709824032768 for more details.

and I can't seem to find the_forward_auth_csrf cookie after the redirect under Application>Storage>Cookies. Not sure if this is the cause of my problem.

[thomseddon]

Hello everyone - the good news is that v2.2 was released yesterday, this includes a significant improvement in logging as well as a warning about some invalid configuration states.

This also includes a tweak to the Google provider to ensure the user is always prompted to select which account they would like to use, this fixes the issue where a user selects the wrong account and is then unable to reselect the right account as their previous selection is "cached" (which may result in a Not Authorized error).

I've also released a large amount of new examples for both k8s and swarm, using both traefik v1.7 and 2+ - this shows the advised way to deploy the service and should help many people ending up in a bad/conflicting configuration state.

As mentioned before, I think this issue may represent a number of different issues that people have had, most of which I believe will have been caused by me not providing enough configuration examples - which I hope is now fixed. Due to this, and the other fixes outlined above, I'm going to close this issue.

If anyone is still having a problem when using v2.2, following the new examples, please please please open a new issue with your configuration and logs and I will dig into it right away.

Thank you everyone for your contributions to this, it's very much appreciated.

[strazto]

I can confirm that although a similar issue (intermittently) persisted on tags :2 AND 2.2, they are resolved on :latest ( b364aa6a4117)

[MortenRickiRasmussen]

I Received that too and i seems to be some cookie/cache related issue. Either wait a couple of hours or try a different browser/ Incognito

[osotechie]

I Received that too and i seems to be some cookie/cache related issue. Either wait a couple of hours or try a different browser/ Incognito

OMG,so in all my testing I had forgot to try Incognito mode... I had tried both Chrome and Edge... But that was prob whilst the config was not right... so retested with Incognito mode and hey presto its working...

Thanks this can be closed 👍

Related #20

[thomseddon]

Hmm, I have a similar issue where I've clicked the wrong google account, my only option is to clear cookies and try again - sounds like this could be the same root cause - I wonder if there is a better UX for this?

It does happen if you do choose the wrong account or it happens once the authorization expires with the right account.

I believe it may be due to a missing header on traefik and/or its like it can't refresh the token on it's own.

[thomseddon]

@robertbaker what do you mean it's missing a header?

In the #20 I believe it's mentioned. The Forwarded URI or auth trust heaters. Although I think it's more likely a bug where its not refreshing the expired token

…

On Tue, Mar 19, 2019, 9:07 AM Thom Seddon ***@***.***> wrote: @robertbaker <https://github.com/robertbaker> what do you mean it's missing a header? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA3YJQccmkPKzNmrrTAL6W3Ncj8AumK7ks5vYQtcgaJpZM4bWnaR> .

[whi-tw]

I don’t suppose a “logout” endpoint is in the works?

@zetas simply completely closing chrome (including tray icon) should be the only thing you need to do to fix it.

[Queuecumber]

The weird thing is that it lets me access some of my endpoints but not others when this happens. E.g. some work fine some give the "not authorized" text

[thomseddon]

So I think I might have found the cause of this, after a cookie expires the existing behavior was just to throw an error.

I have just pushed a change which means that the users will be redirected to Google login instead.

If someone would be willing to test the latest tag on docker, with error logging enabled, that would be hugely helpful.
I know this can take some time to replicate, so if anyone has error logs enabled then there should be more info in that log already, so if you could post existing logs that would also help!

[thomseddon]

Can you double check you're running the image with the tag: 3e6ccc8

Do you get any logs with the 401?

For extra info, 401 can returned from 4 places:

• AuthHandler - invalid cookie:

  traefik-forward-auth/internal/server.go

  Lines 79 to 89 in 3e6ccc8

  	// Validate cookie
  	email, err := ValidateCookie(r, c)
  	if err != nil {
  	if err.Error() == "Cookie has expired" {
  	logger.Info("Cookie has expired")
  	s.authRedirect(logger, w, r)
  	} else {
  	logger.Errorf("Invalid cookie: %v", err)
  	http.Error(w, "Not authorized", 401)
  	}
  	return

• AuthHandler - invalid user:

  traefik-forward-auth/internal/server.go

  Lines 92 to 100 in 3e6ccc8

  	// Validate user
  	valid := ValidateEmail(email)
  	if !valid {
  	logger.WithFields(logrus.Fields{
  	"email": email,
  	}).Errorf("Invalid email")
  	http.Error(w, "Not authorized", 401)
  	return
  	}

• AuthCallbackHandler - invalid CSRF:

  traefik-forward-auth/internal/server.go

  Lines 115 to 121 in 3e6ccc8

  	// Check for CSRF cookie
  	c, err := r.Cookie(config.CSRFCookieName)
  	if err != nil {
  	logger.Warn("Missing csrf cookie")
  	http.Error(w, "Not authorized", 401)
  	return
  	}

• AuthCallbackHandler - invalid state:

  traefik-forward-auth/internal/server.go

  Lines 123 to 129 in 3e6ccc8

  	// Validate state
  	valid, redirect, err := ValidateCSRFCookie(r, c)
  	if !valid {
  	logger.Warnf("Error validating csrf cookie: %v", err)
  	http.Error(w, "Not authorized", 401)
  	return
  	}

There's a log message in each location, so if tfa is returning an error then it should be showing more info in the logs

[Queuecumber]

One thing that seems to work consistently is getting to the "not authorized" page, then visiting a different endpoint that for whatever reason works, then going back to my original endpoint lets me continue to the page I wanted.

[whi-tw]

I've done a thing in branch add-logout-endpoint on my fork that, in theory, adds a /_tfa-logout endpoint which will clear the local _forward_auth cookie.

I have written tests, which pass, but I want to do some functional testing before I actually submit a PR. However, I flubbed an unrelated IP on my home setup, and so can't actually access my own environment to deploy the updated version, so I'll do it tomorrow. Thanks, sslh.

I have built the image manually on https://hub.docker.com/r/tnwhitwell/traefik-forward-auth/tags, tag: testing, but obviously I'd recommend building and deploying your own version if you do want to test.

[Queuecumber]

I actually added a logout endpoint to my landing page which does the same thing, I've also been manually deleting the cookie and disabling caching before running these tests so I don't know if that will solve the problem

[thomseddon]

Could you do a docker inspect on the container, mine is showing: Image": "sha256:97d5ed375d0c1fa946b74d4381ab912199056da56d5c2af7a78c87ee074d240f"

I actually think my changes in 3e6ccc8 are a red herring (which I will partially revert), the browser will expire the cookie after that time so it is actually an error if such a cookie is received.

Missing csrf cookie suggests you're hitting the callback endpoint, you should only hit this endpoint after you've done the login on Google?

[Queuecumber]

Could you do a docker inspect on the container, mine is showing: Image": "sha256:97d5ed375d0c1fa946b74d4381ab912199056da56d5c2af7a78c87ee074d240f"

I'll double check the image hash

Missing csrf cookie suggests you're hitting the callback endpoint, you should only hit this endpoint after you've done the login on Google?

Indeed it's directly after I log into google that I get this screen, which is part of why it makes no sense since I've just logged in.

Do you rely on the X-Forwarded-User header to check the username at the callback endpoint? I don't think I configured traefik to send it for that endpoint

[thomseddon]

Interesting, could you check what cookies are sent before you're redirected to Google (you can inspect the headers on the 307 redirect in your browser)? Are you using the auth host?

[Queuecumber]

Sorry for the late reply, I'm in the process of rebuilding that cluster right now, when it's settled down I'll look into the cookies for you

[Queuecumber]

In the request to my endpoint, I see

cookie: _forward_auth_csrf=0adc70a3cbde78d0b09e37e7c9602c83

The 307 response has

set-cookie: _forward_auth_csrf=cabca8a0d088bddc863932ec8704c953; Path=/; Domain=<redacted correct domain>; Expires=Wed, 19 Jun 2019 00:10:51 GMT; HttpOnly; Secure

[thomseddon]

Hmm, so that's why it doesn't work - but why doesn't the cookie match? My first thought would be a domain missmatch?

[Queuecumber]

Domain is correct and if it was that I would expect it to fail for every domain. The only obvious difference is that the working domains set x-forwarded-user and the failing ones don't

[westfood]

I am experiencing exactly same issue as Queuecumber. "Not authorized" after google authentication. Docker Image: 97d5ed375d0c. Traefik services handle traffic on port 8080.

Command is: --cookie-domain=somewhere.com --cookie-domain=test.somewhere.com --cookie-domain=auth.test.somewhere.com
In my env: AUTH_HOST: auth.test.somewhere.com:8080

services I am authenticating are path based m1-eu1.test.somewhere.com:8080/traefik/dashboard or m1-eu1.test.somewhere.com:8080/cadvisor

[frankforpresident]

Same issue here, After google login I get "Not authorized".. No clue yet why
msg="Missing csrf cookie"

[thomseddon]

@frankforpresident (or anyone else in the thread) could you post:

• Confg
• Request log (headers from all requests+responses)
• Debug log from traefik-forward-auth

I'm sure we can get to the bottom of this :)

[burntoc]

I can confirm I'm seeing the same behavior here. Initiating an incognito browser allows me to access the service properly though.

[frankforpresident]

@thomseddon, (or anyone else) Any update on this issue? Is there something I could help you with? If more logs are required don't hesitate to contact me.

I would love to implement this in my setup.

[thomseddon]

Sorry I missed your previous comment, thanks for the detailed info. Could you post your entire compose config (including traefik) - the absolute jackpot would be if you can give me a docker-compose.yaml + associated config so I can exactly replicate it?

With regards to capturing the requests, if you export them from the chrome network tab, that would be great (although if I can replicate that won't be needed)

[0xknon]

I am having the same issue here. I guess my config are more naive.

Error:
time="2020-02-27T08:25:03Z" level=warning msg="Missing csrf cookie" source_ip=192.168.9.195

Traefik (installed with helm - stable/traefik):
values.yaml:

ssl:
  enabled: true
  enforced: true
  insecureSkipVerify: true
dashboard:
  enabled: true
  domain: traefik.stg.example.com
  ingress:
    annotations:
      kubernetes.io/ingress.class: traefik
      ingress.kubernetes.io/auth-type: forward
      ingress.kubernetes.io/auth-url: http://traefik-forward-auth.kube-system.svc.cluster.local
      ingress.kubernetes.io/auth-response-headers: X-Forwarded-User
rbac:
  enabled: true
deployment:
  hostPort:
    httpsEnabled: true

traefik-forward-auth (reference to Example for k8s):

##
# Secrets to store Google's client secret and the app's secret
##
kind: Secret
apiVersion: v1
metadata:
  name: traefik-forward-auth-secrets
  namespace: kube-system
  labels:
    name: traefik
type: Opaque
data:
  CLIENT_SECRET: <- CLIENT_SECRET -> 
  SECRET: <- SECRET ->
---

##
# Main deployment
## 
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: traefik-forward-auth
  name: traefik-forward-auth
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik-forward-auth
  template:
    metadata:
      labels:
        app: traefik-forward-auth
    spec:
      containers:
        - name: traefik-forward-auth
          image: thomseddon/traefik-forward-auth
          ports:
            - containerPort: 4181
              protocol: TCP
          env:
            - name: CLIENT_ID
              value: <- CLIENT_ID ->
            - name: CLIENT_SECRET
              valueFrom:
                secretKeyRef:
                  name: traefik-forward-auth-secrets
                  key: CLIENT_SECRET
            - name: SECRET
              valueFrom:
                secretKeyRef:
                  name: traefik-forward-auth-secrets
                  key: SECRET
            - name: DOMAIN
              value: example.com
          livenessProbe:
            tcpSocket:
              port: 4181
            initialDelaySeconds: 20
            failureThreshold: 3
            successThreshold: 1
            periodSeconds: 10
            timeoutSeconds: 2
---

kind: Service
apiVersion: v1
metadata:
  name: traefik-forward-auth
  namespace: kube-system
spec:
  selector:
    app: traefik-forward-auth
  ports:
    - port: 80
      targetPort: 4181
      protocol: TCP

[kk17]

I found out the clear the browser data can solve this problem.

[ghhv]

@thomseddon, I did my best to gather some information for you. My setup was based on: https://www.smarthomebeginner.com/google-oauth-with-traefik-docker/

Description:
Open chrome in incognite mode (or normal). Visit the configured url and login with configured email address.

Expected: Proper redirection after the authentication. Content is visible.

Actual: After login the webpage redirects and returns with 401 Not authorized.

Note: I did changed some personal data and revoked the 0auth cert just after this run.

Logs:

time="2020-01-20T22:20:42Z" level=debug msg="Starting with options: {\"LogLevel\":\"debug\",\"LogFormat\":\"text\",\"AuthHost\":\"oauth.fakedomainforissue31.com\",\"CookieDomains\":[{\"Domain\":\"fakedomainforissue31.com\",\"DomainLen\":16,\"SubDomain\":\".fakedomainforissue31.com\",\"SubDomainLen\":17}],\"InsecureCookie\":true,\"CookieName\":\"_forward_auth\",\"CSRFCookieName\":\"_forward_auth_csrf\",\"DefaultAction\":\"auth\",\"Domains\":null,\"LifetimeString\":0,\"Path\":\"/_oauth\",\"Whitelist\":[\"[email protected]\"],\"Providers\":{\"Google\":{\"ClientId\":\"xxxxxxxxxxxxx.apps.googleusercontent.com\",\"Scope\":\"https://www.googleapis.com/auth/userinfo.profile https://www.googleapis.com/auth/userinfo.email\",\"Prompt\":\"\",\"LoginURL\":{\"Scheme\":\"https\",\"Opaque\":\"\",\"User\":null,\"Host\":\"accounts.google.com\",\"Path\":\"/o/oauth2/auth\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"TokenURL\":{\"Scheme\":\"https\",\"Opaque\":\"\",\"User\":null,\"Host\":\"www.googleapis.com\",\"Path\":\"/oauth2/v3/token\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"},\"UserURL\":{\"Scheme\":\"https\",\"Opaque\":\"\",\"User\":null,\"Host\":\"www.googleapis.com\",\"Path\":\"/oauth2/v2/userinfo\",\"RawPath\":\"\",\"ForceQuery\":false,\"RawQuery\":\"\",\"Fragment\":\"\"}}},\"Rules\":{},\"Lifetime\":0,\"CookieDomainsLegacy\":null,\"CookieSecureLegacy\":\"\",\"ClientIdLegacy\":\"\",\"PromptLegacy\":\"\"}"
time="2020-01-20T22:20:42Z" level=info msg="Listening on :4181"
time="2020-01-20T22:21:10Z" level=debug msg="Authenticating request" headers="map[Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9] Accept-Encoding:[gzip, deflate, br] Accept-Language:[en-US,en;q=0.9,nl;q=0.8,af;q=0.7] Sec-Fetch-Mode:[navigate] Sec-Fetch-Site:[none] Sec-Fetch-User:[?1] Upgrade-Insecure-Requests:[1] User-Agent:[Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36] X-Forwarded-For:[192.168.0.1] X-Forwarded-Host:[debug.fakedomainforissue31.com] X-Forwarded-Method:[GET] X-Forwarded-Proto:[https] X-Forwarded-Uri:[/]]" rule=default source_ip=192.168.0.1
time="2020-01-20T22:21:10Z" level=debug msg="Set CSRF cookie and redirecting to google login" source_ip=192.168.0.1
time="2020-01-20T22:21:10Z" level=debug msg=Done source_ip=192.168.0.1
time="2020-01-20T22:21:13Z" level=debug msg="Handling callback" headers="map[Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9] Accept-Encoding:[gzip, deflate, br] Accept-Language:[en-US,en;q=0.9,nl;q=0.8,af;q=0.7] Referer:[https://accounts.google.com/signin/oauth/oauthchooseaccount?client_id=xxxxxxxxxxxxxxxxxxx.apps.googleusercontent.com&as=siqf3xKp8E6XuK8hiWhYSw&destination=https%3A%2F%2Foauth.fakedomainforissue31.com&approval_state=!ChRjUnBHenZ0aWlkMlJOWjB1UmZBZRIfODFnTTFFUjNYR1VkY0t0V0xtY192ZHBvNWJsUV9CWQ%E2%88%99AJDr988AAAAAXid5VqkCKO41SE29VN3yjfxvWPdXqezJ&oauthgdpr=1&xsrfsig=ChkAeAh8TzAGW13wZT6mNVUkgf1VI3Un__LKEg5hcHByb3ZhbF9zdGF0ZRILZGVzdGluYXRpb24SBXNvYWN1Eg9vYXV0aHJpc2t5c2NvcGU&flowName=GeneralOAuthFlow] Sec-Fetch-Mode:[navigate] Sec-Fetch-Site:[cross-site] Sec-Fetch-User:[?1] Upgrade-Insecure-Requests:[1] User-Agent:[Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36] X-Forwarded-For:[192.168.0.1] X-Forwarded-Host:[oauth.fakedomainforissue31.com] X-Forwarded-Method:[GET] X-Forwarded-Proto:[https] X-Forwarded-Uri:[/_oauth?state=f480a65f3c0c40bc1ec395dd5884d862%3Ahttps%3A%2F%2Fdebug.fakedomainforissue31.com%2F&code=4%2FvgFS7t4mz4NGZPiFu_WqGI4IM-rwRbZpSLkhC8lsjakaQ3wZcRe1VJIuXxL_8bzruh7dP5LfKUGJraM3UyPUQc4&scope=email+profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+openid&authuser=0&prompt=consent&session_state=efcdb271432c17ec7c1ac1b246b908fbc2dd43eb..413a]]" rule=default source_ip=192.168.0.1
time="2020-01-20T22:21:13Z" level=warning msg="Missing csrf cookie" source_ip=192.168.0.1
time="2020-01-20T22:21:13Z" level=debug msg="Authenticating request" headers="map[Accept:[image/webp,image/apng,image/*,*/*;q=0.8] Accept-Encoding:[gzip, deflate, br] Accept-Language:[en-US,en;q=0.9,nl;q=0.8,af;q=0.7] Referer:[https://oauth.fakedomainforissue31.com/_oauth?state=f480a65f3c0c40bc1ec395dd5884d862%3Ahttps%3A%2F%2Fdebug.fakedomainforissue31.com%2F&code=4%2FvgFS7t4mz4NGZPiFu_WqGI4IM-rwRbZpSLkhC8lsjakaQ3wZcRe1VJIuXxL_8bzruh7dP5LfKUGJraM3UyPUQc4&scope=email+profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+openid&authuser=0&prompt=consent&session_state=efcdb271432c17ec7c1ac1b246b908fbc2dd43eb..413a] Sec-Fetch-Mode:[no-cors] Sec-Fetch-Site:[same-origin] User-Agent:[Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36] X-Forwarded-For:[192.168.0.1] X-Forwarded-Host:[oauth.fakedomainforissue31.com] X-Forwarded-Method:[GET] X-Forwarded-Proto:[https] X-Forwarded-Uri:[/favicon.ico]]" rule=default source_ip=192.168.0.1
time="2020-01-20T22:21:13Z" level=debug msg="Set CSRF cookie and redirecting to google login" source_ip=192.168.0.1
time="2020-01-20T22:21:13Z" level=debug msg=Done source_ip=192.168.0.1

docker-compose.yaml

whoami:
    container_name: debug
    hostname: debug
    image: containous/whoami
    labels:
      - traefik.frontend.rule=Host:debug.fakedomainforissue31.com
      - traefik.port=80
      - traefik.backend=whoami
      - traefik-docker.network=internal
      - traefik.frontend.auth.forward.address=http://oauth:4181
      - traefik.frontend.auth.forward.authResponseHeaders=X-Forwarded-User
      - traefik.frontend.auth.forward.trustForwardHeader=true
      - traefik.redirectorservice.frontend.entryPoints=http
      - traefik.redirectorservice.frontend.redirect.entryPoint=https
      - traefik.webservice.frontend.entryPoints=https
	  
oauth:
    image: thomseddon/traefik-forward-auth:latest
    container_name: oauth
    hostname: oauth
    restart: always
    env_file:
      - oauth.env
    labels:
      - traefik.enable=true
      - traefik.port=4181
      - traefik.backend=oauth
      - traefik.docker.network=internal
      - traefik.frontend.rule=Host:oauth.fakedomainforissue31.com
      - traefik.frontend.headers.SSLHost=oauth.fakedomainforissue31.com
      - traefik.frontend.auth.forward.address=http://oauth:4181
      - traefik.frontend.auth.forward.authResponseHeaders=X-Forwarded-User
      - traefik.frontend.auth.forward.trustForwardHeader=true
      - traefik.frontend.headers.SSLRedirect=true
      - traefik.frontend.headers.browserXSSFilter=true
      - traefik.frontend.headers.contentTypeNosniff=true
      - traefik.frontend.headers.forceSTSHeader=true
      - traefik.frontend.headers.STSSeconds=315360000
      - traefik.frontend.headers.STSIncludeSubdomains=true
      - traefik.frontend.headers.STSPreload=true
      - traefik.frontend.headers.frameDeny=true
      - traefik.frontend.passHostHeader=true
      - traefik.frontend.headers.SSLForceHost=true
      - traefik.frontend.headers.customResponseHeaders=X-Robots-Tag:noindex,nofollow,nosnippet,noarchive,notranslate,noimageindex
      - traefik.redirectorservice.frontend.entryPoints=http
      - traefik.redirectorservice.frontend.redirect.entryPoint=https
      - traefik.webservice.frontend.entryPoints=https	  

oauth.env

PROVIDERS_GOOGLE_CLIENT_ID=XXXXXXXXXXXXXXXXXX.apps.googleusercontent.com
PROVIDERS_GOOGLE_CLIENT_SECRET=XXXXXXXXXXXXXX
SECRET=XXXXXXXXXXXXXXX
COOKIE_DOMAIN=fakedomainforissue31.com
INSECURE_COOKIE=true
AUTH_HOST=oauth.fakedomainforissue31.com
URL_PATH=/_oauth
[email protected]
LOG_LEVEL=debug
LIFETIME=2592000 # 30 days
OIDC_ISSUER=https://accounts.google.com
CSRF_COOKIE_NAME=_forward_auth_csrf

Note: I also tried INSECURE_COOKIE=false, changing _forward_auth_csrf to a different key and a different email address

How do you want me to collect the request logs? or is this enough to get started?

I did saw this message in the console after landing on the 401 page.

/_oauth?state=014308…49a7a48f6c..576a#:1 A cookie associated with a cross-site resource at https://accounts.google.com/ was set without the `SameSite` attribute. A future release of Chrome will only deliver cookies with cross-site requests if they are set with `SameSite=None` and `Secure`. You can review cookies in developer tools under Application>Storage>Cookies and see more details at https://www.chromestatus.com/feature/5088147346030592 and https://www.chromestatus.com/feature/5633521622188032.
/_oauth?state=014308…49a7a48f6c..576a#:1 A cookie associated with a cross-site resource at http://google.com/ was set without the `SameSite` attribute. A future release of Chrome will only deliver cookies with cross-site requests if they are set with `SameSite=None` and `Secure`. You can review cookies in developer tools under Application>Storage>Cookies and see more details at https://www.chromestatus.com/feature/5088147346030592 and https://www.chromestatus.com/feature/5633521622188032.
/_oauth?state=014308…49a7a48f6c..576a#:1 A cookie associated with a cross-site resource at https://google.com/ was set without the `SameSite` attribute. A future release of Chrome will only deliver cookies with cross-site requests if they are set with `SameSite=None` and `Secure`. You can review cookies in developer tools under Application>Storage>Cookies and see more details at https://www.chromestatus.com/feature/5088147346030592 and https://www.chromestatus.com/feature/5633521622188032.
Cross-Origin Read Blocking (CORB) blocked cross-origin response https://accounts.google.com/o/oauth2/auth?client_id=110929115210-iebt17j9me…b0168a5…%3Ahttps%3A%2F%2Foauth.computerheld.net%2Ffavicon.ico with MIME type text/html. See https://www.chromestatus.com/feature/5629709824032768 for more details.

and I can't seem to find the_forward_auth_csrf cookie after the redirect under Application>Storage>Cookies. Not sure if this is the cause of my problem.

Hey Frank, I also used the SmartHomeBeginner sample. I was also getting 'unauthorized' because the Google email address I was logging in with, was not in the WHITELIST environment variable passed to the forward-auth! Adding this fixed it.. Clear cookies to ensure it's re-authenticating.

This error clue is also apparent from the service log for the container which will say invalid email.
time="2020-04-04T12:13:37Z" level=error msg="Invalid email" [email protected]

Hope that helps someone!

[frankforpresident]

Hey Frank, I also used the SmartHomeBeginner sample. I was also getting 'unauthorized' because the Google email address I was logging in with, was not in the WHITELIST environment variable passed to the forward-auth! Adding this fixed it.. Clear cookies to ensure it's re-authenticating.

This error clue is also apparent from the service log for the container which will say invalid email.
time="2020-04-04T12:13:37Z" level=error msg="Invalid email" [email protected]

Hope that helps someone!

I will try it again tonight but if i remember correctly it was already set to the correct email address.

[frankforpresident]

My email address is ok and I don't see this message in the logs.

I have a question, could it be related to my ip range? I've noticed in the logs a 192.168.0.1 ip address but that would be the range of my modem while my internal ip range of my router is 10.0.0.X

Anyway. on every attempt I see the same result. missing cookie, setting cookie, done

time="2020-04-09T10:53:09Z" level=warning msg="Missing csrf cookie" source_ip=192.168.0.1
time="2020-04-09T10:53:10Z" level=debug msg="Set CSRF cookie and redirecting to google login" source_ip=192.168.0.1
time="2020-04-09T10:53:10Z" level=debug msg=Done source_ip=192.168.0.1

[ghhv]

Ahh, OK.. that should be an easy fix.. Did you generate and pass a cookie?
e.g.
environment:
- CLIENT_ID=$OAUTH2_PROXY_CLIENT_ID
- CLIENT_SECRET=$OAUTH2_PROXY_CLIENT_SECRET
- SECRET=$OAUTH2_PROXY_COOKIE_SECRET

[frankforpresident]

Ahh, OK.. that should be an easy fix.. Did you generate and pass a cookie?
e.g.
environment:

• CLIENT_ID=$OAUTH2_PROXY_CLIENT_ID
• CLIENT_SECRET=$OAUTH2_PROXY_CLIENT_SECRET
  - SECRET=$OAUTH2_PROXY_COOKIE_SECRET

Yes, generated with: openssl rand -hex 16

You can check my config here, it didn't changed that much.
#31 (comment)

[ghhv]

OK, comparing yours to mine, I have 2 differences:

• INSECURE_COOKIE=false

And I do not have the "CSRF_COOKIE_NAME=_forward_auth_csrf"

So that could be why you are getting your error.

Note also that SmartHomeBeginner has drastically changed his sample code in the past week..

[bekoeppel]

I got the same errors and was able to reproduce it as follows:

• traefik-forward-auth configured in Host Mode
• Tab A: open an application that makes requests to its backend every second to refreshing its data
• Wait until the authentication session expires
• Tab A keeps making requests, which fail because the session expired. From my understanding, this is issuing new _forward_auth_csrf cookies every second
• Tab B: open another application, go through the OIDC login to the consent screen. When I was redirected back to my auth backend, I got Not authorized and saw in the log Error validating csrf cookie: CSRF cookie does not match state.

Closing Tab A, so that it would not make requests and issue new _forward_auth_csrf cookies then allowed me to log in to the second application in tab B.

[frankforpresident]

@thomseddon, Will do

[0xknon]

I am having the same issue here. I guess my config are more naive.

Error:
time="2020-02-27T08:25:03Z" level=warning msg="Missing csrf cookie" source_ip=192.168.9.195

Traefik (installed with helm - stable/traefik):
values.yaml:

ssl:
  enabled: true
  enforced: true
  insecureSkipVerify: true
dashboard:
  enabled: true
  domain: traefik.stg.example.com
  ingress:
    annotations:
      kubernetes.io/ingress.class: traefik
      ingress.kubernetes.io/auth-type: forward
      ingress.kubernetes.io/auth-url: http://traefik-forward-auth.kube-system.svc.cluster.local
      ingress.kubernetes.io/auth-response-headers: X-Forwarded-User
rbac:
  enabled: true
deployment:
  hostPort:
    httpsEnabled: true

traefik-forward-auth (reference to Example for k8s):

##
# Secrets to store Google's client secret and the app's secret
##
kind: Secret
apiVersion: v1
metadata:
  name: traefik-forward-auth-secrets
  namespace: kube-system
  labels:
    name: traefik
type: Opaque
data:
  CLIENT_SECRET: <- CLIENT_SECRET -> 
  SECRET: <- SECRET ->
---

##
# Main deployment
## 
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: traefik-forward-auth
  name: traefik-forward-auth
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: traefik-forward-auth
  template:
    metadata:
      labels:
        app: traefik-forward-auth
    spec:
      containers:
        - name: traefik-forward-auth
          image: thomseddon/traefik-forward-auth
          ports:
            - containerPort: 4181
              protocol: TCP
          env:
            - name: CLIENT_ID
              value: <- CLIENT_ID ->
            - name: CLIENT_SECRET
              valueFrom:
                secretKeyRef:
                  name: traefik-forward-auth-secrets
                  key: CLIENT_SECRET
            - name: SECRET
              valueFrom:
                secretKeyRef:
                  name: traefik-forward-auth-secrets
                  key: SECRET
            - name: DOMAIN
              value: example.com
          livenessProbe:
            tcpSocket:
              port: 4181
            initialDelaySeconds: 20
            failureThreshold: 3
            successThreshold: 1
            periodSeconds: 10
            timeoutSeconds: 2
---

kind: Service
apiVersion: v1
metadata:
  name: traefik-forward-auth
  namespace: kube-system
spec:
  selector:
    app: traefik-forward-auth
  ports:
    - port: 80
      targetPort: 4181
      protocol: TCP

Just want to share my Experience on debugging my issue.

My architecture is like this:

Route53 -> CloudFront -> EKS -> traefik -> traefik-forward-auth -> The application

The biggest challenge to me is the Cloudfront since i am not familiar with it. Here are the errors I experienced

First Error - Missing csrf cookie

The reason is that I didn't allow the Cloudfront to Forward Cookies

Second Error - Error validating csrf cookie: Invalid CSRF state value

The reason of getting this is because the Allowed HTTP Methods are GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE. After I change it to just GET, HEAD, The error changed.

Third Error - Invalid email

It seems like the config in the kubeconfig.yaml is incorrect. I am still investigating it.

[hollie]

Hello all,

for reference to other users: I too was experiencing the error:

msg="Error validating csrf cookie" 
error="CSRF cookie does not match state"

with the in the documentation recommended docker v2 image from docker hub.

The error happened after the initial cookie expired (which was 30 days after initial authentication).

I could work around the issue by using a private browser session.

After updating to the latest docker image the issue seems to be resolved.

Best regards,
Lieven.

[ghhv]

I find myself revisiting this thread again but this time, my issue was I didn't have the user in the WHITELIST - added that and voila! If that helps anyone..

[GuyKh]

I can confirm that although a similar issue (intermittently) persisted on tags :2 AND 2.2, they are resolved on :latest ( b364aa6a4117)

What would we, the ones running on arm do? :(

[blampe]

I can confirm that although a similar issue (intermittently) persisted on tags :2 AND 2.2, they are resolved on :latest ( b364aa6a4117)

What would we, the ones running on arm do? :(

@GuyKh you can use the 2.2.1 multi-arch image here https://github.com/Beanow/traefik-forward-auth/pkgs/container/traefik-forward-auth. Works like a charm.

Credit to @Beanow who built it as part of #275.

[GuyKh]

Nope... still getting

level=info msg="Missing csrf cookie" handler=AuthCallback ...