djc / gcp_auth

Minimal authentication library for Google Cloud Platform (GCP)

GCP Auth

GCP auth provides authentication using service accounts Google Cloud Platform (GCP)

GCP auth is a simple, minimal authentication library for Google Cloud Platform (GCP) providing authentication using service accounts. Once authenticated, the service account can be used to acquire bearer tokens for use in authenticating against GCP services.

The library supports the following methods of retrieving tokens in the listed priority order:

1. Reading custom service account credentials from the path pointed to by the GOOGLE_APPLICATION_CREDENTIALS environment variable. Alternatively, custom service account credentials can be read from a JSON file or string.
2. Look for credentials in .config/gcloud/application_default_credentials.json; if found, use these credentials to request refresh tokens. This file can be created by invoking gcloud auth application-default login.
3. Use the default service account by retrieving a token from the metadata server.
4. Retrieving a token from the gcloud CLI tool, if it is available on the PATH.

For more detailed information and examples, see the docs.

This crate does not currently support Windows.

Simple usage

The default way to use this library is to select the appropriate token provider using provider(). It will find the appropriate authentication method and use it to retrieve tokens.

let provider = gcp_auth::provider().await?;
let scopes = &["https://www.googleapis.com/auth/cloud-platform"];
let token = provider.token(scopes).await?;

License

Parts of the implementation have been sourced from yup-oauth2.

Licensed under MIT license.

gcp_auth's Issues

gcloud stores application user credentials in an SQLLite database

gcloud auth appears to no longer write a user's application credentials to application_default_credentials.json but instead to a user specific path under .config/gcloud/legacy_credentials/. I believe the change was done when the authors moved to use SQLite to manage the tokens.

I'd like to add support for using gcloud auth print-access-token directly but wanted to get feedback before committing to the process. Are there concerns or opinions about adding support for gcloud?

No tests but "Master's student" what a joke

Make it possible to parse secrets from string or bytes rather than providing path to file

Best wishes for the New Year 🎆

For one of my projects I need to generate tokens to interact with the GCP API. It looks like this is exactly what I need, especially since I'm looking for something simple instead of a full SDK.

In my case the JSON secrets file is stored in a database. Unless I have missed something, I believe reading the file from disk is currently the only option to use your library. Would you consider adding a function like:

gcp_auth::from_credentials_str

In this case the given string or slice of bytes already contains the content of the JSON file.

Possible to release new gcp_auth?

It looks like ring was upgraded in main branch but the latest release in crates.io hasn't picked this up and pins to "0.16.20"

My token looks like a morse code

Maybe I'm just dumb and didn't understand how to use the library. Sorry if that's the case, but I'm perplexed about this output.

Output:
ya29.c.b0AT7lpjA9DDSaBgwjFjWyjZ6iXNU3Lmok9NQmStz4hCc3RuPOw2GQB2x5rHiKqNkNyEJHlLzEUbkJ1NF_g7wNSgCXeLnLJonMLSfmi1Sr8ZrMFCG1nDZV1tymD5dnYWX2S3g2fECKqXvJrUwxDcwe2WRhuPZmse-rFTGAm3xUdk9RpLQYLpwzVHz8rLvg33ULx7agLUcuGLf81s5GW9M82PjiEcDIgjk........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

get_token never finishes

I am using the latest version (0.4.0)

I am getrring token from the following lines:

let authentication_manager = gcp_auth::init().await.unwrap();
authentication_manager.get_token(scopes).await.unwrap();

Execution never returns from this point.

Unable to load `application_default_credentials.json` on Windows

On 0.12.1, my application started failing to get the credentials. Upon investigation, I noticed that 0.11.1 wasn't using the application_default_credentials.json file, as I expected but using the gcloud command to get the authorized user (via GCloudAuthorizedUser).

The reason for failing to load application_default_credentials.json is that, according to the Google Cloud documentation, on Windows the application_default_credentials.json file is located at %APPDATA%\gcloud\application_default_credentials.json , but the the application tries to load it from %HOMEPATH%/.config/gcloud/application_default_credentials.json (looking at this line and this line.

gcp_auth::init() requires GOOGLE_APPLICATION_CREDENTIALS be defined

The absence of the environment variable GOOGLE_APPLICATION_CREDENTIALS should not preclude using DefaultServiceAccount and DefaultAuthorizedUser.

Using `#[tracing::instrument]` on `refresh_token()` leaks sensitive information to logs

I have discovered a lot of entries such as the following in my log files:

2022-05-20T08:29:28.053898Z DEBUG refresh_token: hyper::client::connect::http: connecting to 142.250.74.138:443 self=CustomServiceAccount { credentials: ApplicationCredentials { type: Some("service_account"), ...

The problem here, is that ApplicationCredentials contains a plaintext private key which is getting printed repeatedly to our log files, as well as a lot of other information one might not choose to expose so liberally such as service-account email and id, etc.

Perhaps using something like #[instrument(skip_all)] would be safer in this case?

Adjust logging level for `tracing` crate

A number of the functions in this library are instrumented with tracing::instrument, but that defaults to an INFO logging level. It would be helpful to set these to something higher like DEBUG so we can manage whether these low level methods will show up in the logs when we instrument our own applications.

Credentials loading order?

Hi!

Thanks for a great library, I am using it to build a small hobby project and so far everything has been running smoothly.

I noticed however, that the order of the credentials being loaded is not the same as other tools.

For examle, the Google documentation called Authenticating as a service account states:

ADC looks for service account credentials in the following order:

1. If the environment variable GOOGLE_APPLICATION_CREDENTIALS is set, ADC uses the service account file that the variable points to.

2. If the environment variable GOOGLE_APPLICATION_CREDENTIALS isn't set, ADC uses the default service account that Compute Engine, Google Kubernetes Engine, App Engine, Cloud Run, and Cloud Functions provide.

3. If ADC can't use either of the above credentials, an error occurs.

Version 0.2.2 of gcp_auth loads the credentials in this order:

1. Default service account aka instance metadata
2. Custom service account aka "GOOGLE_APPLICATION_CREDENTIALS"
3. "DefaultAuthorizedUser" aka "Application default credentials"

Would you consider changing the order to be more in line with other google tools?

I would personally prefer this order:

1. Custom service account aka "GOOGLE_APPLICATION_CREDENTIALS"
2. Default service account aka instance metadata
3. "DefaultAuthorizedUser" aka "Application default credentials"

Thanks!

Support audience

When invoking Cloud Run endpoints, token needs to have audience of the endpoint url.

Examples for using AuthenticationManager from many threads?

Hello!

I would like to know if there are any examples on how to use AuthenticationManager in a multithreaded context?

Using GOOGLE_APPLICATION_CREDENTIALS to specify an authorized_user or impersonated_service_account JSON key file giving "missing field `private_key`" error

Hi there, thanks for creating this nice package.

Does it support all kinds of different key jsons.

I tried to use the the json generated via
gcloud auth application-default login
and the key is located at /Users/fuyangl/.config/gcloud/application_default_credentials.json

CustomServiceAccountCredentials(Error("missing field `private_key`", line: 11, column: 1))

The key as "client_id", "client_secret", "refresh_token" and "type" as "authorized_user", that is all the fields there.

Unable to find the `gcloud` command on Windows

On 0.12.0 and 0.12.1, gcp_auth is unable to find the cloud command. This regression may be caused by the removal of the which crate, which was being used to locate the command on 0.11.1, but is not used on 0.12.0 and 0.12.1

Log for `0.12.0`

2024-05-29T07:58:59.067572Z DEBUG provider: gcp_auth: initializing gcp_auth
2024-05-29T07:58:59.067780Z DEBUG provider: gcp_auth::custom_service_account: check for GOOGLE_APPLICATION_CREDENTIALS env var
2024-05-29T07:58:59.072484Z DEBUG provider: gcp_auth::config_default_credentials: try to load credentials from .config/gcloud/application_default_credentials.json
2024-05-29T07:58:59.072926Z DEBUG provider: gcp_auth::metadata_service_account: try to fetch token from GCP instance metadata server
2024-05-29T07:58:59.073294Z DEBUG provider:fetch_token: gcp_auth::types: requesting token url=http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token provider="MetadataServiceAccount"
2024-05-29T07:58:59.073929Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="metadata.google.internal"
2024-05-29T07:58:59.078290Z  WARN provider:fetch_token: gcp_auth::types: failed to refresh token, trying again... err=Other("HTTP request failed", hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Os { code: 11001, kind: Uncategorized, message: "No such host is known." }))) provider="MetadataServiceAccount" retries=0
2024-05-29T07:58:59.079130Z DEBUG provider:fetch_token: gcp_auth::types: requesting token url=http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token provider="MetadataServiceAccount"
2024-05-29T07:58:59.079422Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="metadata.google.internal"
2024-05-29T07:58:59.079742Z  WARN provider:fetch_token: gcp_auth::types: failed to refresh token, trying again... err=Other("HTTP request failed", hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Os { code: 11001, kind: Uncategorized, message: "No such host is known." }))) provider="MetadataServiceAccount" retries=1
2024-05-29T07:58:59.080038Z DEBUG provider:fetch_token: gcp_auth::types: requesting token url=http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token provider="MetadataServiceAccount"
2024-05-29T07:58:59.080306Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="metadata.google.internal"
2024-05-29T07:58:59.080620Z  WARN provider:fetch_token: gcp_auth::types: failed to refresh token, trying again... err=Other("HTTP request failed", hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Os { code: 11001, kind: Uncategorized, message: "No such host is known." }))) provider="MetadataServiceAccount" retries=2
2024-05-29T07:58:59.080921Z DEBUG provider:fetch_token: gcp_auth::types: requesting token url=http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token provider="MetadataServiceAccount"
2024-05-29T07:58:59.081178Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="metadata.google.internal"
2024-05-29T07:58:59.081468Z  WARN provider:fetch_token: gcp_auth::types: failed to refresh token, trying again... err=Other("HTTP request failed", hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Os { code: 11001, kind: Uncategorized, message: "No such host is known." }))) provider="MetadataServiceAccount" retries=3
2024-05-29T07:58:59.081753Z DEBUG provider:fetch_token: gcp_auth::types: requesting token url=http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token provider="MetadataServiceAccount"
2024-05-29T07:58:59.082011Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="metadata.google.internal"
2024-05-29T07:58:59.082268Z  WARN provider:fetch_token: gcp_auth::types: failed to refresh token, trying again... err=Other("HTTP request failed", hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Os { code: 11001, kind: Uncategorized, message: "No such host is known." }))) provider="MetadataServiceAccount" retries=4
2024-05-29T07:58:59.082537Z DEBUG provider: gcp_auth::gcloud_authorized_user: try to print access token via `gcloud`
Error: NoAuthMethod(Io("failed to run `gcloud`", Error { kind: NotFound, message: "program not found" }), Other("HTTP request failed", hyper_util::client::legacy::Error(Connect, ConnectError("dns error", Os { code: 11001, kind: Uncategorized, message: "No such host is known." }))), Io("failed to open application credentials file", Os { code: 3, kind: NotFound, message: "The system cannot find the path specified." }))

Log for 0.11.1

2024-05-28T19:58:57.622868Z DEBUG new: gcp_auth::authentication_manager: Initializing gcp_auth
2024-05-28T19:58:57.627715Z DEBUG new: gcp_auth::default_authorized_user: Loading user credentials file
2024-05-28T19:58:57.628029Z DEBUG new:get_token{client=Client}: gcp_auth::default_service_account: Getting token from GCP instance metadata server
2024-05-28T19:58:57.628913Z DEBUG hyper::client::connect::dns: resolving host="metadata.google.internal"
2024-05-28T19:58:57.645776Z  WARN new:get_token{client=Client}: gcp_auth::default_service_account: Failed to get token from GCP instance metadata server: error trying to connect: dns error: No such host is known. (os error 11001), trying again...
2024-05-28T19:58:57.646494Z DEBUG hyper::client::connect::dns: resolving host="metadata.google.internal"
2024-05-28T19:58:57.646779Z  WARN new:get_token{client=Client}: gcp_auth::default_service_account: Failed to get token from GCP instance metadata server: error trying to connect: dns error: No such host is known. (os error 11001), trying again...
2024-05-28T19:58:57.646919Z DEBUG hyper::client::connect::dns: resolving host="metadata.google.internal"
2024-05-28T19:58:57.647205Z  WARN new:get_token{client=Client}: gcp_auth::default_service_account: Failed to get token from GCP instance metadata server: error trying to connect: dns error: No such host is known. (os error 11001), trying again...
2024-05-28T19:58:57.647400Z DEBUG hyper::client::connect::dns: resolving host="metadata.google.internal"
2024-05-28T19:58:57.647647Z  WARN new:get_token{client=Client}: gcp_auth::default_service_account: Failed to get token from GCP instance metadata server: error trying to connect: dns error: No such host is known. (os error 11001), trying again...
2024-05-28T19:58:57.647834Z DEBUG hyper::client::connect::dns: resolving host="metadata.google.internal"
2024-05-28T19:58:57.648058Z  WARN new:get_token{client=Client}: gcp_auth::default_service_account: Failed to get token from GCP instance metadata server: error trying to connect: dns error: No such host is known. (os error 11001), trying again...
2024-05-28T19:58:59.817829Z DEBUG new: gcp_auth::authentication_manager: Using GCloudAuthorizedUser
2024-05-28T19:58:59.819169Z DEBUG reqwest::connect: starting new connection: https://us-central1-aiplatform.googleapis.com/
2024-05-28T19:58:59.819376Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="us-central1-aiplatform.googleapis.com"
2024-05-28T19:58:59.820883Z DEBUG hyper_util::client::legacy::connect::http: connecting to 142.250.179.234:443
2024-05-28T19:58:59.829227Z DEBUG hyper_util::client::legacy::connect::http: connected to 142.250.179.234:443
2024-05-28T19:59:00.487182Z DEBUG hyper_util::client::legacy::pool: pooling idle connection for ("https", us-central1-aiplatform.googleapis.com)

background token refresh

We're seeing that refreshing tokens from outside of a GCP env is slow, often around 1 second for us. Because of this we'd like to get the refresh done in background outside of the get_token call. This is possible but awkward currently as there's no direct way for a consumer to trigger refresh other than to call get_token in the last 20 seconds of the token lifetime.

Ideally the library would offer a background refresh option, or if not that, the ability to force a refresh.

I can put together a PR for either of these approaches if this is something you're interested in.

License contains references to yup-oauth2

You might want to fix that up. :)

Tokens fetched via a GCloudAuthorizedUser do not refresh

It seems that GCloudAuthorizedUser::from_string defaults the token's expiry_time to None, which causes Token::has_expired to always return false. This results in AuthenticationManager::get_token never refreshing a token that was retrieved via a GCloudAuthorizedUser.

Are these tokens not intended to be refreshed? Would it be unreasonable to default the expiry_time to one hour from when from_string is called? Something like:

expires_at: Some(OffsetDateTime::now_utc() + Duration::hours(1))

I don't see anything about extending the token's expiry time in the documentation for the CLI command used to fetch the token, but I suppose it's possible the expiry time's default could be overridden somewhere else.