aws-beam / aws_credentials Goto Github PK

I've noticed that the dialyzer will return an error when calling get_credentials() saying

The pattern can never match the type.

Pattern:
:undefined

Type:
%{
  :access_key_id => binary(),
  :credential_provider => _,
  :secret_access_key => binary(),
  :region => binary(),
  :token => binary()
}

But we have observed that it does return :undefined in failure cases.
This is being called from Elixir. Not sure if that should matter though.

Hey there!

It would be helpful to support web identity token authentication. This would allow using IRSA on EKS (https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/setting-up-enable-IAM.html) instead of having to rely on outdated strategies like kube2iam or hard-coded user access keys.

Here is an example implementation on the golang sdk: aws/aws-sdk-go@2e1d76a

The URL to fetch credentials information via the EC2 provider is constructed incorrectly.

Hi,

I'm using AWS's Security Hub service, and it recently created a finding for my environment, specifically [EC2.8] EC2 instances should use IMDSv2. The remediation details can be found at https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-standards-fsbp-controls.html#ec2-8-remediation, but in short, the new version of the service requires an extra API call to generate a token, and then subsequent API calls need to pass that token in a header.

I'm new to this project and thought I'd start by asking if you are aware of this change to IMDS and if anyone is already planning on making the corresponding code changes in the project. I'm willing to roll up my sleeves, but given that I'm new here, I thought I'd start with just bringing up the issue and starting a discussion.

Thanks for the projects you've created here. I'm just gaining familiarity but I like the idea of clients that uses code gen.

Hello,

I am trying to use this library and I found a problem with aws_credentials_provider:options().

According to the type declaration this type appear to be a nested map as shown below:

%% type declaration in aws_credentials_provider
%% -type options() :: #{provider() => map()}.

%% maybe like this
Options = #{
    aws_credentials_file => #{
        credential_path => "/path/to/aws/",
        profile => "my_profile"
    }
}.

However, in practice, the variable needs to be a flat map without a provider name, as shown in this example:

%% code in aws_credentials_file
-spec get_file_path(aws_credentials_provider:options()) -> {error, any()} | string().
get_file_path(Options) ->
  case maps:get(credential_path, Options, undefined) of
    undefined -> maybe_add_home("/.aws/");
    Path -> Path
  end.

Possible fixes:

1. Keep the current code and modify aws_credentials_provider:options() type to be a flat map to match current usage.
2. Change option values and usage to match the type declaration.

Method 1 is easier but makes it much harder to have different options for different providers in the future.
Method 2 is cleaner but breaks current library users, though some compatibility compromises could be made.

If I could choose, I would take method 2 with special credentials_path and profile options to maintain compatibility for current aws_credentials_file users.
(Luckily, there are no other usage except for those two options in aws_credentials_file.)

Thank you!

Build: https://github.com/aws-beam/aws_credentials/actions/runs/4450317306/jobs/7815626766 released version 0.0.0 rather than the intended 0.1.11 version. It seems to have been a transient issue on the Github Actions side from as far as I can tell due to the error below. Opening an issue in case it happens again it'll require some further investigation.

===> Getting log of git repo failed in /github/workspace. Falling back to version 0.0.0

Seeing this error when compiling with OTP 27:

===> Compiling aws_credentials
===> Compiling _build/default/plugins/katana_code/src/ktn_code.erl failed
_build/default/plugins/katana_code/src/ktn_code.erl:52:34: syntax error before: 'else'

_build/default/plugins/katana_code/src/ktn_code.erl:23:2: type tree_node_type() undefined
_build/default/plugins/katana_code/src/ktn_code.erl:62:15: type tree_node_type() undefined

==> mercury
** (Mix) Could not compile dependency :aws_credentials, "xxx/.asdf/installs/elixir/1.17.0-otp-27/.mix/elixir/1-17/rebar3 bare 
compile --paths xxx/_build/dev/lib/*/ebin" command failed. Errors may have been logged above. 
You can recompile this dependency with "mix deps.compile aws_credentials --force", update it with 
"mix deps.update aws_credentials" or clean it with "mix deps.clean aws_credentials"

Any help would be greatly appreciated.

As a developer using aws_credentials I would like the ability to run the application locally without any providers configured, and not get a huge exception stack trace in my logs that steals unnecessary focus.

The documentation says:

If you prefer to not log errors as they happen (the default behavior), but accumulate them for later, you can set erlang environment variable log_errors_immediately to false as in this example

However, both cases log immediately, except one uses ?LOG_INFO and the other uses ?LOG_ERROR. Is that intentional? Maybe log_level would be a better config name? Or maybe we just discard the config altogether?

@mrallen1 The aws_credentials module contains a un-hygienic macro (in the sense that relies on a variable which is defined outside of its scope) and that is supposed to keep backward compatibility with OTP 20 or older. Given that OTP 20 is from 2017, what's your view on dropping compatibility?

The aws-erlang client is using jsx. If we want to use aws_credentials as a dependency it will bring yet another JSON library. Moving from jsone to jsx solves this situation.

Using aws_credentials_ec2 provider failed with this error

Kernel pid terminated (application_controller) ("{application_start_failure,aws_credentials,{{shutdown,{failed_to_start_child,aws_credentials,{{badmatch,{error,enoent}},[{pubkey_os_cacerts,get,0,[{file,\"pubkey_os_cacerts.erl\"},{line,38}]},{httpc,ssl_verify_host_options,1,[{file,\"httpc.erl\"},{line,476}]},{httpc,'-http_options_default/0-fun-5-',0,[{file,\"httpc.erl\"},{line,1015}]},{httpc,http_options,3,[{file,\"httpc.erl\"},{line,961}]},{httpc,handle_request,9,[{file,\"httpc.erl\"},{line,771}]},{aws_credentials_httpc,request,6,[{file,\"/app/deps/aws_credentials/src/aws_credentials_httpc.erl\"},{line,71}]},{aws_credentials_ec2,fetch_session_token,0,[{file,\"/app/deps/aws_credentials/src/aws_credentials_ec2.erl\"},{line,52}]},{aws_credentials_ec2,fetch,1,[{file,\"/app/deps/aws_credentials/src/aws_credentials_ec2.erl\"},{line,40}]}]}}},{aws_credentials_app,start,[normal,[]]}}}")

It seems the package not compatible with the restricted security constraint introduced by erlang 26. https://www.erlang.org/blog/otp-26-highlights/#ssl-safer-defaults. We miss to explicitly append SSL options to the httpc calls, and maybe find a way to pull configs.

Calling force_credentials_refresh/0 twice, with the initial call not containing a finite timer, causes a failure with the following exception (in Elixir):

     ** (exit) exited in: :gen_server.call(:aws_credentials, {:force_refresh, %{}})
         ** (EXIT) an exception was raised:
             ** (ArgumentError) errors were found at the given arguments:
     
       * 1st argument: not a reference
     
                 :erlang.cancel_timer(:ok)
                 (aws_credentials 0.0.1) /Users/chrismoore/Projects/mercury/deps/aws_credentials/src/aws_credentials.erl:137: :aws_credentials.handle_call/3
                 (stdlib 4.0.1) gen_server.erl:1146: :gen_server.try_handle_call/4
                 (stdlib 4.0.1) gen_server.erl:1175: :gen_server.handle_msg/6
                 (stdlib 4.0.1) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

After some investigation, this seems to arise from the initial force_credentials_refresh call setting tref to ok atom:

    erlang:display(T),
    erlang:display(NewT),
    erlang:cancel_timer(T),

....

#Ref<0.760700117.1686896644.31319>
ok

This causes the subsequent call to fail, as there is no timer to cancel.

I believe here we should not perform a timer cancellation if T is ok.

EC2 credentials provider is throwing following error.

aws_credentials ignoring exception :error:{:badmatch, {:ok, 403, "", [ {'date', 'Mon, 23 Jan 2023 16:29:31 GMT'}, {'server', 'EC2ws'}, {'content-length', '0'}, {'content-type', 'text/plain'} ]}} ([ {:aws_credentials_ec2, :fetch_session_token, 0, [file: '/app/deps/aws_credentials/src/aws_credentials_ec2.erl', line: 46]},

Confirmed the error with CURL. It's because of extra '/' in the end of SESSION_TOKEN_URL.

`curl -XPUT -v http://169.254.169.254/latest/api/token/ -H "x-aws-ec2-metadata-token-ttl-seconds: 21600" -H "te: " -H "connection: keep-alive" -H "Host: 169.254.169.254"

• Trying 169.254.169.254:80...
• Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)

PUT /latest/api/token/ HTTP/1.1
Host: 169.254.169.254
User-Agent: curl/7.74.0
Accept: /
x-aws-ec2-metadata-token-ttl-seconds: 21600
connection: keep-alive

• Mark bundle as not supporting multiuse
  < HTTP/1.1 403 Forbidden
  < Content-Length: 0
  < Content-Type: text/plain
  < Date: Fri, 27 Jan 2023 15:36:10 GMT
  < Server: EC2ws
  <
• Connection #0 to host 169.254.169.254 left intact`

Works fine after removing / from the end in the URL

`curl -XPUT -v http://169.254.169.254/latest/api/token -H "x-aws-ec2-metadata-token-ttl-seconds: 21600"

• Trying 169.254.169.254:80...
• Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)

PUT /latest/api/token HTTP/1.1
Host: 169.254.169.254
User-Agent: curl/7.74.0
Accept: /
x-aws-ec2-metadata-token-ttl-seconds: 21600

• Mark bundle as not supporting multiuse
  < HTTP/1.1 200 OK
  < Content-Length: 56
  < Content-Type: text/plain
  < Date: Fri, 27 Jan 2023 15:34:39 GMT
  < Server: EC2ws
  < X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 21600
  <
• Connection #0 to host 169.254.169.254 left intact
  AQAEAJzntKEe7z7Zrt7o2zRKKtkff5mH4n6h3mfzxiFWIJA1fhiNlQ==`

Documentation also doesn't have a '/' in the end or URL. Source: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html

Let's introduce the Elvis linter, so we can start working with consistent style.

Hi, running into this issue when testing some elixir code in the Github workflow. The worflows run on some Azure servers, but apparently the http://169.254.169.254 is available on these servers for some microsoft purposes. This causes:

-spec fetch_session_token() -> {ok, session_token()}.
fetch_session_token() ->
  RequestHeaders = [{?SESSION_TOKEN_TTL_HEADER, ?SESSION_TOKEN_TTL_SECONDS}],
  {ok, 200, Body, _Headers} =
    aws_credentials_httpc:request(put, ?SESSION_TOKEN_URL, RequestHeaders),
  {ok, Body}.

to fail. Since there's no match for a 400 as evidenced:

** (Mix) Could not start application aws_credentials: :aws_credentials_app.start(:normal, []) returned an error: shutdown: failed to start child: :aws_credentials
    ** (EXIT) an exception was raised:
        ** (MatchError) no match of right hand side value: {:ok, 400, "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<Error xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\">\n    <Code>InvalidHttpVerb</Code>\n    <Message>The HTTP verb specified was not recognized by the server.</Message>\n    <Details>'PUT' is not a supported verb.</Details>\n</Error>", [{'date', 'Tue, 17 May 2022 16:25:43 GMT'}, {'server', 'Microsoft-IIS/10.0'}, {'content-length', '322'}, {'content-type', 'text/xml; charset=utf-8'}]}

I would expect this to simply fail to return credentials for the provider, rather than break the application start.

Am I thinking about this wrong? Would appreciate any advice.

When the config file (~/.aws/config) contains multiple profiles, then the header for each section contains a space, such as [profile dev]. Seems that eini cannot parse this. Error message:

** (Mix) Could not start application aws_credentials: :aws_credentials_app.start(:normal, []) returned an error: shutdown: failed to start child: :aws_credentials
    ** (EXIT) an exception was raised:
        ** (MatchError) no match of right hand side value: {:error, {:syntax_error, 5, [~c"syntax error before: ", [~c"\" \""]]}}
            (aws_credentials 0.1.11) /home/david/git/badger/cloud/cloud_dev_tools_cluster/tools/awsman/deps/aws_credentials/src/aws_credentials_file.erl:130: :aws_credentials_file.parse_config_file/2
            (aws_credentials 0.1.11) /home/david/git/badger/cloud/cloud_dev_tools_cluster/tools/awsman/deps/aws_credentials/src/aws_credentials_file.erl:87: :aws_credentials_file.maybe_add_region/3
            (aws_credentials 0.1.11) /home/david/git/badger/cloud/cloud_dev_tools_cluster/tools/awsman/deps/aws_credentials/src/aws_credentials_provider.erl:76: :aws_credentials_provider.evaluate_providers/3
            (aws_credentials 0.1.11) /home/david/git/badger/cloud/cloud_dev_tools_cluster/tools/awsman/deps/aws_credentials/src/aws_credentials.erl:182: :aws_credentials.fetch_credentials/1
            (aws_credentials 0.1.11) /home/david/git/badger/cloud/cloud_dev_tools_cluster/tools/awsman/deps/aws_credentials/src/aws_credentials.erl:123: :aws_credentials.init/1
            (stdlib 5.2) gen_server.erl:980: :gen_server.init_it/2
            (stdlib 5.2) gen_server.erl:935: :gen_server.init_it/6
            (stdlib 5.2) proc_lib.erl:241: :proc_lib.init_p_do_apply/3

We are currently experiencing dependency conflicts in our project due to the version of the JSX library we are using. To resolve these conflicts, we require an upgrade to at least version 3.1.0 of the JSX library.

Could you please consider bumping the version of the JSX package to a minimum of 3.1.0?

Currently no docs are published to hexdocs.pm

Until now there hasn't been a very clean way to fetch credentials from STS/AssumeRole functions, but it should be straightforward to use the aws_signature library to implement such a provider

I can imagine a scenario where I need to have, for the same execution profile, providers _env and _ec2 defined in that order.

e.g. locally, I use env. variables (so the test is Ok on the first provider), but in EC2 (for the same execution profile), _env fails, but _ec2 succeeds.

I'd like to not have an error in the logs unless all possible providers fail (I believe it plays well with the existing notion of order in provider-based credential fetching).