aws / aws-encryption-sdk-javascript

AWS Encryption SDK for Javascript and Node.js

AWS Encryption SDK for Javascript

The AWS Encryption SDK for Javascript provides a fully compliant, native Javascript implementation of the AWS Encryption SDK

Security issue notifications

See Support Policy for for details on the current support status of all major versions of this library.

Client Packages

Package	Description
@aws-crypto/client-browser	Client SDK for Web applications
@aws-crypto/client-node	Client SDK for Node.js client applications

These client packages have everything you need to encrypt/decrypt. They are the primary starting point. The AWS Encryption SDK for Javascript is built from a group of modularized packages. You can also compose the functional packages you need.

Functional Packages

Package	Description
@aws-crypto/encrypt-browser	Encrypt function for Web applications
@aws-crypto/encrypt-node	Encrypt function for Node.js client applications
@aws-crypto/decrypt-browser	Decrypt function for Web applications
@aws-crypto/decrypt-node	Decrypt function for Node.js client applications
@aws-crypto/kms-keyring-browser	Kms keyring for Web applications
@aws-crypto/kms-keyring-node	Kms keyring for Node.js client applications
@aws-crypto/raw-rsa-keyring-browser	Raw RSA keyring for Web applications
@aws-crypto/raw-rsa-keyring-node	Raw RSA keyring for Node.js client applications
@aws-crypto/raw-aes-keyring-browser	Raw AES keyring for Web applications
@aws-crypto/raw-aes-keyring-node	Raw AES keyring for Node.js client applications
@aws-crypto/caching-materials-manager-browser	Caching Materials Manager for Web applications
@aws-crypto/caching-materials-manager-node	Caching Materials Manager for Node.js client applications

Concepts

There are four main concepts that you need to understand to use this library:

Cryptographic Materials Managers

Cryptographic materials managers (CMMs) are resources that collect cryptographic materials and prepare them for use by the Encryption SDK core logic.

An example of a CMM is the default CMM, which is automatically generated anywhere a caller provides a keyring. The default CMM collects encrypted data keys from it's keyrings.

An example of a more advanced CMM is the caching CMM, which caches cryptographic materials provided by another CMM.

Keyrings

Keyrings use wrapping keys to generate, encrypt, and decrypt data keys. The keyring that you use determines the source of the unique data keys that protect each message, and the wrapping keys that encrypt that data key. An example of a keyring is the KmsKeyringNode.

An example of a more advanced keyring is the multi keyring. A multi keyring can be used to compose keyrings together.

Wrapping Keys

Wrapping keys are used to protect data keys. An example of a wrapping key is a KMS customer master key (CMK).

Data Keys

Data keys are the encryption keys that are used to encrypt your data. If your algorithm suite uses a key derivation function, the data key is used to generate the key that directly encrypts the data.

test

npm test

License

This SDK is distributed under the Apache License, Version 2.0, see LICENSE.txt and NOTICE.txt for more information.

Breaking changes from preview to 1.0.0

The AWS Encryption SDK for JavaScript is generally available as of October 1, 2019. There were breaking changes during the preview.

• Passing encryption context to encrypt is now { encryptionContext?: EncryptionContext } #148
• The return value of encrypt is now {result: Uint8Array, messageHeader: MessageHeader} #211
• encrypt strictly enforces plaintextLength #213

aws-encryption-sdk-javascript's Issues

We should support custom frame lengths in encrypt-browser

Currently, the frame length in encrypt-browser is determined by the plaintext size. We should support arbitrary frame lengths.

Not a launch blocker.

Browsers do not support 192 Bit Keys, remove these Algorithm Suites for browsers.

Improve UI/UX for keyring trace

@mattsb42-aws
mattsb42-aws 3 days ago
Not sure where the best place for this comment is (probably not this PR), but I am going to assert that returning the trace to the caller as this assembled binary representation is the wrong user experience. If a user has to do post-processing to understand the contents of this structure, that means we did the wrong thing.

An alternative might be to return the trace effectively as black box, then provide functions to call to determine whether that trace contains a certain flag.

Writing this down, I do realize that this function already exists for the value as written in the form of the bitwise AND operator.

I think the question we need to ask ourselves is: what is a language-idiomatic way of looking for a value in a container? If it is idiomatic to pass around a binary blob an perform bitwise AND and XOR operations, then ok; if it is not, then we should provide an idiomatic interface to that underlying definition.

@seebees
seebees 6 hours ago Author

This conversation should be in the material-management PR, but as you say ;)

I am not adverse to wrapping the raw object with some helper functions. From the spec conversation: I think it would be best to pin down what traces are and how they are used. What I have implemented here is minimally functional because I'm not clear on how people will use them.

In JS binary operations are not insane, but I would argue that they are never intuitive.

@mattsb42-aws
mattsb42-aws 2 minutes ago

I'm fine to move this to an issue and treat it as an UX enhancement.

Client-browser readme is missing fallback strategies and alternatives

The fallback section of the readme for the client-browser module lists the features of the SDK that are not supported by certain browsers. However, while it mentions msrcrypto as an example of a fallback library, it doesn't recommend any fallback strategies or alternatives.

Also, the fallback section points to @aws-crypto/web-crypto-backend, but that file has very little information, and just sends readers back to the readme. Loops like this are very frustrating to users.

[Verify] Humans should never need to write code that creates default CMMs

Related to #44, no human should ever need to write code that creates default CMMs.

I do not remember if this is the case yet, so am making this issue to track this requirement.

Add a decrypt oracle both client-browser and client-node

For the other Encryption SDK's to use for compatibility testing.

Browser Encryption signatures are not always encoded corectly

DER encoding is always signed.
Browsers use 'raw' encoding.
Where the r and s values are just concatenated.
If the first bit of r or s is 1,
an additional 0 byte should be added.
The current implementation of raw2der does not do this.

JavaScript instanceOf and different versions of the underlying Encryption SDK modules.

We use instanceOf to insure some security properties of Cryptographic Materials, and Encrypted Data Keys. However, under some conditions, versions of the dependent packages may result in copies of these files being installed (local node_modules folders) for the packages that have differing versions.
This means that these instanceOf checks may fail. Because while the code is the same, the actual instances are not.
What is the best direction?
How pervasive can this be?

client-node x-ray integration

Hi,

I've been looking into the source code and issues but could not find a way to trace the KMS requests made by client-node using AWS X-Ray.

Could you please confirm if that's possible?

Thank you.

investigate interaction between tags and branch protection

It would be great if we could do releases without disabling branch protection.
The blocker for this is whether or not we can push tags with branch protection turned on.
I think we can, because we can make them through the web UI, but I have not tested this.

Default CMM is a specific thing, not a set of default behavior

NodeCryptographicMaterialsManager and WebCryptoCryptographicMaterialsManager are not implementations of the root behavior that all CMMs need to do, they are explicitly implementations of the specific thing called the "default CMM". They should be named and structured accordingly.

Add CI checks to verify that our copied KMS types have not drifted

@mattsb42-aws
mattsb42-aws 3 days ago

That doesn't mean they'll never change. What happens if/when the SDK types change and we don't?

If having a common dependency is intractable, can we put something in our CI to test and make sure that the types are continuing to match?

@seebees
seebees 6 hours ago Author

If the types change and we don't nothing will happen to the run time. (In TypeScript all type information is removed at compile time).

The worst that will happen is that people who are doing development on new KMS Keyrings will not have as pleasant an experience until the types are updated.

However. This problem is not unique to me. Anyone who wants to consume the v3 sdk in this way will have a similar problem and if/as this happens it will be the JS SDK team that will have the burden to fix it.

Finally, breaking up the SDK in this way is explicitly done to avoid common dependancies. So while I could just include them, both the JS SDK team and myself have gone to great lengths to avoid this. I think it would be better to drive to a world where we don't need to.

(A CI style solution is in principle possible, but annoying as I changed the types so that they are independent of other things)

@mattsb42-aws
mattsb42-aws 2 minutes ago

The worst that will happen is that people who are doing development on new KMS Keyrings will not have as pleasant an experience until the types are updated.

My concern is more that I'm not seeing any way that we would know if the types are updated.

That said, this is also something that I am fine moving into an enhancement feature.

Add CI checks that verify boilerplate across modules

We have a lot of modules in this meta-repo and each module has some important boilerplate including but not limited to things like license, notice, etc.

We should have something in our CI that verifies that these are all consistent across all modules and the top-level repo.

Specific unit tests for ParseHeaderStream to decrypt-node

Support AWS SDK v2 and v3

I should support both versions. And I can use import types (https://www.typescriptlang.org/docs/handbook/release-notes/typescript-2-9.html) to simplify the dependancies.

Support KeyObjects in Node.js

From cursory investigation of Node Crypo module native code implementation, Node makes use of smart pointers to zero memory (for symmetric keys) or call OpenSSL helpers (pkey_free()) to clear private key memory contents when keys are no longer needed.

https://nodejs.org/api/crypto.html#crypto_class_keyobject

Encryption SDK Requires Node11

Node11 is not LTS and isn't even supported by AWS Lambda.

IMO, the encryption SDK should support NodeJS10 and maybe NodeJS8.

FWIW, these are the two versions which are currently supported by lambda.

Readme is hard to understand

Another HMAC-based Key Derivation Function for node.js.

Link to HKDF or KDF in Wikipedia

The function is very simple, but having a controlled, reviewed, and blessed version is valuable.

What does it mean for a function to be controlled? Also substitute a synonym (approved?) for blessed.

As a crypto primitive

Is this function a cryptographic primitive?

this is helpful to have, but the nodejs crypto module only exposes what openssl supports. Since it is so simple and derivable, there are no plans to include this in core.

If we're not using it, why do we have it in a module?

Decide on behavior when a keyring fails an operation

When a keyring or sub-keyring fails an operation, what side effect do we want?

Log a record of what happened? Do nothing? Something else?

This affects all keyring implementations.

Integration Browser investigation

Make sure that integration-browser can be run locally to test compatibility.

Encryption/Decryption Materials should verify that the curve matches the stuite

https://github.com/awslabs/aws-encryption-sdk-javascript/blob/master/modules/material-management/src/cryptographic_material.ts#L402

https://github.com/awslabs/aws-encryption-sdk-javascript/blob/master/modules/material-management/src/cryptographic_material.ts#L442

Serialized byte concatenation order should really be in serialize, not in each encrypt module

Ditto on decrypt and parsing.

Fix KDF behavior

The current behavior, as least for the webcrypto backend[1], is to only attempt a KDF if suite.kdf === 'HKDF' and suite.kdfHash is set. Otherwise, it treats this as a non-KDF suite.

The correct behavior is:

if (!kdf && !kdfHash) {
  // This is a non-KDF suite.
} else if (kdf === 'HKDF' && kdfHash) {
  // This is a valid HKDF suite. Do that.
} else {
  // This is an invalid or unsupported suite. Throw an appropriate error.
}

[1] https://github.com/awslabs/aws-encryption-sdk-javascript/blob/19786794821f282b61779d70e2db3d68fd95b28e/modules/material-management-browser/src/material_helpers.ts#L197

add Verdaccio publish/integration-test step to CI

To avoid issues with accidentally creating invalid/incomplete package artifacts, let's add a CI stage that spins up a Verdaccio server, publishes all packages to that server, then goes into a temp directory, installs everything from the Verdaccio server, and runs all integration tests.

Decide if we want to limit the input forms for RSA keys for the raw RSA keyrings

Encryption context vs context

In the process of requiring encryption context there are still many references to context. All these words would converge towards encryptionContext.
It is helpful to have the codebase be consistent and this kind of change can be especially tolerated in this beta period.

client-node relies on browser kms client

Importing the client-node imports the the sdk v3 kms node and kms browser clients. This defeats point of separating the browser and node clients

Bring browser and node encrypt/decrypt return values in sync

Change to:

ciphertext
plaintext

// browser-decrypt
{ messageHeader, clearMessage }
// node-decrypt
{ plaintext, messageHeader ?
// node-encrypt
{ ciphertext, messageHeader }
// browser-encrypt
{ messageHeader, cipherMessage }

plaintextLength should be and optional value for Node and Browser encrypt API

Wrap UTF8 encoding/decoding errors

All UTF8 encoding/decoding failures should be normalized to our external-facing errors.

Encryption context should be required everywhere but the initial user entry points

There are a lot of points in the codebase that accept an optional encryption context and default to an empty object if so. We should only do that default value assignment once, at the top-level user interface. Everywhere else, we should require that it is actually provided.

FrameLength can not be zero, and must be less than Maximum. FRAME_SIZE

Both of these need to be preconditions in serialize at a minimum

Better error messaging when no keyring is found.

Currently if no configured keyring is found to unwrap any of the encrypted data keys,
the CMM will throw an unencryptedDataKey has not been set error.
While this error is accurate, it is unhelpful.

Browser framed encryption is not slicing the given plaintext into frames.

If the plaintext given to encrypt is larger than the frame size,
encrypt is not breaking up the plaintext into frames properly.

Use oaepHash to support additional padding options.

Node.js v12.9 now supports oaepHash

https://nodejs.org/api/crypto.html#crypto_crypto_publicencrypt_key_buffer

Determine future for non-Arn KMS keyring key IDs

Either:

• Add support for non-Arn key IDs (key ID or alias name).
• Document that KMS keyring only supports Arns.

Make sure all HKDF test vectors from the AWE Encryption SDK for C exist here

Remove frame length field from encryption materials request

https://github.com/awslabs/aws-encryption-sdk-javascript/blob/master/modules/material-management/src/types.ts#L48

Decrypting AWS DB Activity Stream

I get the following error

Error: unencryptedDataKey has not been set
    at Object.needs (/var/task/node_modules/@aws-crypto/material-management/build/main/needs.js:29:15)
    at NodeDecryptionMaterial.getUnencryptedDataKey (/var/task/node_modules/@aws-crypto/material-management/build/main/cryptographic_material.js:180:17)
    at NodeDefaultCryptographicMaterialsManager.decryptMaterials (/var/task/node_modules/@aws-crypto/material-management-node/build/main/node_cryptographic_materials_manager.js:49:46)
    at process._tickCallback (internal/process/next_tick.js:68:7)

while running a lambda function to decrypt DB activity streams.

const aws = require("aws-sdk");
const {
    decrypt,
    RawAesKeyringNode,
    RawAesWrappingSuiteIdentifier,
} = require('@aws-crypto/client-node')

console.log('Loading function');
aws.config.logger = console;


exports.handler = async (event, context) => {
    const kms = new aws.KMS({ region: "us-west-2" });
    try {
        const output = await Promise.all(
            event.records.map(async (record) => {
                const data = Buffer.from(record.databaseActivityEvents, 'base64');
                const key = Buffer.from(record.key, 'base64');
                const promise = await kms.decrypt({
                    CiphertextBlob: key,
                    EncryptionContext: {
                        "aws:rds:dbc-id": process.env.cluster_id,
                    }
                }).promise();
                console.log(typeof promise.Plaintext, promise.Plaintext);
                const wrappingSuite = RawAesWrappingSuiteIdentifier.AES256_GCM_IV12_TAG16_NO_PADDING;
                const unencryptedMasterKey = new Uint8Array(promise.Plaintext);
                console.log(unencryptedMasterKey.byteLength);
                console.log(promise.Plaintext)
                const keyring = new RawAesKeyringNode({
                    keyName: "aes-name",
                    keyNamespace: "aes-namespace",
                    wrappingSuite: wrappingSuite,
                    unencryptedMasterKey: unencryptedMasterKey,
                });

                const d = await decrypt(keyring, record.databaseActivityEvents, {encoding: 'base64'});
                console.log(d);
            })
        );
        console.log(`Processing completed.  Successful records ${output.length}.`);
    } catch (err) {
        console.log(err);
    }
};

with Test Data

{
  "invocationId": "invocationIdExample",
  "deliveryStreamArn": "arn:aws:kinesis:EXAMPLE",
  "region": "us-west-2",
  "records": [
    {
      "type": "DatabaseActivityMonitoringRecords",
      "version": "1.0",
      "databaseActivityEvents": "AYADeNXig1LTUwpx7uXCSRBTBuUAXwABABVhd3MtY3J5cHRvLXB1YmxpYy1rZXkAREFqSVhETGpKajJ5djh2TFV0RHNuMFM1eDQ5emIvZy9CN1pmUSs4cUtGenJudXJwT3VnSlR0Nlhwc1p4RnZXL0hxUT09AAEAAkJDABtEYXRhS2V5AAAAgAAAAAw70KtkoLqQkIBTM7IAMMOHE9BBr0oWz9kbwm6gadhx4cL0BnBmyxZxwr+dNzYIQN8mF5MwNB2pTwvYc7eJqQIAAAAADAAAEAAAAAAAAAAAAAAAAAAn5FJ3KVLaFhkoQwqhk73u/////wAAAAEAAAAAAAAAAAAAAAEAAAJxBlzFL/29OBU67dwDfo6/iX3XxnwQsGHx65Ts8oNAHrjwZJ9Wl/nnQ2KaY/OdCq0Tv8+u9TUnTlRdWcsXEXZ2PAP9/ejJYsdlIW0kda/jeNfb2PLkj7vzvC7KzYpY8A9BQhc8a8bm5NNeTgh/ARjkUA7pdeG6AA4DXc5w8yl6aN42o+l3c1NNX2DedT+OF9xc6fs+dqkUEzdoHK6mShuaPWdSFwzLKtGfHC5z6bqpe5mI313pccO1WSEzVioFC0g5G7mQ2f2oMuh86BjEAxT9bkU8VmtxVstaDYcXvjvu4S91Lj3UGx1dHnkA3n5P1SSvy99N1xHZxz+sAndAZ9IlcC0+EnBmWppeLwqjRJSm0qxENOkqDF3AFCkiE70HVxmXyLg70+njWXEmHhlyzBCsw2RzeGfiqXuBV12EGWud3kPKHZ3+3UXkkmvHrt/01G8MdCgvhYawsP3K4MqoNt5iZcXtsgCx+6XouIaP+6qIKVFRPVW6T6ue5pEJzYU3AG0tZ8s0/F/I44bUXRKXeEIhfcJGiDdix/fOTVB5iy946RkaCWBxam7AAmC3JjvM5uVe8ZQ5ZjjrPLD+42n1/ny+rRLtY83AfmGCAF8Nv/yj+NQZE7at91gRNYeBjGQk8sX8JchTWwQNYrsgEMWOEZ9B34q6Hqb4im4lm32oJFpwecgTrWdHQ5afSt4A9u4y5BAJXtXqdpBkzP1Ail4Fqit+NYDxdiwuVM37a6baeslaJppzDOH9r40kxiDVDMgFTl/PgXSpL7RbQhWcIA0vcke2DHiO2kBWmHpphK7jLSpprFuVP/WJagSJbJ+ZGvI406UBYacZqWvAyfknVnGttJucETsAZzBlAjEArAzQjkTRcSAUdn0pXvR3D3HhtxGzLvkR48Frso4GzDlXkUIgGHgV7kS0AksB0OdxAjArbCAXaH1CXcMkt1OKlMXZfyeLwJn10fVJ/zxbrxl3b6aaGIR4EI4yTS6T2l8T82o=",
      "key": "AQIDAHj5sC4V75fw9OgpNzg8eJz30SjZJKlkaeCghpgU0ZZpcwGyagVf0Vv0OdZEh9ge6wKPAAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMIZT3HfWUMjZXZFp0AgEQgDtqf5tHswwRHSQNqlkXMuoVe2N+zfnJVJ0njS2es8vDqGm54lDCbUVMAIkaSZAx62ygv0IFD8UpExo3og=="
    }
  ]
}

I followed https://github.com/awslabs/aws-encryption-sdk-javascript/blob/master/modules/example-node/src/aes_simple.ts and https://docs.amazonaws.cn/en_us/AmazonRDS/latest/AuroraUserGuide/DBActivityStreams.html#DBActivityStreams.CodeExample

Not sure if this is a bug or I did something wrong. Any help would be much appreciated

Add test to verify that browsers still do not support 192-bit keys

Browsers do not support 192-bit AES keys at this time.

Just in case this ever changes, we should add a test to our browser test suite to verify this and error if this ever changes.

https://github.com/awslabs/aws-encryption-sdk-javascript/pull/57/files#diff-7e07c157f1bc227ae9c628e3fa51d0e1R48

Need data key caching example

Before release, or as a fast-follow, we need an example in JavaScript that is analogous to the ones in Java and Python.

https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/sample-cache-example.html

client-node relies on browser DOM types

Importing the client-node imports DOM types, this defeats the purpose of separating the browser and node clients.

Add max body size contraint on decrypt

We need to provide a control on decrypt that will allow the caller to specify a maximum body component size that they are willing to accept.

This value specifies the largest frame size or non-framed body size that they are willing to accept. If in processing the header and/or body header we find that the message contains a frame size or non-framed body larger than the specified size, we will stop and error out.

This is to allow the caller to protect themselves against resource DoS attacks.

Document and/or expand functionality for raw RSA keyring algorithm support

The four algorithm combinations we support for the raw RSA keyring are:

1. PKCS1v15
2. OAEP with SHA1 and MGF1 with SHA1
3. OAEP with SHA1 and MGF256 with SHA256
4. OAEP with SHA1 and MGF384 with SHA384
5. OAEP with SHA1 and MGF512 with SHA512

There is a feature gap between the node and browser keyrings. We need to either close or document this gap.

Message format default values should be centrallized in one place.

Verify correct behavior for plaintext length == frame size

We need to check the other implementations and write down the result, but I am 80% certain that if the plaintext is equal to the frame length, the resulting ciphertext should have a single frame that is a final frame.

If we determine that this is correct, we need to come back to encrypt-browser and change the frame size to plaintextLength rather than plaintextLength + 1.

investigate options for verifying package.json

After preview.0, let's see if we can find a checker that will analyze package.json files to look for issues.

publish code to npm

version 0.1.0-preview.1

Encyrption SDK requires you have aws sdkv3 and sdkv2 installed

AFAIK, the sdkv2 + v3 compatibility in https://github.com/awslabs/aws-encryption-sdk-javascript/blob/master/modules/kms-keyring/src/kms_types.ts#L28-L34 is not working as expected. It requires the user have both SDKV2 and V3 installed.

Update all copyright year statements appropriately

A lot of the copyright notices just state 2018. We need to update those to the correct statement.