awsdocs / aws-doc-sdk-examples

Welcome to the AWS Code Examples Repository. This repo contains code examples used in the AWS documentation, AWS SDK Developer Guides, and more. For more information, see the Readme.md file below.

AWS SDK Code Examples

This repository contains code examples that demonstrate how to use the AWSK SDKs to interact with AWS services.

Many examples are injected into the AWS Documentation.

How this repository is organized

Code examples for each language's SDK can be found within the following subdirectories. The examples here demonstrate the most common uses of the SDK for each language.

SDK	folder	SDK version
.NET	dotnetv3/	3.5+
.NET	dotnet/	<3.5
C++	cpp/	1
Go	gov2/	2
Go	go/	1
Java	javav2/	2
Java	java/	1
JavaScript	javascriptv3/	3
JavaScript	javascript/	2
Kotlin	kotlin/	1
PHP	php/	3
Python	python/	3
Ruby	ruby/	3
Rust	rustv1/	1
Swift	swift/	preview

Within each directory, you will find SDK-specific instructions for understanding and invoking example code.

NOTE

In alignment with our SDKs and Tools Maintenance Policy, the AWS SDK for Java v1.x will enter maintenance mode on July 31, 2024, and reach end-of-support on December 31, 2025.

For more information, see Announcing end-of-support for AWS SDK for Java v1.x.

Additional directories

directory	purpose	usage
/applications	Contains the non-language-specific components of example applications, which show how the SDKs can be used in the context of a production-like application.	To view the language-specific components for each example application, see the cross-service folder in the sub-directory for your desired language (such as python/cross-service).
/test	Contains all components supporting the custom test automation framework used to routinely test the code in this repository.	Deploys to AWS as a polyglot container-based integration testing solution. WARNING: Still under active construction as of 2023.
/resources	Contains shared components used by many code examples across this repository.	Deploys as frontend (/clients) or backend (/cdk or /cfn) components.

Invoke example code

To invoke this example code, you must have an AWS account. For more information about creating an account, see AWS Free Tier.

You must also have AWS credentials configured. For steps on using the AWS Command Line Interface (AWS CLI) to configure credentials, see CLI Configuration basics

⚠️ Usage disclaimer

These code examples interact with services that may incur charges to your AWS account. For more information, see AWS Pricing.

Additionally, example code might theoretically modify or delete existing AWS resources. As a matter of due diligence, do the following:

• Be aware of the resources that these examples create or delete.
• Be aware of the costs that might be charged to your account as a result.
• Back up your important data.

Contributing

This repository thrives on your contributions! ❤️ To get involved, see the CONTRIBUTING.md. 🙏

Copyright and license

All content in this repository, unless otherwise stated, is Copyright © Amazon Web Services, Inc. or its affiliates. All rights reserved.

Except where otherwise noted, all examples in this collection are licensed under the Apache license, version 2.0 (the "License"). The full license text is provided in the LICENSE file accompanying this repository.

aws-doc-sdk-examples's Issues

C++ S3 PutObjectRequest where source is a buffer rather than a file

I would love to see an example where I can take some in-memory data (eg: uint8_t * and upload it to S3 as an object. Can you help!?

JavaScript Template Example | TypeError: AWS.config.mediaconvert is not a function

I'm experiencing an error trying to run the JavaScript template example out-of-the-box.

AWS.config.mediaconvert({endpoint: customEndpoint});
           ^

TypeError: AWS.config.mediaconvert is not a function

If I adjust the code to skip .mediaconvert, then I see another error:

Adjustment:

AWS.config.update({endpoint: customEndpoint});

Error:

Error { MissingRequiredParameter: Missing required key 'OutputGroups' in params.Settings
    at ParamValidator.fail (.../node_modules/aws-sdk/lib/param_validator.js:50:37)
    at ParamValidator.validateStructure (.../node_modules/aws-sdk/lib/param_validator.js:61:14)
    at ParamValidator.validateMember (.../node_modules/aws-sdk/lib/param_validator.js:88:21)
    at ParamValidator.validateStructure (.../node_modules/aws-sdk/lib/param_validator.js:75:14)
    at ParamValidator.validateMember (.../node_modules/aws-sdk/lib/param_validator.js:88:21)
    at ParamValidator.validate (.../node_modules/aws-sdk/lib/param_validator.js:34:10)
    at Request.VALIDATE_PARAMETERS (.../node_modules/aws-sdk/lib/event_listeners.js:125:42)
    at Request.callListeners (.../node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at callNextListener (.../node_modules/aws-sdk/lib/sequential_executor.js:95:12)
    at .../node_modules/aws-sdk/lib/event_listeners.js:85:9
  message: 'Missing required key \'OutputGroups\' in params.Settings',
  code: 'MissingRequiredParameter',
  time: 2018-08-23T13:31:06.278Z }

This error seems to be consistent when I try to the call the endpoint in other tests.

node -v: v8.11.3
npm -v: 6.3.0

"dependencies": {
    "aws-sdk": "^2.283.1"
}

var AWS = require('aws-sdk');

var fileInput = "s3://bucket/input/sample-video.mp4";
var destination = "s3://bucket/output/";
var role = "arn:aws:iam::000000000000:role/MediaConvertRole";
var queue = "arn:aws:mediaconvert:us-west-2:000000000000:queues/Default";
var customEndpoint = "https://something.mediaconvert.us-west-2.amazonaws.com";
var templateName = "My-Template";

AWS.config.update({region: 'us-west-2'});
AWS.config.update({endpoint: customEndpoint});

var params = {
  "Queue": queue,
  "JobTemplate": templateName,
  "Role": role,
  "Settings": {
    "Inputs": [
      {
        "AudioSelectors": {
          "Audio Selector 1": {
            "Offset": 0,
            "DefaultSelection": "NOT_DEFAULT",
            "ProgramSelection": 1,
            "SelectorType": "TRACK",
            "Tracks": [
              1
            ]
          }
        },
        "VideoSelector": {
          "ColorSpace": "FOLLOW"
        },
        "FilterEnable": "AUTO",
        "PsiControl": "USE_PSI",
        "FilterStrength": 0,
        "DeblockFilter": "DISABLED",
        "DenoiseFilter": "DISABLED",
        "TimecodeSource": "EMBEDDED",
        "FileInput": fileInput
      }
    ]
  }
};

// Create a promise on a MediaConvert object
var templateJobPromise = new AWS.MediaConvert({apiVersion: '2017-08-29'}).createJob(params).promise();

// Handle promise's fulfilled/rejected status
templateJobPromise.then(
  function(data) {
    console.log("Success! ", data);
  },
  function(err) {
    console.log("Error", err);
  }
);

Any way to configure the manager to download to HDFS in Java?

aws-doc-sdk-examples/java/example_code/s3/src/main/java/aws/example/s3/XferMgrDownload.java

Line 39 in 0893f12

try {

cpp build issue

Any idea why I have this build issue when I try to build the cpp example code?

---

cpp/example_code/s3/get_bucket_policy.cpp:54:50: error: no match for ‘operator<<’ (operand types are ‘std::basic_ostream::__ostream_type {aka std::basic_ostream}’ and ‘Aws::IOStream {aka std::basic_iostream}’)
std::cout << "Policy: " << std::endl <<
^

---

Method used to get InstanceID in CreateInstance.java example is incorrect

While trying to play around with the example(CreateInstance.java) snippets, I noticed that below is used to get instanceID:-

String instance_id = run_response.getReservation().getReservationId();

This is incorrect, as this gives the reservation ID of the format "r-" and correct way to get Instance ID is actually this:-

run_response.getReservation().getInstances();

Where are you connected to aws ?

Hi,
in all your ruby code for aws, I did not see where are you using the credentials to connect to aws, can you please show me where ?
Thanks

Batch Write is written wrong to get the correct response

// Load the AWS SDK for Node.js
var AWS = require('aws-sdk');
// Set the region
AWS.config.update({region: 'us-west-2'});

// Create DynamoDB service object
var ddb = new AWS.DynamoDB({apiVersion: '2012-08-10'});

var params = {
RequestItems: {
"CUSTOMER_LIST": [
{
PutRequest: {
Item: {
"CUSTOMER_ID": { "N": "005" },'CUSTOMER_NAME' : {S: 'Andy Owens'},
"DOG": { "S": "YES" },
"CAT": { "S": "YES" },
"HOUSE": { "S": "NO" },
"CAR": { "S": "NO" }
}
}
},
{
PutRequest: {
Item: {
"CUSTOMER_ID": { "N": "006" },'CUSTOMER_NAME' : {S: 'Grim Reaper'},
"DOG": { "S": "NO" },
"CAT": { "S": "NO" },
"HOUSE": { "S": "NO" },
"CAR": { "S": "NO" }
}
}
}
]
}
};

ddb.batchWriteItem(params, function(err, data) {
if (err) {
console.log("Error", err);
} else {
console.log("Success", data);
}
});
Instead of the array containing a comma separated list with two put requests each within own brackets. I've made a pull request, I hope that it gets accepted : )

scan sample code question

why is the line
KeyConditionExpression: 'Season = :s and Episode > :e',

present in the params of the scan sample code ? this seems to cause an error at runtime:
UnexpectedParameter: Unexpected key 'KeyConditionExpression' found in params

feedback welcome ...

[aws-sdk-cpp DynamoDB] BatchGetItem sample (Difference with GetItem)

I check the API and tried to implement a BatchGetItem Similar to GetItem(req).

Aws::DynamoDb:::Model::BatchGetItemOutcome &result = dynamodbClient.batchGetItem(req);

From the looks of it, I could not get it working. Is there a better example or plan to have a sample committed here for the users to get familiar with how the internals works and identify the difference between them to decide when to use them.

Policy is broken

&region-arn; should be arn:aws: here:

aws-doc-sdk-examples/iam_policies/secretsmanager/asm-resource-policy-grant-gsv-on-only-awscurrent-to-role.json

Line 6 in b51d535

"Principal": {"AWS": "&region-arn;iam::123456789012:role/EC2RoleToAccessSecrets",

How to config the credentials for DynamoDB using aws-sdk js?

import AWS = require('aws-sdk');

 AWS.config.region = region;
 AWS.config.credentials = new AWS.CognitoIdentityCredentials({
      IdentityPoolId: identityPoolId,
 }, {
      region
});
this.docClient = new DynamoDB.DocumentClient({ region: AWS.config.region });

I am using cognito as well. With the above setup, it complains about missing credentials.
The aws-sdk document is up-to-date?

using "aws-sdk": "^2.209.0"

Missing result from describeInstances call in JavaScript

Hi, I am stuck at testing few basic calls with missing result so I am looking for some help here.
Here's the link for the sample code.
https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/javascript/example_code/ec2/ec2_describeinstances.js

var AWS = require('aws-sdk');
AWS.config.update({region:'us-west-2'});

var ec2 = new AWS.EC2({apiVersion: '2016-11-15'});

var params = {
  DryRun: false,
}

ec2.describeInstances(params, function(err, data) {
  if (err) {
    console.log('Error', err.stack);
  }
  else {
    console.log(JSON.stringify(data, null, 4));
  }
});

It's very basic call but it does not return all EC2 instances. It only returns 62 results.
All EC2 instances are under same region and all of them are clearly shown at console.

Am I missing anything?
I use node 8.11.3.
Will be very much appreciated if I get some clue. Thanks,

Sync objects to another bucket

I'm looking for some code to copy a bunch of objects (an HLS stream with very many segments) to another bucket with a prefix, as efficiently as possible.

BONUS: this sync job might have been already run and it will be a waste to redo the copy inefficiently.

I noticed https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/go/example_code/s3/s3_copy_object.go but it would be better to have some sort of aws s3 sync starting points in golang for example.

Query VS Scan

I was reading query vs scan dynamoDb on AWS docs page.

aws-doc-sdk-examples/javascript/example_code/dynamodb/ddb_scan.js

Line 35 in 978bae5

ddb.query(params, function(err, data) {

Should it not be ddb.scan(params, function(err, data) { ?

Put object example

Not sure whether I don't understand the example or whether it contains a bug.

Aws::MakeShared<Aws::FStream>(key_name.c_str(), dir_name.c_str(), std::ios_base::in);

key_name is passed into the allocationTag parameter of MakeShared. Is this correct? Usage example says it's the filename to upload..
dir_name (".") is passed to FStream, I doubt that's correct.

[AWS SDK for Java 2.0] Cloudfront Example

[AWS SDK for Java 2.0]
Could you add a Create Cloudfront Distribution from S3 example?
Thanks~~~~

Scan example is empty!

See aws-doc-sdk-examples/java/example_code/dynamodb/src/main/java/aws/example/dynamodb/Scan.java on master branch

Could not unzip uploaded file. ...

In creating a lambda function in Ruby, when supplying a local zip file myfile.zip via the zip_file option as follows

resp = lambda.create_function({
  function_name: lfparams['func_name'], # required
  runtime: lfparams['runtime'], # required, accepts nodejs, nodejs4.3, java8, python2.7
  role: lfparams['func_role'], # required
  handler: lfparams['handler'], # required
  code: { # required
   zip_file: "myfile.zip",
  },
  description: lfparams['func_desc'], # may be the version or stage environment ?
  timeout: lfparams['timeout'],
  memory_size: lfparams['memory'],
  publish: false,
})

I got the error but the zip file is OK if I manually upload it to S3 and use S3 inside code: {...} like
...
code: { # required
s3_bucket: "bucker_name",
s3_key: "myfile.zip",
},
...

Any idea please ?

Thanks

Ken

[C++ SDK] example code to access Redshift

I'm trying to use this SDK to access Redshift in my c++ application, but couldn't find any example code.
By looking at the documentation, I couldn't figure out the API to get items from tables.
Is there an API that I can use to fetch data from Redshift tables? Similar to Aws::DynamoDB::Model::GetItemRequest

Go examples imports and convention

For a handful of the examples imports need to be updated, Additionally golang convention is to never use underscores in variable names, that exists in a few files under s3/

Wrong variable used in JavaScript code. It should be only ACCOUNT_ID.

aws-doc-sdk-examples/javascript/example_code/cognito/cognito_getcreds.html

Line 33 in 33335a4

AccountId: AWS_ACCOUNT_ID

Go Create Enable Alarms example doesn't compile

It's missing a comma at the end of this line.

aws-doc-sdk-examples/go/example_code/cloudwatch/create_enable_alarms.go

Line 49 in 50d7b39

AlarmName: aws.String(name)

Missing result from describeInstances call

Hi, I reopen the issue as it's closed without proper explanation.

I am stuck at testing few basic calls with missing result so I am looking for some help here.
Here's the link for the sample code.
https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/javascript/example_code/ec2/ec2_describeinstances.js

var AWS = require('aws-sdk');
AWS.config.update({region:'us-west-2'});

var ec2 = new AWS.EC2({apiVersion: '2016-11-15'});

var params = {
  DryRun: false,
}

ec2.describeInstances(params, function(err, data) {
  if (err) {
    console.log('Error', err.stack);
  }
  else {
    console.log(JSON.stringify(data, null, 4));
  }
});

It's very basic call but it does not return all EC2 instances. It only returns 63 results.
All EC2 instances are under same region and all of them are clearly shown at console.

And it does not return NextToken.

When I tried describe instances using aws cli at same instance, I get all 76 instances.
All instances are in same region.

$ aws ec2 describe-instances --query 'Reservations[].Instances[].[InstanceId]' --output text | wc -l
76

Am I missing anything?
I use node 8.11.3.
Will be very much appreciated if I get some clue.

Add (mostly C++; one Python) SDK examples

I have been thinking of adding,

• SNS
  • C++ #189
• GuardDuty
  • C++ #209
• SES
  • C++ #208
  • Python #188
• EBS
  • C++ #221
• Elastic File System
  • C++ #212
• CloudTrail
  • C++ #210
• CodeCommit
  • C++ #215
• CodeBuild
  • C++ #214
• ElastiCache
  • C++ #220
• Neptune
  • C++ #211
• Glacier
  • C++ #213
• Storage Gateway
  • C++ #219

The C++ code samples are near to none for the above over the web and code examples of later ones are not present in the documentation as sample codes like other example codes. Please let me know if you think it would be useful to add example code for the above in this repository, would be happy to add also if I should expand any of the above from the list to describe the specific api calls.

AWS SES GoLang Credentials

Hi,

I'm trying to use this example from your GoLang SDK, to send mails programmatically.

But, when I run a similar code, I have the following error:

NoCredentialProviders: no valid providers in chain. Deprecated. 
For verbose messaging see aws.Config.CredentialsChainVerboseErrors

Which is quite normal as the example code doesn't mention at all any API keys.

I think there is something (i.e. API id and key) to provide somewhere, like here

Can you provide me some informations about that?
Regards,
Adrien

Missing pagination

The delete example https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/go/example_code/s3/s3_delete_objects.go seems to max out at 1000 objects with it's listObjects call. I.e. can I assume it's not paginating?

Be good if this example could be please be improved. For e.g. underscores in variable names num_Objs is a no go.

If empty queue, example crashes

Suggest this change:

From:
sqs.receiveMessage(params, function(err, data) {
if (err) {
console.log("Receive Error", err);
} else {

To:
sqs.receiveMessage(params, function(err, data) {
if (err) {
console.log("Receive Error", err);
} else if (data.Messages) {

How to create parameter store ?

I'm trying to create parameter store to store access & secret keys. But I seems I couldn't find it.
Here is what I have tried but, I don't if this the right API :

func createParameterStoreKey(svc *kms.KMS, key, value string) error {
	// Create the key
	_, err := svc.CreateKey(&kms.CreateKeyInput{
		Tags: []*kms.Tag{
			{
				TagKey:   aws.String(key),
				TagValue: aws.String(value),
			},
		},
	})

	if err != nil {
		return err
	}

	return nil
}

using above code doesn't create anything in parameter store. please help ...

NextToken example

Hit a pagnating API so I grepped this repo for good examples for getting output from such an API.

However the two grepped mentions of "NextToken" I found appear invalid and misleading:

• https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/go/example_code/cloudwatch/describe_alarms.go
• https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/go/example_code/cloudwatch/describe_alarm_history.go

Perhaps they could be please be fixed and as a good example?

From https://github.com/search?l=Go&q=aws+%22NextToken%22&type=Code&utf8=%E2%9C%93 I think has starting points.

Dynamo Get/Delete item doesn't match the rest of the tutorial so it doesn't work

https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/javascript/example_code/dynamodb/ddb_getitem.js

all of the ddb_* files reference a CUSOMTER_LIST, ddb_getitem.js, ddb_deleteitem.js is referencing a table that doesn't exist

Unicode support in cpp S3 example broken on Linux

The new Unicode example code is using Windows WString which does not exist on Linux. The core SDK is fine, it is the examples that need to be fixed.

[ 44%] Building CXX object CMakeFiles/put_object_unicode.dir/put_object_unicode.cpp.o
/home/jonsmirl/aosp/aws-doc-sdk-examples/cpp/example_code/s3/put_object_unicode.cpp: In function ‘int main(int, char**)’:
/home/jonsmirl/aosp/aws-doc-sdk-examples/cpp/example_code/s3/put_object_unicode.cpp:29:20: error: ‘WString’ in namespace ‘Aws’ does not name a type
const Aws::WString key_name = L"unicode-Ⓢ③Ⓢ③Ⓒⓛⓘ.txt";
^
/home/jonsmirl/aosp/aws-doc-sdk-examples/cpp/example_code/s3/put_object_unicode.cpp:30:20: error: ‘WString’ in namespace ‘Aws’ does not name a type
const Aws::WString file_name = L"Ⓢ③Ⓢ③Ⓒⓛⓘ.txt";
^
/home/jonsmirl/aosp/aws-doc-sdk-examples/cpp/example_code/s3/put_object_unicode.cpp:42:31: error: ‘FromWString’ is not a member of ‘Aws::Utils::StringUtils’
object_request.SetKey(Aws::Utils::StringUtils::FromWString(key_name.c_str()));
^
/home/jonsmirl/aosp/aws-doc-sdk-examples/cpp/example_code/s3/put_object_unicode.cpp:42:68: error: ‘key_name’ was not declared in this scope
object_request.SetKey(Aws::Utils::StringUtils::FromWString(key_name.c_str()));
^
/home/jonsmirl/aosp/aws-doc-sdk-examples/cpp/example_code/s3/put_object_unicode.cpp:46:13: error: ‘file_name’ was not declared in this scope
file_name.c_str(), std::ios_base::in | std::ios_base::binary);
^
/home/jonsmirl/aosp/aws-doc-sdk-examples/cpp/example_code/s3/put_object_unicode.cpp: At global scope:
/home/jonsmirl/aosp/aws-doc-sdk-examples/cpp/example_code/s3/put_object_unicode.cpp:22:14: error: unused parameter ‘argc’ [-Werror=unused-parameter]
int main(int argc, char** argv)
^
/home/jonsmirl/aosp/aws-doc-sdk-examples/cpp/example_code/s3/put_object_unicode.cpp:22:27: error: unused parameter ‘argv’ [-Werror=unused-parameter]
int main(int argc, char** argv)
^
cc1plus: all warnings being treated as errors
make[2]: *** [CMakeFiles/put_object_unicode.dir/put_object_unicode.cpp.o] Error 1
make[1]: *** [CMakeFiles/put_object_unicode.dir/all] Error 2

Error in Go sample?: certificate signed by unknown authority

While executing, I am getting below error

[rahul:~/Softwares/GO/src/s3_download_object]$ go run s3_download_object.go rahuljain-s3-bucket Not+Required.txt
Unable to download item "Not+Required.txt", RequestError: send request failed
caused by: Get https://rahuljain-s3-bucket.s3.ap-south-1.amazonaws.com/Not%2BRequired.txt: x509: certificate signed by unknown authority
exit status 1

how to know IAM user is admin or not in Java

Hi Team,
With the help of java aws sdk, I can create AmazonIdentityManagementClient IAM object. And also I can get all the list of users those are associated with this aws account. Now I want to know that what all users are admin. How can I do this with the help of java aws sdk?

S3 Transfer Manager bytesTransferred not being updated

I am using the S3 Transfer Manager within a Scala project. It seems like the bytesTransferred is not being updated. Please see below sample code:

val amazonTransferManager : TransferManager = ???
val copy = amazonTransferManager.copy(sourceBucket, sourceKey, destinationBucket, destinationKey)
while (!copy.isDone) {
Thread.sleep(100)
println(s"Transfer progress ${copy.getProgress.getBytesTransferred} / ${copy.getProgress.getTotalBytesToTransfer}")
}

The getTotalBytesToTransfer always returns the correct value but the getBytesTransferred always returns 0 even when the TransferState is Completed.

Is this expected behaviour or is there something I can do differently to ensure the getBytesTransferred method returns the expected result?

Thanks,

Steve

SNS examples are missing for Go

It looks like the most complete set of examples for SNS is in the javascript subdirectory, so those could be used as a reference.

java.lang.NoSuchFieldError: ALLOW_FINAL_FIELDS_AS_MUTATORS for put object java example

Here is my code

AWSCredentials credentials = new BasicAWSCredentials(passing my credentials here);
AmazonS3 s3Client = new AmazonS3Client(credentials);
s3Client.putObject(bucketname", file.getName(), file);

Warning: StandardWrapperValve[springapp]: Servlet.service() for servlet springapp threw exception
java.lang.NoSuchFieldError: ALLOW_FINAL_FIELDS_AS_MUTATORS
at com.amazonaws.partitions.PartitionsLoader.(PartitionsLoader.java:52)
at com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30)
at com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:66)
at com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:54)
at com.amazonaws.regions.RegionUtils.getRegion(RegionUtils.java:107)
at com.amazonaws.services.s3.AmazonS3Client.createSigner(AmazonS3Client.java:3288)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3984)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1570)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1428)

S3 local "the target server failed to respond"

Sorry, error of repo

Help Wanted

Do you have any ideas for new examples? For any particular service? In any particular language?

We would love to hear from you.

Suggestions? Questions? Whatever keeps you awake at night.

Valgrind report memory leaks on C++ SDK examples

Hello,

Valgrind is reporting a memory leak on the SQS C++ SDK send_message example (function Aws::Utils::Crypto::Sha256OpenSSLImpl::Calculate)

Please find the logs below:

$ valgrind --leak-check=full --track-origins=yes ./send_message "<queue_url>" "message"
==8878== Memcheck, a memory error detector
==8878== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==8878== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==8878== Command: ./send_message <queue_url> message
==8878== 
Successfully sent message to <queue_url>
==8878== 
==8878== HEAP SUMMARY:
==8878==     in use at exit: 73,296 bytes in 17 blocks
==8878==   total heap usage: 8,764 allocs, 8,747 frees, 1,080,092 bytes allocated
==8878== 
==8878== 144 bytes in 3 blocks are definitely lost in loss record 11 of 13
==8878==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8878==    by 0x6502DB2: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==8878==    by 0x658DEA6: EVP_MD_CTX_create (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==8878==    by 0x543BEE1: Aws::Utils::Crypto::Sha256OpenSSLImpl::Calculate(std::istream&) (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x542526E: Aws::Utils::Crypto::Sha256::Calculate(std::istream&) (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x54065AF: Aws::Client::AWSAuthV4Signer::ComputePayloadHash(Aws::Http::HttpRequest&) const (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x54046AC: Aws::Client::AWSAuthV4Signer::SignRequest(Aws::Http::HttpRequest&, bool) const (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x54128BB: Aws::Client::AWSClient::AttemptOneRequest(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod) const (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x541173D: Aws::Client::AWSClient::AttemptExhaustively(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod) const (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x541539E: Aws::Client::AWSXMLClient::MakeRequest(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod) const (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x4F46589: Aws::SQS::SQSClient::SendMessage(Aws::SQS::Model::SendMessageRequest const&) const (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-sqs/libaws-cpp-sdk-sqs.so)
==8878==    by 0x40EE9B: main (in /home/aviggiano/aws-test/aws-doc-sdk-examples/cpp/build/send_message)
==8878== 
==8878== 144 bytes in 3 blocks are definitely lost in loss record 12 of 13
==8878==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8878==    by 0x6502DB2: CRYPTO_malloc (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==8878==    by 0x658DEA6: EVP_MD_CTX_create (in /lib/x86_64-linux-gnu/libcrypto.so.1.0.0)
==8878==    by 0x543BDE5: Aws::Utils::Crypto::Sha256OpenSSLImpl::Calculate(std::basic_string<char, std::char_traits<char>, Aws::Allocator<char> > const&) (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x542522C: Aws::Utils::Crypto::Sha256::Calculate(std::basic_string<char, std::char_traits<char>, Aws::Allocator<char> > const&) (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x5404E17: Aws::Client::AWSAuthV4Signer::SignRequest(Aws::Http::HttpRequest&, bool) const (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x54128BB: Aws::Client::AWSClient::AttemptOneRequest(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod) const (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x541173D: Aws::Client::AWSClient::AttemptExhaustively(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod) const (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x541539E: Aws::Client::AWSXMLClient::MakeRequest(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod) const (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-core/libaws-cpp-sdk-core.so)
==8878==    by 0x4F46589: Aws::SQS::SQSClient::SendMessage(Aws::SQS::Model::SendMessageRequest const&) const (in /home/aviggiano/aws/deps/aws-sdk-cpp/build/aws-cpp-sdk-sqs/libaws-cpp-sdk-sqs.so)
==8878==    by 0x40EE9B: main (in /home/aviggiano/aws-test/aws-doc-sdk-examples/cpp/build/send_message)
==8878== 
==8878== LEAK SUMMARY:
==8878==    definitely lost: 288 bytes in 6 blocks
==8878==    indirectly lost: 0 bytes in 0 blocks
==8878==      possibly lost: 0 bytes in 0 blocks
==8878==    still reachable: 73,008 bytes in 11 blocks
==8878==         suppressed: 0 bytes in 0 blocks
==8878== Reachable blocks (those to which a pointer was found) are not shown.
==8878== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==8878== 
==8878== For counts of detected and suppressed errors, rerun with: -v
==8878== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

Confusing Java examples for S3

Hi there -- it's a bit confusing that the Java examples for S3 (e.g. java/example_code/s3) provide a working pom.xml and require Maven but then don't use Maven to actually run the examples via run_example.sh. Thus, the dependencies get pulled down, but aren't used -- and the docs suggest downloading the SDK locally somewhere (I'm not sure why Maven is required at all if we're going to manually expect folks to do that).

Perhaps the run_example.sh could leverage Maven to run, so the dependency already pulled in could be used?

diff --git a/java/example_code/s3/run_example.sh b/java/example_code/s3/run_example.sh
index 1f8e6cd..cf8bf0d 100755
--- a/java/example_code/s3/run_example.sh
+++ b/java/example_code/s3/run_example.sh
@@ -5,5 +5,4 @@ if [[ -z $* ]] ; then
 fi
 export CLASSPATH=target/s3examples-1.0.jar:$JAVA_SDK_HOME/lib/*:$JAVA_SDK_HOME/third-party/lib/*
 echo "## Running $1..."
-java aws.example.s3.$@
-
+mvn exec:java -Dexec.mainClass="aws.example.s3.$@"

Ruby, how to send/receive xml documents

Hi,
Thanks for this example in ruby that send/receive a text message, Can you please show us how to send and receive an xml document ?
Thanks, your help is appreciated.

Why Base64?

aws-doc-sdk-examples/python/example_code/kms/kms-python-example-encrypt-data.py

Line 23 in c5c9ace

text_64 = base64.b64encode(text)

Why is this base64 operation being done? I don't see this pattern in any of the other examples, and it seems to me that it just complicates the example with unnecessary code.

Example request: fetch a Cloudwatch log stream in Go

My use case is that I need to be able to give my colleagues a quick way of checking the logs of a job

Giving them a URL like: https://us-west-2.console.aws.amazon.com/cloudwatch/home?region=us-west-2#logEventViewer:group=/aws/batch/job;stream=example/default/a69d2235-549e-42db-9c03-6d68dbd3ae23

Is just too painful for them, since they have to login et al, and sometimes on new accounts we don't have users for them.

Be good if there was an example that basically fetched the last 100 lines or something at

https://github.com/awsdocs/aws-doc-sdk-examples/tree/master/go/example_code/cloudwatch

put_object.cpp sample returning error ( timeout )

I am experiencing the problem as described like this : RequestTimeout Message: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed

I have been trying to test S3 with AWS SDK wiht c++ .. but this putobject sample is the one that I can not get working

So this one is the problematic one now:
aws-doc-sdk-examples/cpp/example_code/s3/put_object.cpp

I tried putting file object into the bucket created by me to S3 storage

Sample apps for "creating bucket" , "listing of bucket content" and e.g "deleting buckets" are working well

Credentials for S3 are set properly I have compiled AWS sdk, latest from master branch with Windows 10 and VIsual Studio 2013

list all versions objects from s3 in specific folder location

I'm able to get all objects from specific location using this code :

package main

import (
	"fmt"
	"os"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3"
)

// Lists the items in the specified S3 Bucket
//
// Usage:
//    go run s3_list_objects.go BUCKET_NAME FOLDER_LOCATION
func main() {
	bucket := os.Args[1]
	prefix := os.Args[2]

	// Initialize a session in us-west-2 that the SDK will use to load
	// credentials from the shared credentials file ~/.aws/credentials.
	sess, err := session.NewSession(&aws.Config{
		Region: aws.String("ap-southeast-1")},
	)

	// Create S3 service client
	svc := s3.New(sess)

	// Get the list of items
	resp, err := svc.ListObjects(&s3.ListObjectsInput{
		Bucket: aws.String(bucket),
		Prefix: &prefix,
	})
	if err != nil {
		exitErrorf("Unable to list items in bucket %q, %v", bucket, err)
	}

	for _, item := range resp.Contents {
		fmt.Println("Name:         ", *item.Key)
		fmt.Println("Last modified:", *item.LastModified)
		fmt.Println("Size:         ", *item.Size)
		fmt.Println("Storage class:", *item.StorageClass)
		fmt.Println("")
	}

	fmt.Println("Found", len(resp.Contents), "items in bucket", bucket)
	fmt.Println("")
}

func exitErrorf(msg string, args ...interface{}) {
	fmt.Fprintf(os.Stderr, msg+"\n", args...)
	os.Exit(1)
}

But it seems that the Go AWS SDK doesn't provide to get all versions objects in specific location. I'm able to get it via CLI s3api.

Is there any way to do the same approach in Go?

Question: on Go example for deleting s3 objects

aws-doc-sdk-examples/go/example_code/s3/s3_delete_objects.go

Line 47 in 5d8954e

// Get the list of objects

Hi Instead of this looping example would it be possible to update the documentation with the below?, let me know if you think this is worth doing and I'll make a PR.

	iter := s3manager.NewDeleteListIterator(svc, &s3.ListObjectsInput{
		Bucket: aws.String(bucket),
		Prefix: aws.String(prefix),
	})

	if err := s3manager.NewBatchDeleteWithClient(svc).Delete(aws.BackgroundContext(), iter); err != nil {
		return err
	}

[aws-sdk-cpp sqs] Is there a purge queue example ?

Unused parameters in cpp example, won't build on Linux

This will fix it....

diff --git a/cpp/example_code/s3/list_buckets.cpp b/cpp/example_code/s3/list_buckets.cpp
index 7847e4e..e46248c 100644
--- a/cpp/example_code/s3/list_buckets.cpp
+++ b/cpp/example_code/s3/list_buckets.cpp
@@ -20,6 +20,9 @@
/
int main(int argc, char* argv)
{

• (void)argc;

• (void)argv;

• Aws::SDKOptions options;
  Aws::InitAPI(options);
  {
  diff --git a/cpp/example_code/s3/set_acl.cpp b/cpp/example_code/s3/set_acl.cpp
  index b6a8aa8..576cc10 100644
  --- a/cpp/example_code/s3/set_acl.cpp
  +++ b/cpp/example_code/s3/set_acl.cpp
  @@ -39,6 +39,8 @@ Aws::S3::Model::Permission GetPermission(Aws::String access)
  void SetAclForBucket(Aws::String bucket_name, Aws::String email,
  Aws::String access, Aws::String user_region)
  {

• (void)access;

• std::cout << "Setting ACL for bucket: " << bucket_name << std::endl;

  Aws::Client::ClientConfiguration config;
  @@ -69,6 +71,8 @@ void SetAclForBucket(Aws::String bucket_name, Aws::String email,
  void SetAclForObject(Aws::String bucket_name, Aws::String object_key,
  Aws::String email, Aws::String access, Aws::String user_region)
  {

• (void)access;

• std::cout << "Setting ACL for object: " << object_key << std::endl
  << " in bucket: " << bucket_name << std::endl;

AWS SNS: Example client JavaScript

Hi all @AWSRandall @huntsper @joshbean @mwunderl

Do we have any document/example code that show how we can use Client JavaScript with AWS SNS service? For example I want to push notification from Server Back-end to Browser client using AWS SNS, is there some guide to do?

Thank you for you help!

Best regards
Johnny

[javascript SQS receive/delete] VisibilityTimeout === 0 guarantees most delete attempts will fail?

Running the (slightly altered) sample with a fifo queue gives:

Delete Error { InvalidParameterValue: Value AQEBTHl<snip> for parameter ReceiptHandle is invalid. Reason: The receipt handle has expired.
    at Request.extractError ...

Setting the Visibility Timeout

Set to default or don't set?