aws-samples / amazon-dynamodb-labs

Hands on workshop and design scenarios for Amazon DynamoDB

Amazon DynamoDB Labs / Amazon DynamoDB Immersion Day

The repo for https://catalog.workshops.aws/dynamodb-labs/en-US , formerly https://amazon-dynamodb-labs.com

Dev:

Local development

You can make code changes and markdown changes, but in order to test the build you need to be an Amazon employee with access to preview_build to compile the documentation and run the site locally. Amazon employees click here for instructions.

Clone this repo:

We suggest you make a fork. From wherever you are you can checkout the repo:

git clone [email protected]:aws-samples/amazon-dynamodb-labs.git (or your fork)

Making edits:

Amazon employees only: Make changes, run preview_build, check localhost:8080 to see the site locally

Everyone else: make changes, make a pull request, and wait for the automations to run. They will tell you if you have errors in your changes.

Pull requests

Make a pull request with changes. PRs will be automatically checked to make sure their markdown and other files are correct and without error using an automatic GitHub action. With each commit in a PR, the action will run to verify.

On merge to master

On merge to master, a GitHub action will deploy the assets to amazon-dynamodb-labs.com and verify the build to ensure the markdown and other files are correctly formatted. From there, a maintainer must manually pull the changes and push to https://catalog.workshops.aws/dynamodb-labs/en-US

License

This project is licensed under the Apache-2.0 License.

amazon-dynamodb-labs's Issues

Clarify language on compound and composite keys

Composite key - should refer to Partition + Sort key combo
Compound key - Concatenated key, especially the sort key.

Ex7 Step 4 should end with an exercise summary

https://amazon-dynamodb-labs.com/design-patterns/ex7adjlists/step4.html

We shouldn't just move on to the next exercise without a summary of what was learned.

Design scenarios language standarization

Between my edits and the PRs we've merged I think the design challenges need a quick edit. For example, the designs says the size or item size is such and such kilobytes, but the word item is used in the retail cart exercise for the products in the shopping cart.

Ex8 Step2 IAM policy should have a resource restriction for the DYnamoDB table

https://github.com/aws-samples/amazon-dynamodb-labs/blob/master/content/design-patterns/ex8streams/Step2.en.md

The Lambda function can put and delete items only in the logfile_replica table.

However, the IAM policy has Resource:"*". We should change this IAM policy to actually restrict access to only 1 table.

Sometimes the EC2 instance fails to configure yum and the cache is corrupted somehow

The symptom is that an instance won't display the right version of python when python --version is run. This is because yum is unable to install packages. Sample logs:

[2020-09-14T23:50:50.840Z] Updating yum packages.
Loaded plugins: priorities, update-motd, upgrade-helper
http://packages.us-west-2.amazonaws.com/2018.03/main/c31535f74c6e/x86_64/repodata/primary.sqlite.bz2?instance_id=i-06dbf2976fb3b9fb3&region=us-east-1: [Errno -1] Metadata file does not match checksum
Trying other mirror.
...
14 Sep 23:51:00 ntpdate[2500]: adjust time server 47.144.196.17 offset -0.416039 sec
Starting ntpd: [  OK  ]
...
http://packages.ap-southeast-2.amazonaws.com/2018.03/updates/fd3054b03123/x86_64/repodata/primary.sqlite.bz2?instance_id=i-06dbf2976fb3b9fb3&region=us-east-1: [Errno -1] Metadata file does not match checksum
Trying other mirror.


 One of the configured repositories failed (amzn-updates-Base),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Disable the repository, so yum won't use it by default. Yum will then
        just ignore the repository until you permanently enable it again or use
        --enablerepo for temporary usage:

            yum-config-manager --disable amzn-updates

     4. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=amzn-updates.skip_if_unavailable=true

failure: repodata/primary.sqlite.bz2 from amzn-updates: [Errno 256] No more mirrors to try.
http://packages.us-west-1.amazonaws.com/2018.03/updates/fd3054b03123/x86_64/repodata/primary.sqlite.bz2?instance_id=i-06dbf2976fb3b9fb3&region=us-east-1: [Errno -1] Metadata file does not match checksum
...
http://packages.ap-southeast-2.amazonaws.com/2018.03/updates/fd3054b03123/x86_64/repodata/primary.sqlite.bz2?instance_id=i-06dbf2976fb3b9fb3&region=us-east-1: [Errno -1] Metadata file does not match checksum
Warning: cannot update yum packages. Continue...
[2020-09-14T23:51:21.465Z] Completed updating yum packages.
[2020-09-14T23:51:21.468Z] Installing Python3.6
Loaded plugins: priorities, update-motd, upgrade-helper
Not using downloaded repomd.xml because it is older than what we have:
  Current   : Thu Sep  3 23:14:32 2020
  Downloaded: Thu Nov 14 22:09:23 2019
http://packages.us-east-1.amazonaws.com/2018.03/updates/fd3054b03123/x86_64/repodata/primary.sqlite.bz2?instance_id=i-06dbf2976fb3b9fb3&region=us-east-1: [Errno -1] Metadata file does not match checksum
Trying other mirror.
...
http://packages.ap-southeast-2.amazonaws.com/2018.03/updates/fd3054b03123/x86_64/repodata/primary.sqlite.bz2?instance_id=i-06dbf2976fb3b9fb3&region=us-east-1: [Errno 16] error setting timestamp on file /var/cache/yum/x86_64/latest/amzn-updates/primary.sqlite.bz2 from http://packages.ap-southeast-2.amazonaws.com/2018.03/updates/fd3054b03123/x86_64/repodata/primary.sqlite.bz2?instance_id=i-06dbf2976fb3b9fb3&region=us-east-1, OSError: [Errno 2] No such file or directory: '/var/cache/yum/x86_64/latest/amzn-updates/primary.sqlite.bz2'
Trying other mirror.
...
http://packages.ap-northeast-1.amazonaws.com/2018.03/updates/fd3054b03123/x86_64/repodata/primary.sqlite.bz2?instance_id=i-06dbf2976fb3b9fb3&region=us-east-1: [Errno -1] Metadata file does not match checksum
Trying other mirror.


 One of the configured repositories failed (amzn-updates-Base),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Disable the repository, so yum won't use it by default. Yum will then
        just ignore the repository until you permanently enable it again or use
        --enablerepo for temporary usage:

            yum-config-manager --disable amzn-updates

     4. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=amzn-updates.skip_if_unavailable=true

failure: repodata/primary.sqlite.bz2 from amzn-updates: [Errno 256] No more mirrors to try.

Link to DDB docs instead

On reference-materials.html page, I recommend you either embed or just link to this page in the DDB docs.

Replace “DocumentDB” with DynamoDB

Replace “DocumentDB” with DynamoDB
on Prerequisites page: https://amazon-dynamodb-labs.com/hands-on-labs/setup/prerequisites.html

LEDA: Switch to resource based identity policies for scoreboard

At present, the LEDA scoreboard is updated by assuming a role in each team account. This is done in a loop, assuming credentials via WS who then gather those for us from STS.

We need to switch to RBAC, passing the central account ID to each team stack.

Consolidate labs (adv design & basic) under Cloud9 IDE + add provisioning into EE blueprint

1. In Adv design patterns labs scripts are reviewed via EC2 that is provisioned - it'll be better to leverage Cloud9 IDE from basic labs here - to provide better UX for end users when reviewing the python scripts
2. Add EE module provisioning for VPC/Cloud9 (currently you need to download and run CFn on your own and setup C9 env manually) - to provide more smooth end user experience. If we have python code in Public gitrepo - we could even auto-clone that into C9 during provisioning as well.
3. Internal labs inside AMZN needing to be updated

Should CLI and Python versions be updated?

on pages like this one it mentions AWS CLI 1.x, but we really should be using 2.x.

Also, should we switch Python version as well? For python, we should likely be on a newer version as well. This mentions 3.6.x when Python is already up to 3.9.

Nesting is lost in the nav menu for the Setup -> on your own and setup-> at an aws event pages

Issue: the two sub-items on this page should appear on the nav menu on the left. You should be able to see the section you're in. Without these two items appearing, students are confused about which set of instructions to follow.

Page with the two items: https://amazon-dynamodb-labs.com/design-patterns/setup.html

Code folder where the start-here section sits: https://github.com/aws-samples/amazon-dynamodb-labs/tree/master/content/design-patterns/setup

CloudFormation stack is not available - Access Denied

Stack is not a available under link provided in workshop. AWS return Access Denied.

LGME / LMR - Misc updates

LGME

• content/game-player-data/setup/step1.en.md - code zip only has scripts, but doc page says there are two folders

LMR

• chalice deploy warning:
  /usr/local/lib/python3.10/dist-packages/_distutils_hack/__init__.py:26: UserWarning: Setuptools is replacing distutils. warnings.warn("Setuptools is replacing distutils.") Creating deployment package.

Recommend Targeting a New Tab for Console Links

This is a minor annoyance. I would suggest all console links target a new tab. It's just annoying to click the link and have to go back and open in a new tab.

LHOL - Backups - unpredictable runtime

In LHOL we have a section where customers create a backup of a table and restore from it. It takes time, one estimate says 20 minutes, to do an end to end restore. The lab guide doesn't explain this process and how long it will take, and IIRC it might also be missing instructions / providing incorrect screenshots which add confusion to the process. We should go through LHOL's backup/PITR section and re-work the guide and screenshots.

Migration from ACL to Bucket policy

Starting in April 2023, Amazon S3 will introduce two new default bucket security settings by automatically enabling S3 Block Public Access and disabling S3 access control lists (ACLs) for all new S3 buckets. The template used by the Event Engine solution needs to be updated.

https://aws.amazon.com/about-aws/whats-new/2022/12/amazon-s3-automatically-enable-block-public-access-disable-access-control-lists-buckets-april-2023/

Update information on Transactions Lab (DynamoDB now supports 100 actions)

On the transactions lab guide we need to update the very first sentence as DynamoDB now supports 100 actions;

[Hands-on Labs > Explore DynamoDB with the CLI > Deleting Data] Wrong description of delete operation.

Probably because of copy-paste mistake the description for delete operation states that it creates or replace item:

The DynamoDB DeleteItem API is used to create a new item or to replace existing items completely with a new item. It is invoked using the delete-item CLI command.

Prevent stack deletion unless DynamoDB resources are cleaned up

Issue: Tables are left scaled up after stack deletion. We have banners in the lab explaining the importance of deleting DDB tables, but these aren't foolproof.

Recommended resolution:

• 1. Tables in the hands on python labs are tagged on creation
• 2. We should create a CFN custom resource or similar that blocks stack deletion if the DDB tables are not cleaned up
• 3. We should send an error message in the stack events if the tables still exist, and return a hyperlink to AWS resource manager to review the list of tables and clean them up

Change references from master to leader or root in examples

We should not use the term master in this lab to denote the primary item for an entity.

Change references to hash key on the scenario steps to partition key

When we show the tables with schema, we should prefer to call the primary keys partition and sort key instead of hash/sort or hash/range as it stands today.

NoSQL Workbench import issue

NoSQL Workbench (Windows) import fails with Model validation error: "Invalid version"
for the following files:

scenario-solutions/bank-payments/BankModel.json
scenario-solutions/retail-cart/CartModel.json