krvax / aws-api-gateway-developer-portal

A Serverless Developer Portal for easily publishing and cataloging APIs

Introduction

The Amazon API Gateway Serverless Developer Portal is an application that you use for developer engagement by making your API Gateway APIs available to your customers through self-service discovery of those APIs. Your customers can use the developer portal to browse API documentation, register for – and immediately receive – their own API key that can be used to build applications, test published APIs, monitor their own API usage, generate SDKs, and submit feedback on your APIs design.

For more information about Amazon API Gateway, visit the API Gateway product page.

Setup

There are three main ways to deploy the Developer Portal today:

1. Deploy using SAR
2. Deploy using SAM
3. Deploy using the development scripts

1. Deploy using SAR

If all you want to do is deploy it as it is out of the box, you can do it by simply following the instructions in the Serverless Developer Portal documentation.

2. Deploy using SAM

If you plan to automate the deployment through your own infrastructure or if you just want to customize the template, you can just use the SAM CLI to do it. See the SAM Deployment Guide for how to do this.

3. Deploy using the development scripts

This deployment model is better if you choose to modify the developer portal assets and/or design itself or if you need to do something else more advanced. See the development guide for how to do this.

Prerequisites

First, ensure you have the latest version of the SAM CLI installed. Note that while the instructions specify Docker as a pre-requisite, Docker is only necessary for local development via SAM local. Feel free to skip installing Docker when you first set up the developer portal.

Then, clone this repo into a local directory. Ensure that you have an S3 bucket to put zipped lambda functions into. It can be private, and will be referred to in this readme as YOUR_LAMBDA_ARTIFACTS_BUCKET_NAME.

If you have not used the AWS CLI or SAM CLI before, you may need to configure your AWS credentials file.

If you have previously set up a v1 developer portal (non-SAM deployed), you will need to either remove all the v1 developer portal resources (dynamo tables, roles, etc.) or provide new names for the v2 developer portal by passing in parameter overrides for every resource.

Deploy

Run the following, with YOUR_LAMBDA_ARTIFACTS_BUCKET_NAME replaced with the name of a bucket that you manage and that already exists.

If you're on Windows, you'll need to remove the backslashes and line breaks here for these commands to work.

sam package --template-file ./cloudformation/template.yaml \
    --output-template-file ./cloudformation/packaged.yaml \
    --s3-bucket YOUR_LAMBDA_ARTIFACTS_BUCKET_NAME

Then run the following, with YOUR_LAMBDA_ARTIFACTS_BUCKET_NAME the same as above and CUSTOM_PREFIX in all cases replaced with some shared prefix that is globally unique across AWS, like your org name or username.

Note: do not use the sam deploy command suggested by the sam package command.

sam deploy --template-file ./cloudformation/packaged.yaml \
    --stack-name "dev-portal" \
    --s3-bucket YOUR_LAMBDA_ARTIFACTS_BUCKET_NAME \
    --capabilities CAPABILITY_NAMED_IAM \
    --parameter-overrides \
    DevPortalSiteS3BucketName="CUSTOM_PREFIX-dev-portal-static-assets" \
    ArtifactsS3BucketName="CUSTOM_PREFIX-dev-portal-artifacts" \
    CognitoDomainNameOrPrefix="CUSTOM_PREFIX"

The command will exit when the stack creation is successful. If you'd like to watch it create in real-time, you can log into the cloudformation console.

To get the URL for the newly created developer portal instance, find the websiteURL field in the cloudformation console's outputs or run this command:

aws cloudformation describe-stacks --query \
    "Stacks[?StackName=='dev-portal'][Outputs[?OutputKey=='WebsiteURL']][][].OutputValue"

You can override any of the parameters in the template using the --parameter-overrides key="value" format. This will be necessary if you intend to deploy several instances of the developer portal or customize some of the features. You can see a full list of overridable parameters in cloudformation/template.yaml under the Parameters section.

Registering Users

Users can self-register by clicking the 'Register' button in the developer portal. Cognito calls the CognitoPreSignupTriggerFn lambda to determine if the user is allowed to register themselves. By default, this function always accepts the user into the user pool, but you can customize the body of the function either in a local repository (followed by packaging and deploying) or in the lambda console. If you intend for the developer portal to be 'private' to some group of users (and not globally / freely accessible), you will need to write a lambda function that enforces your business logic for user registration. Documentation on this lambda function's use can be found here.

Promoting a User to an Admin

Admin users can manage what APIs are visible to normal users and whether or not SDK generation is enabled (per api) for normal users. To promote a user to an admin, go to the Cognito console in the account the developer portal is in, select User Pools, then select the correct User Pool for the dev portal. From there, choose Users and groups, click on the users' name, choose Add to group, and select the group named STACK-NAMEAdminsGroup. This user is now an admin; if they're currently logged in, they will have to log out and back in to receive admin credentials.

Populate the API catalog

By default the Developer Portal won't list any APIs. You will have to pick and choose which APIs to show. There are 2 types of APIs:

Subscribable APIs

For an API to be subscribable, they must be managed by Amazon API Gateway. The Developer Portal can let a user associate their API Key with these APIs (via the Subscribe button) so they can start calling and developing on these APIs.

To list a subscribable API:

1. In API Gateway's console or CLI, associate that API & stage to a usage plan.
2. Log into the developer portal using an admin account and go to the Admin Panel tab.
3. In the "Displayed" column, click "False".

Non-subscribable APIs

The Developer can also list APIs that are managed outside of Amazon API Gateway (e.g., APIs hosted on-premise). The Developer Portal won't be able to associate an API Key with the API automatically; however, customers can still test the APIs.

To list a non-subscribable API:

1. Log into the developer portal using an admin account and go to the Admin Panel tab.
2. In the "Generic APIs" table, click "Add API", select an API specification file (Swagger or OAS3 in .json, .yaml, or .yml), and upload it.

Testing your APIs

When logged into the developer portal with an account that has a provisioned api key, you should be able to test your APIs by selecting a resource/method in them and clicking "Try it out!". Note that this requires CORS to be set up on your API to allow the developer portal to call it. Note that the default PetStore has CORS enabled on all resources but /.

Before going to production

Setup a custom domain for your Developer Portal

You should request and verify an ACM managed certificate for your custom domain name. Then, redeploy the CFN stack with the domain name and ACM cert ARN as parameter overrides. Additionally, you can control if Route 53 nameservers are created using the UseRoute53Nameservers override. A value of true will result in the creation of a Route 53 hosted zone and record set; false will skip the creation of these resources.

Use the same CUSTOM_PREFIX and YOUR_LAMBDA_ARTIFACTS_BUCKET_NAME you used in the deploy before.

sam deploy --template-file ./cloudformation/packaged.yaml \
    --stack-name "dev-portal" \
    --s3-bucket YOUR_LAMBDA_ARTIFACTS_BUCKET_NAME \
    --capabilities CAPABILITY_NAMED_IAM \
    --parameter-overrides \
    DevPortalSiteS3BucketName="CUSTOM_PREFIX-dev-portal-static-assets" \
    ArtifactsS3BucketName="CUSTOM_PREFIX-dev-portal-artifacts" \
    CustomDomainName="my.acm.managed.domain.name.com" \
    CustomDomainNameAcmCertArn="arn:aws:acm:us-east-1:111111111111:certificate/12345678-1234-1234-1234-1234567890ab" \
    UseRoute53Nameservers="false"

This creates a cloudfront distribution in front of the S3 bucket serving the site, optionally sets up a Route53 hosted zone with records aliased to that distribution, and require HTTPS to communicate with the cloudfront distribution.

If you chose UseRoute53Nameservers=true, after the deployment finishes, go to the Route53 console, find the nameservers for the hosted zone created by the deployment, and add those as the nameservers for your domain name through your registrar. The specifics of this process will vary by registrar.

If you chose UseRoute53Nameservers=false, instead point your nameservers at the cloudfront distribution's URL.

Add custom content and brand the Developer Portal

See this page on customization

Updating to a new version

The Developer Portal follows the semantic versioning scheme (major.minor.patch). Changes to the minor or patch version are backwards compatible so you should feel safe to get the latest version. For changes to major versions, please see this page on updating.

To update a SAM deployment:

1. Get the latest version from GitHub (Clone/Pull/Download).
2. When deploying follow the same steps as previous and use the same values for the parameters. The only difference is passing in a new value for the stack parameter StaticAssetRebuildToken. You can use any string for this as long as it is different than previously used.

To update a SAR deployment

1. When deploying follow the same steps as previous and use the same values for the parameters. The only difference is passing in a new value for the stack parameter StaticAssetRebuildToken.

Components

For an overview of the components of the developer portal, please see this page.

Debugging

You can trace and troubleshoot the Lambda functions using CloudWatch Logs. See this blog post for more information.

Tear-down

Deleting the developer portal should be as easy as deleting the cloudformation stack. This will empty the ArtifactsS3Bucket and DevPortalSiteS3Bucket s3 buckets, including any custom files! Note that this will not delete any api keys provisioned by the developer portal. If you would like to delete api keys provisioned through the developer portal but not those provisioned through other means, make sure to download a backup of the Customers DDB table before deleting the cloudformation stack. This table lists the provisioned api keys that will need to be cleaned up afterwards.