aws-samples / lambda-refarch-imagerecognition

The Image Recognition and Processing Backend reference architecture demonstrates how to use AWS Step Functions to orchestrate a serverless processing workflow using AWS Lambda, Amazon S3, Amazon DynamoDB and Amazon Rekognition.

Serverless Reference Architecture: Image Recognition and Processing Backend

The Image Recognition and Processing Backend demonstrates how to use AWS Step Functions to orchestrate a serverless processing workflow using AWS Lambda, Amazon S3, Amazon DynamoDB and Amazon Rekognition. This workflow processes photos uploaded to Amazon S3 and extracts metadata from the image such as geolocation, size/format, time, etc. It then uses image recognition to tag objects in the photo. In parallel, it also produces a thumbnail of the photo.

This repository contains sample code for all the Lambda functions depicted in the diagram below as well as an AWS CloudFormation template for creating the functions and related resources. There is also a test web app that you can run to interact with the backend.

Walkthrough of the architecture

1. An image is uploaded to the PhotoRepo S3 bucket under the private/{userid}/uploads prefix
2. The S3 upload event triggers the S3Trigger Lambda function, which kicks off an execution of the ImageProcStateMachine in AWS Step Functions, passing in the S3 bucket and object key as input parameters.
3. The ImageProcStateMachine has the following sub-steps:

• Read the file from S3 and extract image metadata (format, EXIF data, size, etc.)
• Based on output from previous step, validate if the file uploaded is a supported file format (png or jpg). If not, throw NotSupportedImageType error and end execution.
• Store the extracted metadata in the ImageMetadata DynamoDB table
• In parallel, kick off two processes simultaneously:
  • Call Amazon Rekognition to detect objects in the image file. If detected, store the tags in the ImageMetadata DynamoDB table
  • Generate a thumbnail and store it under the private/{userid}/resized prefix in the PhotoRepo S3 bucket

How to deploy

Follow these instructions to deploy the application (both backend and frontend):

1. Use 1-click deployment button above. Amplify Console will fork this repository in your GitHub account, and deploy the backend and frontend application.
   • Note: If you forked and changed the repository first, you can use the Amplify console and select "Connect App" to connect to your forked repo.
2. For IAM Service Role, create one if you don't have one or select an existing role. (This is required because the Amplify Console needs permissions to deploy backend resources on your behalf. More info)
3. Within your new app in Amplify Console, wait for deployment to complete (this may take a while)
4. Once the deployment is complete, you can test out the application!

If you want to make changes to the code locally:

1. Clone the repo in your Github account that Amplify created
2. In the Amplify console, choose Backend environments, and toggle "Edit backend" on the environment with categories added
3. Under Edit backend, copy the amplify pull --appId <your app id> --envName <your env name> command displayed
   • If you don't see this command and instead see amplify init --appId, try refreshing the backend environment tab after waiting a few minutes (cloudformation could still be provisioning resources)
4. Within your forked repository locally, run the command you copied and follow the instructions

   - This command synchronizes what's deployed to your local Amplify environment
   - Do you want to use an AWS profile: Yes
   - default
   - Choose your default editor: Visutal Studio Code
   - Choose the type of app that you're building: javascript
   - What javascript framework are you using: react
   - Source Directory Path:  src/react-frontend/src
   - Distribution Directory Path: src/react-frontend/build
   - Build Command:  npm.cmd run-script build  
   - Start Command: npm.cmd run-script start
   - Do you plan on modifying this backend? (Yes)

If at anytime you want to change these options. Look into amplify/.config/project-config.json and make your changes there.

Using the test web app

You can use the test web app to upload images and explore the image recognition and processing workflow.

Sign up and log in

1. Go to the URL of the Amplify app that was deployed

2. In the login page, click on "Create account"

3. Register an account by following the sign up instructions

   

4. After confirming the account, sign in

Album list

1. create albums using the "Add a new album"
2. You may need to referresh

Photo gallery

1. Click into an album you created

2. Upload a photo

3. You can follow the Step Functions execution link to review the details of the workflow execution Below is the diagram of the state machine being executed every time a new image is uploaded (you can explore this in the Step Functions Console):

   

4. When the processing finishes, the photo and extracted information is added to the display

Cleaning Up the Application Resources

To remove all resources created by this example, do the following:

1. Go to AWS CloudFormation console, delete the 2 stacks with name "amplify-photoshare-"
2. Go to the AWS Amplify console and delete the app.

License

This reference architecture sample is licensed under Apache 2.0.

lambda-refarch-imagerecognition's Issues

Module not found: Error: Can't resolve '@aws-amplify/cache'

Hi there,
I've uploaded the cfn lambda runtime to node 14, but now getting this error:
2022-06-08T12:47:14.729Z [INFO]: Creating an optimized production build...
2022-06-08T12:47:35.071Z [INFO]: Failed to compile.
2022-06-08T12:47:35.073Z [INFO]: Module not found: Error: Can't resolve '@aws-amplify/cache' in '/codebuild/output/src644004588/src/lambda-refarch-imagerecognition/src/react-frontend/node_modules/@aws-amplify/api/lib-esm'
Any suggestions how can I fix this?

Build fails to complete successfully

Everytime I deploy this application to the Amplify console, the build fails.
It gives me the following errors:

2022-04-03T17:03:17.454Z [ERROR]: !!! Build failed
2022-04-03T17:03:17.455Z [ERROR]: !!! Non-Zero Exit Code detected

In the CloudFormation stack details, these are events captured:

Has anyone else experienced the same?
Let me know what else may be required to troubleshoot this issue.

Unable to build a dist folder

Hello,

I have successfully deployed this locally and want to say thanks for some great code!

I am trying npm build -dist to produce the code to deploy the web app to an S3 bucket as a static website rather than have this working locally.

I am getting no error at the command line, instead npm doesn't produce a prod build/populated dist folder.

Any ideas please?

Many thanks :)

App fails to deploy with Amplify 1-click

I followed the directions for the Amplify 1-click deployment. The Backend build failed.

The relevant parts of the Amplify Build console show:

2022-06-30T17:12:58.779Z [INFO]: CREATE_IN_PROGRESS AmplifyResourcesPolicy AWS::IAM::Policy      Thu Jun 30 2022 17:12:54 GMT+0000 (Coordinated Universal Time)
  CREATE_IN_PROGRESS LambdaFunction         AWS::Lambda::Function Thu Jun 30 2022 17:12:54 GMT+0000 (Coordinated Universal Time)
  CREATE_IN_PROGRESS AmplifyResourcesPolicy AWS::IAM::Policy      Thu Jun 30 2022 17:12:55 GMT+0000 (Coordinated Universal Time) Resource creation Initiated
  CREATE_IN_PROGRESS LambdaFunction         AWS::Lambda::Function Thu Jun 30 2022 17:12:56 GMT+0000 (Coordinated Universal Time) Resource creation Initiated
2022-06-30T17:13:03.775Z [INFO]: 
2022-06-30T17:13:03.776Z [INFO]: CREATE_FAILED LambdaFunction AWS::Lambda::Function Thu Jun 30 2022 17:13:00 GMT+0000 (Coordinated Universal Time) 
    Resource handler returned message: "User: arn:aws:sts::111111111111:assumed-role/amplifyconsole-backend-role/BuildSession is not authorized to perform: lambda:PutFunctionConcurrency on resource: 
    arn:aws:lambda:us-west-2:111111111111:function:S3Trigger984fb593-devb because no identity-based policy allows the lambda:PutFunctionConcurrency 
    action (Service: Lambda, Status Code: 403, Request ID: 3a417a78-3416-4ab9-a589-0e6945fe3947)" (RequestToken: 1a65c44a-93cb-86c6-b612-4a48af0dfe0f, HandlerErrorCode: AccessDenied)
2022-06-30T17:13:18.778Z [INFO]: 
2022-06-30T17:13:18.781Z [INFO]: CREATE_COMPLETE AmplifyResourcesPolicy                                                 AWS::IAM::Policy           Thu Jun 30 2022 17:13:13 GMT+0000 (Coordinated Universal Time)
                                 CREATE_FAILED   amplify-photoshare-devb-170520-functionS3Trigger984fb593-19MV1K7RNRV4I AWS::CloudFormation::Stack Thu Jun 30 2022 17:13:14 GMT+0000 (Coordinated Universal Time) The following resource(s) failed to create: [LambdaFunction].
2022-06-30T17:13:28.887Z [INFO]: 
2022-06-30T17:13:28.888Z [INFO]: CREATE_FAILED functionS3Trigger984fb593 AWS::CloudFormation::Stack Thu Jun 30 2022 17:13:28 GMT+0000 (Coordinated Universal Time) 
    Embedded stack arn:aws:cloudformation:us-west-2:111111111111:stack/amplify-photoshare-devb-170520-functionS3Trigger984fb593-19MV1K7RNRV4I/d0a04fc0-f897-11ec-abb8-02cf258d1d13 
   was not successfully created: The following resource(s) failed to create: [LambdaFunction].

node10 does not include ImageMagick

After updating lambda functions to node10, execution fails due to following error. Assuming that node8 includes dependencies and node10 does not.

photo-sharing-backend-ExtractImageMetadataFunction-WO8YHMQB5V98

2020-02-17T17:53:52.893Z	9e746276-c1f2-45ff-a0f1-2ef293f8fdd4	ERROR	Invoke Error	
{
    "errorType": "ImageIdentifyError",
    "errorMessage": {
        "errorType": "Error",
        "errorMessage": "Could not execute GraphicsMagick/ImageMagick: identify \"-ping\" \"-verbose\" \"-\" this most likely means the gm/convert binaries can't be found",
        "cause": {
            "errorType": "Error",
            "errorMessage": "Could not execute GraphicsMagick/ImageMagick: identify \"-ping\" \"-verbose\" \"-\" this most likely means the gm/convert binaries can't be found",
            "stack": [
                "Error: Could not execute GraphicsMagick/ImageMagick: identify \"-ping\" \"-verbose\" \"-\" this most likely means the gm/convert binaries can't be found",
                "    at ChildProcess.<anonymous> (/var/task/node_modules/gm/lib/command.js:232:12)",
                "    at ChildProcess.emit (events.js:198:13)",
                "    at ChildProcess.EventEmitter.emit (domain.js:448:20)",
                "    at Process.ChildProcess._handle.onexit (internal/child_process.js:246:12)",
                "    at onErrorNT (internal/child_process.js:415:16)",
                "    at process._tickCallback (internal/process/next_tick.js:63:19)"
            ]
        },
        "isOperational": true,
        "stack": [
            "Error: Could not execute GraphicsMagick/ImageMagick: identify \"-ping\" \"-verbose\" \"-\" this most likely means the gm/convert binaries can't be found",
            "    at ChildProcess.<anonymous> (/var/task/node_modules/gm/lib/command.js:232:12)",
            "    at ChildProcess.emit (events.js:198:13)",
            "    at ChildProcess.EventEmitter.emit (domain.js:448:20)",
            "    at Process.ChildProcess._handle.onexit (internal/child_process.js:246:12)",
            "    at onErrorNT (internal/child_process.js:415:16)",
            "    at process._tickCallback (internal/process/next_tick.js:63:19)"
        ]
    },
    "name": "ImageIdentifyError",
    "message": {
        "errorType": "Error",
        "errorMessage": "Could not execute GraphicsMagick/ImageMagick: identify \"-ping\" \"-verbose\" \"-\" this most likely means the gm/convert binaries can't be found",
        "cause": {
            "errorType": "Error",
            "errorMessage": "Could not execute GraphicsMagick/ImageMagick: identify \"-ping\" \"-verbose\" \"-\" this most likely means the gm/convert binaries can't be found",
            "stack": [
                "Error: Could not execute GraphicsMagick/ImageMagick: identify \"-ping\" \"-verbose\" \"-\" this most likely means the gm/convert binaries can't be found",
                "    at ChildProcess.<anonymous> (/var/task/node_modules/gm/lib/command.js:232:12)",
                "    at ChildProcess.emit (events.js:198:13)",
                "    at ChildProcess.EventEmitter.emit (domain.js:448:20)",
                "    at Process.ChildProcess._handle.onexit (internal/child_process.js:246:12)",
                "    at onErrorNT (internal/child_process.js:415:16)",
                "    at process._tickCallback (internal/process/next_tick.js:63:19)"
            ]
        },
        "isOperational": true,
        "stack": [
            "Error: Could not execute GraphicsMagick/ImageMagick: identify \"-ping\" \"-verbose\" \"-\" this most likely means the gm/convert binaries can't be found",
            "    at ChildProcess.<anonymous> (/var/task/node_modules/gm/lib/command.js:232:12)",
            "    at ChildProcess.emit (events.js:198:13)",
            "    at ChildProcess.EventEmitter.emit (domain.js:448:20)",
            "    at Process.ChildProcess._handle.onexit (internal/child_process.js:246:12)",
            "    at onErrorNT (internal/child_process.js:415:16)",
            "    at process._tickCallback (internal/process/next_tick.js:63:19)"
        ]
    },
    "stack": [
        "Error",
        "    at Object.<anonymous> (/var/task/index.js:40:32)",
        "    at Module._compile (internal/modules/cjs/loader.js:778:30)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)",
        "    at Module.load (internal/modules/cjs/loader.js:653:32)",
        "    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:585:3)",
        "    at Module.require (internal/modules/cjs/loader.js:692:17)",
        "    at require (internal/modules/cjs/helpers.js:25:18)",
        "    at _tryRequire (/var/runtime/UserFunction.js:75:12)",
        "    at _loadUserApp (/var/runtime/UserFunction.js:95:12)",
        "    at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)",
        "    at Object.<anonymous> (/var/runtime/index.js:45:30)",
        "    at Module._compile (internal/modules/cjs/loader.js:778:30)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)",
        "    at Module.load (internal/modules/cjs/loader.js:653:32)",
        "    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)"
    ]
}

AlbumID - expression not defined

Hi - running into this error when triggering via a direct upload to s3 bucket, which prevents the metadata from being written to dynamoDB. Any suggestions?

"errorMessage": "Invalid UpdateExpression: An expression attribute value used in expression is not defined; attribute value: :albumID",

Backend Build Failure

npm start fails

When I enter npm start, it fails with the following errors:

app/config.ts(2,13): error TS1005: ';' expected.
app/config.ts(3,67): error TS1005: ';' expected.
app/config.ts(9,1): error TS1128: Declaration or statement expected.

npm ERR! Darwin 16.1.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "start"
npm ERR! node v7.7.1
npm ERR! npm v4.1.2
npm ERR! code ELIFECYCLE
npm ERR! [email protected] start: tsc && concurrently "tsc -w" "lite-server"
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the [email protected] start script 'tsc && concurrently "tsc -w" "lite-server" '.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the serverless-image-proc-backend package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! tsc && concurrently "tsc -w" "lite-server"
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs serverless-image-proc-backend
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls serverless-image-proc-backend
npm ERR! There is likely additional logging output above.

Frontend build fails with: Can't resolve 'maplibre-gl'

As of this morning, the install fails on the Frontend build with this error "Module not found: Error: Can't resolve 'maplibre-gl' in '/codebuild/output/src687896026/src/lambda-refarch-imagerecognition/src/react-frontend/node_modules/@aws-amplify/ui-react/dist/esm/components/Geo/MapView'"

Does anyone know how to resolve this?

                             # Executing command: npm run build

2022-04-07T13:12:55.950Z [INFO]: > [email protected] build /codebuild/output/src687896026/src/lambda-refarch-imagerecognition/src/react-frontend
> react-scripts build
2022-04-07T13:12:57.173Z [INFO]: Creating an optimized production build...
2022-04-07T13:13:05.259Z [INFO]: Failed to compile.
2022-04-07T13:13:05.262Z [INFO]: Module not found: Error: Can't resolve 'maplibre-gl' in '/codebuild/output/src687896026/src/lambda-refarch-imagerecognition/src/react-frontend/node_modules/@aws-amplify/ui-react/dist/esm/components/Geo/MapView'

Support required

Hello,
I am fairly new to most of the coding and front end related stuff. I deployed your solution as a cloud formation template and it worked seamlessly.
I want to re-purpose this for my application and hence want to have a conversation on the understanding. Can you help me with that?

Unable to produce prod build

Hello,

I have successfully deployed this locally and want to say thanks for some great code!

I am trying npm build -dist to produce the code to deploy the web app to an S3 bucket as a static website rather than have this working locally.

I am getting no error at the command line, instead npm doesn't produce a prod build/populated dist folder.

Any ideas please?

Many thanks :)

Front-end build failure

Hi,

The Front-end build fails to locate "@aws-amplify/cache" and ends with an error. Would you please be able to look into this when you have the bandwidth?

I have attached a log snippet for more information.
log.txt

Thank you,
Rushi

deployment not working

in my log... maybe something is missing in the role...

T02:35:37.838Z [WARNING]: Traceback (most recent call last):
File "/root/.local/lib/python3.7/site-packages/samcli/lib/samlib/wrapper.py", line 68, in run_plugins
parser.parse(template_copy, all_plugins) # parse() will run all configured plugins
File "/root/.local/lib/python3.7/site-packages/samcli/lib/samlib/wrapper.py", line 102, in parse
raise InvalidDocumentException(document_errors)
samtranslator.model.exceptions.InvalidDocumentException: [InvalidResourceException('ImageMagick', 'User: arn:aws:sts::123456789abc:assumed-role/AmplifyBackend/BuildSession is not authorized to perform: serverlessrepo:CreateCloudFormationTemplate on resource: arn:aws:serverlessrepo:us-east-1:cba987654321:applications/image-magick-lambda-layer')]
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/root/.local/bin/sam", line 8, in
sys.exit(cli())

Error with CloudFormation

Hi,
during the creation with the cloudformation I see this error:

The runtime parameter of nodejs4.3 is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (nodejs8.10) while creating or updating functions. (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: d5fedcf4-215f-11e9-984c-1b75dc1aca1b)

do you have any idea how to solve it?

thank you

Role policy need to be updated in Step3

Following the documentation, if you have created a new IAM role for Step Functions, this only includes permission to invoke the ExtractMetadata Lambda function.

When adding two new Lambda function invocations in the parallel path in Step3, the execution would fail with an 403 Access denied error.

{
  "error": "Lambda.AWSLambdaException",
  "cause": "User: arn:aws:sts::012345678910:assumed-role/StepFunctionExecutionRoleForImageProcessing/ahHmwmWNKuHXeWtEOlzRZuOFiwpoCyKH is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:eu-west-1:012345678910:function:sfn-workshop-setup-DetectLabel (Service: AWSLambda; Status Code: 403; Error Code: AccessDeniedException; Request ID: 7b05b2e7-a480-48e1-aa04-a77181f3e179)"
}

To fix this error, add the arn of the newly added functions to the Resource in the IAM Role Permissions.

Unable to create new album

Whenever I try to create a new album it throws an error.

cloudformation templates still ref node8

PR #21 changed the runtime of lambda functions to node10 but did not update cloudformation template's s3 source files.

Execution of CFT per README.md at s3://media-sharing-refarch/cloudformation/image-processing-v3-node8.output.yaml builds node8 (actually fails on lambda function creation)