aws-samples / aws-serverless-openai-chatbot-demo

Introduction

ChatGPT is a very popular artificial intelligence (AI) technology that enables natural language conversations between humans and machines. It is based on the open-source GPT-3 language model developed by OpenAI, the model has been used in a variety of applications, from customer service chatbots to virtual assistants. It has also been used to generate human-like text in a wide range of formats, including conversation, story-telling, news articles, and more. ChatGPT has received positive feedback from the public and the research community for its ability to understand natural language, generate high-quality, coherent text, and meaningful responses. As the technology continues to evolve, it is expected that ChatGPT will become an increasingly important tool for businesses and individuals alike.
In this sample , we will demonstrate using the GPT-3 text completion endpoint from OpenAI, to build a web application as your personal AI assistant on AWS using a serverless architecture. And the services used in this project are all eligible for free tier. The services to be used are :

• Amazon API Gateway
• Amazon Lambda
• Amazon S3
• Amazon DynamoDB

Architecture

This application is built on a totally serverless architecture:

• An Amazon S3 bucket is hosting the HTML, JS, CSS files of the front-end client.
• An Amazon API Gateway is deployed to route the requests from client devices to the backend services.
• The backend services are built on top of Amazon Lambda, which includes a function to authorize the request, a function to process user sign in, a function to handle chat requests from the client and revoke OpenAI SDK function to get the response text from the OpenAI server.
• An Amazon DynamoDB table also needs to be created to store the username and credential to give some basic authorization of this application.

A new architecture adds a WebSocket API and SNS topic subscription to decouple the remote call to OpenAI from API gateway, thus to extend the 30s timeout limitation, and reduce 503 error. please refer to v2-websocket

Prerequisites

• You need to have an OpenAI account, and create an API key in the portal.
• An AWS account. If your account is still eligible for free tier, then your application might cost zero under the free tier quota.

Setup local environment

• Install Node.js (if you've already installed it, please skip this) Install Node.js in your local environment, to build static website files and some dependencies packages for Amazon Lambda.
• Get source code and build the packages The server folder contains the code for the Lambda functions. The client folder contains the code for the front-end website.
• Go to each Lambda function folder under the server folder, install the dependencies and make .zip file archives for uploading to Amazon Lambda. For example, to make a lambda_login.zip:
  >cd server/lambda_login
>npm install <br/>
>zip -r lambda_login.zip
  We will create 3 Amazon Lambda functions, that means we will make 3 .zip files (lambda_auth.zip/ lambda_login.zip/ lambda_chat.zip)in total.

Create Lambda functions

Create a Lambda function to handle chat.

1. In AWS console, create a Lambda function from scratch named openai-chat, choose Node.js 18.x for the runtime.

If you are using a Mac with M1/M2 chips to build the NodeJS dependencies locally, please remember to choose "arm64" for architecture option.

2. Upload the lambda_chat.zip created from last step to the Lambda.
3. Configure your own OpenAI API key in environment variables as key named "OPENAI_API_KEY".
   
4. OpenAI needs time to process the request, which is longer than 3 secs, so please change the runtime timeout to a greater value, e.g 1 min.
   

Create a Lambda authorizer for API request authorization.

1. In AWS console, create a Lambda function from scratch named chat-authorizer, choose Node.js 18.x for the runtime.
2. Configure an arbitrary value as your own token key. We use jsonwebtoken to authorize and validate the request, and this key will be used to sign the token. Store this token key in environment variables named "TOKEN_KEY".
3. Upload the lambda_auth.zip file to the console, similar to the openai-chat function.

Create a Lambda function to handle user login.

1. In the AWS console, create a Lambda function from scratch named openai-login, choose Node.js 18.x for the runtime.
2. Add the environment variables for TOKEN_KEY as same as the lambda authorizer function.
3. This function will invoke DynamoDB service API to verify the user credentials, so we need to attach AmazonDynamoDBReadOnlyAccess policy or create an inline policy to the role of this function.
4. Upload the lambda_login.zip file to the console, similar to the openai-chat function.

Create API gateway

1. Create the HTTP API gateway.
   
2. Create two Routes using POST method: /chat , /login.
3. For /chat route, we need to attach the Lambda authorizer and integrate it to the openchat Lambda function.
   -- Create a Lambda authorizer and attach it to /chat. In Lambda function field, select the "chat-authorizer" that you have created above. -- Create and attach an integration. In the Lambda function field, select the "openai-chat" that you have created above.
   4 For /login route, we need to integrate to the Lambda function created for login. In Lambda function field, select the "openai-login" that you have created above.
   5 Set the CORS configuration as below:
   

Create DynamoDB table

We use Amazon DynamoDB to store username and password credentials. To simplify the demo, we will not implement the signup function, you can directly add the user and unencrypted password into the table. Or you can use the AWS CLI command in the code package.

1. Sign in to the AWS Management Console and open the Amazon DynamoDB console, create a table name chat_user_info with the partition key username. .
2. Add your username and password pairs to the table.
   .

Host the website in S3

When you configure a bucket as a static website, you must enable static website hosting, configure an index document, and set the permissions. You can read the detail from AWS doc Reference.

1. Create an S3 bucket named bucket-name on the Amazon S3 console.
2. Enable static website hosting of this bucket. In Index document, enter the file name of the index document index.html.
3. By default, the S3 bucket blocks public access. You need to change the setting by unchecking the option in the "Permissions" tab of the bucket detail page.
4. Add the policy below to the bucket policy to allow public access.

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "PublicReadGetObject",
               "Effect": "Allow",
               "Principal": "*",
               "Action": [
                  "s3:GetObject"
               ],
               "Resource": [
                  "arn:aws:s3:::bucket-name/*"
               ]
           }
       ]
   }
   

5. Then your Amazon S3 website follows one of these two formats:

• http://bucket-name.s3-website-Region.amazonaws.com
• http://bucket-name.s3-website.Region.amazonaws.com

Build the static files for client

1. In your local environment, go to the client folder, change the first line of apigw.js to the actual API gateway endpoint which you created in the previous step.

const API_endpoint = 'https://xxx.amazonaws.com/';

2. Then run these commands to install and build the packages for the front-end website.
   >cd client
>npm install
>npm run build

3. After the "npm run build" completes, it will create a folder named "build" in the client folder. That folder has all the files to be deployed to the Amazon S3 website. You can upload this folder to the bucket through AWS S3 console or use AWS CLI as below:
   `>aws s3 sync ./build/ s3://bucket-name/

4. After all steps are done, now you can visit the S3 website endpoint in your PC/Mobile browser, and login the page with your username and password you stored in the DynamoDB table. .

5. You can change your model settings in the menu (This is a new feature updated on Feb-11-2023) .

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.

aws-serverless-openai-chatbot-demo's Issues

How to Set the CORS configuration

In the step 5 Set the CORS configuration as below:

I'm not following - I doubt it means to put a giant X in each of the three fields, so how does one set the CORS config? Thanks very much.

This is incredible, and need help with this Step

Thanks for this awesome repo. Tore through it up till here.
For Step 3 in the openai_chat function setup:

"...This function will invoke DynamoDB service API to verify the user credentials, so we need to attach AmazonDynamoDBReadOnlyAccess policy or create
an inline policy to the role of this function."

I found the property in the docs by searching and it looks like you set this up in the console like so? Thanks very much!

搭建完成了，但是无法登录

sign in issue "Unexpected end of input"

follow the guidance in v2-websocket readme file, but got this issue when login.

login完成后/chat返回500

login和auth似乎都是成功的，但是/chat会返回500，不知道是什么原因？本地nodejs是V16, lambda配置的v18会对/chat有影响吗？

chatGPT不是最新的版本？

搭建完成后发现，和chatgpt沟通，它的时间一致停留在2020年。这个需要哪里调整呢？

Dot missing

zip -r lambda_login.zip .
There is a dot missing in the instruction to zip the lambda functions.

Reopening Closed Issue.

Earlier issue was closed without fixing. API Gateway after deploying still getting this auth error after removing the backslash in Principal property in bucket policy. Thanks for sticking with me on this.

Access to fetch at 'https://3cncgd7ya0.execute-api.us-east-2.amazonaws.com/login' from origin 'http://bucket-name99.s3-website.us-east-2.amazonaws.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled. POST https://3cncgd7ya0.execute-api.us-east-2.amazonaws.com/login net::ERR_FAILED 404

503, {“message”：“Service Unavailable”}

有时候能正常回复有时候却出现503，是否是OpenAI 的接口的响应时间有关呢？已经调到3分钟了。

另外新增的Change model settings功能不清楚如何使用，会出现一堆重复的文字。

Error: 500,{"message":"Internal Server Error"}

每一步都按照要求配置后。报 500 error。

去看了 aws Lambda的 chat-authorizer 和 openai-login都是有log的，但是openai-chat没有log。

v2聊天窗口经常出现 remote socket connection error

如下图，聊天窗口经常出现 remote socket connection error

Ops! Error: 503,{"message":"Service Unavailable"}

API GW 30秒超时，问题长了，都来不及回答。有解吗？

How to use new openai model in this project?

Is there anyway to revise codes to add model "gpt-3.5-turbo" besides "text-davinci-003"/“code-davinci-002” ？or put ""gpt-3.5-turbo" as default option? Thanks

Support Global Region?

I tried to build it in CN region at first, but due to ICP restrction, I can't make my bucket public.

Then I tried to build in Global Region, there is 500 error when I tried to log in:

JSON in bucket policy malformed

Placing the cut and paste JSON into the S3 bucket policy results in a syntax error:

Support with streaming outputs from OpenAI APIs

In the lambda function, I could see that 'stream' parameter is not taken as an argument. That would need significant refactoring of the code as one needs to wait for the data to arrive. Has someone implemented with streaming?

Lark-ChatGptBot 部署成功后 会话不能连续提问，连续提问时机器人无法记住上一句

如题，机器人无法记住上一句的问题

请问如何解决？

copy the chatbot message

when i select the message of chatbot result, that will renew the select item and change to select all item. so how can i copy the result?

i have to use v2-websocket

V2版本不能选中文字用于复制

我在手机上、电脑上打开了V2版本网页，发现在聊天界面不能选中文字。
用鼠标选中文字后，文字焦点马上消失，导致不能选中，无法复制。望解决。。

Support GPT-4

As GPT-4 has been released, when could the team update the codes to support the new GPT-4 API?

Auth error

Almost there! Thanks for the help along the way. So I've set the CORS policies as indicated on my API Gateway but after deploying still getting this auth error.

Access to fetch at 'https://3cncgd7ya0.execute-api.us-east-2.amazonaws.com/login' from origin 'http://bucket-name99.s3-website.us-east-2.amazonaws.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
POST https://3cncgd7ya0.execute-api.us-east-2.amazonaws.com/login net::ERR_FAILED 404

v2-websocket deployment: Unrecognized resource types: [AWS::Lambda::Url]

When deploy v2-websocket in AWS China, it raises error deploying the websocket-api-gateway-chat stack.

OpenAIChatBotWsApiGwChatStack-test: creating CloudFormation changeset...

 ❌  websocket-api-gateway-chat (OpenAIChatBotWsApiGwChatStack-test) failed: Error [ValidationError]: Template format error: Unrecognized resource types: [AWS::Lambda::Url]
    at Request.extractError (/usr/local/lib/node_modules/aws-cdk/lib/index.js:254:46127)
    at Request.callListeners (/usr/local/lib/node_modules/aws-cdk/lib/index.js:254:88989)
    at Request.emit (/usr/local/lib/node_modules/aws-cdk/lib/index.js:254:88437)
    at Request.emit (/usr/local/lib/node_modules/aws-cdk/lib/index.js:254:194844)
    at Request.transition (/usr/local/lib/node_modules/aws-cdk/lib/index.js:254:188396)
    at AcceptorStateMachine.runTo (/usr/local/lib/node_modules/aws-cdk/lib/index.js:254:153268)
    at /usr/local/lib/node_modules/aws-cdk/lib/index.js:254:153598
    at Request.<anonymous> (/usr/local/lib/node_modules/aws-cdk/lib/index.js:254:188688)
    at Request.<anonymous> (/usr/local/lib/node_modules/aws-cdk/lib/index.js:254:194919)
    at Request.callListeners (/usr/local/lib/node_modules/aws-cdk/lib/index.js:254:89157) {
  code: 'ValidationError',
  time: 2023-03-08T03:04:25.241Z,
  requestId: '43205935-2295-4757-9d42-4a018ce5178f',
  statusCode: 400,
  retryable: false,
  retryDelay: 516.2804870621469
}

 ❌ Deployment failed: Error: Stack Deployments Failed: ValidationError: Template format error: Unrecognized resource types: [AWS::Lambda::Url]
    at deployStacks (/usr/local/lib/node_modules/aws-cdk/lib/index.js:362:129094)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async CdkToolkit.deploy (/usr/local/lib/node_modules/aws-cdk/lib/index.js:362:146822)
    at async exec4 (/usr/local/lib/node_modules/aws-cdk/lib/index.js:417:51795)

Stack Deployments Failed: ValidationError: Template format error: Unrecognized resource types: [AWS::Lambda::Url]

OS: macOS Ventura 13.2.1 (Intel)
cdk: 2.67.0 (build b6f7f39)
go: go1.19.6 darwin/amd64
node: v16.19.1 (nodejs 18.x not support in AWS China Lambda)

V2版本部署后登录提示401

你好，
按照V2.0版本部署上去后提示401，请问是哪里有问题吗？

lamada如下：

Ops! Error: 500

Ops! Error: 500,{"message":"Request failed with status code 429","name":"Error","stack":"Error: Request failed with status code 429\n at createError (/var/task/node_modules/axios/lib/core/createError.js:16:15)\n at settle (/var/task/node_modules/axios/lib/core/settle.js:17:12)\n at IncomingMessage.handleStreamEnd (/var/task/node_modules/axios/lib/adapters/http.js:322:11)\n at IncomingMessage.emit (node:events:525:35)\n at endReadableNT (node:internal/streams/readable:1359:12)\n at process.processTicksAndRejections (node:internal/process/task_queues:82:21)","config":{"transitional":{"silentJSONParsing":true,"forcedJSONParsing":true,"clarifyTimeoutError":false},"transformRequest":[null],"transformResponse":[null],"timeout":0,"xsrfCookieName":"XSRF-TOKEN","xsrfHeaderName":"X-XSRF-TOKEN","maxContentLength":-1,"maxBodyLength":-1,"headers":{"Accept":"application/json, text/plain, /","Content-Type":"application/json","User-Agent":"OpenAI/NodeJS/3.1.0","Authorization":"Bearer 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","Content-Length":135},"method":"post","data":"{"model":"text-davinci-003","prompt":"\nHi","temperature":0,"max_tokens":2000,"top_p":1,"frequency_penalty":0.5,"presence_penalty":0.1}","url":"https://api.openai.com/v1/completions"},"status":429}

code error?

Hello,
I followed your above sequence and did it all the way down, but found that I couldn't log in successfully.
I suspect that there may be an error in the nodejs code, can you confirm it?

v2-websocket always failed with status code 429

Any input returned same error messages as below:

Request failed with status code 429|Error: Request failed with status code 429
at createError (/var/task/node_modules/axios/lib/core/createError.js:16:15)
at settle (/var/task/node_modules/axios/lib/core/settle.js:17:12)
at IncomingMessage.handleStreamEnd (/var/task/node_modules/axios/lib/adapters/http.js:322:11)
at IncomingMessage.emit (node:events:525:35)
at endReadableNT (node:internal/streams/readable:1359:12)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)