shelfio / jest-dynamodb Goto Github PK
View Code? Open in Web Editor NEWJest preset for DynamoDB local server
License: MIT License
Jest preset for DynamoDB local server
License: MIT License
I try to upgrade a project to Serverless 3 which uses this code from readme in jest-dynamodb-config.js
.
module.exports = async () => {
const serverless = new (require('serverless'))();
// If using monorepo where DynamoDB serverless.yml is in another directory
// const serverless = new (require('serverless'))({ servicePath: '../../../core/data' });
await serverless.init();
const service = await serverless.variables.populateService();
const resources = service.resources.filter(r => Object.keys(r).includes('Resources'))[0];
const tables = Object.keys(resources)
.map(name => resources[name])
.filter(r => r.Type === 'AWS::DynamoDB::Table')
.map(r => r.Properties);
return {
tables,
port: 8000
};
};
Works perfect for Serverless 2. But with Serverless 3 I get this error:
TypeError: Jest: Got error running globalSetup - /path/to/project/node_modules/@shelf/jest-dynamodb/setup.js, reason: undefined is not an array
at module.exports (/path/to/project/node_modules/type/lib/resolve-exception.js:12:14)
at module.exports (/path/to/project/node_modules/type/array/ensure.js:15:25)
at new Serverless (/path/to/project/node_modules/serverless/lib/serverless.js:84:22)
at module.exports (/path/to/project/jest-dynamodb-config.js:4:24)
at module.exports (/path/to/project/node_modules/@shelf/jest-dynamodb/setup.js:21:44)
at /path/to/project/node_modules/@jest/core/build/runGlobalHook.js:125:19
at ScriptTransformer.requireAndTranspileModule (/path/to/project/node_modules/@jest/transform/build/ScriptTransformer.js:903:24)
at async _default (/path/to/project/node_modules/@jest/core/build/runGlobalHook.js:116:9)
at async runJest (/path/to/project/node_modules/@jest/core/build/runJest.js:369:5)
at async _run10000 (/path/to/project/node_modules/@jest/core/build/cli/index.js:320:7)
at async runCLI (/path/to/project/node_modules/@jest/core/build/cli/index.js:173:3)
at async Object.run (/path/to/project/node_modules/jest-cli/build/cli/index.js:155:37)
Is there an updated version of that code that works with Serverless 3?
Thanks!
Using jest-dynamodb
after updating to jest 28 (specifically 28.1.0
at the time of writing, as well as updating ts-jest
to 28.0.3
) I get errors such as the following in all test suites:
● Test suite failed to run
TypeError: Class extends value #<Object> is not a constructor or null
at Object.<anonymous> (node_modules/@shelf/jest-dynamodb/environment.js:5:52)
at Object.newLoader (node_modules/pirates/lib/index.js:141:7)
at ScriptTransformer.requireAndTranspileModule (node_modules/@jest/transform/build/ScriptTransformer.js:895:66)
This appears to be because Jest 28 no longer exports NodeEnvironment
as a default import, but is now a named import
In addition, test environments are now exported with the name TestEnvironment, instead of simply exporting the class directly
https://jestjs.io/docs/upgrading-to-jest28#test-environment
It's an easy fix but I guess it would break anyone still on Jest 27? So it would be a major version bump?
Due to the sqlite4java did not support the Apple SIlicon platform, I have set up the JEnv to switch the Java SDK to X64 platform for my project. (I would like to keep the system one to use the native version)
Jest-dynamodb can run in the Unit test without issue with JEnv Shell (set the variable for the Shell) or Global, but it does not work with the JEnv Local. It seems the SDK cannot load with the jest-dynamodb.
I have tested the jenv local is working correctly.
Did anyone have the experience to up the Jest-DynamoDB with Jenv to work correctly?
Hi
I am using this package for testing dynamo operations in a serverless framework project.
My tests were timing out, after increasing the timeout value i am getting the following error:
UnknownEndpoint: Inaccessible host: 'localhost'. This service may not be available in the 'local-env' region.
My configuration can be found here
So it seems like the local instance of dynamo is not running along with my tests. If I use serverless-offline, and start the "offline server" - my tests pass.
Is there any specific configuration required for this work in a serverless framework project that i might have missed?
Any advise would be much appreciated
Maybe something is wrong in my monorepo setup...
I am getting an issue similar to #113 but even with his PR fix I'm still failing to create the tables
My structure:
jest-dynamodb-config.js
package-a/
jest.config.js
package-b/
jest.config.js
this is the inside of package-a and package-b jest.config.js
const shelfPreset = require('@shelf/jest-dynamodb/jest-preset');
const path = require('path');
// dynamodb-local
process.env.JEST_DYNAMODB_CONFIG = path.resolve(__dirname, '../../../jest-dynamodb-config.js');
module.exports = {
...shelfPreset,
testEnvironment: 'node',
};
this is the content of jest-dynamodb-config.js
located at the root:
module.exports = {
tables: [
{
TableName: 'table-a-tests',
},
{
TableName: 'table-b-tests',
},
],
port: 8000,
};
When running the tests, both package-a and package-b fail with:
ResourceInUseException: ResourceInUseException
at deserializeAws_json1_0CreateTableCommandError (/Users/xxx/dev/repos/serverless/node_modules/@aws-sdk/client-dynamodb/dist-cjs/protocols/Aws_json1_0.js:973:41)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async /Users/xxx/dev/repos/serverless/node_modules/@aws-sdk/client-dynamodb/node_modules/@aws-sdk/middleware-serde/dist-cjs/deserializerMiddleware.js:6:20
at async /Users/xxx/dev/repos/serverless/node_modules/@aws-sdk/client-dynamodb/node_modules/@aws-sdk/middleware-signing/dist-cjs/middleware.js:11:20
at async StandardRetryStrategy.retry (/Users/xxx/dev/repos/serverless/node_modules/@aws-sdk/client-dynamodb/node_modules/@aws-sdk/middleware-retry/dist-cjs/StandardRetryStrategy.js:51:46)
at async /Users/xxx/dev/repos/serverless/node_modules/@aws-sdk/client-dynamodb/node_modules/@aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:6:22
at async Promise.all (index 0)
at async module.exports (/Users/xxx/dev/repos/serverless/node_modules/@shelf/jest-dynamodb/setup.js:72:5)
at async /Users/xxx/dev/repos/serverless/node_modules/@jest/core/build/runGlobalHook.js:82:11
at async waitForPromiseWithCleanup (/Users/xxx/dev/repos/serverless/node_modules/@jest/transform/build/ScriptTransformer.js:207:5) {
'$fault': 'client',
'$metadata': {
httpStatusCode: 400,
extendedRequestId: undefined,
cfId: undefined,
attempts: 1,
totalRetryDelay: 0
}
Dynamodb-local seems to be not compatible with Apple Silicon
Dec 24, 2021 9:26:36 PM com.almworks.sqlite4java.Internal log WARNING: [sqlite] cannot open DB[1]: com.almworks.sqlite4java.SQLiteException: [-91] cannot load library: java.lang.UnsatisfiedLinkError: /private/var/folders/q2/smpmq5gs1vlclkfn3s971b800000gp/T/dynamodb-local/DynamoDBLocal_lib/libsqlite4java-osx.dylib: dlopen(/private/var/folders/q2/smpmq5gs1vlclkfn3s971b800000gp/T/dynamodb-local/DynamoDBLocal_lib/libsqlite4java-osx.dylib, 0x0001): tried: '/private/var/folders/q2/smpmq5gs1vlclkfn3s971b800000gp/T/dynamodb-local/DynamoDBLocal_lib/libsqlite4java-osx.dylib' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e')), '/usr/lib/libsqlite4java-osx.dylib' (no such file) Dec 24, 2021 9:26:36 PM com.almworks.sqlite4java.Internal log SEVERE: [sqlite] SQLiteQueue[]: error running job queue com.almworks.sqlite4java.SQLiteException: [-91] cannot load library: java.lang.UnsatisfiedLinkError: /private/var/folders/q2/smpmq5gs1vlclkfn3s971b800000gp/T/dynamodb-local/DynamoDBLocal_lib/libsqlite4java-osx.dylib: dlopen(/private/var/folders/q2/smpmq5gs1vlclkfn3s971b800000gp/T/dynamodb-local/DynamoDBLocal_lib/libsqlite4java-osx.dylib, 0x0001): tried: '/private/var/folders/q2/smpmq5gs1vlclkfn3s971b800000gp/T/dynamodb-local/DynamoDBLocal_lib/libsqlite4java-osx.dylib' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e')), '/usr/lib/libsqlite4java-osx.dylib' (no such file) at com.almworks.sqlite4java.SQLite.loadLibrary(SQLite.java:97) at com.almworks.sqlite4java.SQLiteConnection.open0(SQLiteConnection.java:1441) at com.almworks.sqlite4java.SQLiteConnection.open(SQLiteConnection.java:282) at com.almworks.sqlite4java.SQLiteConnection.open(SQLiteConnection.java:293) at com.almworks.sqlite4java.SQLiteQueue.openConnection(SQLiteQueue.java:464) at com.almworks.sqlite4java.SQLiteQueue.queueFunction(SQLiteQueue.java:641) at com.almworks.sqlite4java.SQLiteQueue.runQueue(SQLiteQueue.java:623) at com.almworks.sqlite4java.SQLiteQueue.access$000(SQLiteQueue.java:77) at com.almworks.sqlite4java.SQLiteQueue$1.run(SQLiteQueue.java:205) at java.base/java.lang.Thread.run(Thread.java:833) ...
According to the log, sqlite4java cannot find the corresponding support platform for Apple Silicon - (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64e')
As the last update of the sqlite4db was before Apple M1 release, Jest-dynamodb is based on dynamodb-local which is based on sqlite4db.
Is there any suggestion on how to solve this issue? is it possible to wrap the docker version for the Jest-dynamodb?
Thanks
Hi,
this repository is great. As AWS released v3 of their SDK I was wondering if there are plans to support v3 as well?
Cheers,
Michael
➜ aws-lambda-graphql git:(dev) ✗ node '/home/myfreax/work/aws-lambda-graphql/node_modules/.bin/jest' '/home/myfreax/work/aws-lambda-graphql/src/users/users.service.spec.ts' -t 'UsersService should be defined' --detectOpenHandles
PASS src/users/users.service.spec.ts (6.155 s)
UsersService
✓ should be defined (1 ms)
○ skipped should be create user
○ skipped should be update user
○ skipped should be only update user phone
○ skipped should be only update user password
○ skipped should be find one user
○ skipped should be find all user
console.info
http://localhost:8000
at initialization (../node_modules/nestjs-dynamoose/dist/dynamoose-core.module.js:34:21)
Test Suites: 1 passed, 1 total
Tests: 6 skipped, 1 passed, 7 total
Snapshots: 0 total
Time: 6.191 s, estimated 7 s
Ran all test suites matching /\/home\/myfreax\/work\/aws-lambda-graphql\/src\/users\/users.service.spec.ts/i with tests matching "UsersService should be defined".
ReferenceError: You are trying to `import` a file after the Jest environment has been torn down.
at Object.userAgent (../node_modules/aws-sdk/lib/util.js:34:43)
at HttpRequest.setUserAgent (../node_modules/aws-sdk/lib/http.js:111:78)
at new HttpRequest (../node_modules/aws-sdk/lib/http.js:104:10)
at new Request (../node_modules/aws-sdk/lib/request.js:328:24)
at features.constructor.makeRequest (../node_modules/aws-sdk/lib/service.js:202:19)
test.ts
import { Test, TestingModule } from '@nestjs/testing';
import { DynamooseModule } from 'nestjs-dynamoose';
import { CreateUserInput } from './dto/create-user.input';
import { UpdateUserInput } from './dto/update-user.input';
import { User } from './entities/user.entity';
import { userSchema } from './models/schema';
import { UsersService } from './users.service';
describe('UsersService', () => {
let service: UsersService;
let user: User;
beforeAll(async () => {
const module: TestingModule = await Test.createTestingModule({
providers: [UsersService],
imports: [
DynamooseModule.forRoot({
local: 'http://localhost:8000',
aws: { region: 'ap-east-1' },
model: {
update: true,
create: true,
prefix: `test-`,
suffix: '-table',
},
}),
DynamooseModule.forFeature([{ name: 'user', schema: userSchema }]),
],
}).compile();
service = module.get<UsersService>(UsersService);
});
// afterEach(async () => {
// if (user && user.id) {
// await service.remove({ id: user.id });
// }
// });
it('should be defined', () => {
expect(service).toBeDefined();
});
})
Have not any error throw when preset start Dynamodb with docker.
docker run -itd -p 8000:8000 --name dynamodb amazon/dynamodb-local:latest
Sorry this bug report is incomplete, I'm just filing what I have now until I can look into it later.
After updating from 3.2.0 to 3.3.0 I get the following errors:
$ jest --runInBand
● Validation Warning:
Unknown option "default" with value {"globalSetup": "/home/birtles/project/node_modules/@shelf/jest-dynamodb/lib/setup.js", "globalTeardown": "/home/birtles/project/node_modules/@shelf/jest-dynamodb/lib/teardown.js", "testEnvironment": "/home/birtles/project/node_modules/@shelf/jest-dynamodb/lib/environment.js"} was found.
This is probably a typing mistake. Fixing it will remove this message.
Configuration Documentation:
https://jestjs.io/docs/configuration
● Validation Warning:
Unknown option "default" with value {"globalSetup": "/home/birtles/project/node_modules/@shelf/jest-dynamodb/lib/setup.js", "globalTeardown": "/home/birtles/project/node_modules/@shelf/jest-dynamodb/lib/teardown.js", "testEnvironment": "/home/birtles/project/node_modules/@shelf/jest-dynamodb/lib/environment.js"} was found.
This is probably a typing mistake. Fixing it will remove this message.
Configuration Documentation:
https://jestjs.io/docs/configuration
FAIL src/changes/changes-stack.change-handler.messages.test.ts
● change handler › test
AWS SDK error wrapper for Error: connect ECONNREFUSED 127.0.0.1:8090
I'd like to test my DB queries in the case that it has to paginate a result. I was hoping there was a way for me to cause the DB to page at a lower size than 1Mb so that I don't have to generate a bunch of data just to fill it up to cause pagination.
Is there a way to either set the pagination size manually to a few kb or force a pagination result?
Thanks!
We use Jenkins to test, build and deploy our code for multiple clients.
The jobs run in parallel, so when we write tests which use this library, we encounter an issue where only one job will pass the tests, with the rest failing with "Cannot create preexisting table", which makes sense, as they all use the same codebase, which all reference the same local address for jest-dynamodb, https://localhost:8000.
Is there anyway around this?
Would using random ports work, or does it have to be 8000?
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
Error type: Cannot find preset's package (github>shelfio/renovate-config)
// Self-answering question to help others with the same issue
After updating to Jest 28, when running tests I am getting the error:
● Test suite failed to run
TypeError: Class extends value #<Object> is not a constructor or null
at Object.<anonymous> (node_modules/@shelf/jest-dynamodb/environment.js:5:52)
at Object.newLoader (node_modules/pirates/lib/index.js:141:7)
at ScriptTransformer.requireAndTranspileModule (node_modules/@jest/transform/build/ScriptTransformer.js:895:66)
This appears to be related to DynamoDBEnvironment
extending NodeEnvironment
(from jest-environment-node
).
For those of us using SAM-CLI directly, it would be nice to support parsing and transforming a SAM template.yam
file to set tables
in jest-dynamodb-config.js
When jest.config.js
contains "globalSetup" property jest-dynamodb-config.js
is not ran.
A workaround for this is to move setup script to jest-dynamodb-config.js
.
Hey there
I'm running shelf on my CI, "github actions"
I get the following error below. Do I need both "@aws-sdk/client-dynamodb" & "aws-sdk" installed 🤔 I just have aws-sdk installed as a peer dep. I missing something here?
Thank you
Versions
Error: Jest: Got error running globalSetup - /home/runner/work/api/api/node_modules/@shelf/jest-dynamodb/setup.js, reason: Cannot find module '@aws-sdk/client-dynamodb'
Require stack:
- /home/runner/work/api/api/node_modules/@shelf/jest-dynamodb/setup.js
- /home/runner/work/api/api/node_modules/jest-util/build/requireOrImportModule.js
- /home/runner/work/api/api/node_modules/jest-util/build/index.js
- /home/runner/work/api/api/node_modules/@jest/core/build/cli/index.js
- /home/runner/work/api/api/node_modules/@jest/core/build/jest.js
- /home/runner/work/api/api/node_modules/jest-cli/build/cli/index.js
- /home/runner/work/api/api/node_modules/jest-cli/bin/jest.js
- /home/runner/work/api/api/node_modules/jest/bin/jest.js
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:889:15)
at Function.Module._load (internal/modules/cjs/loader.js:745:27)
at Module.require (internal/modules/cjs/loader.js:961:19)
at require (internal/modules/cjs/helpers.js:92:18)
at Object.<anonymous> (/home/runner/work/api/api/node_modules/@shelf/jest-dynamodb/setup.js:3:20)
at Module._compile (internal/modules/cjs/loader.js:1072:14)
at Module._extensions..js (internal/modules/cjs/loader.js:1101:10)
at Object.newLoader (/home/runner/work/api/api/node_modules/pirates/lib/index.js:104:7)
at Module.load (internal/modules/cjs/loader.js:937:32)
at Function.Module._load (internal/modules/cjs/loader.js:778:12)
npm ERR! Test failed. See above for more details.
Error: Process completed with exit code 1.
This is the error I'm getting when trying to test a put command, like in the example.
I am using the following config options for the DDB client:
...(isTest && {
endpoint: 'localhost:8000',
sslEnabled: false,
tls: false,
region: 'local-env',
credentials: {
accessKeyId: 'fakeMyKeyId',
secretAccessKey: 'fakeSecretAccessKey',
},
}),
Do I need to explicity launch a local DynamoDB "instance" or should this library do that for me? The documentation isn't too clear on that part.
Below is my "put" class I made that I want to test
import { DynamoDB, PutItemCommand, PutItemCommandInput } from '@aws-sdk/client-dynamodb';
const isTest = process.env.JEST_WORKER_ID;
export class Put {
#client: DynamoDB;
constructor(region: string) {
this.#client = new DynamoDB(
{
...(isTest && {
endpoint: 'localhost:8000',
sslEnabled: false,
tls: false,
region: 'local-env',
credentials: {
accessKeyId: 'fakeMyKeyId',
secretAccessKey: 'fakeSecretAccessKey',
},
}),
},
);
}
async putItem(tableName: string, data: {[key: string]: any}) {
const params: PutItemCommandInput = {
TableName: tableName,
Item: data,
};
return this.#client.send(new PutItemCommand(params));
}
}
The get class
import { DynamoDB, GetItemCommand, GetItemCommandInput } from '@aws-sdk/client-dynamodb';
import { convertToAttr } from '@aws-sdk/util-dynamodb';
const isTest = process.env.JEST_WORKER_ID;
export class Get {
#client: DynamoDB;
constructor() {
this.#client = new DynamoDB(
{
...(isTest && {
endpoint: 'localhost:8000',
sslEnabled: false,
tls: false,
region: 'local-env',
credentials: {
accessKeyId: 'fakeMyKeyId',
secretAccessKey: 'fakeSecretAccessKey',
},
}),
},
);
}
async getItem(tableName: string, keyName: string, keyValue: string) {
const params: GetItemCommandInput = {
TableName: tableName,
Key: {
[keyName]: convertToAttr(keyValue),
},
};
return this.#client.send(new GetItemCommand(params));
}
}
And my test
import { Put } from '..';
import { Get } from '..';
test('should insert item into table', async () => {
const put = new Put();
const get = new Get();
await put.putItem('files', {id: '1', hello: 'world'});
const {Item} = await get.getItem('files', 'id', '1');
expect(Item).toEqual({
id: '1',
hello: 'world',
});
});
I followed all the steps on the Jest website and I can't make any progress. Why is the mock not working? from the error message the test is trying to connect to a dynamodb on port 8000, which is not what I want.
This is the code I'm writing tests
const { getAccessToken, getUser, getUserEmails } = require('../services/github');
const { createRedirectResponse } = require('../utils/redirect');
const { User } = require('../dynamodb');
const qs = require('qs');
exports.main = async (event, context, callback) => {
const redirect_uri = `https://${event.headers.Host}/${event.requestContext.stage}${event.path}`;
const code = event.queryStringParameters.code;
try{
const requestBody = {
client_id: process.env.CLIENT_ID,
client_secret: process.env.CLIENT_SECRET,
code,
redirect_uri
}
const githubCredentialsResponse = await getAccessToken(requestBody);
console.log(githubCredentialsResponse.data);
const githubCredentials = qs.parse(githubCredentialsResponse.data);
try{
const userDetailsPromises = [
getUser(githubCredentials.access_token),
getUserEmails(githubCredentials.access_token)
];
const success = await Promise.all(userDetailsPromises);
const githubUserResponse = success[0];
console.log(githubUserResponse.data);
const githubUserEmailsResponse = success[1];
console.log(githubUserEmailsResponse.data);
const user = githubUserResponse.data;
const emails = githubUserEmailsResponse.data;
var params = {
TableName: process.env.USER_TABLE,
Item: {
email: emails.find(email => email.primary).email,
github: {
credentials: githubCredentials,
user : user
},
name: user.name
}
};
try {
await User.put(params).promise();
console.log("Save operation was successful.");
callback(null, createRedirectResponse({
user : user,
access_token : githubCredentials.access_token,
expires_in : githubCredentials.expires_in
}));
} catch (error) {
console.error(error);
callback(null, createRedirectResponse({
error: 'Could not save User'
}));
}
}catch(error){
console.error(error);
callback(null, createRedirectResponse({
error: 'Could not get Github User and Emails'
}));
}
}catch(err){
console.log(err);
callback(null, createRedirectResponse({
error: 'Could not get Github Access Token'
}));
};
}
This is the Test that I have at the moment. The only place the test fails in when it reaches the await User.put(params).promise();
const qs = require('qs');
const axios = require('axios');
const lambda = require('../../../handlers/github-auth');
const redirect = require('../../../utils/redirect');
const userDocument = require('../../data/user_document.json');
const githubCredentials = require('../../data/github_credentials.json');
const userEmails = require('../../data/user_emails.json');
const user = require('../../data/user.json');
const AWS = require('aws-sdk');
const responseData = {
user,
access_token : githubCredentials.access_token,
expires_in : githubCredentials.expires_in
}
const response = {
statusCode: 301,
headers: {
"Location": `http://localhost:7482?${qs.stringify(responseData)}`
},
body: null
};
jest.mock('aws-sdk', () => {
return {
DynamoDB: jest.fn(() => {
return {
put: jest.fn(() => {
return {
promise: jest.fn(() => true)
};
})
};
})
};
});
jest.mock('axios');
jest.setTimeout(30000);
describe('Test github-auth handler', function () {
afterEach(() => jest.resetAllMocks());
it('should authenticate a user with github auth and add this user to dynamodb', async () => {
axios.post.mockImplementationOnce(() => Promise.resolve({ data: githubCredentials}));
axios.get.mockImplementationOnce(() => Promise.resolve({data: user}));
axios.get.mockImplementationOnce(() => Promise.resolve({data: userEmails}));
jest.spyOn(redirect, 'createRedirectResponse').mockImplementationOnce(() => Promise.resolve(response));
process.env.USER_TABLE = 'User';
process.env.REDIRECT_URI = 'http://localhost:7248';
process.env.CLIENT_ID = 'test';
process.env.CLIENT_SECRET = 'test';
const event = {
httpMethod: 'GET',
queryStringParameters: {
code: 'dsa7d6a7da9s7da9sd7a9'
},
headers: {
Host: 'dsadsadasdadsa.aws.com'
},
requestContext: {
stage: 'Prod'
},
path: '/auth/github'
};
await lambda.main(event, null, (error, result) => {
const responseData = {
user,
access_token: githubCredentials,
expires_in
}
const expectedResult = createRedirectResponse(responseData);
expect(result).toEqual(expectedResult);
});
});
});
Partially it's my fault as I wanted to use the library for unit tests and bypass it for integration tests, but what's the need in the teardown to delete all tables?
https://github.com/shelfio/jest-dynamodb/blob/master/teardown.js#L15
I'm not sure what causes this, but sometimes if my tests crash teardown is not called and dynamodb local continues to run. The problem is, when I run the tests again I get:
ResourceInUseException: Cannot create preexisting table
.
Tow possible solutions:
ResourceInUseException: Cannot create preexisting table
exception.I'm happy to implement either of the above and submit a PR.
Hello,
I've followed the steps outlined in the readme but I am getting the following error when executing jest.
Validation Error:
Preset @shelf/jest-dynamodb not found.
jest.config.js
module.exports = {
setupFiles: ['dotenv/config'],
testEnvironment: 'node',
preset: '@shelf/jest-dynamodb'
}
package.json
{
"name": "foo",
"version": "1.0.0",
"main": "index.js",
"dependencies": {
"aws-sdk": "^2.507.0",
"jsonwebtoken": "^8.5.1",
"serverless-dotenv-plugin": "^2.1.1"
},
"devDependencies": {
"@shelf/jest-dynamodb": "^1.7.0",
"jest": "^22.4.3",
"serverless-offline": "^5.10.1",
"supertest": "^3.0.0",
"timekeeper": "^2.2.0"
},
"jest": {
"verbose": true,
"testURL": "http://localhost:3000/"
},
"scripts": {
"dynamo:start": "cd __tests__ && java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar",
"dev": "sls offline start --port 4500",
"test": "jest"
},
"license": "ISC"
}
I tried changing the preset-package-name string by a relative path but didn't work.
Would you point me in the right direction, please?
In the example code, and my own tests based on the example provided, I had to add new code to my DDB wrapper class, such as the JEST_WORKER_ID
env var, and the config for the DDB client.
Is there a "neater" way of doing this, where I don't have to modify my actual code?
Simply by setting the following in jest config, all other AWS services (e.g. Secrets Manager) stop working:
{ "preset": "@shelf/jest-dynamodb" }
import { SecretsManager } from 'aws-sdk';
export default async function getSecret() {
const secretsManager = new SecretsManager();
const res = await secretsManager
.getSecretValue({
SecretId: "name",
})
.promise();
return res;
}
UnrecognizedClientException: The security token included in the request is invalid.
at Request.extractError (node_modules/aws-sdk/lib/protocol/json.js:51:27)
at Request.callListeners (node_modules/aws-sdk/lib/sequential_executor.js:106:20)
at Request.emit (node_modules/aws-sdk/lib/sequential_executor.js:78:10)
at Request.emit (node_modules/aws-sdk/lib/request.js:688:14)
at Request.transition (node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (node_modules/aws-sdk/lib/state_machine.js:14:12)
at node_modules/aws-sdk/lib/state_machine.js:26:10
at Request.<anonymous> (node_modules/aws-sdk/lib/request.js:38:9)
at Request.<anonymous> (node_modules/aws-sdk/lib/request.js:690:12)
at Request.callListeners (node_modules/aws-sdk/lib/sequential_executor.js:116:18)
Hello.
When running with jest watch, when jest re-runs tests, Setup fails with a ResourceInUseException
on setup.js line 65. I'm using the latest version of jest-dynamodb, v2.1.0.
Here's the debug logging right before the failure:
jest-dynamodb fallback to launch DB due to TypeError: Cannot destructure property 'TableNames' of '(intermediate value)' as it is undefined.
jest-dynamodb dynamodb-local is ready on port 8000
The type error is caused by
Lines 41 to 44 in 8be18c9
Although jest-dynamodb (correctly) does not stop dynamodb-local in Teardown, waitForLocalhost
wins the race against dynamoDB.listTables({})
.
I believe the solution is if dynamodb-local is still running during Setup, don't wait for localhost.
Sometimes when running tests, the tests fail with the following error: ResourceNotFoundException: Cannot do operations on a non-existent table
. This happens probably 5% of the time and it happens for every test.
I believe it is due to a race condition with the table creation. Seems like the tests run before the tables are created
Relevant dependencies:
"@shelf/jest-dynamodb": "^1.8.1"
"aws-sdk": "2.998.0"
"jest": "27.0.6"
"ts-jest": "27.0.3"
I've installed specific packages in aws-sdk version 3.
"@aws-sdk/client-dynamodb": "^3.20.0",
When I install jest-dynamodb it says it is missing listTables
TypeError: Jest: Got error running globalTeardown - /Users/jrobens/NetBeansProjects/azuron/winpay-uploader/node_modules/@shelf/jest-dynamodb/teardown.js, reason: Cannot read property 'listTables' of undefined at module.exports (/Users/jrobens/NetBeansProjects/azuron/winpay-uploader/node_modules/@shelf/jest-dynamodb/teardown.js:16:53) at /Users/jrobens/NetBeansProjects/azuron/winpay-uploader/node_modules/@jest/core/build/runGlobalHook.js:135:19 at ScriptTransformer.requireAndTranspileModule (/Users/jrobens/NetBeansProjects/azuron/winpay-uploader/node_modules/@jest/transform/build/ScriptTransformer.js:895:24) at /Users/jrobens/NetBeansProjects/azuron/winpay-uploader/node_modules/@jest/core/build/runGlobalHook.js:126:9 at pEachSeries (/Users/jrobens/NetBeansProjects/azuron/winpay-uploader/node_modules/p-each-series/index.js:8:23) at _default (/Users/jrobens/NetBeansProjects/azuron/winpay-uploader/node_modules/@jest/core/build/runGlobalHook.js:110:5) at runJest (/Users/jrobens/NetBeansProjects/azuron/winpay-uploader/node_modules/@jest/core/build/runJest.js:391:5) at _run10000 (/Users/jrobens/NetBeansProjects/azuron/winpay-uploader/node_modules/@jest/core/build/cli/index.js:408:7) at runCLI (/Users/jrobens/NetBeansProjects/azuron/winpay-uploader/node_modules/@jest/core/build/cli/index.js:261:3) at Object.run (/Users/jrobens/NetBeansProjects/azuron/winpay-uploader/node_modules/jest-cli/build/cli/index.js:163:37) run index.js:170 Async call from async function run index.js:163
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
@babel/cli
, @babel/core
)@aws-sdk/client-dynamodb
, @aws-sdk/lib-dynamodb
, @aws-sdk/util-dynamodb
)@types/jest
, jest
).circleci/config.yml
package.json
@aws-sdk/client-dynamodb 3.113.0
@aws-sdk/lib-dynamodb 3.113.0
@aws-sdk/util-dynamodb 3.113.0
cwd 0.10.0
debug 4.3.4
dynamodb-local 0.0.31
@babel/cli 7.19.3
@babel/core 7.20.2
@shelf/babel-config 1.2.0
@shelf/eslint-config 2.22.0
@shelf/prettier-config 1.0.0
@shelf/tsconfig 0.0.8
@types/aws-sdk 2.7.0
@types/cwd ^0.10.0
@types/jest 28.1.8
eslint 8.28.0
husky 8.0.2
jest 28.1.3
lint-staged 13.0.3
prettier 2.8.0
typescript 4.8.4
node >=16
Upgrading from 1.7 to 1.8 causes our tests to hang.
It seems that this change is causing a hang on await dynamoDB.listTables().promise()
.
It seems as if it was expected that local dynamo will throw if wasn't launched but seems to hang.
The endpoint
configured in here can be overridden by jest-dynamodb-config.js
. This is typically useful for docker-compose environment.
// jest-dynamodb-config.js
module.exports = () => {
return {
clientConfig: {
endpoint: 'http://dynamodb:8000',
region: 'localhost-test',
credentials: { accessKeyId: "access-key", secretAccessKey: "secret-key" },
},
};
};
However, waitForLocalhost
does not respect the custom endpoint config. It checks only localhost. This causes hanging-up with non-localhost endpoint.
Note that since jest-dynamodb 1.8.1 does not have waitForLocalhost
check, custom endpoint worked well on that version. Thus, it feels like a regression for docker-compose users who do not want to launch dynamodb-local.
In setup, the library checks to see if DynamoDB local is already running and only installs if the dynamodb local is not running and the directory ${os.tmpdir()}/dynamodb-local
is not setup correctly. I'd like an option that allows the user to explicitly disallow the library from downloading the jar.
This would allow users like me to keep using their existing dynamodb setup like this docker image while using this preset to setup and teardown tables.
I'm trying to move a load of Lambda functions to MJS, so I have set the type to "module" in package.json.
However, this means the jest-dynamodb-config.js gets interpreted as a module file, buts its commonjs.
I tried changing jest-dynamodb-config.js to jest-dynamodb-config.cjs but it fails with the following:
Error: Jest: Got error running globalSetup - /Users/me/Programming/Work/myproject/node_modules/@shelf/jest-dynamodb/setup.js, reason: Cannot find module '/Users/me/Programming/Work/myproject/jest-dynamodb-config.js'
Instead of using the DynamoDBLocal jar, have the ability to use the docker version of DynamoDBLocal
Hi!
Since updating from v1.8.1 to v2.0.3 my tests no longer run and get stuck at Determining test suites to run...
.
Adding debug logging I see:
birtles@Bristation:~/project$ DEBUG=jest-dynamodb yarn jest
yarn run v1.22.5
$ /home/birtles/project/node_modules/.bin/jest
Determining test suites to run... jest-dynamodb fallback to launch DB due to Error: connect ECONNREFUSED 127.0.0.1:8090
+0ms
jest-dynamodb spinning up a local ddb instance +0ms
After which point it hangs indefinitely.
It appears to be getting stuck here:
Line 57 in a9ef89e
and it appears to be because it insists on using DEFAULT_PORT
which resolves to 8000 but I have configured the client to run on port 8090 in my jest-dynamodb-config.js
.
Note that setup.js
is correctly resolving port
to 8090 but it simply fails to use that, preferring to use DEFAULT_PORT
instead.
That appears to be a regression introduced in 6192908 / 4ee04ea and then propagated further in fb6d175.
jest.config.js
const tsPreset = require('ts-jest/jest-preset');
const dynamodbPreset = require('@shelf/jest-dynamodb/jest-preset');
// ^ Note that the docs say `@shelf/jest-dynamodb` but that doesn't work for me
module.exports = {
...tsPreset,
...dynamodbPreset,
testMatch: ['**/*.test.ts'],
transform: {
'^.+\\.tsx?$': 'ts-jest',
},
modulePathIgnorePatterns: ['.aws-sam', 'cdk.out']
};
jest-dynamodb-config.js
module.exports = {
tables: [
// ...
],
port: 8090,
};
ddb-client.ts
import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { isTest } from '../utils/is-test';
export const ddbClient = new DynamoDBClient({
...(isTest && {
endpoint: {
hostname: 'localhost',
path: '/',
protocol: 'http',
port: 8090,
},
tls: false,
region: 'local-env',
}),
});
This happens to us inside of Github actions, which run on some Azure instance with a strange Java version installed (Azul built for Azure instance or something like that).
So it's not clear how to fix this but it would be enough to make it work.
Right now we can see that most of the time the tables are successfully created after 6-8 retries.
What we'd like is just to raise the max-retry configuration to ~20.
Would be best to have jest-dynamodb
accepts some general options to pass to the DynamoDb client. Should be pretty simple.
Should I move forward and open a PR?
I am trying to run a docker container that would perform the tests, but it seems that the dynamo instance shuts down just before the tests start.
It happens only within the container, it runs perfectly fine with the same config on my local machine.
That's the output of yarn test run against this github repository within docker container.
This is the Dockerfile I am using:
FROM openjdk:8-jre-alpine
RUN apk -v --no-cache add \
curl \
build-base \
groff \
jq \
less \
py-pip \
python openssl \
python3 \
python3-dev \
yarn \
git \
&& \
pip3 install --upgrade pip awscli boto3 aws-sam-cli
RUN git clone https://github.com/shelfio/jest-dynamodb
WORKDIR /jest-dynamodb
RUN yarn install
ENV AWS_ACCESS_KEY_ID='access_key'
ENV AWS_SECRET_ACCESS_KEY='secret_key'
RUN yarn test
I'm receiving the following error message after updating to v3.3.1:
● Validation Error:
Module ./node_modules/@shelf/jest-dynamodb/setup.js in the globalSetup option was not found.
<rootDir> is: /Users/mahdi.taghizadeh/Gousto/repos/recommendationApi-function
Configuration Documentation:
https://jestjs.io/docs/configuration
Running npm i
after update, removes the following files from /node_modules/@shelf/jest-dynamodb
: setup.js / teardown.js / wait-for-localhost.js
Have I missed a breaking change between versions 3.2.0 and 3.3.1?
Hello,
If I run my tests with the --watchAll flag the first time it runs just fine, but if I try to rerun using one of the watch commands it will output CONNECTION refused after a while.
What I think is happening is that the DynamoDB instance isn't persisting after first test run,
is there any way to preserve the watch flag and rerun tests with them making a new instance of dynamo?
Currently ParamValidator fails if a TimeToLiveSpecification is provided in the table properties.
Expected behaviour: TimeToLiveSpecification does not cause it to fail, and ideally would be used by the db
UnexpectedParameter: Unexpected key 'TimeToLiveSpecification' found in params
at ParamValidator.fail (node_modules\aws-sdk\lib\param_validator.js:50:37)
at ParamValidator.validateStructure (node_modules\aws-sdk\lib\param_validator.js:77:14)
at ParamValidator.validateMember (node_modules\aws-sdk\lib\param_validator.js:88:21)
at ParamValidator.validate (node_modules\aws-sdk\lib\param_validator.js:34:10)
at Request.VALIDATE_PARAMETERS (node_modules\aws-sdk\lib\event_listeners.js:126:42)
at Request.callListeners (node_modules\aws-sdk\lib\sequential_executor.js:106:20)
at callNextListener (node_modules\aws-sdk\lib\sequential_executor.js:96:12)
at node_modules\aws-sdk\lib\event_listeners.js:86:9
at finish (node_modules\aws-sdk\lib\config.js:357:7)
at node_modules\aws-sdk\lib\config.js:375:9
at SharedIniFileCredentials.get (node_modules\aws-sdk\lib\credentials.js:127:7)
at getAsyncCredentials (node_modules\aws-sdk\lib\config.js:369:24)
at Config.getCredentials (node_modules\aws-sdk\lib\config.js:389:9)
at Request.VALIDATE_CREDENTIALS (node_modules\aws-sdk\lib\event_listeners.js:81:26)
at Request.callListeners (node_modules\aws-sdk\lib\sequential_executor.js:102:18)
at Request.emit (node_modules\aws-sdk\lib\sequential_executor.js:78:10)
at Request.emit (node_modules\aws-sdk\lib\request.js:683:14)
at Request.transition (node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (node_modules\aws-sdk\lib\state_machine.js:14:12)
at Request.runTo (node_modules\aws-sdk\lib\request.js:403:15)
at node_modules\aws-sdk\lib\request.js:792:12
at new Promise (<anonymous>)
at Request.promise (node_modules\aws-sdk\lib\request.js:778:12)
at Promise.all.tables.map.table (node_modules\@shelf\jest-dynamodb\setup.js:27:70)
at Array.map (<anonymous>)
at createTables (node_modules\@shelf\jest-dynamodb\setup.js:27:29)
at module.exports (node_modules\@shelf\jest-dynamodb\setup.js:23:9)
at process._tickCallback (internal/process/next_tick.js:68:7)
Hi, I've been wrapping my head around this for a few days now, but it seems that jest refuses to exit after the tests are finished. I checked the debug, the teardown is properly executed, but I have the following message popping up every time whatever I try:
Jest did not exit one second after the test run has completed.
This usually means that there are asynchronous operations that weren't stopped in your tests. Consider running Jest with `--detectOpenHandles` to troubleshoot this issue.
I've been running the test on windows 10, both on a directory with real test and on a directory with a single expect(true).toBe(true) and nothing seems to work. Any idea of what I could do?
I am seeing this in VSCode.
I can run the tests (written in TypeScript) without issue, it's just that my IDE is flagging these in the editor window and I'd like to get rid of them properly.
I have installed @types/jest but that doesn't seem to have helped.
Could not run jest when added @shelf/jest-dynamodb as preset in the jest.config.js
file.
Error:
Determining test suites to run...Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/eclipse/jetty/server/handler/AbstractHandler
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:650)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:632)
Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.server.handler.AbstractHandler
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 19 more
Package Versions
"@shelf/jest-dynamodb": "^2.2.4"
"jest": "^26.4.2",
"ts-jest": "^26.2.0"
Does this work with the release of the v3 of aws-sdk?
I keep receiving connection errors after the update. Is this compatible with the major of aws-sdk or still in the works?
Specifically that's the error:
Error: AWS SDK error wrapper for Error: connect ECONNREFUSED 127.0.0.1:80
Which is weird considering the port for the local tests is 8000.
The connection details is as per suggestions from the Jest website.
Hello! I am seeing this error when I try to run some tests locally using jest-dynamodb wrapper. I seem to be able to run dynamo using dynamodb-local (https://github.com/rynop/dynamodb-local) and the tests I fail to run locally is running fine in Github Actions - there must be something off on my side but I cannot figure it out.
Determining test suites to run...Error: Unable to initialize main class com.amazonaws.services.dynamodbv2.local.main.ServerRunner
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException
Error: connect ECONNREFUSED 127.0.0.1:8000
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16)
Java --version output
java 10.0.1 2018-04-17
Java(TM) SE Runtime Environment 18.3 (build 10.0.1+10)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.1+10, mixed mode)
Any ideas on what to check?
I currently use preset: 'ts-jest'
. Does changing it to @shelfio/jest-dynamodb
preclude me from writing my tests in TypeScript?
There is a section in README.md talking about how to setup aws-sdk v2, but when installing the package a error requiring the version 3 shows up.
Thats is confusing, why the new sdk is needed even when it will not be used, and will the docs dont explicity say this.
Hi,
installed jest-dynamodb and my tests fail due to the following error:
TypeError: Class extends value #<Object> is not a constructor or null
at Object.<anonymous> (node_modules/@shelf/jest-dynamodb/environment.js:5:52)
at Object.newLoader (node_modules/pirates/lib/index.js:141:7)
at ScriptTransformer.requireAndTranspileModule (node_modules/@jest/transform/build/ScriptTransformer.js:890:66)
So this should be about class DynamoDBEnvironment extends NodeEnvironment
. Could this be due to a circular dependency? (see this)
This is kind of annoying but having an issue here that doesnt play well with Serverless. I have a DynamoDB table and using the jest dynamodb package to run a local DDB when running unit tests. I am using version 1.8.1
, I have a table that has a StreamSpecification set, which deploys fine using serverless (although I DO see StreamEnabled
property on AWS docs). But when I run unit tests, I get this error:
MissingRequiredParameter: Missing required key params.StreamSpecification
so I add it, tests run fine. Then when I try to deploy using serverless I get this error:
serverless Encountered unsupported property StreamEnabled
I assume this is a bug, but any workarounds?
Testing would be easier if database state could be reset at the beginning of each test
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.