livshitz / dojo-sdk

⛩ Dojo-SDK - Create simulated and deterministic environments with system components such as DB, MQ, Orchestrated MicroServices to practice and solve system-design challenges

⛩ Dojo-SDK

Introduction:

⛩ Dojo-SDK is a framework that provides components simulating contracts and behaviors of real-world system components like database, auto-scaled services, message queues, etc.
Inspired from Dojo halls, where participants are using dummies as Wing Chun Dummy (Mu ren zhuang) to learn and practice before going to real world.
With Dojo-SDK you can practice real-world and a complete system designs & concepts. It is designed to be deterministic and re-playable so you could test your concepts with unit-tests tools and verify your concepts end to end.
With a few commands on a single 'matrix' handler you can spawn local DB, messages queue, micro-service & orchestrator, scheduler and more.
See in Dojo-Recipes's repo a list of system-design challenges in which you can practice and implement using Dojo-SDK.
This is essentially a system-wide playground you can play with from within your command line or even run within your browser 🤯.

Features:

1. ⭐️ Database: CRUD and query operations with multiple persistency options, memory and disk. Stores the data on a local JSON file so you can see in real time changes and actually intercept and change values directly and alter the running DB.
2. ⭐️ MessageQueue: Complete working stripped down messaging queue Provider and Consumer so you could implement easily your own consumer. Supports auto scaling to simulate real world behavior.
3. ⭐️ Micro Services & Orchestrator: HTTP micro services that can auto-scale based on traffic to simulate real-world orchestrated environment.
4. Matrix: Handy root level object that consolidates all needed methods to spawn your environment, instead of manually creating objects. Inspired by the jQuery 'jQuery master' object.
5. Scheduler: CRON scheduler to simulate scheduled jobs.
6. Browser compatible: Setup your simulated environment within the browser and play with it!
7. Future:
   1. StreamProcessor (Kafka-like simulator): Simulate complex environments that include event streaming and events bus.
   2. SqlDatabase: Current Database implementation is NoSql in mind, in the future will better support SQL behavior database.

Examples:

Check src/examples/matrix.ts or browser version (play with it!)

const matrix = new Matrix();

// spawn DB with local disk persistency (will listen to local/external changes):
await matrix.addDB(new DiskPersistencyManager('./.tmp/db.json', true), {
	col: {
		'618230709af3ade104bee1ff': {
			a: 100,
			_id: '618230709af3ade104bee1ff',
		},
	},
});

// spawn MessageQueue service and a consumer:
await matrix.addMQ('queue1', { treat: (item) => console.log('This is my consumer treating item: ', item) });

// spawn a scheduler that will run every 5 seconds:
matrix.addScheduler(
	'*/5 * * * * *',
	() => {
		log.i('Scheduler: TICK!', faker.random.words(10));
	},
	SchedulerTypes.Recurring
);

// spawn a micro-service with auto-scaling from min 1 to max 10:
await matrix.addService(
	'/my-resource',
	() =>
		new (class extends BaseService {
			async handle(req: IRequest, res: IResponse) {
                console.log('Service:', req);
                res.type = ResponseTypes.OK;
                res.body = `You got it!`;
                return res;
            }
		})(),
	1,
	10
);

// perform actual call to the microservice 
await matrix.request(new RequestX('/my-resource', RequestMethods.GET));

Recipes:

Go to Dojo-Recipes to start on with suggested scenarios to utilize dojo-sdk and solve them.

Install:

$ yarn add dojo-sdk
or
$ npm install --save dojo-sdk   

Develop:

Build:

$ yarn build

Watch & Build:

$ yarn watch

Run tests:

$ yarn test <optional: path-to-test-file>

Debug:

Select 'typescript' debug configuration, open file in vscode (to run it specifically) and run debugger

or:

Select 'Node Attach' debug configuration, run specific file in debug mode (you can pass also args):

$ node --inspect build/Main.js

Use:

Run:

Install this npm package

Credits:

Logo SVG - Wing Chun by Icongeek26 from NounProject.com
Dojo GIF - makeagif.com

---

Scaffolded with 🏗 TS-scaffold .

dojo-sdk's Issues

Add support for Kafka-like `StreamProcessor` module

Currently the SDK supports DB, Scheduler, Microservice orchestrator and Messages Queue.
Streaming processors like Kafka, Amazon SQS, Google Pub/Sub, etc, are integral part of modern system architecture. It'd be very beneficial once the SDK will support such simulator.
Placeholder: https://github.com/Livshitz/Dojo-SDK/tree/master/src/StreamProcessor

Contract should resemble MessageQueue module and expose consumer (custom logic for consuming the messages) and producer (module that handles the enqueued messages, ordering, etc).

If you're new to Kafka, here's a nice intro video: https://kafka.apache.org/intro

Add missing recipe details

There are few high-level challenges I've listed here.
Help is needed to add more in-details description, like in the Email Scheduler recipe for instance.

Format:

Context:
Provide some background to the challenge to create a guided scenario in which the reader would solve. Try to tell a story.

Recipe:
What components or behaviors are expected to be seen in the result.

Validation:
Actual steps/conditions that could help the reader make sure his solution answers the challenge.

Add example for JS

The repo currently contains examples for TS usage of the SDK, it would be great to have an equivalent demo for JS developers.