peter-gy / snip-man

💻 A full-stack, dual-db code snippet manager. Purposefully overengineered for demonstration purposes.

SnipMan

This project was generated using Nx.

🔎 Smart, Fast and Extensible Build System

Table of contents

• Development Environment Setup
• Production Build & Run
• Working with nx

One-liner App Start

docker build . -t snip-man:nx-base && docker-compose up --build

The app start might take up to 10-15 minutes based on your internet connection and your machine's performance. The system will be fully functional, once snip-man-prisma-migrate exits with code 0. Please note that it is possible that snip-man-prisma-migrate will exit with code 1 first, indicating an error, but this will be the case iff the mongo replica set snip-man-mongo starts up too slow. The system will self-heal automatically, you should just wait until you see snip-man-prisma-migrate exited with code 0 in your logs.

You will be able to access the app via https://localhost. You will need to accept the self-signed certificates, or ignore them in your browser. Both of the urls below need to be accessible by your browser:

• https://localhost
• https://localhost:8443/api-docs/

Development Environment

The development environment of this project is fully dockerized via Dockerfile.dev and docker-compose.dev.yml. The setup makes it possible to edit the source files locally and run the nx commands inside a docker container. In addition, the database services can be also managed via docker-compose.

Steps to start the development environment:

1. Build and run the services: make dev-up
2. Start Nx services of your choice (scripts in package.json might be helpful)

Note that the repository root gets mounted bi-directionally into the /usr/snip-man-dev/ directory of the nx-dev service, therefore file-system changes made on the host will be immediately present inside the container, and vice-versa.

The web and server components can be served in development mode at once by executing npm run project:serve.

Truly Local Development

Install node modules:

npm install

Start the database instances with Docker and serve the project from your local machine:

make dev-up-local

The following services will be started:

• postgres: localhost:15432
• mongo: localhost:37017
• server: localhost:3333
• web: localhost:4200

Production Build & Run

The system can be built using Docker only:

1. Build the base nx image:

docker build . -t snip-man:nx-base

Note that this base image containing the whole repository source is necessary for the build to succeed, as nx needs access to the whole monorepo to do shared module transpilation and more advanced dependency-graph based dependency resolution.

2. Build the Docker services:

docker-compose build

The full system can be started by executing:

docker-compose up

Convenience command to execute the steps above at once:

Using pure bash:

docker build . -t snip-man:nx-base && docker-compose build && docker-compose up -d

Using make:

make prod-up

Accessing App Components

Only the web and server components are exposed publicly. They can be accessed both with and without using SSL. Please note that self-signed SSL certificates are generated upon container startup, hence your browser will complain about security risks when visiting any of the below HTTPS links.

• web:
  • http://localhost
  • https://localhost
• server:
  • http://localhost:3333/api
  • https://localhost:8443/api

Please note that it is necessary to ignore SSL certificate errors with browsers when visiting the above links.

• macOS, Chromium: open /Applications/Chromium.app --args --ignore-certificate-errors
• Windows, Chrome: using flag --ignore-certificate-errors

Working with nx

Serving Project web

In order to serve the Next application, residing in ./apps/web, execute:

npx nx run web:serve

The app should be accessible under http://localhost:4200.

Serving Project server

In order to serve the Nest application, residing in ./apps/server, execute:

npx nx run server:serve

The app should be accessible under http://localhost:3333/api.

Parallel Serve

In order to execute both the Next and Nest applications, execute:

npx nx run-many --target=serve --projects=web,server

Formatting the Whole Project

The whole project source code can be formatted by executing:

npx nx format:write

Understand your workspace

Run nx graph to see a diagram of the dependencies of your projects.

Further help

Visit the Nx Documentation to learn more.

☁ Nx Cloud

Distributed Computation Caching & Distributed Task Execution

Nx Cloud pairs with Nx in order to enable you to build and test code more rapidly, by up to 10 times. Even teams that are new to Nx can connect to Nx Cloud and start saving time instantly.

Teams using Nx gain the advantage of building full-stack applications with their preferred framework alongside Nx’s advanced code generation and project dependency graph, plus a unified experience for both frontend and backend developers.

Visit Nx Cloud to learn more.

Working With Prisma

Useful links

• Prisma Schema
• Data Modelling

Generating Code

Generate TypeScript clients:

npm run generate:prisma

Generate SQL:

npm run migrate:prisma

snip-man's Issues

feat: mock snippets

Description

Actual code snippets should be taken as mock data

Requirements

• various different snippets as mock data

feat: use-case create topic

feat: environment handling

Description

Set up environment variable handling for development and production modes, both for web and server.

feat(web): add `Geist UI` to the web app

Description

The Geist UI component framework is a light-weight library with an elegant look & feel, giving us the following elements "for free":

• Icons
• Input with hooks
• File Tree

Requirements

• Install the library and set up its top-level provider
• Remove DaisyUI (but keep TailwindCSS) in favour of Geist UI

perf: enhance NoSQL schema

Description

As I have been working on the submission report discussing the NoSQL schema, I realised that we have room for improvement (a.k.a chance not to get tons of points deducted)

Considerations

User

• Users are read frequently as a collection on the home page ->
• Own collection for users

Programming Topic

Topic <> Snippet

• Consider adding array of Snippet IDs

Topic <> User

• Programming topics are read frequently, but not globally, and only on a per-user basis
• It should be easy to query for prog topics by user
• Embedding them into user documents is not sensible, as topics might end up being large documents because of the nesting possibility ->
• Embed topic ids to user documents

Topic <> Topic

• Nested topics are read frequently, written probably also frequently
• It should be easy to access a child topic from a parent topic
• Embedding is not an option, as the recursive structure would end up in huge documents
• Our use case is to build a tree of the topics and display it in the UI and to create subtopics ->
• Embed parent id to topics

Tag

• Tags are read frequently, written seldom
• Tiny documents ->
• Embed tags to topics documents
• Create own collection to efficiently query

Programming Snippet

• Snippets are read frequently and written frequently
• They are always used in transactions together with topics, as snippets are weak entities
• Embedding snippets into topics is not sensible, snippets might be pretty big and there are lots of snippets
• We just need to make it efficient to access all snippets belonging to a topic
• Create own collection for snippets

Programming Language

• Read frequently as a collection (for language selection) ->
• Create own collection for languages
• Read frequently belonging to a prog topic & is a tiny document ->
• Embed language on topics

@S-ecki, please let me know if there are other considerations I did not think about!

feat: report language dominance

feat: implement postgres repository services

Description

Fully implement each postgres repository service and hook it up with the use-case services. Create db populator service.

chore: setup `docker-compose` for development

Description

Given that the usage of Docker is a hard requirement of this project, it would make sense to dockerize the development environment too.

Requirements

• Create a docker-compose file with development services
• Extend the README with docs on how the dev environment is meant to be used

chore: upgrade to an `NX` monorepo

Description

To leverage model sharing between frontend and backend & to have an overall more modern workflow, let's use NX for this project.

Obviously, it is a significant overkill for this project, but hey, it is a cool piece of technology, let's mess around with it!

feat: mongo schema setup

Description

Setup mongo schema with prisma.

fix: general fixes

Description

• Fix topic creation [postgres]
• Fix language dropdown [mongo]

feat: report users active in languages [postgres]

feat: finish milestone 2

Description

The title should be pretty self-explanatory. The sole purpose of this issue is to act as a list with TODOs to be ticked off.

Requirements

• Configuration of Infrastructure (6 points)
  • IS container creation and composition (every service is deployable locally)
  • Secure deployment (https, port exposure, data volumes, networks)
  • Implement build-pipeline (everything build-related is done in Docker)
• Logical/Physical database design (2 points)
  • Relational model and normalization
  • Implementation with relational DBMS
• Data import (2 points)
  • Automated and randomized data creation for the SQL database
• Implementation Web system (relational DBMS) (7 points)
• Implementation of Use Cases (relational DBMS) (8 points)
  • Realization of one main use case per team member (#17, #18)
  • Realization of one report per team member (#21, #23)
• NoSQL design decisions (15 points)
• Data migration (2 points)
• Implementation IS (NoSQL) (8 points)

feat: use-case create snippet

feat: report users active in languages [mongo]

feat: infrastructure configuration

Description

Setup the infrastructure of the system with full dockerization.

Requirements

• web component: NextJS, Typescript, Tailwind & DaisyUI
• server component: NestJS API
• postgres component: PostgreSQL database for the RDBMS phase
• mongo component: MongoDB database for the NoSQL phase
• reverse-proxy component: nginx reverse-proxy to expose the web and server components and make them accessible through HTTPS

feat: create relational data model

Description

Our ER model should be converted into TypeScript models & get relationally mapped.

Requirements

• Use Prisma for ORM
• Define our model schema as .prisma files

Additional Context

We have contemplated using TypeORM for the RDBMS phase, then Mongoose for the NoSQL phase; however, given that we are kind of short on time, I think it makes sense to go with Prisma, as it supports both PostgreSQL and MongoDB, hence we will have to get used to only one library instead of two.

ci: deploy app

Description

The app should be deployed somewhere. Anywhere.

feat: postgres API endpoints

Description

Extend the server app with API endpoints to communicate with Postgres.

Requirements

• Create base entity interfaces in libs/entities
• Create the nest controllers in a reasonably generic way so that we can re-use the structure for the mongo endpoints too

chore: use docker for all build steps

Description

The app should be executable via Docker commands only. Currently, there is a build step which needs to be done locally using nx. We cannot expect by default that all the required system dependencies are installed for that, so the first build step should be dockerized too.