Coder Social home page Coder Social logo

girish21 / speed-metal-stack Goto Github PK

View Code? Open in Web Editor NEW
162.0 4.0 16.0 1.86 MB

The Remix Blog Stack for deploying to Fly with MDX, SQLite, testing, linting, formatting, etc.

Home Page: https://remix.run/stacks

Shell 0.26% JavaScript 14.52% Dockerfile 1.75% TypeScript 82.71% CSS 0.77%
remix-stack

speed-metal-stack's Introduction

Remix Speed Metal Stack

Learn more about Remix Stacks.

npx create-remix --template Girish21/speed-metal-stack

Remix Blog ๐Ÿ“–

This blog starter template was inspired heavily by Kent C. Dodds implementation of kentcdodds.com. You can read more about the architecture and the idea behind it at How I built a modern website in 2021.

Architecture ๐Ÿ’ก

website-architecture

Important ๐Ÿšง

Fly requires a globally unique name for all the apps, and we've used the directory name and random hash as the app name. Of course, you can change this anytime you want BEFORE launching the app with Fly CLI. But it's not a big deal since you can reassign the internal Fly URL to any custom domain by adding a CNAME record to your custom domain pointing to the Fly internal URL. We'll see that later when deploying the app to production.

Quickstart

# run database migrations and set up the initial blog
npm run setup
# run the initial build for the development server
npm run build
# start the development server and run other processes in parallel in watch mode
npm run dev

Available scripts

  • build - compile and build the express server, Remix app, Tailwind in production mode
  • dev - starts the express server, Remix watcher, Tawilwind CLI in watch mode
  • format - runs prettier on the codebase and fixes fixable issues
  • lint - runs ESLint on the codebase
  • new:blog - create a new Blog post template from the command line
  • start - starts the express server (should only be executed after running npm run build)
  • test - runs vitest
  • test:e2e:dev - starts the cypress runner in development mode
  • test:e2e:run - starts the cypress runner in CI mode
  • typecheck - runs type check on the codebase

Fly Setup ๐Ÿ› 

  1. Install Fly

  2. Sign up and log in to Fly

    flyctl auth signup

Database ๐Ÿ—ƒ

We use SQLite as the database in this template. SQLite is a fast database engine, a great option to persist data without reaching for advanced database engines like Postgres.

Installation โš™๏ธ

SQLite comes pre-installed on all Macs. You can check the official installation guides for other OS's SQLite Installation Guides

Why do we need a database โ“

We use MDX-Bundler to compile the MDX files, and MDX-Bundler uses esbuild internally to compile the MDX files. Though esbuild is very fast, there is a noticeable lag during this process which is not suitable for the performance of the site and the user experience. And since there is no need to compile MDX on every request when the data does not change seems like a waste of time and performance. So instead, we can cache the compiled MDX and recompile it only when we know the content has changed.

Prisma โ–ณ

We use Prisma as the ORM in this template. To create the SQLite database and initialise the database schema, run:

npx prisma migrate dev

The above command will prompt for a migration name, and you can name it as initial migration. This command will also install Prisma Client for interacting with the database.

Development ๐Ÿ’ป

We can start our development server with the migrations run and the SQLite database populated with the initial schema. Then, from a new tab in your terminal, run the command.

npm run dev

This command starts four processes concurrently.

  • The Remix dev server starts in development mode and rebuilds assets on file change.
  • Tailwind CLI which rebuilds the stylesheet when the styles change
  • An MSW server which intercepts the API calls to GitHub and serves the content from the local instead of calling the remote API
  • A file watcher watches over the content directory and rebuilds the assets.

Relavant files ๐Ÿ”

Deployment ๐Ÿš€

Initial setup ๐Ÿ‘€

Before proceeding to deploy our app, we have some steps to take care of:

  • Create a GitHub account GitHub
  • Create a new app on Fly
flyctl launch --name YOUR_APP_NAME --copy-config --no-deploy

โš ๏ธ Remember not to deploy since we have some setup steps left to complete!

Environment variables and Secrets ๐Ÿคซ

This template comes with GitHub actions workflows to automatically deploy the app to Fly.io. First, we need to set up our GitHub actions and the Fly app with some secrets. Let's do that now.

To push the build image to the remote Fly registry from GitHub action, we need an access token from Fly. We can generate that using the Fly command line, run:

flyctl auth token

The command will generate an access token. You can then add this token to your GitHub actions secrets by visiting your GitHub repository's settings page https://github.com/:owner/:repo/settings/secrets/actions and then click New repository secret. Next, GitHub will prompt for a key and a value. The key should be FLY_API_TOKEN, and the value will be the token generated by the command line.

We also need to set the Fly app name as a secret, the key should be FLY_APP_NAME, and the value will be the app name specified in fly.toml

Now we need to set up secrets in our Fly app.

Since we're fetching the content from GitHub on demand instead of building all the pages upfront, we need an access token from GitHub to call the GitHub API and fetch the content. Also, GitHub won't rate-limit the app from calling the GitHub API more often. So, you can generate an access token at Personal access token. Then, you can copy the generated token and set it to your app's secret. We can do that by running the following command:

flyctl secrets set GITHUB_TOKEN={GITHUB_TOKEN}

We also need a secret to sign our session. We can do that by running the command:

flyctl secrets set SESSION_SECRETS=$(openssl rand -hex 32)

If openssl is not available, you can generate a secure token using a password generating service like 1Password.

The last secret required is a token for securely communicating between the GitHub action and our app deployed on a remote server since we need a public-facing API for this communication.

openssl rand -hex 32

We have to set this secret as part of the GitHub actions secret and a Fly secret. The key should be REFRESH_TOKEN. You can create a new actions secret in GitHub and create a new secret for the Fly app by running the command.

flyctl secrets set REFRESH_TOKEN={GENERATED_PASSWORD}

Volumes ๐Ÿ’พ

We also need to create a volume in Fly to persist our app data (SQLite DB) so that Fly can persist the data stored across deployments and container restarts. Again, we can do that using the Fly command line.

flyctl volumes create data --region [REGION] --size 1

Note: REGION should be the region selected when launching the app. You can check the region chosen by running flyctl regions list.

It's important to note that Volumes are bound to an app in a region and cannot be shared between apps in the same region or across multiple regions.

You can learn more about Fly Volumes here

Push to Prod ๐Ÿฅณ

We are ready for our first deployment. GitHub actions workflows are configured to run on push to the main branch. So let's push the local branch main to remote, triggering the workflows.

Once all the checks are passed, and the deployment is complete, you can run:

flyctl info

To get the current app URL and IP address. The app URL will be https://YOUR_APP_NAME.fly.dev. You can visit that URL, and the site should be online. That's it. You have deployed your blog built using REMIX!.

Adding Custom Domain ๐Ÿ”–

To add a custom domain to the app, you first must buy a domain from a Domain Name Register, and you can choose one of your preferences. Some popular options are Domain.com, Google, Cloudflare.

After buying the domain, we can add a DNS record to point to the domain or create a subdomain and point that to the Fly app URL. We can do that by adding a DNS record using the CNAME option and entering the Fly URL https://YOUR_APP_NAME.fly.dev.

We also have to create an SSL certificate on Fly with the domain name. We can do that by running the command:

flyctl certs create [DOMAIN]

You can read more about this at SSL for Custom Domains

That's it, and we are ready to share our blog with the rest of the world! But there is one more step to take care of before sharing it.

Scaling โš–๏ธ

There are two ways of scaling an application, vertical and horizontal.

In vertical scaling, the system is scaled by adding more compute resources to the server (increasing the CPU/RAM). Fly supports vertical scaling, and you can check the docs here scaling virtual machines.

Horizontal scaling is achieved by adding more replicas of the same application, either in the same region or in other regions worldwide. Fly supports many regions worldwide, and you can read more about them here Fly regions.

Our app is currently deployed in only one region we selected when we ran flyctl launch. This is fine during prototyping and development, but when pushing for production, we would want our app to be accessible from regions worldwide and have similar performances for users worldwide. In this case, we can add more replicas of our app worldwide, at least in the regions with many targetted users, so that the app will run on the servers closer to the users, and all the users will have comparable performance.

Since Fly anchors the regions based on the volumes created, we can add more regions by creating a volume in the new region or adding a replica in the same region. For example, we can do that by:

flyctl volumes create data --region [NEW_REGION] --size 1

You can check this list of available regions at Fly regions

And then increase the scale count of the Fly app by running the command:

flyctl scale count 2

The above command will set the scale count to 2, meaning two instances of our app will run on the specified regions where the volumes are created.

You can read more about scaling at Fly scaling

API Mocks ๐Ÿ”ธ

Our architecture is to fetch the blog content from the GitHub repository on demand and not bundle the content as part of the build. Therefore, we will be making a significant amount of API calls to GitHub. And with any API, there come restrictions such as rate limit, calls per minute, etc. And when we're writing an article, the process becomes tedious since we're making calls to the GitHub API; the article has to be on GitHub so that the API can return the content. This process is not ideal. We can do better.

Instead, we can mock the API request to GitHub and serve the articles locally, providing a better experience. Instead of calling the GitHub API, we use MSW to intercept the request and return a mocked response that serves the content from the local file system.

Linting โฌก

This template comes preconfigured with ESLint and Prettier rules and is integrated with the workflow as a build step. For example, you can run npm run lint to run ESLint on the project and npm run format to run prettier.

Styling ๐Ÿ’…๐Ÿป

This template comes preconfigured with Tailwindcss with all the scripts required during development and production.

The template also comes with a theme toggler preconfigured and can detect the suitable theme and prevent FART

Testing ๐Ÿ”ฌ

This template comes preconfigured with Jest and React testing library for unit testing and Cypress for e2e testing and is configured to run as part of the GitHub actions. You can run the npm run test command to run the Jest test suite. And npm run test:e2e:run to run the Cypress tests in a headless mode. You can check package.json for the available commands.

Type check สฆ

You can run npm run typecheck to run tsc on the codebase. Type check is also included as part of the deployment workflow.

Debugging

Some helpful commands to debug the application on Fly using the command line

Logs

You can check the logs using the command flyctl logs from the project directory, containing the fly.toml file in the project's root. You can also check the logs from the console by visiting fly.io/apps.

Console

You can also log in to the remote console using the flyctl ssh console command.

Database

After logging in to the console, you can also inspect the SQLite DB. But first, we have to install SQLite on the remote machine. We can do that using the apt-get install sqlite3 command. Then, cd into the volume using the cd data command (Note: data refers to the volume's name created from the command line). And then run the command sqlite3 sqlite.db to open a command-line interface into the database.

Important links

speed-metal-stack's People

Contributors

girish21 avatar kory-smith avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

speed-metal-stack's Issues

refresh-on-content-change should trigger if content is deleted

Currently, the refresh-on-content-change script does not trigger when content (i.e a blog post) is deleted. It only triggers when the file(s) are changed, but not on delete.

I think ideally the script should run when anything is added, removed or modified in the content directory.

Error on `npm run dev`

Hello! ๐Ÿ‘‹

I'm doing an initial setup using speed-metal-stack and am running into an error.

Steps to reproduce

  1. Run npx create-remix --template Girish21/speed-metal-stack
  2. cd into newly created directory and run npm run setup
  3. Run npm run build
  4. Run npm run dev and get the error below (It doesn't seem to like any of the ~/ imports?):
> dev
> run-p dev:*


> dev:server
> cross-env NODE_ENV=development node --inspect --require dotenv/config --require ./mocks ./build/server.js


> dev:build
> cross-env NODE_ENV=development npm run build:server -- --watch


> dev:css
> tailwindcss -w -i styles/app.css -o app/styles/app.css


> dev:content
> node -r esbuild-register -r dotenv/config others/refresh-on-content-change.ts


> dev:remix
> cross-env NODE_ENV=development remix watch

Debugger listening on ws://127.0.0.1:9229/15a1f155-0c16-4ad3-a794-dc8fed5de229
For help, see: https://nodejs.org/en/docs/inspector
MSW initialised

Rebuilding...

> build:server
> esbuild --platform=node --format=cjs ./server/index.ts --outfile=build/server.js "--watch"

[watch] build finished, watching for changes...
(node:32298) ExperimentalWarning: stream/web is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
Error: Cannot find module '~/components/nav'
Require stack:
- /Users/janessa/Dev/remix/mdx-stack/build/index.js
- /Users/janessa/Dev/remix/mdx-stack/build/server.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/Users/janessa/Dev/remix/mdx-stack/build/index.js:98:26)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Module._compile (/Users/janessa/Dev/remix/mdx-stack/node_modules/esbuild-register/dist/node.js:2258:26)
    at Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Object.newLoader (/Users/janessa/Dev/remix/mdx-stack/node_modules/esbuild-register/dist/node.js:2262:9)
    at Object.extensions..js (/Users/janessa/Dev/remix/mdx-stack/node_modules/esbuild-register/dist/node.js:4801:24)
ERROR: "dev:server" exited with 1.

Environment

  • Node v16.13.2
  • npm v8.1.2
  • MacOS

Any ideas what might be happening?

TypeError: Cannot read properties of undefined (reading 'sha')

Hello! Thank you for your work on this stack, it's really great!

I'm running into a strange issue. I've published some posts, but I can't seem to update them. If I commit and deploy a changes to the markdown file associated with a post, I am still seeing the initial version of that post when deployed to production.

I'm not sure entirely what is causing this issue, but I did notice this error during the Determine Changes step on CI:

Run echo ::set-output name=DEPLOYABLE::$(node ./others/is-deployable.js)
TypeError: Cannot read properties of undefined (reading 'sha')
    at go (/home/runner/work/***/***/others/is-deployable.js:7:30)
    at processTicksAndRejections (node:internal/process/task_queues:9[6](https://github.com/tconroy/my-repo/runs/7243487957?check_suite_focus=true#step:5:7):5)

Wonder if that could be related? I've seen this undefined error relating to sha crop up a few times in my CLI with this stack.

Refresh all content

How can I manually refresh all content in production?
My environment got completely out of sync

Error on npm run build

When i create a brand new project and run npm run build i get the following error:

/path/to/project/node_modules/@remix-run/dev/cli/create.js:26
var sortPackageJSON = require('sort-package-json');
                      ^

Error [ERR_REQUIRE_ESM]: require() of ES Module /path/to/project/node_modules/sort-package-json/index.js from /path/to/project/node_modules/@remix-run/dev/cli/create.js not supported.
Instead change the require of index.js in /path/to/project/node_modules/@remix-run/dev/cli/create.js to a dynamic import() which is available in all CommonJS modules.

I took the following steps:

  1. create a new project
npx create-remix --template Girish21/speed-metal-stack
  1. run npm run setup
  2. run npm run build

i am using Node.js v17.2.0

New Deploy for each new blog post

Hi, thank you very much for this awesome starter stack. I played a little bit with it and i notice, that the Github Action Deploy doesn't exit, even when i only change the content under content/blog. I checked the logs under โš™๏ธ Determine Changes

Run echo ::set-output name=DEPLOYABLE::$(node ./others/is-deployable.js)
  echo ::set-output name=DEPLOYABLE::$(node ./others/is-deployable.js)
  shell: /usr/bin/bash -e {0}
  env:
    FLY_APP_NAME: ***
TypeError: Cannot read properties of undefined (reading 'sha')
    at go (/home/runner/work/blog/blog/others/is-deployable.js:7:[3](https://github.com/:repo/blog/runs/6241374380?check_suite_focus=true#step:5:3)0)
    at processTicksAndRejections (node:internal/process/task_queues:96:[5](https://github.com/:repo/blog/runs/6241374380?check_suite_focus=true#step:5:5))

I follow the readme for Deployment and setup the following Github Actions Secret

FLY_API_TOKEN=*****
FLY_APP_NAME=*****
REFRESH_TOKEN=*****

I also checked the buildInfo endpoint. When i manuell go to https://{app}.fly.dev/build/info.json i only receive this values:

{
  "timestamp": 1651339488348
}

So sha is not defined and so isDeployable is always true

Have I forgotten anything or done anything wrong? Interestingly, despite the new deployment, the new posts do not show up in the blog list under /blog unless I go manuell to the appropriate slug like /blog/hello-world.

Error failed to fetch an image or build from source

Not sure if others are experiencing this issue, but I just recently started with this stack (great work and thank you for making it ๐Ÿ˜„ ) and I had to troubleshoot the below error that was causing my deploy to fail.

==> Verifying app config
--> Verified app config
==> Building image
Searching for image 'registry.fly.io/***:[...] locally...
Searching for image 'registry.fly.io/***:[...] remotely...
Error failed to fetch an image or build from source: Could not find image "registry.fly.io/***:[...]

Of course now I can't find the discussion link where I found the solution but this is what got it working for me.

In deploy.yml changed:

 - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1

to:

 - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
        with:
          version: v0.9.1

The main thing was specifying the version. Hopefully this helps if others are also running into the issue. Not sure if it's just me or the issue should be resolved another way, but if it'd be helpful I can add a PR.

TypeError: Cannot read properties of undefined (reading 'ServerClient')

Hi, after i used your stack and added some new pages, i wanted to restart my dev server with npm run dev but i get the following error in the console:

/path/to/project/app/routes/blog.rss[.]xml.ts:23
            const frontMatter = JSON.parse(post.frontmatter)
                                                ^
TypeError: Cannot read properties of undefined (reading 'ServerClient')
    at Object.<anonymous> (/path/to/project/app/routes/blog.rss[.]xml.ts:23:49)

Do you have any idea what this error means and how to fix it? I have not changed anything related to the stack.

Initial Setup Fails

Hi ๐Ÿ‘‹

I followed the instructions to get this setup.

Ran npx create-remix --template Girish21/speed-metal-stack with no errors

Ran npm run setup with no errors.

And followed with

npm run dev

I am then stuck with this error:

โœ˜ [ERROR] [plugin server-bare-modules] Cannot find module '~/package.json'
Require stack:
- /Users/mikesmith/Github/mikerhyssmith/node_modules/@remix-run/dev/compiler/plugins/serverBareModulesPlugin.js
- /Users/mikesmith/Github/mikerhyssmith/node_modules/@remix-run/dev/compiler.js
- /Users/mikesmith/Github/mikerhyssmith/node_modules/@remix-run/dev/cli/commands.js
- /Users/mikesmith/Github/mikerhyssmith/node_modules/@remix-run/dev/cli/run.js
- /Users/mikesmith/Github/mikerhyssmith/node_modules/@remix-run/dev/cli/index.js
- /Users/mikesmith/Github/mikerhyssmith/node_modules/@remix-run/dev/index.js
- /Users/mikesmith/Github/mikerhyssmith/node_modules/@remix-run/dev/cli.js

    app/components/footer.tsx:4:22:
      4 โ”‚ import GitHubSvg from '~/assets/icons/github.svg'
        โ•ต                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~

  This error came from the "onResolve" callback registered here:

    node_modules/@remix-run/dev/compiler/plugins/serverBareModulesPlugin.js:46:12:
      46 โ”‚       build.onResolve({
         โ•ต             ~~~~~~~~~

    at setup (/Users/mikesmith/Github/mikerhyssmith/node_modules/@remix-run/dev/compiler/plugins/serverBareModulesPlugin.js:46:13)
    at handlePlugins (/Users/mikesmith/Github/mikerhyssmith/node_modules/@remix-run/dev/node_modules/esbuild/lib/main.js:842:23)

Any idea what might be happening here ?

Node version: v16.14.2
NPM versiomn: 8.5.0
OS: MacOS

Question: What is a good way to implement images?

Hi, thank you very much for this stack. That's exactly what I was looking for ๐Ÿ‘ I'm honestly still a newbie when it comes to Remix and I need to understand all the workings of this stack. So my question is: What is a good and scalable way to integrate images into this stack, which will be displayed in a blogpost? Of course I could upload any image to the /public folder, but would that be recommended? I saw that Kent C. Dodds used cloudinary for his website. How could that be integrated into this stack as well, or is there perhaps another "simpler solution"?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.