graphile / starter Goto Github PK

Problem

tsc is unable to navigate imports from any workspace in a workspace that is tsconfig configured with a rootDir.

Repro: https://github.com/jkantr/starter/tree/repo_import__to_workspace_with_rootdir
Error:

@app/server: error TS6059: File '/mnt/d/src/graphql-starter-repro/@app/graphql/index.tsx' is not under 'rootDir' '/mnt/d/src/graphql-starter-repro/@app/server/src'. 'rootDir' is expected to contain all source files.

This error persists even when added the workspace in question to the local deps. See:
https://github.com/jkantr/starter/blob/repo_import__to_workspace_with_rootdir/%40app/server/package.json#L15

Solution:

Use TS3.0 Project References - example referencer tsconfig:

{
  "extends": "../../tsconfig.json",
  "compilerOptions": {
    "noEmit": false,
    "rootDir": "src",
    "outDir": "dist",
    "tsBuildInfoFile": "dist/tsconfig.tsbuildinfo",
    "declarationDir": "dist",
    "lib": ["es2018", "esnext.asynciterable"],
    "target": "es2018",
    "module": "commonjs"
  },
  "include": ["src"],
  "references": [{ "path": "../graphql" }]
}

Example refrencee tsconfig:

{
  "extends": "../../tsconfig.json",
  "compilerOptions": {
    "jsx": "react",
    "target": "es2017",
    "module": "commonjs",
    "noEmit": false,
    "composite": true
  },
  "include": ["index.tsx"]
}

(the above with @app/server and @app/graphql fixes the error in the above repo branch)

Possible Resolution:

Either include certain references / reference paths by default in the starter for workspaces with rootDirs or include a section in the README pertaining to this enhancement when needed.

in setting up the starter, i had to comment some lines out from the server index.ts file. this is because they reference the starter's authorization design (User). This could be refactored to be a bit easier, but not so hard to work around manually. in fact, it gives me a roadmap to implement my own stuff on all fronts. some reorganization of this code and comments explaining as much might be good:

  /*
   * Middleware is installed from the /server/middleware directory. These
   * helpers may augment the express app with new settings and/or install
   * express middleware. These helpers may be asynchronous, but they should
   * operate very rapidly to enable quick as possible server startup.
   */
  await middleware.installDatabasePools(app);
  // await middleware.installSession(app);
  // await middleware.installPassport(app);
  await middleware.installLogging(app);
  // These are our assets: images/etc; served out of the /client/public folder
  await middleware.installSharedStatic(app);
  // if (isTest || isDev) {
  //   await middleware.installCypressServerCommand(app);
  // }
  await middleware.installPostGraphile(app);
  // await middleware.installNext(app);

also PassportLoginPlugin and SubscriptionsPlugin had to be commented out of installPostgraphile.ts

for the db migrations, i created a new template database by pg_dump -s my old db, then using psql to run that file on the new template

i added TEMPLATE_DATABASE_NAME to my .env

added preAfterReset to .gmrc

  "afterReset": [
    {
      "_": "command",
      "command": "scripts/preAfterReset"
    },
    "afterReset.sql",
    {
      "_": "command",
      "command": "cd backend/dist/worker && npx --no-install graphile-worker --once"
    }
  ],

in scripts, added preAfterReset

#!/usr/bin/env bash

set -e
export NODE_ENV=development

if [ -x .env ]; then
  . .env

  fail () {
    echo ".env already exists, but it doesn't define $1 - aborting!"
    exit 1;
  }

  if [ "$DATABASE_HOST" = "" ]; then fail DATABASE_HOST; fi;
  if [ "$DATABASE_OWNER" = "" ]; then fail DATABASE_OWNER; fi;
  if [ "$DATABASE_OWNER_PASSWORD" = "" ]; then fail DATABASE_OWNER_PASSWORD; fi;
  if [ "$DATABASE_AUTHENTICATOR" = "" ]; then fail DATABASE_AUTHENTICATOR; fi;
  if [ "$DATABASE_AUTHENTICATOR_PASSWORD" = "" ]; then fail DATABASE_AUTHENTICATOR_PASSWORD; fi;
  if [ "$TEMPLATE_DATABASE_NAME" = "" ]; then fail DATABASE_RESET_TEMPLATE; fi;

  echo "Configuration already exists, using existing secrets."
fi

echo "Rebuilding database from template..."
echo $DATABASE_NAME     $ROOT_DATABASE_URL
PSQL="psql"
$PSQL -X -v ON_ERROR_STOP=1 "${ROOT_DATABASE_URL}" <<SQL
DROP DATABASE IF EXISTS ${DATABASE_NAME};
CREATE DATABASE ${DATABASE_NAME} TEMPLATE ${TEMPLATE_DATABASE_NAME};

DROP DATABASE IF EXISTS ${DATABASE_NAME}_shadow;
CREATE DATABASE ${DATABASE_NAME}_shadow TEMPLATE ${TEMPLATE_DATABASE_NAME};

DROP DATABASE IF EXISTS ${DATABASE_NAME}_test;
CREATE DATABASE ${DATABASE_NAME}_test TEMPLATE ${TEMPLATE_DATABASE_NAME};
SQL

echo "✅ preAfterReset success"

Server timezone may differ from client timezone leading to dates being rendered differently on client vs server and causing hydration issues. For example, reload the email management page.

The instructions say to start by downloading the .zip file.
If yarn start is run on this folder before git init, the process fails with the error

(TEST) [jest] --watch is not supported without git/hg, please use --watchAll
(TEST) [jest] node --inspect=9876 node_modules/.bin/jest -i --watch exited with code 1

The start script should check if it's being run in a git repo, and if not, fail and instruct the user to git init.

First of all, thanks for this starter project. I personally found it very helpful when learning Postgraphile and working on my first project:

1. it shows how Postgraphile features work together in practice;
2. it shows the best practices;
3. it's well documented, up-to-date and complete;
4. it drastically reduces costs of setting up a new project from scratch, especially for beginners.

Obviously it makes sense that the starter project contains an authentication solution. Most of real world apps require it. But having this solution hardcoded in the project has several drawbacks, IMO. An authentication solution is:

1. Quite complex when done right, lots of code, features and security requirements;
2. Critical, must be maintained and bug-free.

With this in mind, it might not match the goals of the starter project for the following reasons:

1. The project is intended to be used at "point in time". If an issue is discovered and fixed at a later date regarding the authentication, any previous user of the starter project won't get the fix easily;
2. The project is incredibly useful for beginners, as a tutorial/playground for learning Postgraphile. However the authentication code takes the big part of the spotlight. And it's not the best learning material. As said above, it's rather complex and domain-specific.

So what about extracting this code into a flexible authentication plugin, as an alternative to JWT + external auth solutions?

• It can evolve over time with security fixes and additional features;
• It will greatly simplify the starter project code and lower the barrier for beginners. The starter can then showcase a simpler model like a forum or a todo-list;
• Right now if a developer doesn't want to implement authentication manually, he's required to go with JWT and an external auth service. But I believe many developers would gladly not deal with the complexities of JWT for a simple auth. Coming from Ruby on Rails development, I know libraries like Devise (https://github.com/plataformatec/devise) are very popular and a go-to choice for implementing a simple, in-house auth system;
• For developers that just want the boilerplate code for auth in order to customize the code internally, cloning the starter project or cloning the auth plugin project is pretty much the same;

Now I understand it's big task to make it as an external plugin, especially to make it flexible and pluggable with the rest of the stack. But most of the work is already done in the starter project and it looks good!

I think it will be a great long-term addition to the Postgraphile ecosystem.
What do you think?

This should do the trick:

docker/package.json

...
"preinstall": "echo $(cat NOOP_MESSAGE.txt); exit 126",
...

docker/NOOP_MESSAGE.txt

Exiting without running `yarn install` because this is a helper package that has no dependencies created by graphile/starter.

This is a noop that prevents a yarn.lock file from being accidentally generated.

Maybe a "Danger Zone" area in settings like on GitHub. It should require password entry, or an email roundtrip, or something.

Setup does not work with AWS RDS due to no SUPERUSER role

Having @app/client-cra for a Create-React-App based version of the @app/client might be a better option for apps that sit behind the login wall and don't need any SSR.

It seems to me, that this error handling code is never reached

 } catch (e) {
      /*
       * If anything goes wrong, let the test runner know so that it can fail
       * the test.
       */
      res.status(500).json({
        error: {
          message: e.message,
          stack: e.stack,
        },
      });

because it get's handled by /work/@app/server/src/middleware/installErrorHandler.ts before that.

Did I just mess up my configuration or can somebody confirm that?

On my end this is bad for cypress testrunner because it just waits forever until a timeout without any information (there is a error log in the server cli printed by installErrorHandler.ts ofc)

I expected to be able to see what the application would look like but didnt get the repo running. :(

Reproducable example

$ git clone https://github.com/graphile/starter
$ cd starter

Reading the readme or the docker readme https://github.com/graphile/starter/blob/master/docker/README.md i run the command:

$ export UID; yarn docker setup
yarn run v1.21.1
$ yarn --cwd ./docker setup
$ node ./scripts/yarn-setup.js
$ yarn compose down --remove-orphans
$ docker-compose -f ../docker-compose.yml down --remove-orphans
Stopping starter_db_1 ... done
Removing starter_server_run_2 ... done
Removing starter_server_run_1 ... done
Removing starter_db_1         ... done
Removing network starter_default
$ yarn compose up -d db
$ docker-compose -f ../docker-compose.yml up -d db
Creating network "starter_default" with the default driver
Creating starter_db_1 ... done
$ docker-compose -f ../docker-compose.yml run server sudo bash -c 'chmod o+rwx /var/run/docker.sock && chown -R node /work/node_modules /work/@app/*/node_modules'
Starting starter_db_1 ... done
$ docker-compose -f ../docker-compose.yml run server yarn setup starter
Starting starter_db_1 ... done
yarn run v1.21.1
$ yarn && node ./scripts/setup.js starter
[1/4] Resolving packages...
success Already up-to-date.
Installing or reinstalling the roles and database...
Database is not ready yet (attempt 1)
Database is not ready yet (attempt 2)
Database is not ready yet (attempt 3)
Database is not ready yet (attempt 4)
Database is not ready yet (attempt 5)
Database is not ready yet (attempt 6)
Database is not ready yet (attempt 7)
Database is not ready yet (attempt 8)
Database is not ready yet (attempt 9)
Database is not ready yet (attempt 10)
Database is not ready yet (attempt 11)
Database is not ready yet (attempt 12)
Database is not ready yet (attempt 13)
Database is not ready yet (attempt 14)
Database is not ready yet (attempt 15)
Database is not ready yet (attempt 16)
Database is not ready yet (attempt 17)
Database is not ready yet (attempt 18)
Database is not ready yet (attempt 19)
Database is not ready yet (attempt 20)
Database is not ready yet (attempt 21)
Database is not ready yet (attempt 22)
Database is not ready yet (attempt 23)
Database is not ready yet (attempt 24)
Database is not ready yet (attempt 25)
Database is not ready yet (attempt 26)
Database is not ready yet (attempt 27)
Database is not ready yet (attempt 28)
Database is not ready yet (attempt 29)
Database is not ready yet (attempt 30)
Database never came up, aborting :(
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

trying to setup locally with Windows10 and WSL but I have this error (coval is my windows user):

`✘ coval@coco-desk  ~/winhome/Documents/projects/plants/starter-2.2.0   master  yarn setup
yarn run v1.21.1
$ yarn && node ./scripts/setup.js
[1/4] Resolving packages...
success Already up-to-date.
? What would you like to call your database? graphile_starter
? What's the hostname of your database server (include :port if it's not the default :5432)? localhost:6543
? Please enter a superuser connection string to the database server (so we can drop/create the 'graphile_starter' and 'graphile_starter_shadow' databases) - IMPORTANT: it must not be a connection to the 'graphile_starter' database itself, instead try 'template1'. postgres://localhost:6543/template1
? We're going to drop (if necessary):

• database graphile_starter
• database graphile_starter_shadow
• database role graphile_starter_visitor (cascade)
• database role graphile_starter_authenticator (cascade)
• database role graphile_starter Yes
  Installing or reinstalling the roles and database...
  error: password authentication failed for user "coval"
  at Connection.parseE (/c/Users/Coco/Documents/projects/plants/starter-2.2.0/node_modules/pg/lib/connection.js:604:13)
  at Connection.parseMessage (/c/Users/Coco/Documents/projects/plants/starter-2.2.0/node_modules/pg/lib/connection.js:403:19)
  at Socket. (/c/Users/Coco/Documents/projects/plants/starter-2.2.0/node_modules/pg/lib/connection.js:123:22)
  at Socket.emit (events.js:203:13)
  at addChunk (_stream_readable.js:295:12)
  at readableAddChunk (_stream_readable.js:276:11)
  at Socket.Readable.push (_stream_readable.js:210:10)
  at TCP.onStreamRead (internal/stream_base_commons.js:166:17) {
  name: 'error',
  length: 101,
  severity: 'FATAL',
  code: '28P01',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'auth.c',
  line: '337',
  routine: 'auth_failed'
  }
  error Command failed with exit code 1.
  info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.`

I am not sure where to input my password. I have also creating a user coval in the database but still the same error.
I think it is a stupid mistake but can't find solution.

This project uses PostGraphile's default CRUD mutations, which is perfectly reasonable and expected. However, when a developer takes this codebase and customizes it for their own project, they will probably turn off the default CRUD mutations in favor of custom mutations -- and at that point, things go a bit off the rails.

GraphQL Code Generator is designed to validate that our queries match our schema, but once these mutations are disabled, the queries no longer match, and running yarn build results in a lot of ugly errors. Furthermore, it's not immediately obvious that these errors are due to disabling the default mutations, since they involve code and queries that the developer didn't write and is not familiar with. Even after the developer determines the cause of the errors, it's not clear what is the best way to resolve the issue: delete the code? Disable graphql-codegen? Write custom mutations that match the autogenerated ones that you need? Something else?

To be clear, the problem here is not using autogenerated mutations. That is a very handy feature for PostGraphile, and it makes sense to show it off in this repo. The problem is that there are no clear paths for how to disable this feature without breaking everything. This problem could be solved with documentation, with informative error messages, with a migration file that defines the database functions necessary to recreate these mutations, or some combination of these. (Maybe other ways, too?)

When I attempt to log in via GitHub, this error throws after the callback:

error: null value in column "email" violates not-null constraint
    at Connection.parseE (/home/frank/git/starter/node_modules/pg/lib/connection.js:601:11)
    at Connection.parseMessage (/home/frank/git/starter/node_modules/pg/lib/connection.js:398:19)
    at Socket.<anonymous> (/home/frank/git/starter/node_modules/pg/lib/connection.js:120:22)
    at Socket.emit (events.js:193:13)
    at addChunk (_stream_readable.js:295:12)
    at readableAddChunk (_stream_readable.js:276:11)
    at Socket.Readable.push (_stream_readable.js:231:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:154:17)

Not sure, but the authorization screen on Github was only accessing public data, perhaps it needs to request access to Email address?

As privately discussed, it would be super useful for various different setups if we would avoid to use psql in setup.js. @benjie says he could maybe rewrite it to use node-postgres.

Example for setups which would then be possible:

• using the starter natively on windows OS. where the postgreSQL server is eg. on another machine or started via docker
• use inside our docker-compose setup.

Explanation about docker-compose

While trying to combine the latest commits about dotenv, js scripts, etc, it turned out the single major hassle is to to call docker-compose run pgtools psql or $PSQL via spawnSync from inside the running webapp/dev container.

For reference:

• the current workaround with calling a .sh: https://github.com/JoeSchr/graphile-starter/blob/0ec5bf8c7850f9c4f54820de1796675d6242a322/scripts/setup.js#L367
• (not working, because ENOENT) try to use spawnSync directly: https://github.com/JoeSchr/graphile-starter/blob/501240363fc0e70544c8e08b6fc618b9ab9048d6/scripts/setup.js#L358

Since with my current work we wouldn't need a special DOCKER_MODE anymore, they only inside container calls to docker-compose are psql to reset database at setup and further on pg_dump to update schema.sql afaik .

Also it turns out, the current way we are doing pg_dump via pgtools/docker-compose.yml is actually not working, just was not failing. The dump never actually updates schema.sql. I tested this by using a different databasename then graphile-starter

Reason it seems you can't map a volume if you call docker-compose from inside another running container. I'm currently exploring other solutions, maybe via using docker cp and a special Dockerfile.

But it would be really great, if we could just use node-postgresql or any other "native JS" solution so we don't need to support psql tooling on different plattforms.

Hi @benjie,

On February 4th, 2020 at 08:51 (UTC) your application (Graphile Starter (Heroku)) used an access token (with the User-Agent passport-github) as part of a query parameter to access an endpoint through the GitHub API:

https://api.github.com/user/emails

Please use the Authorization HTTP header instead as using the access_token query parameter is deprecated.

Depending on your API usage, we'll be sending you this email reminder once every 3 days for each token and User-Agent used in API calls made on your behalf.
Just one URL that was accessed with a token and User-Agent combination will be listed in the email reminder, not all.

Visit https://developer.github.com/changes/2019-11-05-deprecated-passwords-and-authorizations-api/#authenticating-using-query-parameters for more information.

Thanks,
The GitHub Team

Might be sufficient to just upgrade passport-github?

After uncommenting example commands in current.sql, the following graphql query unexpectedly fails. The reason I expect it to work is in the docs: https://www.graphile.org/postgraphile/relations/

query MyQuery {
  posts {
    edges {
      node {
        userByAuthorId {
          email
        }
        body
        headline
      }
    }
  }
}

On inspection of data/schema.graphql, I notice that authorId is an int, instead of a reference. In other words,

type Post {
  authorId: Int
}

should probably be

type Post {
  author: User!
}

Am I missing something? Pardon my ignorance.

Here is my changeset: https://github.com/bdombro/starter/pull/1

Require Docker and git-bash on Windows. And make it smooth.

e.g. https://github.com/graphile/bootstrap-react-apollo/blob/master/db/CONVENTIONS.md

See also: https://www.troyhunt.com/everything-you-ever-wanted-to-know/

Should dump on commit.

When run from any subdirectory, say scripts, eslint clean.js says:

Error: Cannot read config file: /Users/jj/src/table-stakes/.eslintrc.js
Error: ENOENT: no such file or directory, open './data/schema.graphql'

That is what my editor does (I suppose others run eslint from the top, and give the path to the file in question). It seems right to allow both. The fix is:

 const { readFileSync } = require("fs");
-const schemaString = readFileSync("./data/schema.graphql", "utf8");
+const schemaString = readFileSync(`${__dirname}/data/schema.graphql`, "utf8");
 
 module.exports = {
   parser: "@typescript-eslint/parser",

Looking for an easy way to create a generic React package to supply my @app/client package with "dumb" components, but it seems like there is a lot of boilerplate needed that is slightly different from @app/graphql or @app/config, meaning that for newbies to the project, the overhead of creating a new React-specific module is non-trivial, and will probably leave others and myself creating sub-optimal implementations.

if i manually create the db, then:

Solution: force them to follow the "forgot password" flow, but maybe wrap it up nice?

Problem

The starter properly generates cryptographically secure secrets in the setup script, but has a 'fallback' using a very dubious default:

If one misconfigures their env vars after setup, or circumvents the setup, this could result in using a very insecure secret for session signing.

Solution

Provide no default here and fail if there is no secret available.

https://discordapp.com/channels/489127045289476126/498852330754801666/655062567240073219
https://discordapp.com/channels/489127045289476126/498852330754801666/655066636516327454

[10:04 AM] Benjie:IMO things like unused vars should be something ESLint tells you about, not TypeScript. I think TypeScript was overreaching with that rule.

[10:20 AM] Benjie: Oh and add these to tsconfig:

    "noUnusedLocals": false,
    "noUnusedParameters": false,


I'd actually be tempted to add these to the official starter TBH; as I say my opinion is
that's ESLint's responsibility.

Explain about how migrations/committed works/etc.

https://www.npmjs.com/package/helmet

(Note to self: use the one you did for Steve.)

Reduce the number of root-level dependencies and commands to make it easier to grok what's going on.

When running the production worker docker image, the following error occurs when an email is sent.

[worker(worker-2405044886759662)] ERROR: Failed task 30 (send_email) with error ENOENT: no such file or directory, open '/app/@app/worker/dist/../templates/verify_email.mjml' (17.34ms):
  Error: ENOENT: no such file or directory, open '/app/@app/worker/dist/../templates/verify_email.mjml'

This is fixed by placing the following on line 51 of production.Dockerfile

COPY --from=builder /app/@app/worker/templates/ /app/@app/worker/templates/

but, is there a way to only include the template folder if the $TARGET is worker?

The table name in the starter scripts

does not conform to the naming convention started in the schema design guide on postgraphile.
https://www.graphile.org/postgraphile/postgresql-schema-design/#the-person-table

Expected the tables to be named after the guide like user and not users.

If the error code is 'ENOTFOUND' then it's not likely the attempt to setup the database will succeed. Perhaps change the code to:

  // Wait for PostgreSQL to come up
  let attempts = 0;
  while (true) {
    try {
      await pgPool.query('select true as "Connection test";');
      break;
    } catch (e) {
      if (e.code === 'ENOTFOUND') {
        console.log(
          'Could not find database connection at host:',
          answers.DATABASE_HOST
        );
        process.exit(1);
      }
      attempts++;
      if (attempts <= 60) {
        console.log(`Database is not ready yet (attempt ${attempts})`);
      } else {
        console.log('Database never came up, aborting :(');
        process.exit(1);
      }
      await sleep(2000);
    }
  }

Setup:

• MacOS Catalaina
• yarn v1.21.1
• ZSH & iTerm

Steps to reproduce:

• git clone https://github.com/graphile/starter.git
• yarn docker setup

I also tried pruned all containers and images to force rebuild but it did not help.
Found this issue #3 and checked commit history but did not find anything useful in this date range which was very different to the current setup

Looking at the last merge to master here, I can see the error for error TS5072: Unknown build option '--noEmit'.

I can see noEmit is set in config throughout:

https://github.com/graphile/starter/search?q=emit&unscoped_q=emit

however, when using tsc --build for project references, many of of your options change, and noEmit is one of the things you can no longer do. you do have the option of doing tsc --build --dry which is effectively the same.

https://www.typescriptlang.org/docs/handbook/project-references.html#tsc--b-commandline

Try using the types entry in the root tsconfig to solve.

Used in this file.

I have asked how to move from having this:

  {
    getDataFromTree: 'ssr',
  }

...to the new format.

I'll post an update here when I figure out how to update.

This could maybe be helped by adding a sample new table to the current.sql the user can customise

HT @voodooattack

Doesn't seem like it would hurt: https://nextjs.org/docs#changing-x-powered-by

We cannot do so until dotansimha/graphql-code-generator#3326 is solved.