sanity-io / groq-cli

Run GROQ in your command line

groq-cli

Currently in alpha.

Easy wrangling of JSON documents with GROQ in the command line.

The CLI tool consumes both JSON and NDJSON) documents. You can pass in data from a local file, or from piping to standard input.

Read the announcement blog post, and the getting started guide.

Install

npm install --global groq-cli

Requirements

This CLI requires Node v10 or later.

Usage

$ groq --help
  Run GROQ in the command line

  Usage
    $ groq '*[<filter>]{<projection>}'
    # Remember to alternate quotation marks inside of the query

  Options
    -i, --input   One of: ndjson, json, null
    -o, --output  One of: ndjson, json, pretty
    -p, --pretty  Shortcut for --output=pretty
    -n, --ndjson  Shortcut for --input=ndjson --output=ndjson

  Input formats
    json      Reads a JSON object from stdin.
    ndjson    Reads a JSON stream from stdin.
    null      Reads nothing.

  Output formats
    json      Formats the output as JSON.
    pretty    Formats the output as pretty JSON.
    ndjson    Streams the result as NDJSON.

  Examples
    # Query data in a file
    $ cat blog.json | groq 'count(posts)'

    # Query data in a NDJSON file
    $ cat blog.ndjson | groq --input ndjson '*[_type == "post"]{title}'

    # Query JSON data from an URL
    $ curl -s https://jsonplaceholder.typicode.com/todos | groq  --pretty '*[completed == false]{title}'

Similar tools

GROQ-cli isn't the only tool to work with JSON data in the command line. If it doesn't do exactly what you need, you can check out these other tools that might help you:

• jq — a lightweight and flexible command-line JSON processor.
• gron - Make JSON greppable!

License

MIT – Copyright 2019–present Sanity Inc.

groq-cli's Issues

support for interactive mode

Would be nice to be able to just pass in a filename and enter a REPL mode to experiment with the syntax fast.

Possible memory leak

Seeing the below warning.

(node:27236) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 end listeners added. Use emitter.setMaxListeners() to inc
rease limit
[...]

irena $ wc -l migration/ndjson/employees.ndjson 
     343 migration/ndjson/employees.ndjson
irena $ du -sh !$
du -sh migration/ndjson/employees.ndjson
1.1M    migration/ndjson/employees.ndjson
irena $ cat migration/ndjson/employees.ndjson | groq --input ndjson '*[published==true]{name}'

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

---

Using a curated preset maintained by

Sanity: The Composable Content Cloud

---

Pending Approval

These branches will be created by Renovate only once you click their checkbox below.

• fix(deps): Update non-major (groq-js, node)

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• chore(deps): lock file maintenance

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): Update dependency get-stdin to v9

Detected dependencies

github-actions

.github/workflows/test.yml

• actions/checkout v4
• actions/setup-node v4
• actions/checkout v4
• actions/setup-node v4

npm

package.json

• chalk ^4.0.0
• get-stdin ^8.0.0
• groq-js ^1.4.3
• json-lexer ^1.2.0
• meow ^9.0.0
• ndjson ^2.0.0
• regenerator-runtime ^0.14.1
• stream-to-async-iterator ^1.0.0
• @sanity/semantic-release-preset ^4.1.7
• eslint ^8.57.0
• eslint-config-prettier ^9.1.0
• eslint-config-sanity ^7.1.2
• prettier ^3.2.5
• node >=18

nvm

.nvmrc

• node 20.11.1

---

• Check this box to trigger a request for Renovate to run again on this repository

'--primary' flag no longer works

At https://www.sanity.io/blog/we-re-open-sourcing-groq-a-query-language-for-json-documents we see the query

$ groq '*[gender == "female" && "physics" in prizes[].category]{firstname, surname}' \
  --url http://api.nobelprize.org/v1/laureate.json \
  --pretty --primary laureates

resulting in

[
    {
      "firstname": "Marie",
      "surname": "Curie, née Sklodowska"
    },
    {
      "firstname": "Maria",
      "surname": "Goeppert Mayer"
    },
    {
      "firstname": "Donna",
      "surname": "Strickland"
    }
  ]

but today it prints null.

It appears that --primary is no longer there, and that that is the culprit. I say that because the query works as expected when we use jq instead of groq's --primary flag:

$ curl -s http://api.nobelprize.org/v1/laureate.json | jq .laureates | groq '*[gender == "female" && "physics" in prizes[].category]{firstname, surname}' --pretty

`TypeError` when using `-n` CLI flag

When using -n CLI flag, the following error is thrown: TypeError: ndjson is not a function

This regression likely happened after upgrading ndjson package from version 1.x.x to version 2.x.x, where the API of the package changed. The error takes place in the following two places:

groq-cli/src/cli.js

Line 132 in c726590

const dataset = new S2A(process.stdin.pipe(ndjson()))

groq-cli/src/cli.js

Line 108 in c726590

if (result.getType() == 'array') {

In order to fix this issue, update inputNDJSON and outputNDJSON functions. Consider using the following implementations:

function inputNDJSON() {
  const dataset = new S2A(process.stdin.pipe(ndjson.parse()))
  return {dataset}
}

async function* outputNDJSON(result) {
  const value = await result.get()

  if (Array.isArray(value)) {
    for await (const row of value) {
      yield JSON.stringify(row)
      yield '\n'
    }
  } else {
    yield JSON.stringify(value)
    yield '\n'
  }
}