metriql / metriql-lookml

metriql Looker integration

metriql Looker Integration

Generates LookML model from your metriql datasets. The idea is to leverage metriql datasets in your Looker setup without any additional modeling.

Usage

The library is available in PyPI so you can install it via pip as follows:

pip install metriql-lookml

The library expects stdin for the metriql metadata and outputs a ZIP file to stdout. Here is an example:

curl http://metriql-server.com/api/v0/metadata | metriql-lookml --connection myproject > metriql.zip

You can use --file argument instead of reading the metadata from stdin as an alternative.

The command outputs a zip file to stdout by default, if you pass --out myfile.zip argument, it will write to the specified file instead.

How does it work?

1. Create a connection to metriql from Looker using its Trino interface.

2. Run metriql-lookml passing the relevant connection argument for metriql.

You need to enable JDBC in your metriql server to be able to use metriql's Trino interface.

Build from source

Requires pipenv and python >=3.7

# From root folder
# Install
pipenv install --dev

# Run
pipenv run metriql2lookml

metriql-lookml's Issues

Implement relations inside measures

Implement relations inside measures. We don’t need to create joins inside models instead all the measures will be in the same explore.

Support downloading files as a zip file

We will call this library from our web app and let people generate LookML projects locally if they wish so. In order to integrate with the web app, the users need to be able to download the LookML files as a zip file. Here is how it looks like: https://cln.sh/wVY9s9

We can have a command-line argument called --stdout that outputs a zip file to stdout so that we can call the Python library from our application or the users can run the following command:

metriql-looker --stdout > files.zip

Implement timeframes

Looker supports dimension groups for different time and duration values. While metriql doesn't support duration yet, we should generate appropriate LookML that lets people to drill down into different time buckets. If we have a dimension as follows:

name: created_at
column: created_at
timeframes: [day, week, month]
type: timestamp

We should generate the following LookML:

view: view_name {
  dimension: created_at__day {
    label: Day
    sql: ${TABLE}.created_at::day ;;
    group_label: "Created At"
  }
  dimension: created_at__week {
    label: Week
    sql: ${TABLE}.created_at::week ;;
    group_label: "Created At"
  }
 dimension: created_at__month {
    label: Month
    sql: ${TABLE}.created_at::month ;;
    group_label: "Created At"
  }
}

Since we don't want Looker to take care of the timeframes themselves, we create different dimensions and group them using the group_label attribute.

Publish library to pip

We should publish the library to PyPI and let people install the library and run them locally as follows:

pip install metriql-looker
metriql-looker [ARG]

Create Dimensions inside each explore

Create Dimensions inside same explore

Rather than creating one model for each dataset, let’s create a single file

1. If the description is an empty string, let’s ignore it
2. Rather than creating one model for each dataset, let’s create a single file called explores.model and include all the explores.
3. Here is an example:

connection: "Project Name"
include: "/views/*"
explore: source_snowflake_tpch_sample_tpch_orders {
  description: ""
}
explore: source_snowflake_tpch_sample_tpch_customers {
  description: ""
}

3. We should probably implement command line arguments for changing the name but the default will be explores