dbt-labs / jaffle_shop_duckdb Goto Github PK

Also got a bunch of 'wheel' is not installed, Would it make sense to add wheel to the requirements.txt ?

Collecting text-unidecode>=1.3
  Using cached text_unidecode-1.3-py2.py3-none-any.whl (78 kB)
Using legacy 'setup.py install' for dbt-core, since package 'wheel' is not installed.
Using legacy 'setup.py install' for dbt-postgres, since package 'wheel' is not installed.
Using legacy 'setup.py install' for configobj, since package 'wheel' is not installed.

Notes from working through this:

• Encountered an error with python3 -m pip install -r requirements.txt and had to both update pip and brew install postgresql. Rerunning after updating pip and installing postrgesql worked! Had I not encountered this error, definitely would have been up and running in 1 min 🙂
• Forgot to run source venv/bin/activate after installing requirements, so it was defaulting to my local snowflake adapter instead of the duckdb one. Don't be like me and read the instructions carefully!
• Everything else ran super super smoothly! This is such a wildly fantastic idea and I can't wait to see it in use.

Hello -- thanks for this repo! I noticed that currently this repo uses code on a fork by @dbeatty10 .

Are there plans to open a PR and move these changes back into https://github.com/jwills/dbt-duckdb ? It seems like this might be related to duckdb/dbt-duckdb#21

Git cloning from [email protected]:dbt-labs/jaffle_shop_duckdb.git will create a folder named "jaffle_shop_duckdb" by default.

But if using a custom, non-default folder name in the git clone command, this will point to a folder that doesn't exist:

Hi, I'm quite new to DBT and DuckDB. I'm still a bit confused about setting up primary and foreign keys in the models.

I saw that this is how it is being implemented in the models:

When I open the resulting .db file in DBeaver or Metabase, the constraints doesn't seem to be implemented.

My question is:

• Is this something that is expected in DBT or am I missing something?
• Why don't we implement this in the .sql scripts?

PS:
I also found these tools:

• dbt_constraints - Not sure if it support duckdb
• dbt-metabase - works great

follow the readme toggles and copy them into github action format.

This will serve as auto checks too.

It'll be cool for new folks coming across this repo for the first time thinking, "Yeah right, no way I can get started that fast" then click through the github actions and go, "Oh they already proved it's that fast..."

Let's avoid this error for people

dbt-labs/dbt-core#5425

Hi I've attempted to start the project by first copying the following instructions:

git clone https://github.com/dbt-labs/jaffle_shop_duckdb.git
cd jaffle_shop_duckdb
python3 -m venv venv
source venv/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install -r requirements.txt
source venv/bin/activate
dbt build
dbt docs generate
dbt docs serve

And that led to this error.

So then I tried by using the provided vscode dev container but then it somehow fails to find the csv seed files. (even tho they exist in the repo)

When running this repo in VS Code (locally or via codespaces), it looks like the tasks in the devcontainer complete okay, but the dbt power user extension pops up an error.

Any idea how to prevent the error? It seems maybe related to needing DBT_PROFILES_DIR set?

For example, the windows cmd instructions don't work. I tried in a github action the syntax was wrong.

Synced with @sungchun12 on some minor updates to the README to hopefully improve readability :)

• Linking to documentation on profiles.yml in step 4
• Removing the "What is a jaffle?" section; I know that's a cornerstone of jaffle shop, but I can't tell if it's just adding dead weight to a semi-long readme
• Moving Doug's callout/dropdown to the virtual environment reactivation earlier up (maybe at the step it occurs)
• The note in Step 2 here is potentially confusing—this is bey default using a duckDB database/adapter, so the code should work without having to change any of the model code. I think the note could be reworded along the lines of, "If you decided to use this project in your own data warehouse, make sure....blah blah"
• Linking to documentation on dbt build in step 5

The accepted values for payment_method field for stg_customers are defined in `models/staging/schema.yml'.

version: 2

models:
  - name: stg_customers
    columns:
      - name: customer_id
        tests:
          - unique
          - not_null

  - name: stg_orders
    columns:
      - name: order_id
        tests:
          - unique
          - not_null
      - name: status
        tests:
          - accepted_values:
              values: ['placed', 'shipped', 'completed', 'return_pending', 'returned']

  - name: stg_payments
    columns:
      - name: payment_id
        tests:
          - unique
          - not_null
      - name: payment_method
        tests:
          - accepted_values:
              values: ['credit_card', 'coupon', 'bank_transfer', 'gift_card']

These values are used in models/orders.sql.

-- ...

order_payments as (

    select
        order_id,

        {% for payment_method in payment_methods -%}
        sum(case when payment_method = '{{ payment_method }}' then amount else 0 end)
            as {{ payment_method }}_amount,
        {% endfor -%}

        sum(amount) as total_amount

    from payments

    group by order_id

),

Can payment_methods defined once in `models/staging/schema.yml', so that we could avoid data inconsistencies in the future. If it is possible, how can I do it?

Currently, the README is a giant wall of text that is intimidating to me.

As a user, I always appreciate the main focus to be install instructions that help me get hands-on. Everything else is a distraction from that goal, in my opinion.

What good looks like

• minimum prerequisites
  • minimum software to install
  • minimum skills/knowledge required
• the user can quickly find all the commands they need to run
  • the relevant commands don't need to be fished out from the middle of sentences (see example below)
• work for the greatest number of users
  • instructions that are cross-platform (zsh, PowerShell, etc)
• ideally fits on a single page

Example of fishing out relevant commands

• pip upgrade command needs to be fished out from here
• Should just be a stand-alone line that says this instead:

  pip install --upgrade pip

Hi Team,

we would like to seed or read Parquet file from local storage and Azure Blob Storage in project , Can you please guide the configuration which we can do to read the file.

Regards,
Akash