firebolt-db / dbt-firebolt Goto Github PK

Background

Minor version v1.8 is targeted for final release within dbt Core on May 9, 2024.

As a maintainer of a dbt adapter, we strongly encourage you to release a corresponding minor version increment to ensure users of your adapter can make use of this new minor version.

As of dbt-core v1.8.0, we no longer need to encourage you to release a new minor version anytime we do. After following the linked upgrade guide, we guarantee your adapter will be forward compatible with all future minor versions of dbt-core (at least until v2.0 which is not yet planned).

Another major win: you can now make your adapter truly SemVer compliant, as you can release new versions of your adapter without needing to wait for a new dbt-core release. You can actually follow

How to upgrade

dbt-labs/dbt-core#9798 is an open discussion with more detailed information. If you have questions, please put them there!

Background

Minor version v1.6 is targeted for final release on July 27, 2023. As a maintainer of a dbt adapter, we strongly encourage you to release a corresponding minor version increment to ensure users of your adapter can make use of this new minor version.

How to upgrade

dbt-labs/dbt-core#7958 is an open discussion with more detailed information. If you have questions, please put them there!

The above linked guide has more information, but below is a high-level checklist of work that would enable a successful 1.6.0 release of your adapter:

### Tasks
- [ ] SUPPORT: materialized views
- [ ] SUPPORT: new `clone` command
- [ ] BEHIND THE SCENES: Drop support for Python 3.7 (if you haven't already)
- [ ] BEHIND THE SCENES: new arg for `adapter.execute()`
- [ ] BEHIND THE SCENES: ensure support for revamped `dbt debug``
- [ ] BEHIND THE SCENES: Add support for new/modified relevant tests

the next minor release: 1.7.0

FYI, dbt-core==1.7.0 is expected to be released on October 12, 2023 in time for Coalesce, the annual analytics engineering conference!

does the firebolt-sdk support actual status messages yet? Or, do we still have to hard-code "OK"? Not a must have for now, but something we'll want on our backlog probably. for example:

• dbt-postgres parses cursor.statusmessage
• dbt-snowflake uses a cursor.sqlstate

Originally posted by @swanderz in #10 (comment)

We've just published the release cut of dbt-core 1.2.0, dbt-core 1.2.0rc1 (PyPI | GitHub release notes).

dbt-labs/dbt-core#5468 is an open discussion with more detailed information, and dbt-labs/dbt-core#5474 is for keeping track of the communities progress on releasing 1.2.0

Below is a checklist of work that would enable a successful 1.2.0 release of your adapter.

• migrate necessary cross-db macros into adapter and ensure they're tested accordingly
• remove any copy-and-pasted materialization (if your adapter inherits from another adapter)
• add new basic tests BaseDocsGenerate and BaseDocsGenReferences
• consider checking and testing support for Python 3.10

dbt-labs/dbt-core#5432 might make it into the second release cut in the next week, in which case, you'll also might want to:

• implement method and tests for connection retry logic

in order to more easily browse the QUERY_TEXT column of catalog."query_history", it would be better to move the dbt-generated query_comment`to the end of the SQL string, or perhaps even remove it.

[x] Update __version.py__
[ ] Add profile_template.yml as per https://docs.getdbt.com/docs/contributing/building-a-new-adapter#other-files
[ ] Reorganize file structure

It seems I have to eat my words ("let's see what happens!") when I approved #20. Pip now uses setup.cfg to install instead of setup.py. The error message from @octavianzarzu30 shows this snippet is causing pip to fail to install the pacakage:

(dbt_firebolt) octavianzarzu@Octavians-MBP dbt-firebolt % pip install git+https://github.com/firebolt-db/dbt-firebolt.git
Collecting git+https://github.com/firebolt-db/dbt-firebolt.git
  Cloning https://github.com/firebolt-db/dbt-firebolt.git to /private/var/folders/6l/_2bfz8wj36l2_bznx7_j07680000gn/T/pip-req-build-jn0imu3c
  Running command git clone -q https://github.com/firebolt-db/dbt-firebolt.git /private/var/folders/6l/_2bfz8wj36l2_bznx7_j07680000gn/T/pip-req-build-jn0imu3c
  Resolved https://github.com/firebolt-db/dbt-firebolt.git to commit f8c36f7797974ac46143cba059a45caa59d75ff0
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/octavianzarzu/opt/anaconda3/envs/dbt_firebolt/bin/python /Users/octavianzarzu/opt/anaconda3/envs/dbt_firebolt/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /var/folders/6l/_2bfz8wj36l2_bznx7_j07680000gn/T/tmp3kjyrrn_
       cwd: /private/var/folders/6l/_2bfz8wj36l2_bznx7_j07680000gn/T/pip-req-build-jn0imu3c
  Complete output (2 lines):
  running egg_info
  error: error in 'egg_base' option: 'src' does not exist or is not a directory
  ----------------------------------------
WARNING: Discarding git+https://github.com/firebolt-db/dbt-firebolt.git. Command errored out with exit status 1: /Users/octavianzarzu/opt/anaconda3/envs/dbt_firebolt/bin/python /Users/octavianzarzu/opt/anaconda3/envs/dbt_firebolt/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /var/folders/6l/_2bfz8wj36l2_bznx7_j07680000gn/T/tmp3kjyrrn_ Check the logs for full command output.
ERROR: Command errored out with exit status 1: /Users/octavianzarzu/opt/anaconda3/envs/dbt_firebolt/bin/python /Users/octavianzarzu/opt/anaconda3/envs/dbt_firebolt/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /var/folders/6l/_2bfz8wj36l2_bznx7_j07680000gn/T/tmp3kjyrrn_ Check the logs for full command output.

Background

The latest release cut for 1.3.0, dbt-core==1.3.0rc2 was published on October 3, 2022 (PyPI | Github). We are targeting releasing the official cut of 1.3.0 in time for the week of October 16 (in time for Coalesce conference).

We're trying to establish a following precedent w.r.t. minor versions:
Partner adapter maintainers release their adapter's minor version within four weeks of the initial RC being released. Given the delay on our side in notifying you, we'd like to set a target date of November 7 (four weeks from today) for maintainers to release their minor version

How to upgrade

dbt-labs/dbt-core#6011 is an open discussion with more detailed information, and dbt-labs/dbt-core#6040 is for keeping track of the community's progress on releasing 1.2.0

Below is a checklist of work that would enable a successful 1.2.0 release of your adapter.

• Python Models (if applicable)
• Incremental Materialization: cleanup and standardization
• More functional adapter tests to inherit

#20 adds a setup.cfg to our project even though we already have a setup.py. This will only cause problems for us by not having a single source of truth for package configuration. This should be addressed at somepoint

Might be wrong here, but I'm fairly certain we don't need this file. We're currently using setup.py instead of a .cfg. relevant SO question?

Originally posted by @swanderz in #20 (comment)

test

We've just published the release cut of dbt-core 1.2.0, dbt-core 1.2.0rc1 (PyPI | GitHub release notes).

dbt-labs/dbt-core#5468 is an open discussion with more detailed information, and dbt-labs/dbt-core#5474 is for keeping track of the communities progress on releasing 1.2.0

Below is a checklist of work that would enable a successful 1.2.0 release of your adapter.

• migrate necessary cross-db macros into adapter and ensure they're tested accordingly
• remove any copy-and-pasted materialization (if your adapter inherits from another adapter)
• add new basic tests BaseDocsGenerate and BaseDocsGenReferences
• consider checking and testing support for Python 3.10

dbt-labs/dbt-core#5432 might make it into the second release cut in the next week, in which case, you'll also might want to:

• implement method and tests for connection retry logic

Background

The latest release cut for 1.3.0, dbt-core==1.3.0rc2 was published on October 3, 2022 (PyPI | Github). We are targeting releasing the official cut of 1.3.0 in time for the week of October 16 (in time for Coalesce conference).

We're trying to establish a following precedent w.r.t. minor versions:
Partner adapter maintainers release their adapter's minor version within four weeks of the initial RC being released. Given the delay on our side in notifying you, we'd like to set a target date of November 7 (four weeks from today) for maintainers to release their minor version

How to upgrade

dbt-labs/dbt-core#6011 is an open discussion with more detailed information, and dbt-labs/dbt-core#6040 is for keeping track of the community's progress on releasing 1.2.0

Below is a checklist of work that would enable a successful 1.2.0 release of your adapter.

• Python Models (if applicable)
• Incremental Materialization: cleanup and standardization
• More functional adapter tests to inherit

We've just published the release cut of dbt-core 1.2.0, dbt-core 1.2.0rc1 (PyPI | GitHub release notes).

dbt-labs/dbt-core#5468 is an open discussion with more detailed information, and dbt-labs/dbt-core#5474 is for keeping track of the communities progress on releasing 1.2.0

Below is a checklist of work that would enable a successful 1.2.0 release of your adapter.

• migrate necessary cross-db macros into adapter and ensure they're tested accordingly
• remove any copy-and-pasted materialization (if your adapter inherits from another adapter)
• add new basic tests BaseDocsGenerate and BaseDocsGenReferences
• consider checking and testing support for Python 3.10

dbt-labs/dbt-core#5432 might make it into the second release cut in the next week, in which case, you'll also might want to:

• implement method and tests for connection retry logic

Background

The latest version of dbt Core,dbt-core==1.4.0, was published on January 25, 2023 (PyPI | Github). In fact, a patch, dbt-core==1.4.1 (PyPI | Github), was also released on the same day.

How to upgrade

dbt-labs/dbt-core#6624 is an open discussion with more detailed information. If you have questions, please put them there! dbt-labs/dbt-core#6849 is for keeping track of the community's progress on releasing 1.4.0

The above linked guide has more information, but below is a high-level checklist of work that would enable a successful 1.4.0 release of your adapter.

• support Python 3.11 (only if your adapter's dependencies allow)
• Consolidate timestamp functions & macros
• Replace deprecated exception functions
• Add support for more tests

the next minor release: 1.5.0

FYI, dbt-core==1.5.0 is expected to be released at the end of April. Please plan on allocating a more effort to upgrade support compared to previous minor versions. Expect to hear more in the middle of April.

At a high-level expect much greater adapter test coverage (a very good thing!), and some likely heaving renaming and restructuring as the API-ification of dbt-core is now well underway. See https://github.com/dbt-labs/dbt-core/milestone/82 for more information.

Background

Minor version v1.7 is targeted for final release on Ocotber 26, 2023. As a maintainer of a dbt adapter, we strongly encourage you to release a corresponding minor version increment to ensure users of your adapter can make use of this new minor version.

How to upgrade

dbt-labs/dbt-core#8307 is an open discussion with more detailed information. If you have questions, please put them there!

see dbt-labs/dbt-core#4282

Background

The latest version of dbt Core,dbt-core==1.5.0rc1, was published on April 13, 2023 (PyPI | Github).

How to upgrade

dbt-labs/dbt-core#7213 is an open discussion with more detailed information. If you have questions, please put them there!

The above linked guide has more information, but below is a high-level checklist of work that would enable a successful 1.5.0 release of your adapter.

• Add support Python 3.11 (if you haven't already)
• Add support for relevant tests (there's a lot of new ones!)
• Add support model contracts
• Add support for materialized views (this likely will be bumped to 1.6.0)

the next minor release: 1.6.0

FYI, dbt-core==1.6.0 is expected to be released at the end of July, with a release cut at least two weeks prior.

related to: dbt-labs/dbt-core#4187 and dbt-labs/dbt-core#3835

Using the standard two models that dbt comes prepackaged.

The first run ends successfully.
The second run ends with an error:

Completed with 1 error and 0 warnings:

Compilation Error in model my_first_dbt_model (models/example/my_first_dbt_model.sql)
  When searching for a relation, dbt found an approximate match. Instead of guessing 
  which relation to use, dbt will move on. Please delete "oz_my_first_dbt_model", or rename it to be less ambiguous.
  Searched for: oz_my_first_dbt_model
  Found: "oz_my_first_dbt_model"
  
  > in macro materialization_table_firebolt (macros/materializations/table.sql)
  > called by model my_first_dbt_model (models/example/my_first_dbt_model.sql)

Done. PASS=0 WARN=0 ERROR=1 SKIP=1 TOTAL=2

Shouldn't the model be recreated? (drop & create)