sintef-9012 / vesselai-dagster

vesselai_dagster

Welcome to your new Dagster repository.

Contents

Name	Description
README.md	A description and guide for this code repository
setup.py	A build script with Python package dependencies for this code repository
workspace.yaml	A file that specifies the location of the user code for Dagit and the Dagster CLI
vesselai_dagster/	A Python directory that contains code for your Dagster repository
vesselai_dagster_tests/	A Python directory that contains tests for vesselai_dagster

Getting up and running

1. Create a new Python environment and activate.

Pyenv

export PYTHON_VERSION=X.Y.Z
pyenv install $PYTHON_VERSION
pyenv virtualenv $PYTHON_VERSION vesselai_dagster
pyenv activate vesselai_dagster

Conda

export PYTHON_VERSION=X.Y.Z
conda create --name vesselai_dagster python=PYTHON_VERSION
conda activate vesselai_dagster

2. Once you have activated your Python environment, install your repository as a Python package. By using the --editable flag, pip will install your repository in "editable mode" so that as you develop, local code changes will automatically apply.

pip install --editable .

Local Development

1. Set the DAGSTER_HOME environment variable. Dagster will store run history in this directory.

mkdir ~/dagster_home
export DAGSTER_HOME=~/dagster_home

2. Start the Dagit process. This will start a Dagit web server that, by default, is served on http://localhost:3000.

dagit

3. (Optional) If you want to enable Dagster Schedules or Sensors for your jobs, start the Dagster Daemon process in a different shell or terminal:

dagster-daemon run

Local Testing

Tests can be found in vesselai_dagster_tests and are run with the following command:

pytest vesselai_dagster_tests

As you create Dagster ops and graphs, add tests in vesselai_dagster_tests/ to check that your code behaves as desired and does not break over time.

For hints on how to write tests for ops and graphs in Dagster, see our documentation tutorial on Testing.