conda-forge / flytekit-feedstock

A conda-smithy repository for flytekit.

About flytekit-feedstock

Feedstock license: BSD-3-Clause

Home: https://github.com/flyteorg/flytekit

Package license: Apache-2.0

Summary: Flyte SDK for Python

Development: https://github.com/flyteorg/flytekit

Documentation: https://flytecookbook.readthedocs.io/en/latest/

Flytekit Python is the Python Library for easily authoring, testing, deploying, and interacting with Flyte tasks, workflows, and launch plans.

• Flyte HomePage: https://flyte.org
• Flyte Docs: http://docs.flyte.org/

PyPI: https://pypi.org/project/flytekit/

Current build status

All platforms:

Current release info

Name	Downloads	Version	Platforms

Installing flytekit

Installing flytekit from the conda-forge channel can be achieved by adding conda-forge to your channels with:

conda config --add channels conda-forge
conda config --set channel_priority strict

Once the conda-forge channel has been enabled, flytekit can be installed with conda:

conda install flytekit

or with mamba:

mamba install flytekit

It is possible to list all of the versions of flytekit available on your platform with conda:

conda search flytekit --channel conda-forge

or with mamba:

mamba search flytekit --channel conda-forge

Alternatively, mamba repoquery may provide more information:

# Search all versions available on your platform:
mamba repoquery search flytekit --channel conda-forge

# List packages depending on `flytekit`:
mamba repoquery whoneeds flytekit --channel conda-forge

# List dependencies of `flytekit`:
mamba repoquery depends flytekit --channel conda-forge

About conda-forge

conda-forge is a community-led conda channel of installable packages. In order to provide high-quality builds, the process has been automated into the conda-forge GitHub organization. The conda-forge organization contains one repository for each of the installable packages. Such a repository is known as a feedstock.

A feedstock is made up of a conda recipe (the instructions on what and how to build the package) and the necessary configurations for automatic building using freely available continuous integration services. Thanks to the awesome service provided by Azure, GitHub, CircleCI, AppVeyor, Drone, and TravisCI it is possible to build and upload installable packages to the conda-forge anaconda.org channel for Linux, Windows and OSX respectively.

To manage the continuous integration and simplify feedstock maintenance conda-smithy has been developed. Using the conda-forge.yml within this repository, it is possible to re-render all of this feedstock's supporting files (e.g. the CI configuration files) with conda smithy rerender.

For more information please check the conda-forge documentation.

Terminology

feedstock - the conda recipe (raw material), supporting scripts and CI configuration.

conda-smithy - the tool which helps orchestrate the feedstock. Its primary use is in the construction of the CI .yml files and simplify the management of many feedstocks.

conda-forge - the place where the feedstock and smithy live and work to produce the finished article (built conda distributions)

Updating flytekit-feedstock

If you would like to improve the flytekit recipe or build a new package version, please fork this repository and submit a PR. Upon submission, your changes will be run on the appropriate platforms to give the reviewer an opportunity to confirm that the changes result in a successful build. Once merged, the recipe will be re-built and uploaded automatically to the conda-forge channel, whereupon the built conda packages will be available for everybody to install and use from the conda-forge channel. Note that all branches in the conda-forge/flytekit-feedstock are immediately built and any created packages are uploaded, so PRs should be based on branches in forks and branches in the main repository should only be used to build distinct package versions.

In order to produce a uniquely identifiable distribution:

• If the version of a package is not being increased, please add or increase the build/number.
• If the version of a package is being increased, please remember to return the build/number back to 0.

Feedstock Maintainers

• @asford
• @bmcconeghy
• @bstadlbauer
• @cosmicBboy
• @eapolinario
• @sugatoray

flytekit-feedstock's Issues

@conda-forge-admin, please add user @eapolinario

Additional comment:

@eapolinario is a maintainer of flytekit and would love to also get involved with this feedstock.

@conda-forge-admin,please add bot automerge

Additional comment:

No response

@conda-forge-admin, please update team

@conda-forge-admin, please update team

Appears that team wasn't updated on repo creation.

https://conda-forge.org/docs/maintainer/infrastructure.html#conda-forge-admin-please-update-team

@conda-forge-admin, please add user @bstadlbauer

Additional comment:

No response

Builds currently failing

Looks like the Azure CI platform is down (see: conda-forge/status#125), so current build failed (and therefore didn't update on anaconda).

• I read the conda-forge documentation and could not find the solution for my problem there.

Issue:

Environment (conda list):

$ conda list

Details about conda and system ( conda info ):

$ conda info

Build from .tar.gz sources once flytekit sourc includes package data

Reason for Build Failure: 1/N

There is a file, template.html that is not found in the pypi source. This needs fixing the upstream repository by:

• adding this template.html file to MANIFEST.in if somehow it is unavailable in the pypi source
• fixing jinja2 template loading with autoescaping

See: flyteorg/flytekit#1088

• https://github.com/sugatoray/flytekit/blob/fix_jinja_template/flytekit/deck/html/template.html

1102 jinja2.exceptions.TemplateNotFound: template.html

(error)

Some of the places in the code where template.html is called from.

• jinja2/loaders.py:

  raise TemplateNotFound(template)

• errors originating from flytekit:

  import flytekit
  
  # flytekit/__init__.py", line 164, in <module>
  from flytekit.core.base_sql_task import SQLTask
  
  # flytekit/core/base_sql_task.py", line 4, in <module>
  from flytekit.core.base_task import PythonTask, TaskMetadata
  
  # flytekit/core/base_task.py", line 42, in <module>
  from flytekit.deck.deck import Deck
  
  # flytekit/deck/__init__.py", line 1, in <module>
  from .deck import Deck
  
  # flytekit/deck/deck.py", line 115, in <module>
  template = env.get_template("template.html")

See Error Details | Time Elasped: 3:57 s

import: 'flytekit'
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/flytekit_1656731293461/test_tmp/run_test.py", line 2, in <module>
    import flytekit
  File "/home/conda/feedstock_root/build_artifacts/flytekit_1656731293461/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.10/site-packages/flytekit/__init__.py", line 164, in <module>
    from flytekit.core.base_sql_task import SQLTask
  File "/home/conda/feedstock_root/build_artifacts/flytekit_1656731293461/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.10/site-packages/flytekit/core/base_sql_task.py", line 4, in <module>
    from flytekit.core.base_task import PythonTask, TaskMetadata
  File "/home/conda/feedstock_root/build_artifacts/flytekit_1656731293461/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.10/site-packages/flytekit/core/base_task.py", line 42, in <module>
    from flytekit.deck.deck import Deck
  File "/home/conda/feedstock_root/build_artifacts/flytekit_1656731293461/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.10/site-packages/flytekit/deck/__init__.py", line 1, in <module>
    from .deck import Deck
  File "/home/conda/feedstock_root/build_artifacts/flytekit_1656731293461/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.10/site-packages/flytekit/deck/deck.py", line 115, in <module>
    template = env.get_template("template.html")
  File "/home/conda/feedstock_root/build_artifacts/flytekit_1656731293461/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.10/site-packages/jinja2/environment.py", line 1010, in get_template
    return self._load_template(name, globals)
  File "/home/conda/feedstock_root/build_artifacts/flytekit_1656731293461/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.10/site-packages/jinja2/environment.py", line 969, in _load_template
    template = self.loader.load(self, name, self.make_globals(globals))
  File "/home/conda/feedstock_root/build_artifacts/flytekit_1656731293461/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.10/site-packages/jinja2/loaders.py", line 126, in load
    source, filename, uptodate = self.get_source(environment, name)
  File "/home/conda/feedstock_root/build_artifacts/flytekit_1656731293461/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.10/site-packages/jinja2/loaders.py", line 218, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: template.html
Tests failed for flytekit-1.1.0-pyhd8ed1ab_0.tar.bz2 - moving package to /home/conda/feedstock_root/build_artifacts/broken
WARNING:conda_build.build:Tests failed for flytekit-1.1.0-pyhd8ed1ab_0.tar.bz2 - moving package to /home/conda/feedstock_root/build_artifacts/broken
TESTS FAILED: flytekit-1.1.0-pyhd8ed1ab_0.tar.bz2

Originally posted by @sugatoray in #14 (comment)

flyekit requires importlib-metadata>=4.0

• I read the conda-forge documentation and could not find the solution for my problem there.

Issue:

Upstream package does not specify importlib-metadata as a pinned requirements, however features in the 4.0 release series are required for the standard entrypoints:

Repro trace with importlib-metadata < 4.

This is a little confused, because it looks like importlib-metadata and importlib_metadata are both present in conda-forge.

% python mvp.py 
Traceback (most recent call last):
  File "mvp.py", line 1, in <module>
    from flytekit import task, workflow
  File "/Users/brianmcconeghy/abcd/.conda/lib/python3.8/site-packages/flytekit/__init__.py", line 245, in <module>
    load_implicit_plugins()
  File "/Users/brianmcconeghy/abcd/.conda/lib/python3.8/site-packages/flytekit/__init__.py", line 239, in load_implicit_plugins
    discovered_plugins = entry_points(group="flytekit.plugins")
TypeError: entry_points() got an unexpected keyword argument 'group'

@eapolinario is added to the list of maintainers but doesn't have write permissions

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

According to https://conda-forge.org/docs/maintainer/updating_pkgs.html#maint-updating-maintainers, in order to become a maintainer I'd need to open an issue like #39.

I show up in the recipe's maintainers list, but I dont' have permissions to approve/merge PRs (e.g. #49).

Installed packages

n/a

Environment info

n/a

@conda-forge-admin, please add user @cosmicBboy

Additional comment:

From #26