Comments (2)
Ah, sorry, but I think Fugue side has finished the change to depend on dask-sql when the backend is dask, I will make a pr to dask-sql to remove the unnecessary part.
from fugue.
Hey, I'm also affected by this bug, using Spark backend.
Some relevant requirements:
adtk==0.6.2
pyod==1.0.1
numpy==1.22.3
fugue==0.8.3
My application suddenly stopped working from one deployment to another. It seems like the setup.py of fugue pins down versions in a rather risky way:
"triad>=0.9.3",
"adagio>=0.2.4",
# sql dependencies
"qpd>=0.4.4",
"fugue-sql-antlr>=0.1.6",
So I'm guessing triad
was updated, which caused my pinned down version of fugue
to break. I see two long term solutions to this problem:
- Change the pinning strategy of the fugue project, to something like this:
"triad<=0.9.3",
"adagio<=0.2.4",
# sql dependencies
"qpd<=0.4.4",
"fugue-sql-antlr<=0.1.6",
This would ensure sub-dependencies are only updated with a specific version update on fugue.
- Add a disclaimer in the installation steps that proper requirement locking is required from end-users, by using something like
pipenv
. Currently documentation recommends installation bypip
which is a ticking bomb: https://github.com/fugue-project/fugue#installation
from fugue.
Related Issues (20)
- [QUESTION] Add ClickHouse as backend HOT 1
- [QUESTION] Unable to run Ray example from documentation HOT 5
- [QUESTION] Statsforecast + Ray + JupyterHub HOT 7
- [FEATURE] Use `_collect_as_arrow` for `fugue_api.as_arrow(spark_df)` HOT 3
- [BUG] Unable to use spark backend on High Concurrency Databricks Clusters HOT 4
- [BUG] HOT 5
- [FEATURE] `as_dicts` for DataFrames
- [BUG] Lambda function for passing data frame to the setup function in pycaret doesn't work HOT 1
- [COMPATIBILITY] Remove ibis version cap
- Apache Beam Dataframe and SQL support HOT 3
- Unable to display transformed output in pyspark HOT 1
- [COMPATIBILITY] Fix compatibility issues with Dask 2024.4+
- [COMPATIBILITY] Fix Ray 2.10+ compatibility issues.
- [COMPATIBILITY] Change table temp view names to uppercase
- [FEATURE] Support type hinting with standard collections
- [BUG] Spark connect import issue on worker side
- [COMPATIBILITY] Support Python 3.12
- [COMPATIBILITY] Deprecate Python 3.8 support
- [FEATURE] Support single `dict[str,Any]` as transformer input and output
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fugue.