fsspec / community Goto Github PK
View Code? Open in Web Editor NEWDiscussion space for conversations across multiple fsspec repos
Discussion space for conversations across multiple fsspec repos
We have lots of inconsistencies across fsspec projects (ci, styling, linters, supported python versions, etc), which make it a pain to work with multiple projects at the same time.
We should create one recommended project template, put all recommended stuff into it and use cruft to keep the created projects up-to-date automatically. E.g. the template will contain a skeleton for new fs implementation, so automatic PRs would be a reminder to create new methods, as well as will try to add or remote particular python version support (though that could be handled by just creating an ABC for *FileSystem).
We use a similar approach in dvc plugins: https://github.com/iterative/cookiecutter-dvc-plugin
Should we set up a monthly get-together to keep people tied in to progress? We could publish notes or maintain a roadmap.
It seems like we currently use plain merging everywhere, which makes for a very messy commit history, where it is very common to have commits that break the test suit. Should we switch to squash-merging (the most hustle free, we get 1 commit in history, but all the granularity in the PR and don't have to enforce any practices) or rebasing (requires a lot of discipline and enforcement for contributors, but makes for the nicest linear history)? Also, should we adopt a common practice or let every repo decide? WDYT?
In addition, we should consider who should hold rights to pypi, conda-forge and RTD.
Originally posted by @martindurant in #3 (comment)
Currently, there is only one owner https://github.com/orgs/fsspec/people , which means that if @martindurant for some reason becomes unavailable, no one else will be able to do org-level actions (e.g. inviting people, changing teams, granting permissions to repositories, adopting new repos that are being transfered from elsewhere, etc), which is a problem. I think we should have at least one more owner for everyone's security.
From fsspec/sshfs#13
@efiop & @martindurant, I'm interested in having a CLI utility that brings the power of fsspec to an interface similar to curl, wget or rsync.
Have you guys thought about anything like this? Would you be interested in having something like this be a part of the fsspec community of repos? I'm open to having it be completely separate to start but if it's something you guys know already exists and I'm just not finding it, I'd rather not write yet another python package. Thanks.
We currently have a super unofficial logo: https://avatars.githubusercontent.com/u/92825505?s=96&v=4 which is not much used anywhere.
I asked Anaconda marketing to come up with something, and the proposed this:
Thoughts?
RTD is reporting the following for s3fs and gcsfs:
/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/s3fs-2021.10.1+10.g0ac0362-py3.7.egg/s3fs/core.py:docstring of s3fs.core.S3File:72: WARNING: toctree references unknown document 's3fs.core.S3File.writelines'
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/sphinx/cmd/build.py", line 304, in build_main
app.build(args.force_all, filenames)
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/sphinx/application.py", line 341, in build
self.builder.build_update()
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 347, in build_update
len(to_build))
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 360, in build
updated_docnames = set(self.read())
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 468, in read
self._read_serial(docnames)
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 490, in _read_serial
self.read_doc(docname)
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 534, in read_doc
doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/sphinx/io.py", line 318, in read_doc
pub.publish()
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/docutils/core.py", line 219, in publish
self.apply_transforms()
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/docutils/core.py", line 200, in apply_transforms
self.document.transformer.apply_transforms()
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/sphinx/transforms/__init__.py", line 90, in apply_transforms
Transformer.apply_transforms(self)
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
transform.apply(**kwargs)
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/sphinx/transforms/__init__.py", line 245, in apply
apply_source_workaround(n)
File "/home/docs/checkouts/readthedocs.org/user_builds/s3fs/envs/latest/lib/python3.7/site-packages/sphinx/util/nodes.py", line 94, in apply_source_workaround
for classifier in reversed(node.parent.traverse(nodes.classifier)):
TypeError: 'generator' object is not reversible
Do we need to update something?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.