greydanus / mnist1d Goto Github PK

Congrats on this dataset being prominently used in the Prince's ML textbook: https://udlbook.github.io/udlbook/ !

It made me wonder: was your paper published anywhere? It seems like it wasn't; but have you tried to publish it? I think it's really cool, so I am wondering if it kept getting rejected or if you haven't really tried to publish it.

This worked last week, but today when I try to import mnist1d in Colab after running !git clone https://github.com/greydanus/mnist1d I get no errors, but a yellow underline under import mnist1d that says:

Import "mnist1d" could not be resolved(reportMissingImports) in Colab

Then:
args = mnist1d.data.get_dataset_args() fails with AttributeError: module 'mnist1d' has no attribute 'data'

The default dataset seems to be broken. Not a big deal since it just automatically rebuilds it, but I though I should bring it to your attention.

from mnist1d.data import get_dataset_args, get_dataset
args = get_dataset_args()
data = get_dataset(args,  path='./mnist1d_data.pkl', download=True)
-> 
Downloading MNIST1D dataset from https://github.com/greydanus/mnist1d/raw/master/mnist1d_data.pkl   
Saving to ./mnist1d_data.pkl
Did or could not load data from ./mnist1d_data.pkl. Rebuilding dataset...

minimal repo: https://colab.research.google.com/drive/16RBVCMLXegLZ6xidI3ZBe_visT_gR5EA?usp=sharing

Hi, thanks for contributing this dataset as open source. Motivated by "Understanding Deep Learning", I'd love to start playing with it.

I saw that the repo is not ready for packaging. As the code is rather straight forward, I was wondering if any PRs would be appreciated to package this as a python module.

This would also make it more easier to use more broadly than by using the requests package. People could just go: pip install mnist1d and have the code for augmenting and changing under their fingers. Just a thought!

Right now, I get the following:

python -m pip install git+https://github.com/greydanus/mnist1d.git@master

Collecting git+https://github.com/greydanus/mnist1d.git@master
  Cloning https://github.com/greydanus/mnist1d.git (to revision master) to /tmp/pip-req-build-eg1a_6cu
  Running command git clone --filter=blob:none --quiet https://github.com/greydanus/mnist1d.git /tmp/pip-req-build-eg1a_6cu
  Resolved https://github.com/greydanus/mnist1d.git to commit 39dd6c03785eefe60f349af94e61f864fc449644
ERROR: git+https://github.com/greydanus/mnist1d.git@master does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

Hey thank you for this, but pickled objects in Python can be used to invoke remote code execution. It means I have to trust you in order to use this data. I'd be grateful if you'd consider distributing this data in some other way?

with #9 being merged, I suggest we work towards pushing this repo to pypi