CSV import about django-todo

[shacker]

This is a very good idea. I like it. The CSV would have to be very rigidly formatted (unless we provided a mapping tool, which would be complex). Basically every field on the Task object in django-todo would need to be specified in the right order. And maybe a dropdown to let you select which list it should go into. The tool would be available to Staff users only (or would a command line tool be acceptable?). Other thoughts on the specific implementation?

Is this something you'd be willing to contribute?

[datatalking]

Yes, I have about 25 different values, will have to play with a main importing function then a hub people could connect to. Any aesthetic or technical requests for this?

[shacker]

There are only 10 fields on the Task model - where are the other 15 values coming from / going to? And what do you mean by "a hub people could connect to" ?

Technical requests: We'll need tests to ensure this is working properly. Let's start simple, with a required field order mirroring the order of fields currently shown in the Task model. No mapping tool for now. Thanks in advance!

[datatalking]

Ok lets start with the first ten, and go from there. I can write tests but I'll need to know the values we are testing for as I'm relatively new to writing tests.

I have another project and we could definitely add features to this in helping to support/contribute.
One challenge is we use unittests vs pytest, do the two play well together ?

[shacker]

pytest can officially run standard unit tests (it is backwards compatible) but pytest are so much more capable and easier to read. For consistency, it sure would be nice if you include them in pytest format. See the existing tests directory in this project for examples. Not sure what you mean by "what the values are" - you will get the CSV importer up and running, and importing a csv should result in new Tasks in the database, owned by the right people, assigned to the right people, in the right groups and lists, with valid due dates, etc. Your tests of course will feed mock data into the system and then check that tasks exist in the db reflecting the inputs (and of course also testing that bad input data results in no new tasks being created).

If you can get something off the ground, I can help,

[datatalking]

I was curious that I would be the first one to create an importer for data into a Django app and found that Django has built in JSON app. So this might be multiple functional uses; a format selectable, multi document importer to Django as a side app for the current built in JSON recognition?

[shacker]

@datatalking As far as I know, Django doesn't include any JSON-specific support, but you can of course call the JSON libs that are part of the Python standard library. It is not clear to me how your comment relates to the point of this issue, which is the ability to import tasks from CSV.

[datatalking]

I've been wanting to work on this but it's possibly out of my depth right now. The process looks like a django serialization model. https://docs.djangoproject.com/en/2.1/topics/serialization/

It looks like we could serialize the data into Json

[shacker]

I could probably build the feature in a day or less, just struggling for spare time. One wouldn't need JSON or serializers - Python has a built-in CSV reader. Just scoop it in and write out db records. The hard part is the tooling around it - defining the import format, and all of the small decisions to be made during import. Not hard, just need to jump into it.

[shacker]

FYI this feature is now under way and I'm making good progress. Should be done in a few days. Would you be willing to beta test it? (that offer is open to anyone).

[datatalking]

When you say test you mean try to run data through it? I’m open and have an application that has a few formats I could try.

…

Sent from my iPhone

On Mar 9, 2019, at 11:31 AM, Scot Hacker ***@***.***> wrote: FYI this feature is now under way and I'm making good progress. Should be done in a few days. Would you be willing to beta test it? (that offer is open to anyone). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[shacker]

Okey doke, all set for beta testing - just check out the csv branch of this repo, then see the new Importing Tasks via CSV section of the readme for usage notes. This provides both a web upload interface and a management command (./manage.py import_csv -f /path/to/file.csv). An example spreadsheet is provided in todo/data/import_example.csv.

Please provide any and all feedback - were you able to crash it? Did you get the tasks created/updated that you expected? Did you find any bugs? Thanks and looking forward to your feedback.

[shacker]

@datatalking I'd like to merge the pull request tomorrow and get this released. Have you had a chance to test? I built it for you! (j/k) If you don't have time, you can always file follow-on issues later.

[shacker]

Sorry, we never heard beta feedback. But we do have tests! The CSV import feature is now merged into master, and going to pypi shortly.