support full twitter archive format about forget

[codl]

?? i just requested a twitter archive and it still has /data/js/tweets. no big jsonp file at the root either. maybe this is some kinda A/B test?

@rrix can you provide an example archive? or more details about the format of the file & its filename?

[rrix]

Hey sorry about the delay, I finally found a browser session that was logged in to Github.

The file is a "little" large, I'll upload a version of it with media files dropped out of it to my nextcloud and DM you a link to it through the fediverse.

[codl]

ok i get it now. it's not a new format, it's a different archive. it's the full account archive you get from https://twitter.com/settings/your_twitter_data instead of the tweet archive you get from https://twitter.com/settings/account#tweet_export

it could be supported but it would require extracting the zip in the browser and parsing it there, cos we cant reasonably upload gigabytes of images and videos just to get a few thousand tweet IDs. I'm not going to do that. but if someone's up to the task I'd be happy to help and to merge it in

what i will do is document it, link to the right page, and pop a warning before uploading if the archive is more than, say, 25 MB. mine reaches 6.3 MB with 30k ish tweets so i figure that's a safe value

[codl]

it sure did take me a whole month to figure that out huh 😅

[rrix]

Oh wow, I had no idea there were two different things there. I'll upload the archive you're expecting, thanks for the pointer and investigating!

[codl]

the old "tweets archive" format has apparently been phased out. support for the new archive format is now essential. reopening

[codl]

so, not only is the new format huge and inconvenient to upload, it also has a lot of personal data that i would rather never came even close to my server

my current plan is:

1. let users upload the tweet.js file from within that archive (like #119 suggested)
2. if the user selects a full zip archive, unzip it in browser and only upload tweet.js

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[codl]

Hi all. Sorry for taking so long. I intend to get this done by the end of the month.