wlerin / social48-ameblo Goto Github PK

The NMB48 Official JSON is 411 MB in size, and causes the entire system to hang once the script reaches it. The SKE48 one is not much better, at 209 MB. I might be able to solve this with a more efficient json parsing module (or one that could read/write json in pieces), but switching to a proper (relational or nosql) database would also provide many other benefits, not just on the ameblo side of things but also for the rest of the social48 project.

Add proper commandline arguments similar to other social48 scripts, or to showroom.

Encapsulate the update functionality ( rip_ameblo() ) and search functions in an object that can be imported into a generalised update script. Add print, encode, etc. functions similar to those in other social48 scripts.

In particular it needs to be threadable.