writes millions of temp files, never cleans up about tardis-python

[thaaddeus]

Hi, yes right now, there's no cleanup function, so you'd have to remove/clear cache as needed.
Also in general it's much more efficient to request data for multiple symbols at once, vs one-by-one (it will also create less files).

[manos]

Thanks for the reply @thaaddeus! The issue with fetching multiple symbols is that if one doesn't have data at the --start date, the API doesn't just return what it has, it throws an error.
I could sort by that, e.g. bucket everything together that has data for my desired 60-day range (for example), and fetch those together.

[thaaddeus]

Actually it should still return what is available at a given day (CSV downloads would fail indeed). Can you share the sample payload that would case this issue for you by any chance? I'll investigate.

[manos]

Oh indeed, you're right - I was thinking of when I was calling the API directly. I've restructured as you recommended and fetched 60 days of all FTX PERP symbols in a few hours. InfluxDB is happily full of data :)
Much better; thanks for the help!