ovyerus / bandsnatch

A CLI batch downloader for your Bandcamp collection.

big nerd

bandsnatch's Issues

Error when running

I have downloaded to Linux Mint 21.2, I have made the file executable, created a cookies.json file containing the bandcamp.com cookie and I am using the following command:

./bandsnatch run -f flac <username>

After running that command I get:

thread 'main' panicked at 'Failed to deserialise collection page data blob.: Error("invalid type: null, expected u16", line: 1, column: 68813)', src/api/mod.rs:76:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Please can you help?

Outdated `html5ever` version required by `soup` produces future-incompatibilities warning

Just as an heads-up. Building with Rust-1.72.0 spits-out the following warning:

   Compiling bandsnatch v0.3.0 (/usr/pkgsrc/wip/bandsnatch/work/bandsnatch-0.3.0)
warning: use of deprecated method `chrono::TimeZone::datetime_from_str`: use `DateTime::parse_from_str` instead
  --> src/api/structs/digital_item.rs:50:34
   |
50 |             Some(d) => match Utc.datetime_from_str(d, FORMAT) {
   |                                  ^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(deprecated)]` on by default

warning: function `command` is never used
  --> src/cmds/release.rs:18:14
   |
18 | pub async fn command(_args: Args) -> Result<(), Box<dyn std::error::Error>> {
   |              ^^^^^^^
   |
   = note: `#[warn(dead_code)]` on by default

warning: `bandsnatch` (bin "bandsnatch") generated 2 warnings
    Finished release [optimized] target(s) in 9m 01s
warning: the following packages contain code that will be rejected by a future version of Rust: html5ever v0.22.5
note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`

Auto detect user to pull releases for

The identities.fan properties on the pagedata blob can probably be used to reliably automatically pull the user's name/page url without needing to be provided via --user. This can be retrieved from the homepage.

Replace invalid folder names.

Currently Bandsnatch has an error when it comes across a release with characters in its name which results in the release being skipped.
We should either have a map of replacement characters to use, which are visually similar, or just remove them straight up.

Disallowed . or space at end of directory/filename

I was moving my collection to an NTFS partition, and then I ran into the limitation that Windows doesn't allow files/folders to end in a space or a period (.), quick fix would be to add a legal character like - or _, that way you'd know it was slightly altered to work Windows.

Better logging

Currently logging kinda sucks big time. Here's a list of some improvements

• Show information about files/account being used right at the start like bandcamp-collection-downloader does.
  • Format
  • Username
  • Cookies path
  • Output path
  • etc.
• Remove wonky debug logs that screw with the progress.
• Use the log stuff we have already for these.
• Have another progress bar/line for progress through the user's collection

Strange message is output during downloading

Thank you for making great software. This seems to be working for me.

However, a strange message appeared, so I thought I would report it here.
Cookie.json was exported from Cookie Quick Manager.
Here is the command I used. I ran it on macOS 13.2.

bandsnatch -c cookies.json -f flac MY_NAME

thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: Error("missing field `download_type`", line: 1, column: 3752)', src/api/mod.rs:188:91
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: bandsnatch::main::{{closure}}::{{closure}}::{{closure}}
   4: tokio::runtime::task::harness::Harness<T,S>::poll
   5: std::thread::local::LocalKey<T>::with
   6: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
   7: tokio::runtime::scheduler::multi_thread::worker::Context::run
   8: tokio::macros::scoped_tls::ScopedKey<T>::set
   9: tokio::runtime::scheduler::multi_thread::worker::run
  10: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
  11: tokio::runtime::task::harness::Harness<T,S>::poll
  12: tokio::runtime::blocking::pool::Inner::run
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

I get this message, but the download seems to be working correctly.
Therefore, there may be no harm done, but I reported it just in case.

Again, your software is very useful. Thank you for making it.

Previous fix was good, but not perfect

As mentioned in a previous issue, I was stuck at 104 releases of ~950, now it's down to 13, so great improvement

-> https://deadsara.bandcamp.com/album/self-titled
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("data did not match any variant of untagged enum StringOrNumber", line: 1, column: 2246)', ~/.cargo/registry/src/github.com-1ecc6
299db9ec823/bandsnatch-0.2.0/src/api/mod.rs:194:74

-> https://stuntrock.bandcamp.com/album/were-not-assholes-were-just-smarter-than-you-split-release-with-tuyo
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 1, column: 3492)', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/bandsnatch-0.2.0/src/api/mod.rs:194:74

-> https://stuntrock.bandcamp.com/album/fuck-canada-fuck-america-split-release-with-venetian-snares
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 1, column: 3464)', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/bandsnatch-0.2.0/src/api/mod.rs:194:74

-> https://stuntrock.bandcamp.com/album/this-is-stunt-rock-volume-one-in-memory-of-bob-seger
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 1, column: 3443)', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/bandsnatch-0.2.0/src/api/mod.rs:194:74

-> https://remute.bandcamp.com/album/technoptimistic
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 1, column: 1879)', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/bandsnatch-0.2.0/src/api/mod.rs:194:74

-> https://remute.bandcamp.com/album/red-eyes-ep
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 1, column: 1862)', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/bandsnatch-0.
2.0/src/api/mod.rs:194:74

-> https://remute.bandcamp.com/album/the-cult-of-remute
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 1, column: 1892)', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/bandsnatch-0.2.0/src/api/mod.rs:194:74

-> https://remute.bandcamp.com/album/electronic-lifestyle
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 1, column: 1899)', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/bandsnatch-0.2.0/src/api/mod.rs:194:74

-> https://stuntrock.bandcamp.com/album/this-is-stunt-rock-volume-two-in-memory-of-tray-loren
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 1, column: 3442)', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/bandsnatch-0.2.0/src/api/mod.rs:194:74

-> https://65daysofstatic.bandcamp.com/album/utopian-frequencies-disquiet-vinyl-bundle
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 1, column: 4133)', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/bandsnatch-0.2.0/src/api/mod.rs:194:74

-> https://remute.bandcamp.com/album/theme-tunes-for-10-games-never-made
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 1, column: 1958)', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/bandsnatch-0.2.0/src/api/mod.rs:194:74

-> https://redrumrecordz.bandcamp.com/album/threnody-bewildering-thoughts-1995
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 1, column: 4210)', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/bandsnatch-0.2.0/src/api/mod.rs:194:74

-> https://remute.bandcamp.com/album/the-sound-of-money-radio-edit
thread 'tokio-runtime-worker' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: null, expected a string", line: 1, column: 1936)', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/bandsnatch-0.2.0/src/api/mod.rs:194:74

Because the JSON isn't anonymized, I'm not posting it here, the locations seem to be off too, so it's hard to pin point the exact offending key value pair. I apparently have a bandcamp key left for a few of the remute's, so there is a way to test it.

Cookie parsing error `called `Result::unwrap()` on an `Err` value: RelativeUrlWithoutBase`

Hi,

bandsnatch currently isn't parsing my cookies.txt file correctly:

.bandcamp.com	TRUE	/	TRUE	2028157437	client_id	XXXXXXXXXXXXXXXXXXXXXXXXX

 $ RUST_BACKTRACE=1 ./target/debug/bandsnatch run --format mp3-320 mjsir911 -c cookies.txt
thread 'main' panicked at src/cookies.rs:89:43:
called `Result::unwrap()` on an `Err` value: RelativeUrlWithoutBase
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: core::result::Result<T,E>::unwrap
             at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/core/src/result.rs:1073:23
   4: bandsnatch::cookies::fill_cookie_jar
             at ./src/cookies.rs:89:20
   5: bandsnatch::api::Api::new
             at ./src/api/mod.rs:37:26
   6: bandsnatch::cmds::run::command
             at ./src/cmds/run.rs:115:24
   7: bandsnatch::main
             at ./src/main.rs:40:36
   8: core::ops::function::FnOnce::call_once
             at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Debugging subcommands

Bandsnatch should have a bunch of different subcommands which grab different information from Bandcamp and displays it as JSON for debugging/testing purposes.