erasure encoded backends? about rclone

[ncw]

This method of splitting a file is dependent on everything working (all the cloud providers) to recover the data.

I'd probably recommend something more like this https://github.com/jesseduffield/horcrux where you don't need every copy of the data to reconstruct the original.

You could use that in a simple script - horcrux split then rclone copy for each part to the remote backends. The recovery script would do the opposite, rclone copy to retreive the parts and then horcrux bind to join them back together.

Ideally rclone would have a horcrux backend where you could say, split my data over these N providers in such a way that the data isn't recoverable unless you have M (<=N) parts.

There have been issues about erasure encoded backends which are a similar idea but not encrypted - so more like your current scheme but with redundancy.

PS I sync my keepass file to Google Drive then I can use it with KeepassXC on my computer and Keepasss2Android on my phone. The keepass file is protected with a very strong password, however Keepass2Android can store it in the secure enclave on the phone so you can unlock it with your finger print. All very convenient!

[lucasnuic]

Hi @ncw

This method of splitting a file is dependent on everything working (all the cloud providers) to recover the data.

yes. But that was the goal of my initial algorithm. In theory, if there are no providers or a lack of internet connection, the keepass password bank is useless. This makes sense when you want greater security, privacy and anonymity. In theory, no provider can know which n providers you shared the information with. It's like shamir, but used in reverse.

The bad thing is that even with this supposed security, privacy and certain anonymity... you still depend on suppliers, which is bad. Because without n suppliers you lose the file. It's exactly what you mention here.

I'd probably recommend something more like this https://github.com/jesseduffield/horcrux where you don't need every copy of the data to reconstruct the original.
You could use that in a simple script - horcrux split then rclone copy for each part to the remote backends. The recovery script would do the opposite, rclone copy to retreive the parts and then horcrux bind to join them back together.

yes. Initially I created an issue whether I could complement the horcrux with the idea above or not. In a second use case, horcrux could be useful for those who don't trust certain companies but still need them. In the case of password managers, I could split my data with n password managers: Bitvarden, 1password, Lastpass. That makes sense?

(My initial idea was to do encryption through division without there being a password or anything like that.)

Ideally rclone would have a horcrux backend where you could say, split my data over these N providers in such a way that the data isn't recoverable unless you have M (<=N) parts.

That's what I thought too and what you described perfectly here.

There have been issues about erasure encoded backends which are a similar idea but not encrypted - so more like your current scheme but with redundancy.

yes.

PS I sync my keepass file to Google Drive then I can use it with KeepassXC on my computer and Keepasss2Android on my phone. The keepass file is protected with a very strong password, however Keepass2Android can store it in the secure enclave on the phone so you can unlock it with your finger print. All very convenient!

Thank you for your kind words. 🙂

[lucasnuic]

@ncw I left the my code public, I don't intend to implement it for now for the good reasons you mentioned. But I learned a lot and I'm grateful for the opportunity to improve.

Edit: I had opened two issues to check if my algorithm could be added to the horcrux project as a reference or as part of the core. Look here: horcrux with rclone?, "alternatives"

I closed this issue because it is very similar to what you mentioned here: erasure encoded backend