Comments (14)
I suspect the blob is triggered with private windows because it has some local storage limits in private mode; testing private windows with 2gb+ downloads triggers a "noStreamsOptionFirefox" message and downloads fail.
No worries, thanks for responding so quickly and for keeping Send alive!
from send.
Looks like I had a typo in my redirect, taking back the "works on blob/doesn't work otherwise" comment. Can confirm just doing a redirect for /api/download and /api/download/blob BOTH do work.
May be able to simply add a variable to specify the blob download url format ?
from send.
Feel free to give it a try for sure. If you get stuck feel free to ask here.
Otherwise I may be able to implement it after the weekend.
from send.
You're right, that S3 objects are currently proxied through the Send instance.
There is currently nothing in place to support something like this, but I'm sure it can be implemented as long as the S3 instance has some public endpoint.
I wonder what security implications this might have though. I can think of the following things:
- this would allow unlimited downloads of the encrypted blob
- this would allow downloading the encrypted blob after the file has expired (but hasn't been cleaned from S3)
These two points aren't a problem when Send is proxying because it enforces these limits. You might consider it an acceptable risk though.
I've managed to get it working with a rewrite of /api/download/blob to my backblaze b2 url for each file via
/api/download/blob/(DOWNLOADid) -> /file/BUCKETNAME/1-DOWNLOADid but it would be nice if it were actually supported. Also I can't figure out how it sometimes calls /download/blob and sometimes just /download/ and it only works properly with blob.
Are you sure that, in the latest version, different paths are used for downloading the actual blobs? I don't recall there being any logic for this. The URL should have the /api/download/<ID>
format.
Just a heads up, the 1-
prefix stands for the expiry time in number of days rounded up, and so this might be different for some uploads.
from send.
At least for the B2 account I'm testing with, there's a way to allow only authorized connections using an ID and key. See here:
https://help.backblaze.com/hc/en-us/articles/360010017893-How-to-allow-Cloudflare-to-fetch-content-from-a-Backblaze-B2-private-bucket
edit: more specifically https://www.backblaze.com/b2/docs/b2_get_download_authorization.html
The method described there doesn't help a whole lot with preventing unlimited downloads since it authorizes ALL downloads from cloudflare, but there's no reason the same authorization method won't work directly from send. I thought that might be a separate feature request after allowing direct connections in the first place though, especially since I think the different S3 providers probably handle authorization for public links differently and it could be a lot of work to support several of them.
After some more testing, the /api/download/blob link seems to only show up in the Firefox private windows I was testing with. All other browsers and Firefox normal windows use just the /api/download/ format, which for some reason I can't get to work using the MITM rewrite I'm currently experimenting with. The blob works, the non-blob doesn't.
Thanks for the headsup with the prefix, I was only testing and should've probably read the code a bit to figure where that came from.
from send.
At least for the B2 account I'm testing with, there's a way to allow only authorized connections using an ID and key. See here:
https://help.backblaze.com/hc/en-us/articles/360010017893-How-to-allow-Cloudflare-to-fetch-content-from-a-Backblaze-B2-private-bucket
edit: more specifically https://www.backblaze.com/b2/docs/b2_get_download_authorization.htmlThe method described there doesn't help a whole lot with preventing unlimited downloads since it authorizes ALL downloads from cloudflare, but there's no reason the same authorization method won't work directly from send. I thought that might be a separate feature request after allowing direct connections in the first place though, especially since I think the different S3 providers probably handle authorization for public links differently and it could be a lot of work to support several of them.
I assume this to be a comment on the two points I made regarding security here. There problem here lies in URL/token re-usability. All downloads through Send decrease the file download counter, once the file expires there is no way to download the file afterwards, even through the same link.
If you provide a direct link to the client liking to the blob, you might decrease the download count with -1 on the Send server when sharing the direct link, but the client will be able to reuse that direct blob link as many times as he'd like (given it's token doesn't expire). There is no way for Send to take control of this. A single-use token for S3 files would solve this, but I'm not aware of this being a thing.
Note that this may be a perfectly fine risk depending on your use case. I'd be perfectly fine with it if this was implemented in Send.
After some more testing, the /api/download/blob link seems to only show up in the Firefox private windows I was testing with. All other browsers and Firefox normal windows use just the /api/download/ format, which for some reason I can't get to work using the MITM rewrite I'm currently experimenting with. The blob works, the non-blob doesn't.
Nice find! Looking into the route configuration, it does indeed show two different blob download routes:
Lines 96 to 101 in 825e394
Downloading worked differently back in the day. When the Send client detects an old browser it might fall back to this method, this old downloading method likely uses the old route then. Running a private window might somehow trigger this.
The routes both link to the same download logic here, so I'm not really sure why this is the case.
I'm sorry I don't know the exact details right now. I don't have much time right now to look into this further.
from send.
Fantastic!
Yeah, supporting this would be a nice addition. I wonder what configuration options Send should provide for this with regard to configuration.
Would you like to give implementing this a try?
from send.
I'm not much of a programmer myself, more sysadmin, but I can try and find help 🙂
from send.
I think there's 3 most popular S3 backends: AWS, Backblaze, and Minio. All three have their own formats for direct downloads:
AWS: https://[BUCKETNAME].s3.amazonaws.com/[FILENAME]
B2: https://[SERVER].backblazeb2.com/file/[BUCKETNAME]/[FILENAME]
Minio: https://[SERVER]/BUCKETNAME]/[FILENAME]
So maybe a single variable something like:
S3DIRECT=[NO|AWS|B2|MINIO]
and
S3DIRECT_SERVER=""
would be enough?
Or do it via a URL template, more flexible but harder to configure and pretty niche cases.
from send.
For https://send.vis.ee/ I'm using Digital Ocean spaces, which uses a different format.
Implementing a template system might actually be pretty easy as it shouldn't do much more than setting the file name an possibly some authentication details.
Replacing a {{FILE}}
needle in a URL string is probably good enough.
from send.
I assume this to be a comment on the two points I made regarding security here. There problem here lies in URL/token re-usability. All downloads through Send decrease the file download counter, once the file expires there is no way to download the file afterwards, even through the same link.
If you provide a direct link to the client liking to the blob, you might decrease the download count with -1 on the Send server when sharing the direct link, but the client will be able to reuse that direct blob link as many times as he'd like (given it's token doesn't expire). There is no way for Send to take control of this. A single-use token for S3 files would solve this, but I'm not aware of this being a thing.
Use presigned URLs
to have Send set the expiration as 10 seconds, 60 seconds or 5 mins, something small from what I've read it only needs to be enough time for the redirect to happen. S3 only checks the expiration time at the time of the request, if you start a request and then the URL expires, the request will continue on without issue, and any subsequent request attempts thereafter will fail.
While it isn't an actual single-use token, it effectively is a single-use token.
As for download counter, have a client-side filestream send events back to the server, only decrease on a completed download event. (I realize this isn't necessary)
from send.
I assume this to be a comment on the two points I made regarding security here. There problem here lies in URL/token re-usability. All downloads through Send decrease the file download counter, once the file expires there is no way to download the file afterwards, even through the same link.
If you provide a direct link to the client liking to the blob, you might decrease the download count with -1 on the Send server when sharing the direct link, but the client will be able to reuse that direct blob link as many times as he'd like (given it's token doesn't expire). There is no way for Send to take control of this. A single-use token for S3 files would solve this, but I'm not aware of this being a thing.Use presigned URLs to have Send set the expiration as 10 seconds, 60 seconds or 5 mins, something small from what I've read it only needs to be enough time for the redirect to happen. S3 only checks the expiration time at the time of the request, if you start a request and then the URL expires, the request will continue on without issue, and any subsequent request attempts thereafter will fail. While it isn't an actual single-use token, it effectively is a single-use token.
I agree that such approach could be a nice middle ground. It'll then be up to the webmaster to determine whether to enable such functionality. I sadly don't have any bandwidth to implement such feature right now though.
As for download counter, have a client-side filestream send events back to the server, only decrease on a completed download event. (I realize this isn't necessary)
This must be implemented on the server side, and the counter must be decreased when requesting the download to prevent abuse.
from send.
As for download counter, have a client-side filestream send events back to the server, only decrease on a completed download event. (I realize this isn't necessary)
This must be implemented on the server side, and the counter must be decreased when requesting the download to prevent abuse.
This suggestion was meant to go hand in hand with the previous presigned URL suggestion, with the "effectively" 1 use token I figured the counter didnt need to be server side as only the one stream was capable of being used anyways. I know it would still be abusable and isn't perfect, but it's the small work around for not being able to monitor the stream server-side. For abuse prevention with this method you could set a hard download cap and a soft cap, soft cap is determined by the client-side completion event and the hard cap is the server-side download request count that is a few more than the client-side.
I realize part of the reason for send's existence is to be as secure as possible so it's not a great recommendation. I'm just trying to think of ways to make the presigned url's integrate okay
from send.
Has this issue been abandoned or is it something that is still being worked on?
from send.
Related Issues (20)
- Total max file size of an instance HOT 2
- Transfer speed drops considerably as soon as the tab is not active HOT 3
- SSO or LDAP auth
- Fix permission on upload folder in docker HOT 2
- Anyone figured out a way to force dark mode? HOT 4
- Any way to make the upload accessible only for internal users and download for external? HOT 2
- If you click the logo then you're redirected to the homepage. How can I turn this off? HOT 2
- Logo reverse back after npm run build
- Error with redirection because of the # in the last part of the link url
- Add `unlimited` as an option for max downloads when sharing a file
- Uploading a folder not working HOT 2
- CUSTOM_DESCRIPTION does not seem to work
- Using send with password behind apache reverse proxy failed HOT 2
- Uploads fail after Send has been running for a extended period fo time HOT 1
- Can`t install Send
- Is it possible to recover an expired link data? HOT 1
- customizing the text under Simple, private file sharing HOT 2
- Is there a possibility to keep the unexpired send-archive always visible? HOT 1
- [FEATURE REQUEST] Add folder picker option like WeTransfer HOT 3
- [Request] ARM64 Support HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from send.