Comments (3)
KISS. I am unsure if multiple range requests are really common in a request, and I already had to deal with http.ServeContent
myself and would rather not duplicate/have core code be borrowed.
If needed, revisit when someone asks, and then/or make a 3rd interface.
Let the backends deal with the problem, and don't try to be too smart/know better than the storage systems without a good reason.
from tusd.
As mentioned in #1048, I prefer having explicit support for fetching ranged content inside the storage implementations, so that we can implement efficiently. My first idea was to add a new interface for storage, such as:
type Range struct {
start int64
end int64
}
type RangedDownloadsUpload interface {
GetRangedReaders(ctx context.Context, ranges []Range) ([]io.ReadCloser, error)
}
The slices are necessary because the Range
header can include multiple ranges.
However, after looking at the source code of Go's implementation for ranged GET requests (see https://cs.opensource.google/go/go/+/refs/tags/go1.21.6:src/net/http/fs.go;l=169-355), I am more hesitant with this approach. Go implements additional logic for preventing abuse and ensuring wide compatibility with even odd clients. If we go with my above proposal, we would have to duplicate this logic inside tusd as well, which I would like to avoid.
As an alternative, we could go with a more lazy route and basically pass the request and response into the storage to let it handle the ranged request as efficiently as possible:
type ServableUpload interface {
ServeContent(ctx context.Context, w ResponseWriter, r *Request) (error)
}
The filestore
could simply use http.ServeContent
internally for this. The s3store
(and similar cloud storages) could implement the interface by relaying the Range
header to S3 and then just piping the response from S3 directly to the end-user's client. The logic for handling ranges is then outsourced to the cloud storage. A downside would be that this way, we also inherit the limitations of the cloud storages. For example, S3 does not support multiple ranges in one request. This limitation would then also be passed to the end-user. However, overall I like this approach for now.
Let me know what you think!
from tusd.
Agreed. I am not sure when I will have time to work on this. If anybody is interested in starting this, let me know and I can help :)
from tusd.
Related Issues (20)
- Allow stopping uploads in PatchRequest before any data is passed to the store HOT 3
- Support RUFH draft-01 and -02
- S3 backend with TUS handler fails to handle PATCH HOT 10
- Keep old metadata after modifying it in pre-create hook HOT 1
- S3 Storage GetReader Range Support HOT 1
- Uniform Termination Process for Store Implementations: Ensuring Consistent Removal of Binary and .info Files HOT 1
- Cleaning up .info files HOT 3
- s3store: Experiment with optimistic part uploads to S3 HOT 2
- Tus returning http in initial response causing redirect to https to strip authorization headers HOT 3
- docs: Add developing and contributing guide
- docs: Add example usage with Minio and Azurite
- Many post-receive hooks after post-finish hook HOT 3
- Unable to fetch Upload Id from Path HOT 1
- cli: Option log-format=json is ignored for plugin logs HOT 1
- using tusd handler with gorilla mux causing "NetworkControlError" warnings HOT 10
- Given nginx configuration slows down uploads over https HOT 1
- NetworkControlError/NetworkTimeoutError "feature not supported" in logs HOT 1
- Mismatched offset caused by duplicate part name with old file use s3store.
- Unclear if filelocker has to be used with filestore HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tusd.