Coder Social home page Coder Social logo

Comments (3)

pcfreak30 avatar pcfreak30 commented on August 17, 2024 1

KISS. I am unsure if multiple range requests are really common in a request, and I already had to deal with http.ServeContent myself and would rather not duplicate/have core code be borrowed.

If needed, revisit when someone asks, and then/or make a 3rd interface.

Let the backends deal with the problem, and don't try to be too smart/know better than the storage systems without a good reason.

from tusd.

Acconut avatar Acconut commented on August 17, 2024

As mentioned in #1048, I prefer having explicit support for fetching ranged content inside the storage implementations, so that we can implement efficiently. My first idea was to add a new interface for storage, such as:

type Range struct {
  start int64
  end int64
}

type RangedDownloadsUpload interface {
  GetRangedReaders(ctx context.Context, ranges []Range) ([]io.ReadCloser, error)
}

The slices are necessary because the Range header can include multiple ranges.

However, after looking at the source code of Go's implementation for ranged GET requests (see https://cs.opensource.google/go/go/+/refs/tags/go1.21.6:src/net/http/fs.go;l=169-355), I am more hesitant with this approach. Go implements additional logic for preventing abuse and ensuring wide compatibility with even odd clients. If we go with my above proposal, we would have to duplicate this logic inside tusd as well, which I would like to avoid.

As an alternative, we could go with a more lazy route and basically pass the request and response into the storage to let it handle the ranged request as efficiently as possible:

type ServableUpload interface {
  ServeContent(ctx context.Context, w ResponseWriter, r *Request) (error)
}

The filestore could simply use http.ServeContent internally for this. The s3store (and similar cloud storages) could implement the interface by relaying the Range header to S3 and then just piping the response from S3 directly to the end-user's client. The logic for handling ranges is then outsourced to the cloud storage. A downside would be that this way, we also inherit the limitations of the cloud storages. For example, S3 does not support multiple ranges in one request. This limitation would then also be passed to the end-user. However, overall I like this approach for now.

Let me know what you think!

from tusd.

Acconut avatar Acconut commented on August 17, 2024

Agreed. I am not sure when I will have time to work on this. If anybody is interested in starting this, let me know and I can help :)

from tusd.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.