Very slow download speed from `ObjectDownloadView` about django-downloadview

[johnthagen]

Interesting that if I read the file into memory and use a vanilla Django HttpResponse directly from a DRF ViewSet, it transfers extremely quickly.

class MyModelViewSet(RetrieveModelMixin, GenericViewSet):
    ...

    def retrieve(self, request: Request, *args: Any, **kwargs: str) -> HttpResponse:
        my_model = self.get_object()
        export_data = Path(my_model.model.path).read_bytes()
        return HttpResponse(
            export_data,
            content_type= "application/octet-stream",
            headers={
                "Content-Disposition": "attachment; "
                'filename="out.bin"',
            },
        )

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  118M  100  118M    0     0   162M      0 --:--:-- --:--:-- --:--:--  161M

I think that is roughly 1.3Gbps. I'm seeing anywhere around a 40x speed improvement.

[johnthagen]

I suspect that the underlying issue is inefficiency in how Django/Python/WSGI stream bytes by looping over the input in the Python interpreter:

django-downloadview/django_downloadview/views/base.py

Line 30 in 338e171

response_class = DownloadResponse

django-downloadview/django_downloadview/response.py

Line 86 in 338e171

class DownloadResponse(StreamingHttpResponse):

django-downloadview/django_downloadview/response.py

Lines 99 to 102 in 338e171

	For large files, or for specialized uses of HTTP streaming,
	applications will usually return an iterator (often a
	generator-iterator) that produces the output in a block-by-block
	fashion.

[johnthagen]

The way I solved this was to simply use pure DRF views to serve files rather than django-downloadview, since I haven't been able to get NGINX acceleration working:

• #177

In case this helps others, here is an implementation:

from pathlib import Path
from typing import Any

from django.db.models import Model
from django.forms import FileField
from django.http import HttpResponse
from drf_spectacular.utils import extend_schema, extend_schema_view
from rest_framework.filters import BaseFilterBackend
from rest_framework.mixins import RetrieveModelMixin
from rest_framework.renderers import BaseRenderer
from rest_framework.request import Request
from rest_framework.viewsets import GenericViewSet


class BinaryRenderer(BaseRenderer):
    """A renderer that supports binary file downloads."""

    media_type = "application/octet-stream"
    format = "bin"

    def render(
        self,
        data: bytes,
        accepted_media_type: str | None = None,
        renderer_context: dict[str, Any] | None = None,
    ) -> bytes:
        return data


@extend_schema_view(retrieve=extend_schema(responses=bytes))
class FileDownloadViewSet(RetrieveModelMixin, GenericViewSet):
    """A generic ViewSet that supports serving downloaded files.

    Rather than streaming the file, the entire file is read into memory and sent out, which can
    improve the download speed of files at the cost of maximum memory.

    Fill in the `file_field` and `filename_field` attributes when deriving from this class to set
    which file should be downloaded from the Model.
    """

    filter_backends: list[BaseFilterBackend] = []
    renderer_classes = [BinaryRenderer]

    file_field: str = NotImplemented
    filename_field: str = NotImplemented

    def retrieve(self, request: Request, *args: Any, **kwargs: str) -> HttpResponse:
        model: Model = self.get_object()
        file: FileField = getattr(model, self.file_field)
        file_data = Path(file.path).read_bytes()  # type: ignore[attr-defined]
        file_name: str = getattr(model, self.filename_field)
        return HttpResponse(
            file_data,
            content_type=BinaryRenderer.media_type,
            headers={"Content-Disposition": f'attachment; filename="{file_name}"'},
        )

And an example usage:

class MyModelDownloadViewSet(FileDownloadViewSet):
    queryset = MyModel.objects.all()
    permission_classes = [DjangoObjectPermissions]

    file_field = "file"
    filename_field = "filename"