downloadFile by streaming about tiny-storage-client

[steevepay]

Hello, thanks for your support!

You don't need to pipe yourself, everything is already managed by tiny-storage-client.
Just return the res on the createOutputSteam.
When the callback is called, it is either because:

• The video stream pipe finished
• An error returned from the Object storage or HTTP request

res.setHeader('Content-Type', 'video/mp4');

function getOutputStream() {
    // The magic is here
    return res;
}

 s3storage.downloadFile('coachtest', '5be9a29f97caa8a38e030f0b18ac6fb1.mp4', { output: createOutputStream }, (err, resp) => {
    if (err) {
        // Handle error 1: Error coming from Node/tiny-storage-client
        return res.status(500).end();
    }
     if (resp?.statusCode !== 200) {
         // Handle error 2: Error coming from the Object Storage
         return res.status(500).end();
     }
     res.end();
});

Let me know if it is good on your side.
I am available if you have questions.
In the meantime, have a good day!

[serolgames]

Great ! So it means I don't have to manage chunk ect for the streaming ?

I have a lack of knowledge about stream, if you can help me to make it work I would be greatful (and maybe it can help others too).
I understood that when the callback is called it means finished/error (hopefully error won't happen often ^^)

You told me to return res in a getOutputStream function , but where should I call it ? I tried in a middleware but it doesn't load on my client

router.get('/stream', (req, res) => 
{                
       const getOutputStream = (req, res) => 
       {
               return res
       }

       function createOutputStream(opts, res) {
               const writer = fs.createWriteStream('5be9a29f97caa8a38e030f0b18ac6fb1.mp4')
               writer.on('error', (e) => { console.log("erreur : " + e) })
               return writer
       }

       res.setHeader('Content-Type', 'video/mp4');

       s3storage.downloadFile('coachtest', '5be9a29f97caa8a38e030f0b18ac6fb1.mp4', { output: createOutputStream }, getOutputStream, (err, resp) => {
               if (err) {
                 return console.log("Error on download: ", err);
               }
       })
})

What do I miss ?
Have a good day too, thank you

[steevepay]

Sorry I misnamed the function name, pass the getOutputStream to the output option on the 3rd argument of downloadFile:

res.setHeader('Content-Type', 'video/mp4');

function getOutputStream() {
    return res;
}

 s3storage.downloadFile('coachtest', '5be9a29f97caa8a38e030f0b18ac6fb1.mp4', { output: getOutputStream }, (err, resp) => {
    if (err) {
        // Handle error 1: Error coming from Node/tiny-storage-client
        return res.status(500).end();
    }
     if (resp?.statusCode !== 200) {
         // Handle error 2: Error coming from the Object Storage
         return res.status(500).end();
     }
     res.end();
});

[serolgames]

Got it !
It works.

But... not completely, my client receive the video and can play it. But he needed to wait the file to completely load before playing.
I want a realtime stream so it doesn't fit my need.

I tried that :

router.get('/stream', (req, res) => 
{                

       const getOutputStream = (req, res) => 
       {
                const headers = {
                        "Content-Type": "video/mp4",
                }; 
              res.writeHead(206, headers)
               return res
       }

       function createOutputStream(opts, res) {
               const writer = fs.createWriteStream('5be9a29f97caa8a38e030f0b18ac6fb1.mp4')
               writer.on('error', (e) => { console.log("erreur : " + e) })
               return writer
       }

       res.setHeader('Content-Type', 'video/mp4');

       s3storage.downloadFile('coachtest', '5be9a29f97caa8a38e030f0b18ac6fb1.mp4', { output: createOutputStream }, getOutputStream, (err, resp) => {
               if (err) {
                 return console.log("Error on download: ", err);
               }
       })
})

but it's not working. Any idea ?

[steevepay]

Hello,

First of all, on the code you shared, it is not possible to pass another argument to downloadFile, basically, the getOutputStream will be called as the callback function. And the last argument, the callback function (err, resp) => {} will never be executed.

The downloadFile function expects only four arguments; here is the prototype:

 function downloadFile (container, filename, options, callback) 

But he needed to wait the file to completely load before playing. I want a realtime stream so it doesn't fit my need.

The downloadFile function streams the whole file and not chunks of the videos.

A streaming client requires only chunks (small packets of videos buffered and streamed one after another over a user’s computing device). A client streaming a video will request automatically with specific start and end of the data chunk to be streamed in continuation.

I just discovered Open Stack Swift provides a header Range to return stream chunks, here is the documentation:

Source: https://docs.openstack.org/api-ref/object-store/

With the tiny-storage-client, you can pass the Range option like this, for instance, fetching a chunk of six bytes of data after a 10-byte offset:

function getOutputStream() {
    res.setHeader('Content-Type', 'video/mp4');
    res.status(206);
    return res;
}

 s3storage.downloadFile('coachtest', '5be9a29f97caa8a38e030f0b18ac6fb1.mp4', { output: getOutputStream, headers: { 'Range': 'bytes=10-15' } }, (err, resp) => {
    if (err) {
        // Handle error 1: Error coming from Node/tiny-storage-client
        return res.status(500).end();
    }
     if (resp?.statusCode !== 200) {
         // Handle error 2: Error coming from the Object Storage
         return res.status(500).end();
     }
     res.end();
});

I never tried streaming only chunks, and I did not execute the code above, but I'm sure it is the right solution ;)
Let me know the result on your side.