Pipe multipart uploads direct to S3 or another file service in connect middleware without writing to disk. It uses multiparty to parse the multipart form.
It is tested and soon to be used in production.
Note: If you are coming from the 0.2.2 or below version, the API has changed because express.multipart
is deprecated and no nice multipart middleware exists to replace it, this package had to replicate some of that functionality. Now the only middleware you apply is the one in this package, and two new options (encoding and byte limits) are added to support functionality lost from express to multiparty.
npm install multipart-pipe
var multipartPipe = require('multipart-pipe'),
knox = require('knox'),
express = require('express')
var app = express(), /* or connect() */
s3 = knox.createClient({
bucket: my_bucket,
key: my_key,
secret: my_secret
})
// Pipes to S3
app.use(multipartPipe.s3(s3))
In the request object after the pipe middleware will be two new fields on the request object:
- a new
req.files
field which will contain a map of filenames prior to upload to filenames on the streamed-to fileserver. - a
req.form
field with a map of form field values that might have also come with the multipart file.
For example:
app.use('/upload', multipartPipe.s3(s3))
app.post('/upload', function (req, res) {
res.send({
ok: true,
uploaded_files: req.files,
other_fields: req.form
})
})
The main way to instantiate the middleware is multipartPipe(options)
where options contains the following:
streamer
- Requiredfunction (part, filename, callback)
- Optionally call
multipartPipe.s3(s3_knox_client, options)
to use built-in S3 streamer
- Optionally call
allow
- OptionalString
orRegExp
to test each part's content-type header for acceptabilityfilename
- Optionalfunction (part_filename, part_content_type)
which returns a filename to store. Defaults tofunction (part_filename) { return part_filename; }
encoding
- Set the encoding. Defaults to the usualutf8
.limit
- Set a bytesReceived limit. Can be in string form like'128mb'
,'1gb'
,'512kb'
. Defaults to `128mb'.
When using pipe.s3(s3_knox_client, opts)
, there are additional options:
headers
- Optional object with default headers for each upload to S3. Defaults to enabling public-read.
-
Limit upload size:
app.use(multipartPipe.s3(s3, { limit: '128mb' }))
-
Limit content types (to say, just images):
app.use(multipartPipe.s3(s3, { allow: /^image\/.*$/ }))
-
Use uploaded filename with counter and a path parameter prepended:
var counter = 0; app.use(multipartPipe.s3(s3, { filename: function (fn, mime) { return req.params.prefix + '/' + (counter++) + '_' + fn } }))
-
Find the right mime type extension for a filename. Referenced package: mime
var mime = require('mime'), path = require('path'); app.use(multipartPipe.s3(s3, { filename: function (fn, contentType) { return "some-directory/" + path.basename(fn).split('.')[0] + '.' + mime.extension(contentType); } }))
-
Create your own streamer function
function streamer(part, filename, callback) { // see source s3streamer() for example } app.use(multipartPipe({streamer: streamer}))
-
Restrict the middleware to a specific path
app.use('/upload', multipartPipe.s3(s3, opts))
MIT in LICENSE file.