Coder Social home page Coder Social logo

multiparty's Introduction

multiparty

NPM Version NPM Downloads Node.js Version Build Status Test Coverage

Parse http requests with content-type multipart/form-data, also known as file uploads.

See also busboy - a faster alternative which may be worth looking into.

Installation

This is a Node.js module available through the npm registry. Installation is done using the npm install command:

npm install multiparty

Usage

Parse an incoming multipart/form-data request.

var multiparty = require('multiparty');
var http = require('http');
var util = require('util');

http.createServer(function(req, res) {
  if (req.url === '/upload' && req.method === 'POST') {
    // parse a file upload
    var form = new multiparty.Form();

    form.parse(req, function(err, fields, files) {
      res.writeHead(200, { 'content-type': 'text/plain' });
      res.write('received upload:\n\n');
      res.end(util.inspect({ fields: fields, files: files }));
    });

    return;
  }

  // show a file upload form
  res.writeHead(200, { 'content-type': 'text/html' });
  res.end(
    '<form action="/upload" enctype="multipart/form-data" method="post">'+
    '<input type="text" name="title"><br>'+
    '<input type="file" name="upload" multiple="multiple"><br>'+
    '<input type="submit" value="Upload">'+
    '</form>'
  );
}).listen(8080);

API

multiparty.Form

var form = new multiparty.Form(options)

Creates a new form. Options:

  • encoding - sets encoding for the incoming form fields. Defaults to utf8.
  • maxFieldsSize - Limits the amount of memory all fields (not files) can allocate in bytes. If this value is exceeded, an error event is emitted. The default size is 2MB.
  • maxFields - Limits the number of fields that will be parsed before emitting an error event. A file counts as a field in this case. Defaults to 1000.
  • maxFilesSize - Only relevant when autoFiles is true. Limits the total bytes accepted for all files combined. If this value is exceeded, an error event is emitted. The default is Infinity.
  • autoFields - Enables field events and disables part events for fields. This is automatically set to true if you add a field listener.
  • autoFiles - Enables file events and disables part events for files. This is automatically set to true if you add a file listener.
  • uploadDir - Only relevant when autoFiles is true. The directory for placing file uploads in. You can move them later using fs.rename(). Defaults to os.tmpdir().

form.parse(request, [cb])

Parses an incoming node.js request containing form data.This will cause form to emit events based off the incoming request.

var count = 0;
var form = new multiparty.Form();

// Errors may be emitted
// Note that if you are listening to 'part' events, the same error may be
// emitted from the `form` and the `part`.
form.on('error', function(err) {
  console.log('Error parsing form: ' + err.stack);
});

// Parts are emitted when parsing the form
form.on('part', function(part) {
  // You *must* act on the part by reading it
  // NOTE: if you want to ignore it, just call "part.resume()"

  if (part.filename === undefined) {
    // filename is not defined when this is a field and not a file
    console.log('got field named ' + part.name);
    // ignore field's content
    part.resume();
  }

  if (part.filename !== undefined) {
    // filename is defined when this is a file
    count++;
    console.log('got file named ' + part.name);
    // ignore file's content here
    part.resume();
  }

  part.on('error', function(err) {
    // decide what to do
  });
});

// Close emitted after form parsed
form.on('close', function() {
  console.log('Upload completed!');
  res.setHeader('text/plain');
  res.end('Received ' + count + ' files');
});

// Parse req
form.parse(req);

If cb is provided, autoFields and autoFiles are set to true and all fields and files are collected and passed to the callback, removing the need to listen to any events on form. This is for convenience when you want to read everything, but be sure to write cleanup code, as this will write all uploaded files to the disk, even ones you may not be interested in.

form.parse(req, function(err, fields, files) {
  Object.keys(fields).forEach(function(name) {
    console.log('got field named ' + name);
  });

  Object.keys(files).forEach(function(name) {
    console.log('got file named ' + name);
  });

  console.log('Upload completed!');
  res.setHeader('text/plain');
  res.end('Received ' + files.length + ' files');
});

fields is an object where the property names are field names and the values are arrays of field values.

files is an object where the property names are field names and the values are arrays of file objects.

form.bytesReceived

The amount of bytes received for this form so far.

form.bytesExpected

The expected number of bytes in this form.

Events

'error' (err)

Unless you supply a callback to form.parse, you definitely want to handle this event. Otherwise your server will crash when users submit bogus multipart requests!

Only one 'error' event can ever be emitted, and if an 'error' event is emitted, then 'close' will not be emitted.

If the error would correspond to a certain HTTP response code, the err object will have a statusCode property with the value of the suggested HTTP response code to send back.

Note that an 'error' event will be emitted both from the form and from the current part.

'part' (part)

Emitted when a part is encountered in the request. part is a ReadableStream. It also has the following properties:

  • headers - the headers for this part. For example, you may be interested in content-type.
  • name - the field name for this part
  • filename - only if the part is an incoming file
  • byteOffset - the byte offset of this part in the request body
  • byteCount - assuming that this is the last part in the request, this is the size of this part in bytes. You could use this, for example, to set the Content-Length header if uploading to S3. If the part had a Content-Length header then that value is used here instead.

Parts for fields are not emitted when autoFields is on, and likewise parts for files are not emitted when autoFiles is on.

part emits 'error' events! Make sure you handle them.

'aborted'

Emitted when the request is aborted. This event will be followed shortly by an error event. In practice you do not need to handle this event.

'progress' (bytesReceived, bytesExpected)

Emitted when a chunk of data is received for the form. The bytesReceived argument contains the total count of bytes received for this form so far. The bytesExpected argument contains the total expected bytes if known, otherwise null.

'close'

Emitted after all parts have been parsed and emitted. Not emitted if an error event is emitted.

If you have autoFiles on, this is not fired until all the data has been flushed to disk and the file handles have been closed.

This is typically when you would send your response.

'file' (name, file)

By default multiparty will not touch your hard drive. But if you add this listener, multiparty automatically sets form.autoFiles to true and will stream uploads to disk for you.

The max bytes accepted per request can be specified with maxFilesSize.

  • name - the field name for this file
  • file - an object with these properties:
    • fieldName - same as name - the field name for this file
    • originalFilename - the filename that the user reports for the file
    • path - the absolute path of the uploaded file on disk
    • headers - the HTTP headers that were sent along with this file
    • size - size of the file in bytes

'field' (name, value)

  • name - field name
  • value - string field value

License

MIT

multiparty's People

Contributors

andrewrk avatar baoshan avatar bengourley avatar chalker avatar dougwilson avatar egirshov avatar elmerbulthuis avatar felixge avatar foxxtrot avatar gliese1337 avatar jasondavies avatar jesperp avatar jimbly avatar kapouer avatar maciek-codes avatar mattrobenolt avatar mikefrey avatar mikesamuel avatar monsendag avatar ncr avatar orangedog avatar perelandric avatar psugihara avatar rkusa avatar ryhinchey avatar sreuter avatar svnlto avatar thanpolas avatar tj avatar tojocky avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

multiparty's Issues

Can't install, checksum failed

root@vpsssmis:~# npm install -g multiparty
npm http GET https://registry.npmjs.org/multiparty
npm http 200 https://registry.npmjs.org/multiparty
npm http GET https://registry.npmjs.org/multiparty/-/multiparty-2.1.8.tgz
npm http 200 https://registry.npmjs.org/multiparty/-/multiparty-2.1.8.tgz
npm ERR! Error: shasum check failed for /root/tmp/npm-32055-gckpA-q1/1379698670643-0.04241336928680539/tmp.tgz
npm ERR! Expected: 35a31834323578ee65f5d870568097914739cf4e
npm ERR! Actual:   2d521b9e2fe44cbcc5b2a7d7033190744141745f
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/sha/index.js:38:8
npm ERR!     at ReadStream.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/sha/index.js:85:7)
npm ERR!     at ReadStream.EventEmitter.emit (events.js:125:20)
npm ERR!     at _stream_readable.js:896:16
npm ERR!     at process._tickCallback (node.js:317:11)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <[email protected]>

npm ERR! System Linux 2.6.32-308.8.2.el5.028stab101.1
npm ERR! command "/usr/local/bin/node" "/usr/local/bin/npm" "install" "-g" "multiparty"
npm ERR! cwd /root
npm ERR! node -v v0.11.7
npm ERR! npm -v 1.3.8
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /root/npm-debug.log
npm ERR! not ok code 0

Error while uploading multiple files on S3 using s3.js example.

I am trying to upload multiple files using S3 example on Amazon S3 using streams without touching hard drive. It works for single file but fails in case of multiple file selection. In following code I am using 'multiple' attribute with file input to select one or more files for uploading.

var http = require('http')
    , util = require('util')
    , multiparty = require('multiparty')
    , knox = require('knox')
    , Batch = require('batch')
    , PORT = process.env.PORT || 27372

var s3Client = knox.createClient({
    secure: false,
    key: '',
    secret: '',
    bucket: ''
});

var server = http.createServer(function(req, res) {
    if (req.url === '/') {
        res.writeHead(200, {'content-type': 'text/html'});
        res.end(
            '<form action="/upload" enctype="multipart/form-data" method="post">'+
                '<input type="file" name="upload" multiple><br>'+
                '<input type="submit" value="Upload">'+
                '</form>'
        );
    } else if (req.url === '/upload') {
        var headers = {
            'x-amz-acl': 'public-read'
        };
        var form = new multiparty.Form();
        var batch = new Batch();
        batch.push(function(cb) {
            form.on('part', function(part) {
                if (! part.filename) return;
                cb(null, part);
            });
        });
        batch.end(function(err, results) {
            if (err) throw err;
            form.removeListener('close', onEnd);
            var destPath = '/'+results[0].filename
                , part = results[0];
            headers['Content-Length'] = part.byteCount;
            s3Client.putStream(part, destPath, headers, function(err, s3Response) {
                if (err) throw err;
                res.statusCode = s3Response.statusCode;
                s3Response.pipe(res);
                console.log("https://s3.amazonaws.com/testcmsresource" + destPath);
            });
        });
        form.on('close', onEnd);
        form.parse(req);
    } else {
        res.writeHead(404, {'content-type': 'text/plain'});
        res.end('404');
    }
    function onEnd() {
        throw new Error("no uploaded file");
    }
});
server.listen(PORT, function() {
    console.info('listening on http://0.0.0.0:'+PORT+'/');
});

If I select single file then it works fine otherwise it fails with following error:

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: Request aborted
    at IncomingMessage.onReqAborted (G:\Projects\nodejs\Test\node_modules\multiparty\index.js:131:17)
    at IncomingMessage.EventEmitter.emit (events.js:92:17)
    at abortIncoming (http.js:1911:11)
    at Socket.serverSocketCloseListener (http.js:1923:5)
    at Socket.EventEmitter.emit (events.js:117:20)
    at TCP.close (net.js:466:12)

Kindly suggest how multiple files can be uploaded to S3 without touching to hard drive.

part.on('end') called after form.on('close')

Near line 408, there is:

Form.prototype.onParsePartEnd = function() {
  if (this.destStream) {
    flushWriteCbs(this);
    var s = this.destStream;
    process.nextTick(function() {
      s.end();
    });
  }
  clearPartVars(this);
}

The part's "end" event is triggered in a nextTick. However, the "close" event for the form is triggered immediately.

Because of that, I finish processing my form without the last part.

File size from stream

Hello and thank your work, it's going to be very helpful!

I find difficulties understanding what "part.byteCount" is since it changes in function of the part orders. I get the correct file size if the file is the last part (as stated in the documentation) but I don't understand why it's not the correct file size if it's not the case; and can't figure how to compute it if it's not the last part (and the part doesn't have a content-length header).

Thanks in advance for your answer.
Cheers!

byteCount get negative when using Supertest attach()

Hi, I have some test like this:

var request = require('supertest');

describe('POST /files', function() {
  it('should return json', function(done) {
    request(app)
      .post('/files')
      .set('Accept', 'application/json')
      .attach('upload', __dirname + '/foo.txt')
      .expect('Content-Type', /json/)
      .expect(200)
      .end(function(err, res) {
        if (err) return done(err);
        done();
      });
  });
});

When I do the following in my handler:

form.on('part', function(part) {
  console.log(part)
});

part.byteCount is a negative number.

When using curl -F "[email protected]" http://localhost:3000/files, part.byteCount is the correct value.

And when I use supertest to upload file to a PHP server, the file is successfully uploaded.

The header from supertest is like Content-Type "multipart/form-data; boundary=vfhx8k7a42l2ztka2kdpkp3gsm8tj7xu", does this matter?

Feature suggestion: max file size

Provide a way to configure the parsing process in order to avoid uploading files which are greater than a maximum file size set in configuration (i.e: maxFileSize: x, where x is a number of bytes)
If the file is greater than the maximum value the upload process should be canceled and no file should be written.

csrf with multiparty

Hi,

I'm not sure if this is a bug or just something on my side.
I'm using multiparty and ofcourse with enctype='multipart/form-data' on the form side...
However, whenever I enable csrf - and submit the form, server responds with 403 as Invalid CSRF token.
I did verify that the token does get created and is the same token on the form as an hidden input element. However, I keep geting this error with multipart/form-data and multiparty. If I remove multipart/form-data, ofcourse it works fine. If I disable csrf, it works fine as well.

Has anyone come across this issue? Am I missing something trivial?

Thanks,
Jatin

My setup:
I'm using express 3.5 so it's nothing to do with connect 3.0 and and express 4.0 deprecations...
server.js-

app.use(express.csrf());
app.use(function (req, res, next) {
    res.locals.csrftoken = req.csrfToken();
    next();
});

jade form:


form(id='headerinfo-form', action='/submit-campaign-info2', enctype='multipart/form-data', method='post')
fieldset
input(type='hidden', name='_csrf', value='#{csrftoken}')
ol
li
.form-group
label Small Logo on home page (max width: 120px, max-heigh: 50px).
input(type='file', name='logo')
button.btn.rounded(type='submit', id='wizard-submit')

connect middleware

any possibility you can create and publish your own connect middleware so we can test it out?

Knowing all the parts have been emitted

Maybe I'm missing it, but it would be nice to know when all the parts have been emitted so I can work with the data after the fact. The 'close' event doesn't seem to fire unless all the parts have been emitted and consumed. Any help appreciated.

Issue when using express.bodyParser?

Hi. So if I have express using bodyParser like:

app.use(express.bodyParser());

It does not seem to work, I get an error:

[Error: stream ended unexpectedly]

Any way around this?

Thanks.

Error: Expected CR Received 45

Hello,

I'm using express 3.4.6 which uses connect 2.11.2, which uses node-multiparty :)

When I enable the multipart connect module app.use(express.multipart()); and attempt to do multipart form uploads, multiparty gives me the error Error: Expected CR Received 45 in all instances. Any idea what could be the cause of this?

Thanks!

Feature Request: Method to get non-file field values

What is the best way to get the fields and their values for all non file fields (parts that do not have a fileName)?

On the form.on('part') function, you can retrieve name with part.name but no way to get part.value.

NPM: 404 Not found?

npm http GET https://registry.npmjs.org/multiparty
npm http 304 https://registry.npmjs.org/multiparty
npm http GET https://registry.npmjs.org/multiparty/-/multiparty-3.1.1.tgz
npm http 404 https://registry.npmjs.org/multiparty/-/multiparty-3.1.1.tgz
npm ERR! fetch failed https://registry.npmjs.org/multiparty/-/multiparty-3.1.1.tgz
npm ERR! Error: 404 Not Found

Is it NPM or multiparty is not listed anymore?

avoid creating temp files for empty uploads

Hi,

Multiparty creates temp files for empty uploads. Wouldn't be better to omit empty uploads, e.g. when user did not select file at all?

"files": {
    "image": [
      {
        "fieldName": "image",
        "originalFilename": "",
        "path": "/var/folders/hr/2bmj9h196p56zvtc90p1lp_40000gq/T/2476-esfvcu",
        "headers": {
          "content-disposition": "form-data; name=\"image\"; filename=\"\"",
          "content-type": "application/octet-stream"
        },
        "ws": {
          "_writableState": {
            "highWaterMark": 16384,
            "objectMode": false,
            "needDrain": false,
            "ending": true,
            "ended": true,
            "finished": true,
            "decodeStrings": true,
            "defaultEncoding": "utf8",
            "length": 0,
            "writing": false,
            "sync": true,
            "bufferProcessing": false,
            "writecb": null,
            "writelen": 0,
            "buffer": []
          },
          "writable": true,
          "domain": null,
          "_events": {
            "error": [
              null
            ],
            "close": [
              null
            ]
          },
          "_maxListeners": 10,
          "path": "/var/folders/hr/2bmj9h196p56zvtc90p1lp_40000gq/T/2476-esfvcu",
          "fd": null,
          "flags": "w",
          "mode": 438,
          "bytesWritten": 0
        },
        "size": 0
      }
    ]
  }

2.1.0 installation shasum check failed

I'm not sure if this report is correct here, but I'm havin issues installing multiparty through npm.

Installing version 2.1.0 (and higher) gives me npm ERR! Error: shasum check failed whereas installing 2.0.0 works like a charm (see below).

root@mpl-debian:~/scg-mediaupload# npm install [email protected]
npm http GET https://registry.npmjs.org/multiparty/2.1.0
npm http 200 https://registry.npmjs.org/multiparty/2.1.0
npm http GET https://registry.npmjs.org/multiparty/-/multiparty-2.1.0.tgz
npm http 200 https://registry.npmjs.org/multiparty/-/multiparty-2.1.0.tgz
npm ERR! Error: shasum check failed for /root/tmp/npm-1742-7LA2sM14/1379421553513-0.9426389483269304/tmp.tgz
npm ERR! Expected: 989a593866698946311c2c3e3f4e9deee69dd21c
npm ERR! Actual:   20d0fe62dfd93861c9109ed9ede51d19f4a87cbc
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/sha/index.js:38:8
npm ERR!     at ReadStream.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/sha/index.js:85:7)
npm ERR!     at ReadStream.EventEmitter.emit (events.js:125:20)
npm ERR!     at _stream_readable.js:896:16
npm ERR!     at process._tickCallback (node.js:316:11)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <[email protected]>
npm ERR! System Linux 3.2.0-4-amd64
npm ERR! command "/usr/local/bin/node" "/usr/local/bin/npm" "install" "[email protected]"
npm ERR! cwd /root/scg-mediaupload
npm ERR! node -v v0.11.8-pre
npm ERR! npm -v 1.3.11
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /root/scg-mediaupload/npm-debug.log
npm ERR! not ok code 0
root@mpl-debian:~/scg-mediaupload# npm install [email protected]
npm http GET https://registry.npmjs.org/multiparty
npm http 304 https://registry.npmjs.org/multiparty
[email protected] node_modules/multiparty

EMFILE error thrown during upload

Hello,

I'm trying to handle a multiple file upload with a big amount of file to transfer. An EMFILE error is thrown during the parsing of the form.

I am aware of the ulimit -n limitation and how to increase this limit, but I hope I there is a way to handle this without changing it because even if I change the limit to 10000+ any user could try to upload more files and crashed my server.

I have the error with the following code when i try to upload more than 256 files (depending on your ulimit -n value the amount of file needed to get the error can change).

var http = require('http')
    , multiparty = require('multiparty')
    , PORT = process.env.PORT || 27372

var server = http.createServer(function(req, res) {
    if (req.url === '/') {
        res.writeHead(200, {'content-type': 'text/html'});
        res.end(
            '<form action="/upload" enctype="multipart/form-data" method="post">'+
            '<input type="text" name="title"><br>'+
            '<input type="file" name="upload" multiple="multiple"><br>'+
            '<input type="submit" value="Upload">'+
            '</form>'
        );
    } else if (req.url === '/upload') {
        var form = new multiparty.Form({
            maxFields: 5000
        });
        form.on('file', function(name, file) {
            console.log(file)
        });
        form.on('close', function() {
            console.log('close')
        });

        form.parse(req);
    } else {
        res.writeHead(404, {'content-type': 'text/plain'});
        res.end('404');
    }
});
server.listen(PORT, function() {
    console.info('listening on http://127.0.0.1:'+PORT+'/');
});

rename temp file

Hi
looking into this module since body parser with express 4.x is no longer default.
I want to achieve rename of the file once it is saved. with body parser did it similar to this but now file.originalFileName is not available
var form = new multiparty.Form({uploadDir:'./public/uploads'});

form.parse(req, function(err, fields, files) {
    var file = util.inspect(files.file);
    console.log('FILE '+file);


    var fileExt = file.originalFilename.replace(/^.*\./,'');
    var newFileName=uuid.v4()+ '.'+fileExt;
    return fs.rename(file.path, './public/uploads/' +newFileName, function(err) {
        if (err) {
            res.send({
                error: err
            });
        }
        else{
            console.log('file created on our server');
            return res.send({
                success: true
            });
        }

    });


    //res.end(util.inspect({fields: fields, files: files}));
    return res.send({
        success: true
    });
});

How should I solve now?

maxFilesSize is not being checked properly

I just attempted to use the maxFilesSize for a project, and it appears there is a bug in the calculation of the in progress file size (using StreamCounter). It ends up with files prematurely hitting the limit even if the file size is below the max.

better api for multiple files

I've tried to upload 2 files and I noticed that filesObject only contains one of the files but in fact both files were uploaded.
Moreover using filesList param the 2 files can be found inside it.

var form = new multiparty.Form();
 form.parse(req, function(err, fieldsObject, filesObject, fieldsList, filesList) {
 // Code here
}

Output values:

filesObject value:

{ image:
   { fieldName: 'image',
     originalFilename: 'startup-fair.jpg',
     path: 'C:\\Users\\Al\\AppData\\Local\\Temp\\2284-118w0de.jpg',
     headers:
      { 'content-disposition': 'form-data; name="image"; filename="startup-fai
jpg"',
        'content-type': 'image/jpeg' },
     ws:
      { _writableState: [Object],
        writable: true,
        domain: null,
        _events: [Object],
        _maxListeners: 10,
        path: 'C:\\Users\\Al\\AppData\\Local\\Temp\\2284-118w0de.jpg',
        fd: null,
        flags: 'w',
        mode: 438,
        start: undefined,
        pos: undefined,
        bytesWritten: 87258,
        closed: true },
     size: 87258 } }

filesList value:

filesList [ { fieldName: 'image',
    originalFilename: 'Picture0025.jpg',
    path: 'C:\\Users\\Al\\AppData\\Local\\Temp\\2284-16dt281.jpg',
    headers:
     { 'content-disposition': 'form-data; name="image"; filename="Picture0025.j
g"',
       'content-type': 'image/jpeg' },
    ws:
     { _writableState: [Object],
       writable: true,
       domain: null,
       _events: [Object],
       _maxListeners: 10,
       path: 'C:\\Users\\Al\\AppData\\Local\\Temp\\2284-16dt281.jpg',
       fd: null,
       flags: 'w',
       mode: 438,
       start: undefined,
       pos: undefined,
       bytesWritten: 44394,
       closed: true },
    size: 44394 },
  { fieldName: 'image',
    originalFilename: 'startup-fair.jpg',
    path: 'C:\\Users\\Al\\AppData\\Local\\Temp\\2284-118w0de.jpg',
    headers:
     { 'content-disposition': 'form-data; name="image"; filename="startup-fair.
pg"',
       'content-type': 'image/jpeg' },
    ws:
     { _writableState: [Object],
       writable: true,
       domain: null,
       _events: [Object],
       _maxListeners: 10,
       path: 'C:\\Users\\Al\\AppData\\Local\\Temp\\2284-118w0de.jpg',
       fd: null,
       flags: 'w',
       mode: 438,
       start: undefined,
       pos: undefined,
       bytesWritten: 87258,
       closed: true },
    size: 87258 } ]

clarify fields <> req.body

If i understand well, 'fields' comes from multipart/form-data only,
where req.body comes from application/x-www-form-urlencoded data.
It's a bit confusing when handling a POST - one has to check content type to know where to look. That question came when handling a POST with an optional file input.

Image Resizing

First off, thanks for your great work!

I have a question about uploading an image to node/express, resizing to multiple different variants and then pushing the resized images to amazon s3.

My nodejs application is hosted on Heroku and I copied your s3.js example to get passthrough uploading working. Now I’m wondering about the best way to add in a resizing step where I can create several variants of the uploaded image.

My question is…

Would it be best to:

  • provide a callback to parse() so the file gets written to tmp
  • resize the file stored in tmp with imagemagick
  • push new files to s3
  • when last variant pushed, clean up tmp and send response

OR

Any insight you have would be greatly appreciated :)

npm does not install

Hi,

I cannot install the package thru npm.
node version 0.10.15

npm ERR! fetch failed https://registry.npmjs.org/multiparty/-/multiparty-3.2.3.tgz

regards

Log is as below:

root@nginx-php-fastcgi /var/www# npm install multiparty
npm http GET https://registry.npmjs.org/multiparty
npm http 304 https://registry.npmjs.org/multiparty
npm http GET https://registry.npmjs.org/multiparty/-/multiparty-3.2.3.tgz
npm http 503 https://registry.npmjs.org/multiparty/-/multiparty-3.2.3.tgz
npm ERR! fetch failed https://registry.npmjs.org/multiparty/-/multiparty-3.2.3.tgz
npm http GET https://registry.npmjs.org/multiparty/-/multiparty-3.2.3.tgz
npm http 503 https://registry.npmjs.org/multiparty/-/multiparty-3.2.3.tgz
npm ERR! fetch failed https://registry.npmjs.org/multiparty/-/multiparty-3.2.3.tgz
npm http GET https://registry.npmjs.org/multiparty/-/multiparty-3.2.3.tgz
npm http 503 https://registry.npmjs.org/multiparty/-/multiparty-3.2.3.tgz
npm ERR! fetch failed https://registry.npmjs.org/multiparty/-/multiparty-3.2.3.tgz
npm ERR! Error: 503 Service Unavailable
npm ERR! at WriteStream. (/root/.nvm/v0.10.15/lib/node_modules/npm/lib/utils/fetch.js:57:12)
npm ERR! at WriteStream.EventEmitter.emit (events.js:117:20)
npm ERR! at fs.js:1596:14
npm ERR! at /root/.nvm/v0.10.15/lib/node_modules/npm/node_modules/graceful-fs/graceful-fs.js:103:5
npm ERR! at Object.oncomplete (fs.js:107:15)
npm ERR! If you need help, you may report this log at:
npm ERR! http://github.com/isaacs/npm/issues
npm ERR! or email it to:
npm ERR! [email protected]

npm ERR! System Linux 2.6.32-5-xen-686
npm ERR! command "/root/.nvm/v0.10.15/bin/node" "/root/.nvm/v0.10.15/bin/npm" "install" "multiparty"
npm ERR! cwd /var/www
npm ERR! node -v v0.10.15
npm ERR! npm -v 1.3.5
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /var/www/npm-debug.log
npm ERR! not ok code 0

formdata as arrays?

Hi,

Ran into the problem today with a form, basically every field coming from form.parse() converted to array even though some fields do not repeat in POST request.

{ name: [ 'Blah' ],
  summary: [ 'Blah' ],
  type: [ 'shot' ],
  glassKind: [ 'shot_glass' ],
  strength: [ '40' ],
  'ingredient[name]': [ 'blah', 'blah' ],
  'ingredient[qty]': [ '100', '1' ] }

Is it normal behavior for multiparty?

Receiving files from forms submitted without multipart/form-data

I am currently utilizing http://mean.io/ to develop an app. AngularJS relies on a function call while submitting a form, there is no enctype attribute.

The multipart module doesn't fire its form events because there is no file part coming through the response.

I am attempting to run a putStream() to stream files directly to S3 from my controller in my app.

How can this be configured to pick up the part without using an enctype?

A few ideas I had were to force an enctype in the Angular Service controlling my articles.

The other was to wrap the HTML5 FileReader API in an Angular directive to parse the file data as an object, but this ends up with the file in req.body - and I don't think multipart is configured this way.

I can post some code examples of what I have tried if necessary, but hopefully someone has some insight.

Installing 'multiparty' package problem - with npm on win 8.1

Hi. I'm trying to install the grunt-contrib-connect package on Windows 8.1 (as an administrator). But when I get this lines:

npm http GET https://registry.npmjs.org/multiparty/-/multiparty-2.2.0.tgz
npm http 200 https://registry.npmjs.org/multiparty/-/multiparty-2.2.0.tgz

it stops without any errors. I found this QA, but it didn't help. So, I tried to install [email protected] package separately, But nothing changed and still, it just stops at mentioned lines without any error. Do you know what should I do?

multiparty installing screenshot

Here is my StackOverflow question about the problem

Thanks in advance.

How to get to the file data

Thanks for your great work on node-multiparty (and of course, formidable). Question for you….

In this example: https://github.com/superjoe30/node-multiparty/blob/master/examples/upload.js

…I can see how to get all the metadata about the file, but how do you actually get at the contents of the file?

My goal is to read the bits (small images) and save them into MongoDB. They are well under 4MB, so no need for GridFS.

Would appreciate a sample code snippet or an expanded example on how to actually read the bytes into a variable.

form.parse() not functioning as described in the documentation

form.parse(request, [cb])

Parses an incoming node.js request containing form data. If cb is provided, autoFields and autoFiles are set to true and all fields and files are collected and passed to the callback:


The file event will fire for each and every file selected in a multiple file input. This is expected.

However, when using form.parse and autoFiles: true only one of the many files are passed to the callback.

The documentation indicates that when using autoFiles: true all fields and files are collected and passed to the callback but this is not what I am experiencing.

The problem that this causes for me is that I need to ensure a function is called for each file, after all files have been sent. If I call that function in the file event for each one, I have no way of verifying that.

I would like to be able to loop through any array of files passed to the callback of form.parse, and which point I can ensure that the function is called for each.

Space between "boundary" should be optional

Currently we have
var CONTENT_TYPE_RE = /^multipart\/(form-data|related);\s+boundary=(?:"([^"]+)"|([^;]+))$/i;

However Phonegap for example send by default multipart-form-data;boundary=ffffff (i.e. no spaces).

Phusion Passenger Error: stream ended unexpectedly

Hi there,

We have intermittent file upload failures and the only error message we're seeing in the logs is Error: stream ended unexpectedly followed by a stack trace. The stack trace points to the following code within multiparty

self.on('finish', function() {
    if ((self.state === HEADER_FIELD_START && self.index === 0) ||
        (self.state === PART_DATA && self.index === self.boundary.length))
    {
      self.onParsePartEnd();
    } else if (self.state !== END) {
      self.handleError(new Error('stream ended unexpectedly'));
    }
    self.finished = true;
    maybeClose(self);
  });

In our case in particular, the error is thrown when state = PART_DATA, index = 0, and boundary.length = 42. On successful uploads, the state equals END. I'm trying to figure out how the index is calculated and under what circumstances this function will be called when the state is equal to PART_DATA?

As a test, I removed the self.index === self.boundary.length check from the if statement and this fixed my issue but obviously that check is there for a reason.

I should also note that this Node application is running under Nginx + Passenger (with SSL) and these errors are only noticed when the upload is initiated from a Chrome browser ( boundary is ------WebKitFormBoundary1ONhqXuetuF7yDxs ).

Any help would be greatly appreciated. Thanks.

maxFileSize error event webpage loading forever

I have a problem when multiparty fires an error event (like when the file size exceeds maxFileSize), the page in the browser stays loading the form post forever. I submit a file larger than the max to fire the event and the page load spinner icon keeps spinning forever.

My code:

    form.parse(req, function (err, fields, files) {
            if (err) {
                console.error(err);
                return res.status(500).send(err);
            }
    ...
    }

When an error occurs I send the error to the user but it's like he never receives it or something.

Express v4.3.0
Modules:

    app.use(responseTime());
    app.use(morgan('short'));
    app.use(compression());
    app.use(methodOverride());
    app.use(cookieParser());
    app.use(bodyParser());

Callback never called

Hey, I'm using this to upload images to my server and for some reason my callback never gets called and the request just hangs on form.parse. I've tried making a post without specifying any images, and I've event tried commenting the image part out of the form. This seemed to be working a few weeks ago, so I'm not sure why it isn't now.

Here's what my form looks something like:

<form action="/share/post" method="post" enctype="multipart/form-data">
    <label for="post_images">
        <img src="images.png" alt="Add Some Images!">
    </label>
    <input id="post_images" type="file" accept="image/*" name="images[]" multiple>
    <input type="submit" value="Post!">
    <textarea name="text" placeholder="Post Something Here ..."></textarea>
</form>

And then my code looks something like:

app.post("/share/post",function(req,res){
    console.log("This gets called");
    var form = new multiparty.Form();
    form.parse(req,function(err,fields,files){
        console.log("This never gets called");
        myImageSavingCodeHere();
        res.send("Done");
    });
});

Any idea as to what might be going wrong?

form.parse only returns a single file upload even though multiple form.on('file') fire

I'm uploading 2 files which each show up as a 'file' event but only the first shows up in form.parse:

2 'file' events containing both files:

{ originalFilename: 'SOG2.JPG',
  path: '/tmp/13278-1wddlfi.JPG',
  headers: 
   { 'content-disposition': 'form-data; name="files[]"; filename="SOG2.JPG"',
     'content-type': 'image/jpeg' },
  ws: 
   { domain: null,
     _events: 
      { unpipe: [Function: onunpipe],
        drain: [Function],
        error: [Object],
        close: [Object],
        finish: [Object] },
     _maxListeners: 10,
     path: '/tmp/13278-1wddlfi.JPG',
     fd: 19,
     writable: false,
     flags: 'w',
     encoding: 'binary',
     mode: 438,
     bytesWritten: 86741,
     busy: true,
     _queue: [ [Object] ],
     _open: [Function],
     drainable: true } }
{ originalFilename: 'BenF364_LIB353.zip',
  path: '/tmp/13278-s8mnog.zip',
  headers: 
   { 'content-disposition': 'form-data; name="files[]"; filename="BenF364_LIB353.zip"',
     'content-type': 'application/zip' },
  ws: 
   { domain: null,
     _events: 
      { unpipe: [Function: onunpipe],
        drain: [Function],
        error: [Object],
        close: [Object],
        finish: [Object] },
     _maxListeners: 10,
     path: '/tmp/13278-s8mnog.zip',
     fd: 19,
     writable: false,
     flags: 'w',
     encoding: 'binary',
     mode: 438,
     bytesWritten: 49127,
     busy: true,
     _queue: [ [Object] ],
     _open: [Function],
     drainable: true } }

1 form.parse event:

{ 'files[]': 
   { originalFilename: 'BenF364_LIB353.zip',
     path: '/tmp/13278-s8mnog.zip',
     headers: 
      { 'content-disposition': 'form-data; name="files[]"; filename="BenF364_LIB353.zip"',
        'content-type': 'application/zip' },
     ws: 
      { domain: null,
        _events: [Object],
        _maxListeners: 10,
        path: '/tmp/13278-s8mnog.zip',
        fd: 19,
        writable: false,
        flags: 'w',
        encoding: 'binary',
        mode: 438,
        bytesWritten: 49127,
        busy: true,
        _queue: [Object],
        _open: [Function],
        drainable: true } } }

I tested this in formidable and got the same thing.

node v0.8.17

Here's the upload code:

if (up.pathname === '/upload' && request.method === 'POST') {

        var form = new multiparty.Form({autoFields:true,autoFiles:true});

        form.on('error', function(err) {
            console.log(err);
        });

        form.on('close', function() {
        });

        form.on('file', function(name, file) {
            console.log(file);
        });

        form.parse(request, function(err, fields, files) {
            console.log('---------------NEW UPLOAD REQUEST---------------');
            console.log('username '+fields.username);
            console.log('password '+fields.password);
            console.log('FILES:');
            console.log(files);
        });

}

Wrong fields count

I'm setting maxFields to 1 and sending one file field and it gives me "Error: maxFields 1 exceeded.".

I'm not sure what it is. I have not checked it yet.

Also, when I set it to 2 and check all received fields (files and fields), indeed there is only one file and zero fields.

Parser seems to truncate values over 0x7F from body attribs

(Originally posted here: expressjs/express#1855)

Below is a small illustration. In both cases I'm sending 4 bytes. In the first case (which has a value over 0x7F), the size is incorrectly reported as '1'. In the second case (no values over 0x7F), it is correctly reported as '4' and all the data is intact.

Server:

app.post('/api', multipartMiddleware, function(req, res){
    var body = req.body.name;
    console.log('body',req.body);
    console.log('req body len', body.length);
    res.send("respond with a resource");
});

Python test client:

>>> requests.post('http://localhost:3000/api', files={'file':open('foo','rb')}, data={'name':pack('!HH',0xF0,0x0)})
<Response [200]>
>>> requests.post('http://localhost:3000/api', files={'file':open('foo','rb')}, data={'name':pack('!HH',0x0F,0x0)})
<Response [200]>

Express output:

Express server listening on port 3000
body { name: '\u0000' }
req body len 1
POST /api 200 9ms - 23b

body { name: '\u0000\u000f\u0000\u0000' }
req body len 4
POST /api 200 3ms - 23b

File Size Exceeded while autoFiles: true

I'm noticing its still streaming file (or at least a part of it) to the disk even when the error is thrown, however the on('file'...) event isn't being thrown out so I am unable to find out what the file on the disk is being written from the program; so I can't go and delete it?

multiparty stream upload with pkgcloud

I'm using version 2.1.5 with node 0.8.22 to attempt to upload files to rackspace through my server without storing them on the server disk first. My upload handler (coffeescript) code looks like this:

app.post "/images", (req, res, next) ->
  uris = []

  form = new multiparty.Form()
  form.on 'part', (part) ->
    return unless part.filename?
    path = 'somerandompath'
    logger.debug "received part: #{part.filename}, uploading to rackspace at: #{path}"

    rackspace.upload({container: rackspaceImageContainer, remote: path, stream: part}, (err) ->
      return next err if err?
      uri = rackspaceCdnBaseUrl + "/#{path}"
      uris.push uri)

  form.on 'error', (err) ->
    next new Error err

  form.on 'close', ->
    res.send uris

  form.parse req

The code hits the debug "received part" output, but the file never gets created on rackspace and the request is eventually aborted from the client side. I have also tried removing setting the stream from the options and piping directly to upload function as shown in the examples:

 part.pipe rackspace.upload({....

To no avail. What am I missing?

Content-Type Issue

I'm working with file uploads from Titanium. I spend a lot of type trying to find out the solution for the error:

unrecognized content-type:multipart/form-data; charset=utf-8; boundary=0xKhTmLbOuNdArY-019828C5-31F2-414B-9FC0-F68B537DD62D

Reading the source file I discover the match in line 104 uses CONTENT_TYPE_RE
which is set to
var CONTENT_TYPE_RE = /^multipart/(form-data|related);\s+boundary=(?:"([^"]+)"|([^;]+))$/i;

I found the solution for this error was as simple as changing this variable to :
var CONTENT_TYPE_RE = /^multipart/(form-data|related);\s+charset=utf-8;\s+boundary=(?:"([^"]+)"|([^;]+))$/i;

Not sure if this is an issue caused by a non-standard header from Titanium but I suspect its something this lib should add.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.