jaydenseric / graphql-upload

Middleware and an Upload scalar to add support for GraphQL multipart requests (file uploads via queries and mutations) to various Node.js GraphQL servers.

graphql-upload

Middleware and an Upload scalar to add support for GraphQL multipart requests (file uploads via queries and mutations) to various Node.js GraphQL servers.

Clients implementing the GraphQL multipart request spec upload files as Upload scalar query or mutation variables. Their resolver values are promises that resolve file upload details for processing and storage. Files are typically streamed into cloud storage but may also be stored in the filesystem.

Installation

Note

First, check if there are GraphQL multipart request spec server implementations (most for Node.js integrate graphql-upload) that are more suitable for your environment than a manual setup.

To install graphql-upload and its graphql peer dependency with npm, run:

npm install graphql-upload graphql

Use the middleware graphqlUploadKoa or graphqlUploadExpress just before GraphQL middleware. Alternatively, use the function processRequest to create custom middleware.

A schema built with separate SDL and resolvers (e.g. using the function makeExecutableSchema from @graphql-tools/schema) requires the Upload scalar to be setup.

Then, the Upload scalar can be used for query or mutation arguments. For how to use the scalar value in resolvers, see the documentation in the module GraphQLUpload.mjs.

Examples

• Apollo upload examples repo; a full stack demo of file uploads via GraphQL mutations using apollo-server-koa and @apollo/client.

Tips

• The process must have both read and write access to the directory identified by os.tmpdir().
• The device requires sufficient disk space to buffer the expected number of concurrent upload requests.
• Promisify and await file upload streams in resolvers or the server will send a response to the client before uploads are complete, causing a disconnect.
• Handle file upload promise rejection and stream errors; uploads sometimes fail due to network connectivity issues or impatient users disconnecting.
• Process multiple uploads asynchronously with Promise.all or a more flexible solution such as Promise.allSettled where an error in one does not reject them all.
• Only use the function createReadStream before the resolver returns; late calls (e.g. in an unawaited async function or callback) throw an error. Existing streams can still be used after a response is sent, although there are few valid reasons for not awaiting their completion.
• Use stream.destroy() when an incomplete stream is no longer needed, or temporary files may not get cleaned up.

Architecture

The GraphQL multipart request spec allows a file to be used for multiple query or mutation variables (file deduplication), and for variables to be used in multiple places. GraphQL resolvers need to be able to manage independent file streams. As resolvers are executed asynchronously, it’s possible they will try to process files in a different order than received in the multipart request.

busboy parses multipart request streams. Once the operations and map fields have been parsed, Upload scalar values in the GraphQL operations are populated with promises, and the operations are passed down the middleware chain to GraphQL resolvers.

fs-capacitor is used to buffer file uploads to the filesystem and coordinate simultaneous reading and writing. As soon as a file upload’s contents begins streaming, its data begins buffering to the filesystem and its associated promise resolves. GraphQL resolvers can then create new streams from the buffer by calling the function createReadStream. The buffer is destroyed once all streams have ended or closed and the server has responded to the request. Any remaining buffer files will be cleaned when the process exits.

Requirements

Supported runtime environments:

• Node.js versions ^18.15.0 || >=20.4.0.

Projects must configure TypeScript to use types from the ECMAScript modules that have a // @ts-check comment:

• compilerOptions.allowJs should be true.
• compilerOptions.maxNodeModuleJsDepth should be reasonably large, e.g. 10.
• compilerOptions.module should be "node16" or "nodenext".

Exports

The npm package graphql-upload features optimal JavaScript module design. It doesn’t have a main index module, so use deep imports from the ECMAScript modules that are exported via the package.json field exports:

• GraphQLUpload.mjs
• graphqlUploadExpress.mjs
• graphqlUploadKoa.mjs
• processRequest.mjs
• Upload.mjs

graphql-upload's Issues

is there any example for express?

I am trying to find the way to upload a file using your package but I don't know how to define the schema and resolver, also how to test if it works (normally I test it on graphiql).

please, anyone can provide an example for me.

Payload Verification?

This may seem like a silly question but I am rather new to this lib. Are there any checks that take place to verify that the only files received by the server are those purposefully sent by the client?

I'm imagining a scenario where a MITM could try to add files to the request that do not match what was extracted from the query string.

Absolutely love this application of graphql!

Problem with uploading of identical files

When I am trying to upload 2 identical files with different names in the same request, apollo-upload-server doesn't handle it well. This is probably due strange behavior of busboy. Problematic piece of code:

/src/index.mjs

const upload = new Promise(resolve =>
  busboy.on('file', (fieldName, stream, filename, encoding, mimetype) =>
    fieldName === mapFieldName && resolve({ stream, filename, mimetype, encoding })
  )
)

This statement (mapFieldName && resolve) always returns false when it parses the second file, so the promise never gets resolved.

Is there any way to solve this issue?

Thanks for the cool library, loving it!

How to get working with mergeSchemas

Hello,
I am trying to get this working with graphql-tool's mergeSchemas feature. Here is a contrived example:

import { GraphQLServer } from 'graphql-yoga'
import {
  mergeSchemas,
  makeExecutableSchema,
} from 'graphql-tools';

const UploadScalar = makeExecutableSchema({
  typeDefs: `
    scalar Upload
    type Query {
      _uploadTest: Boolean
    }
    type Mutation {
      _uploadTest: Boolean
      updateProfilePicture(
        picture: Upload!
      ): Boolean
    }
  `,
  resolvers: {
    Mutation: {
      updateProfilePicture(_, args) {
        console.log(args)
        return {
          success: true
        }
      }
    }
  }
})
  const server = new GraphQLServer({
    schema: mergeSchemas({
      schemas: [UploadScalar]
    })
  })
  server.start({
    port: 5444
  }, () => console.log('Server is running on http://localhost:5444'))

Basically, the Upload does not get forwarded to the schema and resolvers. I just get an empty picture object from the args.

If I use UploadScalar directly for the schema (const server = new GraphQLServer({ schema: UploadScalar })), it works. Of course, I don't do this in a real-world scenario. I would be merging multiple remote schemas which may use apollo-upload-server.

Does anyone know of a way I could work around this?

"Must provide Document" Error

Hi there,
today I was trying to implement a Avatar Upload system for user profiles.
So I came across apollo-upload server and client.
After implementing it like shown in the examples I'm getting this error and dont know how to solve it
Error: Must provide document at invariant (/Users/tjarkkuehl/Desktop/semtexxx.de-website/server/node_modules/graphql/jsutils/invariant.js:18:11) at Object.validate (/Users/tjarkkuehl/Desktop/semtexxx.de-website/server/node_modules/graphql/validation/validate.js:58:34) at doRunQuery (/Users/tjarkkuehl/Desktop/semtexxx.de-website/server/node_modules/apollo-server-express/node_modules/apollo-server-core/src/runQuery.ts:143:30) at /Users/tjarkkuehl/Desktop/semtexxx.de-website/server/node_modules/apollo-server-express/node_modules/apollo-server-core/src/runQuery.ts:69:39 at <anonymous>

Server Setup

Index

import { apolloUploadExpress } from 'apollo-upload-server'

app.use(
    '/graphql',
    bodyParser.json(),
    apolloUploadExpress({uploadDir: './user_upload/'})
)

Schema

input Upload {
    name: String!
    type: String!
    size: Int!
    path: String!
}

type Mutation {
    uploadAvatar(file: Upload!) : Boolean!
}

Resolver

uploadAvatar: (parent, args, context) => {
    console.log('Worked!')
    return true
}

Client Setup

Index

import { createUploadLink } from 'apollo-upload-client'

const link = createUploadLink({
    uri: `http://${config.graphqlAddress}:${config.graphqlPort}/graphql`
})

export const apolloClient = new ApolloClient({
    link: from([
        link,
        authMiddleware,
        authAfterware
    ]),
    cache: new InMemoryCache(),
    connectToDevTools: true
})

Profile.vue

<input class="select_file_action"
                           type="file"
                           @change="tryChangeAvatar($event)">

this.$apollo.provider.defaultClient.mutate({
    mutation: gql`
    mutation ($file: Upload!) {
        uploadAvatar(file: $file)
    }`,
    variables: {
        file: target.files[0]
    }
})

Using GridFS

Hi guys, awesome project, i used it with the default conf and works great. But... im dealing with a new path and it is to use GridFS.

How i can connect it to GridFS ?.

Maybe im a bit loose.

Sorry im asking it here.

Error Handling?

There seems to be no error handling. I ran into an error where I had set the files limit to 1 and was trying to upload multiple files and busboy wasn't emitting an event but apollo-upload-server was create a promise that would never resolve and the request would start hanging.

1. We fist need to handle the special limits events in busboy here,
   https://github.com/mscdex/busboy#busboy-special-events
   and reject the promise if any of those are raised.
2. And the inner promise should reject if the mapFieldname does not match the fieldName,

if (fieldName === mapFieldname) { // }
else { reject(new Error(`Field name mismatches ${fieldName} !== ${mapFieldname}`));

3. I guess we should resolve the top level promise only on the busboy finish event.

And finally maybe the request.pipe/busboy might have some error events which need to consumed and reject the promise for safety sake.

Add possibility to rename Upload scalar

Hey,

It would be great to have a possibility to rename scalar. I tried this code, but it is still named Upload:

import { GraphQLUpload } from 'apollo-upload-server'

const imageTypeDefs = `
  scalar Image
`

const imageResolvers = {
  Image: GraphQLUpload
}

Gallery! ?

In the multipart example, could you provide and example type for "Gallery"?

Thanks!

Only absolute urls are supported in example. And get empty object istead of file.

In your example

Client logs:

When i'm trying to make same thing on my server and client. I got empty object istead of file :(

Any ideas?

How to use with Meteor GridFS?

Hello, awesome project !!

I am using apollo with meteor and I have successfully integrated your package with apollo and meteor.
Can you please give an example on how to use the internal meteor gridfs to save the received file?

Thank you

Files don't save

I get this in my resolve

Promise { { stream: FileStream { _readableState: [ReadableState], readable: true, domain: null, _events: [Object], _eventsCount: 2, _maxListeners: undefined, truncated: false, _read: [Function] }, filename: 'Captura de pantalla 2018-02-19 a la(s) 18.31.42.png', mimetype: 'image/png', encoding: '7bit' } }

Trouble with S3.putObject

I'm not sure if this is an issue with the apollo-upload-server, s3.putObject, or my code; but I'm hoping someone can get me pointed in the right direction.

I cloned the repo from https://github.com/jaydenseric/apollo-upload-examples and am running the client. I made a few tweaks to the client but largely the same. I'm trying to do some validation on a csv that gets uploaded, if it passes validation I would like to upload the file to s3. I posted an issue with some code on stackoverflow.

I find it surprising that you need to specify a file length for a stream, isn't the whole point that we don't know the length until it's done? The files that I'll be uploading are too big to keep in memory (multiple GB) so a stream seems like my only option here. I'm wondering if there is some substantial difference between the stream produced by the upload-server and fs.createReadStream()

MultipartParser.end(): stream ended unexpectedly: state = START

Hello,

I have a restify server and I use the apolloUploadExpress.

But it doesn'st work. I have two error:

The first one:

SyntaxError: Unexpected token u in JSON at position 0
    at JSON.parse (<anonymous>)
    at form.parse (/Users/berthel/developpement/pro/missions/gptw/web-reporting-api/node_modules/apollo-upload-server/dist/apollo-upload-server.js:29:25)
    at IncomingForm.<anonymous> (/Users/berthel/developpement/pro/missions/gptw/web-reporting-api/node_modules/formidable/lib/incoming_form.js:102:9)
    at emitOne (events.js:116:13)
    at IncomingForm.emit (events.js:211:7)
    at IncomingForm._error (/Users/berthel/developpement/pro/missions/gptw/web-reporting-api/node_modules/formidable/lib/incoming_form.js:290:8)
    at IncomingMessage.<anonymous> (/Users/berthel/developpement/pro/missions/gptw/web-reporting-api/node_modules/formidable/lib/incoming_form.js:132:14)
    at emitNone (events.js:111:20)
    at IncomingMessage.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1056:12)

The second one:

Error: MultipartParser.end(): stream ended unexpectedly: state = START
        at form.parse (/Users/berthel/developpement/pro/missions/gptw/web-reporting-api/node_modules/apollo-upload-server/dist/apollo-upload-server.js:25:25)
        at IncomingForm.<anonymous> (/Users/berthel/developpement/pro/missions/gptw/web-reporting-api/node_modules/formidable/lib/incoming_form.js:102:9)
        at emitOne (events.js:116:13)
        at IncomingForm.emit (events.js:211:7)
        at IncomingForm._error (/Users/berthel/developpement/pro/missions/gptw/web-reporting-api/node_modules/formidable/lib/incoming_form.js:290:8)
        at IncomingMessage.<anonymous> (/Users/berthel/developpement/pro/missions/gptw/web-reporting-api/node_modules/formidable/lib/incoming_form.js:132:14)
        at emitNone (events.js:111:20)
        at IncomingMessage.emit (events.js:208:7)
        at endReadableNT (_stream_readable.js:1056:12)
        at _combinedTickCallback (internal/process/next_tick.js:138:11)

I think the issue came from formidable but Have you any idea ?

Many thanks :)

if wrap next.js it can't upload the file

if wrap the next.js resolver can get file but can't save to Upload path
https://github.com/polo13999/apolloUpload

Hapi Integration

It would be great to have support for hapijs. I might take a look at this if I find the time.

Large image uploads crashing

Everything on this library has been working wonderfully, so first of all: thank you so much!

I noticed recently that we're failing on large image uploads (somewhere in the range of 3mb+) on the server side. Above is a log on the server.

Is this an inherent issue with post requests coming through gql? Or is there something we should be doing to handle larger images better?

Thank you!

hapi support?

hi does it support for hapi server?
thanks

formidable vs muddler

Hey, great package I got it working pretty quick!

One use case I think was missed is stream files directly to S3 Buckets without ever writing to the disk.

I've noticed it uses formidable which saves it to the disk without an option to hold it in memory but on the @danielbuechele’s Medium article he uses multer which has options for saving to the DiskStorage or MemoryStorage, was there a reason to choose formidable over multer as I see a big lost in flexibility?

FileMissingUploadError: File missing in the request

Please how do we make FileUpload optional?

Uploaded file variable expected "Promise" object, but "false".

currently, I'm using express to use apollo-upload-server middleware. but It works quite strange. the file variable in resolver has "false" value not a "promise".

I've checked with my own middleware to check how apolloUploadExpress(/* ... */) middleware set a variables object. and It successfully assigned promise but It changes from promise to false when it passed to resolver function.

right below code snippet is my own middleware code:

app.use("/graphql",
    bodyParser.json(),
    apolloUploadExpress(),
    (request, _, next) => {
        console.log(request.body.variables.thumbnail); // this prints 'Promise { ... }'
        next();
    },
    graphqlExpress({
        schema: require("./schema"),
    }),
);

and this is resolver code:

{
    push: async (_, args) => {
        // the 'thumbnail' variable accepted as file.
        const { thumbnail } = args;

        // so we could expect that right below code snippet will prints about "Promise" object.
        // but just prints "false".
        console.log(thumbnail);
    },
};

EDIT

I'm using both server and client in a same project.
and I've tested that Promise can be resolved in my own middleware function.
but it still pass a 'false' value instead of 'Promise' to resolver callback.

Could you support documentation on how to use this project with

https://reactql.org/docs/

I tried from your example but it I got hung up on scalars and the different way you defined your schema, were using graphql.js and it's not to easy trying to shoe-horn your library in.

How to use it with apollo-server-lambda?

Hi

This is super great, I am just wondering how I can use it with apollo-server-lambda since I am spinning up my graphql server on a lambda function.

Non-Apollo GraphQL Input Type

Would this be the correct way to do this using just normal graphql types?

server.js

app.use(
  '/graphql',
  bodyParser.json(),
  apolloUploadExpress({
    uploadDir: '/tmp/uploads',
  }),
  expressGraphQL({
    schema,
    graphiql: true,
  })
);

server-side input type:

const graphql = require('graphql');

const {
  GraphQLInputObjectType,
  GraphQLString,
  GraphQLInt,
  GraphQLNonNull,
} = graphql;

const UploadType = new GraphQLInputObjectType({
  name: 'UploadType',
  fields: () => ({
    name: { type: new GraphQLNonNull(GraphQLString) },
    type: { type: new GraphQLNonNull(GraphQLString) },
    size: { type: new GraphQLNonNull(GraphQLInt) },
    path: { type: new GraphQLNonNull(GraphQLString) },
  }),
});

module.exports = UploadType;

server-side mutation:

...
   uploadFile: {
      type: PartType,
      args: {
        file: {
          type: new GraphQLNonNull(UploadType),
        },
        owner: {
          type: new GraphQLNonNull(GraphQLString),
        },
      },
      resolve(rootValue, {
        file,
        owner,
      }) {
        console.log('uploading file...');
        console.log('rootValue: ', rootValue);
        console.log('file: ', file);
        console.log('owner: ', owner);

        return new Promise((resolve, reject) => {
          resolve(null);
        });
      },
    },
...

client side mutation:

import gpl from 'graphql-tag';

export default gpl`
  mutation UploadFile($file:UploadType!,$owner:String!){
    uploadFile(file:$file,owner:$owner){
      id
    }
  }
`;

client-side component function:

...
handleUploadFile({ target }) {
    const {
      UploadFile,
      owner,
    } = this.props;

    if (target.validity.valid) {
      const { name, type, size, path } = target.files[0];

      UploadFile({
        variables: {
          file: {
            name,
            type,
            size,
            path: path || '',
          },
          owner,
        },
      })
      .then(({ data }) => {
        console.log('ImagePartModal -> UploadFile', data);
      });
    }
  }
...

According to the docs, I should be able to send over the File object from the client side, but I'm unable to do that.

When I do this:

 ...
    UploadFile({
        variables: {
          file: target.files[0],
          owner,
        },
      })
     ...

I get an error message:

{"errors":[{"message":"Variable \"$file\" got invalid value {}.\nIn field \"name\": Expected \"String!\", found null.\nIn field \"type\": Expected \"String!\", found null.\nIn field \"size\": Expected \"Int!\", found null.\nIn field \"path\": Expected \"String!\", found null.","locations":[{"line":1,"column":21}]}]}

GraphQL operations is undefined

Hi, I'm having an odd case where the GraphQL operations are undefined when the multipart form request is parsed. We're using Google Cloud Functions at the moment. It works fine in my local GCF emulator but the deployed cloud version is always returning undefined.

The code in question is:

// Decode the GraphQL operation(s). This is an array if batching is
// enabled.
console.log( 'operations', operations )
operations = JSON.parse(operations)

As a result, the server logs the following:

...
SyntaxError: Unexpected token u in JSON at position 0 at Object.parse (native) at form.parse
...
Error: Request aborted at form.parse (/user_code/node_modules/@jstwrt/apollo-upload-server/lib/main/index.js:58:25)

Line 58 being form.parse(request, (error, { operations }, files) => {

Has anyone else experienced this issue. If someone could lead me in the right direction, that'd be helpful, thanks.

How to use with AdonisJs?

Image extensions

Hello! Thanks for your work! I'm trying to use apollo-upload-server to upload images.

The question is how do I add extensions to uploaded files' filenames?

Provide file streams instead of metadata to resolvers

I will flesh out this description out in time. There are lots of little conversations about this hidden around.

Ideal scenario

A new user fills in a registration form. Fields submitted include username (String), password (String) avatar (File), and banner (File). The files are 4 MB each.

The non-file fields can be validated in the resolver as soon as the request comes in to the server, while the uploads asynchronously stream in. It turns out the username is already taken, so a validation error is thrown and the request is aborted. The user gets immediate feedback without having to wait for the whole upload to complete, and the server is not unnecessary burdened.

Because the files are streams in the resolver, they can be individually forwarded into cloud storage, skipping temporary storage on the API server's memory or filesystem. This allows the first avatar file to begin uploading to cloud storage while the second second banner file is still being received by the API. Because the files are not stored on the API server, data security compliance is simpler.

Thoughts

With this setup, a developer could even use different methods to store each file from a single request.

I think it is best to just substitute files for file streams in the resolvers; no more metadata. The API is less opinionated, easier to implement and less complicated to document. Users can extract whatever metadata is useful to the them from the streams.

To prevent file uploads from blocking the resolvers (as they do currently with formidable) we will probably need to provide a new files field in the multipart form sent from the client. This will contain a JSON array of all the file object paths, i.e. ["0.variables.avatar", "0.variables.banner"]. It, along with the existing operations fields should be the very first fields in the multipart form, will all the files following. This will allow us to build an operations object with placeholder file streams to then pass into the GraphQL server and the resolvers. The form can then continue streaming up in parallel to the resolvers running; each file that gets parsed can be streamed into the relevant placeholder stream constructed earlier.

I suspect that to engineer the new system we might have to write a custom multipart form parser, because we need to create streams before the file fields are parsed and options like formidable only create streams as they are encountered. We might also be better off writing the new functionality directly into the GraphQL server middlewares.

Also, I think when we move to streams we should start using a new word like "blob" instead of "file", because people will be able to use any sort of binary data that can't JSON encode in their variables. Relevant: jaydenseric/extract-files#2.

Support for Node 6 (needed for AWS Lambda)

Would be great if you could adjust the babel compilation settings in a way that the resulting output also runs in Node 6 (or even lower). Otherwise it's not possible to deploy apps using apollo-upload-server to AWS Lambda.

Not possible to use with TypeScript

Since apollo is written on typescript lot of people are using it with typescript. Solution is to rewrite this library in typescript with minimal changes which is easy to do or include and maintain typescript typings.

How to connect to gridFS?

I'm trying to implement apollo-upload-server (which is exactly what I'm needing) using gridFS. So I came up to #8 (comment)

But I've some problems how to get connected to the db at all in my express server (do I have to use middleware?), which looks like this:

import express from 'express'
import { graphqlExpress, graphiqlExpress } from 'apollo-server-express'
import { apolloUploadExpress } from 'apollo-upload-server'
import crossOriginRequests from './middlewares/cors'
import schema from './graphql/schema'

const app = express()

app.use('/graphql',
  bodyParser.text({ type: 'application/graphql' }),
  crossOriginRequests,
  apolloUploadExpress(),
  graphqlExpress(req => ({
    schema: schema,
    context: { token: req.headers.authorization }
  }))
)

app.listen(port, () => {
  if (process.env.NODE_ENV !== 'production') {
    console.log('Listening on port ' + port)
  }
})

export default app

In my resolver I would like to write the stream:

import { GraphQLUpload } from 'apollo-upload-server'

const processUpload = async upload => {
  const { stream, filename, mimetype, encoding } = await upload

  // use gfs here to write stream...

  return { id, filename, mimetype, encoding }
}

export default {
  Upload: GraphQLUpload,
  Mutation: {
    singleUpload: (obj, { file }) => processUpload(file)
  }
}

Need help with Express api upload via Insomania

Thanks @jaydenseric for this plugin, I'm working on Express API (mongodb, apollo server off course ) and using Insomania to test all my features.

Here is my code

index.js

const helperMiddleware = [
  bodyParser.json(),
  bodyParser.urlencoded({ extended: true }),
  bodyParser.text({ type: 'application/graphql' }),
  (req, res, next) => {
    if (req.is('application/graphql')) {
      req.body = { query: req.body }
    }
    next()
  }
]

const buildOptions = async (req, res) => {
  res.removeHeader("X-Powered-By") // remove x power headers
  const mongo = await connectMongo()
  const user = await authentication(req, res, mongo.Users)
  return {
    schema,
    context: {
      mongo,
      dataloaders: buildDataloaders(mongo),
      user
    },
    // tracing: true
  }
}

app.use(
  '/api',
  ...helperMiddleware,
  apolloUploadExpress({
    maxFileSize: 5 * 1000,
    maxFiles: 5
  }),
  graphqlExpress(buildOptions)
)

schema

type Image {
  # upload
  id: ID!
  path: String!
  filename: String!
  mimetype: String!
  encoding: String!
  # data
  caption: String
  created_by: User!
  created_date: Datetime
  width: Int
  height: Int
  url: String
  album: Album
  variants: [ImageVariant]
}

mutation schema

uploadImage(files: [Upload!]!): [Image!]

resolver

const imgDir = './public/uploads'
const storeUpload = async ({ stream, filename }) => {
  const id = uuid.v4()
  const path = `${imgDir}/${id}-${filename}`
  return new Promise((resolve, reject) =>
    stream
      .pipe(createWriteStream(path))
      .on('finish', () => resolve({ id, path }))
      .on('error', reject)
  )
}

const processUpload = async (upload, Images) => {
  const { stream, filename, mimetype, encoding } = await upload
  const { id, path } = await storeUpload({ stream, filename })
  const newImage = { id, filename, mimetype, encoding, path }
  const response = await Images.insert(newImage)
  return Object.assign({ id: response.insertedIds[0] }, newImage)
}

export const uploadImage = async (root, data, { mongo: { Images }, user }) => processUpload(data.files, Images)

Insomania
Query

mutation Upload($files: [Upload!]!) { 
	uploadImg(files: $files) {
		filename
		encoding
		mimetype
		path
	}
}

Expected

• Upload multiple images within maximum 5 files (and maximum 5MB for each)
• Insert data to db

Result

Non-stop sending data request in Insomania

I don't know what I missed, would you please help me out, I'm kind of newbie.

Thank you.

App reload itself after uploading done

I'm not really sure what happen to my client app (frontend)..
If the resolver in the backend perform await processUpload(...) and succeed the client will reload itself.

It's hard to debug what the response from the backend was.
Because using Chrome Devtools, the network tab also reloaded.

Does this package sendRefresh HTTP Header or something else happen in the backend that cause this?

PS: I'm using express with middlewares: (in order) bodyParser.json() bodyParser.urlencoded({ extended: true }) cookieParser(theSecret) compress() methodOverride() session({..}) passport.initialize() passport.session() helmet() cors() expressWinston.logger({..}) rest graphql graphiql

"Unexpected token {", due to processRequest function definition syntax?

Hi! First, thank you so much for this and the client module. I'm using Node v7.5.0 edit: correction, seems to be Node v4.8.4, inspecting process.version at runtime and trying to add them to our Meteor based app and hit an early stumbling block with apollo-upload-server v2.0.4. I add with meteor npm install --save apollo-upload-server and import as per your documentation, but the server fails to start with:

W20171013-09:34:07.929(7)? (STDERR) Debugger listening on [::]:45969
W20171013-09:38:24.349(7)? (STDERR) packages/modules.js:19849
W20171013-09:38:24.351(7)? (STDERR) function processRequest(request, { uploadDir } = {}) {
W20171013-09:38:24.353(7)? (STDERR)                                  ^
W20171013-09:38:24.354(7)? (STDERR) 
W20171013-09:38:24.355(7)? (STDERR) SyntaxError: Unexpected token {
W20171013-09:38:24.356(7)? (STDERR)     at Object.exports.runInThisContext (vm.js:53:16)
W20171013-09:38:24.357(7)? (STDERR)     at /home/me/app/.meteor/local/build/programs/server/boot.js:332:30
W20171013-09:38:24.358(7)? (STDERR)     at Array.forEach (native)

Setting a breakpoint for where this is being thrown, I see runInThisContext is part of the meteor bootstrapping process, and has:

var func = require('vm').runInThisContext(wrapped, scriptPath, true);

where wrapped is a large block of javascript code containing apollo-upload-server, containing:

function processRequest(request, { uploadDir } = {}) {
  // Ensure provided upload directory exists
  if (uploadDir) mkdirp.sync(uploadDir);

It looks like the { uploadDir } = {} syntax is causing my node system to throw. Is this new or in any way nonstandard syntax? I'm fairly new to the latest es6 language features like destructuring. Before adding your modules, the es6 style Meteor based server was running fine.

File uploads but getting undefined JSON error

Hi Jayden, thanks for making this awesome package! I'm running into this weird issue, my actual file gets uploaded, but I get this error:

_SyntaxError: Unexpected token u in JSON at position 0
at JSON.parse ()
at form.parse (/Users/vlady/WebProjects/yupty-api/node_modules/apollo-upload-server/src/index.js:20:31)
at IncomingForm. (/Users/vlady/WebProjects/yupty-api/node_modules/formidable/lib/incoming_form.js:102:9)
at emitOne (events.js:96:13)
at IncomingForm.emit (events.js:188:7)
at IncomingForm._error (/Users/vlady/WebProjects/yupty-api/node_modules/formidable/lib/incoming_form.js:290:8)
at IncomingMessage. (/Users/vlady/WebProjects/yupty-api/node_modules/formidable/lib/incoming_form.js:120:12)
at emitNone (events.js:86:13)
at IncomingMessage.emit (events.js:185:7)
at abortIncoming (_http_server.js:284:11)
at Socket.serverSocketCloseListener (_http_server.js:297:5)
at emitOne (events.js:101:20)
at Socket.emit (events.js:188:7)
at TCP._handle.close [as onclose] (net.js:501:12)

My server looks like this:

app.use('/graphql',
	authenticate,
	bodyParser.json(),
	// Uploads
	apolloUploadExpress({
		// Optional, defaults to OS temp directory
		uploadDir: './uploads',
	}),
	graphqlExpress(req => {
		const query = req.query.query || req.body.query;
		if (query && query.length > 2000) {
			// Probably indicates someone trying to send an overly expensive query
			throw new Error('Query too large.');
		}

		return {
			schema: makeExecutableSchema({
				typeDefs: schema,
				resolvers,
			}),
			context: {
				user: req.user,
				token: req.headers.authorization,
			},
		};
	}),
);

I've been trying to fix and I have no idea what's going on...

The query on the client looks like this:

const CreateEventFormWithMutation = graphql(createEventMutation, {
	props: ({ ownProps, mutate }) => ({
		// This function will be available as a prop in the form:
		createEvent: ({
			venue,
			start_time,
			description,
			picture,
		}, artistIDs, genreIDs) => {
			const variables = {
				venue_id: venue.id,
				start_time,
				description,
				artistIDs,
				genreIDs,
			};

			// Add picture to variables if there is one
			if (Object.getOwnPropertyNames(picture).length > 0) {
				variables.picture = picture;
			}

			return mutate({
				variables,
				// Update event feed with data from form, no need for network request
				optimisticResponse: {
					__typename: 'Mutation',
					createEvent: {
						__typename: 'Event',
						// Note that we can access the props of the container at `ownProps`
						venue,
						start_time,
						artists: ownProps.newEvent.artists,
						description,
					},
				},
				// Name the query to update, in this case: the query from EventFeedContainer, "getEvents"
				updateQueries: {
					getEvents: (prev, { mutationResult }) => {
						return update(prev, {
							events: {
								$unshift: [mutationResult.data.createEvent],
							},
						});
					},
				},
			});
		},
	}),
})(CreateEventForm);

mimetype is always 'application/octet-stream'

The mimetype always appear to be defined as 'application/octet-stream'

File size

Hey,

is there a way to get the length of the stream? (size of the uploaded file)

😄

Remove Babel from production dependencies

Thanks for the great module!

Not sure if intentional, but package.json references the "@babel/runtime": "^7.0.0-beta.39",.
My Lambda (or microservice) definitely do not want that big thing in the memory on on the disk.

Is it removable?

Understanding options

Hello,

I want to thank you for this package. But I have a few questions:

I wanted to know what do you mean with this option:
maxFieldSize: Max allowed non-file multipart form field size in bytes

Also, is it possible to have an example on how can I test this in the server side (automatic tests)?

Thanks.

Cannot upload image on localhost server

I am trying to upload an image on my graphql server. The way I am doing is as follows;

My component looks like this;

	handleImageUpload = async ([file]) => {
		console.log('file: ', file);
		const res = await this.props.uploadImageMutation({
			variables: { file },
		});
		console.log('res: ', res);
	};
	render() {
		const { children, disableClick } = this.props;
		return (
			<Dropzone className="ignore" onDrop={this.handleImageUpload} disableClick={disableClick}>
				{children}
			</Dropzone>
		);
	}
}

const UPLOAD_IMAGE_MUTATION = gql`
	mutation UploadImageMutation($file: Upload!) {
		singleUpload(file: $file)
	}
`;

export default graphql(UPLOAD_IMAGE_MUTATION, { name: 'uploadImageMutation' })(FileUpload);

Schema:

scalar Upload

type Mutation {
	singleUpload(file: Upload!): Boolean!
}

And resolver is as follow;

{
       Upload: GraphQLUpload,
       Mutation: {
              singleUpload: async (parent, { file }) => {
                 const fil = await file;
                 return true;
              },
       }
}

When I upload the image I get the response true but cannot see the upload file on my server. If someone could point out what I am doing wrong that would be really helpful. Thanks!

[AWS Lambda] SyntaxError: Unexpected end of JSON input

Hey there! 👋

First off, thanks for all the work you're doing for us, @jaydenseric! 🙏

So, I'm trying to use apollo-upload-server on AWS Lambda using Serverless and aws-serverless-express.

It works fine when there's no file being uploaded, but when a file is uploaded, I get the following error:

SyntaxError: Unexpected end of JSON input
at Object.parse (native)
at Busboy.parser.on (/var/task/node_modules/apollo-upload-server/lib/middleware.js:66:29)
at emitMany (events.js:127:13)
at Busboy.emit (events.js:201:7)
at Busboy.emit (/var/task/node_modules/busboy/lib/main.js:38:33)
at PartStream.onEnd (/var/task/node_modules/busboy/lib/types/multipart.js:261:15)
at emitNone (events.js:91:20)
at PartStream.emit (events.js:185:7)
at Dicer.onPart (/var/task/node_modules/busboy/lib/types/multipart.js:120:13)
at emitOne (events.js:96:13)
at Dicer.emit (events.js:188:7)
at Dicer.emit (/var/task/node_modules/dicer/lib/Dicer.js:80:35)
at Dicer._oninfo (/var/task/node_modules/dicer/lib/Dicer.js:181:12)
at SBMH.<anonymous> (/var/task/node_modules/dicer/lib/Dicer.js:127:10)
at emitMany (events.js:127:13)
at SBMH.emit (events.js:201:7)

The request payload from apollo-upload-client looks like this:

------WebKitFormBoundarymnT1gqXGJmeSwDDv
Content-Disposition: form-data; name="operations"

{"operationName":"ShareMutation","variables":{"state":"SHARE","text":"Hey!","image":null,"childPosts":[{"accountId":"cjekhdqfl2m7d0104fh88d8r8","state":"SHARE","contentSameAsParentPost":true}]},"query":"mutation ShareMutation($state: POST_STATE!, $text: String, $image: Upload, $childPosts: [ChildPostInput!]) {\n  share(state: $state, text: $text, image: $image, childPosts: $childPosts) {\n    id\n    state\n    text\n    image {\n      id\n      size\n      contentType\n      url\n      __typename\n    }\n    errors {\n      id\n      code\n      name\n      message\n      __typename\n    }\n    childPosts {\n      id\n      account {\n        id\n        __typename\n      }\n      state\n      contentSameAsParentPost\n      text\n      image {\n        id\n        size\n        contentType\n        url\n        __typename\n      }\n      errors {\n        id\n        code\n        name\n        message\n        __typename\n      }\n      __typename\n    }\n    __typename\n  }\n}\n"}
------WebKitFormBoundarymnT1gqXGJmeSwDDv
Content-Disposition: form-data; name="map"

{"0":["variables.image"]}
------WebKitFormBoundarymnT1gqXGJmeSwDDv
Content-Disposition: form-data; name="0"; filename="blob"
Content-Type: image/png


------WebKitFormBoundarymnT1gqXGJmeSwDDv--

I'm also setting multipart/form-data as a Binary Media Type in both AWS API Gateway and aws-serverless-express.

I'm using the latest versions of apollo-upload-server and apollo-upload-client.

Thanks for your time! 😃

JavaScript Schema

I'm currently using a JavaScript schema and I don't want to rewrite as GraphQL, Your examples don't show how to implement the Upload scalar using JavaScript.
Are you able to do an example showing how to implement it using JavaScript?
It will help me and probably other people.

"SyntaxError: Unexpected token {"

Just tried to import "apollo-upload-server" in a new blank meteor project (v1.5) and it still throwing this error:
error "function processRequest(request, { uploadDir } = {}) {"
"SyntaxError: Unexpected token {"

The error is shown when trying to import 'apollo-upload-server' to 'server/main.js'.
Any thoughts?

Resolver example in the README

Thanks for the great piece of tech!

I'm new to GraphQL and Apollo, thus having trouble understanding.
Where would I put my code which streams data to S3?

Should it be a mutation or something?

Would be great to have a typical file streaming example in the README to avoid similar questions from newbies.

Cheers

Support regular clients again

I guess operations here would be undefined when using a client other than apollo-upload-client

undefined:1
undefined
^

SyntaxError: Unexpected token u in JSON at position 0
at Object.parse (native)
at form.parse (/Users/t/apps/thai.run-api/node_modules/apollo-upload-server/dist/apollo-upload-server.js:29:25)
at IncomingForm. (/Users/t/apps/thai.run-api/node_modules/formidable/lib/incoming_form.js:102:9)
at emitOne (events.js:96:13)
at IncomingForm.emit (events.js:188:7)
at IncomingForm._error (/Users/t/apps/thai.run-api/node_modules/formidable/lib/incoming_form.js:290:8)
at IncomingMessage. (/Users/t/apps/thai.run-api/node_modules/formidable/lib/incoming_form.js:120:12)
at emitNone (events.js:86:13)
at IncomingMessage.emit (events.js:185:7)
at abortIncoming (_http_server.js:283:11)
at Socket.serverSocketCloseListener (_http_server.js:296:5)
at emitOne (events.js:101:20)
at Socket.emit (events.js:188:7)
at TCP._handle.close [as _onclose] (net.js:498:12)

ECMAScript module loading error

Thank you for your great package. I have one question:

When using the program with ESM (.mjs import) I get an error with the current version of Node (9.4.0).

(node:4290) ExperimentalWarning: The ESM module loader is experimental.
file:///Users/.../node_modules/apollo-upload-server/lib/types.mjs:1
import { GraphQLScalarType } from 'graphql'
         ^^^^^^^^^^^^^^^^^
SyntaxError: The requested module does not provide an export named 'GraphQLScalarType'
    at ModuleJob._instantiate (internal/loader/ModuleJob.js:86:19)
    at <anonymous>
error Command failed with exit code 1.

The problem should be resolved by changing the following lines in types.mjs:

import graphql from 'graphql'

const { GraphQLScalarType } = graphql

I think GraphQL exports currently only with CJS and not yet with ESM and therefore there is only the "default export". Since you are running this program with Babel, you will not get an error. I use the package directly with Node.

https://nodejs.org/api/esm.html#esm_interop_with_existing_modules

Auto delete file from directory?

I think if there was an error thrown later in the resolver (or other middleware), the uploaded files should be deleted (optionally).

MaxFileSizeUploadError can't be caught and crashes the server

Hi!

I have a problem with a custom middleware and the use of the processRequest function.
Here is my custom middleware:

try {
    return processRequest(request, options)
                    .then(body => {
                        console.log("body is : ", body);
                        ctx.request.body = body;
                        return next();
                    })
                    .catch(error => {
                        console.log("Error catched : ", error);
                        if (error.status && error.expose) response.status(error.status);
                        return next(error)
                    });
} catch (error) {
    console.log("Error catched bis : ", error);
}

If I try to upload a file bigger than the allowed limit, I get the following error and my app crash.

events.js:137
      throw er; // Unhandled 'error' event
      ^

MaxFileSizeUploadError: File truncated as it exceeds the size limit.
    at FileStream.file.stream.once (/Volumes/lake/projects/evibe/api-lovejs/node_modules/apollo-upload-server/lib/middleware.js:35:13)
    at Object.onceWrapper (events.js:255:19)
    at FileStream.emit (events.js:160:13)

I can't capture the error.
Any idea?

Thanks :)

Cannot name file

I don't see a way to name the files as they are uploaded, they just get a name of "upload_" is there a way to change that, or append an extension?

file extension problem

First of all, Thanks. This is an amazing work. My problem is that When a file is uploaded the extension is removed. Why is that? I can see from the source code that it uses formidable. Making the keepExtensions: true inside the IncomingForm options makes it work. if we could set the keepExtensions field like the uploadDir filed it would be awesome.