mixmaxhq / batchelor Goto Github PK

Google is deprecating BATCH endpoints soon
https://developers.googleblog.com/2018/03/discontinuing-support-for-json-rpc-and.html

Does this library use the /batch/library/v1 suffix so it will continue to work after August 2020?

By looking at the source code, it seems like this is similiar to

Does this library use /batch endpoints at all, or it is just special sauce to run multiple rest api calls in series or parallel and return the final output?

Thanks for the help!

https://www.npmjs.com/package/request

Any plans to refactor? Thanks.

This may not be a bug on your part, just wondering if you've seen this? We've been using the access_token acquired by the usual oauth2 exchange auth code for tokens flow, server side. Is this even the correct token to use here? It seems no matter what we put in for Bearer we get this response. I would expect a 400 or 401 response, not { parts: [], errors: 0 } Thanks for any info you can provide!

https://github.com/wapisasa/batchelor/blob/master/lib/batchelor.js#L49

the type is listed as multipart/mixed; -- NOTE: trailing semi-colon

I get 400 error from Google Calendar Batch API, but if I override the value without the semi-colon it works fine.

Hi,

Great package btw

If invalid json is returned in the part.data.body then JSON.parse throws an error. It should be wrapped in a try catch. Example:

//  Parse response JSON
if (part.data.body) {
  try {
	  part.data.body = JSON.parse(part.data.body);
  } catch (e) {
	  console.error(e); 
  }
}

The error was happening when I was trying to label a old gmail message. It works most of the time. Google must be returning something strange in certain instances.

"err": {
        "message": "Unexpected token N in JSON at position 0",
        "name": "SyntaxError",
        "stack": "SyntaxError: Unexpected token N in JSON at position 0\n    at JSON.parse (<anonymous>)\n    at PartStream.<anonymous> (/src/node_modules/batchelor/lib/batchelor.js:223:33)\n    at emitNone (events.js:91:20)\n    at PartStream.emit (events.js:186:7)\n    at endReadableNT (_stream_readable.js:974:12)\n    at _combinedTickCallback (internal/process/next_tick.js:74:11)\n    at process._tickDomainCallback (internal/process/next_tick.js:122:9)"
    }

To replicate yourself, change the following on line 223

JSON.parse('make this bad json' + part.data.body);

Hope you able to fix. I can raise a PR if needed, but felt the change was simple enough to be done by established committers.

Regards

Dan

Hey guys, I just switched to latest to get the concurrency fix, but getting this:

/server/auth/node_modules/batchelor/lib/batchelor.js:167
boundary = boundaryRegex.exec(responseObj.headers['content-type'])[2],
^
TypeError: Cannot read property '2' of null
at Request. (/server/auth/node_modules/batchelor/lib/batchelor.js:167:73)
at Request.emit (events.js:107:17)
at Request.onRequestResponse (/server/auth/node_modules/request/request.js:1139:10)
at ClientRequest.emit (events.js:107:17)
at HTTPParser.parserOnIncomingClient as onIncoming
at HTTPParser.parserOnHeadersComplete (_http_common.js:111:23)
at TLSSocket.socketOnData (_http_client.js:310:20)
at TLSSocket.emit (events.js:107:17)
at readableAddChunk (_stream_readable.js:163:16)
at TLSSocket.Readable.push (_stream_readable.js:126:10)

I have verfied the content-type header is present and valid. If I switch back to older version of Batchelor and make same request, it parses fine.

This may not be a bug on your part, just wondering if you've seen this? We've been using the access_token acquired by the usual oauth2 exchange auth code for tokens flow, server side. Is this even the correct token to use here? It seems no matter what we put in for Bearer we get this response. I would expect a 400 or 401 response, not { parts: [], errors: 0 } Thanks for any info you can provide!

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: Cannot find preset's package (local>mixmaxhq/renovate-config)

In case you hadn't seen the PR #15 :)

Thanks for work on Batchelor!

Hello,
If i'm using the following code, everything is ok:

const google = require('googleapis');
const OAuth2Client = google.auth.OAuth2;
const tokenGA = {
    "access_token": "ya29.XXXX",
    "expires_in": 3600,
    "refresh_token": "YYYY",
    "token_type": "Bearer",
    "created": 1485874552
};
const oAuth2ClientGA = new OAuth2Client(webGA.client_id, webGA.client_secret, webGA.redirect_uris[0]);
const analytics = google.analytics({version: 'v3',auth: oAuth2ClientGA});
oAuth2ClientGA.setCredentials(tokenGA);
analytics.data.ga.get({
        'ids': VIEW_ID,
        'start-date': '14daysAgo',
        'end-date': 'today',
        'metrics': 'ga:users,ga:sessions,ga:bounceRate,ga:avgSessionDuration,ga:goalCompletionsAll,ga:pageviewsPerSession,ga:uniquePageviews,ga:goalConversionRateAll',
        'dimensions': 'ga:date,ga:hour,ga:channelGrouping',
        'filters': 'ga:date=~' + _today + '|' + _1weekAgo + '|' + _2weeksAgo + ';ga:sessions>0'
    }, function (err, response) {
        if (err) {
            next(err);
        }
        res.send(response);
    });

But with Batchelor i always get an error
This is the code i used:
```
var Batchelor = require('batchelor');

var batch = new Batchelor({
    'uri':'https://www.googleapis.com/batch',
    'method':'POST',
    'auth': {
        'bearer': tokensGA.access_token
    },
    'headers': {
        'Content-Type': 'multipart/mixed'
    }
});

batch.add({
    'method':'GET',
    'path':'/analytics/v3/data/ga?ids='+ VIEW_ID+ '&start-date=14daysAgo&end-date=today&metrics=ga:users,ga:sessions,ga:bounces,ga:sessionDuration,ga:goalCompletionsAll,ga:pageviewsPerSession,ga:uniquePageviews,ga:pageViews&dimensions=ga:date,ga:hour&filters=ga:date=~' + _today + '|' + _1weekAgo + '|' + _2weeksAgo + ';ga:sessions>0'
});

I'm getting the following response:

{ parts:
[ { statusCode: '401',
statusMessage: 'Unauthorized',
headers: [Object],
body: [Object] } ],
errors: 0 }

Thanks in advance for any help you can provide, really appreciate it.

Currently run() throws exceptions when errors happen. See https://github.com/wapisasa/batchelor/blob/master/lib/batchelor.js#L167. It should call: callback(new Error('..')) instead. The result should be the second parameter of the callback.

I'm running a HTTP server that accepts user requests. So I'm trying to use multiple instances of Batchelor to service different users. However, the API is entirely singleton, so calling Batchelor.init() with a different user will overwrite the previous call.

What's your recommended way to create two instances of Batchelor at the same time?

Bathes are limited to 100 requests per batch, will bachelor automatically split requests?

If no, then it could be good feature,

Hello,

Using dicer dependency may result in crashes, according to npm when running npm install with batchelor 2.0.2 in package.json:

dicer  *
Severity: high
Crash in HeaderParser in dicer - https://github.com/advisories/GHSA-wm7h-9275-46v2
No fix available
node_modules/dicer
  batchelor  *
  Depends on vulnerable versions of dicer
  node_modules/batchelor

Please check the corresponding issue opened in dicer repo. A PR on dicer is opened since more than one year, a quick fix should not be expected on that side...

Hey @jamesmhaley 👋 thanks for your great work on this library!

The official Node JS Google client still doesn't support the batching endpoint 😞 so there's still a need for a lib such as this.

Currently, we use the Node JS Google client to interact with the the Google Calendar API. I've been using Nock to mock responses in our tests. Nock is also used for the tests in the official Google client too.

Unfortunately, Nock doesn't seem to pick up requests made by batchelor. I can see batchelor is using the request lib, which seems as though it should be compatible with nock.

Do you have any suggestions as to why Nock isn't picking up batchelor's requests?

I can provide an example to reproduce if that's helpful.

As an aside, how actively are you looking to maintain this lib? Would be keen to make some contributions if this is still being actively worked on! 👍