mixmaxhq / batchelor Goto Github PK
View Code? Open in Web Editor NEWA lovely little Node.js module to perform batch requests with the Google REST API
Home Page: https://www.npmjs.com/package/batchelor
License: MIT License
A lovely little Node.js module to perform batch requests with the Google REST API
Home Page: https://www.npmjs.com/package/batchelor
License: MIT License
Google is deprecating BATCH endpoints soon
https://developers.googleblog.com/2018/03/discontinuing-support-for-json-rpc-and.html
Does this library use the /batch/library/v1
suffix so it will continue to work after August 2020?
By looking at the source code, it seems like this is similiar to
Does this library use /batch endpoints at all, or it is just special sauce to run multiple rest api calls in series or parallel and return the final output?
Thanks for the help!
https://www.npmjs.com/package/request
Any plans to refactor? Thanks.
This may not be a bug on your part, just wondering if you've seen this? We've been using the access_token acquired by the usual oauth2 exchange auth code for tokens flow, server side. Is this even the correct token to use here? It seems no matter what we put in for Bearer we get this response. I would expect a 400 or 401 response, not { parts: [], errors: 0 } Thanks for any info you can provide!
https://github.com/wapisasa/batchelor/blob/master/lib/batchelor.js#L49
the type is listed as multipart/mixed;
-- NOTE: trailing semi-colon
I get 400 error from Google Calendar Batch API, but if I override the value without the semi-colon it works fine.
Hi,
Great package btw
If invalid json is returned in the part.data.body then JSON.parse throws an error. It should be wrapped in a try catch. Example:
// Parse response JSON
if (part.data.body) {
try {
part.data.body = JSON.parse(part.data.body);
} catch (e) {
console.error(e);
}
}
The error was happening when I was trying to label a old gmail message. It works most of the time. Google must be returning something strange in certain instances.
"err": {
"message": "Unexpected token N in JSON at position 0",
"name": "SyntaxError",
"stack": "SyntaxError: Unexpected token N in JSON at position 0\n at JSON.parse (<anonymous>)\n at PartStream.<anonymous> (/src/node_modules/batchelor/lib/batchelor.js:223:33)\n at emitNone (events.js:91:20)\n at PartStream.emit (events.js:186:7)\n at endReadableNT (_stream_readable.js:974:12)\n at _combinedTickCallback (internal/process/next_tick.js:74:11)\n at process._tickDomainCallback (internal/process/next_tick.js:122:9)"
}
To replicate yourself, change the following on line 223
JSON.parse('make this bad json' + part.data.body);
Hope you able to fix. I can raise a PR if needed, but felt the change was simple enough to be done by established committers.
Regards
Dan
Hey guys, I just switched to latest to get the concurrency fix, but getting this:
/server/auth/node_modules/batchelor/lib/batchelor.js:167
boundary = boundaryRegex.exec(responseObj.headers['content-type'])[2],
^
TypeError: Cannot read property '2' of null
at Request. (/server/auth/node_modules/batchelor/lib/batchelor.js:167:73)
at Request.emit (events.js:107:17)
at Request.onRequestResponse (/server/auth/node_modules/request/request.js:1139:10)
at ClientRequest.emit (events.js:107:17)
at HTTPParser.parserOnIncomingClient as onIncoming
at HTTPParser.parserOnHeadersComplete (_http_common.js:111:23)
at TLSSocket.socketOnData (_http_client.js:310:20)
at TLSSocket.emit (events.js:107:17)
at readableAddChunk (_stream_readable.js:163:16)
at TLSSocket.Readable.push (_stream_readable.js:126:10)
I have verfied the content-type header is present and valid. If I switch back to older version of Batchelor and make same request, it parses fine.
This may not be a bug on your part, just wondering if you've seen this? We've been using the access_token acquired by the usual oauth2 exchange auth code for tokens flow, server side. Is this even the correct token to use here? It seems no matter what we put in for Bearer we get this response. I would expect a 400 or 401 response, not { parts: [], errors: 0 } Thanks for any info you can provide!
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
Error type: Cannot find preset's package (local>mixmaxhq/renovate-config)
In case you hadn't seen the PR #15 :)
Thanks for work on Batchelor!
Hello,
If i'm using the following code, everything is ok:
const google = require('googleapis');
const OAuth2Client = google.auth.OAuth2;
const tokenGA = {
"access_token": "ya29.XXXX",
"expires_in": 3600,
"refresh_token": "YYYY",
"token_type": "Bearer",
"created": 1485874552
};
const oAuth2ClientGA = new OAuth2Client(webGA.client_id, webGA.client_secret, webGA.redirect_uris[0]);
const analytics = google.analytics({version: 'v3',auth: oAuth2ClientGA});
oAuth2ClientGA.setCredentials(tokenGA);
analytics.data.ga.get({
'ids': VIEW_ID,
'start-date': '14daysAgo',
'end-date': 'today',
'metrics': 'ga:users,ga:sessions,ga:bounceRate,ga:avgSessionDuration,ga:goalCompletionsAll,ga:pageviewsPerSession,ga:uniquePageviews,ga:goalConversionRateAll',
'dimensions': 'ga:date,ga:hour,ga:channelGrouping',
'filters': 'ga:date=~' + _today + '|' + _1weekAgo + '|' + _2weeksAgo + ';ga:sessions>0'
}, function (err, response) {
if (err) {
next(err);
}
res.send(response);
});
But with Batchelor i always get an error
This is the code i used:
```
var Batchelor = require('batchelor');
var batch = new Batchelor({
'uri':'https://www.googleapis.com/batch',
'method':'POST',
'auth': {
'bearer': tokensGA.access_token
},
'headers': {
'Content-Type': 'multipart/mixed'
}
});
batch.add({
'method':'GET',
'path':'/analytics/v3/data/ga?ids='+ VIEW_ID+ '&start-date=14daysAgo&end-date=today&metrics=ga:users,ga:sessions,ga:bounces,ga:sessionDuration,ga:goalCompletionsAll,ga:pageviewsPerSession,ga:uniquePageviews,ga:pageViews&dimensions=ga:date,ga:hour&filters=ga:date=~' + _today + '|' + _1weekAgo + '|' + _2weeksAgo + ';ga:sessions>0'
});
I'm getting the following response:
{ parts:
[ { statusCode: '401',
statusMessage: 'Unauthorized',
headers: [Object],
body: [Object] } ],
errors: 0 }
Thanks in advance for any help you can provide, really appreciate it.
Currently run() throws exceptions when errors happen. See https://github.com/wapisasa/batchelor/blob/master/lib/batchelor.js#L167. It should call: callback(new Error('..'))
instead. The result should be the second parameter of the callback.
I'm running a HTTP server that accepts user requests. So I'm trying to use multiple instances of Batchelor to service different users. However, the API is entirely singleton, so calling Batchelor.init()
with a different user will overwrite the previous call.
What's your recommended way to create two instances of Batchelor at the same time?
Bathes are limited to 100 requests per batch, will bachelor automatically split requests?
If no, then it could be good feature,
Hello,
Using dicer dependency may result in crashes, according to npm when running npm install
with batchelor 2.0.2 in package.json:
dicer *
Severity: high
Crash in HeaderParser in dicer - https://github.com/advisories/GHSA-wm7h-9275-46v2
No fix available
node_modules/dicer
batchelor *
Depends on vulnerable versions of dicer
node_modules/batchelor
Please check the corresponding issue opened in dicer repo. A PR on dicer is opened since more than one year, a quick fix should not be expected on that side...
Hey @jamesmhaley ๐ thanks for your great work on this library!
The official Node JS Google client still doesn't support the batching endpoint ๐ so there's still a need for a lib such as this.
Currently, we use the Node JS Google client to interact with the the Google Calendar API. I've been using Nock to mock responses in our tests. Nock is also used for the tests in the official Google client too.
Unfortunately, Nock doesn't seem to pick up requests made by batchelor. I can see batchelor is using the request lib, which seems as though it should be compatible with nock.
Do you have any suggestions as to why Nock isn't picking up batchelor's requests?
I can provide an example to reproduce if that's helpful.
As an aside, how actively are you looking to maintain this lib? Would be keen to make some contributions if this is still being actively worked on! ๐
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.