mauricebutler / bunyan-loggly Goto Github PK
View Code? Open in Web Editor NEWA bunyan stream to send logs through to loggly
License: MIT License
A bunyan stream to send logs through to loggly
License: MIT License
I am logging a request from koa-bunyan-logger
. When I use bunyan-loggly
it errors like below. When I use a regular bunyan logger. No error.
TypeError: Converting circular structure to JSON
at Object.stringify (native)
at Bunyan2Loggly.write (xxx/node_modules/bunyan-loggly/index.js:24:32)
at Logger._emit (xxx/node_modules/bunyan/lib/bunyan.js:939:22)
at Logger.info (xxx/node_modules/bunyan/lib/bunyan.js:1044:24)
at requestLogger (xxx/node_modules/koa-bunyan-logger/index.js:136:13)
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at logger (xxx/node_modules/koa-bunyan-logger/index.js:39:12)
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at logger (xxx/node_modules/koa-logger/index.js:49:12)
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
at xxx/node_modules/koa-mount-hot/index.js:116:26
at dispatch (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:43:32)
at next (xxx/node_modules/live-koa-server/node_modules/koa/node_modules/koa-compose/index.js:44:18)
the problem is that this tag will be added even more than once, depending on how someone is using this module and that it is not possible to remove it...
Not sure if this belongs here but since the statement has [log,connection,client,error],
it seems likely its from Buynan as that's the only logging I'm using.
I've opened an issue w/ trentm/node-bunyan#353
I am running Nodejs 4.2.3 on OpenShift and logging to Loggly
Once this error is displayed, the server stops
[log,connection,client,error], data: {"code":"EPIPE","errno":"EPIPE","syscall":"write"}
Is there a way for me to handle this error?
Tags are not making their way to Loggly.
In order to get my tags to appear in Loggly, I need to modify index.js line 60 from:
this.client.log(content);
To something like this:
this.client.log({ content: content }, this.logglyConfig.tags);
My test program:
var bunyan = require('bunyan'),
Bunyan2Loggly = require('bunyan-loggly').Bunyan2Loggly,
logger;
// create the logger
var logger = bunyan.createLogger({
name: 'logglylog',
streams: [
{
type: 'raw',
stream: new Bunyan2Loggly({
token: 'your-account-token',
subdomain: 'your-sub-domain',
tags: [ 'global-tag' ]
})
}
]
});
logger.info("foobar");
As was brought up in #33 it seems there is still a number of people using node version 6.x
Node 6.x will go end of life in April 2019 (https://github.com/nodejs/Release#release-schedule).
I am interested in what the user base thinks about dropping support in a major version bump.
There is no new functionality or features being relied on from newer versions, however things like trailing commas etc are built in to most editors, formatters, etc these days make it much harder to keep support for these older versions. Hence the issue introduced in #33
Also the lack of package-lock.json
support or explicit version locking in package.json
means that npm versions shipping with node 6 expose / exacerbate these issues and more.
I am assuming @rizowski @dncrews @stephan-nordnes-eriksen @epiales you all may have an opinion / some insight into the usage of node 6 and your migration plans?
Hey,
Just curious if you've ever considered making bunyan-loggly inherit from the built-in Stream class? The reason I ask is that I need to pipe the output of another stream into loggly. Since bunyan-loggly is not a full-fledged stream, that doesn't work. The workaround I intend to attempt is to write my own Stream class that wraps bunyan-loggly. I don't see why that wouldn't work. But it would be sweet if that was unnecessary.
Thanks!
Eric
Is there any particular reason why the bulk endpoint isn't used when sending buffered log records?
https://github.com/MauriceButler/bunyan-loggly/blob/master/index.js#L41
From what I gather, the log
function could be invoked by passing an array of log records, instead of being called in a loop, and that would take advantage of bulk posting and the network overhead.
It isn't currently possible to turn off bulk through options. It would be nice to have that option in the loggly config:
new BunyanLoggly({
token: 'foo',
subdomain: 'bar',
isBulk: false
})
So I created a PR for that. #30
Line 5 in d53c766
How does this implement bunyan.Stream?
I delved a little into this and found that the resultant type has no write
member as it's supposed to either.
Perhaps it's deviated a little from the mainline bunyan type, unless someone can point out where I'm missing something?
With the most recent version published (1.4.0) I am now getting an error on line 31 of index
/..../node_modules/bunyan-loggly/index.js:31
);
^
SyntaxError: Unexpected token )
I added subdomain + token, created new object stream and add it into bunyan... I nothing see in loggly. In callback error is null and response is 200. (code is in typescript)
import * as Bunyan2Loggly from "bunyan-loggly"
let streams: any[] = []
let logglyStream = new Bunyan2Loggly({
token: logglyToken,
subdomain: logglySubdomain,
}, 5)
this.log = Bunyan.createLogger({
name: "logglyTest",
streams: streams
})
Hi,
As shown in your usage example in the readme file that we require both bunyan
and bunyan-loggly
to setup the logging you must also provide explicit instructions to run both the commands npm install bunyan
and npm install bunyan-loggly
. If we only run the command npm install bunyan-loggly
then we get the below error-
module.js:327
throw err;
^
Error: Cannot find module 'bunyan'
at Function.Module._resolveFilename (module.js:325:15)
at Function.Module._load (module.js:276:25)
at Module.require (module.js:353:17)
at require (internal/module.js:12:17)
at Object.<anonymous> (/home/dell/bunyan-loggly/testbun.js:1:76)
at Module._compile (module.js:409:26)
at Object.Module._extensions..js (module.js:416:10)
at Module.load (module.js:343:32)
at Function.Module._load (module.js:300:12)
at Function.Module.runMain (module.js:441:10)
After installing bunyan
, logging works fine.
This is failing NSP check because of a dependency it uses. More information here: https://nodesecurity.io/advisories/533
Since dependency might not get patched soon, is there a possibility to remove it from this project or use a secure one?
I got this error while updating to from 1.3.1
to 1.3.2
. Previous versions worked perfectly.
throw new TypeError('"string" must be a string, Buffer, or ArrayBuffer');
^
TypeError: "string" must be a string, Buffer, or ArrayBuffer
at Function.byteLength (buffer.js:477:11)
at exports.Loggly.Loggly.log (/.../node_modules/node-loggly-bulk/lib/loggly/client.js:161:32)
at Bunyan2Loggly._processBuffer (/.../node_modules/bunyan-loggly/index.js:48:32)
at Bunyan2Loggly._checkBuffer (/.../node_modules/bunyan-loggly/index.js:61:21)
at Bunyan2Loggly.write (/.../node_modules/bunyan-loggly/index.js:37:10)
at Logger._emit (/.../node_modules/bunyan/lib/bunyan.js:923:22)
at Logger.info (/.../node_modules/bunyan/lib/bunyan.js:1045:24)
I'm using node v8.4.0
and this packages dependencies:
"dependencies": {
"bunyan": "1.8.12",
"bunyan-loggly": "1.3.1",
"bunyan-prettystream": "0.1.3",
"config": "1.26.2",
"jsonschema": "1.2.0",
"mongodb": "2.2.31",
"restify": "5.2.0",
"restify-clients": "1.5.0"
},
To add the timestamp
field and remove time
, it modifies the passed log
object.
Regrettably, this might break other configured streams (for example, it does break bunyan-cloudwatch).
Though it's of course more expensive, it should create a new object. Maybe an option could be provided to do one or the other, but IMO the right thing to do is to treat the log object as immutable, so it should default to that behavior.
Once loggly/node-loggly-bulk#21 is merged, re-enable bulk message processing.
Hey,
First of all, thank you for the project.
Have you considered adding typings to add support for TypeScript? Here's link to issue to help doing that.
typings/typings#322
As of #16 (v1.3), this library is no longer usable in AWS Lambda. Because that library waits for 30 seconds before sending ANY logs, it has broken all of my Lambda calls that go through API Gateway, which times out at 30 seconds. This library now causes my entire stack to be locked up and always respond with timeouts.
I've set up bunyan-loggly as described below (and I've tried every variation of this I could think of). I get no errors, but I also get no logs in my Loggly account. I have successfully used pm2-loggly with the same credentials (token and subdomain) and it works fine, so I feel confident in the credentials and the account configuration. How can I troubleshoot?
const logStreams = [];
const loggly = require('bunyan-loggly');
logStreams.push({
type: 'raw',
level: CONFIG.LOGGER.LOGGLY.LEVEL,
stream: new loggly({
token: CONFIG.LOGGER.LOGGLY.TOKEN,
subdomain: CONFIG.LOGGER.LOGGLY.SUBDOMAIN,
tags: CONFIG.LOGGER.LOGGLY.TAGS
}, 10, 1000)
});
const log = require('bunyan').createLogger({
name: 'ES',
streams: logStreams
});
Please consider updating the dependencies. Right now we have node-uuid
generating warnings: [SECURITY] node-uuid: crypto not usable, falling back to insecure Math.random()
Tracking back ... loggly package seems to be unmaintained (issue open five months back without answers). Loggly team created a fork repo to keep up security updates and created a new package.
Great to see some updates on this project, thanks!
The bump from 0.0.5
to 1.0.0
is much appreciated โ but some release notes and a tag of the new release would have been welcome, so one could easily spot whether this move from 0.0.5
means that the entire API has been remade or whether one can expect at least some API-compatibility (by looking through the diff I assume the latter)
The release notes could either be placed in eg. a CHANGELOG.md
file or as part of the tag message โ or preferably both. (One can subscribe to new tags on GitHub using an RSS reader and thus get the release notes easily that way for example)
A good note on how and why etc to keep a good changelog can be found at: http://keepachangelog.com/
By making it easy for people to move to new versions one can as a maintainer ensure that one has more of ones users on ones latest versions and thus can ensure that one gets feedback on stuff that matters to push the project forward rather than having people point out either outdated errors or mishandled upgrades.
Thanks again for updating the project!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.