winstonjs / node-loggly Goto Github PK

Does is provide asynchronous logging support?

Tests are currently broken, both on node v0.4.12 and 0.6.0.

♢ node-loggly/common

  When using the common module the clone() method
    ✓ should return a deep clone of the object

♢ node-loggly/devices

  When using the node-loggly client the getDevices() method
    ✓ should return a list of valid devices
  When using the node-loggly client the addDeviceToInput() method
    ✗ should respond with 200 status code
      » expected null, got {
    stack: 'Error: Loggly Error (400): Bad Request\n    at Request.callback (/Users/maciej/Programowanie/JavaScript/node-loggly/lib/loggly/common.js:130:25)\n    at Request.<anonymous> (/Users/maciej/Programowanie/JavaScript/node-loggly/node_modules/request/main.js:314:21)\n    at Request.emit (events.js:64:17)\n    at IncomingMessage.<anonymous> (/Users/maciej/Programowanie/JavaScript/node-loggly/node_modules/request/main.js:295:19)\n    at IncomingMessage.emit (events.js:81:20)\n    at HTTPParser.onMessageComplete (http.js:133:23)\n    at CleartextStream.ondata (http.js:1231:22)\n    at CleartextStream._push (tls.js:303:27)\n    at SecurePair.cycle (tls.js:577:20)\n    at EncryptedStream.write (tls.js:96:13)',
    arguments: undefined,
    type: undefined,
    message: 'Loggly Error (400): Bad Request'
} // common.js:130

♢ node-loggly/inputs

  When using the node-loggly client the getInputs() method
    ✓ should return a list of valid inputs
  When using the node-loggly client the getInput method when called with a json input
    ✓ should return a valid input
    ✗ of the format 'json'
      » expected 'json',
    got  'text' (==) // input-test.js:64
    ✗ that matches the second input in the test configuration
      » expected 'efe9d37e-a9f1-4e28-a30f-a489ab545d36',
    got  undefined (==) // input-test.js:67
  When using the node-loggly client the getInput method when called with a plaintext input
    ✓ should return a valid input
    ✓ of the format 'text'
    ✗ that matches the first input in the test configuration
      » expected 'f67ef8d2-9482-4aba-9a82-2c5740f5f18e',
    got  undefined (==) // input-test.js:49
  When using the node-loggly client the log() method to a 'text' input when passed a callback
    ✗ should log messages to loggly
      » expected null, got {
    stack: 'Error: Loggly Error (500): Internal Server Error\n    at Request.callback (/Users/maciej/Programowanie/JavaScript/node-loggly/lib/loggly/common.js:130:25)\n    at Request.<anonymous> (/Users/maciej/Programowanie/JavaScript/node-loggly/node_modules/request/main.js:314:21)\n    at Request.emit (events.js:64:17)\n    at IncomingMessage.<anonymous> (/Users/maciej/Programowanie/JavaScript/node-loggly/node_modules/request/main.js:295:19)\n    at IncomingMessage.emit (events.js:81:20)\n    at HTTPParser.onMessageComplete (http.js:133:23)\n    at CleartextStream.ondata (http.js:1231:22)\n    at CleartextStream._push (tls.js:303:27)\n    at SecurePair.cycle (tls.js:577:20)\n    at EncryptedStream.write (tls.js:96:13)',
    arguments: undefined,
    type: undefined,
    message: 'Loggly Error (500): Internal Server Error'
} // common.js:130
  When using the node-loggly client the log() method to a 'json' input when passed a callback
    ✗ should log messages to loggly
      » expected null, got {
    stack: 'Error: Loggly Error (500): Internal Server Error\n    at Request.callback (/Users/maciej/Programowanie/JavaScript/node-loggly/lib/loggly/common.js:130:25)\n    at Request.<anonymous> (/Users/maciej/Programowanie/JavaScript/node-loggly/node_modules/request/main.js:314:21)\n    at Request.emit (events.js:64:17)\n    at IncomingMessage.<anonymous> (/Users/maciej/Programowanie/JavaScript/node-loggly/node_modules/request/main.js:295:19)\n    at IncomingMessage.emit (events.js:81:20)\n    at HTTPParser.onMessageComplete (http.js:133:23)\n    at CleartextStream.ondata (http.js:1231:22)\n    at CleartextStream._push (tls.js:303:27)\n    at SecurePair.cycle (tls.js:577:20)\n    at EncryptedStream.write (tls.js:96:13)',
    arguments: undefined,
    type: undefined,
    message: 'Loggly Error (500): Internal Server Error'
} // common.js:130


✗ Errored » Asynchronous Error
    in node-loggly/inputs
    in test/input-test.js

✗ Errored » callback not fired
    in When using the node-loggly client the log() method to a 'text' input when not passed a callback
    in node-loggly/inputs
    in test/input-test.js
✗ Errored » callback not fired
    in When using the node-loggly client the log() method to a 'json' input when not passed a callback
    in node-loggly/inputs
    in test/input-test.js
✗ Errored » 6 honored ∙ 6 broken ∙ 1 errored ∙ 2 dropped

Logging an array makes use of the Bulk Endpoint of Loggly's HTTP API. However tags are not being send over.
(Using node-loggly 1.0.3)

var loggly = require('loggly');

var client = loggly.createClient({
  token     : '12345678-1234-1234-1234-123456789ab',
  subdomain : 'abc'
});

client.log('single', ['test']);
client.log(['mul', 'ti'], ['test']);

note that the tags are missing with mul and ti.

I tried the same thing with Fiddler and that works, so Loggly Bulk Endpoint is not broken.

You can facet on json fields, so there is no need to check for date, ip or input in facet.js

It looks like this issue has re-appeared. Minimal use case:

const loggly = require("loggly");

const client = loggly.createClient({
  token: <loggly-token>,
  subdomain: <subdomain>
});

const circular = { a : "123"};
circular.b = circular;

client.log(circular);

gives:

RangeError: Maximum call stack size exceeded
    at Object.common.serialize (/.../tmp-loggly-testcase/node_modules/loggly/lib/loggly/common.js:154:29)
    at Object.common.serialize (/.../tmp-loggly-testcase/node_modules/loggly/lib/loggly/common.js:187:21)
    at Object.common.serialize (/.../tmp-loggly-testcase/node_modules/loggly/lib/loggly/common.js:187:21)

See https://nodesecurity.io/advisories/664
Only affects node.js <= 4.

Loggly doesn't require an auth header when you are POSTing to the hash input URLs. The URL itself serves as the key for entry to the input in the account.

node-loggly currently puts the auth header in the POST, but it's being ignored by the Loggly servers.

I'd suggest taking it out to avoid confusion if someone is looking at the code.

This code should be able to work through an HTTP(S) proxy if there is a need to do so.

We should be checking the FROM and UNTIL parameters in the Loggly.prototype.search method to ensure that:

1. They are from the set of accepted parameters by Loggly: http://wiki.loggly.com/searchguide
2. The time range actually exists (i.e. disallow things like FROM=NOW&UNTIL=NOW-1DAY

@kordless Any input on this?

My app had a memory leak and would go from 130mb to 512 over a couple hours and then crash. After commenting out the code for node-loggly, the memory no longer climbs.

I used it as:

var Logger = new Loggly({                              
  token: "<token>",       
  subdomain: "<subdomain>",                                 
  auth: {                                              
    username: "<username>",                     
    password: "<password>"                               
  },                                                                                                    
  tags: ["customer_purchase"],                            
  json: "true"                                         
});                                                    

// a bunch of log statements in various parts of the code
Logger.info('no products in' + storeData._id);

I have about 700 logs per minute in bursts every 5 seconds.

If it helps, I am using the meteor package miktam:loggly which is a wrapper around node-loggly.

nt

I have been having issues with loggly return slowly with complex queries that return a lot of data. Is there an easy way to increase the request time out?

Hi!

Installing node-loggly
$ curl [sudo] npm install loggly

Should that "curl" really be there?

This deep clone method may cause infinite recursion on Mongoose objects in version 2.7.1 of mongoose when trying to log the mongoose object as meta. Maybe prevent this by adding a maximum depth to this function?

common.clone = function (obj, lvl) {
  lvl = lvl || 1;
  var clone = {};
  if (lvl > MAX_DEPTH) {
    return clone;
  }

  for (var i in obj) {
    clone[i] = obj[i] instanceof Object ? common.clone(obj[i], lvl+1) : obj[i];
  }

  return clone;
};

The request version being used right now is vulnerable to the issue discussed here: https://snyk.io/vuln/npm:request:20160119. It should be updated to 2.69.x. Thanks!

You are using [email protected] that requires deprecated [email protected]. Latest version of request is 2.79 that is properly updated to use "uuid": "^3.0.0"

Please either use request as a semver dependency via "request": "^2.75.0" or hardcode fixed 2.79 version.

Thank you

It would be nice to provide a convenience method for outputting a combined-access format from the node server. I'll whip up the format string and then present it for inclusion in the logging lib if everyone is jiggy with it.

I've tried node 0.2.6, but this package requires node >= 0.3.0, so I built 0.3.8 (latest unstable) and I can't install because this package requires 'request >= 1.1.1' and request 1.1.1 (and 1.2.0) aren't supported by node >= 0.3.6

A few things, can you put pressure on your dependencies to support current releases of node, and second, can you recommend a version of nodejs that works with this? I'd prefer not to play binary search with versions as each build takes 5-10 minutes of node ;)

We wanted to use your package for one of our https://github.com/WindowsAzure projects.

Could you please include a license file, so we know if we are allowed to?

request is now at 2.45.0 while node-loggly requires it at 2.27.0. The biggest problem here is that at 2.27.0, request requires the qs module at 0.6.6 which has known security vulnerabilities. See:
https://nodesecurity.io/advisories

There is a nice tool to check potential security vulnerabilities in node modules called nsp, which I recommend using:
https://github.com/nodesecurity/nsp

I use request-time to measure how long the deps need to be found with node:

# ./node_modules/.bin/require-time
1146ms  express
2038ms  loggly
425ms   node-cache
1ms q
389ms   request
374ms   winston

• Platform: armv7l
• Node: v4.4.7
• npm: 3.9.6

The facet call is currently returning search results instead of facet results. The documentation is also incorrect in showing an example of passing in rows to the facet call. Facets only support 'buckets'. Attempting to fix this issue and submit now.

I'm getting errors when an error callback is trying to be called:

root@ubuntu:/home/kord/code# node foo.js
401

/usr/local/lib/node/.npm/loggly/0.3.1/package/lib/loggly/interns.js:122
callback(new Error('Loggly Error (' + statusCode + '): ' + failCodes
^
TypeError: object is not a function
at Object.CALL_NON_FUNCTION (native)
at [object Object].callback (/usr/local/lib/node/.npm/loggly/0.3.1/package/lib/loggly/interns.js:122:20)
at IncomingMessage. (/usr/local/lib/node/.npm/request/1.9.0/package/main.js:214:44)
at IncomingMessage.emit (events.js:59:20)
at HTTPParser.onMessageComplete (http.js:111:23)
at CleartextStream.ondata (http.js:1140:22)
at CleartextStream._push (tls.js:263:27)
at SecurePair._cycle (tls.js:514:18)
at EncryptedStream.write (tls.js:72:13)
at Socket.ondata (stream.js:15:26)

Here's the code I'm calling it with:

geekceo.facet('date', '404', { rows: 1 }, function (err, results) { //do something } );

Otherwise installing with npat turned on fails, because it can't find the test runner.

When sending a log to loggly, can I echo it to the console as well? Right now I have to print them manually to console.

Problem:
The current dependent lib request version of request": "2.75.x" has an old dependency of aws-sign2: 0.6.0 which has a serious bug of duplicated exporting names: request/aws-sign#7

And this is preventing bundle tools such as rollup to fail because it can't allow duplicated exports from the same module.

I followed the instructions but I'm always getting
[Error: Loggly Error (403): Forbidden]

I created the client with the necessary data and in the log command I indicated my customer token.

I'm not sure what I'm doing wrong? Is it that the API is only for non-free?

Hi,

I can see in loggly docs that OAUTH used to be supported in GP1. But I don't see it now. Does anybody know if token based events retrieval is possible in the latest API version?

Trying to use the library on Node.js 0.3.4, and I'm getting:

node.js:68
throw e; // process.nextTick error, or 'error' event on first tick
^
TypeError: Cannot read property 'auth' of undefined
at Object.loggly (/usr/local/lib/node/.npm/loggly/0.1.0/package/lib/loggly/interns.js:98:36)
at [object Object].log (/usr/local/lib/node/.npm/loggly/0.1.0/package/lib/loggly/core.js:49:11)
at Object. (/root/test/test.js:14:8)

That's this line:
auth = auth || loggly.config.auth;

Apparently auth isn't being populated from args if..else blocks above, and loggly.config.auth isn't returning anything meaningful either.

Looks like maybe the way a callback is being passed in (or not) and handled.

For an application that sends a lot of data it can help to send a batch instead of a single log item.

The new /bulk endpoint supports this, http://www.loggly.com/docs/api-sending-data/#bulk

Would be great if this module could support it.

This is failing NSP check because of a dependency it uses. More information here: https://nodesecurity.io/advisories/533
Since dependency might not get patched soon, is there a possibility to remove it from this project or use a secure one?

I'm using winston-loggly to send logs to Loggly.
Every log with json.true option give me following notification:

LogglyNotifications:
[0]:
message: Timestamp field extracted from Syslog header is missing TZ
type: SyslogClockOrTimezoneWrong

I have succeeded to fix this issue by overriding message.timestamp at Loggly.prototype.log, but I think this was not ultimate solution.

I was previously using node 0.3.4 (maybe) and this was working. I installed Node on a new machine and got 0.3.6. When I try to install node-loggly, I got this:

root@aquaman:/usr/local/lib/node/.npm/.cache/loggly/0.1.4# npm install loggly
npm info it worked if it ends with ok
npm info using [email protected]
npm info using [email protected]
npm ERR! sudon't!
npm ERR! sudon't! Running npm as root is not recommended!
npm ERR! sudon't! Seriously, don't do this!
npm ERR! sudon't!
npm info range pool@>= 0.4.1
npm info latest = [email protected] not supported by [email protected]
npm info range request@>= 1.1.1
npm ERR! Error: No satisfying version found for 'request'@'>=1.1.1'
npm ERR! Valid install targets for request: "latest", "0.10.0", "0.8.3", "0.9.0", "0.9.1", "0.9.5", "1.0.0"
npm ERR! at /usr/local/lib/node/.npm/npm/0.2.16/package/lib/install.js:143:29
npm ERR! at /usr/local/lib/node/.npm/npm/0.2.16/package/lib/install.js:178:5
npm ERR! at /usr/local/lib/node/.npm/npm/0.2.16/package/lib/utils/registry/get.js:55:5
npm ERR! at /usr/local/lib/node/.npm/npm/0.2.16/package/lib/utils/registry/request.js:37:10
npm ERR! at IncomingMessage. (/usr/local/lib/node/.npm/npm/0.2.16/package/lib/utils/registry/request.js:174:14)
npm ERR! at IncomingMessage.emit (events.js:59:20)
npm ERR! at HTTPParser.onMessageComplete (http.js:111:23)
npm ERR! at Socket.ondata (http.js:990:22)
npm ERR! at Socket._onReadable (net.js:623:27)
npm ERR! at IOWatcher.onReadable as callback
npm ERR! Report this entire log at http://github.com/isaacs/npm/issues
npm ERR! or email it to [email protected]
npm ERR! Just tweeting a tiny part of the error will not be helpful.
npm not ok

I'm assuming this is a problem with the request lib 1.1.1 not supporting the new version of Node.

Looking into rolling back to a previous version, but some guidance would be great! :)

I am running node pretest after running npm install and getting the following errors

module.js:340
    throw err;
          ^
Error: Cannot find module 'C:\Users\Jayant\Documents\GitHub\node-loggly-fresh\no                                                                                                                de-loggly\pretest'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:906:3

I get it; you are busy!
When a repo gets to a point where the low-review-effort PRs go stale it might be a good idea to be explicit about the repo being unmaintained or asking if people are willing to help out.

There is an issue that warns about a - now fixed - vulnerability, bad docs, and a deprecated dependency. This package is still downloaded 150K times a month. Let's maintain it or clearly state it isn't 😄. Both could help this repo back to life making a significant positive change for the ecosystem.

/cc @indexzero

Something must have changed that was not adjusted.

The request lib was relying on a module called node-uuid. For some reason the author decided it was necessary to deprecate this module and continue with a module called uuid. Using node-uuid now spits warnings to users.

request fixed this in version 2.79.x, however because you chose to lock minor versions the many thousands of users that download this package each month have been getting warnings at build / install time.

Please bump the minor version or unlock the minor version. Thank you!

Hi,

Thanks for the awesome work on this repository. I notice you have patched the vulnerabilities in request into master, but not published to npm as of yet. When do you plan to publish?

Best,
Roy.

There was a pull request #14 which fixed an issue I'm having a few months ago.

However, it appears there hasn't been a bump in version number in the package.json for this module for almost 2 years: aa14779

This means that other modules, dependent on the change, haven't gotten any of the recent updates, including #14 . The issue specifically I ran into is that with https://github.com/indexzero/winston-loggly it's impossible to override the inputUrl.

However, the default url set in the 0.3.11 tagged version of this module (https://logs.loggly.com/input/...) is no longer valid. It will only work if I change that url (to https://logs-01.loggly.com/input/...) See http://loggly.com/docs/api-sending-data/

What this all means is that, with the default URL to loggly wrong and no way to override that, it's impossible to use winston-loggly or even node-loggly out of the box from npm.

This is very not ideal and burned a ton of my productivity today as I spent a long time trying to track down this issue which doesn't really have a good solution.

Thanks!
Victor

Hello,
In the last few days, we got quite a few errors that seem to be from inside the node loggly library.

The error looks like this:

json: level:error message:Unhandled promise rejection: Unspecified error from Loggly: SyntaxError: Unexpected token < in JSON at position 0 stack:Error: Unspecified error from Loggly: SyntaxError: Unexpected token < in JSON at position 0 at /root/fstg-service-monolith/node_modules/loggly/lib/loggly/client.js:161:18 at Request._callback (/root/fstg-service-monolith/node_modules/loggly/lib/loggly/common.js:139:7) at Request.self.callback (/root/fstg-service-monolith/node_modules/loggly/node_modules/request/request.js:187:22) at emitTwo (events.js:106:13) at Request.emit (events.js:191:7) at Request.<anonymous> (/root/fstg-service-monolith/node_modules/loggly/node_modules/request/request.js:1048:10) at emitOne (events.js:96:13) at Request.emit (events.js:188:7) at IncomingMessage.<anonymous> (/root/fstg-service-monolith/node_modules/loggly/node_modules/request/request.js:969:12) at emitNone (events.js:91:20) at IncomingMessage.emit (events.js:185:7) at endReadableNT (_stream_readable.js:974:12) at _combinedTickCallback (internal/process/next_tick.js:80:11) at process._tickDomainCallback (internal/process/next_tick.js:128:9)

It's occurring quite a lot on our part. We wrote to the Loggly support and they suggested also opening an issue here: "This is the first report of this kind we received. I also suggest to report it to the GitHub page of the library that you are using for a wider visibility to the community.".

It seems that we get an error response that is probably HTML and not JSON so node-loggly breaks because it can't handle this. Any idea on why this happens?

Thank you,
Nick