0 records sent successfully to destinations. about amazon-kinesis-agent

[kramalin]

I am just 2 months in to software and the mistake i was doing was the size of the fake log generated by apache was less than buffer size so records were not send to S3 bucket and as soon as i ran infinite fake log generation , i can see that data being transferred by the agent

[chaochenq]

Sorry for the inconvenience. You can enable the debug logging by adding

"log.level": "DEBUG"

or

"log.level": "TRACE"

to agent config file.

But I'll try to switch some of the messages over to ERROR level instead of DEBUG level. The reason why we don't throw these errors by default was that we don't want your log to be screwed (the error could be with each parse and send, thus could be quite frequently)

[maimonoded]

using debug is a pain, once open in debug we start getting 10 of this logs every SECOND:

[DEBUG] FileTailer[fh:pop-bi-data:/home/logs/test.log]: We reached end of file TrackedFile(id=(dev=ca01,ino=401409), path=/home/logs/test.log, lastModifiedTime=1459950502000, size=1070)

[chaochenq]

Switching the error message over to ERROR rather than trace/debug may not make it better. The error code is per record. And there could be up to 500 records per single batch.

You can decide if you want to change it: https://github.com/awslabs/amazon-kinesis-agent/blob/master/src/com/amazon/kinesis/streaming/agent/tailing/KinesisSender.java#L128

[chaochenq]

At the same time, could you also try changing this message from "debug" to "error" to see if it helps?

https://github.com/awslabs/amazon-kinesis-agent/blob/master/src/com/amazon/kinesis/streaming/agent/tailing/SimplePublisher.java#L206

I'm patching another release to fix the error message too.

[chaochenq]

I've pushed a patch to switch some of the error messages from DEBUG to ERROR.

[thesmart]

Hey sorry for the delay in responding. Thanks for looking into this an providing a solution.