awslabs / amazon-kinesis-client-nodejs Goto Github PK

View Code? Open in Web Editor NEW

294.0 30.0 202.0 430 KB

Amazon Kinesis Client Library for Node.js

License: Apache License 2.0

JavaScript 99.65% Batchfile 0.35%

amazon-kinesis-client-nodejs's People

Stargazers

Watchers

Forkers

ospreyx mgreschke iotboy truthkos tylertuan atlassian-forks samcday nemo annie2010 leon2555 viktort noahchense arnaugm vlebedev peebles twistedpear taplytics christianthindberg brandingbrand fanmilesgmbh devsaik tinderbackend dughibot mengyiqi toflow edrabc jeffreyleespokeo mitesh91 witsoej rishiloyola noppoman ronakr larsonmpdx hungtd9 madsquare rbramwell rajeevhans isobe-toshiyuki jcoreio servicem8 pfifer pablitovicente colorwalf jeremyskogen tomafc330 easter-egg webstergrill loudbinary someoneweird iflix devicescape samjeffress vanipm arkator7 ganoro malakam leonardojperez sahilpalvia stezh msmmajor68 freshkeep jaystevency sandeep-jay kevinvancleef anudeepakumar yknx4 lordgraysith jbaczuk hejnal ihor-bondarenko poterlee cncoder bdiesel npatel88 kobaska hyandell aheuermann benluteijn meeks91 opticsponge zhenyu-aws-lab brennentsmith therealmtam mscharp sharethis-github qwalker nm226 sapataf tapaderster kenwalger stavank sampathpushengage ellerbrock cloud-architecture ernetas tanmay-at-sellanycar prabz anthonywc burnzoire pawellabudafitit

amazon-kinesis-client-nodejs's Issues

upgrade to kcl 2.0

Can you provide an implementation with the most recent kcl version (2.0)?

Proper way to implement processRecord retries

I am wondering what the proper way to implement retry logic in the processRecords function that will attempt (indefinitely) to reprocess records from a checkpointed sequence number.

For example, in our KCL application, in the processRecord function, we do some processing on the record data and then send the result via TCP to another service. If for some reason that service is down, we do not want to continue processing subsequent records but rather continue to retry processing the same record until it succeeds. In this scenario there is no issue with the data, it's just that we aren't able to push the processed data to it's desired destination.

From testing I've discovered that within the processRecord function, even if completeCallback() is called without calling checkpoint(), the KCL will still attempt to grab the next record and not retry the previous record that failed to be "processed". But, I've also noticed that if completeCallback() is called without calling checkpoint() and the KCL application is restarted, then it will start again from after the last checkpoint, thus retrying the records that failed to "process".

So by this, it seems that in order to implement retry logic, we should be stopping the KCL application and restarting it. Is there a recommended way to do that? In the java KCL library I noticed there is a worker.requestShutdown() method, but I don't see that in the Nodejs library, is it just a feature that hasn't been implemented yet? Either way, is the suggested work around to just do a process.exit() call when we want to retry (and having something watch the process that restarts it)? That seems like it may not be the best in regards to graceful a shutdown, is there a better way?

I believe this is related to the conversation here awslabs/amazon-kinesis-client#10

Cannot find module 'log4js'

After npm install, I tried to run the producer, I got this error:

Error: Cannot find module 'log4js'
    at Function.Module._resolveFilename (module.js:325:15)
    at Function.Module._load (module.js:276:25)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (/usr/src/app/samples/util/logger.js:18:14)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Module.require (module.js:353:17)

So, what's wrong?

Release new version

The latest release is v2.1.0 which is 2 years ago https://www.npmjs.com/package/aws-kcl/v/2.1.0.
There are couple of changes since v2.1.0, is it possible to release a new version?

Thanks!

Unable to locate java

when I run the command

kcl-bootstrap -e -p ..\samples\basic_sample\consumer\sample.properties

from the location of the bootstrap file, I get:
ERROR: Valid --java value is required or alternatively JAVA_HOME environment must be set.

I have set the JAVA_HOME value to point to my java.exe file, and have added the --java [path-to-java] option to my command and still am unable to run the code

Zombie nodejs processes

After KCL is working long time I see zombie nodejs processes which don't do anything. KCL starts new nodejs processes, but don't kill zombie processes.
Is anybody receive this problem?

Add New KCL Features to Client

Request for awslabs/amazon-kinesis-client#77

UserRecord's fields are missing. for example 'subSequenceNumber'
The ability to specify the subsequence number at which to checkpoint
Additional features

Update npm package

Hello,

When pulling the npm package, it pulls the 0.5.0 tag by default which contains an out of date /bin/kcl-bootstrap file. The dependecies were recently updated so it would be nice if you could update the package to pull the latest code.

Thank you

Record processor not shutdown on IO close

I received an uncaught exception that meant the consumer was no longer connected to Kinesis. In this case, I would imagine that the KCL would either continuously attempt to reconnect or die and call shutdown. However, neither happened, which got my consumer process stuck in an invalid and dangerous state: thinking it was connected when it wasn't.

Stack trace:

Error: Kinesis Client Library is in the invalid state. Cannot proceed further.
  File "/vitally/server/node_modules/aws-kcl/lib/kcl/kcl_manager.js", line 399, col 11, in KCLManager._handleStateInput
    throw new Error('Kinesis Client Library is in the invalid state. Cannot proceed further.');
  File "/vitally/server/node_modules/aws-kcl/lib/kcl/kcl_manager.js", line 265, col 8, in KCLManager._onActionEnd
    this._handleStateInput(this._context, 'cleanup');
  File "events.js", line 106, col 13, in emitNone
  File "events.js", line 208, col 7, in ActionHandler.emit
  File "/vitally/server/node_modules/aws-kcl/lib/kcl/action_handler.js", line 82, col 8, in ActionHandler._onIOClose
    this.emit('end');
  File "events.js", line 106, col 13, in emitNone
  File "events.js", line 208, col 7, in IOHandler.emit
  File "/vitally/server/node_modules/aws-kcl/lib/kcl/io_handler.js", line 93, col 8, in IOHandler._onInputClose
    this.emit('close');
  File "events.js", line 111, col 20, in emitNone
  File "events.js", line 208, col 7, in Interface.emit
  File "readline.js", line 370, col 8, in Interface.close
  File "readline.js", line 149, col 10, in Socket.onend
  File "events.js", line 111, col 20, in emitNone
  File "events.js", line 208, col 7, in Socket.emit
  File "_stream_readable.js", line 1055, col 12, in endReadableNT
  File "internal/process/next_tick.js", line 138, col 11, in _combinedTickCallback
  File "internal/process/next_tick.js", line 218, col 9, in process._tickDomainCallback

Native only

Why java daemon.. it's like having to speak English, I need a Greek translator.. bad approach:(

run consumer as background service

I have been trying to use it as background service on ubuntu 16.06. absolute paths gives exception
/home/ubuntu/kcl/bin/kcl-bootstrap --java /usr/bin/java -e -p /home/ubuntu/kcl/samples/basic_sample/consumer/sample.properties -s
results in

Starting Multi-Lang Daemon ...
Exception in thread "main" java.lang.NullPointerException
        at java.util.Properties$LineReader.readLine(Properties.java:434)
        at java.util.Properties.load0(Properties.java:353)
        at java.util.Properties.load(Properties.java:341)
        at com.amazonaws.services.kinesis.multilang.MultiLangDaemonConfig.loadProperties(MultiLangDaemonConfig.java:135)
        at com.amazonaws.services.kinesis.multilang.MultiLangDaemonConfig.<init>(MultiLangDaemonConfig.java:85)
        at com.amazonaws.services.kinesis.multilang.MultiLangDaemonConfig.<init>(MultiLangDaemonConfig.java:70)
        at com.amazonaws.services.kinesis.multilang.MultiLangDaemonConfig.<init>(MultiLangDaemonConfig.java:57)

also tried with /etc/init.d/ and changed directory to consumer folder and ran ../../../bin/kcl-bootstrap --java /usr/bin/java -e -p ./sample.properties -s

is there any way I can run it as background service (relaunch if it crashes and start on boot)
thanks for any help

dynamodb-streams-kinesis-adapter?

Does this thing work with the DDB Streams Kinesis adapter? I am writing NodeJS worker. I need to process DDB streams. I am not finding any information about how this might be done using KCL. I might imagine that kcl-bootstrap needs to be modified to add getMavenPackageInfo('com.amazonaws', 'dynamodb-streams-kinesis-adapter', '1.0.0') in the MAVEN_PACKAGE_LIST and then in sample.properties, some way to identify the DDB stream ARN instead of a Kinesis stream name.

this._reportError is not a function

_reportError function is called in kcl_manager but it is not defined

amazon-kinesis-client-nodejs/lib/kcl/kcl_manager.js

Line 67 in 4790a36

this._reportError(util.format('Invalid action received: %j', action));

How to disable cloudwatch?

Empty S3 Bucket Name

Sorry, maybe I didn't fully understand the application, but perhaps it would be better to states in README that S3 Bucket Name is required when using the CloudFormation template associated with the clickStream example, or remove the S3 Bucket from the template so that the stack creation will not fail. It will be created by the application anyway.

Can we specify the nodejs version we want to use

I notice that we specify the language in the configuration as such

# Appended to the user agent of the KCL. Does not impact the functionality of the
# KCL in any other way.
processingLanguage = nodejs/0.10

If i I have nopdejs v4.2.1 installed how am i able to specify that in the configuration file.

Customizing log level

In the default configuration, the system generates extremely verbose logs.

Is it possible to adjust the log level downwards by editing the application.properties file somehow?

Error while running consumer

I am trying the to run the sample as it is, producer works fine but i am getting errors while running the consumer:
Here is the console dump:
Apr 24, 2018 2:55:02 PM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker info
INFO: Current stream shard assignments: shardId-000000000001, shardId-000000000000
Apr 24, 2018 2:55:02 PM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker info
INFO: Sleeping ...
Apr 24, 2018 2:55:15 PM com.amazonaws.services.kinesis.multilang.LineReaderTask call
INFO: Stopping: Reading next message from STDIN for shardId-000000000000
Apr 24, 2018 2:55:15 PM com.amazonaws.services.kinesis.multilang.LineReaderTask call
INFO: Stopping: Reading STDERR for shardId-000000000000
Apr 24, 2018 2:55:15 PM com.amazonaws.services.kinesis.multilang.MultiLangProtocol futureMethod
SEVERE: Failed to get status message for initialize action for shard shardId-000000000000
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Reached end of STDIN of child process for shard shardId-000000000000 so won't be able to return a message.
at java.base/java.util.concurrent.FutureTask.report(Unknown Source)
at java.base/java.util.concurrent.FutureTask.get(Unknown Source)
at com.amazonaws.services.kinesis.multilang.MultiLangProtocol.futureMethod(MultiLangProtocol.java:197)
at com.amazonaws.services.kinesis.multilang.MultiLangProtocol.waitForStatusMessage(MultiLangProtocol.java:171)
at com.amazonaws.services.kinesis.multilang.MultiLangProtocol.waitForStatusMessage(MultiLangProtocol.java:138)
at com.amazonaws.services.kinesis.multilang.MultiLangProtocol.initialize(MultiLangProtocol.java:78)
at com.amazonaws.services.kinesis.multilang.MultiLangRecordProcessor.initialize(MultiLangRecordProcessor.java:94)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.InitializeTask.call(InitializeTask.java:90)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.MetricsCollectingTaskDecorator.call(MetricsCollectingTaskDecorator.java:49)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.MetricsCollectingTaskDecorator.call(MetricsCollectingTaskDecorator.java:24)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: Reached end of STDIN of child process for shard shardId-000000000000 so won't be able to return a message.
at com.amazonaws.services.kinesis.multilang.GetNextMessageTask.returnAfterEndOfInput(GetNextMessageTask.java:84)
at com.amazonaws.services.kinesis.multilang.GetNextMessageTask.returnAfterEndOfInput(GetNextMessageTask.java:31)
at com.amazonaws.services.kinesis.multilang.LineReaderTask.call(LineReaderTask.java:70)
... 4 more

Apr 24, 2018 2:55:15 PM com.amazonaws.services.kinesis.multilang.MultiLangRecordProcessor stopProcessing
SEVERE: Encountered an error while trying to initialize record processor
java.lang.RuntimeException: Failed to initialize child process
at com.amazonaws.services.kinesis.multilang.MultiLangRecordProcessor.initialize(MultiLangRecordProcessor.java:95)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.InitializeTask.call(InitializeTask.java:90)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.MetricsCollectingTaskDecorator.call(MetricsCollectingTaskDecorator.java:49)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.MetricsCollectingTaskDecorator.call(MetricsCollectingTaskDecorator.java:24)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)

Apr 24, 2018 2:55:15 PM com.amazonaws.services.kinesis.multilang.LineReaderTask call
INFO: Starting: Draining STDOUT for shardId-000000000000
Apr 24, 2018 2:55:15 PM com.amazonaws.services.kinesis.multilang.LineReaderTask call
INFO: Stopping: Draining STDOUT for shardId-000000000000
Apr 24, 2018 2:55:15 PM com.amazonaws.services.kinesis.multilang.MultiLangRecordProcessor childProcessShutdownSequence
INFO: Child process exited with value: 1
Apr 24, 2018 2:55:15 PM com.amazonaws.services.kinesis.multilang.MultiLangDaemon run
INFO: Process terminanted, will initiate shutdown.

kcl-bootstrap can't download dependencies on Maven repository

The kcl-bootstrap script can't download dependencies on Maven repository anymore.

The root cause is pretty straight forward (and frightening): the script does not respect the URL scheme, and never use https while downloading dependencies.

The issue raised because Maven repository does not accept http requests anymore.

I submitted a PR solving this #75

Checkpoint Performance Cost?

I'm wondering what the cost of checkpoint is.

If I needed to checkpoint after each record in a batch, I'm curious if anyone knows if there will be a major performance impact or if it's a cheap operation.

Thanks!

Consuming multiple streams using KCL

Hi I have an application that consuming multiple streams by starting several KCL processes, which works fine, but when the application is interrupted rather than an exception raised from my record processor those multiple instances of MultiLangDaemon will continue running, which causes problems.

When I restart the application a new set of MultiLangDaemon processes will be started and from this point on wards I am running into situation in which:

either the DynamoDB table will contain a different worker ID i.e leaseOwner than my newly started worker's ID so nothing will happen,
Or I can possibly alter the table and betting on MultiLangDaemon to repopulate the table with new leaseOwner ID but eventually the host machine will have many "java.exe" processes running and run out of memory.

I can stop those multiple node-js KCL bootstrap processes by executing OS commands in my application but I don't know how to ONLY stop MultiLangDaemon processes (they are just java.exe on windows).

All these can be resolved "relatively properly", if I can stop the MultiLangDaemon process, hence when new MultiLangDaemon process start the leaseOwner ID will be changed and the process starts normally and the machine won't run out of memory. (I can't just kill all "java.exe" on windows or java on Unix system)

You suggestion and help will be much appreciated !

MultiLang Daemon for KCL 2.0

We intend to update the MultiLang Daemon for KCL 2.0. As part of this update we are considering incorporating other enhancements.

If you have a feature or change request please comment on awslabs/amazon-kinesis-client#451.

Graceful Shutdown of Workers

Since September 30, 2016, there's a new method in KCL's library (for Java) that handles very well the shutdown. Its called "requestShutdown", and it has already been implemented there.

awslabs/amazon-kinesis-client#109

We are missing that method here in Node... when KCL decides to shutdown a worker, how can we handle and perform a Graceful Shutdown? Meaning that would be cool to let the worker to finish processing the current batch, then checkpoint, and only then start the shutdown.

Is there any way to run some logic only on one child process?

I want to run my application which uses KCL to k8s cluster, i need some healthcheck endpoint and i want to use this library - https://github.com/gajus/lightship. But i have a problem because MultiLangDaemon runs three child processes and each one try to listen on some port, so application dies. Is there any way to run listen only in one child process (if this child is not healthy then whole app will be killed), or maybe i should do it in another way like configure many ports and each child process should use next port number (but how can i determine which child process i am when calling listen??)

Vulnerable lodash in dependency chain

[email protected] depends on [email protected] which depends on [email protected]

Versions of lodash before 4.17.5 are vulnerable to prototype pollution.

The vulnerable functions are 'defaultsDeep', 'merge', and 'mergeWith' which allow a malicious user to modify the prototype of Object via proto causing the addition or modification of an existing property that will exist on all objects.

https://www.npmjs.com/advisories/577

High CPU usage with node

Hi aws team!

I'm running the KCL client for reading 10 shards.
It seems like the for some simple log operations, each process take up to 45% cpu usage on a m3.xlarge. I was wondering if that was normal, or if I'm missing anything ?

Thank you for your insights

Use a custom aws endpoint

I have a running localstack server, I want to update the sample.properties file to point to another endpoint, for example: http://localhost:4456
Can anyone help me?

Not able to download MultiLangDaemon from maven repo

I'm receiving the following error when running the kcl-bootstrap script as following;

node kcl-bootstrap --java /usr/bin/java -e -p ../../../tests/sensor_datum_extra_field_node/kinesis.properties

ERROR: Unable to download Multi-Language Daemon jar files from maven: Error: connect ECONNREFUSED

using sudo did not change anything.

Broken and outdated kcl-bootstrap - Here is an alternative

The kcl-bootstrap script is kinda broken. It downloads a hardcoded list of maven dependencies, when the package POM itself tells us what all dependencies are required. Also its really out of date. MultiLangDaemon is at version 1.6.1, whereas this library ships with 1.2.0

Here is a simple bootstrap script that you can copy/paste and run on your own. It first downloads Ivy, the uses Ivy to download the latest MultiLangDaemon JAR (plus all its dependencies properly) and runs it with the kcl properties file. Its actually very simple.

#!/usr/bin/env sh
# Usage:
# kcl.sh <path-to-kcl-properties>
# e.g.
# kcl.sh kcl.properties

MULTI_LANG_DAEMON_CLASS='com.amazonaws.services.kinesis.multilang.MultiLangDaemon'
IVY_URL='http://search.maven.org/remotecontent?filepath=org/apache/ivy/ivy/2.3.0/ivy-2.3.0.jar'
mkdir -p kcl

if [ ! -f "kcl/ivy.jar" ]; then
  echo 'Downloading ivy...'
  wget -q $IVY_URL -O kcl/ivy.jar
fi

echo 'Downloading dependencies...'
java -jar kcl/ivy.jar \
     -dependency com.amazonaws amazon-kinesis-client 1.6.1 \
     -retrieve "kcl/[artifact]-[revision](-[classifier]).[ext]"

echo 'Starting KCL MultiLang daemon...'
java -cp .:"kcl/*" "$MULTI_LANG_DAEMON_CLASS" "$1"

Not able to send data from firehose to ES

In my project im streaming logs from cloudwatch kinesis stream. From kinesis stream i have linked to Kinesis firehose which invokes a lambda and transforms the data. The lambda code is one of the blueprints provided when we setup the firehose. I need to send data from firehose lambda to Elastic search as destination.

Now the problem is when i see the firehose logs in cloudwatch i get the below error:

{
"deliveryStreamARN": "arn:aws:firehose:us-east-2:620209065378:deliverystream/testSBFirehose",
"destination": "arn:aws:es:us-east-2:620209065378:domain/amcs-test",
"deliveryStreamVersionId": 2,
"message": "The ES cluster returned a JsonParseException. Ensure that the data being put is valid.",
"errorCode": "ES.JsonParseException",
"processor": "arn:aws:lambda:us-east-2:620209065378:function:streamCloudwatchLogsByKinesis:$LATEST"
}

Please let me know in which form/structure i need to send the data to ES from firehose lambda. Currently im sending data as below format:-

{
records: [
{
recordId:''some id",
result:"ok",
data:"base64 data",
}
]
}

Override dynamoDBEnpoint

Hi,

I'm running Localstack with Kinesis and DynamoDB locally, but I still don't know how to override DynamoDB endpoint. I was looking at the kinesis application client java library, but I guess it uses the default constructor to create the DynamoDbAsyncClient instance.

DynamoDbAsyncClient dynamoDBClient = DynamoDbAsyncClient.builder() .credentialsProvider(DefaultCredentialsProvider.create()).build();

Does anyone know how to do it?

Unable to run the consumer sample

I tried to run the kinesis click_stream_sample, the producer is working fine but the consumer has an issue.
The issue is that when I'm trying to run the command kcl-bootstrap <javapath> -e -p ./sample.properties I got this error: ERROR: Valid --java value is required or alternatively JAVA_HOME environment variable must be set. although I put the <javapath> with the my java path and also put JAVA_HOME in system variables so I can't determine why I have that problem till now.

Note: I'm trying this sample locally on my machine with Windows-10

@pfifer

KCL consumer on kubernetes does not pick up IAM role via serviceAccount.

Deploying KCL consumer via kubernetes on aws EKS service does not pick up IAM role from serviceAccount with AWSCredentialsProvider = DefaultAWSCredentialsProviderChain, instead it uses node level role.

Following environment variables are present in the service, however it still doesn't seem to pick it up. However it works when I provide AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

AWS_DEFAULT_REGION=<region>
AWS_REGION=<region>
AWS_ROLE_ARN=<role-arn>
AWS_WEB_IDENTITY_TOKEN_FILE=<token-path>

How to catch KCL error

Hello,
I'm using KCL in nodeJS to send data from kinesis to some other streams in my app (for example websocket), first thing i do is initialiation of websocket connection, i want to be able to close connection when KCL initialization fails or when some errors occurs. Is it possible to register some handler in child process or listening on some message to be notified about errors?

For now i'm using uncaughtException to gracefully disable my connections but maybe it is another way ?

processRecords failure - correct handling ?

if processRecords() fails for any one record, how should one handle it ? @sahilpalvia

what is the difference between the amazon kinesis client library and the aws-sdk library for kinesis?

Readme links to MultiLangDaemon return 404

Is there a way to use other credentials for dynamoDB?

The kinesis will be consumed by an user that hasn't dynamodb permissions.

So I need a way to create a dynamodb table in other aws account that has dynamodb permissions.

Is there a way to use two distincts credentials??

Cannot change the Credentials Provider

In the sample.properties fie, I added a different credentials Provider:
"AWSCredentialsProvider = EnvironmentVariableCredentialsProvider".
I have also set the Environment variable with correct credentials
When I run the consumer, it errors out with ::

java.util.concurrent.ExecutionException: com.amazonaws.AmazonClientException: Unable to load AWS credentials from any provider in the chain
at java.util.concurrent.FutureTask.report(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at com.amazonaws.services.kinesis.multilang.MultiLangDaemon.main(MultiLangDaemon.java:217)
Caused by: com.amazonaws.AmazonClientException: Unable to load AWS credentials from any provider in the chain
at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:117)
at com.amazonaws.services.kinesis.multilang.MultiLangDaemon.prepare(MultiLangDaemon.java:123)
at com.amazonaws.services.kinesis.multilang.MultiLangDaemon.call(MultiLangDaemon.java:148)
at com.amazonaws.services.kinesis.multilang.MultiLangDaemon.call(MultiLangDaemon.java:61)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

processRecords with async operations?

Decided to use the new basic consumer sample as a template for my current processor..

So on this project, we have some asynchronous tasks to do for each kinesis message, we also want to checkpoint after each message. Was wondering if you could take a peek at my processRecords impl to see if it looks solid:

fyi to decouple a lot of our processing and retry logic from the consumer we pass a recordProcessingStrategyCallback to the consumer.. @sahilpalvia

'use strict';

var util = require('util');

module.exports = function strategizedKinesisConsumer(logger, recordProcessingStrategyCallback) {

  var shardId;
  var logger = logger;
  var recordProcessingStrategyCallback = recordProcessingStrategyCallback;

  return {

    initialize: function(initializeInput, completeCallback) {
      shardId = initializeInput.shardId;

      logger.info('strategizedKinesisConsumer', {customText: `New KCL consumer initializing with config: ${JSON.stringify(initializeInput)}`});

      completeCallback();
    },

    processRecords: function(processRecordsInput, completeCallback) {
      
      if (!processRecordsInput || !processRecordsInput.records) {
        completeCallback();
        return;
      }

      var records = processRecordsInput.records;
      
      return new Promise( async (resolve, reject) => {

        for (var i = 0 ; i < records.length ; ++i) {
          let record = records[i];
          let data = new Buffer(record.data, 'base64').toString();
          let dataObj = JSON.parse(data);
          let sequenceNumber = record.sequenceNumber;
          let partitionKey = record.partitionKey;
          
          logger.info(util.format('ShardID: %s, Record: %s, SeqenceNumber: %s, PartitionKey:%s', shardId, data, sequenceNumber, partitionKey));

          //call logic
          logger.info('event', {eventId: dataObj.eventId, correlationId: dataObj.correlationId, customText: 'Delegating message to blackbox strategy for processing'});
          await recordProcessingStrategyCallback(dataObj);
          logger.info('event', {eventId: dataObj.eventId, correlationId: dataObj.correlationId, customText: '...back from blackbox strategy'});

          if (!sequenceNumber) {
            completeCallback();
            return;
          }

          const checkpoint = util.promisify(processRecordsInput.checkpointer.checkpoint).bind(processRecordsInput.checkpointer);

          logger.info('event', {eventId: dataObj.eventId, correlationId: dataObj.correlationId, customText: 'Checkpointing message...'});
          await checkpoint(sequenceNumber);
          logger.info('event', {eventId: dataObj.eventId, correlationId: dataObj.correlationId, customText: '..message checkpointed'});
  
        
        }

        completeCallback()
        
        resolve()

      });  
    },

    leaseLost: function(leaseLostInput, completeCallback) {
     ...
    },

    shardEnded: function(shardEndedInput, completeCallback) {
     ..
    },

    shutdownRequested: function(shutdownRequestedInput, completeCallback) {
      ..
    }
  };
}

listFragments return different FragmentLengthInMilliseconds then EXTINF

I'm trying to calculate my stream duration by using the list fragments API and then sum the FragmentLengthInMilliseconds for all the return fragments, however, the FragmentLengthInMilliseconds in the returned result is not the same as the time duration I get in the M3U file of the video, for example, the list of fragment I get for the following time range (Using PRODUCER_TIMESTAMP in both cases):

TimestampRange: {
      "StartTimestamp": "2022-01-28T07:00:35.000Z",
      "EndTimestamp": "2022-01-28T07:01:34.000Z"
    }

List of sorted fragments by ProducerTimestamp:

 [
  {
    FragmentNumber: '91343852333185057105875891022195998207386831087',
    FragmentSizeInBytes: 788604,
    ProducerTimestamp: 2022-01-28T07:00:35.931Z,
    ServerTimestamp: 2022-01-28T07:00:47.067Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057110827651179337519666762451998',
    FragmentSizeInBytes: 868779,
    ProducerTimestamp: 2022-01-28T07:00:37.787Z,
    ServerTimestamp: 2022-01-28T07:00:48.407Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057115779411336479041184133649199',
    FragmentSizeInBytes: 820392,
    ProducerTimestamp: 2022-01-28T07:00:39.643Z,
    ServerTimestamp: 2022-01-28T07:00:49.963Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057120731171493620562686992995215',
    FragmentSizeInBytes: 847071,
    ProducerTimestamp: 2022-01-28T07:00:41.499Z,
    ServerTimestamp: 2022-01-28T07:00:51.466Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057125682931650762084177611210926',
    FragmentSizeInBytes: 763593,
    ProducerTimestamp: 2022-01-28T07:00:43.355Z,
    ServerTimestamp: 2022-01-28T07:00:52.922Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057130634691807903605621447927987',
    FragmentSizeInBytes: 835200,
    ProducerTimestamp: 2022-01-28T07:00:45.211Z,
    ServerTimestamp: 2022-01-28T07:00:54.205Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057135586451965045127000030563099',
    FragmentSizeInBytes: 906677,
    ProducerTimestamp: 2022-01-28T07:00:47.071Z,
    ServerTimestamp: 2022-01-28T07:00:55.244Z,
    FragmentLengthInMilliseconds: 1792
  },
  {
    FragmentNumber: '91343852333185057140538212122186648471533739708',
    FragmentSizeInBytes: 685842,
    ProducerTimestamp: 2022-01-28T07:00:48.927Z,
    ServerTimestamp: 2022-01-28T07:00:56.629Z,
    FragmentLengthInMilliseconds: 1792
  },
  {
    FragmentNumber: '91343852333185057145489972279328169811940549220',
    FragmentSizeInBytes: 751127,
    ProducerTimestamp: 2022-01-28T07:00:50.783Z,
    ServerTimestamp: 2022-01-28T07:00:57.527Z,
    FragmentLengthInMilliseconds: 1791
  },
  {
    FragmentNumber: '91343852333185057150441732436469691175863905321',
    FragmentSizeInBytes: 733375,
    ProducerTimestamp: 2022-01-28T07:00:52.638Z,
    ServerTimestamp: 2022-01-28T07:00:58.511Z,
    FragmentLengthInMilliseconds: 1792
  },
  {
    FragmentNumber: '91343852333185057155393492593611212512597876737',
    FragmentSizeInBytes: 719137,
    ProducerTimestamp: 2022-01-28T07:00:54.494Z,
    ServerTimestamp: 2022-01-28T07:00:59.395Z,
    FragmentLengthInMilliseconds: 1792
  },
  {
    FragmentNumber: '91343852333185057160345252750752733848109724600',
    FragmentSizeInBytes: 704671,
    ProducerTimestamp: 2022-01-28T07:00:56.350Z,
    ServerTimestamp: 2022-01-28T07:01:00.274Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057165297012907894255185015258598',
    FragmentSizeInBytes: 726122,
    ProducerTimestamp: 2022-01-28T07:00:58.206Z,
    ServerTimestamp: 2022-01-28T07:01:01.158Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057170248773065035776518554640603',
    FragmentSizeInBytes: 781660,
    ProducerTimestamp: 2022-01-28T07:01:00.062Z,
    ServerTimestamp: 2022-01-28T07:01:02.029Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057175200533222177297851537695855',
    FragmentSizeInBytes: 670833,
    ProducerTimestamp: 2022-01-28T07:01:01.918Z,
    ServerTimestamp: 2022-01-28T07:01:02.899Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057180152293379318819121358297568',
    FragmentSizeInBytes: 756430,
    ProducerTimestamp: 2022-01-28T07:01:03.774Z,
    ServerTimestamp: 2022-01-28T07:01:03.533Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057185104053536460340391192576793',
    FragmentSizeInBytes: 901088,
    ProducerTimestamp: 2022-01-28T07:01:05.630Z,
    ServerTimestamp: 2022-01-28T07:01:04.167Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057190055813693601862044276052305',
    FragmentSizeInBytes: 773312,
    ProducerTimestamp: 2022-01-28T07:01:07.486Z,
    ServerTimestamp: 2022-01-28T07:01:06.229Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057195007573850743383632007202899',
    FragmentSizeInBytes: 768660,
    ProducerTimestamp: 2022-01-28T07:01:09.346Z,
    ServerTimestamp: 2022-01-28T07:01:08.047Z,
    FragmentLengthInMilliseconds: 1791
  },
  {
    FragmentNumber: '91343852333185057199959334007884905218529030717',
    FragmentSizeInBytes: 748856,
    ProducerTimestamp: 2022-01-28T07:01:11.201Z,
    ServerTimestamp: 2022-01-28T07:01:09.861Z,
    FragmentLengthInMilliseconds: 1792
  },
  {
    FragmentNumber: '91343852333185057204911094165026426933483564105',
    FragmentSizeInBytes: 763292,
    ProducerTimestamp: 2022-01-28T07:01:13.057Z,
    ServerTimestamp: 2022-01-28T07:01:12.153Z,
    FragmentLengthInMilliseconds: 1792
  },
  {
    FragmentNumber: '91343852333185057209862854322167948478491151138',
    FragmentSizeInBytes: 831664,
    ProducerTimestamp: 2022-01-28T07:01:14.913Z,
    ServerTimestamp: 2022-01-28T07:01:13.813Z,
    FragmentLengthInMilliseconds: 1792
  },
  {
    FragmentNumber: '91343852333185057214814614479309470058753450625',
    FragmentSizeInBytes: 751255,
    ProducerTimestamp: 2022-01-28T07:01:16.769Z,
    ServerTimestamp: 2022-01-28T07:01:15.603Z,
    FragmentLengthInMilliseconds: 1792
  },
  {
    FragmentNumber: '91343852333185057219766374636450991675003849379',
    FragmentSizeInBytes: 769545,
    ProducerTimestamp: 2022-01-28T07:01:18.625Z,
    ServerTimestamp: 2022-01-28T07:01:17.528Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057224718134793592513326318973487',
    FragmentSizeInBytes: 757835,
    ProducerTimestamp: 2022-01-28T07:01:20.481Z,
    ServerTimestamp: 2022-01-28T07:01:19.583Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057229669894950734034908059013802',
    FragmentSizeInBytes: 790785,
    ProducerTimestamp: 2022-01-28T07:01:22.337Z,
    ServerTimestamp: 2022-01-28T07:01:21.380Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057234621655107875556490501268794',
    FragmentSizeInBytes: 800209,
    ProducerTimestamp: 2022-01-28T07:01:24.193Z,
    ServerTimestamp: 2022-01-28T07:01:23.178Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057239573415265017078113082715299',
    FragmentSizeInBytes: 784770,
    ProducerTimestamp: 2022-01-28T07:01:26.049Z,
    ServerTimestamp: 2022-01-28T07:01:25.127Z,
    FragmentLengthInMilliseconds: 1795
  },
  {
    FragmentNumber: '91343852333185057244525175422158599662409003435',
    FragmentSizeInBytes: 733595,
    ProducerTimestamp: 2022-01-28T07:01:27.904Z,
    ServerTimestamp: 2022-01-28T07:01:26.802Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057249476935579300121271711940811',
    FragmentSizeInBytes: 863091,
    ProducerTimestamp: 2022-01-28T07:01:29.760Z,
    ServerTimestamp: 2022-01-28T07:01:28.701Z,
    FragmentLengthInMilliseconds: 1796
  },
  {
    FragmentNumber: '91343852333185057254428695736441642978804126889',
    FragmentSizeInBytes: 794767,
    ProducerTimestamp: 2022-01-28T07:01:31.620Z,
    ServerTimestamp: 2022-01-28T07:01:30.964Z,
    FragmentLengthInMilliseconds: 1792
  },
  {
    FragmentNumber: '91343852333185057259380455893583164563849312914',
    FragmentSizeInBytes: 815843,
    ProducerTimestamp: 2022-01-28T07:01:33.476Z,
    ServerTimestamp: 2022-01-28T07:01:32.772Z,
    FragmentLengthInMilliseconds: 1792
  }
]

the total duration of all fragments: 57421 MS

The M3U file:


#EXTM3U
#EXT-X-VERSION:7
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-TARGETDURATION:2
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MAP:URI="getMP4InitFragment.mp4?SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1"
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:35.931Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057105875891022195998207386831087&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:37.787Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057110827651179337519666762451998&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:39.643Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057115779411336479041184133649199&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:41.499Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057120731171493620562686992995215&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:43.355Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057125682931650762084177611210926&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:45.211Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057130634691807903605621447927987&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:47.071Z
#EXTINF:1.853,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057135586451965045127000030563099&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:48.927Z
#EXTINF:1.853,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057140538212122186648471533739708&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:50.783Z
#EXTINF:1.852,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057145489972279328169811940549220&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:52.638Z
#EXTINF:1.853,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057150441732436469691175863905321&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:54.494Z
#EXTINF:1.853,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057155393492593611212512597876737&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:56.350Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057160345252750752733848109724600&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:00:58.206Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057165297012907894255185015258598&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:00.062Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057170248773065035776518554640603&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:01.918Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057175200533222177297851537695855&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:03.774Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057180152293379318819121358297568&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:05.630Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057185104053536460340391192576793&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:07.486Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057190055813693601862044276052305&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:09.346Z
#EXTINF:1.852,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057195007573850743383632007202899&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:11.201Z
#EXTINF:1.853,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057199959334007884905218529030717&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:13.057Z
#EXTINF:1.853,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057204911094165026426933483564105&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:14.913Z
#EXTINF:1.853,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057209862854322167948478491151138&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:16.769Z
#EXTINF:1.853,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057214814614479309470058753450625&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:18.625Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057219766374636450991675003849379&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:20.481Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057224718134793592513326318973487&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:22.337Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057229669894950734034908059013802&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:24.193Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057234621655107875556490501268794&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:26.049Z
#EXTINF:1.856,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057239573415265017078113082715299&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:27.904Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057244525175422158599662409003435&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:29.760Z
#EXTINF:1.857,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057249476935579300121271711940811&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:31.620Z
#EXTINF:1.853,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057254428695736441642978804126889&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-PROGRAM-DATE-TIME:2022-01-28T07:01:33.476Z
#EXTINF:1.853,
getMP4MediaFragment.mp4?FragmentNumber=91343852333185057259380455893583164563849312914&SessionToken=CiBFQ5hwk0gJ4y0yZel0pfEM048uaoiqfynqb3Or3PnbBRIQCNXzXfY3UUFHJCBRHnqwUhoZHHw48q4PWtF3iPhcl1vK0ilc18UrzzrePiIgSBupoTd0SDcBQUc4wGmBkVGaMshh0QRhSG3Y8Ku8I2Q~&TrackNumber=1
#EXT-X-ENDLIST

The total duration of all fragments: 5900 MS

As you can see the return fragments' numbers are identical but their length is different.
My question is how can it be?

Which other why do I have to get the duration of the fragments for some time range without retrieving the entire stream for that time range?

Thank you.

Is it possible for KCL to drop records?

Tried asking on StackOverflow but got no answer, perhaps one of the core maintainers can comment.

Basically AWS docs say that there are times where KCL can drop records if it receives an error. Whichever way we crash our node app, it just restarts with the same checkpoint as before, thus not dropping records.

What's the scenario where it can drop records? How do we replicate it? For our application it's paramount we don't, so if that scenario exists need to build in protection.

PS note we're already handling failures on the producer side, this is for the consumer specifically.

Would really appreciate feedback 🙏

Error when updating to aws-kcl 2.0.0 unrecognized classpath option "-cp"

==========================================================
Starting MultiLangDaemon ...
Unrecognized option: -cp "/var/app/current/node_modules/aws-kcl/lib/jars/amazon-kinesis-client-2.1.2.jar:/var/app/current/node_modules/aws-kcl/lib/jars/amazon-kinesis-client-multilang-2.1.2.jar:/var/app/current/node_modules/aws-kcl/lib/jars/animal-sniffer-annotations-1.14.jar:/var/app/current/node_modules/aws-kcl/lib/jars/annotations-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/apache-client-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/auth-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/aws-cbor-protocol-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/aws-core-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/aws-java-sdk-core-1.11.477.jar:/var/app/current/node_modules/aws-kcl/lib/jars/aws-json-protocol-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/aws-query-protocol-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/checker-qual-2.5.2.jar:/var/app/current/node_modules/aws-kcl/lib/jars/cloudwatch-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/commons-beanutils-1.9.3.jar:/var/app/current/node_modules/aws-kcl/lib/jars/commons-codec-1.10.jar:/var/app/current/node_modules/aws-kcl/lib/jars/commons-collections-3.2.2.jar:/var/app/current/node_modules/aws-kcl/lib/jars/commons-collections4-4.2.jar:/var/app/current/node_modules/aws-kcl/lib/jars/commons-io-2.6.jar:/var/app/current/node_modules/aws-kcl/lib/jars/commons-lang3-3.8.1.jar:/var/app/current/node_modules/aws-kcl/lib/jars/commons-logging-1.1.3.jar:/var/app/current/node_modules/aws-kcl/lib/jars/dynamodb-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/error_prone_annotations-2.1.3.jar:/var/app/current/node_modules/aws-kcl/lib/jars/flow-1.7.jar:/var/app/current/node_modules/aws-kcl/lib/jars/guava-26.0-jre.jar:/var/app/current/node_modules/aws-kcl/lib/jars/http-client-spi-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/httpclient-4.5.6.jar:/var/app/current/node_modules/aws-kcl/lib/jars/httpcore-4.4.10.jar:/var/app/current/node_modules/aws-kcl/lib/jars/ion-java-1.0.2.jar:/var/app/current/node_modules/aws-kcl/lib/jars/j2objc-annotations-1.1.jar:/var/app/current/node_modules/aws-kcl/lib/jars/jackson-annotations-2.9.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/jackson-core-2.9.8.jar:/var/app/current/node_modules/aws-kcl/lib/jars/jackson-databind-2.9.8.jar:/var/app/current/node_modules/aws-kcl/lib/jars/jackson-dataformat-cbor-2.9.8.jar:/var/app/current/node_modules/aws-kcl/lib/jars/jcommander-1.72.jar:/var/app/current/node_modules/aws-kcl/lib/jars/joda-time-2.8.1.jar:/var/app/current/node_modules/aws-kcl/lib/jars/jsr305-3.0.2.jar:/var/app/current/node_modules/aws-kcl/lib/jars/kinesis-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/logback-classic-1.2.3.jar:/var/app/current/node_modules/aws-kcl/lib/jars/logback-core-1.2.3.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-buffer-4.1.32.Final.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-codec-4.1.32.Final.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-codec-http-4.1.32.Final.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-codec-http2-4.1.32.Final.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-common-4.1.32.Final.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-handler-4.1.32.Final.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-nio-client-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-reactive-streams-2.0.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-reactive-streams-http-2.0.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-resolver-4.1.32.Final.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-transport-4.1.32.Final.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-transport-native-epoll-4.1.32.Final.jar:/var/app/current/node_modules/aws-kcl/lib/jars/netty-transport-native-unix-common-4.1.32.Final.jar:/var/app/current/node_modules/aws-kcl/lib/jars/profiles-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/protobuf-java-2.6.1.jar:/var/app/current/node_modules/aws-kcl/lib/jars/protocol-core-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/reactive-streams-1.0.2.jar:/var/app/current/node_modules/aws-kcl/lib/jars/regions-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/rxjava-2.1.14.jar:/var/app/current/node_modules/aws-kcl/lib/jars/sdk-core-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/slf4j-api-1.7.25.jar:/var/app/current/node_modules/aws-kcl/lib/jars/sts-2.4.0.jar:/var/app/current/node_modules/aws-kcl/lib/jars/utils-2.4.0.jar:/var/app/current:/var/app/current"
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Can not run the sample.

I`m trying to run KCL with localstack

Always getting:
Caused by: io.netty.handler.codec.http2.Http2Exception: First received frame was not SETTINGS. Hex dump for first 5 bytes: 3c21444f43

Error when running the basic producer sample

When I run the basic producer sample according to the docs, I get this error:

/Users/ttt/repo/ratingbutler/kinesis/amazon-kinesis-client-nodejs/node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^

TypeError: "callback" argument must be a function
    at exports.setTimeout (timers.js:327:11)
    at /Users/ttt/repo/ratingbutler/kinesis/amazon-kinesis-client-nodejs/samples/basic_sample/producer/sample_producer.js:101:11

The offending line is in 101 where the pass in param is not a function. I will open a PR to address this.

SEVERE: class FastBuffer extends Uint8Array {}]

Used the KPL to ingest data and got this one:

INFO: Starting: Reading next message from STDIN for shardId-000000000017
Jul 11, 2017 9:05:28 PM com.amazonaws.services.kinesis.multilang.MessageWriter call
INFO: Message size == 719833 bytes for shard shardId-000000000017
Jul 11, 2017 9:05:28 PM com.amazonaws.services.kinesis.multilang.DrainChildSTDERRTask handleLine
SEVERE: Received error line from subprocess [buffer.js:8] for shard shardId-000000000017
buffer.js:8
Jul 11, 2017 9:05:28 PM com.amazonaws.services.kinesis.multilang.DrainChildSTDERRTask handleLine
SEVERE: Received error line from subprocess [class FastBuffer extends Uint8Array {}] for shard shardId-000000000017
class FastBuffer extends Uint8Array {}
Jul 11, 2017 9:05:28 PM com.amazonaws.services.kinesis.multilang.DrainChildSTDERRTask handleLine
SEVERE: Received error line from subprocess [^] for shard shardId-000000000017
Jul 11, 2017 9:05:28 PM com.amazonaws.services.kinesis.multilang.DrainChildSTDERRTask handleLine
SEVERE: Received error line from subprocess [] for shard shardId-000000000017
Jul 11, 2017 9:05:28 PM com.amazonaws.services.kinesis.multilang.DrainChildSTDERRTask handleLine
SEVERE: Received error line from subprocess [RangeError: Maximum call stack size exceeded] for shard shardId-000000000017
RangeError: Maximum call stack size exceeded
Jul 11, 2017 9:05:28 PM com.amazonaws.services.kinesis.multilang.DrainChildSTDERRTask handleLine
SEVERE: Received error line from subprocess [ at Buffer.Uint8Array (native)] for shard shardId-000000000017
at Buffer.Uint8Array (native)

my dependencies:
"dependencies": {
"async": "^2.4.1",
"aws-kinesis-agg": "^2.2.2",
"aws-sdk": "^2.79.0",
"aws-xray-sdk": "^1.1.1",
"requestretry": "^1.12.0",
"util": "^0.10.3",
"uuid": "^3.1.0",
"winston": "^2.3.1",
"aws-kcl": "^0.6.0"
},

Memory leak when writing intensively

I have a memory leak problem writing to kinesis from node and I wanted to verify if it happens as well with this sample producer.

I verified that if instead of 10 writes, you modify the code to write endlessly 10 times per second, the memory consumption grows slowly but steadily.

Any idea why this happens?

missing java class when attempting to kick off the consumer in clickstream sample

Getting the following errors when executing the consumer in clickstream sample.

vagrant@vagrant:~/amazon-kinesis-client-nodejs/samples/click_stream_sample/consumer$ ~/amazon-kinesis-client-nodejs/bin/kcl-bootstrap --java /usr/bin/java -p ./sample.properties -e
/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/commons-codec-1.3.jar downloaded. 9 files remain.
/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/joda-time-2.4.jar downloaded. 8 files remain.
/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/aws-java-sdk-1.7.13.jar downloaded. 7 files remain.
/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/jackson-databind-2.1.1.jar downloaded. 6 files remain.
/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/commons-logging-1.1.1.jar downloaded. 5 files remain.
/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/amazon-kinesis-client-1.2.0.jar downloaded. 4 files remain.
/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/jackson-core-2.1.1.jar downloaded. 3 files remain.
/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/httpclient-4.2.jar downloaded. 2 files remain.
/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/httpcore-4.2.jar downloaded. 1 files remain.
/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/jackson-annotations-2.1.1.jar downloaded. 0 files remain.
==========================================================
/usr/bin/java -cp /home/vagrant/amazon-kinesis-client-nodejs/lib/jars/amazon-kinesis-client-1.2.0.jar:/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/aws-java-sdk-1.7.13.jar:/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/commons-codec-1.3.jar:/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/commons-logging-1.1.1.jar:/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/httpclient-4.2.jar:/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/httpcore-4.2.jar:/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/jackson-annotations-2.1.1.jar:/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/jackson-core-2.1.1.jar:/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/jackson-databind-2.1.1.jar:/home/vagrant/amazon-kinesis-client-nodejs/lib/jars/joda-time-2.4.jar:/home/vagrant/amazon-kinesis-client-nodejs/samples/click_stream_sample/consumer:/home/vagrant/amazon-kinesis-client-nodejs/samples/click_stream_sample/consumer com.amazonaws.services.kinesis.multilang.MultiLangDaemon ./sample.properties
==========================================================
Starting Multi-Lang Daemon ...
Exception in thread "main" java.lang.NoClassDefFoundError: com/amazonaws/auth/AWSCredentialsProvider
    at com.amazonaws.services.kinesis.clientlibrary.config.KinesisClientLibConfigurator.<init>(KinesisClientLibConfigurator.java:63)
    at com.amazonaws.services.kinesis.multilang.MultiLangDaemon.main(MultiLangDaemon.java:197)
Caused by: java.lang.ClassNotFoundException: com.amazonaws.auth.AWSCredentialsProvider
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    ... 2 more

Updating pom.xml

Can we just update pim.xml file to point to the latest version of https://github.com/awslabs/amazon-kinesis-client or do we need to make additional changes to the code base.. I am interested in using the AT_TIMESTAMP option to sync streams.

Any help would be appreciated

Notify nodejs code when process terminates

Is there anyway to notify the nodejs code when the process shuts down?

awslabs / amazon-kinesis-client-nodejs Goto Github PK

amazon-kinesis-client-nodejs's People

Stargazers

Watchers

Forkers

amazon-kinesis-client-nodejs's Issues

Recommend Projects

Recommend Topics

Recommend Org