Coder Social home page Coder Social logo

ges-client's Introduction

ges-client

![Gitter](https://badges.gitter.im/Join Chat.svg)

A nodejs client library for (Get) Event Store

Build Status

NPM NPM

Want to help out? Check out a waiting issue Stories in Ready

Introduction

This client assumes that you already have an Event Store instance running.

Basic usage

Install

$ npm install ges-client

Append and read from an event stream

var ges = require('ges-client')

// 1) Create a connection to a running EventStore
//    using default connection options and credentials
var connection = ges()
	, stream = 'intro-events'

// 2) Append events that can be read
connection.on('connect', function() {
	var appendData = {
	      expectedVersion: ges.expectedVersion.emptyStream
	    , events: [array of events]
			}
	connection.appendToStream(stream, appendData, function(err, appendResult) {
	  if(err) return console.log('Ooops!', err) // connection error
  	
	  // 3) Read first events from the stream
  	connection.readStreamEventsForward(stream, { start: 0, count: 1 }, function(err, readResult) {
	    if(err) return console.log('Ooops!', err) // connection error or stream does not exist

	    // ta da!
  		console.log(readResult.Events)
  	})
  })
})

Subscribe to stream updates

var ges = require('ges-client')

// 1) Create a connection to a running EventStore
//    using default connection options and credentials
var connection = ges()
	, stream = 'intro-events'

// 2) Create a subscription
connection.on('connect', function() {
  var subscription = connection.subscribeToStream(stream)

	// 3) Listen for events
  subscription.on('event', function(evt) {
  	// ta da!
  	console.log(evt)
  })

	var appendData = {
	      expectedVersion: ges.expectedVersion.emptyStream
	    , events: [array of events]
			}
	connection.appendToStream(stream, appendData, function(err, appendResult) {
	  if(err) return console.log('Ooops!', err) // connection error
  })
})

Major Feature Status

  • Connection (Partial - not all events are available)
  • Append to stream (Complete)
  • Read from stream (Complete)
  • Subscriptions (Complete)
  • Read from all (Complete)
  • Stream ACLs (Not Started)
  • Transactions (Complete)
  • Stream deletes (Complete)
  • Stream metadata (Complete)

License & copyright

Copyright (c) 2014 Brian Mavity.

ges-client is licensed under the MIT license. All rights not explicitly granted in the MIT license are reserved. See the included LICENSE.md file for more details.

Special Thanks

Special thanks to Ken Pratt for writing the first version of this over at https://github.com/kenpratt/nodejs-EventStore . It made early development go by much faster.

ges-client's People

Contributors

bmavity avatar gitter-badger avatar waffle-iron avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ges-client's Issues

Consider using protobufjs for compatibility with recent node version

Explore the possibility of using protobufjs package instead of protobuf.

The protobuf package use native code which is blocking compatibility with node version higher than 0.10.x. Relying on a package that doesn't require native bindings would be better. protobufjs seems to be implementing the full protocol. Of course someone would need to dive into the specific to make sure that it supports what is needed for ges-client.

Note: the protobuf package seems to compile with node 0.12 despite the warning, but I guess it is untested.

Implement Connection Tests

The current hacky version of the connection is making it difficult to diagnose false build failures. All equivalent connection related tests should be migrated.

Simple append example fails

const uuid = require('node-uuid');
const EventStore = require('ges-client');

const connection = EventStore();

connection.on('connect', ()=> {
    const streamId = 'intro-stream';
    const appendData = {
        expectedVersion:EventStore.expectedVersion.noStream,
        events: [
            {
                eventType: 'Foo',
                eventId: uuid.v4(),
                data: {
                    name: `Hello World!`,
                    number: Math.random()
                },
                metadata: {
                    addedDate: new Date().toISOString()
                }
            }
        ]
    };

    connection.appendToStream(streamId, appendData, (err, results)=> {
        if (err) return console.error(`Ooops`, err);

        connection.readStreamEventsForward(streamId, {start: 0, count: 1}, (err, results)=> {
            if (err) return console.error(`Oppps read:`, err);

            console.log(results.Events);
        });
    });
});

This gives me an error of

node_modules\node-uuid\uuid.js:67
    s.toLowerCase().replace(/[0-9a-f]{2}/g, function(oct) {
     ^

TypeError: Cannot read property 'toLowerCase' of undefined

Looking through the code it appears you are using EventId, though the README.md doesn't discuss the internals of what events look like. Captializing EventType, EventId, Data and Metadata however gives me

protobufjs\node_modules\bytebuffer\dist\ByteBufferNB.js:378
                throw TypeError("Illegal buffer");
                ^

TypeError: Illegal buffer

What would be the correct version of the initial test code?

data type/schema issue

Hey, I'm getting this error
throw TypeError("Illegal str: Length not a multiple of 4");
so I looked at a bunch of stuff then I looked at the data schema. I'm not sure if i have the right format for my data. The schema ( in ges_client.proto ) is
message NewEvent {
required bytes event_id = 1;
required string event_type = 2;
required int32 data_content_type = 3;
required int32 metadata_content_type = 4;
required bytes data = 5;
optional bytes metadata = 6;
}
the schema of my events looks like this
EventId : uuid.v4(),
Type : eventName,
IsJson : true,
Data : data,
Metadata: metadata
this is what I was using in the past, and it seemed to work ok. However, it certainly doesn't look like a direct map.
lastly here is what my test event data looks like
EventId: uuid.v4() ,Type:'bootstrapApplication', IsJson: true, Data: { data:'bootstrap please' }, Metadata: {commandTypeName:'bootstrapApplication'}
Here's my stack trace.
thanks
Raif

            throw TypeError("Illegal str: Length not a multiple of 4");

plugin_1 | ^
plugin_1 | TypeError: Illegal str: Length not a multiple of 4
plugin_1 | at TypeError (native)
plugin_1 | at Function.module.exports.ByteBuffer.fromBase64 (/home/opt/app/node_modules/ges-client/node_modules/protobufjs/node_modules/bytebuffer/dist/ByteBufferNB.js:2864:23)
plugin_1 | at Function.module.exports.ByteBuffer.wrap (/home/opt/app/node_modules/ges-client/node_modules/protobufjs/node_modules/bytebuffer/dist/ByteBufferNB.js:332:39)
plugin_1 | at [object Object].ProtoBuf.Reflect.ElementPrototype.verifyValue (/home/opt/app/node_modules/ges-client/node_modules/protobufjs/dist/ProtoBuf.js:1937:39)
plugin_1 | at [object Object].ProtoBuf.Reflect.FieldPrototype.verifyValue (/home/opt/app/node_modules/ges-client/node_modules/protobufjs/dist/ProtoBuf.js:3536:33)
plugin_1 | at [object Object].MessagePrototype.set (/home/opt/app/node_modules/ges-client/node_modules/protobufjs/dist/ProtoBuf.js:2542:59)
plugin_1 | at [object Object].MessagePrototype.set (/home/opt/app/node_modules/ges-client/node_modules/protobufjs/dist/ProtoBuf.js:2533:38)
plugin_1 | at [object Object].Message (/home/opt/app/node_modules/ges-client/node_modules/protobufjs/dist/ProtoBuf.js:2462:34)
plugin_1 | at [object Object].ProtoBuf.Reflect.ElementPrototype.verifyValue (/home/opt/app/node_modules/ges-client/node_modules/protobufjs/dist/ProtoBuf.js:1976:28)
plugin_1 | at [object Object].ProtoBuf.Reflect.FieldPrototype.verifyValue (/home/opt/app/node_modules/ges-client/node_modules/protobufjs/dist/ProtoBuf.js:3517:43)
plugin_1 | at [object Object].MessagePrototype.set (/home/opt/app/node_modules/ges-client/node_modules/protobufjs/dist/ProtoBuf.js:2542:59)
plugin_1 | at [object Object].MessagePrototype.set (/home/opt/app/node_modules/ges-client/node_modules/protobufjs/dist/ProtoBuf.js:2533:38)
plugin_1 | at [object Object].Message (/home/opt/app/node_modules/ges-client/node_modules/protobufjs/dist/ProtoBuf.js:2462:34)
plugin_1 | at Object.serialize (/home/opt/app/node_modules/ges-client/tcp/messageParser.js:28:9)
plugin_1 | at Object.toRequestPayload (/home/opt/app/node_modules/ges-client/tcp/operations.js:70:26)
plugin_1 | at OperationItem.toTcpMessage (/home/opt/app/node_modules/ges-client/tcp/operations.js:19:24)
plugin_1 | at OperationsManager.scheduleOperation (/home/opt/app/node_modules/ges-client/tcp/operationsManager.js:35:38)
plugin_1 | at EsConnectionLogicHandler.states.Connected.StartOperation (/home/opt/app/node_modules/ges-client/tcp/connectionLogicHandler.js:77:23)
plugin_1 | at EsConnectionLogicHandler._processNextMessage (/home/opt/app/node_modules/ges-client/tcp/connectionLogicHandler.js:236:10)
plugin_1 | at Immediate._onImmediate (/home/opt/app/node_modules/ges-client/tcp/connectionLogicHandler.js:212:6)
plugin_1 | at processImmediate as _immediateCallback
plugin_1 |

Testing Matrix (node and EventStore)

Hi All,

Right now, the test are running the latest node version is 0.10.x and EventStore v3.0.5. We have to expand out our test coverage, but I am not sure how to prioritize. What versions of node and/or EventStore are people targeting?

B

Improve user friendliness of createEventData

Issues like #54 will come up because client.createEventData requires that data and metadata objects are buffers. Need to modify eventData.js to allow objects to be passed in and then automatically serialize and set property as a Buffer.

v0.10.0 Release

Hi All,

I need to figure out how to automatically generate a changelog, but for now I'd like to make a v0.9.3 release this weekend. It will definitely include:

  • Memory Leak Fix #40 ( via #39 )
  • Babel error Fix ( via #47 ) (thanks @reharik)
  • More stable test runner #33 ( via #46 )
  • Dropping a Catch Up Subscription is now async, it was causing subscription to not properly update it's last processed event
  • Subscription Operations now verify a connection is writable before attempting to send a request to the server
  • Change to use protobufjs library instead of abandoned protobuf library ( via #51 )

I will try to get to the connection retry, or at least raising the connection event. However, if I cannot get to it I would still like to get a release out. Anyone have any thoughts?

Tcp disconnect doesn't emit an error on tcpConnection

When the underlying tcp connection is disconnected by an error the error doesn't bubble up to the tcpConnection.

Right now there is no way to know that we are disconnected, so we can't take any compensating actions.

I think I could submit a PR for this one.

Is this still being used/maintained?

Sorry for making this an issue, but we just found this and it's pretty nice. Almost exactly what we're looking for.

I tried the Gitter room, but it was empty and the last message was long ago and far away...

Anyway...are the maintainers still using this?

Reading a stream to the end?

Currently, readStreamEventsForward (and Backward) require a count field to be in the options (see code here).

Is there a reason for requiring this? This doesn't allow for reading a stream to the end, which I think is a fairly common use case.

Performance

I would be interested in seeing some performance benchmarks on the client. Are their any?

Properly identify operation errors

If a server response message has an .error property, the operation code still identifies it as a successful response and does not report an error.

Code to reproduce an example

var ges = require('ges-client')
    , connection = ges({})

module.exports.readStream = readStream

function readStream(streamName, cb) {
    connection.readStreamEventsForward(streamName, function(err, events) {
        if(err) return cb(err)

        cb(err, events)
    })
}

With response

{ auth: undefined,
  cb: undefined,
  requestType: 'ReadStreamEventsForward',
  toRequestPayload: [Function],
  responseType: 'ReadStreamEventsCompleted',
  toResponseObject: [Function] } { result: 'Error',
  nextEventNumber: -1,
  lastEventNumber: -1,
  isEndOfStream: true,
  lastCommitPosition: '3463749',
  error: 'maxCount should be positive.\nParameter name: maxCount' }

Fix Travis Build

Travis build keeps failing while running tests because (educated guess) the version of protobuf is not parsing nested structures properly.

Tests all pass locally and on an EC2 instance, but this may be an issue for people going forward. Might have to move to a node module that does not require protobuf to be installed.

Nodify Client API

After feature parity is achieved, add some node specific features to be discussed here.

  1. Read streams for read calls and subscriptions.

error on operations.js no callback

Hey, sorry for the dumb title can't think of how to say it.
I'm running a test that calls connection.setStreamMetadata. It doesn't assert anything which is why it passes. oh and I have the call to connection.setStreamMetadata in a setTimeout so that the connection can be established before making the call. Hope this explains the sequence of logs (shown below)
The connection is live as indicated by a console.log (shown below).
I put a console.log in connectionLogicHandler ln 235. It says "next" then dumps the next var.
at the end of my console dump below, you'll see the error statement and all the "next"s.
perhaps the best thing is to start below and see the stacktrace and this may make more sense.
Please let me know if there is anything else I can provide.

thanks,
r

next
{ name: 'StartConnection',
data: { endPoint: { host: '172.17.0.76', port: 1113 } },
cb: [Function] }
next
{ name: 'StartConnection',
data: { endPoint: { host: '172.17.0.76', port: 1113 } },
cb: [Function] }
appendToStreamPromiseTester
next
{ name: 'EstablishTcpConnection',
data: { endPoint: { host: '172.17.0.76', port: 1113 } },
cb: [Function] }
next
{ name: 'EstablishTcpConnection',
data: { endPoint: { host: '172.17.0.76', port: 1113 } },
cb: [Function] }
append to stream
next
{ name: 'TcpConnectionEstablished',
data:
{ connection:
{ domain: null,
_events: [Object],
_maxListeners: undefined,
_socket: [Object],
_closeCallbacks: [] } } }
next
{ name: 'TcpConnectionEstablished',
data:
{ connection:
{ domain: null,
_events: [Object],
_maxListeners: undefined,
_socket: [Object],
_closeCallbacks: [] } } }
โœ“ should resolve with success

1 passing (374ms)

next
{ name: 'HandleTcpPackage',
data:
{ connection:
{ domain: null,
_events: [Object],
_maxListeners: undefined,
_socket: [Object],
_closeCallbacks: [] },
package:
{ messageName: 'HeartbeatRequestCommand',
flag: 'None',
correlationId: '73b8a4de-adae-7c47-8160-30ee9bc3ae9d',
payload: } } }
----------- connection object ----------------
{ domain: null,
_events: {},
_maxListeners: undefined,
_endPoint: { host: '172.17.0.76', port: 1113 },
_handler:
{ domain: null,
_events: { connect: [Function] },
_maxListeners: undefined,
_handlers: {},
_connection:
{ domain: null,
_events: [Object],
_maxListeners: undefined,
_socket: [Object],
_closeCallbacks: [] },
_endPoint: { host: '172.17.0.76', port: 1113 },
_state:
{ CloseConnection: [Function: performCloseConnection],
EstablishTcpConnection: [Function: noOp],
HandleTcpPackage: [Function: handlePackage],
StartConnection: [Function],
StartOperation: [Function],
StartSubscription: [Function],
TcpConnectionEstablished: [Function: noOp] },
_queuedMessages: [],
_operations: { _activeOperations: {}, _waitingOperations: [] },
_subscriptions: { _activeSubscriptions: {}, _waitingSubscriptions: [] },
_connectingPhase: 'Connected' } }
next
{ name: 'StartOperation',
data:
{ name: 'AppendToStream',
stream: '$$$all',
auth: { username: 'admin', password: 'changeit' },
data: { expectedVersion: -1, events: [Object] },
cb: undefined } }
next
{ name: 'HandleTcpPackage',
data:
{ connection:
{ domain: null,
_events: [Object],
_maxListeners: undefined,
_socket: [Object],
_closeCallbacks: [] },
package:
{ messageName: 'HeartbeatRequestCommand',
flag: 'None',
correlationId: '63cb297e-5bd9-1449-87b6-b651c74f67af',
payload: } } }
next
{ name: 'HandleTcpPackage',
data:
{ connection:
{ domain: null,
_events: [Object],
_maxListeners: undefined,
_socket: [Object],
_closeCallbacks: [] },
package:
{ messageName: 'WriteEventsCompleted',
flag: 'None',
correlationId: 'a27e6264-d590-485d-8c7e-456c5f132743',
payload: <Buffer 08 04 12 17 57 72 6f 6e 67 20 65 78 70 65 63 74 65 64 20 76 65 72 73 69 6f 6e 2e 18 80 80 80 80 f8 ff ff ff ff 01 20 80 80 80 80 f8 ff ff ff ff 01 28 ... > } } }
/home/rharik/Development/MethodFitness/mf_workflows/node_modules/eventstore/node_modules/ges-client/tcp/operations.js:52
return cb(new Error(payload.message))
^
TypeError: undefined is not a function
at OperationItem.finish (/home/rharik/Development/MethodFitness/mf_workflows/node_modules/eventstore/node_modules/ges-client/tcp/operations.js:52:11)
at EsConnectionLogicHandler.handlePackage (/home/rharik/Development/MethodFitness/mf_workflows/node_modules/eventstore/node_modules/ges-client/tcp/connectionLogicHandler.js:122:13)
at EsConnectionLogicHandler._processNextMessage (/home/rharik/Development/MethodFitness/mf_workflows/node_modules/eventstore/node_modules/ges-client/tcp/connectionLogicHandler.js:238:10)
at Immediate._onImmediate (/home/rharik/Development/MethodFitness/mf_workflows/node_modules/eventstore/node_modules/ges-client/tcp/connectionLogicHandler.js:212:6)
at processImmediate as _immediateCallback

Memory Leak

There seem to be some memory leak issues which are causing some problems on long-running tasks.

maxRecordCount generates an error when reading events

Hi,

I was trying to wrap ges-client in a repository in my code.
And I wanted to query every single events of a stream to reload my aggregate state.
So as the API needs us to provide a start and count, I provided 0 and client.maxRecordCount.
But my test was failing and I had this as a log:

{ Status: 'Error',
  Events: [],
  NextEventNumber: -1,
  LastEventNumber: -1,
  IsEndOfStream: true }

For some reason, I tried to use a different value for the property count.
And after some dichotomy, I finally found that when count is above 4096, I get the same error.
But when count is lower than 4096 or equal, then I get my events.
Here are my events:

[ { Event: 
     { EventStreamId: 'TestStream-1',
       EventId: 'cb0a92bb-ae66-492f-901e-f259f7b38d88',
       EventNumber: 50,
       EventType: 'TestStream',
       Data: <Buffer 7b ...>,
       Metadata: <Buffer 7b ...>,
       IsJson: true,
       Created: '635881237258614740',
       CreatedEpoch: '1452526925861' },
    IsResolved: false,
    Link: null,
    OriginalEvent: 
     { EventStreamId: 'TestStream-1',
       EventId: 'cb0a92bb-ae66-492f-901e-f259f7b38d88',
       EventNumber: 50,
       EventType: 'TestStream',
       Data: <Buffer 7b ...>,
       Metadata: <Buffer 7b ...>,
       IsJson: true,
       Created: '635881237258614740',
       CreatedEpoch: '1452526925861' },
    OriginalEventNumber: 50,
    OriginalPosition: null,
    OriginalStreamId: 'TestStream-1' } ]

Any idea on what's happening ?

I used EventStore v3.4.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.