happner / happn-cluster Goto Github PK

0.0 0.0 0.0 1.9 MB

Extends happn with cluster ability

License: MIT License

JavaScript 99.08% Dockerfile 0.92%

happn-cluster's Introduction

Happner is a cloud application framework ideal for integrating multiple micro services into a unified offering. It enables the creation of an interconnected mesh of local and remote components. The mesh components can easily call upon each other's methods or listen to each other's events in a manner uncomplicated by remoteness.

HAPPNER-2 RELEASED

this version makes use of happn-3 as part of its data layer, the repo can be found here, with a migration plan.

installation

npm install happner --save

demonstration

These walkthroughs use happner to create a rudimentary monitoring service.

The Basics

documentation

happn-cluster's People

Contributors

Watchers

happn-cluster's Issues

swim faulty-list lives on indefinately

The list of members detected as faulty (not responding) is preserved by swim in order to resume at previous incarnation.

This list lives for ever and is sent (along with the list of active members) in response to join requests from new members. That is how any given joining member learns it's previous incarnation.

This would not be a problem in fairly static clusters.

But if a cluster grows and shrinks according to load and has dynamic ip that list will become very bloated over time.

Perhaps what is missing is a fourth swim message type depart to allow members to cleanly depart the cluster such that no remaining members recall a failed state whose incarnation needs to be exceeded.

Onward release latest happn/happner

A stopping cluster member does not properly disconnect replication clients that are already in a reconnect loop to a previously stopped cluster member.

This affects mainly tests and is generally only a problem where multiple nodes are started and stopped without ending their containing process.

The tests step around the problem by using unique ports per test, (grep testSequence).

eg.

given cluster (node1, node2, node3)
node3 is stopped, node2 and node1 both had replication client logged into node3
these clients are now in a reconnect loop
node2 is stopped, but its replication client remains attempting to log into node3

Test for possible issue in cluster stabilise.

https://travis-ci.com/FieldServer/tunnel-proxy/jobs/102157757

Persisted members: try to connect (swim) before http

Since persisted members are stored in DB, and will automatically try to be joined on container start, container should try to connect htem via swim (?) before http. Se also previous issue about member persistence

batching on intra-cluster messages

There is a saturation issue where a single cluster node that is hammered with requests would push changes to all the internal nodes and thus saturate the entire cluster with traffic. One mitigation against this would be the batching of internal cluster notifications, so the network is more efficiently utilised internally.

cluster master out-of-sync does not rejoin existing peers

this is documented in the README, hosts must be configured to contain the entire cluster if we don't want members to be orphaned.

We need to persist members based on configuration so the hosts can be concatenated with previously joined members on startup, so we can grow the cluster without modifying everyones hosts config.

persistMembers lives on indefinitely

lives on indefinitely:
after release 6.1.0: nodes that are started with persistMembers:true will grow the list of cluster members indefinitely. We will need to be able to use ping and a timestamp to ensure the persisted members are pruned for inactive hosts.

Intermittent duplicate messages in cluster.

Could be happn-3 doing it.

1 in 30 or so appear to be duplicated. And not on all members.

No test.

Fix options.strategy equals null bug by forking Primus

In primus/dist/primus.js, on line 2380, test for primus.options before attempting to use it, ie:

if(primus.options){
  if(~primus.options.strategy.indexOf('disconnect')) {
    return primus.recovery.reconnect();
  }
}

Simultaneous cluster arrival/departure leak

node2 joins cluster
node1 detects node2 and begins connect
node1 is stopped before connect completed
node1 connection to node2 callback after node1 being stopped

see lib/orchestrator/member.stop()

swim payloads are insecure

If at any point we want to run a private cluster spanning a public network any 3rd party with a capacity to view network traffic between cluster members can learn the swim update dissemination payload structure, craft their own payload and send it to an existing member - at which point that member will disseminate the 3rd party's inclusion in the cluster to all members, each of which will then log into the 3rd party's happn server with the cluster's password. Thus exposing the password.