HI all,

When I submit docker-compose command and found that the gmaster could not be started normally, with exit state 1,

root@nb1:/proj/speedd/speedd-docker-master# docker-compose ps
Name Command State Ports

speedddockermaster_client_1 /bin/sh -c /opt/start-clie ... Exit 137
speedddockermaster_gmaster_1 bash start.sh --with-gmond Exit 1
speedddockermaster_kafka_1 /bin/sh -c start-kafka.sh Exit 137
speedddockermaster_nimbus_1 /bin/sh -c /usr/bin/start- ... Exit 137
speedddockermaster_storm-ui_1 /bin/sh -c /usr/bin/start- ... Exit 137
speedddockermaster_supervisor_1 /bin/sh -c /usr/bin/start- ... Exit 137
speedddockermaster_ui_1 /bin/sh -c /opt/start-ui.sh Exit 0
speedddockermaster_zookeeper_1 /bin/sh -c /usr/sbin/sshd ... Exit 137
root@nb1:/proj/speedd/speedd-docker-master#

Should uniformly distribute events based on the sensor id.

Real vs. Simulated time

• replaying a csv file – requires simulated time
• currently limitation in Proton – does not allow simulated time

Time Zones

• limitation – not supported in Proton

Timestamp (long) vs. Date vs. jsr310 ZonedDateTime

• see [http://www.threeten.org/|(http://www.threeten.org/)] for more info

Unable to start the UI with NPM error
when I tried to run ui docker, and found the following error to install npm, any ideas why?

ui_1 | SPEEDD UI - starting
ui_1 | Starting sshd
ui_1 | Installing required node modules
ui_1 | /opt/start-ui.sh: 7: cd: can't cd to /opt/speedd/speedd_ui_tm_version2
ui_1 | npm info it worked if it ends with ok
ui_1 | npm info using [email protected]
ui_1 | npm info using [email protected]
ui_1 | npm ERR! install Couldn't read dependencies
ui_1 | npm ERR! Linux 4.13.0-21-generic
ui_1 | npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install"
ui_1 | npm ERR! node v4.2.6
ui_1 | npm ERR! npm v2.14.12
ui_1 | npm ERR! path /package.json
ui_1 | npm ERR! code ENOPACKAGEJSON
ui_1 | npm ERR! errno -2
ui_1 | npm ERR! syscall open
ui_1 |
ui_1 | npm ERR! package.json ENOENT: no such file or directory, open '/package.json'
ui_1 | npm ERR! package.json This is most likely not a problem with npm itself.
ui_1 | npm ERR! package.json npm can't find a package.json file in your current directory.
ui_1 |
ui_1 | npm ERR! Please include the following file with any support request:
ui_1 | npm ERR! /npm-debug.log
ui_1 | Sleep 1m till kafka is up and topics initialized

Need to have a mechanism to configure various DM (and not only) parameters at runtime.
A use case: the user can define highest/lowest metering rate limits in UI, and the new values will be used by the DM.

ProtonOnStorm uses {Name, attributes} tuple scheme for events.
Worth consideration using Event API (org.speedd.data.Event) for better encapsulation and semantics (e.g. Event.getDetectionTime() where detection time is not just one of the attributes).
We could possibly make Event part of ProtonOnStorm API.

In the TrafficDecisionMakerBolt.get_criticalDensity(location), if location is null or wrong, the get_cellId() returns -1, and the index addressed is '-2' - out of bounds.
Should harden this code.

The stack trace is below.
java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: -2
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQue
ue.java:128) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(Disrupt
orQueue.java:99) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.disruptor$consume_batch_when_available.invoke(disrupto
r.clj:80) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.daemon.executor$fn__3441$fn__3453$fn__3500.invoke(exec
utor.clj:748) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463) ~[storm-c
ore-0.9.3.jar:0.9.3]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
Caused by: java.lang.ArrayIndexOutOfBoundsException: -2
at org.speedd.dm.TrafficDecisionMakerBolt.get_criticalDensity(TrafficDec
isionMakerBolt.java:198) ~[stormjar.jar:na]
at org.speedd.dm.TrafficDecisionMakerBolt.computeNewRate(TrafficDecision
MakerBolt.java:153) ~[stormjar.jar:na]

I followed the following steps
Stream events into SPEEDD (example for the traffic management use case):

1. Open ssh to the client container
2. cd /opt/src/speedd/speedd-runtime/scripts/traffic
3. ./playevents-traffic-docker

However, it shows ERROR on kafka I/O, may I know anyone know why?
root@db5e62928fe5:/opt/speedd/speedd/speedd-runtime/scripts/traffic# ./playevents-traffic-docker
Loading producer configuration from ../producer-docker.properties
Event file: simulator_data_incident_shortened.csv
Sending events to topic: speedd-traffic-in-events
Properties loaded:{bootstrap.servers=kafka:9092, compression.codec=none, value.serializer=org.apache.kafka.common.serialization.StringSerializer, producer.type=sync, key.serializer=org.apache.kafka.common.serialization.StringSerializer}
15 [main] INFO org.speedd.TimedEventFileReader - Open file simulator_data_incident_shortened.csv for read.
29 [main] INFO org.apache.kafka.clients.producer.ProducerConfig - ProducerConfig values:
value.serializer = class org.apache.kafka.common.serialization.StringSerializer
key.serializer = class org.apache.kafka.common.serialization.StringSerializer
block.on.buffer.full = true
retry.backoff.ms = 100
buffer.memory = 33554432
batch.size = 16384
metrics.sample.window.ms = 30000
metadata.max.age.ms = 300000
receive.buffer.bytes = 32768
timeout.ms = 30000
max.in.flight.requests.per.connection = 5
bootstrap.servers = [kafka:9092]
metric.reporters = []
client.id =
compression.type = none
retries = 0
max.request.size = 1048576
send.buffer.bytes = 131072
acks = 1
reconnect.backoff.ms = 10
linger.ms = 0
metrics.num.samples = 2
metadata.fetch.timeout.ms = 60000

155 [main] WARN org.apache.kafka.clients.producer.ProducerConfig - The configuration compression.codec = null was supplied but isn't a known config.
155 [main] WARN org.apache.kafka.clients.producer.ProducerConfig - The configuration producer.type = null was supplied but isn't a known config.
155 [main] INFO org.speedd.TimedEventFileReader - Open producer to send to kafka: {bootstrap.servers=kafka:9092, compression.codec=none, value.serializer=org.apache.kafka.common.serialization.StringSerializer, producer.type=sync, key.serializer=org.apache.kafka.common.serialization.StringSerializer}
156 [main] INFO org.speedd.TimedEventFileReader - Start streaming events, start timestamp: 1513567782570
258 [kafka-producer-network-thread | producer-1] WARN org.apache.kafka.common.network.Selector - Error in I/O with kafka/172.17.0.4
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
at org.apache.kafka.common.network.Selector.poll(Selector.java:238)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
at java.lang.Thread.run(Thread.java:748)

In v1 architecture we have a single big topology that includes CEP and DM. From scalability, testability, and manageability perspectives it may be easier to build it as 2 topologies, separating CEP and DM into separate topologies.
This way, the system would become more modular, facilitating composition of the resulting environment for each use case. For instance, traffic use case and CCF use case each require completely different implementation of DM, however CEP has the same topology for all use cases.

Read https://storm.incubator.apache.org/documentation/Guaranteeing-message-processing.html

Unable to start STORM and NIMBUS with following error... any thoughts?

ui_1 | Setting up Kafka clients
zookeeper_1 | 2018-01-14 00:46:00,032 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /172.17.0.6:49746
zookeeper_1 | 2018-01-14 00:46:00,037 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@882] - Connection request from old client /172.17.0.6:49746; will be dropped if server is in r-o mode
zookeeper_1 | 2018-01-14 00:46:00,037 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@928] - Client attempting to establish new session at /172.17.0.6:49746
zookeeper_1 | 2018-01-14 00:46:00,050 [myid:] - INFO [SyncThread:0:ZooKeeperServer@673] - Established session 0x160f22047fb0001 with negotiated timeout 30000 for client /172.17.0.6:49746
ui_1 | Offset data: {"speedd-traffic-actions":{"0":[545]},"speedd-traffic-out-events":{"0":[21421]}}
ui_1 | Actions offset: 545
ui_1 | Events offset: 21421
ui_1 | Setting up Consumer on-message event

nimbus_1 | 2018-01-14 00:47:09,704 INFO exited: storm-nimbus (exit status 13; not expected)
nimbus_1 | 2018-01-14 00:47:10,709 INFO spawned: 'storm-nimbus' with pid 148
supervisor_1 | 2018-01-14 00:47:10,741 INFO exited: storm-supervisor (exit status 13; not expected)
nimbus_1 | 2018-01-14 00:47:11,712 INFO success: storm-nimbus entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
supervisor_1 | 2018-01-14 00:47:11,743 INFO spawned: 'storm-supervisor' with pid 142
supervisor_1 | 2018-01-14 00:47:12,746 INFO success: storm-supervisor entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
nimbus_1 | 2018-01-14 00:49:00,851 INFO exited: storm-nimbus (exit status 13; not expected)
nimbus_1 | 2018-01-14 00:49:01,856 INFO spawned: 'storm-nimbus' with pid 218
supervisor_1 | 2018-01-14 00:49:01,946 INFO exited: storm-supervisor (exit status 13; not expected)

Uniformly partition based on the card pan code