nhomble / temp-readings

temperature readings

Setup

$ cd deploy/
$ docker-compose --env-file .env up -d
$ create-topic SOME_TOPIC

Setup your [Kaggle] credentials by downloading kaggle.json with your own credentials.

$ cd deploy/
$ download-data.sh
$ cat temperature-readings-iot-devices.zip

Start publishing 'real time' data to kafka by running

$ cd deploy/
$ publish.sh SOME_TOPIC

Build our topology jar

$ mvn clean package
$ cp ./path/to/target/jar ./deploy/nimbus/deploy

If you are using the deploy/ directory as is, then the docker-compose will mount deploy/nimbus/deploy to /deploy on the container. So open a shell in that container

$ docker exec -it nimbus /bin/bash

Once you are in the container, then the last step will be to submit the topology into storm. To make it easier to script, I just added a run.sh into that location so that you can just exec in the shell.

# cd /deploy
# TYPE=temp
# sh run_${TYPE}.sh

Finally, you can observe the topology via storm-ui.

To kill a running topology, just open a shell in the nimbus container and run:

# TOPOLOGY_NAME=temperature
# WAIT_SECONDS=1
# storm kill $TOPOLOGY_NAME -w $WAIT_SECONDS