Coder Social home page Coder Social logo

rbccps-iisc / ideam Goto Github PK

View Code? Open in Web Editor NEW
14.0 9.0 9.0 91.02 MB

City Data Exchange & Analytics Stack (CDEAS) for Smart Cities

Home Page: https://smartcity.rbccps.org/

Shell 39.20% Lua 9.63% Python 48.23% JavaScript 2.94%
iot-platform smartcity-middleware iot smartcities

ideam's Introduction

City Data Exchange & Analytics Middleware (CDX) for Smart Cities

image

Architecture

image

image

Resources

Getting Started -------------

This guide will help you to quickly get started with a single node instance of CDX and start publishing data from a device.

  1. Install Docker :

    sudo apt-get install docker.io
  2. Add permission to user :

    sudo usermod -a -G docker $USER
  3. Logout and log back in.
  4. Run ifconfig and note down the name of the interface you're connected to.

    alternate textI

    In the above example the interface name is enp9s0

  5. Now run:

    sudo nmcli device show enp9s0

    Replace enp9s0 with your respective interface name. The output should be something like this

    alternate textI

    Note down the DNS from IP4.DNS section

  6. Add DNS obtained in the previous step to the file /etc/docker/daemon.json If the file does not exist, create it. The entry should look something like this :

    {"dns": ["8.8.8.8", "8.8.4.4"]}

    Add the obtained DNS to the above JSON array. So the final entry should look something like this:

    {"dns": ["8.8.8.8","8.8.4.4","168.95.1.1"]}

    Of course, the DNS would change according to your network.

  7. Add DNS in /etc/default/docker file as follows:

    DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4 --dns 168.95.1.1"
  8. Restart Docker :

    service docker restart

CDX Installation

  1. Clone CDX git repo :

    git clone https://github.com/rbccps-iisc/ideam.git

    CDX repository comes with a default configuration file ideam.conf:

    [APIGATEWAY]
    https = 8443
    
    [BROKER]
    http = 12080
    management = 12081
    amqp = 12082
    mqtt = 12083
    
    [ELASTICSEARCH]
    kibana = 13081
    
    [WEBSERVER]
    http = 14080
    
    [LDAP]
    ldap = 15389
    
    [CATALOGUE]
    http = 16080
    
    [KONGA]
    http = 17080
    
    [VIDEOSERVER]
    rtmp = 18935
    hls = 18080
    http = 18088
    
    [PASSWORDS]
    ldap = ? 
    broker = ? 
    cdx.admin = ? 
    database = ?

    This file contains details about the ports used by different microservices. It also allows the user to configure passwords that should be used for certain services during installation. By default, the password fields in the config file is set to ?, which indicates that the system will generate random passwords during runtime.

  2. Install CDX :

    cd ideam/
    ./install

Registering your first device

  • Once CDX has installed you can now start registering devices with it. Let's create a simple test device for the sake of illustration:

    sh tests/create_entity.sh testStreetlight
  • This will give you the details of the registration

    {
      "Registration": "success",
      "entityID": "teststreetlight",
      "apiKey": "EHQilai5cF_tNmWOwg-oiPdncmRPdfGCIhFHM85zDDW",
      "subscriptionEndPoint": "https://smartcity.rbccps.org/api/{version}/followid=teststreetlight",
      "accessEndPoint": "https://smartcity.rbccps.org/api/{version}/db?id=teststreetlight",
      "publicationEndPoint": "https://smartcity.rbccps.org/api/{version}/publish?id=teststreetlight",
      "resourceAPIInfo": "https://rbccps-iisc.github.io"
    }     

Publishing from your device

  • You can now publish data from this device using:

    sh tests/publish.sh teststreetlight EHQilai5cF_tNmWOwg-oiPdncmRPdfGCIhFHM85zDDW

    This will publish {"body": "testdata"} to the exchange teststreetlight.protected

  • That's it! You can similarly register more devices and apps with the middleware.

For detailed guides and resources visit https://cdx.readthedocs.io/en/latest/

ideam's People

Contributors

arun-babu avatar harishanand95 avatar manojkumar010 avatar pct960 avatar swaminathanvasanth avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ideam's Issues

Make quick install the default installation process

Quick install option uses pre-built alpine based docker images that are very small and take significantly lesser time to install. This can be made the default way IDEAM installs. However, the option to install with Ubuntu based images can still be available.

Auto-Start Failure

Observed pushpin and catalogue server auto-start failure after docker restart.

Email for follow request

The follow request should be sent as an email to the owner so that they can approve the share request either after receiving the payment or on a need basis.

Pushpin fails when started in screen or supervisor

Submission type

  • Bug report
  • Request for enhancement (RFE)

Middleware version the issue has been seen with

ideam v1.0.0 latest

Bug report

Pushpin fails when started in screen or supervisor which results in landing page not shown.

RFE: Avoid vagrant approach for LDAP, use dockerised approach

Submission type

  • Bug report
  • Request for enhancement (RFE)

Middleware version the issue has been seen with

smartcity-middleware v0.1.0.alpha

Feature Request

Avoid the usage of Vagrant/VirtualBox for LDAP. This can help mitigate a lot of docker based issues like accessing host ports from inside the container, avoid modification of IPtables in the host system etc. LDAP is currently running on an OpenBSD image in VirtualBox.
Here is a list of the opensource LDAP options available,

https://en.wikipedia.org/wiki/List_of_LDAP_software#Server_software

Tomcat module fails during installation

The tomcat module fails during the middleware installation. However, when the installation is limited to tomcat using

python smartcity-middleware.py install -f middleware.conf -l tomcat

It succeeds. It has now become necessary to do this as the tomcat installation fails every single time the middleware installation is carried out.

Have container specific users in docker

For e.g., Kong container should have kong_admin user, tomcat container should have tomcat_admin and so on.

Each of the users must have a random sudo password.

Remove guest user from Kong

guest user is a provider and can add or remove devices. Create it only for the test script and delete once the tests have completed.

Kong locale problem

The locale settings created by ansible do not persist. Due to this, the pg_createcluster fails with

perl: warning: Please check that your locale settings:  
        LANGUAGE = (unset),  
        LC_ALL = (unset),  
        LANG = "en_IN"  
are supported and installed on your system.  
perl: warning: Falling back to the standard locale ("C").  
Error: The locale requested by the environment is invalid.

Pushpin: build process takes very long time

Submission type

  • Bug report
  • Request for enhancement (RFE)

Middleware version the issue has been seen with

smartcity-middleware v0.1.0.alpha

Bug Report

Pushpin takes a very long time to build. This could be because of make or git commands used in the Dockerfile.

/follow and /share APIs stop working

The /follow and /share APIs do not work from time to time although supervisor is running in the background. A manual intervention is required everytime this happens.

Restrict share API

The /share API should only be allowed to go through if the corresponding entry of the requesting entity is present in the follow queue of the owner's device

RFE: IDEAM installation time can be improved

Request For Enhancement:

A quick installation is possible by avoiding ansible based software installation. Instead use docker images that has all software prepackaged. Things to make sure,

  • SSH certificates for containers are correct and generated during run time
  • User inside should have random passwords generated during each runs

Move IDPS to a container

Setup IDPS in a container so that its installation is independent of the host operating system. Furthermore, use Kong's IP restriction and Request Termination plugins to block a consumer or an IP

Add rate limit plugin to Kong

This can be used for limiting the number of HTTP requests made by a consumer. This can help in monetisation as well as in security

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.