Coder Social home page Coder Social logo

scorelab / openioe Goto Github PK

View Code? Open in Web Editor NEW
24.0 14.0 42.0 4.61 MB

Open-source middleware platform for building, managing, and integrating connected products with the Internet of Everything.

License: Apache License 2.0

JavaScript 25.33% Java 38.09% HTML 15.26% CSS 15.49% Scala 1.57% Shell 0.62% Dockerfile 0.14% C 3.50%

openioe's Introduction

Gitter Build Status GitHub license Codacy Badge

OpenIoE

OpenIoE is an Open-source middleware platform for building, managing, and integrating connected products with the Internet of Everything. It enables you to subscribe to data streams and get data from the sensors and store them. The application was generated using JHipster application generator.

Main features

  • A high-perfomance Java server side with Spring Boot
  • Message broker support with ActiveMQ Artemis
  • Multi Protocol support (MQTT, AMQP)
  • NoSQL data storage for sensor data (Cassandra)
  • Support for timeline series modelling
  • Publish-subscribe feature using MQTT

Docker

  • All the services required to run the platform can be easily started via bundled docker-compose.yml located in rest_api/src/main/docker
  • Browse to rest_api/src/main/docker and run docker-compose up -d
  • Open 127.0.0.1 in the web browser where you can configure a user account and create devices and sensors
  • To stop the platform, run docker-compose stop in the same folder as above
  • Logs can be observed via docker-compose logs [CONTAINER] (ex: ioe-app)

Installation

Usage

  • Make sure Cassandra server is up. You can start Cassandra with 'sudo service cassandra start' and stop it with 'sudo service cassandra stop'. Verify that Cassandra is running by invoking nodetool status from the command line.
  • To run the artemis message broker (Compiled using ActiveMQ Artemis 1.5.x),
    • In '/message_broker/etc/artemis.profile' file, change the ArtemisHome directory to match your Apache ActiveMQ Artemis installation directory and java.security.auth.login.config of JAVA_ARGS to match OpenIOE 'message_broker/etc/login.config'.
    • In '/message_broker/etc/bootstrap.xml' file, change the server configuration path to match OpenIOE 'message_broker/etc/broker.xml' file.
    • The executable for running the artemis message broker is found in '/message_broker/bin'. Run ./artemis run in order to start the broker.
  • Spring Boot is run by 'gradlew' executable in /rest_api. Run sudo ./gradlew to start running the application.
  • Open 127.0.0.1 in the web browser where you can configure a user account and create devices, sensors, subscriptions and publications.

Publish - subscribe behavior

The sensor data can be published using rest end point and message broker with MQTT support. Both methods accept the payload as JSON. An example for publishing data with rest endpoint using curl command is

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'Authorization: Bearer <access_token>' -d '{
  "data": "24",
  "description": "test",
  "sensorId": 1,
  "timestamp": "2017-03-19T09:47:44.526Z",
  "topic" : "gps"
}' 'http://127.0.0.1:8080/api/sensors/{id}'

The user can also publish data using MQTT protocol with MQTT client applications by sending the data of the following form to the topic 'gps'.

  "data": "24",
  "description": "test",
  "sensorId": 1,
  "timestamp": "2017-03-19T09:47:44.526Z"

The user can subscribe to topics defined for subscription entity using MQTT client applications by providing the topic name as gps. Whenever a new data is published, it will be stored in Cassandra and can be received by the user.

Storing data

Data can be stored via the REST endpoint and the message broker. Both methods accept the payload as JSON. An example is {"sensorId":1,"data":"24","description":"test","timestamp":"2016-07-30T21:29:34+02:00"}

Retreiving data

Using the REST end point data can be retreived. http://127.0.0.1:8080/api/sensors/1/data where 1 is the sensor id http://127.0.0.1:8080/api/sensors/1/data?dates=2016-07-30,2016-07-2 where multiple dates can be specified

  • The requests should pass the Authentication token

Future work

  • Need to implement authentication for Publish Subscribe

openioe's People

Contributors

agentmilindu avatar charithccmc avatar dande9304 avatar emgaurv avatar keshwithan avatar padamchopra avatar prabushitha avatar priyapappachan avatar shlokj avatar tharidu avatar thisa32 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

openioe's Issues

Sensor list getting updated with new sensor incorrectly

Sensor list is getting updated in the application with new sensor entry even when there is internal server error obtained while trying to create sensor. The Cassandra table is not created for the new sensor because of the error, but the sensor is shown in the sensors list in the front end. Without Cassandra table creation, sensor data cannot be handled.

Repo should follow Gitflow Workflow

This repos should follow Gitflow workflow and follow are the items you have to check.

  • There is a master branch and a develop branch.
  • master branch is locked for direct commits and,
    • Require pull request reviews before merging is active
    • Require status checks to pass before merging is active
      • Require branches to be up to date before merging
      • Appropriate Status checks are required
    • Enforce all configured restrictions for administrators is active
    • Restrict only maintainers group can push to the master branch
  • develop branch is locked for direct commits and,
    • Require pull request reviews before merging
      • Dismiss stale pull request approvals when new commits are pushed is active
    • Require pull request reviews before merging is active
    • Require status checks to pass before merging is active
      • Require branches to be up to date before merging
      • Appropriate Status checks are required
    • Enforce all configured restrictions for administrators is active
    • Restrict only maintainers group can push to the master branch

Update of entity fields required

Foreign key fields such as sensor, device and user are not defined as mandatory for the subscription entity. The implementation of subscription is not clear as it is not known whether the user can use the subscriptions for retrieving data now.

Also it is not sure why serial and table id field is used for sensor. Only name field is mandatory for the device and sensor creation. While creating device and sensor, user can select user id. Does it mean user can create device and sensor for other users?

Installation and running automation, documentation reorganisation (wiki and README)

  • The most important issue is that of installing and running the required programs again and again for running the project-
    • A setup.sh file should be made and Cassandra should be installed automatically without any additional user task.
      Cassandra install
    • The "Usage" section on the README.md file has a list of commands to be run everytime the program is to be run. Commands like sudo service cassandra start, ./artemis, sudo ./gradlew and opening the web browser at 127.0.0.1 should be done automatically by a shell script file.
      Usage section
  • The Java backend should be moved to React-native or some other modern technology because the app is docker-based.
  • The wiki for the project is empty and should be made and updated. Content from the individual sections of the README.md file should be taken and each section should have a new dedicated wiki page.

[Bug] Errors encountered in running process

After successfully completing the installation part for the program, when I executed docker-compose up in ~/OpenIoE/rest_api/src/main/docker I found these two errors in the output:

Error #1

5

This error appeared in the middle of the process, although it continued after. It might not be major but it would nice to clear all the minor errors to have a clean process.

Error #2

6

This error terminated the process. I can guess it's something about nodejs. I'd like to know how to fix this error so that the process can run without any flaws.

Java_Home Variable

To run the program java_home variable is required. Though in the README.md file nothing is described about this. The README.md has no briefing on how to set it up. So the README.md file should be updated at the earliest with more additional details.

Google Code-In 2017: Getting Started Issue

THIS IS A NOT A REAL ISSUE BUT A PLACEHOLDER.

All the practising PRs for Google Code-In 2017 should refer this issue. PRs referring this issue will be closed without merging. Do not refer this issue if you are NOT submitting a practice PR and need your work merged.

Add License

The license is very important for the open-source project.

Add license

Add a license file ,its an important requirement.

No Badges/Shields Available

Expected Behavior

There should be following Shields available to the projects;which gives first-timers and old-timers an at-a-glance peek into the status of the project.

badges

  • Travis
  • Codacy
  • Docker build
  • Gitter
  • Other Microbadgers

Current Behavior

There is no Badges/Shields Available

Possible Solution

All the commands/Documentation that are required To integrate Shields are present Here

Sensor Data

The README.md file days that the sensor data can be published using rest endpoint and message broker with MQTT support. But the README.md file should make more briefing on how to do so because many may not know about MQTT support

Add confirm password input field

Is your feature request related to a problem? Please describe.
Currently there is no confirm password input field which can assure the user he has input his desired password.

Describe the solution you'd like
A confirm password input field ,when the password and confirm password doesn't match a message will be shown below that indicating passwords doesn't match.

Describe alternatives you've considered

Additional context

Implement authentication for message broker and rest API

Currently, OpenIOE does not implement any security mechanisms for publishing or subscribing to any registered sensors in the backend.In order to support more clients, MQTT authentication and GUID based URL routing can be used.

Npm issue

When I cloned the repository, I decided install the npm. I did it, but when I went to test the npm, it didn't work-there were many errors.
capture
The npm tests failed.

[ GCI - 2018 ] Issues jn Files and Code review

  • Being that this tool is meant to be used as a tool for developers, there should also be more documentation on available functions and methods that they have available to use, and how to use them.

  • Use a Continuous Integration for code review ( I suggest Codeship )

  • Add badges to he README

  • Currently it only has a few examples, but doesn't quite show what is available to use. Further explaining how openIoE works will allow for developers to better integrate it into existing projects - as well as allow for people to understand the inner workings for education purposes or open source contribution.

  • Some files are missing like the ISSUE SUBMITTING TEMPLATE and CODE OF CONDUCT.

  • Add a LICENSE to this application in order to make people follow some rules.

Npm issue

When I cloned the repository, I went in the rest_api folder and did an npm install and it worked. After that I did an npm test, but it failed. I even had Docker installed on my computer.
capture2
capture3
Most of the problems were in the loader.js files, but there was also index.js problems. I think that they were not able to sense these files for some reason, or that they were missing. I looked everywhere but I could not find the answer. Also, I am pretty sure it was missing modules, but I was not able to identify which ones after a thorough search. It has to be something about a important java script file which is missing.

Also, there are some important files like the code of conduct, License and proper documentation missing.
Please fix these problems so that your project is more successful and more user friendly.

No License

License File is very important in an open source project. So it must be added.

[GCI2018] Update Dependencies on /rest-api/package.json

Problem :

All the dependencies in /rest-api/package.json has not been updated since Jun 11, 2016, as a result, most of the dependencies are deprecated now, which makes the program vulnerable to security and performance issues.

As an example, the bower library is now deprecated and developers suggest to use Webpack, Parcel and such similar libraries.

Possible Fix:

  • Update the dependencies
  • Update the source code wherever needed to eliminate the problems created with updating libraries

Password validity is weak

Is your feature request related to a problem? Please describe.
Currently password validation is very weak .

Describe the solution you'd like
It would be nice if the password is more strong,like it contains a number,chars and special chars.

Describe alternatives you've considered

Additional context
We can use alert if the user doesn't enter a strong password

README update

Please update the readme so that it is easier to follow. I read about the main features and it seemed interesting, but when I tried installing Cassandra, I encountered some issues. I'm not experienced and I would really appreciate it if you could make the installation steps a bit more detailed. I think that many other people are in the same situation, and these small changes would help the project a lot.

[Suggestion] Proper documentation for developers

Being that this tool is meant to be used as a tool for developers, there should also be more documentation on available functions and methods that they have available to use, and how to use them.

Currently it only has a few examples, but doesn't quite show what is available to use. Further explaining how openIoE works will allow for developers to better integrate it into existing projects - as well as allow for people to understand the inner workings for education purposes or open source contribution

TTL value for EXPIRE storeType sensor is not collected

A TTL value of 0 is used to insert data into cassandra when storeType is EXPIRE. TTL value is not provided by the user for sensorData belonging to a sensor of storeType EXPIRE. Insert operation in cassandra using ttl supports setting a time for data in a column to expire.

Also a timestamp field is used for the sensorData. Is the timestamp used to store when sensorData is collected or inserted?

Fix README.md file

  1. The README.md should contain more text.
  2. No Wiki for project so new user can't understand how to use the app.

Dynamically show and hide password at SignUp screen

Is your feature request related to a problem? Please describe.
Sometimes user needs to see which password he has typed,currently there is no feature to see password on typing.

Describe the solution you'd like
An eye icon which can show password on clicking.

Describe alternatives you've considered

Additional context
Something just like this
1_HOz8wr_pxG5OIrZKL_rFrQ

Implement user interface for OpenIoE

Currently, we are using the generated jHipster UI for OpenIoE. We need to implement a customized user interface for providing a better user experience. Angular or React can be used.

OpenIoE icon is not fitted properly

Describe the bug
OpenIoE is icon is not fitted properly

To Reproduce
Steps to reproduce the behavior:

  1. Open the front page of OpenIoE(App.js)

Expected behavior
The OpenIoE icon should have been fitted properly,but it isn't due to bad CSS styling.

Screenshots
Capture

Desktop (please complete the following information):

  • OS: Windows 10
  • Browser [Chrome]
  • Version [e.g. 22].

Smartphone (please complete the following information):

Additional context
Minor improvement in UI.

Submit Button should be disabled on SignUp Screen until all fields are filled correctly.

Is your feature request related to a problem? Please describe.
Submit button is enabled even though the fields are empty.

Describe the solution you'd like
I will use the state to check whether all fields are filled and then button will be enabled only if all fields are filled.

Describe alternatives you've considered
An Alert Dialog can also be displayed.

Additional context
Something like this
image

[Suggestion] Help for young developers

Please make instructions on readme bit understandable. Because there are so many young developers are going to contribute project like these soon.

Add how to use these in windows platforms because everyone cant deal with linux and add some more info for the installation part and heres the most important thing ** please add the common errors can occur during installation and running and how to get over that error according to the experience of the main developers

[Error] Running Docker compose up

I follow READ.ME and run docker
Browse to rest_api/src/main/docker and run docker-compose up -d
screenshot from 2018-11-21 15-04-49
Step 1/4 : FROM ubuntu:xenial
xenial: Pulling from library/ubuntu
7b8b6451c85f: Pulling fs layer
ab4d1096d9ba: Pulling fs layer
e6797d1788ac: Pulling fs layer
e25c5c290bde: Waiting
ERROR: Service 'ioe-app' failed to build: error pulling image configuration: Get https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/a5/a51debf7e1eb2018400cef1e4b01f2e9f591f6c739de7b5d6c142f954f3715a7/data?verify=1542790201-Y0ATOVI3AsWwTcxjJupUBN5gReM%3D: dial tcp: lookup production.cloudflare.docker.com on 127.0.0.53:53: read udp 127.0.0.1:33806->127.0.0.53:53: i/o timeout

Important Files Missing

Here should be
code_of_conduct.md
issue_template.md
pull_request_template.md

please add them in the repository to guide the developers.So please add it.

[Opinion] Migrate from java backend

The current project is using java as the back-end. Java is slowly becoming less and less used and out of date. Migrating from java to python (using flask) or NodeJS (using Express), would allow for more people to contribute as these two are more well known, with better documentation and tutorials. Furthermore, these popular backed languages/libraries, are constantly evolving with new features, and tools for debugging.

Internal Server error response for POST /sensors/:id and GET /sensors/:id/data methods in sensorResource

I used the following command for retrieving the data using REST endpoint
curl -X GET --header 'Accept: application/json' --header 'Authorization: Bearer <access_token>' 'http://127.0.0.1:8080/api/sensors/1/data'

I got the following response {
"message": "error.internalServerError",
"description": "Internal server error",
"fieldErrors": null
}

The same error occurs for POST http://127.0.0.1:8080/api//sensors/:id. The sensor with id 1 is present.

Dockers should have proper Label Schema labels

This repos has a Dockerfile (OpenIoE/blob/master/rest_api/src/main/docker/Dockerfile) but it does not contain any Label Schema labels. Following are the lables we have to have on our Dockers.

  • name
  • description
  • url
  • vcs-url
  • vcs-ref
  • vendor
  • version
  • schema-version

Update to the wiki

I have created a wiki page for OpenIoE to help users to set up and use the OpenIoE application easily following the instructions in it. I request you to review the page.

Missing use of Strict mode

FIX: Delete node_modules (if there), and update all the packages. npm install to have all the latest dependencies. Remove redundant & deprecated libraries as well for optimal performance.

The current codebase has not been updated with the latest modules and libraries. Using strict mode is mandatory with node. But due to it's unavailability in certain places, the terminal unexpectedly exits, which stops the docker compose. See below.

crash
Find this task on my website: http://itsrohan.com/gci/gcitasks

[GCI-2018] Some issues

  • Being that this tool is meant to be used as a tool for developers, there should also be more documentation on available functions and methods that they have available to use, and how to use them.

  • Currently it only has a few examples, but doesn't quite show what is available to use. Further explaining how openIoE works will allow for developers to better integrate it into existing projects - as well as allow for people to understand the inner workings for education purposes or open source contribution.

  • Some templates are missing like the ISSUE SUBMITTING TEMPLATE and CODE OF CONDUCT.

  • Add a LICENSE to this application in order to make people follow some rules.

[Suggestion] Missing files for open source development

The repo itself is missing push and issue templates, which are needed for efficient open-source development. This will set the guidelines on what makes a good issue and what is a good pull request, and what is expected of newcomers

Improvement and Issues

  1. Lack of license.
  2. The repository should follow git flow work flow.
  3. Wiki should be updated and should be more instructive.
  4. It doesn't have UI/UX and it should.
  5. Should add Graphical user interface to OpenloE

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.