The dsdummies from airavata-courses

Feedback for Project Milestone 1

Hello,

Me and my team were trying to review your project. But we came across few issues while setting-up the project.

As you mentioned in the wiki we tried cloning your repo but we got checkout error(Attached the screenshot below). So we went ahead and downloaded the zip to run the project.
The wiki description is not specific enough, for example you did not explicitly mention to download Anaconda to run the Flask-app in conda environment. But we went ahead and installed Anaconda.
After installing Anaconda we ran the command for setting up the virtual environment. But we encountered an error(Attached screenshot below) because of which we couldn't proceed further.
Next we tried to follow the Docker steps which were mentioned. But "make" command is not working on my machine(Attached screenshot below). Is there a specific software which needs to be installed for it?

We are looking forward to hear from you in order to run your project successfully.

Git Checkout error

Virtual environment error

Make command error

Best,
Team Ampersand

Kubernetes : Create YAML file for Data ingestor Microservice

Create a YAML file for the Data Ingestor microservice to spawn replicas of this service in form of pods.

Common yaml file for CI

Added common yml file using GitHub/workflows.

Cache-CI branch for CI test

This branch will test Cache service in terms of CI.

Bug fix : Add yaml for cache connection to mongodb

The cache service is broken as a result of which the connection of mongodb doesn't happens. This needs to be fixed,

Validation for Requests and Response

validate request coming from client and response coming from Data investor service

Data Ingestor: Add Docker file

Added docker file for this microservice

Project 1 Feedback

Overall, the one-liner start.sh file looks really awesome.

Team Zilean 🦄

Frontend Changes | Enhancements

This commit should involve the UI enhancements. This would include the CSS and PlotPage.js changes.

Dockerfile and Makefile updates

Paths need to be updated in the Dockerfile and Makefile app names to be fixed.

Cache Service | JUnit Test Cases

JUnit test case for checking storing and fetching of data from database.

Dockerize Auth server

Would need to commit Dockerfile, .dockerignore and Makefile for dockerizing this authentication server application.

The docker build is taking more than 15 minutes. Exits with the error -
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.feed_queue: Pending: set()
compose.parallel.parallel_execute_iter: Failed: ServiceName(project='dsdummies', service='react-app', number=1)
compose.parallel.feed_queue: Pending: set()

ERROR: for dsdummies_react-app_1 UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
compose.parallel.parallel_execute_iter: Failed: <Service: react-app>
compose.parallel.feed_queue: Pending: set()

ERROR: for react-app UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
ERROR: compose.cli.errors.log_timeout_error: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regular

Documentation: Added technology stack

https://github.com/airavata-courses/DSDummies/wiki

Front end commit

This issue would be used to commit the front end code in React.js

Documentation: Cache service

Documentation for Cache service with details on setup, usage etc.

[Improvement] Bash files doesn't work directly on machine

Suggestions:

You could add the below line in your documentation.

Replace #!/bin/bash/ to #!/bin/bash in your bash script

chmod 755 start.sh

Thanks for the constructive feedback. We have made some changes to fix this and updated our wiki accordingly. Really appreciate it . Updated wiki : https://github.com/airavata-courses/DSDummies/wiki

Team Zilean 🦄

Feedback for Project 1 [CoDist]

There could have been a hashing structure for repetitive input queries to reduce the computational load on the Data Ingestor.

Add input box for station code

we need to add an input box for taking station code from user

Cache Microservice - make flask app accessable

add the Kubernetes flask URL in cache app, while calling for plots

Docker-compose

This is a single file that shall be used to dockerize and bring up the containers for all the images, namely for React-app(Frontend), Nodejs-app(Auth-service), Cache(Springboot) and Data Ingestor(Flask App).

Project 1 Feedback [CoDist]

The structure of the project seems simple enough to understand. But, from the Data flow Diagram there seems to be no connection between authentication and the spring boot service, what if the request is from an outside source it goes straight to the cache service.

Every connection seems to be Synchronous call, which is good for small scale application. But will not be able to hold a heavy workload.

Talking about Caching instead of caching the whole data file, you could have cached the result of the preprocessor keeping the params of the input from user as keys, and the result as the key. That could have saved a lot of storage, and load up time.

UI fixes for plot form and loading screen

added loading screen and validation for user inputs

Complete dockerization of the application and CI .

This commit shall involve both the changes for dockerization of the complete application and fixes an issue listed in the wiki https://github.com/airavata-courses/DSDummies/wiki#note--step-6-doesnt-works-yet-on-docker-but-the-complete-functionality-works-end-to-end-perfectly-on-local-were-in-the-process-of-fixing-it-but-until-that-time-for-grading--please-refer-to-the-local-setup.
This shall also include the CI changes .

Cache Service | Store AWS data

This service is being used for two purposes-

Fetch existing weather data from the database
Receive data from Data Investor service and store it in database

Data Ingestor: Added environment dependencies

added FFmpeg to environment.yml file

cache: changes for adding Video functionality

validating request from client and validating response from Data ingestor

Frontend Documentation

This wiki lists down the documentation for the Frontend.

Frontend directory structure fix

We want to organize the frontend commit into specific directories so as to properly organize our code.

Frontend Service I Docker

This commit shall be used to commit the changes for Dockerizing the frontend.

Flask-CI branch for CI

This branch is used for testing CI for python flasks service (data ingestor).

Authentication Service Provider

This issue shall be used to commit the code changes for the authentication service provider.

MoM updates

https://github.com/airavata-courses/DSDummies/tree/main/Documentation/MoM

Docker-compose changes for springboot

AUTH-CI branch creation

This branch will be used to test CI for Auth Service.

Issue template

Add template for bugs and features.

Kubernetes - Create yaml file for Auth micro service

create auth microservice and mongo YAML files for deploying to Kubernetes.

Data Ingestor for plotting Data

This is a Flask service that interacts with the S3 server to fetch weather data. Then it plots that data using Pyart and other plotting libraries.

Project 1 Feedback

I tried following the steps on the wiki page. I am unable to find any steps to checkout branches. It is unclear on how to change directory or run the bash command given on the wiki page. We need to pull and checkout the branch in order to run any commands. Could you please provide a little more detailed steps on how to go about that?

Team Orion

Data Ingestor: changing response Status

Changing response status for handling edge cases at cache microservice

Script to run the application.

Using this script only, we can run the entire application.

add auth routes to client app

sorting out login and logout for the frontend app.

Project Milestone 1 Feedback(Team Ampersand)

Hi Team,

Thank you for fixing the errors which I mentioned in the previous issue(#52). I will definitely try and setup the project some time in future.
On a side note, to run 'chmod' and 'make' command one requires a Linux based machine. I have a Windows machine hence was facing the issues.

Best,
Team Ampersand

airavata-courses / dsdummies Goto Github PK

dsdummies's Introduction

DSDummies

dsdummies's People

Contributors

Watchers

Forkers

dsdummies's Issues

Recommend Projects

Recommend Topics

Recommend Org