stalej / weathersim

Welcome to the WeatherSim!
This program will simulate a weather database and server in which for you to retrieve data from. Our main goal is to demonstrate basic socket programming with UDP and TCP.

---

Where to find the files for the assignment

For the MVP:

weather_station_data_sender.py  # Using UDP socket 
server.py                       # Using UDP socket 
client_server.py                # Using TCP socket
weather_app_client.py           # Using TCP socket

The extra features:

A beatiful website
Flask REST api
Docker

Installation

Configuration

With Python 3.8+

To install requirements/dependencies:

$ pip install -r requirements.txt  # or
$ pip3 install -r requirements.txt

Production

Automatically with Docker

For a very quick and easy deploy of your app we recommend hosting the app as a Docker multi-container and running that. It is fairly straight forward:

$ sudo docker-compose up -d && sudo docker attach user_client

Press Return after you have run the command above, and you should be greeted with the message:

### WeatherAPP 2.0 ###
WebApp running on http://0.0.0.0:5000/
Press 'help' for Commands | press Enter to exit 

If changes to the project are made you can redeploy with:

$ sudo docker-compose up -d --build --force-recreate && sudo docker attach user_client

This will force Docker to use the newly edited files for the images that is run in the containers

Manually

For manually deploying you need to run these python scripts simultaneous in their specific folders:

$ python client_server.py                # /server
$ python server.pu                       # /server
$ python weather_station_data_sender.py  # /station
$ python app.py                          # /server/webapp
$ python weather_app_client.py           # /client

NOTE: We highly recommend you to use Docker instead of the manual approach.

Usage

We have three ways of interacting with the app, a shell WeatherAppClient, Website, and REST api.

How to use the shell/CLI

TheWeatherAppClient is a command line program.
You can use it to view the weather data that has been generated

To see possible commands, type:

$ WAclient> help

To receive all data since the simulation start, type:

$ WAclient> get data all

You can also use the flask app we made to view the data in a browser

About

Data will be periodically generated from station.py. weather_station_data_sender.py then transmits the data by UDP to server.py which stores it to data.json. This data can then get read from client_server.py with TCP by the command line program weather_app_client.py. The flask webapp will be able to read the data from the data.json and renders an index.html webpage with data from the last 24h in a graph and last readings in big bold letters. The webapp also has an REST API that allows for specific get request over HTTP.

Illustrated model

REST API Documentation

We use REST API Documentation Tool | Swagger UI for documenting of the REST API. Swagger UI allows visualizing and interact with the API's resources.

Known bugs

data.json will eventually get very big, limiting the whole god-damn operation.

docker_entrypoint.sh may be changed by Git automatically to have a CLRF ending on Windows machine if this option is enabled in your local Git. When docker_entrypoint.sh has End of line sequence set as CLRF the docker compose up command will fail to launch the server, and you will get a error that may look like standard_init_linux.go:219: exec user process caused: no such file or directory. The fix is either no not use Windows EOL symbols or change just docker_entrypoint.sh EOL to have Unix-style EOL LF.

---

Participants:

Students:
Ståle Jacobsen the tech lead
Erlend Haugen the bug killer
Kim Andre Grønstøl the web maestro
Martin Eide the composer
Henrik Eide the architect

TI's:
🇷🇸Lenanomous
Sedrick Varnes

CopyRight

cartoon png from pngtree.com

weathersim's Issues

get data -all command in client

• Arbeidsoppgave:
  • Make the "get data -all" return all the data form the database

Refactor WeatherAppClient.py

• Arbeidsoppgave:
  • make code look bootyful 🏭

Catch OSError by error code

Må finne en måte å catche OSError

• Arbeidsoppgave:
  • finne en måte å catche OSError
  • klare å catche OSError error code

Create Server and Database

Fix broken get all command

This method is flawed. We need to make a second request to find the size file we are sending.

def request_to_server(request: str) -> str:
    sock.sendall(request.encode())
    resp = sock.recv(16384)
    return resp.decode()

We need

sock.reciv(file_size)

So that we don't overflow the recv buffer

Implement more get request commands on client side

Vi må lage flere muligheter til flere kommandoer som vi kan sende i "terminalen".

Forslag kan være

get data <day>
get data <place>
get data <month>
get data rain
get data temp

Either the Readme is not correct or the code is not correct

Describe the bug:
When in the project root directory, if you run the code from as described in the readme the code will fail eventually. It is caused by that the files/servers are coded to be run from the folder they are in.

So we could either change the readme or the code:

A change in the readme would look like:

Then run from project root:

$ cd server
$ python server.py
$ python weather_station_data_sender.py
$ python client_server.py
$ cd ../client
$ python WeatherAppClient.py

or a code change would look going from, for example this:

DATABASE = "data.json"
HELP = "requests.json"

to this:

DATABASE = "./server/data.json"
HELP = "./server/requests.json"

in every relevant files. What you guys think?

Create help command in client

• Arbeidsoppgave:
  • Make help command return all available commands.

README fix

A small error in the readme. data sender is not on the server but station

$ python server/server.py
$ python server/weather_station_data_sender.py
$ python server/client_server.py
$ python client/WeatherAppClient.py

get data all fails in WeatherAppClient.py

Error when trying to fetch all data form data.json

WAclient> get data all
Traceback (most recent call last):
  File "WeatherAppClient.py", line 107, in <module>
    get_all_data()
  File "WeatherAppClient.py", line 29, in get_all_data
    all_data = get_json("getall")  # loads converts str -> dict
  File "WeatherAppClient.py", line 16, in get_json
    _data = json.loads(request_to_server(request))
  File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.8/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 1 (char 0)

Documentation is not very correct.

In the diagram model ther eis inacurate URL for the website. It says 30group.code.
Se picture
The correct URL is group30.codes

Update README

We need to update the README.md

Problems on mac?

Can someone one Linux or windows check if weatherappclient can still talk to the server? or it's a mac problem?

λ python3 WeatherAppClient.py                                                                                                                        ⏎ [3m] ✭
Press Enter to exit
WAclient> get data all
Traceback (most recent call last):
  File "/Users/stale/Skole/oblig1-142/client/WeatherAppClient.py", line 102, in <module>
    get_all_data()
  File "/Users/stale/Skole/oblig1-142/client/WeatherAppClient.py", line 36, in get_all_data
    all_data = get_json("getall")  # loads converts str -> dict
  File "/Users/stale/Skole/oblig1-142/client/WeatherAppClient.py", line 23, in get_json
    _data = json.loads(request_to_server(request))
  File "/Users/stale/Skole/oblig1-142/client/WeatherAppClient.py", line 14, in request_to_server
    size_of_file = int(sock.recv(512).decode())
ValueError: invalid literal for int() with base 10: ''

client_server.py talking to database

• Retrieve all json data
• Parse data on server side for request
• Implement -> getall, close, help
• Implement -> getplace
• Implement -> getrain
• Implement -> gettemp

client_server.py talking to client

• TCP connection to client
• Handle simple requests: getall, help etc