herover / itu-devops-h Goto Github PK

Used here: https://github.com/Herover/itu-devops-h/blob/dev/minitwit/src/main/resources/templates/timeline.vm#L45 and maybe other places

Sign up > Sign in > Post message > Sign out > Sign up > Sign in > Post message
This path messes up the user's timeline. You cannot see the messages you post with the second account, and you can see the messages of the first account on your own timeline without having followed them. Following and unfollowing other accounts also doesn't work correctly.

Port flag_tool so a (sys-)admin can delete/hide bad messages.
Maybe just modify SQLite tool to use args for deleting? https://github.com/Herover/itu-devops-h/blob/dev/minitwit/src/main/java/SQLite.java#L61

https://github.com/Herover/itu-devops-h/blob/dev/minitwit-legacy/minitwit.py#L140
https://github.com/Herover/itu-devops-h/blob/dev/minitwit-legacy/minitwit.py#L155

We need to use a different database, for example Postgres. In relation to that, we need to get rid of the direct SQL queries in our code - not very secure.

The version we have haven't been updated for 12 years. It might be enough to just replace the dependency, see:
https://velocity.apache.org/tools/devel/upgrading.html
https://mvnrepository.com/artifact/org.apache.velocity.tools/velocity-tools-parent/3.1
https://velocity.apache.org/tools/3.1/apidocs/org/apache/velocity/tools/generic/DateTool.html

Given Guideline Explanation from BetterCodeHub:

• When writing new units, don't let them grow above 15 lines of code.
• When a unit grows beyond 15 lines of code, you need to shorten it by splitting it in smaller units of no longer than 15 lines of code.

Listing given units from highest to lowest severity:

1. WebApplication.main() (55 LoC)
2. control.sh:$default (45 LoC)
3. WebApplication.register() (34 LoC)
4. WebApplication.render() (29 LoC)
5. control.sh:build-in-docker() (28 LoC)
6. update_image.sh:$default (27 LoC)
7. WebApplication.getMessages() (24 LoC)
8. install_docker.sh:$default (21 LoC)
9. SQLite.main() (19 LoC)

Tested code forked on 03.04.2022

Task description on course page.

• Instrument Minitwit
• Add prometheus server
• Add Grafana server
• Add useful graphs to Grafana, keep in mind the "story" that is being told and who consumes which dashboards

Stuff to consider monitoring

• Use official Prometheus Java Client
• Monitor resources used by Java with above "Hotspot JVM metrics" package
• Monitor various actions (register, follow, post...) or endpoint results and response time
• Monitor Drone
• Monitor Linux resource usage

There's a official tutorial but it's from 2015 so is probably not current anymore.

Here https://github.com/Herover/itu-devops-h/blob/dev/minitwit/src/main/resources/templates/layout.vm#L3 a jinja block is used to get title from ex. https://github.com/Herover/itu-devops-h/blob/dev/minitwit/src/main/resources/templates/register.vm#L3. Can/should this be ported? We can just put a title value into the view model and define it in code instead of child templates.
Maybe macros can do it.

Add a get_url equivalent function and fix navbar links, just set models urls to WebApplication.URLS or WebApplication.URLS.class.

Exercise setup: https://github.com/itu-devops/itu-minitwit-ci

However, try to use Terraform instead of Vagrant. And Drone instead of Travis CI, because we're poor.

Something is leaking memory causing linux to kill the process after a few days, see graph

It is currently only possible to see the last 30 messages submitted unless you go into the individual users page to see all their previous messages. But this is not a possibility once their last message isn't loaded anymore.

Not sure if this is needed if project is going to run in Docker anyways, but consider porting other features from https://github.com/Herover/itu-devops-h/blob/dev/minitwit-legacy/control.sh to work with Java project.

Create docker image automatically when pushing to main

Upload it automatically to some docker image hub, so the server can fetch the latest image

There are different ways to do this, but I think it should either be every Sunday, or after every PR merge to dev/main.

Username does not show correctly on "my timeline".

This issue already has a branch flashes with some work. Right now it shows a prompt after registering, but it does not go away again.

The Python version gives the user feedback in the shape of prompts that say "You were logged in", "You were logged out", "Your message was recorded", "You are now following 'user'", "You are no longer following 'user'" and "You were successfully registered and can login now".

All timelines use the same template at https://github.com/Herover/itu-devops-h/blob/dev/minitwit-legacy/templates/timeline.html
https://github.com/Herover/itu-devops-h/blob/dev/minitwit-legacy/minitwit.py#L92
https://github.com/Herover/itu-devops-h/blob/dev/minitwit-legacy/minitwit.py#L112
https://github.com/Herover/itu-devops-h/blob/dev/minitwit-legacy/minitwit.py#L121

For now just returns true, but needs to be updated to return correct http code, and redirect to users page like the legacy version

Used here: https://github.com/Herover/itu-devops-h/blob/dev/minitwit/src/main/resources/templates/timeline.vm#L41

Needs both GET and POST requests.

Needs GET request.

It looks like it tries to do something with query parameters that should have been a session, and use the wrong session key later.

Probably pretty easy to fix?

Stuff to check for.

• Minitwit http://minitwit.waygroup.net/
• Github https://github.com/AlexBMJ/minitwit

Figure out how to compile a Maven project. Consider using this base image https://hub.docker.com/_/maven/ for build stage.

When logging in with an account, posting a message, and then clicking the "public timeline" link in the navbar: trying to post another message after this will produce a "database is locked" error. Has to do with a missing conn.close() statement.

Certain commands break when running this script, such as build and build-in-docker.

Use sparks built-in sessions.
https://github.com/Herover/itu-devops-h/blob/dev/minitwit-legacy/minitwit.py#L184
https://github.com/Herover/itu-devops-h/blob/dev/minitwit-legacy/minitwit.py#L234

The messages of the people that a user follows should display on their timeline (aka. "my timeline").

@smilladion noticed that these steps will make throw errors about a locked database on creating new users

• Create new user
• Sign in with a user (required)
• Sign out
• Create a second user, following message shows in browser: org.sqlite.SQLiteException: [SQLITE_BUSY] The database file is locked (database is locked)
  In the console this will print

org.sqlite.SQLiteException: [SQLITE_BUSY]  The database file is locked (database is locked)
	at org.sqlite.core.DB.newSQLException(DB.java:1030)
	at org.sqlite.core.DB.newSQLException(DB.java:1042)
	at org.sqlite.core.DB.execute(DB.java:881)
	at org.sqlite.jdbc3.JDBC3PreparedStatement.execute(JDBC3PreparedStatement.java:54)
	at WebApplication.lambda$static$9(WebApplication.java:466)
	at spark.RouteImpl$1.handle(RouteImpl.java:72)
	at spark.http.matching.Routes.execute(Routes.java:61)
	at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:134)
	at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1584)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:501)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:833)

https://github.com/Herover/itu-devops-h/blob/dev/minitwit-legacy/minitwit.py#L170

Needs to be done by Sunday so that the simulator can run.

We also need to make a PR to the course repo with the IP of our running program.

Consider using https://github.com/drone-plugins/drone-snyk

Needs POST request.

Needs both GET and POST requests.

Our main class is a monster - would be nice to divide it out into some more classes, for our own sake.

For example, we could have one class for routes, one for API routes, one for util methods and so on.

We can also do an MVC structure with classes like User, Message etc.