jesperancinha / vma-archiver

Video Media Archiver

vma-archiver

---

---

Technologies used

Please check the TechStack.md file for details.

Introduction

We are going to make a VMA archiver. VMA in this case does not stand for Video Media Archive anymore. That was version 0.0.0. It now stands for Video Music Awards. So we are going to make a service to register nominees and winners of a VMA show. In this case, we'll use a fake MTV Awards Show. Instead of using contemporary music, we'll make an MTV awards with music and artists from the 1920's.

Our application will be built around the Reactive CoRoutine Paradigms. We'll be using five important elements: Hazelcast, SpringWebFlux, Flow, Kotlin Co-Routines and a nice architecture.

Stable Releases

---

This repo is also the official support repo to my article on medium:

- Stable releases

• 0.0.0 - 36e8f4b35b0a1226d440d6698194e989b8601c1b - First Video Media Archive version / Gradle / Spring 5 / JDK 11
• 1.0.0 - 992cdb7f6262b8e366142814f6b6547ae922cf52 - Video Music Awards App / JDK 17 / Coroutines / Kotlin 1.7.20 / Spring Boot 2.7.4 / Docker with fixed IP's
• 2.0.0 - 17ae13b0c8c1bdc282225ef1515215f7f5906997 - Video Music Awards App / JDK 17 / Coroutines / Kotlin 1.7.20 / Spring Boot 3.0.1 / Docker with dynamic IP's

---

Walk Through

---

---

Knowledge Cloud

---

Knowledge Cloud

BuildersKt.class, Builders.common, withContext, ifEmpty, tailrec, suspend, runBlocking, shuffled , sortedBy, async , await, launch, delay, runBlocking, coroutineScope, Channel, consumeEach, produce, produceSquares , produceNumbers, yield, @Volatile, withContext, Dispatchers.Default, Mutex, CoroutineScope , counterActor, override val massiveRun,

---

Knowledge for Integration Tests Cloud

@Mockk, @Test, @SpringBootTest

---

Project Layout

• VMA Demo Generation - Generates The Demo Awards - It is customizable in terms of categories, music and artists.
• VMA Common - A Common Library to support Spring Boot Startup Services.
• VMA Play - A way to play with co-routines separated from the project. It has examples on the main and test directories.
• VMA Spring CoRoutine Reactive Service - The reactive service facing the front-end - It posts votes in Kafka, registers a new award show directly and reads voting results from the database.
• VMA Service Event Listener - Listens to incoming votes sent in a massive load fashion.
• Locust - The location of locust service. It creates workers in kotlin that simulate random massive voters.

---

Sequence diagram

To visualize this diagram you may need the mermaid-diagrams plugin installation.

To visualize it in Intellij, please install the mermaid plugin.

    sequenceDiagram
        participant VMA Voting Client 1
        participant VMA Voting Client 2
        participant VMA Voting Client n
        participant VMA BE NGINX LB
        participant VMA Services Backend
        participant VMA Kafka Streams
        participant VMA Services Event Listeners
        participant VMA BE HA LB
        participant VMA PostgreSQL
        
        rect rgba(0, 0, 255, .1)
        par
        VMA Voting Client 1-->> VMA BE NGINX LB: Cast vote
        and
        VMA Voting Client 2-->> VMA BE NGINX LB: Cast vote
        and
        VMA Voting Client n-->> VMA BE NGINX LB: Cast vote
        end
        end
        VMA BE NGINX LB-->>VMA Services Backend: Distribute Votes
        VMA Services Backend-->>VMA Kafka Streams: Register Votes
        par
        VMA Kafka Streams-->>VMA Services Backend: Vote registered!
        and
        VMA Kafka Streams-->>VMA Services Event Listeners: Send Register Vote event
        end
        VMA Services Backend-->>VMA BE NGINX LB: Vote registered!
        VMA BE NGINX LB-->>VMA Voting Client 1: Vote registered!
        VMA BE NGINX LB-->>VMA Voting Client 2: Vote registered!
        VMA BE NGINX LB-->>VMA Voting Client n: Vote registered!
        VMA Services Event Listeners-->>VMA PostgreSQL: Create Vote database record
        rect rgba(0, 0, 255, .1)
        par
        VMA Voting Client 1-->> VMA BE NGINX LB: Read votes
        and
        VMA Voting Client 2-->> VMA BE NGINX LB: Read votes
        and
        VMA Voting Client n-->> VMA BE NGINX LB: Read votes
        end
        end
        VMA BE NGINX LB-->>VMA Services Backend: Distribute read votes request
        VMA Services Backend-->>VMA BE HA LB: Request Database Votes
        VMA BE HA LB-->>VMA PostgreSQL: Read Votes
        VMA PostgreSQL-->>VMA BE HA LB: Response with Votes
        VMA BE HA LB-->>VMA Services Backend: Response With Votes
        VMA Services Backend-->>VMA BE NGINX LB: Response With Votes
        VMA BE NGINX LB-->>VMA Voting Client 1: Response With Votes
        VMA BE NGINX LB-->>VMA Voting Client 2: Response With Votes
        VMA BE NGINX LB-->>VMA Voting Client n: Response With Votes

---

How to run

In general:

make docker-clean-build-start

Start all containers

---

make dcup-full

If it fails, you can always try make dcup.

---

Start Locust

---

make locust

---

Cast your vote

---

• Go to http://localhost:8080 and cast your votes

---

Wait for locust to stop and check the result

---

• Go to http://localhost:8080/result

---

Swagger tests

---

You can make tests for this application using the Swagger UI at

• WebFlux Reactive Backend

---

Endpoints

---

• http://localhost:8080/api/vma/welcome

---

Install essential libraries and commands

---

make install

---

Serving Spring Boot (LOCAL)

---

In order to let Kafka know where to get to locally you need to define locally that jofisaes-vma-broker is also in 127.0.0.1.

This is done in MAC-OS and Linux machines on /etc/hosts:

127.0.0.1   jofisaes-vma-broker

---

Java setup

---

sdk install java 17-open
sdk use java 17-open

---

---

References

• How to fix the LEADER_NOT_AVAILABLE error in Kafka?
• Using Kotlin Coroutines with Spring
• Going Reactive with Spring, Coroutines and Kotlin Flow
• How to Set Up PostgreSQL Cluster using Patroni on Ubuntu 20.04
• Creating a single HAProxy and two Apache containers with Docker compose
• HAProxy - The Reliable, High Performance TCP/HTTP Load Balancer
• Ingress Gateways
• Amazon Aurora connection management
• PostgreSQL Load Balancing with HAProxy
• PostgreSQL HAProxy: Proxy for HA and Load Balance
• How Does a Database Load Balancer Work?
• HTTP Load Balancing
• Using nginx as HTTP load balancer
• Markdown Badges
• Kotlin coroutines on Android
• Full Kotlin Coroutines Design Reference
• Kotlin Coroutines Design Document
• Guide to UI programming with coroutines
• Coroutine Channels
• Coroutine
• Best practices for coroutines in Android
• Imagining your Repository Layer with Coroutines
• 17.5. Coroutines
• 17.5.2. How Reactive translates to Coroutines?

About me

vma-archiver's Issues

Profile starter tests and cypress tests