learn-video / mosaic-video Goto Github PK

Is your feature request related to a problem? Please describe.
No changes needed.

Describe the solution you'd like

Currently, the program pushes all content through HTTP to an uploader, which then saves it in a storage backend.

Another option is to enable saving the files locally. Users can then expose the path (where the files are written) via a web server.

This diagram shows how users could take advantage of the local storage backend:

Describe alternatives you've considered
No changes needed.

Additional context
Pros:

• A major strength of this approach is that external storage is not necessary, reducing costs and simplifying the process.
  Cons:
• Implementing service discovery and a web server is required for a CDN to know where to fetch the content from.

Currently, every mosaic configuration is retrieved from the following struct:

Mosaic struct {
	Name      string  `json:"name"`
	Medias    []Media `json:"medias"`
	WithAudio bool    `json:"with_audio"`
}

WithAudio field should be respected while firing the ffmpeg command.

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
We already have a command to run the linter locally. It would be great if the linter could run on CI.
Today we have a considerable amount of warnings, I'm not sure if we should fix them or apply some kind of checkpoint (to run the linter after a specific commit.

Is your feature request related to a problem? Please describe.
Currently, the program is targeted to run on a single instance. We need high availability, and a way to scale it horizontally.

Describe the solution you'd like
Every new worker should process number of mosaics / number of nodes mosaics. It grants an even distribution of the workload.

We also need to handle topics like:

• Uniqueness - a mosaic should not run in two differentes nodes at the same time.
• Healthcheck - a way to check if the process is still running. In negative case, we should release the lock so other servers can start process the mosaic.

Suggested approach 1:

Whenever a worker initiates, it should insert or update a key in Redis under the workers namespace. We could use the hostname or a predefined ID for each worker. This is crucial for workload distribution. For instance, if two workers are registered and there are 10 mosaics to be processed, each worker would handle number of mosaics / number of nodes, as initially outlined

Maybe we can expose some metrics of which mosaics are being processed by each worker.

When a worker dies (or is intentionally terminated), the node must be unregistered, either through TTL or graceful shutdown. Subsequently, other workers should take over the tasks that were being processed by the now unavailable worker.

When a new worker joins the cluster, it is necessary to redistribute the workload. For instance, if there are 2 workers processing a total of 12 tasks, each worker handles 6 tasks. If a new worker joins, we need to terminate 2 tasks (possibly selected at random) from each of the currently running workers. The new worker will then take on 4 of these tasks, ensuring that each worker is now processing 4 tasks.

Suggested approach 2:

We can use a framework like asynq and use a Redis as a queue.

Some things like healthchecks and workload distribution will still happen but in a lower degree.

Describe alternatives you've considered
No changes needed.

Additional context
We already use a strategy to lock the mosaic, but it needs to have the node ID in it.

Is your feature request related to a problem? Please describe.
No changes needed.

Describe the solution you'd like
A mosaic might be generated from videos on demand instead of using HLS playlists.

Imagine you are streaming a mosaic with 3 videos: 2 live streams and 1 promo video (with a QR code, for example) linking your website. This promo video could be a VoD.

Describe alternatives you've considered
No changes needed.

Additional context
No changes needed.

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
Currently, we are parsing env configs in a struct managed entirely by ourselves, calling os.Getenv without any kind of validation.

Some libs like env and envconfig are designed to move away this burden from developers.

Describe alternatives you've considered
I did not consider any alternatives other than the ones above but feel free to bring your ideas.

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
Instead os serving the files through a static web server, we can save every content into a S3 like storage.

When ffmpeg start processing video inputs, it also starts writing every manifest and segment in the local filesystem.

We can send the content through HTTP using another server, ready to accept PUT/POST requests.

Describe alternatives you've considered
Serving through static web server.

Additional context

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
It would be cool if we could assemble the mosaic using pre defined templates.

Maybe some presets? And the process that dispatches the FFMPEG process coud interpret this preset and transform the presets into positions for the FFMPEG command.

This is the payload we have today:

{
    "name": "mosaic",
    "medias": [
        {
            "url": "https://rtvelivestream.akamaized.net/rtvesec/24h/24h_main_720.m3u8",
            "position": "0_0"
        },
        {
            "url": "https://canadaremar2.todostreaming.es/live/peque-pequetv.m3u8",
            "position": "w0_0"
        }
    ],
    "with_audio": true
}

Describe alternatives you've considered
Free descriptions of the positions could work but might be hard to translate and validate them.

Additional context
N/A