hbomb79 / thea

Thea is (going to be) a no compromises home media solution written with Go + Svelte which facilitates the discovery, transcoding, organization and streaming of your content.

Hello world 👋

My name is Harry and I'm a BInfSc graduate currently working on Geneious Prime/Biologics with a brilliant team of people at Biomatters, Auckland, New Zealand.

Most of my experience lies in Lua, Java, C/C++, Ruby, Python, PHP, Go, JavaScript, and TypeScript (including various web stacks/frameworks such as Ruby on Rails, Svelte, AngularJS and RxJS).

In my free time, I'm working on TPA, a concurrent importer for a home movie server, implemented in Go.

Oh... I run arch btw

thea's Issues

Improve transcode workflows code

Transcoder Profiles

Top-Level Overview

Currently, I'm at the stage of development where ffmpeg integration is the next logical step; however, it raises the problems of what exactly the user wants ffmpeg to do. This includes transcoder properties, thread control, etc.

It would be in the best interest of everyone if instead of hardcoding the way that ffmpeg operates, we could instead provide each user with an easy UI to control exactly how ffmpeg operates. Sounds easy right? Well, let's take it one step further.

It would be even better if a set of these profiles could be created, each customized to a particular type of content (1080p, 4k, movie, tv show, etc) and they could be edited on the fly via the web UI, and applied to different types of content manually/automatically.

Profiles

Certain types of content may require different types of transcoding. For instance, perhaps we'd want ffmpeg to target a different resolution depending on the resolution of the source. Perhaps we'd like ffmpeg to output the files to a directory tree if it's a tv-show, but a movie should go to a different file server. Profiles with advanced features such as these could be implemented to service these requirements.

A very rough overview of each major component of a profile can be found below.

Configuration w/ Interpolation

Profiles will require a wealth of options to control exactly how ffmpeg operates. To minimize the complexity for the user, a string interpolation method will be used such that the user is provided a set of keywords, such as %DEFAULT_THREADS%, %TITLE%, %SEASON_NUMBER%, %EPISODE_NUMBER%, %SOURCE_RESOLUTION%.

These variables (and many more) can be inserted into any of the inputs provided when configuring a profile. As an example, perhaps we're specifying the output path of files from a certain profile; we could set it is as "/mnt/my_nas/movies/%TITLE%_%YEAR%/%SOURCE_RESOLUTION%.%EXTENSION%" which would place a 4k render of "Joker 2019" in /mnt/my_nas/movies/Joker_2019/4k.mp4

Of course, the specifics of these examples are irrelevant; it's only to demonstrate that a profile could control fully exactly how ffmpeg operates. Some preset options could be provided (thread count, output directly, export format), but an advanced text input could be provided that simply allows the user to type in the command-line ffmpeg arguments manually (still featuring the above variable interpolation).

Multiple Targets

To accommodate users of slower internet connections, or those of us who don't wish to stream 4k content, the option for a profile to specify multiple 'targets' is a nice-to-have. This would mean, as an example, a movie ingested at 4k may have two or even three ffmpeg targets at different resolutions.

However, at this level of customization, there is only so much TPA can do to assist the end-user. When configuring a target (each profile will have one by default) TPA will simply provide the user with some basic options, and then an advanced text entry for command-line arguments.

While we'd love to be able to provide a discrete option for each and every eventuality, the reality is that ffmpeg is a huge and complex piece of software, and end-users will be better served by TPA if there is full transparency with regards to what ffmpeg is being provided as arguments.

Threading and Blocking

One of the features of TPA is that it's fully parallelized. This means that our profiles will need to have a concept of threads, more specifically, how many they want. To follow on from the above Options, each profile target must be able to specify an explicit thread count as well as whether or not the target is blocking.

For instance, a profile may have 3 targets however the first target should be fully completed before the next targets are started in parallel with each other. To achieve this, target 1 must be the first in the list of the profiles targets, and it should be configured as 'blocking'. Additionally, each target could specify how many threads it wants. Target 1 may want 4 threads, while targets 2 and 3 are happy transcoding more slowly with 1 thread as the target resolution is low.

Applying

These profiles provide a huge degree of flexibility, but what makes them truly useful is that these are not server-level settings. They are individual groups of settings that can be applied to a particular item in the queue. Rather than the server having a concept of these settings, we instead have profiles (one of which is default) that can be selected for an item and all targets inside of that profile will be run against the item.

During the transcoding of this item, the server will lookup the profile that has been selected for this item and retrieve the configuration for the ffmpeg instances (how many, how the threading should be achieved, and what arguments for each ffmpeg instance).

Automatically Applying Profiles

Applying these profiles sounds good on paper, however, TPA is designed to be configured and then left to run with no user intervention. Therefore it's only natural that profiles be fitted with some form of auto-application logic. For instance, a regex match against a title, a specific resolution, perhaps only movies, etc. Each profile in the server's config will be tried for a match against these settings, and the first to match will be applied to the item.

Cancellation with cleanup of partial formatting

Cancelling an item, or any of it's ffmpeg instances, should see that the instance is correctly terminated and any partial transcoding output is removed from the file system.

QueueItem FormatTitle method needs to find resolution

The regular expressions used to parse out rudimentary file information currently don't search for a resolution, which is needed when writing the output file via FFmpeg. This needs to be resolved for #1

Failing to run on other machines

Receiving this error when running go run ..

internal/ffmpeg/commander.go:7:2: github.com/floostack/[email protected]: replacement directory /home/haz/go/src/transcoder does not exist
internal/core_service.go:4:2: github.com/floostack/[email protected]: replacement directory /home/haz/go/src/transcoder does not exist

I assume you have some hardcoded paths still referencing your machine (i.ie haz). But could be wrong...

Authentication

Implement basic username/password user authentication so we can know who is making a given request so that we can retrieve their granted permissions and block any actions which they are not permitted to do (in a follow up ticket, #25)

Profile creation does not trigger commander scheduling

When a new profile is created, the FFmpeg commander should check for any items that match the profile match conditions, and spawn an FFmpeg instance for each that does.

However, this doesn't happen until the user performs some other action (e.g. reorders the list, cancels an item, etc). This indicates that the commander is not correctly subscribed to the changes to the ProfileList... Should be a simple bug to track down and squash.

Authorization/permissions

Following on from #24, Thea should be equipped with a permissions system that allows authenticated users to perform certain restricted actions if they're authorized.

However logging in should not be required to access the fundamental Thea experience, so the below represents some example actions - and whether an un-authenticated user is allowed to perform them:

• movies:read ✅

• moves:write ❌

• queue:read ✅

• queue:write ❌

• profiles:read ✅

• profiles:write ❌

• users:read ❌

• users:add ❌

• users:remove ❌

• users:edit ❌

• users:edit_permissions ❌

Some other permissions are likely to have been missed out here, but it's important that the connection between these permissions and each of the Thea endpoints is clearly understood by the UI - so that forbidden actions can be disabled/hidden for unauthorized users.

Item and FFmpeg instance pausing

Pausing an item or specific FFmpeg instance is pretty useful and I'd like to implement it as part of this issue. Pausing an item solves the problem of a user not wanting work to be done on a specific item yet as cancelling an item is not a useful solution to this problem. However the behaviour of "pausing" needs to be fully explored.

Behaviour

Use Case 1

Pausing an "Item" in the queue

When a user elects to pause an entire item, the following is the expected behaviour:

• If the Item is at any stage OTHER THAN the format stage, the request to PAUSE will be accepted but not acted upon immediately as pausing at these stages is not possible. This means the item will either continue to be processed until it reaches the FFmpeg stage, where it will then pause, OR the item will complete it's database interaction and then become "Completed".
• If the Item is being formatted, all ffmpeg instances will be paused. Each instance will have to do the work required to make this happen, including suspending any active ffmpeg commands and blocking any future instances from starting.

It's important to note that the dispatcher algorithm for TPA will consider a paused instance as not using any threads, and will essentially ignore it and move on to the next item. This means that pausing an item is essentially instructing TPA to temporarily skip that item.

Use Case 2

Pausing the "Queue"

This is a front-end UX flow that will allow the user to pause the entire Queue of items with a single click. This will behave exactly as above for each individual item in the Queue. Of course, as all items are paused this means that the dispatcher will have no work to do and the server will essentially come to a stable IDLE state.

New items will still be detected and ingested at this time, however, they will have a Paused status applied to them automatically and will therefore be processed up to the FFMpeg stage where they will promptly pause.

Use Case 3

Pausing a specific FFmpeg instance

Pausing a specific instance/target for an item is a useful function as well, as it allows a user to "skip" a specific format item for a temporary amount of time. It's important to note that an item will not progress to the next stage of the pipeline until all format tasks are either canceled or completed, paused items are taken in to account here and will be waited on.

If the containing 'Item' is paused in it's entirety, then all new and existing ffmpeg instances will automatically be paused. If an item is un-paused, all instances will be un-paused, even if they were specifically paused before the Item was paused. That is to say that TPA does not know nor care why or what mechanism an instance was paused via - un-pausing an Item will un-pause all instances.

Implementation

Firstly, QueueItems will have a PAUSED status which will be set when the user pauses a specific item or the entire Queue, this flag is used when a new FFmpeg instance is started, and if set, will cause new instances to automatically pause themselves.
When a QueueItem is unpaused, this PAUSED status will be un-set, and IF all instances are paused, they will be un-paused as well.

Secondly, all FFmpeg instances will have a PAUSED status too. When an FFmpeg instance is paused directly, only this one instance is set to paused; The parent item is still considered to be functioning normally. However, if all other instances are complete except for this paused instance, the Commander will flag the item as PAUSED to indicate that work is blocked, but not due to error (which is represented by NEEDS_ATTENTION).

If an ITEM is paused, all FFmpeg instances will be PAUSED. If any specific instance is unpaused, then the entire item becomes un-paused... In essence, while an item is in the FFmpeg stage, it's 'paused' status is a reflection of all non-complete ffmpeg instances.

Live/on-the-fly transcoding for source media

Ability to import existing libraries

Users adopting Thea should be able to have a seamless and simple experience when it comes to importing existing libraries of movies/tv shows... Lots to go over here, and there are likely many ways to approach this... however an interactive UI for the users is probably needed.

Store media genre information, and add optional filtering on the existing media list endpoints

Currently we aren't storing any of the genre information for the media we ingest - this needs to be rectified in order to facilitate searching/listing/filtering media based on what genres it's associated it.

Tricky part is that TMDB provides genre information for movies and series, which means we cannot simply have an associative entity between two tables (e.g. genres <-> media). Best idea I have at the moment is to just have a genres table which we reference using a series_genres and movies_genres associative entity. We can join on these in the relevant queries to apply filtering. Not as ideal as having a single associative entity though.

Implement a trouble system

The trouble system should be capable of tracking errors in the program to prevent the API from panicking. The troubles should expose the information regarding the error, and also a means to recover (perhaps restarting the queue item pipeline stage, or maybe manually providing information that couldn't be found from OMDB).

This system should also integrate with external APIs, such as the CLI interface and the provided HTTP RESTful API

Reclaim FFmpeg thread resources when paused (maybe?)

Open to debate whether or not this makes sense, but it's reasonable to assume a user may pause a transcode in order to get a different transcode to start, however currently the 'thread resources' are claimed when the transcode task begins, and they're only released when the task completes. This includes when the task is successful, cancelled, or troubled.

However, we may want to consider this behavior for paused transcodes too. It's not as trivial as the other three states as those don't need to actually keep any state around. A pause needs to carefully dance with the service to properly release threads on suspension and re-acquire threads before resuming (which could mean the task waits for many minutes/hours).
Currently the transcode service simply does not allow for this type of behavior, but anything is possible... it's just whether the complexity is worth it.

Error log spam due to 'OutputPath' being called on an invalid ffmpeg instance

If an ffmpegInstance is asked for it's OutputPath() when it has no ffmpeg command associated with it (i.e. before it's started), then an error is logged.

Figure out WHY this is being called before the instance is ready... Thea should not be interested in this items output path before it's finished... This probably indicates a problem in the way Thea is handling 'finished' ffmpeg instances.

To repro, follow instructions in readme to provide dummy data to Thea, ensure that the item progresses to the FFmpeg stage, watch error log emitted:

[Commander] (!!) Cannot get output path of command &{<TRUNCATED>}: no ffmpeg command is available

Add title filtering to list endpoint

Implement CLI to view status, troubles and queue

JSON cache file to keep track of processed files

As we plan to remove processed files after a user-configurable amount of time has passed since they were processed by this software, we need some way to keep track of files that have already been processed.

Option 1.

Simply look at the output directory to see if the file has been processed

Pros:

• Easy

Cons:

• Requires some amount of processing to determine what the output path of an item should be. This would mean allowing a QueueItem to proceed through 3 of the 4 stages of the pipeline before realizing it's already been processed... Waste of time

Option 2.

A cache file - likely JSON - that contains a persistent list of items that we've already processed by using their exact filename from the source directory. When a source item is deleted automatically we can very easily purge this JSON file of items not needed anymore.

Pros:

• Also easy
• Can determine if an item should be added to the Queue during the first pipeline stage

Cons:

• None really. The needs of the cache file are very simple.

I think Option 2 makes the most sense here. A database is another option to consider but it's simply overkilling a very small task that will require a small amount of disk space...

Notification pane with 'toasts' for errors

Currently errors from the Thea server are popped up in a horrible vanilla alert()... We can do better!

Some simple toast notifications would do well here, ideally allowing for a LOT of information which the user could see upon clicking (an 'X' to close, but clicking on the notif 'expands' it somehow - perhaps a modal).

Additionally a notification 'pane' would be great to allow the user to view auto-hidden toasts later.

Requirements:

• Multiple toasts can be displayed at one time (e.g. in top-right of window)
• Toasts auto-hide after some time has passed
• Auto-hidden toasts can be viewed later by opening notification pane (e.g. notifications button in the top-right to open notifications pane)
• On click the toast opens a modal to show extra information
• Each toast has a dismiss button which closes the toast (no longer visible in notifications pane)
• Notification pane must also allow for dismissal of individual toasts OR all toasts at once (i.e. 'Dismiss All')

Svelte: Refactor existing components + styling

The Svelte front-end is getting a little out of hand... some components have far too much business logic in them, some are using stores for state management (good!) other's aren't (for no good reason).

The Thea front-end tries to be consistent by re-using styling... however this has come at the cost of some CSS being duplicated due to poor component design. Therefore, it's not always clear where to look when trying to change a style. Some re-usable Svelte components for basic grouping and layouts would help a lot.

• Split up existing components in to smaller, re-usable components
• Refactor existing CSS to be coupled to the above new re-usable components
• Remove business logic from components and find a home for them in a 'service' or somethin

Completed items processed in to database correctly

Allow FFmpeg execution to be killed

Currently, we have the ability to terminate running FFmpeg instances (by canceling the context passed to the transcoder)... However, the endpoint to cancel instances is no-op, so some work needs to be done to 'wire' the incoming commands to do so throughout Thea. We also need to ensure that partially transcoded content is obliterated

Websocket API

Having thought about the internals of the API some more, the currently available HTTP RESTful API is not particularly well suited to the type of web application that might be written for this API.

For instance, we'd like to be able to view the real-time progress of the queue without having to periodically poll the server for status. By using a WebSocket, the server can communicate/PUSH changes via the WebSocket so the client can immediately update the state it's displaying to reflect the new state.

Additionally, rather than the client having to poll for the entire queue state, the client can instead be delivered the new state for a particular queue item - this cuts down dramatically on the load being transferred to and from the server when a single queue item changes its state.

It's currently undecided if the RESTful API will be replaced by the WebSocket implementation. It's unlikely to be useful for the web app I plan to develop for this server. Might be worth keeping around as a fallback option, however as this project is in its early stages it feels like now would be the best time to deprecate the REST API if WebSockets prove to be superior.

Implement HTTP RESTful API

Replace /media/latest with a generic list endpoint with user-defined ordering

The current GET /api/thea/v1/media/latest endpoint does the trick, however it would serve other purposes better if the ordering was user-defined. This could be accessible by GET /api/thea/v1/media/ and would behave the same except with user-defined ordering (with some server side default to ensure consistent paging).