dusk-labs / dim

Dim, a media manager fueled by dark forces.

Dim

Dim is a self-hosted media manager. With minimal setup, Dim will organize and beautify your media collections, letting you access and play them anytime from anywhere.

Running from binaries

Dependencies

• libva2
• libva-drm2
• libharfbuzz
• libfontconfig
• libfribidi
• libtheora
• libvorbis
• libvorbisenc
• libtheora0

You can then obtain binaries from the release tab in github:

1. Unpack with unzip ./release-linux.zip && tar -xvzf ./release.tar.gz
2. Run cd release && ./dim
3. Then you can access the Dim web UI through your browser with http://0.0.0.0:8000 (assuming it's running locally.)

Running with docker

The following command runs dim on port 8000, storing configuration in $HOME/.config/dim. You may change that path if you'd like to store configuration somewhere else. You can mount as many directories containing media as you like by repeating the -v HOST_PATH:CONTAINER_PATH option. In this example, the path /media on the host is made available at the same path inside the Docker container. This name "media" is arbitrary and you can choose whatever you like.

docker run -d -p 8000:8000/tcp -v $HOME/.config/dim:/opt/dim/config -v /media:/media:ro ghcr.io/dusk-labs/dim:dev

Mutiarch image resides at ghcr.io/dusk-labs/dim:master

To use hardware acceleration, mount the relevant device:

docker run -d -p 8000:8000/tcp -v $HOME/.config/dim:/opt/dim/config -v /media:/media:ro --device=/dev/dri/renderD128 ghcr.io/dusk-labs/dim:dev

Refer to docker-compose-template.yaml to run dim using Docker Compose.

Running from source

Dependencies

To run from source, you'll first need to install the following dependencies on your system:

• sqlite
• cargo
• rustc (nightly)
• yarn, npm
• libssl-dev
• libva2 (only if you're using Linux)
• libva-dev (only if you're using Linux)
• libva-drm2 (only if you're using Linux)
• ffmpeg

Once the dependencies are installed, clone the repository and build the project:

git clone https://github.com/Dusk-Labs/dim
yarn --cwd ui/ && yarn --cwd ui/ build
mkdir utils && ln -nfs $(which ffmpeg) utils/ffmpeg && ln -nfs $(which ffprobe) utils/ffprobe

If you're on Linux, run dim with:

cargo run --features vaapi --release

On other platforms where libva isn't available, run dim with:

cargo run --release

License

Dim is licensed under the AGPLv3 license (see LICENSE.md or https://opensource.org/licenses/AGPL-3.0)

Screenshots

dim's Issues

Create bootstrap script that fetches the latest ffmpeg and ffprobe binaries in case they dont exist

Add activity tab

The activity page is a page meant for administrators to see information about their server, what users are doing in Dim, if there are any transcoding sessions, the admin can see info about the stream, progress, performance and other information.

Invalid redirect when clicking on a genre on a media page

Add feature flag that disables auth on devel envs

Implement collections.

Truncate genres in the pop up cards

Add anime support.

Improving the streaming routes

• Add ability to specify an arbitrary timeout when fetching chunks at manifest creation. This could be useful for players that do not retry fetching a chunk if the chunk route returns 102 (such as mpv).

Prettify media page

Add/Delete/Move FS events not handled

streaming: remove unnecessary dependency to `stoppable_thread`

We are currently using stoppable_thread to wrap the thread that parses ffmpeg's stdout, however this is now useless and no longer needed as the Session logic is decoupled from the TranscodeHandler (should be rename to FfmpegStdoutParser).

Use enums for jwt roles

Fix dockerfile and ci/cd

Sexify the dashboard when empty

Design a plugin system.

switch ws-rs in favour of a different ws lib and re-enable ws events.

When scanning, download posters and backdrops locally.

Save sidebar preference

Allow users to manually rematch a card or a unmatched card.

Fix video streaming

Currently video.js is very picky about streams. Streaming with nightfall works fine in MPV and Vlc, however its partially broken in browsers and more specifically video.js.

This is because video.js is very strict about segment continuity. Because of this seeking doesnt work, and hard timeouts (ffmpeg getting killed after a timeout instead of being issues a SIGSTP) have to be disabled. On further inspection video.js crashes because when ffmpeg is restarted with a new timestamp offset, the fragments it outputs also have a timestamp offset that is reset to 0, and because init.mp4 is never reloaded by videojs automatically video.js thinks that the segment requested is actually older than it is, thus it tries to fetch the next segment infinitely until it gets a valid segment timestamp offset.

We can fix this two ways, either we patch video.js to be more lenient, or when we seek in a non-transcoded region we restart video.js and reload the dash manifest (this needs to be paired with soft-timeouts for transcoding sessions).

Figure out how to handle and display unmatched media.

In some cases mediafiles may not get matched because of weird naming conventions. In these cases we still need to display the files in each library somehow so a user can manually match them.

Patch up settings page

Store parent directory id for each mediafile.

nightfall: panics during streaming.

on the experimental branch fix-video-streaming I get playback halting it looks like its just buffering on the UI side but if you check out the server stderr I see panics in nightfall

Mar 24 21:35:10.404 INFO request, uri: "/api/v1/stream/9c374e3a-4d2e-4388-9dbd-38185ee28c60/data/5.m4s", method: Get
try wait, ffmpeg died, killing
thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value', /home/mental/.cargo/git/checkouts/nightfall-b84a4513de79ba91/9ade607/src/lib.rs:177:53
Mar 24 21:35:38.464 INFO response, status: 500 Internal Server Error, route: GET /api/v1/stream/<id>/data/<chunk..> [2] (get_chunk)
try wait, ffmpeg died, killing
thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value', /home/mental/.cargo/git/checkouts/nightfall-b84a4513de79ba91/9ade607/src/lib.rs:177:53
Mar 24 21:35:38.473 INFO response, status: 500 Internal Server Error, route: GET /api/v1/stream/<id>/data/<chunk..> [2] (get_chunk)

Handle cases where files dont have a title and we have to parse the root dir

Feb 21 09:04:34.064 INFO Scanning file: /media/media3/debugging/Shows/The Good Place Season 2 1080p x264/S02E08 Leap To Faith.mkv for lib=1
Feb 21 09:04:34.064 ERRO Failed to mount file into the database: FilenameParserError
Feb 21 09:04:34.064 INFO Scanning file: /media/media3/debugging/Shows/The Good Place Season 2 1080p x264/S02E04 Existential Crisis.mkv for lib=1
Feb 21 09:04:34.065 ERRO Failed to mount file into the database: FilenameParserError
Feb 21 09:04:34.065 INFO Scanning file: /media/media3/debugging/Shows/The Good Place Season 2 1080p x264/S02E03 Team Cockroach.mkv for lib=1
Feb 21 09:04:34.065 ERRO Failed to mount file into the database: FilenameParserError
Feb 21 09:04:34.065 INFO Scanning file: /media/media3/debugging/Shows/The Good Place Season 2 1080p x264/S02E12 Somewhere Else.mkv for lib=1
Feb 21 09:04:34.066 ERRO Failed to mount file into the database: FilenameParserError
Feb 21 09:04:34.066 INFO Scanning file: /media/media3/debugging/Shows/The Good Place Season 2 1080p x264/S02E11 The Burrito.mkv for lib=1
Feb 21 09:04:34.066 ERRO Failed to mount file into the database: FilenameParserError
Feb 21 09:04:34.066 INFO Scanning file: /media/media3/debugging/Shows/The Good Place Season 2 1080p x264/S02E01 Everything Is Great!.mkv for lib=1
Feb 21 09:04:34.066 ERRO Failed to mount file into the database: FilenameParserError
Feb 21 09:04:34.067 INFO Scanning file: /media/media3/debugging/Shows/The Good Place Season 2 1080p x264/S02E06 Janet And Michael.mkv for lib=1
Feb 21 09:04:34.067 ERRO Failed to mount file into the database: FilenameParserError
Feb 21 09:04:34.067 INFO Scanning file: /media/media3/debugging/Shows/The Good Place Season 2 1080p x264/S02E09 Best Self.mkv for lib=1
Feb 21 09:04:34.067 ERRO Failed to mount file into the database: FilenameParserError
Feb 21 09:04:34.067 INFO Scanning file: /media/media3/debugging/Shows/The Good Place Season 2 1080p x264/S02E10 Rhonda, Diana, Jake And Trent.mkv for lib=1
Feb 21 09:04:34.068 ERRO Failed to mount file into the database: FilenameParserError
Feb 21 09:04:34.068 INFO Scanning file: /media/media3/debugging/Shows/The Good Place Season 2 1080p x264/S02E05 The Trolley Problem.mkv for lib=1
Feb 21 09:04:34.068 ERRO Failed to mount file into the database: FilenameParserError
Feb 21 09:04:34.068 INFO Scanning file: /media/media3/debugging/Shows/The Good Place Season 2 1080p x264/S02E07 Derek.mkv for lib=1
Feb 21 09:04:34.069 ERRO Failed to mount file into the database: FilenameParserError
Feb 21 09:04:34.069 INFO Scanning file: /media/media3/debugging/Shows/The Good Place Season 2 1080p x264/S02E02 Dance Dance Resolution.mkv for lib=1
Feb 21 09:04:34.069 ERRO Failed to mount file into the database: FilenameParserError```

Libraries with duplicate paths don't get indexed

• Create one library with a file path and set its media type to movies (name this "xxx")
• Wait for indexing to complete
• Create a second lib with the same file path as "xxx" but the media type is set to "Tv Shows"

Expected behavior: Instantly indexed due to already being indexed before so the results should be cached.

Actual behavior: Indexing never starts.

Update Dashboard when new library is added

Move to rocket 0.4.7 and async

Use sqlite for most unit-tests to simplify CI

Streaming crashes in docker/latest ffmpeg

When streaming from docker the stream crashes because the init fragments for sessions is placed in the dim root and not the streaming cache directory.
I think this can be fixed by including a full path when passing the segment name to -hls_fmp4_init_filename

Streams crash after around chunk 380

Investigate why audio streams crash around this mark. Its something to do with processes being paused but mark as not paused.
This most likely has something to do with how we calculate the current chunk being transcoded.

As a temporary solution disabling timeouts for audio streams seems to fix the issue.

QuotaExceededError in the video player when playing stuff for more than 3min.

If you try to play a video for more than 3mins the player will crash with a QuotaExceededError. The issue doesnt occur with the v4 alpha of dash.js.

This project is related to me

we are both dim

When going to a different route, reset scroll

Add ability to pin metadata to files.

We should add a feature which either pins metadata to media automatically or manually, this can make scanning a library more efficient in the future because we can essentially cache all the correct metadata onto the file itself.

It will also make manual rematches effectively persistent unless overwritten, for large libraries this could be a life saver.

Investigate why removing a library doesnt delete all media

Add icon and other metadata

For the executable file, company name and so on

Simplify and integrate parser daemon into the scanners.

Implement settings page and apis

Explore ffmpeg.wasm for client-side transmuxing

Draft SECURITY.md and ARCHITECTURE.md

strip extensions when passing filenames to torrent-name-parser

extract stills manually rather than rely on tmdb or other metadata providers.

List unmatched medias at the bottom of a library

Fix media page for tv shows

fetching posters should be prioritised over fetching stills

`src/routes/general.rs` requires a bit of a refactor.

src/routes/general.rs is getting quite hefty and ngl ugly. It needs a bit of a touchup, especially the /dashboard routes.

streaming: clean tmp streaming cache after ffmpeg dies.

Some people mount their tmpfs to 10TB drives, others (like me) have it at a few gigabytes. when the cache fills up playback perpetually fails until the cache is empty.

We should clean the streaming cache on a playback exit.

Optimize the tv show scanner for speed.

Images not correctly fetched on library pages.

When the API returns a relative url to a backdrop or a poster, the webui prepends library to the path before fetching it.