sebgl / htpc-download-box Goto Github PK
View Code? Open in Web Editor NEWSonarr / Radarr / Jackett / NZBGet / Deluge / OpenVPN / Plex
Sonarr / Radarr / Jackett / NZBGet / Deluge / OpenVPN / Plex
Hi
First of all, very good job for this tutorial ! very complete and detailed, awesome !
I'm having an issue after the setup of the vpn. I use a PIA account like in your example but when trying to access to the web ui from my personal computer, I get a connection time out. Looks like its trying to reach during 5secs and then response with a timeout message. Im note sure where to investigate, is it a server issue, deluge web ui issue or vpn issue ?
Cheers
I have essentially used the docker-compose example provided but I can not access the deluge WebUI from the network. However from the docker host I can curl 127.0.0.1:8112 and the deluge WebUI responds. Not I did change the vpn -r command to my internal network of 10.0.10./24.
For testing I took the firewall on the host down, so that should take care of that issue.
Any ideas?
Hi,
I prefer to use qBittorrent over Deluge. Also, the setup of openvpn/deluge is quite complicated and prone to failure.
Therefore, I used the repo alpine-qbittorrent-openvpn instead of deluge/openvpn. My setup is:
alpine-qbittorrent-openvpn:
volumes:
- ${ROOT}/media/downloads:/downloads
- ${ROOT}/config/qbittorrent-openvpn:/config
- "/etc/localtime:/etc/localtime:ro"
environment:
- OPENVPN_PROVIDER=PIA
- OPENVPN_CONFIG=netherlands
- OPENVPN_USERNAME=[redacted]
- OPENVPN_PASSWORD=[redacted]
- PUID=${PUID} # default user id, defined in .env
- PGID=${PGID} # default group id, defined in .env
- LAN=192.168.0.0/16
ports:
- "8080:8080"
cap_add:
- NET_ADMIN
image: guillaumedsde/alpine-qbittorrent-openvpn:latest
I used PIA as VPN, for other VPNs please refer to the alpine-qbittorrent-openvpn page.
This might help other people as well, that have issues with deluge/openvpn.
The whole thing works great except for two things. First of all I had to comment out the last two lines of the docker-compose.yml file:
That allowed bazarr to load. Since it's only for subtitles, I probably won't use it anyway.
The second problem is that plex-server won't stay started. I see this in the docker logs over and over:
standard_init_linux.go:219: exec user process caused: exec format error
standard_init_linux.go:219: exec user process caused: exec format error
standard_init_linux.go:219: exec user process caused: exec format error
standard_init_linux.go:219: exec user process caused: exec format error
standard_init_linux.go:219: exec user process caused: exec format error
standard_init_linux.go:219: exec user process caused: exec format error
It just keeps restarting. I'm guessing it has to do with building the docker image for the arm architecture of my Raspberry Pi. I'm new to docker could you please point me in the right direction?
I'm just about to start using this project as I currently have all these items installed directly on my main linux pc but I have to add NzbGet and I didn't want to increase the issues mainly with the split tunneling config which kindof messes up any complex networking on the host pc anyway so i was looking for VM alternatives or docker
I've looked up a few comparisons and it seems transmission is the better choice for lightweight feature rich torrent client compared to deluge though they are very similar
Is there an option to use transmission instead of deluge?
I originally found the project from a search that hit this pi fork first https://github.com/marchah/pi-htpc-download-box
and they have swapped to transmission from deluge so I was going to try the config from there for that part and see if i could get it working as the daemon option is very lightweight and its what i already have working now with high thousands of files with no issues for years
The path that is provided in the docker-compose configuration for plex is inconsistent with the other provided container configurations.
To remain consistent with the completed download location of the other containers:
plex-server:
container_name: plex-server
image: plexinc/pms-docker:latest
restart: unless-stopped
environment:
- TZ=${TZ} # timezone, defined in .env
network_mode: host
volumes:
- ${ROOT}/config/plex/db:/config # plex database
- ${ROOT}/config/plex/transcode:/transcode # temp transcoded files
- ${ROOT}/complete:/data # media library
Should be
plex-server:
container_name: plex-server
image: plexinc/pms-docker:latest
restart: unless-stopped
environment:
- TZ=${TZ} # timezone, defined in .env
network_mode: host
volumes:
- ${ROOT}/config/plex/db:/config # plex database
- ${ROOT}/config/plex/transcode:/transcode # temp transcoded files
- ${ROOT}/downloads/complete:/data # media library
Hello @sebgl , first of all thank you for this project, it is exactly what I was looking for.
After following all of your instructions, I cannot overcome this VPN error:
vpn | grep: /vpn/vpn.conf: No such file or directory vpn | ERROR: VPN CA cert missing! vpn | Options error: In [CMD-LINE]:1: Error opening configuration file: /vpn/vpn.conf vpn | Use --help for more information. vpn | ERROR: VPN not configured!
I am using PIA CA-Vancouver server, renamed the .ovpn to vpn.conf, setup the username/password file and set my ${ROOT}/config/vpn.
I triple checked the .yaml and your instructions but it just does not want to work. I am hoping you or someone else can help me out here.
Much appreciated.
I needed to remove
ports:
- 6767:6767
from my docker-compose file for Bazarr and localhost:6767 still worked so I assume it's not necessary the default port bindings go the right place
ERROR: for bazarr "host" network_mode is incompatible with port_bindings
ERROR: for bazarr "host" network_mode is incompatible with port_bindings
Traceback (most recent call last):
File "docker-compose", line 3, in <module>
File "compose/cli/main.py", line 81, in main
File "compose/cli/main.py", line 202, in perform_command
File "compose/metrics/decorator.py", line 18, in wrapper
File "compose/cli/main.py", line 1188, in up
File "compose/cli/main.py", line 1184, in up
File "compose/project.py", line 713, in up
File "compose/parallel.py", line 108, in parallel_execute
File "compose/parallel.py", line 206, in producer
File "compose/project.py", line 699, in do
File "compose/service.py", line 600, in execute_convergence_plan
File "compose/service.py", line 522, in _execute_convergence_recreate
File "compose/parallel.py", line 108, in parallel_execute
File "compose/parallel.py", line 206, in producer
File "compose/service.py", line 515, in recreate
File "compose/service.py", line 634, in recreate_container
File "compose/service.py", line 335, in create_container
File "compose/service.py", line 959, in _get_container_create_options
File "compose/service.py", line 1091, in _get_container_host_config
File "docker/api/container.py", line 598, in create_host_config
File "docker/types/containers.py", line 339, in __init__
docker.errors.InvalidArgument: "host" network_mode is incompatible with port_bindings
[451362] Failed to execute script docker-compose
I've used jackett since starting this venture but switching to prowlarr as part of the -arr suite... best move evarr..see what I did there..
You add an indexer and it can sync directly to the other -arr programs(sonarr, radarr, readarr, etc)
#Prowlarr
prowlarr:
container_name: prowlarr
image: linuxserver/prowlarr:nightly
ports:
- "9696:9696"
environment:
- PUID=${PUID} # default user id, defined in .env
- PGID=${PGID} # default group id, defined in .env
- UMASK=002
- TZ=${TZ} # timezone, defined in .env
volumes:
- /etc/localtime:/etc/localtime:ro
- /media/config/prowlarr:/config # config files
- /media/downloads/torrent-blackhole:/downloads # place where to put .torrent files for manual download / I added as jackett had it. testing if necessary
remember if using root for everything to comment out the PGID and PUID
when loading...
http://ipaddress:9696/
If that doesnt work.. try http://ipaddress:9696/settings/general initially if the first doesnt work
So where it mentions..
Note: You may need to chown -R $USER:$USER /path/to/root/directory so Sonarr and the rest of the apps have the proper permissions to modify and move around files. This Docker image of Sonarr uses an internal user account inside the container called abc some you may have to set this user as owner of the directory where it will place the media files after download. This note also applies for Radarr.
I think I finally get what its saying?
in the .env file it specifies the user in charge of these will be puid and pgid of 1000
# UNIX PUID and PGID, find with: id $USER
PUID=1000
PGID=1000
in order to find your puid/pgid you will use the command
id yourUsername
This returns the results..
uid=1000(username)` gid=1000(username) groups=1000(username), 998(docker)
or like in my original trial...
uid=27(otherusername) gid=27(otherusername) groups=27(otherusername),998(docker)
the puid and pgid need to match whatever user you are trying to run these as.
I set mine to match root as its a VM anyway..
docker-compose down
docker-compose up -d
it now allows in deluge to set the autoadd where before it was stating I(or maybe it) did not have read/write permissions.
who knows what else this will resolve.
Hope this helps further explain this to someone struggling.
Disclaimer: If I'm incorrect or even just partially, please let me know. going on my 3rd try installing from the beginning.
thanks to..
https://www.reddit.com/r/docker/comments/e2ia04/permissions_with_wordpress_bind_volume/
https://hub.docker.com/r/linuxserver/plex
https://docs.linuxserver.io/general/understanding-puid-and-pgid
https://forum.openmediavault.org/index.php?thread/27226-docker-containers-runs-as-root-ignores-puid-pgid-setting/
Configured as shown, I can only access deluge from localhost
. I can't even access it from my local network with the machine's IP (e.g. 192.168.0.5:8112
). Every other service I can access, so I'm guessing it has something to do with the vpn network.
Any ideas?
Also, in your guide you state:
Important ports are 8112 (web UI) and 58846 (bittorrent daemon)
But I don't see 58846
referenced in the compose file at all. Should it be?
In the proposed config, deluge is indeed downloading requested files over the VPN tunnel defined.. but the indexer used to query the many portals you define is not.. would you not want to pipe that over the vpn service as well?
however in doing so, it appears you can no longer connect to the host on the local LAN. but I would assume the internal subnet exceptions on the vpn service would then route the traffic correctly.. but it does not..
Thoughts?
for plex, where is /data/movies, /data/tv, etc by default located?
If I map directly top the location I know where my files are, it doesn't find them without the /data mapped in plex as a library source
currently the files are located in/media/downloads, because they haven't been sorted into movies or tv.
I have another movie that is not showing up in that dir.
I also see
data at the end of the directory listed near the top but no matter the folder I goto it shows that
I just checked the movie folders from the download, apparently it is moving them but to where!? I need to redir to these folders
Ever though of setting this up with Emby instead of plex?
PIA allows port forwarding to it's users from specific servers. Does this solution leverage this functionality? It would greatly benefit seed ratios for private trackers.
Hello there!
First of all thanks a lot for this project!
I've tried to run it on my laptop with i686 and Ubuntu 16.04.5 LTS on board but got the following errors for all the images:
Recreating plex-server ... done
Recreating radarr ... done
Recreating nzbget ... done
Recreating sonarr ... done
Recreating jackett ... done
Recreating deluge ... done
Attaching to plex-server, jackett, sonarr, nzbget, docker-compose_deluge_1, radarr
plex-server | standard_init_linux.go:178: exec user process caused "exec format error"
jackett | standard_init_linux.go:178: exec user process caused "exec format error"
nzbget | standard_init_linux.go:178: exec user process caused "exec format error"
deluge_1 | standard_init_linux.go:178: exec user process caused "exec format error"
sonarr | standard_init_linux.go:178: exec user process caused "exec format error"
radarr | standard_init_linux.go:178: exec user process caused "exec format error"
jackett exited with code 1
nzbget exited with code 1
radarr exited with code 1
sonarr exited with code 1
Do you know if there is any way to run these images on this particular configuration without issues?
Thanks in advance!
Hello! First of all, thank you for this amazing guide! This is exactly what I wanted, covering all the major tools with a painstakingly detailed guide. I was completely lost with all these tools, and now I think I have a pretty decent grasp on a lot of it.
I followed the tutorial pretty much to a T, so I have a few comments on minor things that I found issue with:
chown
ing directories to the user, as that could be a point of confusion when permission issues appear.My setup wasn't hindered too much by these minor issues, and I got them all sorted. For the most part, everything was perfect.
After finishing the tutorial, and having a working installation, I played around quite a bit with the organizational structure, as it didn't seem cohesive, centralized, or modular (though not bad). Here is my proposed file structure that I've adopted:
Timezone: Now defined in a .env file to be easily changed and shared, rather than hardcoded.
Root directory: This will be the home of all the files in the project, nowhere else. Also defined in the .env file. Mine looks like /media/johnpyp/WD/homemedia
. All of the directories would follow as subdirectories of the root. Having "downloads" as the root directory in the guide is a bit confusing as well, I think something like "homemedia" is more clear.
Downloads dir: ${ROOT}/downloads
, nzb and deluge directory, currently "..."/downloads/ongoing, but I think downloads is a bit more clear.
Complete dir: ${ROOT}/complete
, this is where the movies and tv directories live, and this is where the plex data directory is.
Config dir: ${ROOT}/config
, rather than have dotfiles for the config, separated from everything else, I think it is useful to have a centralized config directory. This would contain all of the .config/* configuration, and also the .plex and .vpn configuration directories. Mine currently has:
- deluge
- jackett
- nzbget
- plex
- radarr
- sonarr
- vpn
These are the three top level directories. The primary objective of this change is 1) Cleaning up the messy directory naming scheme currently present, and more importantly, 2) Creating a single root directory that can be moved around, even onto different computers or whatever, and operate exactly the same by changing a single line in the .env file.
Here is my current docker-compoose.yml file so you can see how what I'm talking about fits together: https://pastebin.com/LHmw6nZE
Hi,
Thanks for all of this. It's really great and useful.
I'm having trouble starting the plex container.
This is what I see in the logs:
standard_init_linux.go:211: exec user process caused "exec format error"
At first I though it had to due with ARM architecture but I see that is covered on the docker plex image repo
Hi all,
I'll be short and sweet. I've set up a VPN through my Asus router and can access everything remotely, except deluge.
I'm guessing it has to do with the unique setup for the VPN within the HTPC Download Box, but I can't figure out what to change to make this work.
Sonarr, Radarr, Synology, Resilio, and even ssh work through the VPN using a local IP, except deluge on port 8112.
I also tried port forwarding to my server box on port 8112 but that just times out as well...
Is there some way of publicly exposing my deluge web UI?
Like the title says, from what I can tell it is impossible to access the deluge client from a machine on the same network as my server (192.168.1.3) while vpn (with username and password) is enabled.
I have ufw
disabled, and I know I can connect to it because with vpn commented out, it shows page. Just wondering if somebody managed to solve the issue?
I am happy to provide any configs/screenshots. My base docker-compose.yml is as good as standard. I have changed nothing.
So the most frustrating thing I've run into is the constant
root@debian:/# docker-compose up -d
ERROR:
Can't find a suitable configuration file in this directory or any
parent. Are you in the right directory?
Supported filenames: docker-compose.yml, docker-compose.yaml, compose.yml, compose.yaml
as well as the error from VPN running before starting docker once more
Created this
cd /bin
nano d-compose.sh
#!/bin/bash
#turn off VPN(if you have a different VPN like OpenVPN, enter the disconnect command here)
windscribe disconnect
#restart docker servers(make sure to change the filepath to reflect yours)
(cd /htpc/htpc-download-box && docker-compose down)
(cd /htpc/htpc-download-box && docker-compose up -d)
#turn VPN on(if you have a different VPN like OpenVPN, enter the connect command here)
windscribe connect
exit
d-compose.sh
I ran into this issue when testing to make sure that Deluge could actually download torrents.
Basically I'd add a torrent via URL (I tried the torguard check) but it would just error out in Deluge. I first thought my VPN configuration was incorrect and spent many hours debugging.
I'm using Ubuntu server if that matters.
I ended up doing a chmod 777
as I don't really know what I'm doing on Linux and that fixed it but is there a better way?
Is this what we should be doing?
Note: You may need to chown -R $USER:$USER /path/to/root/directory so Sonarr and the rest of the apps have the proper permissions to modify and move around files.
I think it would be best if you added an explicit step to the readme where the correct permissions are set on the folder for whatever user is running Docker or the apps.. I don't really know how this works under the hood.
Hey, I really liked the tutorial, thanks a lot about that but I'm struggling to use rclone with docker. Do you know how to use it? It would be awesome if you could help!
Thanks once again and I hope you can help me.
I have been searching for a solution like yours: download apps (r(u)torrent, sonarr, radarr, nzbget) in 1 docker together with OpenVPN. Simply to have them only work via VPN and isolate them from the rest of the system.
Your solution however creates a docker for each app! Why? Isn't it much easier to create 1 docker image with all of them?
And a seperate docker for the playback solution/database like Plex Server?
This is just an honest question. It seems like a lot more work to do it your way, but I am definitely a noob when it comes to Docker and novice when it comes to Linux (just switched from Win10/MacOS to Ubuntu Budgie).
2nd question: Sonarr/Radarr will visit torrent sites, I would want them behind the VPN just like Deluge. In your stack, you clearly allow them to access the internet without VPN.. Why put Deluge behind it but not these two?
Overseerr and Prowlarr have replaced Jackett in a great way, automatic indexing. It wasn't available with seeds before, only nzb. We created all of this in k3s, but make the swap or follow my lead: https://github.com/awknode/docker-media-server as I've even implemented GPU passthrough. I gave this author credit.
This is really well done. Just wanted to say thanks for sharing 🤗
It seems that Sub-Zero is not going to receive any more feature updates and that the lead developer is now starting to contribute to Bazarr instead (60+ commits to bazarr since October).
Bazarr is a companion to Sonarr and Radarr that is exclusively for finding subtitles. It has more features than Sub-Zero, and it isn't tied to Plex.
echo
"deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
fixed:
deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian buster stable
if anyone has issues copying the line from docker install, check the docker source list file.
might need to remove extra spaces, change to debian if necessary
Hello,
So I think I'm stuck on this part:
Note: You may need to chown -R $USER:$USER /path/to/root/directory so Sonarr and the rest of the apps have the proper permissions to modify and move around files. This Docker image of Sonarr uses an internal user account inside the container called abc some you may have to set this user as owner of the directory where it will place the media files after download. This note also applies for Radarr.
What's the deal with the abc user? I'm using a raspberry pi. Is my "$USER" pi or abc? I can get things into Deluge from Radarr and Sonarr just fine. They move to my "complete" directory, but they are not moved correctly. They go to "complete", but not "complete/tv" or "complete/movies" and they are not renamed correctly. Also, the torrents are not deleted from Deluge. Yes, I've taken care of the seeding ratio stuff. Even when no one is downloading from me the torrents are not deleted.
Here are my Linux absolute paths....
/mnt/downloads/incomplete
/mnt/downloads/complete
/mnt/downloads/complete/tv
/mnt/downloads/complete/movies
downloads is a physical 6TB usb hard drive that I have mounted as downloads. Can anyone help a brother out? I think it may have something to do with raspberry pi/docker.
I recently switched to unraid and wanted to figure out an easier way to integrate the vpn in this solution and came across binhex/arch-delugevpn. I had issues for a long time setting up this project originally due to the deluge/vpn combo so I figured I would post my alternative setup in case it helps anyone else. Here is what I replaced the deluge and openvpn docker containers with:
version: '3'
services:
deluge:
container_name: deluge
image: binhex/arch-delugevpn:latest
restart: always
network_mode: host
environment:
- PUID=${PUID} # default user id, defined in .env
- PGID=${PGID} # default group id, defined in .env
- TZ=${TZ} # timezone, defined in .env
- STRICT_PORT_FORWARD=yes
- VPN_ENABLED=yes
- VPN_PROV=pia
- VPN_USER={PIA_USER}
- VPN_PASS={PIA_PASS}
- LAN_NETWORK=<lan ipv4 network>
- NAME_SERVERS=209.222.18.222,84.200.69.80,37.235.1.174,1.1.1.1,209.222.18.218,37.235.1.177,84.200.70.40,1.0.0.1
volumes:
- ${ROOT}/downloads:/downloads # downloads folder
- ${ROOT}/config/deluge:/config # config files
ports:
- 8112:8112 # port for deluge web UI to be reachable from local network
- 8118:8118
- 58846:58846
- 58946:58946
After that, inside the deluge config folder, I added an openvpn
folder and added the files mentioned in your guide, without renaming the location file. This is the video where I followed the instructions for using this package. It doesn't use a docker compose but it could still help people with the general setup.
I don't expect you to necessarily change the guide to use this, however if someone else runs into issues with your vpn setup and go to issues to try and figure it out, maybe this could be a helpful alternative!
Since we want to support things like remote access to your services, adding a web-based filebrowser would compliment that really nicely. It can be pointed at the centralized ROOT folder and allow the user to have pretty much full control over their setup other than full terminal access.
A quick search gives this, and it looks like a decent option: https://filebrowser.xyz/
Edit: Actually, it seems like it supports limited terminal access too!
Maybe you could add a line in the README.MD to help others from running into this problem. If the download folder is on an external drive nzbget will not be able to unpack files that are larger than somewhere around 4-5 GB in size. I know its a problem more related to nzbget and external drives, but it might help others who are not aware using this great setup. Thanks for everything, it works GREAT! :D
Think its a good idea to have Sonarr/Radarr/Jackett behind vpn?
Was thinking of setting it that way since the infrastructure is already there...
First of all I'd like to say thanks for bringing this together and the work you've put in so far documenting it! Unfortunately I haven't been able to get things working just yet though...
I am able to load deluge html with 'curl localhost:8112' in the guest shell using vagrant ssh but 'curl -v 192.168.7.7:8112' from the host machine times out.
I tried using 'config.vm.network :forwarded_port, guest: 8112, host: 8112' in the Vagrantfile instead of the 192.168.7.7 private network, which "connects" with 'curl -v localhost:8112' but resets without loading any html.
The only way I was able to access the deluge webui on my host was by using the above portforwarding, commenting out the entire vpn: section of the docker-compose.yml and changing deluge's network from 'service:vpn' to 'host' - then localhost:8112 loads perfectly. Does this imply it is a VPN firewall issue?
Are other people able to set this up successfully? I'm very new to this so might be making a simple mistake but have spent a few days researching and trying to debug butseem to have run out of options. Any advice would be greatly appreciated!
I'm using Ubuntu 18:04.4 LTS with VirtualBox Version: 6.1 and Vagrant 2.2.7 on MacOS Catalina 10.15.3
Hey,
First of all thank you for this walk-through. I am almost there, however I encountered an error which I don't know how to fix. (I am a newbie in docker).
I basically followed all instruction regarding docker/or your github. Allthough I cannot figure out but my deluge container keep failing to set up.
I think it is due to the network_mode: service:vpn # run on the vpn network
, because if I change it to host
it starts up normally, but then I have no vpn.
Thanks for help in advance
I already have a significant media library and want to be able to make this available to plex as well. I am aware that I likely need to change the mount point for the hard drive and to adjust the docker compose yaml file to pick the folders up as well.
What would be the best way of going about this?
Note - I am using a RPI4 running raspberry pi os.
I added this in to manage all the other containers in one spot.
I did however, run into an issue of the user acoount providing a NGINX bad gateway error. Commenting those out fixed the issue
organizr:
image: organizr/organizr
container_name: organizr
environment:
# - PUID=${PUID} # default user id, defined in .env
# - PGID=${PGID} # default group id, defined in .env
- TZ=${TZ} # timezone, defined in .env
- fpm=true #true or false | using true will provide better performance
- branch=v2-master #v2-master or #v2-develop
volumes:
- /media/config/organizr:/config
ports:
- 80:80
restart: unless-stopped
I can access Jackett, Deluge, and NZBGet, but I cannot access Sonarr, Radarr, or Bazarr. What can I do to troubleshoot, I am new to Docker. Please help
Thanks heaps @sebgl in the first place for your effort putting all this together with well written documentation.
In my network setup I have Unbound configured as the DNS resolver for all requests which I expose by default via DHCP to connected clients.
While I was doing IP and DNS leak testing with the openvpn-client container and routing traffic through it from deluge, having Docker container inherits the DNS settings of the Docker daemon, including the /etc/hosts and /etc/resolv.conf by default
as per Container-Networking/DNS-Services makes it prone to DNS leaks, if you run a local DNS resolver.
A simple suggestion is to override the DNS on the openvpn client by default in the compose file:
dns:
- 209.222.18.222 # PIA DNS
- 209.222.18.218 # PIA DNS
Testing with $ nslookup whoami.akamai.net
which 'should' return the public facing IP of the VPN provider.
Hi,
I would very much like to do this, but as a first attempt, I'd rather not have the vpn yet. I'd do that later on as an enhancement.
My problem is that I am very new to this. Would it be possible to get some hints/suggestions as to how I can implement this without a vpn?
Thx
Can someone help me setup the container to run with WindscribeVPN? Currently, the way I have the VPN setup I get this error when I try to run the container.
ERROR: for vpn Cannot start service vpn: driver failed programming external connectivity on endpoint vpn (a16f61aa3e4c9cf7c3b3e7b24a0744cafac801af144da4403d97ba2d87c964d0): Error starting userland proxy: listen tcp4 0.0.0.0:8112: bind: address already in use
Any help would be appreciated!
Ive tried on 2 different systems now and continue having the same issue. Deluge might work for a day or two but then eventually I end up seeing this warning and am unable to access the front end:
No translation file found for domain: 'deluge'
�
I will go ahead and throw the full log and docker-compose below:
Log: https://pastebin.com/guv0sUrC
Docker-compose.yml: https://pastebin.com/FE3cCgpz
Curious as to why you haven't enabled firewall (-f "") on the vpn container?
I myself am having issues with the firewall in that container so was looking for others using it and reporting issues if they see them.
Hopefully I've mis-configured something
I am getting a working VPN connection but i can't seem to use it
I could be testing it incorrectly potentially
This is my config for transmission and openvpn
services:
vpn:
container_name: vpn
image: dperson/openvpn-client:latest
cap_add:
- net_admin # required to modify network interfaces
restart: unless-stopped
environment:
- PUID=${VPNPUID} # vpn service user id, defined in .env
- PGID=${VPNPGID} # vpn service group id, defined in .env
- TZ=${TZ} # timezone, defined in .env
volumes:
- /dev/net:/dev/net:z # tun device
- ${VPNROOT}:/vpn # OpenVPN configuration
security_opt:
- label:disable
ports:
- 9117:9117 # port for jackett web UI to be reachable from local network
- 9091:9091 # port for transmission web UI to be reachable from local network
command: '-d -f "" -r 192.168.2.0/24' # -d use the vpns DNS, -f enable firewall, -r route local network traffic
transmission:
image: linuxserver/transmission:latest
container_name: transmission
restart: unless-stopped
network_mode: service:vpn # run on the vpn network
environment:
- PUID=${VPNUSERPUID} # default user id, defined in .env
- PGID=${VPNUSERPGID} # default group id, defined in .env
- TZ=${TZ} # timezone, defined in .env
volumes:
- ${DOWNLOADINGROOT}:/downloads/incomplete
- ${COMPLETEDOWNLOADROOT}:/downloads/complete
- ${CONFIGROOT}/config/transmission:/config # config files
This is close to stock but with it running under a different account, and enabling firewall and vpn dns
.env variables
TZ=Australia/Sydney
VPNPUID=1001
VPNPGID=1001
VPNUSERPUID=1001
VPNUSERPGID=1001
VPNROOT=/etc/openvpn
the other paths don't really matter for this issue
Anyway I have got the docker-compose working well for
I can view transmission and jacket web UI's fine
I had to create a vpn.conf file I couldn't seem to change that name (My previous non docker install used different file name)
the env variable VPN_FILES didn't work unfortunately
My certificate and key are included in the vpn file as well as the link to a credentials file, all this seems to work fine
# VPN chosen is ****.nordvpn.com.tcp.ovpn injected on 2020-12-30 13:01:02
client
dev tun
proto tcp
remote NORDVPNSERVERIP
resolv-retry infinite
remote-random
nobind
tun-mtu 1500
tun-mtu-extra 32
mssfix 1450
persist-key
persist-tun
ping 15
ping-restart 0
ping-timer-rem
reneg-sec 0
comp-lzo no
remote-cert-tls server
auth-user-pass /vpn/pass
auth-nocache
verb 3
pull
fast-io
cipher AES-256-CBC
auth SHA512
disable-occ
script-security 2
route-noexec
<ca>
-----BEGIN CERTIFICATE-----
....Cert hash
-----END CERTIFICATE-----
</ca>
key-direction 1
<tls-auth>
#
# 2048 bit OpenVPN static key
#
-----BEGIN OpenVPN Static key V1-----
....Cert key hash
-----END OpenVPN Static key V1-----
</tls-auth>
I've sanitised the Cert hash, key and servername and ip above
This is the VPN container logs
The use of ROUTE or -r may no longer be needed, try it without!,
The use of ROUTE or -r may no longer be needed, try it without!,
Dump terminated,
+ exec sg vpn -c 'openvpn --cd /vpn --config /vpn/vpn.conf --script-security 2 --redirect-gateway def1 --up /etc/openvpn/up.sh --down /etc/openvpn/down.sh --route-up '\''/bin/sh -c " iptables -A OUTPUT -d 127.0.0.11 -j ACCEPT"'\'' --route-pre-down '\''/bin/sh -c " iptables -D OUTPUT -d 127.0.0.11 -j ACCEPT"'\'' ',
Thu Dec 31 05:26:27 2020 WARNING: file '/vpn/pass' is group or others accessible,
Thu Dec 31 05:26:27 2020 OpenVPN 2.4.9 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Apr 20 2020,
Thu Dec 31 05:26:27 2020 library versions: OpenSSL 1.1.1g 21 Apr 2020, LZO 2.10,
Thu Dec 31 05:26:27 2020 WARNING: --ping should normally be used with --ping-restart or --ping-exit,
Thu Dec 31 05:26:27 2020 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts,
Thu Dec 31 05:26:27 2020 NOTE: --fast-io is disabled since we are not using UDP,
Thu Dec 31 05:26:27 2020 Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication,
Thu Dec 31 05:26:27 2020 Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication,
Thu Dec 31 05:26:27 2020 TCP/UDP: Preserving recently used remote address: [AF_INET]NORDVPNSERVERIP:443,
Thu Dec 31 05:26:27 2020 Socket Buffers: R=[131072->131072] S=[16384->16384],
Thu Dec 31 05:26:27 2020 Attempting to establish TCP connection with [AF_INET]NORDVPNSERVERIP:443 [nonblock],
Thu Dec 31 05:26:28 2020 TCP connection established with [AF_INET]NORDVPNSERVERIP:443,
Thu Dec 31 05:26:28 2020 TCP_CLIENT link local: (not bound),
Thu Dec 31 05:26:28 2020 TCP_CLIENT link remote: [AF_INET]NORDVPNSERVERIP:443,
Thu Dec 31 05:26:28 2020 TLS: Initial packet from [AF_INET]NORDVPNSERVERIP:443, sid=26a605b9 fea347d3,
Thu Dec 31 05:26:29 2020 VERIFY OK: depth=2, C=PA, O=NordVPN, CN=NordVPN Root CA,
Thu Dec 31 05:26:29 2020 VERIFY OK: depth=1, C=PA, O=NordVPN, CN=NordVPN CA5,
Thu Dec 31 05:26:29 2020 VERIFY KU OK,
Thu Dec 31 05:26:29 2020 Validating certificate extended key usage,
Thu Dec 31 05:26:29 2020 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication,
Thu Dec 31 05:26:29 2020 VERIFY EKU OK,
Thu Dec 31 05:26:29 2020 VERIFY OK: depth=0, CN=se457.nordvpn.com,
Thu Dec 31 05:26:30 2020 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, 4096 bit RSA,
Thu Dec 31 05:26:30 2020 [se457.nordvpn.com] Peer Connection Initiated with [AF_INET]NORDVPNSERVERIP:443,
Thu Dec 31 05:26:31 2020 SENT CONTROL [se457.nordvpn.com]: 'PUSH_REQUEST' (status=1),
Thu Dec 31 05:26:32 2020 PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,dhcp-option DNS 103.86.96.100,dhcp-option DNS 103.86.99.100,sndbuf 524288,rcvbuf 524288,explicit-exit-notify,comp-lzo no,route-gateway 10.7.3.1,topology subnet,ping 60,ping-restart 180,ifconfig 10.7.3.2 255.255.255.0,peer-id 0,cipher AES-256-GCM',
Thu Dec 31 05:26:32 2020 OPTIONS IMPORT: timers and/or timeouts modified,
Thu Dec 31 05:26:32 2020 OPTIONS IMPORT: --explicit-exit-notify can only be used with --proto udp,
Thu Dec 31 05:26:32 2020 OPTIONS IMPORT: compression parms modified,
Thu Dec 31 05:26:32 2020 OPTIONS IMPORT: --sndbuf/--rcvbuf options modified,
Thu Dec 31 05:26:32 2020 Socket Buffers: R=[131072->425984] S=[87040->425984],
Thu Dec 31 05:26:32 2020 OPTIONS IMPORT: --ifconfig/up options modified,
Thu Dec 31 05:26:32 2020 OPTIONS IMPORT: route options modified,
Thu Dec 31 05:26:32 2020 OPTIONS IMPORT: route-related options modified,
Thu Dec 31 05:26:32 2020 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified,
Thu Dec 31 05:26:32 2020 OPTIONS IMPORT: peer-id set,
Thu Dec 31 05:26:32 2020 OPTIONS IMPORT: adjusting link_mtu to 1659,
Thu Dec 31 05:26:32 2020 OPTIONS IMPORT: data channel crypto options modified,
Thu Dec 31 05:26:32 2020 Data Channel: using negotiated cipher 'AES-256-GCM',
Thu Dec 31 05:26:32 2020 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key,
Thu Dec 31 05:26:32 2020 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key,
Thu Dec 31 05:26:32 2020 ROUTE_GATEWAY 172.18.0.1/255.255.0.0 IFACE=eth0 HWADDR=02:42:ac:12:00:02,
Thu Dec 31 05:26:32 2020 TUN/TAP device tun0 opened,
Thu Dec 31 05:26:32 2020 TUN/TAP TX queue length set to 100,
Thu Dec 31 05:26:32 2020 /sbin/ip link set dev tun0 up mtu 1500,
Thu Dec 31 05:26:32 2020 /sbin/ip addr add dev tun0 10.7.3.2/24 broadcast 10.7.3.255,
Thu Dec 31 05:26:32 2020 /etc/openvpn/up.sh tun0 1500 1587 10.7.3.2 255.255.255.0 init,
Thu Dec 31 05:26:32 2020 Initialization Sequence Completed,
Thu Dec 31 06:26:31 2020 VERIFY OK: depth=2, C=PA, O=NordVPN, CN=NordVPN Root CA,
Thu Dec 31 06:26:31 2020 VERIFY OK: depth=1, C=PA, O=NordVPN, CN=NordVPN CA5,
Thu Dec 31 06:26:31 2020 VERIFY KU OK,
Thu Dec 31 06:26:31 2020 Validating certificate extended key usage,
Thu Dec 31 06:26:31 2020 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication,
Thu Dec 31 06:26:31 2020 VERIFY EKU OK,
Thu Dec 31 06:26:31 2020 VERIFY OK: depth=0, CN=se457.nordvpn.com,
Thu Dec 31 06:26:32 2020 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key,
Thu Dec 31 06:26:32 2020 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key,
Thu Dec 31 06:26:32 2020 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, 4096 bit RSA,
Thu Dec 31 07:26:30 2020 TLS: tls_process: killed expiring key,
Thu Dec 31 07:26:36 2020 VERIFY OK: depth=2, C=PA, O=NordVPN, CN=NordVPN Root CA,
Thu Dec 31 07:26:36 2020 VERIFY OK: depth=1, C=PA, O=NordVPN, CN=NordVPN CA5,
Thu Dec 31 07:26:36 2020 VERIFY KU OK,
Thu Dec 31 07:26:36 2020 Validating certificate extended key usage,
Thu Dec 31 07:26:36 2020 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication,
Thu Dec 31 07:26:36 2020 VERIFY EKU OK,
Thu Dec 31 07:26:36 2020 VERIFY OK: depth=0, CN=se457.nordvpn.com,
Thu Dec 31 07:26:37 2020 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key,
Thu Dec 31 07:26:37 2020 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key,
Thu Dec 31 07:26:37 2020 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, 4096 bit RSA,
I've sanitised the Nord vpn server IP above but left everything else as is
For testing I am running
sudo docker exec -t -i vpn /bin/bash
curl ipinfo.io
This should return the VPN's details not the details I get through my own account
It currently times out after a minute with no response
curl: (28) Failed to connect to ipinfo.io port 80: Operation timed out
I've also tried the same through transmissions container and get the same time out
I think the connection is fine but its not being exposed to itself let alone others
I've tried as configured as
volumes:
- /dev/net:/dev/net:z # tun device
I've also tried
devices:
- /dev/net/tun
both seem to fail the same
I added the timezone as the logs had the incorrect time against them, no difference
I've tried it removing the environment section in the docker-compose file for the vpn service, no difference
I've tried without -d in the command for using the vpn dns servers, no difference
I've tried without -f "" in the command for using the firewall, no difference
Does anyone have any ideas? am i testing it wrong?
I had a split tunnelling connection previously which I believe I've fully removed now multiple reboots renaming all the files, removing the old up and down scripts that the vpn config used to call
First of, thanks for a great sets of tools.
Everything is working perfect, except for one thing:
All data is being duplicated. I.e. - When a TV episode has been downloaded it is then COPIED to the correct folder. But Deluge still has the exact same copy of the file, thus taking up twice the disk space.
Any way to get deluge to understand where the file is being MOVED instead of copied?
I would like to keep seeding all torrents (good for ratio) but I want then to be placed in the correct folder.
Doable??!
Thanks again!
First, I'd like to say thanks for organizing all of this! It will make my transition so much easier since I'd like to start using dockers when possible. But I was wondering if this kind of setup would be more or less resource heavy compared to just a single VM running all native versions of these programs.
The vpn configuration section is probably the most difficult part right now, and for people who don't use PIA there isn't much remedy for them.
I assume that the majority of the people following this guide are using PIA if they are using a vpn, otherwise they would need external help from their individual VPN.
It might be valuable to support something like this: https://hub.docker.com/r/qmcgaw/private-internet-access. It is like the openvpn-client we are using right now, except it is made specifically for PIA, making it a lot easier to setup, and allowing it to be fully configured via the .env file.
Adding this as an option seems like a big benefit to PIA users, while not really affecting the amount of work other vpn users would have to do anyways. Thoughts?
I cant seem to find the data volume that my plex libraries are tied to on Sonarr, anyone know why?
I think adding nginx as a reverse proxy to all of the apps would be a pretty cool addition. I imagine it working it like this:
This solves a couple of issues:
Solves the issue of forgetting which port is which, mapping them instead to paths on a unified domain.
Allows for port-forwarding a single port to be able to access all of the programs in the home server. This would be a really cool addition, because normally you have to port forward every single port individually, and target them individually with a domain if you want to do that too.
Along with NGINX, although this is a less universal issue so it might be reserved for some kind of "advanced" configuration, it could be useful to add LetsEncrypt. What this will let the user do is port forward the single nginx domain and make it all https. I don't know if this would impact any of the apps, or specifically Plex web forwarding.
If supporting public-facing port forwarding is something that is desirable, there would also need to be a section on adding basic auth with nginx to secure it.
Where do I begin... First off, thank you for this write up. This is my first time using docker. I was able to get everything installed following this guide as far as I can tell. (Skipped Plex part, not really intrested in Plex at the moment.) I feel like there was a few things skipped over in it but I believe I was able to connect some dots and get everything installed with a few small changes here and there. As for everything working togeather that is where I am at. It doesn't seem to be working togeather. I can pull up everthing via the URl:port but nothing seems to be talking to eather other. So If you don't mine helping work throught this it would be helpful and a learning experiance.
So they system. It is a VM running Debina 9 (netinstall) with: Docker version 18.09.2, build 6247962, docker-compose version 1.24.0-rc1, build 0f3d4dda.
If we can lets please start with the VPN then move to deluge, jacket, sonarr,radarr, and so forth,
First the VPN: (PIA)
Sat Feb 23 15:09:20 2019 WARNING: file '/vpn/vpn.auth' is group or others accessible
Sat Feb 23 15:09:20 2019 OpenVPN 2.4.6 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Jul 8 2018
Sat Feb 23 15:09:20 2019 library versions: LibreSSL 2.7.4, LZO 2.10
Sat Feb 23 15:09:20 2019 TCP/UDP: Preserving recently used remote address: [AF_INET]46.166.137.237:1198
Sat Feb 23 15:09:20 2019 UDP link local: (not bound)
Sat Feb 23 15:09:20 2019 UDP link remote: [AF_INET]46.166.137.237:1198
Sat Feb 23 15:09:20 2019 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
Sat Feb 23 15:09:21 2019 [8cf34dba64052ef93bb53a1be95d0753] Peer Connection Initiated with [AF_INET]46.166.137.237:1198
Sat Feb 23 15:09:22 2019 TUN/TAP device tun0 opened
Sat Feb 23 15:09:22 2019 do_ifconfig, tt->did_ifconfig_ipv6_setup=0
Sat Feb 23 15:09:22 2019 /sbin/ip link set dev tun0 up mtu 1500
Sat Feb 23 15:09:22 2019 /sbin/ip addr add dev tun0 local 10.8.10.6 peer 10.8.10.5
Sat Feb 23 15:09:22 2019 Initialization Sequence Completed
Sat Feb 23 15:16:04 2019 Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #1223 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Sat Feb 23 15:16:07 2019 Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #2413 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Sat Feb 23 15:16:08 2019 Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #2716 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Sat Feb 23 15:09:20 2019 WARNING: file '/vpn/vpn.auth' is group or others accessible
Sat Feb 23 15:09:20 2019 OpenVPN 2.4.6 x86_64-alpine-linux-musl [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Jul 8 2018
Sat Feb 23 15:09:20 2019 library versions: LibreSSL 2.7.4, LZO 2.10
Sat Feb 23 15:09:20 2019 TCP/UDP: Preserving recently used remote address: [AF_INET]46.166.137.237:1198
Sat Feb 23 15:09:20 2019 UDP link local: (not bound)
Sat Feb 23 15:09:20 2019 UDP link remote: [AF_INET]46.166.137.237:1198
Sat Feb 23 15:09:20 2019 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
Sat Feb 23 15:09:21 2019 [8cf34dba64052ef93bb53a1be95d0753] Peer Connection Initiated with [AF_INET]46.166.137.237:1198
Sat Feb 23 15:09:22 2019 TUN/TAP device tun0 opened
Sat Feb 23 15:09:22 2019 do_ifconfig, tt->did_ifconfig_ipv6_setup=0
Sat Feb 23 15:09:22 2019 /sbin/ip link set dev tun0 up mtu 1500
Sat Feb 23 15:09:22 2019 /sbin/ip addr add dev tun0 local 10.8.10.6 peer 10.8.10.5
Sat Feb 23 15:09:22 2019 Initialization Sequence Completed
Sat Feb 23 15:16:04 2019 Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #1223 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Sat Feb 23 15:16:07 2019 Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #2413 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Sat Feb 23 15:16:08 2019 Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #2716 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.