kiralt / torrent-stream-server Goto Github PK

Create a read-only demo page (which allows only watching predefined torrents). It would be great if users could easily check how does it look before installing.

Dependencies:

• Ability to disable adding new torrents. It would be useful only if admins (admin will require login with password) could create new torrents.
• #267

Maybe it would be possible to automatically deploy to Heroku if we disable seeding.

-----> Building on the Heroku-20 stack
-----> Determining which buildpack to use for this app
-----> Node.js app detected

-----> Creating runtime environment

   NPM_CONFIG_LOGLEVEL=error
   NODE_VERBOSE=false
   NODE_ENV=production
   NODE_MODULES_CACHE=true

-----> Installing binaries
engines.node (package.json): >=12.x
engines.npm (package.json): unspecified (use default)

   Resolving node version >=12.x...
   Downloading and installing node 16.4.1...
   Using default npm version: 7.18.1

-----> Installing dependencies
Installing node modules

   > [email protected] postinstall
   > [ ! -d frontend/src ] || npm --prefix frontend install
   
   
   added 2052 packages, and audited 2053 packages in 1m
   
   141 packages are looking for funding
     run `npm fund` for details
   
   30 vulnerabilities (4 low, 12 moderate, 14 high)
   
   To address issues that do not require attention, run:
     npm audit fix
   
   To address all issues (including breaking changes), run:
     npm audit fix --force
   
   Run `npm audit` for details.
   
   added 1431 packages, and audited 1432 packages in 2m
   
   143 packages are looking for funding
     run `npm fund` for details
   
   4 vulnerabilities (2 moderate, 2 high)
   
   To address all issues, run:
     npm audit fix
   
   Run `npm audit` for details.

-----> Build
Running build

   > [email protected] build
   > npm run build-backend && npm run build-frontend
   
   
   > [email protected] build-backend
   > tsc
   
   
   > [email protected] build-frontend
   > npm run --prefix frontend build
   
   
   > [email protected] build
   > REACT_APP_VERSION=$npm_package_version SKIP_PREFLIGHT_CHECK=true react-scripts build
   
   Creating an optimized production build...
   Failed to compile.
   
   /tmp/build_c1979e7a/frontend/src/components/browse.tsx
   TypeScript error in /tmp/build_c1979e7a/frontend/src/components/browse.tsx(19,20):
   Could not find a declaration file for module 'react-select'. '/tmp/build_c1979e7a/frontend/node_modules/react-select/dist/react-select.cjs.js' implicitly has an 'any' type.
     Try `npm i --save-dev @types/react-select` if it exists or add a new declaration (.d.ts) file containing `declare module 'react-select';`  TS7016
   
       17 | import { sortBy } from 'common-stuff'
       18 | import { useGlobal } from 'reactn'
     > 19 | import Select from 'react-select'
          |                    ^
       20 | 
       21 | import { BrowseApi, ProviderTorrentModel } from '../helpers/client'
       22 | import { useSearchParam, useSearchParamsHandler } from '../helpers/hooks'

-----> Build failed

   We're sorry this build is failing! You can troubleshoot common issues here:
   https://devcenter.heroku.com/articles/troubleshooting-node-deploys
   
   Some possible problems:
   
   - Dangerous semver range (>) in engines.node
     https://devcenter.heroku.com/articles/nodejs-support#specifying-a-node-js-version
   
   Love,
   Heroku

! Push rejected, failed to compile Node.js app.
! Push failed

I wanted to try & tinker with this software.

Since there is Dockerfile and Docker package, I went for it.

This is what happened:

$ docker run --rm -it ghcr.io/kiralt/torrent-stream-server:1.2.0
Unable to find image 'ghcr.io/kiralt/torrent-stream-server:1.2.0' locally
docker: Error response from daemon: Get https://ghcr.io/v2/kiralt/torrent-stream-server/manifests/1.2.0: denied.

I can pull from Docker Hub without problems.

Due to the issue in webtorrent/webtorrent#2194, magnet links do not work when using 0 upload speed.

Temporary there is the default small speed:

But there could be an issue when multiple torrents are added at the same time.

i published this project on heroku... it worked fine for one day but then suddenly my site got suuspended.

Whenever I download a file, it ends up being magnet... instead of the actual file name.

my apologies KiraLT, i am to much on here :) i am having this small issue with a forward slash being added.

i have my domain setup for easy access from terminal like this

https://example.com/5E6C856FB0BB176BB18AA9EC6B11E9DC3BD4ADBE --- > would redirect to https://example.herokuapp.com/stream?torrent= and append the hash

now the issue, it correctly adds hash but also appends a forward slash

example.herokuapp.com/stream?torrent=/5E6C856FB0BB176BB18AA9EC6B11E9DC3BD4ADBE

{"error":"Cannot parse torrent: Invalid torrent identifier, link: /5E6C856FB0BB176BB18AA9EC6B11E9DC3BD4ADBE"}

could torrent stream server be used like this /stream/torrent/hash

Make Donate link with Icon for Our Dev Improvement.
to Paypal or UPI.

hi , i am trying to deploy to heroku, but it seems to show Application error, port is set on 80, if you can check what the problem is, because i have no idea :(

please check here to get some idea about methods to add to TSS.

it would be nice to have, Selectors
Selectors that can be a Parameters or an Endpoint

example

or endpoint

something like live torrent stream api.

very recently forked updated repo and deployed to herokuapp, is the api the same ?

in this version with Chromecast this provides the error {"error":"Unknown query parameter 'dn'"}
https://demo.herokuapp.com/stream?torrent=

in old repo without chromecast support, api worked
https://demo.herokuapp.com/stream?torrent=

I'm going to delete the herokuapp again and fork your repo fresh again . i might have done something wrong.

Hi Kira, have you considered supporting file selection?

At the moment login password is stored in the browser memory. So after each reloads you need to enter the password.

We can add the "Remember me" checkbox to the UI login page. We probably also need a logout button.

As you know, currently the downloading client code is based on webtorrent, as such private torrent trackers can limit which clients could be used, if the private tracker has the webtorrent user-agent WebTorrent/0.102.4 (https://webtorrent.io) blacklisted, the user will not be able to download from the tracker, a work around has been suggested here

it might be a good idea to specify a different user-agent all the trackers allow, such as libtorrent/1.2.2.0

just my suggestions.

I've been using Okteto, and it's working great, but what happens when the Torrents Space reaches the max amount?
Also, why does it show 79.68 / 96.75 for disk space if I've not downloaded something?

Add torlook.info tracker and add authorization on server, user and password for vps.

please check here to get some idea about methods to add to TSS.

it would be nice to have, Selectors
Selectors that can be a Parameters or an Endpoint

example

or endpoint

something like live torrent stream api.

want to run this on ssl port 443 how can i change port from 3000 to 443

When opening a stream on IOS, I get the following error:

The same page/stream works on OSX and Android.

Whenever the system, user or whatever external agent wants to stop a process, the SIGTERM signal is used. SIGTERM is meant to be a clean shutdown signal, which the app can catch (trap) and do its shutdown cleanly.

In Docker, this is done issuing a docker stop command to the container, which will cause the Docker daemon sending the SIGTERM signal to the container's pid 1 process. After a configurable timeout, Docker will send a SIGKILL signal if the container has not stopped.

Since previously TSS was started using a CMD statement, that implied a sh -c ENTRYPOINT. The sh process got the TERM signal and exited, forcefully killing its children.
As of now, node binary runs as pid 1, so it proxy the TERM signal to the running app, which can process it to shutdown cleanly. On most programming languages, you have to explicitly set a handler function to process it (trap the signal). Currently, TSS does not process it and Docker kills it after timeout (by default 10s).

More info: https://www.ctl.io/developers/blog/post/gracefully-stopping-docker-containers/

Use this NPM https://www.npmjs.com/package/torrent-search-api for torrent providers.
or else use core level or some other library.

found this today, might be very useful for nyaa, as many of the mkv are with subtitles that can not be played within html5. but with this, it might be possible to extract and display.

https://www.npmjs.com/package/matroska-subtitles

KiraLT on version v1.8.1 , how can I stream without using swagger to format my URL or go through the web interface, normally I am just typing in a terminal window to stream with the old version of torrent-stream-server , I'm now trying to stream with the V1.8.1 but it's not working as before.

Unformatted magnet does not work in the terminal

This example will not work in the browser or the terminal:

wget "http://127.0.0.1:3000/stream/magnet:?xt=urn:btih:AW33KWPZYIUTNTMU3EDSS2GAGIIXRA7W&dn=%5BSubsPlease%5D%20Re-Main%20-%2003%20%28720p%29%20%5B9EB549CE%5D.mkv&xl=639042804&tr=http%3A%2F%2Fnyaa.tracker.wf%3A7777%2Fannounce&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2F9.rarbg.to%3A2710%2Fannounce&tr=udp%3A%2F%2F9.rarbg.me%3A2710%2Fannounce&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.internetwarriors.net%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.cyberia.is%3A6969%2Fannounce&tr=udp%3A%2F%2Fexodus.desync.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker3.itzmx.com%3A6961%2Fannounce&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451%2Fannounce&tr=udp%3A%2F%2Ftracker.tiny-vps.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fretracker.lanta-net.ru%3A2710%2Fannounce&tr=http%3A%2F%2Fopen.acgnxtracker.com%3A80%2Fannounce&tr=wss%3A%2F%2Ftracker.openwebtorrent.com"

Formatted URL from swagger which works in terminal and browser:

wget "http://127.0.0.1:3000/stream/magnet%3A%3Fxt%3Durn%3Abtih%3AAW33KWPZYIUTNTMU3EDSS2GAGIIXRA7W%26dn%3D%255BSubsPlease%255D%2520Re-Main%2520-%252003%2520%2528720p%2529%2520%255B9EB549CE%255D.mkv%26xl%3D639042804%26tr%3Dhttp%253A%252F%252Fnyaa.tracker.wf%253A7777%252Fannounce%26tr%3Dudp%253A%252F%252Ftracker.coppersurfer.tk%253A6969%252Fannounce%26tr%3Dudp%253A%252F%252Ftracker.opentrackr.org%253A1337%252Fannounce%26tr%3Dudp%253A%252F%252F9.rarbg.to%253A2710%252Fannounce%26tr%3Dudp%253A%252F%252F9.rarbg.me%253A2710%252Fannounce%26tr%3Dudp%253A%252F%252Ftracker.leechers-paradise.org%253A6969%252Fannounce%26tr%3Dudp%253A%252F%252Ftracker.internetwarriors.net%253A1337%252Fannounce%26tr%3Dudp%253A%252F%252Ftracker.cyberia.is%253A6969%252Fannounce%26tr%3Dudp%253A%252F%252Fexodus.desync.com%253A6969%252Fannounce%26tr%3Dudp%253A%252F%252Ftracker3.itzmx.com%253A6961%252Fannounce%26tr%3Dudp%253A%252F%252Ftracker.torrent.eu.org%253A451%252Fannounce%26tr%3Dudp%253A%252F%252Ftracker.tiny-vps.com%253A6969%252Fannounce%26tr%3Dudp%253A%252F%252Fretracker.lanta-net.ru%253A2710%252Fannounce%26tr%3Dhttp%253A%252F%252Fopen.acgnxtracker.com%253A80%252Fannounce%26tr%3Dwss%253A%252F%252Ftracker.openwebtorrent.com?fileType=video"

Unformatted URL

Error reported in url {"error":"Unknown query parameter 'xt'"}:

wget "http://127.0.0.1:3000/stream/magnet:?xt=urn:btih:AW33KWPZYIUTNTMU3EDSS2GAGIIXRA7W&dn=%5BSubsPlease%5D%20Re-Main%20-%2003%20%28720p%29%20%5B9EB549CE%5D.mkv&xl=639042804&tr=http%3A%2F%2Fnyaa.tracker.wf%3A7777%2Fannounce&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2F9.rarbg.to%3A2710%2Fannounce&tr=udp%3A%2F%2F9.rarbg.me%3A2710%2Fannounce&tr=udp%3A%2F%2Ftracker.leechers-paradise.org%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.internetwarriors.net%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.cyberia.is%3A6969%2Fannounce&tr=udp%3A%2F%2Fexodus.desync.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker3.itzmx.com%3A6961%2Fannounce&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451%2Fannounce&tr=udp%3A%2F%2Ftracker.tiny-vps.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fretracker.lanta-net.ru%3A2710%2Fannounce&tr=http%3A%2F%2Fopen.acgnxtracker.com%3A80%2Fannounce&tr=wss%3A%2F%2Ftracker.openwebtorrent.com

Any advice, I am still with the old torrent-stream-server, I do not really visit the WebUI to do streaming.

Create the all media view like images, audios, videos.
Also apart from media files make download button. eg: Application/Game files

Site will be streaming the media content, as well as download options for other files

While setting up TSS behind a reverse proxy, I've set NODE_ENV to development for troubleshooting. I've seen this on logs, even I've set HOST to 0.0.0.0:

Is this message true or the string just needs to include listen address?

While tinkering with code, errors print [email protected], also.

$ node . serve --version
0.0.0

Please show me the this App version in footer or header like v1.5.2.
This must be helpful.

When i stop downloading the file, the torrent may stop downloading and be removed. I see that it is not the correct file and I will download another one, but the old one keeps downloading.

When searching 1337x under all categories the search returns invalid and only works when the specific category is selected this has occurred since v1.7.0 .

Libtorrent?

do you support these protocols?

Protocol obfuscation
DHT
uTP
PEX
Magnet links
IP Blocklists
Some IPv6
HTTP and UDP tracker clients

BEPs:

3: Basic BitTorrent protocol
5: DHT
6: Fast Extension (have all/none only)
7: IPv6 Tracker Extension
9: ut_metadata
10: Extension protocol
11: PEX
12: Multitracker metadata extension
15: UDP Tracker Protocol
20: Peer ID convention ("-GTnnnn-")
23: Tracker Returns Compact Peer Lists
27: Private torrents
29: uTorrent transport protocol
41: UDP Tracker Protocol Extensions
42: DHT Security extension
43: Read-only DHT Nodes

BEPs:

14: Local Peers Discovery
19: WebSeeds

Itd be really cool if you could disable the downloading/storing or torrents, that way this awesome project could work in docker containers with very limited space.

Thank you.

Heroku terms forbid p2p:
https://www.heroku.com/policy/notices-and-license
My account has been suspended (when trying another p2p software).
I've had to go thru the unblock process and learnt it along the way.
Either place a warning or remove the Heroku button completely (the latter, the safer, IMO).

It would be all awesome and cool function to add a button besides stream an download.

Although there seems to be a handful of Chromecast libs around, looks like it it would be doing some gluing to get it. However I can't tell for sure, and it might be easier said than done.

However, while researching about it I've found this code which might be useful (I can't judge it myself due to my poor JS): asapach/peerflix-server@master...stevetuk71:master
It's using the https://chromecast.wyrt.co.uk/ service, which looks like way easier to implement.

I was wondering if the used framework has any feature which allows localization to other languages and how to add one.

is anybody experiencing sudden cut offs while streaming ? I'm not sure if this is something with herokuapp limits or the web torrent engine. the issue basically starts within few minutes after starting viewing the media.

As discuss in #172, we need a reliable free-tier deployment service (or recommended paid service) that allows peer-to-peer.
Heroku and Octeko are great services, but they block p2p services #32.

List of providers which has a free tier: https://free-for.dev/#/?id=paas.

Depending on the torrent you use, if it has a lot of seeders, download speed can consume all resources (in my cases download speed reaches 20 - 30 MB/s).

The idea is to limit download/uploads speed. The limit can be global (for all active torrents) or for each individual torrent/file. The second option would be better, because for streaming 1MB/s it's enough.

Related:

• Limit download/upload speeds (throttling) .
• Add speed limit (pull request)

Now that #7 is fixed and I can pull the Docker image, I've been able to try it out.
Overall, it works wonders and it's a great app!

However, when I run the image:

$ docker run --rm -it -p 3000:3000 ghcr.io/kiralt/torrent-stream-server:1.2.0
{"message":"Starting app on http://127.0.0.1:3000","level":"info"}
{"message":"Preparing torrent directory - /tmp/torrent-stream-server","level":"info"}

Looking at the log, even before opening the web UI, the message with 127.0.0.1 as listening address worried me. I wondered that, in such case, the Docker container won't be accessible from the outside, even with the appropriate port mapping (-p 3000:3000).

However, I could connect to the service on my server over LAN at 192.169.1.100 and everything worked like a charm.

I wanted to investigate further, and installed the netstat tool:

$ docker exec -it torrent-stream-server bash
Ign:1 http://deb.debian.org/debian stretch InRelease
Get:2 http://security.debian.org/debian-security stretch/updates InRelease [53.0 kB]
Get:3 http://deb.debian.org/debian stretch-updates InRelease [93.6 kB]
Get:4 http://deb.debian.org/debian stretch Release [118 kB]
Get:5 http://deb.debian.org/debian stretch Release.gpg [2410 B]
Get:6 http://security.debian.org/debian-security stretch/updates/main amd64 Packages [604 kB]
Get:7 http://deb.debian.org/debian stretch-updates/main amd64 Packages [2596 B]
Get:8 http://deb.debian.org/debian stretch/main amd64 Packages [7080 kB]
Fetched 7953 kB in 4s (1937 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
50 packages can be upgraded. Run 'apt list --upgradable' to see them.
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  net-tools
0 upgraded, 1 newly installed, 0 to remove and 50 not upgraded.
Need to get 248 kB of archives.
After this operation, 963 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stretch/main amd64 net-tools amd64 1.60+git20161116.90da8a0-1 [248 kB]
Fetched 248 kB in 0s (1027 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package net-tools.
(Reading database ... 29937 files and directories currently installed.)
Preparing to unpack .../net-tools_1.60+git20161116.90da8a0-1_amd64.deb ...
Unpacking net-tools (1.60+git20161116.90da8a0-1) ...
Setting up net-tools (1.60+git20161116.90da8a0-1) ...

root@9cda3d8ef4c5:/usr/app# netstat -putanl
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp6       0      0 :::3000                 :::*                    LISTEN      8/node
tcp6       0      0 172.17.0.2:3000         192.168.1.101:58560     ESTABLISHED 8/node
tcp6       0      0 172.17.0.2:3000         192.168.1.101:58559     ESTABLISHED 8/node

I guess I'm able to connect over IPv6, since the only listening address is :::3000, as reported by netstat.

Looks like to me that the 127.0.0.1 listening address advertised is not the real.

I've had a quick look at the code and the listen address is just a hardcoded, text string. Also, it can't be configured otherwise, as opposed to the listening port.

Overall, it's somewhat confusing as a first-time user.

So, my findings are:

• With port mappings it works, and it is OK to listen on :::3000, however...
• I'm not sure if it would work in an IPv4-only scenario (no 0.0.0.0:3000 LISTEN entry).
• If used in host networking mode (--net=host) it's not clear to me which listen address would use in a predictable manner. Also, you might actually want to force it to be 127.0.0.1 in such scenario.
• I could not set explicitly a listen address from configuration.

My proposal would be:

• Get the real listen address from the actual app object property, so it does not cause confusion or, worse yet, gives a false sense of not being exposed.
• Make listen address configurable, as done with the port (both via config and env var).

Hope this helps.

Please try to put some web template apart from Bootstrap basic UI.

Check this Open Source UI

https://ant.design/components/overview

I would be great to have a button to download everything at once (via zip).

Please apply advanced web video player like below open source.

https://video-react.js.org

I've been using torrent-stream-server with Okteto and it's been working so great! Although, it would be great to have an upload to Google Drive button. Thanks!

As I've started to tinker with the code to do an easy frontend test, I could not even make the code run.
In my case, I rely exclusively on Docker to develop (hack/docker build/test/repeat). I don't have NodeJS toolchain installed on my OS, so I can work on any version (and even on any language) without breaking my PC.

I tried some easy frontend changes and it kept me puzzled for a while until I've finally realized that (as stated in the readme for normal development), the Docker build also does not rebuild frontend build dir.
I've done my best and tried to fix it by just following the frontend readme documentation and some googling, but it's way off my abilities (I'm neither a NodeJS dev nor frontend guy).

I think that Docker build should also rebuild frontend files always.
So, after banging my head for a few hours, I'm filing this issue.

Also, I guess that the build dir should not be commited in the repo.
I think it's preferable the run to fail loud out of the box instead of silently working with an outdated frontend. Also applies for any files that should be created by the build/CI process.

In many cases, you don't know which torrent provider has the best torrent you are searching for (the best resolution, most seeders, etc...). So it would be very convenient to search in all torrent providers at the same time. It will be more relevant when more providers will be added #43.

Challenges:

• Category selection - different providers have different category structures. One way would be to disable category selection when multiple providers are selected. Or maybe it is possible to create some kind of mapping between providers with general categories (like movies, TV series...).
• Errors - it is possible that some providers are down. When some providers are not available, API should still return partial result + error list.
• Search results order - when searching by multiple providers, search results should be reranked by relevance (text match + seeders, upload date, downloads).

Getting familiar with the code, I've seen that, although parse-torrent only gets the infoHash, torrent-stream still gets the full magnet URI and can take advantage of trackers embedded in the URI.

Since trackers come and go and some magnet URIs don't even have trackers built-in, adding some well-known (or even private) trackers would speed up stream bootstrap and discovery.

This could be accomplished at add time by just automatically appending a fixed string containing some bootstraping, well-known trackers encoded in a URI parameter string.

As you can't anticipate which URIs will need it and which trackers each user would find best, it would be a nice addition to let the user configure their own string, although it can default to some popular public trackers as example (commented out, for instance).

Notice I'm requesting a fixed generic string, which will initially be of use for adding trackers, but it might be used for other purposes, as long as it is kept generic enough. If torrent-stream lib were to support it, it could also be used to bootstrap the torrent from a well-known peer by adding a x.pe URI parameter, for instance (see BEP 9).

Its just an enhancement request. If it's possible, try to build it in the container stack with ubuntu layer before node layer so as to limit the ban if it happens.
You might have to create heroku.yml file in it.

Also, if streaming can be through ffmpeg so multiple quality support can be added.

i wasn't able to stream from terminal using the long magnet link within windows cmd

this will not work in cmd
mpv http://127.0.0.1:3000/stream/magnet link

but this will perfectly work.
mpv http://127.0.0.1:3000/stream?torrent=magnet link

it could be interesting when clicking play, the system could transcode with ffmpeg to formats compatible with the player. Some formats are not visible on the web.

when i use the command it works but as soon as i leave my ssh it stops wrking
i am begginer in npm and need this for a project please help