Coder Social home page Coder Social logo

webtorrent / ut_metadata Goto Github PK

View Code? Open in Web Editor NEW
111.0 11.0 34.0 194 KB

BitTorrent Extension for Peers to Send Metadata Files (BEP 9)

Home Page: https://webtorrent.io

License: MIT License

JavaScript 100.00%
webtorrent bittorrent metadata peer javascript nodejs browser magnet

ut_metadata's Introduction


WebTorrent
WebTorrent

The streaming torrent client. For node.js and the web.

discord ci npm version npm downloads Standard - JavaScript Style Guide

Sponsored by    Socket - JavaScript open source supply chain security    Wormhole

WebTorrent is a streaming torrent client for node.js and the browser. YEP, THAT'S RIGHT. THE BROWSER. It's written completely in JavaScript – the language of the web – so the same code works in both runtimes.

In node.js, this module is a simple torrent client, using TCP and UDP to talk to other torrent clients.

In the browser, WebTorrent uses WebRTC (data channels) for peer-to-peer transport. It can be used without browser plugins, extensions, or installations. It's Just JavaScript™. Note: WebTorrent does not support UDP/TCP peers in browser.

Simply include the webtorrent.min.js script on your page to start fetching files over WebRTC using the BitTorrent protocol, or import WebTorrent from 'webtorrent' with browserify or webpack. See demo apps and code examples below.

jsdelivr download count

To make BitTorrent work over WebRTC (which is the only P2P transport that works on the web) we made some protocol changes. Therefore, a browser-based WebTorrent client or "web peer" can only connect to other clients that support WebTorrent/WebRTC.

To seed files to web peers, use a client that supports WebTorrent, e.g. WebTorrent Desktop, a desktop client with a familiar UI that can connect to web peers, webtorrent-hybrid, a command line program, or Instant.io, a website. Established torrent clients like Vuze have already added WebTorrent support so they can connect to both normal and web peers. We hope other clients will follow.

Network

Features

  • Torrent client for node.js & the browser (same npm package!)
  • Insanely fast
  • Download multiple torrents simultaneously, efficiently
  • Pure Javascript (no native dependencies)
  • Exposes files as streams
    • Fetches pieces from the network on-demand so seeking is supported (even before torrent is finished)
    • Seamlessly switches between sequential and rarest-first piece selection strategy
  • Supports advanced torrent client features
  • Comprehensive test suite (runs completely offline, so it's reliable and fast)
  • Check all the supported BEPs here

Browser/WebRTC environment features

  • WebRTC data channels for lightweight peer-to-peer communication with no plugins
  • No silos. WebTorrent is a P2P network for the entire web. WebTorrent clients running on one domain can connect to clients on any other domain.
  • Stream video torrents into a <video> tag (webm, mkv, mp4, ogv, mov, etc (AV1, H264, HEVC*, VP8, VP9, AAC, FLAC, MP3, OPUS, Vorbis, etc))
  • Supports Chrome, Firefox, Opera and Safari.

Install

To install WebTorrent for use in node or the browser with import WebTorrent from 'webtorrent', run:

npm install webtorrent

To install a webtorrent command line program, run:

npm install webtorrent-cli -g

To install a WebTorrent desktop application for Mac, Windows, or Linux, see WebTorrent Desktop.

Ways to help

Who is using WebTorrent today?

Lots of folks!

WebTorrent API Documentation

Read the full API Documentation.

Usage

WebTorrent is the first BitTorrent client that works in the browser, using open web standards (no plugins, just HTML5 and WebRTC)! It's easy to get started!

In the browser

Downloading a file is simple:
import WebTorrent from 'webtorrent'

const client = new WebTorrent()
const magnetURI = '...'

client.add(magnetURI, torrent => {
  // Got torrent metadata!
  console.log('Client is downloading:', torrent.infoHash)

  for (const file of torrent.files) {
    document.body.append(file.name)
  }
})
Seeding a file is simple, too:
import dragDrop from 'drag-drop'
import WebTorrent from 'webtorrent'

const client = new WebTorrent()

// When user drops files on the browser, create a new torrent and start seeding it!
dragDrop('body', files => {
  client.seed(files, torrent => {
    console.log('Client is seeding:', torrent.infoHash)
  })
})

There are more examples in docs/get-started.md.

Browserify

WebTorrent works great with browserify, an npm package that lets you use node-style require() to organize your browser code and load modules installed by npm (as seen in the previous examples).

Webpack

WebTorrent also works with webpack, another module bundler. However, webpack requires extra configuration which you can find in the webpack bundle config used by webtorrent.

Or, you can just use the pre-built version via import WebTorrent from 'webtorrent/dist/webtorrent.min.js' and skip the webpack configuration.

Script tag

WebTorrent is also available as a standalone script (webtorrent.min.js) which exposes WebTorrent on the window object, so it can be used with just a script tag:

<script type='module'>
  import WebTorrent from 'webtorrent.min.js'
</script>

The WebTorrent script is also hosted on fast, reliable CDN infrastructure (Cloudflare and MaxCDN) for easy inclusion on your site:

<script type='module'>
  import WebTorrent from 'https://esm.sh/webtorrent'
</script>
Chrome App

If you want to use WebTorrent in a Chrome App, you can include the following script:

<script type='module'>
  import WebTorrent from 'webtorrent.chromeapp.js'
</script>

Be sure to enable the chrome.sockets.udp and chrome.sockets.tcp permissions!

In Node.js

WebTorrent also works in node.js, using the same npm package! It's mad science!

NOTE: To connect to "web peers" (browsers) in addition to normal BitTorrent peers, use webtorrent-hybrid which includes WebRTC support for node.

As a command line app

WebTorrent is also available as a command line app. Here's how to use it:

$ npm install webtorrent-cli -g
$ webtorrent --help

To download a torrent:

$ webtorrent magnet_uri

To stream a torrent to a device like AirPlay or Chromecast, just pass a flag:

$ webtorrent magnet_uri --airplay

There are many supported streaming options:

--airplay               Apple TV
--chromecast            Chromecast
--mplayer               MPlayer
--mpv                   MPV
--omx [jack]            omx [default: hdmi]
--vlc                   VLC
--xbmc                  XBMC
--stdout                standard out [implies --quiet]

In addition to magnet uris, WebTorrent supports many ways to specify a torrent.

Talks about WebTorrent

Modules

Most of the active development is happening inside of small npm packages which are used by WebTorrent.

The Node Way™

"When applications are done well, they are just the really application-specific, brackish residue that can't be so easily abstracted away. All the nice, reusable components sublimate away onto github and npm where everybody can collaborate to advance the commons." — substack from "how I write modules"

node.js is shiny

Modules

These are the main modules that make up WebTorrent:

module tests version description
webtorrent torrent client (this module)
bittorrent-dht distributed hash table client
bittorrent-peerid identify client name/version
bittorrent-protocol bittorrent protocol stream
bittorrent-tracker bittorrent tracker server/client
bittorrent-lsd bittorrent local service discovery
create-torrent create .torrent files
magnet-uri parse magnet uris
parse-torrent parse torrent identifiers
torrent-discovery find peers via dht, tracker, and lsd
ut_metadata metadata for magnet uris (protocol extension)
ut_pex peer discovery (protocol extension)

Enable debug logs

In node, enable debug logs by setting the DEBUG environment variable to the name of the module you want to debug (e.g. bittorrent-protocol, or * to print all logs).

DEBUG=* webtorrent

In the browser, enable debug logs by running this in the developer console:

localStorage.setItem('debug', '*')

Disable by running this:

localStorage.removeItem('debug')

License

MIT. Copyright (c) Feross Aboukhadijeh and WebTorrent, LLC.

ut_metadata's People

Contributors

bjornstar avatar diegorbaquero avatar feross avatar greenkeeper[bot] avatar greenkeeperio-bot avatar jimmywarting avatar renovate-bot avatar renovate[bot] avatar semantic-release-bot avatar thaunknown avatar transitive-bullshit avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ut_metadata's Issues

Merge data from multiple peers

This is a minor optimization.

Merge metadata from multiple peers to return metadata more quickly.

Need to be careful. If one peer consistently sends bad data, you'll get stuck retrying for a long time and have no idea which peer sent the bad data.

Here's how it will work: Assume peers are all good. Merge metadata from all peers together, but also keep metadata per-peer (like we're doing now). Merged metadata should finish fastest. If it matches sha1 hash, great, we're done! If not then just take metadata from first peer (like we're doing now).

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/workflows/ci.yml
  • actions/checkout v3
  • actions/setup-node v3
.github/workflows/release.yml
  • actions/checkout v3
  • actions/setup-node v3
  • actions/cache v3
npm
package.json
  • bencode ^4.0.0
  • bitfield ^4.0.0
  • debug ^4.2.0
  • uint8-util ^2.1.3
  • @webtorrent/semantic-release-config 1.0.10
  • bittorrent-protocol 4.1.11
  • brfs 2.0.2
  • semantic-release 21.1.2
  • standard *
  • tape 5.7.5
  • webtorrent-fixtures 2.0.2

  • Check this box to trigger a request for Renovate to run again on this repository

An in-range update of webtorrent-fixtures is breaking the build 🚨

Version 1.7.1 of webtorrent-fixtures was just published.

Branch Build failing 🚨
Dependency webtorrent-fixtures
Current Version 1.7.0
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

webtorrent-fixtures is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build could not complete due to an error (Details).

Commits

The new version differs by 4 commits.

  • 8dd86e7 1.7.1
  • d07e2d4 Merge pull request #5 from webtorrent/greenkeeper/brfs-2.0.0
  • 7ec91af readme
  • 62f289a fix(package): update brfs to version 2.0.0

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Action required: Greenkeeper could not be activated 🚨

🚨 You need to enable Continuous Integration on all branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because we are using your CI build statuses to figure out when to notify you about breaking changes.

Since we did not receive a CI status on the greenkeeper/initial branch, we assume that you still need to configure it.

If you have already set up a CI for this repository, you might need to check your configuration. Make sure it will run on all new branches. If you don’t want it to run on every branch, you can whitelist branches starting with greenkeeper/.

We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.

Once you have installed CI on this repository, you’ll need to re-trigger Greenkeeper’s initial Pull Request. To do this, please delete the greenkeeper/initial branch in this repository, and then remove and re-add this repository to the Greenkeeper integration’s white list on Github. You'll find this list on your repo or organization’s settings page, under Installed GitHub Apps.

An in-range update of standard is breaking the build 🚨

Version 11.0.0 of standard was just published.

Branch Build failing 🚨
Dependency standard
Current Version 10.0.3
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

standard is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • continuous-integration/travis-ci/push The Travis CI build failed Details

Commits

The new version differs by 50 commits.

There are 50 commits in total.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of tape is breaking the build 🚨

Version 4.9.1 of tape was just published.

Branch Build failing 🚨
Dependency tape
Current Version 4.9.0
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

tape is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

Commits

The new version differs by 10 commits.

  • 050b318 v4.9.1
  • 73232c0 [Dev Deps] update js-yaml
  • 8a2d29b [Deps] update has, for-each, resolve, object-inspect
  • c6f5313 [Tests] add eclint and eslint, to enforce a consistent style
  • 45788a5 [Dev Deps] update concat-stream
  • ec4a71d [fix] Fix bug in functionName regex during stack parsing
  • 7261ccc Merge pull request #433 from mcnuttandrew/add-trb
  • 6cbc53e Add tap-react-browser
  • 9d501ff [Dev Deps] use ~ for dev deps; update to latest nonbreaking
  • 24e0a8d Fix spelling of "parameterize"

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Support BEP 42

It may be worth adding support for BEP 42 to lay the groundwork for better security of the Bittorrent DHT.

An in-range update of brfs is breaking the build 🚨

The devDependency brfs was updated from 2.0.1 to 2.0.2.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

brfs is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build could not complete due to an error (Details).

Commits

The new version differs by 4 commits.

  • 660f6d8 2.0.2
  • eb26fec Merge pull request #93 from jagonzalr/pr/update/static-val-version
  • 434cc8b Update static-module from 3.0.0 to 3.0.2
  • 8fafcf5 Remove stray console.log

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of webtorrent-fixtures is breaking the build 🚨

Version 1.7.0 of webtorrent-fixtures was just published.

Branch Build failing 🚨
Dependency webtorrent-fixtures
Current Version 1.6.0
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

webtorrent-fixtures is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • continuous-integration/travis-ci/push: The Travis CI build could not complete due to an error (Details).

Commits

The new version differs by 3 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of tape is breaking the build 🚨

Version 4.9.0 of tape was just published.

Branch Build failing 🚨
Dependency tape
Current Version 4.8.0
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

tape is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • continuous-integration/travis-ci/push The Travis CI build failed Details

Commits

The new version differs by 27 commits.

  • ea6d91e v4.9.0
  • 6867840 [Deps] update object-inspect, resolve
  • 4919e40 [Tests] on node v9; use nvm install-latest-npm
  • f26375c Merge pull request #420 from inadarei/global-depth-env-var
  • 17276d7 [New] use process.env.NODE_TAPE_OBJECT_PRINT_DEPTH for the default object print depth.
  • 0e870c6 Merge pull request #408 from johnhenry/feature/on-failure
  • 00aa133 Add "onFinish" listener to test harness.
  • 0e68b2d [Dev Deps] update js-yaml
  • 10b7dcd [Fix] fix stack where actual is falsy
  • 13173a5 Merge pull request #402 from nhamer/stack_strip
  • f90e487 normalize path separators in stacks
  • b66f8f8 [Deps] update function-bind
  • cc69501 Merge pull request #387 from fongandrew/master
  • bf5a750 Handle spaces in path name for setting file, line no
  • 3c2087a Test name with spaces

There are 27 commits in total.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Don't recreate the class

V8 is unable to optimize this class if it's reconstructing it everytime.

ut_metadata/index.js

Lines 12 to 13 in b871aca

module.exports = metadata => {
class utMetadata extends EventEmitter {

  • The class should be moved to the top level.
  • A wire.ut_metadata instanceof check should work

Maybe should just get rid of the option to do: utMetadata(x) and instead use the wire.ut_metadata.setMetadata(x) method instead...

An in-range update of brfs is breaking the build 🚨

☝️ Greenkeeper’s updated Terms of Service will come into effect on April 6th, 2018.

Version 1.6.0 of brfs was just published.

Branch Build failing 🚨
Dependency brfs
Current Version 1.5.0
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

brfs is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • continuous-integration/travis-ci/push The Travis CI build failed Details

Commits

The new version differs by 4 commits.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Could you refactory ut_metadata and bittorrent-protocol?

Hi, I am interested in your ut_metadata and bittorrent-protocol. I have been reading the two projects for many days, but I am very confused your code. It is very difficult to understand. Could you refactory it and make it easy to understand?

I am sorry the unreasonable question.

Neat idea: emit data as it's parsed

Each field in the metadata could be emitted as a separate event, as it arrives on the wire.

"Technically, it can start pretty much after the first chunk of data - which will 100% contain the files: field, which is, besides the infoHash, the only critical thing we need in order to start downloading a torrent. Once the verification data (piece hashes) is retrieved and de-bencoded the verification can be enabled.

I guess it's too much work, since you'll need streaming decoding of bencode (there are modules for that though) and the way verification works has to be slightly changed. But I guess it will save some seconds from the loading time, especially if we have a large torrent."

(from mafintosh/peer-wire-protocol#4 (comment))

crash with webtorrent wire

wire = torrent.wires[1];
wire.use(ut_metadata());
wire.ut_metadata.on('metadata', function (metadata) {});

results in crash:

ut_metadata\index.js:189
buf.copy(this.metadata, piece * PIECE_LENGTH)
^

TypeError: argument should be a Buffer
at TypeError (native)
at utMetadata._onData (\ut_metadata\index.js:189:9)

Action required: Greenkeeper could not be activated 🚨

🚨 You need to enable Continuous Integration on all branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because we are using your CI build statuses to figure out when to notify you about breaking changes.

Since we did not receive a CI status on the greenkeeper/initial branch, we assume that you still need to configure it.

If you have already set up a CI for this repository, you might need to check your configuration. Make sure it will run on all new branches. If you don’t want it to run on every branch, you can whitelist branches starting with greenkeeper/.

We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.

Once you have installed CI on this repository, you’ll need to re-trigger Greenkeeper’s initial Pull Request. To do this, please delete the greenkeeper/initial branch in this repository, and then remove and re-add this repository to the Greenkeeper integration’s white list on Github. You'll find this list on your repo or organization’s settings page, under Installed GitHub Apps.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.