Coder Social home page Coder Social logo

notorious's Introduction

Build Status Go Report Card

Notorious

Hello everybody! Notorious aims to be a highy extensible tracker implemented in golang. Right now, Notorious uses Redis to store peer information for quick retrieval and to alleviate some of the burden of storing peer information from the tracker itself. Moreover, some of the core functionalities which Notorious hopes to gain include the following:

1. Improved ratio tracking using a SQL (using an ORM, so it's preference). 
2. IPV6 support
3. UDP & HTTPS tracking (only supports HTTP right now)
4. Speed and scalability are always in the back of my mind, even if my decisions don't always reflect that
5. Remove technical debt. I learned go from this project, so there's definitely non-idiomatic and idiotic code in places.

There's probably a lot more! Check out my issues page

And here for what's planned to be in the next release: Milestone 0.5.0

Deployment

Deployment is one of the funnest things from this project for me so far because I've gotten to use a lot of cloud technologies which I typically wouldn't get to deal with. As of the moment, I'm using Docker Cloud and deploying to a 5 $ Digital Ocean droplet. As you'll see, there's a dockerfile in my repo as well as a supervisord.conf and to build it, you'd want to run:

go build main.go
docker build -t notorious .

and then you can run

docker run <-d> -p 3000:3000 notorious (where the <> indicate the -d is optional, it starts it in daemon mode)

Et voilà, you have a copy of notorious running,.

This will build the docker image which you can then run it either on local bare-metal, or if you're interested in the Docker Cloud route (which I highly recommend) head over to their Documentation They'll explain deployment 1 000% better than I can.

Please note:

I have not yet included instructions for running everything on bare-metal and I'm using docker because it just works (sorry, I try to keep the buzzwordiness down [despite deploying notorious with redis, docker, go...]), but those are soon to come. I just honestly thing the docker way is 10x (just like me, heh) easier to deploy with at this particular moment.

Contact Notorious

Notorious is a project which I've had a ton of fun learning Go in, but do realize I'm still learning Go so I do make non-idiomatic decisions. If you see anywhere that you think I could improve my code or golang usage, please:

open an issue

tweet me

or email me

License

The MIT License (MIT)

Copyright (c) 2016 Ian Clark

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

notorious's People

Contributors

artemnikitin avatar ianleeclark avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

notorious's Issues

Add statistics tracking

  • Total info hashes (total torrent count)
  • Total peer hits
  • Total bytes transferred
  • Peers reaped

Allow for sql-tiering

  • Base level (Ratioless tracking)
    • Store/update only torrent information: file_name, info_hash, downloaders, uploaders
  • Normal tracking
    • Assign a specific key per user to track by
    • Store update peer information like seeds/leeches, downloaded_bytes, uploaded_Bytes

Differentiate between downloaders and uploaders.

This ought to be pretty easily. Just add new redis sub-keys to the info_hash as complete/incomplete and update according to the announce events.

  • Move users to complete upon COMPLETE event
  • Create info_hash upon STARTED event (if info_hash doesn't exist)
  • Remove peers from incomplete/complete upon STOPPED, as client is closing gracefully.

Add unit testing to pure-ish functions

There are a lot of functions that don't have side effects as of this moment which would be easy to test. I don't yet want to commit to mocking unit tests because it's really easy to miss a lot of importanting things when mocking, but we're missing a ton of tests which would be easy to implement.

Bencode should always remain at at 100%
Reaper:

1. Converting time to unix stamps is the only one not requiring mocking.

Server

  • server/announce:GetInt()
  • server/announce:DecodeQueryURL
  • server/announce_response.go:compactIPPort(),compact_all_peers(), create_failure_message()

Add peer reaping

When is probably the biggest question: should I spawn a go channel upon request to reap peers in the redis db or should I have a sentinel-like goroutine spawned along-side the server to handle tasks like this?

Improve bencode library

  • Replace Sprintf with writing to buffers

We can replace any Sprintf() calls with writing to a buffer which should improve performance. Moreover, I'd like to add serialization to these types and add a HTTPResponse model (credit 100% to https://github.com/anacrolix/torrent for the idea)

Add torrent-client whitelisting

A lot of private trackers tend to whitelist or blacklist specific clients (transmission 2.90 as of right now) so this is a much needed feature.

Add a simple web interface

This can include statistics from #64

The next most important feature is to add a torrent to the list of acceptable torrents. The info_hash will be added to Redis (and a backup sent to the database, because Redis persistence is not being targeted), so if a peer requests an info_hash that is not in the Redis database, we can send a response stating that the torrent is not authorized for use on this tracker.

Allow torrent-file whitelisting

Right now, anyone can add a file to the tracker and the tracker will automatically start tracking it. We need to be able to specify specific info_hashes (see issue #65) and reject with a 401 response if the info_hash is found in neither Redis nor MySQL.

TODO(ian): DRY.

File Location: ./bencode/bencode.go
Line Number: 12
This message was auto-generated by Pygemony:Github

Build redis info_hash list from MySQL database

We'll need to switch docker cloud to using a persistent volume, but this should be fairly reasonable. Just query a specific table listing all of the validated info_hashes and create the redis sets based on this data.

Weird redis fragmenting key issue

2016-03-21T03:31:03.754786507Z Adding host 66.69.235.198:51413 to 409f92c9f0312c8fc6a981a2f4d1b011c25bc52f:complete
2016-03-21T03:31:03.754871675Z Event: started from host 66.69.235.198 on port 51413
2016-03-21T03:31:03.756569050Z �Y�n� Ǵ�`��BE��1�BE����
2016-03-21T03:31:03.756609717Z Resp: d8:completei4e10:incompletei4e5:peers23:�Y�n� Ǵ�`��BE��1�BE����e
2016-03-21T03:39:31.321161217Z Adding host 66.69.235.198:12765 to 409f92c9f0312c8fc6a981a2f4d1b011c25bc52f:complete
2016-03-21T03:39:31.321257154Z Event: started from host 66.69.235.198 on port 12765
2016-03-21T03:39:31.321272856Z �Y�n� Ǵ�`��BE��1�BE����
2016-03-21T03:39:31.321287196Z Resp: d8:completei4e10:incompletei4e5:peers23:�Y�n� Ǵ�`��BE��1�BE����e
2016-03-21T03:49:30.901532913Z Adding host 66.69.235.198:12765 to 409f92c9f0312c8fc6a981a2f4d1b011c25bc52f:complete
2016-03-21T03:49:30.901717860Z Event: started from host 66.69.235.198 on port 12765
2016-03-21T03:49:30.904045042Z �Y�n� Ǵ�`��BE��1�BE����

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.