Comments (13)
should move this project to codeberg
from dhtcrawler2.
host own gitea like torrent-csv! google cloud instance free tier would be enough if cloud is must
from dhtcrawler2.
again with this nonsense?
from dhtcrawler2.
I guess since they keep taking down the domain DCMA complaints are going here? If im not mistaken this repo is not the same owner running btdig.com
from dhtcrawler2.
I guess since they keep taking down the domain DCMA complaints are going here? If im not mistaken this repo is not the same owner running btdig.com
Even if it was, this is just code, its not got any of the said files in question. They need to go bother someone else with this nonsense.
from dhtcrawler2.
codeberg?
from dhtcrawler2.
from dhtcrawler2.
lol too easy. i should ave guessed the URL :) but thanks. ill take a peek myself.. new to me
from dhtcrawler2.
Is there anyway to just run the scraper and insert into custom db?
from dhtcrawler2.
not sure, I got sick of messing with SQL. Instead I grabbed every manetico db I could (500GB+ uncompressed), and pulled all of the hashes, names, file sizes etc and just keep it in a python dict (<5GB uncompressed pickle file). Then I hacked together a scraper from a few different implementations out there, and finally a search function. 25M+ hashes, after each search I will scrape against trackers from trackerlist to see whats alive. Also every now and again I'll import torrents-csv to get latest info. when I generate a magnet I'll have it add only trackers that have the hash
from dhtcrawler2.
In the unlikely event the code repo is taken down by bots, i am sure to keep a local copy. I had one clone of another repo removed on me..
from dhtcrawler2.
In the unlikely event the code repo is taken down by bots, i am sure to keep a local copy. I had one clone of another repo removed on me..
I've been running gitea on my server and everytime I find a repo that I value, I'll migrate it to gitea, I have it set to sync/pull the repos every 24h. In the event that a DMCA takes place, I'll still have the code
from dhtcrawler2.
I've been running gitea on my server and everytime I find a repo that I value, I'll migrate it to gitea, I have it set to sync/pull the repos every 24h. In the event that a DMCA takes place, I'll still have the code
i use fossil at home, i should do that too. Right now i just wait until i get release notices, sync, then download a zip, and store it away.
I have had a couple of source repos vanish, unrelated to DMCA and just a pissed off onwer, so i started doing it long ago.
from dhtcrawler2.
Related Issues (20)
- New search results not showing on BTdig since 24hrs HOT 11
- can we get a docker-compose.yml file for this?
- Add dark theme
- loading scren in bitorrent HOT 1
- BTdig site down since 3 days HOT 5
- Seems than the most of your torrents are dead HOT 1
- Bug
- Downloads HOT 2
- Seems than the serbian tv series of your torrents are dead HOT 8
- Weird popup appears when searching HOT 3
- FIFA20
- v2 torrent and Vuze DHT support
- Homeworld 3
- please remove all dead torrents ........
- Please remove my old torrent because I erased the files.
- Stremio Addon
- Please, block ch1ld p0rn from the site.
- Demands the removal of the Korean movie The New Normal.
- COPYRIGHT CLAIMS BTDIG
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dhtcrawler2.