go-phie / gophie Goto Github PK

View Code? Open in Web Editor NEW

289.0 9.0 31.0 34.26 MB

An Aggregator Engine for searching and downloading movies free - NO ADs!

Home Page: https://api.gophie.cam

License: GNU Affero General Public License v3.0

Go 97.89% Makefile 0.69% Shell 0.31% HTML 0.83% Procfile 0.03% Dockerfile 0.25%

cli download-movies scraping-websites web-api no-ads downloader download-videos free movies free-software

gophie's People

Contributors

Stargazers

Watchers

gophie's Issues

Add MoviePerPage to Engine Props

Every Engine should specify how many movies per page are returned during a listing e.g NetNaija returns 14.
Should also reflect on the stoplight documentation

Resume downloads when connection is available

To do this, store details of downloads and their progress in a config file. All unfinished downloads can be resumed when connection is back

Include Torrents options

Users should be able to enable if downloads could include torrents sites also.

Error while using exe on windows "level=info msg="No Results Found""

using any command .exe giving error level=info msg=" No Results Found" (on command list, search etc.)

return series based on Seasons

Using regex, we can manouver around knowing the particular season a series falls under.

Fix besthdmovies parser

The besthdmovies parser is no longer returning valid links

Movie extension

Return the extension type of the movie e.g mp4, mkv

Add BestHDMovies engine

Add https://besthdmovies.top through the ww.9xupload.info alternate links

Create a simple homepage for hostedAPI that shows how the API is used
It should render when homepage of hostedAPI is visited rather than show search and list argument not in url
Make use of packr to store the template within the binary

Change License

Switch license to one that prevents using this free work to make money

Allow overriding variables from ENV Variables using Viper

Using Viper configuration manager, set defaults and override those defaults when some environmental variables are set. Env variables should use the GOPHIE namespace

Another Engine suggestion

Anime: https://gogoanime.pro
Movie: https://nkiri.com

Standardize behaviours of series engine

Standardize the behaviour of all series engines i.e proper search and pagination behaviour in such a way that it slips in perfectly with the seamless SPA nature of the web app
This currently ties into the Tvseries bug

Colly cannot cache FzMovies

FzMovies scraper is slow because we cannot use colly's cache. FzMovies always expires old download keys after a site has been visited so caching will not provide the needed pages to advance to the download link as the old keys used will now be invalid. Find a way to increase scraping speed of FzMovies engine

Not installed

Logs:

go get github.com/go-phie/gophie

github.com/go-phie/gophie/downloader

../../go/src/github.com/go-phie/gophie/downloader/downloader.go:18:37: undefined: downloader.Data
../../go/src/github.com/go-phie/gophie/downloader/downloader.go:29:13: undefined: downloader.URL
../../go/src/github.com/go-phie/gophie/downloader/downloader.go:34:24: undefined: downloader.Stream
../../go/src/github.com/go-phie/gophie/downloader/downloader.go:45:11: undefined: downloader.Data
../../go/src/github.com/go-phie/gophie/downloader/downloader.go:72:10: undefined: downloader.Data
../../go/src/github.com/go-phie/gophie/downloader/downloader.go:93:2: undefined: config.OutputPath
../../go/src/github.com/go-phie/gophie/downloader/downloader.go:100:9: undefined: downloader.Download
../../go/src/github.com/go-phie/gophie/downloader/downloader.go:100:42: undefined: config.ChunkSizeMB

OS: Macos Mojave, Go version go1.14.6 darwin/amd64

New series engine

Add an extra engine for series e.g netnaija

Send Downloadlink as string

Download link ls currently being sent by the API as an object, send as the converted string instead

Add Multiple Engines

The current implementation uses netnaija as the primary source for movies downloads. Other sites that offer movies downloads should be included as well.

We'll need lists of popular scrapable sites for movie downloads. Any help would be appreciated

Integrating selenium for javascript based sites

Sites like tfpdl might need selenium to scrape. We could run selenium in conjunction with colly
https://github.com/tebeka/selenium

Runtime Error: index out of range [2] with length 1 when running Search method on Netnaija Engine

Logs in verbose mode

go run .\main.go search Spider Man -v
time="2022-07-31T03:15:16+01:00" level=debug msg="Switching to ChromeDpTransport"
time="2022-07-31T03:15:18+01:00" level=debug msg="Visiting https://www.thenetnaija.com/search?folder=videos&t=Spider+Man"
time="2022-07-31T03:15:18+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.com/search?folder=videos&t=Spider+Man"
time="2022-07-31T03:15:19+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.com/search?folder=videos&t=Spider+Man"
time="2022-07-31T03:15:19+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:19+01:00" level=debug msg="Done https://www.thenetnaija.com/search?folder=videos&t=Spider+Man"
time="2022-07-31T03:15:19+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-4\n"
time="2022-07-31T03:15:19+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-4"
time="2022-07-31T03:15:20+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-4"
time="2022-07-31T03:15:20+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:20+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-4\n"
time="2022-07-31T03:15:20+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-4/download\n"
time="2022-07-31T03:15:20+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-4/download"
time="2022-07-31T03:15:22+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-4/download"
time="2022-07-31T03:15:22+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:22+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-4/download\n"
time="2022-07-31T03:15:23+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-5\n"
time="2022-07-31T03:15:23+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-5"
time="2022-07-31T03:15:23+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-5"
time="2022-07-31T03:15:23+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:23+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-5\n"
time="2022-07-31T03:15:23+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-5/download\n"
time="2022-07-31T03:15:23+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-5/download"
time="2022-07-31T03:15:25+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-5/download"
time="2022-07-31T03:15:25+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:25+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-5/download\n"
time="2022-07-31T03:15:25+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-2\n"
time="2022-07-31T03:15:25+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-2"
time="2022-07-31T03:15:26+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-2"
time="2022-07-31T03:15:26+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:26+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-2\n"
time="2022-07-31T03:15:26+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-2/download\n"
time="2022-07-31T03:15:26+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-2/download"
time="2022-07-31T03:15:27+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-2/download"
time="2022-07-31T03:15:27+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:27+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-2/download\n"
time="2022-07-31T03:15:28+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-3\n"
time="2022-07-31T03:15:28+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-3"
time="2022-07-31T03:15:28+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-3"
time="2022-07-31T03:15:28+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:28+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-3\n"
time="2022-07-31T03:15:28+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-3/download\n"
time="2022-07-31T03:15:28+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-3/download"
time="2022-07-31T03:15:30+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-3/download"
time="2022-07-31T03:15:30+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:30+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-3/download\n"
time="2022-07-31T03:15:30+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-1\n"
time="2022-07-31T03:15:30+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-1"
time="2022-07-31T03:15:30+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-1"
time="2022-07-31T03:15:30+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:30+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-1\n"
time="2022-07-31T03:15:30+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-1/download\n"
time="2022-07-31T03:15:30+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-1/download"
time="2022-07-31T03:15:32+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-1/download"
time="2022-07-31T03:15:32+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:32+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-1/download\n"
time="2022-07-31T03:15:32+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-6\n"
time="2022-07-31T03:15:32+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-6"
time="2022-07-31T03:15:33+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-6"
time="2022-07-31T03:15:33+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:33+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-6\n"
time="2022-07-31T03:15:33+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-6/download\n"
time="2022-07-31T03:15:33+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-6/download"
time="2022-07-31T03:15:34+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-6/download"
time="2022-07-31T03:15:34+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:34+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/season-3/episode-6/download\n"
time="2022-07-31T03:15:34+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/movies/7938-spider-man-into-the-spider-verse-2018\n"
time="2022-07-31T03:15:34+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/movies/7938-spider-man-into-the-spider-verse-2018"
time="2022-07-31T03:15:35+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/movies/7938-spider-man-into-the-spider-verse-2018"
time="2022-07-31T03:15:35+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:35+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/movies/7938-spider-man-into-the-spider-verse-2018\n"
time="2022-07-31T03:15:35+01:00" level=info msg="Animation, Action, Adventure, Comedy, Family, Sci-FiRelease Date: Dec 14, 2018Stars: Shameik Moore, Jake Johnson, Hailee SteinfeldSource: Spider-Man.Into.the.Spider-Verse.2018.720p.WEB-DL.XviD.AC3-FGTLanguage: EnglishSubtitle Language: EnglishIMDB: https://www.imdb.com/title/tt4633694/"
time="2022-07-31T03:15:35+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/movies/7938-spider-man-into-the-spider-verse-2018/download\n"
time="2022-07-31T03:15:35+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/movies/7938-spider-man-into-the-spider-verse-2018/download"
time="2022-07-31T03:15:36+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/movies/7938-spider-man-into-the-spider-verse-2018/download"
time="2022-07-31T03:15:36+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:36+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/movies/7938-spider-man-into-the-spider-verse-2018/download\n"
time="2022-07-31T03:15:36+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man\n"
time="2022-07-31T03:15:36+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man"
time="2022-07-31T03:15:37+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man"
time="2022-07-31T03:15:37+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:37+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man\n"
time="2022-07-31T03:15:37+01:00" level=info msg="Animation, Action, Adventure, Family, Fantasy, Sci-FiIMDB: https://www.imdb.com/title/tt6135682/Premiere: Aug 19, 2017"
time="2022-07-31T03:15:37+01:00" level=debug msg="Retrieving Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/download\n"
time="2022-07-31T03:15:37+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/download"
time="2022-07-31T03:15:38+01:00" level=debug msg="Set Headers for page https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/download"
time="2022-07-31T03:15:38+01:00" level=debug msg="Successfully retrieved body"
time="2022-07-31T03:15:38+01:00" level=debug msg="Retrieved Download Link https://www.thenetnaija.net/videos/series/11602-marvels-spider-man/download\n"
time="2022-07-31T03:15:38+01:00" level=info msg="Animation, Action, Adventure, Family, Fantasy, Sci-FiIMDB: https://www.imdb.com/title/tt6135682/Premiere: Aug 19, 2017"
panic: runtime error: index out of range [2] with length 1

Fix netnaija and fzmovies parser

Both parsers are no longer returning direct links

Issue installing

go get github.com/go-phie/gophie                                                         ✔  1214  04:24:27

go: downloading github.com/go-phie/gophie v0.3.1
go: github.com/go-phie/gophie upgrade => v0.3.1
go get: github.com/go-phie/[email protected]: parsing go.mod:
	module declares its path as: github.com/bisoncorps/gophie
	        but was required as: github.com/go-phie/gophie

There is also 2 different repos that seem to be confused.

go get github.com/go-phie/gophie  ✔  1216 

04:33:13
# github.com/bisoncorps/gophie/downloader
../bisoncorps/gophie/downloader/downloader.go:18:37: undefined: downloader.Data
../bisoncorps/gophie/downloader/downloader.go:29:13: undefined: downloader.URL
../bisoncorps/gophie/downloader/downloader.go:34:24: undefined: downloader.Stream
../bisoncorps/gophie/downloader/downloader.go:45:11: undefined: downloader.Data
../bisoncorps/gophie/downloader/downloader.go:72:10: undefined: downloader.Data
../bisoncorps/gophie/downloader/downloader.go:93:2: undefined: config.OutputPath
../bisoncorps/gophie/downloader/downloader.go:100:9: undefined: downloader.Download
../bisoncorps/gophie/downloader/downloader.go:100:42: undefined: config.ChunkSizeMB

Am i doing something wrong?

Add Anime Downloads

Popular Anime sites should also be featured in downloads

go install error

➜ ~ go get github.com/bisoncorps/gophie
github.com/bisoncorps/gophie/cmd
go/src/github.com/bisoncorps/gophie/cmd/stream.go:60:4: p.SetStreamURL undefined (type mplayer.Player has no field or method SetStreamURL)

Add series downloads

Add series only engines. Open to suggestions on the sites to use

Integrate GoReleaser

Use goreleaser to update the releases on every tag

Add yesmovies hd parser

Add an extra parser for https://yesmovies.ag

Load balancing using multiple mirrors

Gophie API instance on Heroku keeps crashing, the API is public and a lot of people seem to be accessing it, I know Beetubebot is using it, as well as the React platform and the Native platform I am building. If there is any way we can set up to autodeploy to multiple mirrors and have people use the different mirrors instead of concentrating on just one mirror. Look at this for instance https://github.com/dmytrostriletskyi/heroku-load-balancer
Something like that should do the trick right, Spotlight docs can specify multiple mirrors for an endpoint

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.