Coder Social home page Coder Social logo

flaresolverr / flaresolverr Goto Github PK

View Code? Open in Web Editor NEW
5.8K 58.0 516.0 1.13 MB

Proxy server to bypass Cloudflare protection

License: MIT License

Dockerfile 1.55% HTML 25.54% Python 72.91%
puppeteer cloudflare proxy chrome chromium docker server api rest hacktoberfest

flaresolverr's Introduction

FlareSolverr

Latest release Docker Pulls GitHub issues GitHub pull requests Donate PayPal Donate Bitcoin Donate Ethereum

FlareSolverr is a proxy server to bypass Cloudflare and DDoS-GUARD protection.

How it works

FlareSolverr starts a proxy server, and it waits for user requests in an idle state using few resources. When some request arrives, it uses Selenium with the undetected-chromedriver to create a web browser (Chrome). It opens the URL with user parameters and waits until the Cloudflare challenge is solved (or timeout). The HTML code and the cookies are sent back to the user, and those cookies can be used to bypass Cloudflare using other HTTP clients.

NOTE: Web browsers consume a lot of memory. If you are running FlareSolverr on a machine with few RAM, do not make many requests at once. With each request a new browser is launched.

It is also possible to use a permanent session. However, if you use sessions, you should make sure to close them as soon as you are done using them.

Installation

Docker

It is recommended to install using a Docker container because the project depends on an external browser that is already included within the image.

Docker images are available in:

Supported architectures are:

Architecture Tag
x86 linux/386
x86-64 linux/amd64
ARM32 linux/arm/v7
ARM64 linux/arm64

We provide a docker-compose.yml configuration file. Clone this repository and execute docker-compose up -d (Compose V1) or docker compose up -d (Compose V2) to start the container.

If you prefer the docker cli execute the following command.

docker run -d \
  --name=flaresolverr \
  -p 8191:8191 \
  -e LOG_LEVEL=info \
  --restart unless-stopped \
  ghcr.io/flaresolverr/flaresolverr:latest

If your host OS is Debian, make sure libseccomp2 version is 2.5.x. You can check the version with sudo apt-cache policy libseccomp2 and update the package with sudo apt install libseccomp2=2.5.1-1~bpo10+1 or sudo apt install libseccomp2=2.5.1-1+deb11u1. Remember to restart the Docker daemon and the container after the update.

Precompiled binaries

Warning Precompiled binaries are only available for x64 architecture. For other architectures see Docker images.

This is the recommended way for Windows users.

  • Download the FlareSolverr executable from the release's page. It is available for Windows x64 and Linux x64.
  • Execute FlareSolverr binary. In the environment variables section you can find how to change the configuration.

From source code

Warning Installing from source code only works for x64 architecture. For other architectures see Docker images.

  • Install Python 3.11.
  • Install Chrome (all OS) or Chromium (just Linux, it doesn't work in Windows) web browser.
  • (Only in Linux) Install Xvfb package.
  • (Only in macOS) Install XQuartz package.
  • Clone this repository and open a shell in that path.
  • Run pip install -r requirements.txt command to install FlareSolverr dependencies.
  • Run python src/flaresolverr.py command to start FlareSolverr.

From source code (FreeBSD/TrueNAS CORE)

  • Run pkg install chromium python39 py39-pip xorg-vfbserver command to install the required dependencies.
  • Clone this repository and open a shell in that path.
  • Run python3.9 -m pip install -r requirements.txt command to install FlareSolverr dependencies.
  • Run python3.9 src/flaresolverr.py command to start FlareSolverr.

Systemd service

We provide an example Systemd unit file flaresolverr.service as reference. You have to modify the file to suit your needs: paths, user and environment variables.

Usage

Example Bash request:

curl -L -X POST 'http://localhost:8191/v1' \
-H 'Content-Type: application/json' \
--data-raw '{
  "cmd": "request.get",
  "url": "http://www.google.com/",
  "maxTimeout": 60000
}'

Example Python request:

import requests

url = "http://localhost:8191/v1"
headers = {"Content-Type": "application/json"}
data = {
    "cmd": "request.get",
    "url": "http://www.google.com/",
    "maxTimeout": 60000
}
response = requests.post(url, headers=headers, json=data)
print(response.text)

Example PowerShell request:

$body = @{
    cmd = "request.get"
    url = "http://www.google.com/"
    maxTimeout = 60000
} | ConvertTo-Json

irm -UseBasicParsing 'http://localhost:8191/v1' -Headers @{"Content-Type"="application/json"} -Method Post -Body $body

Commands

+ sessions.create

This will launch a new browser instance which will retain cookies until you destroy it with sessions.destroy. This comes in handy, so you don't have to keep solving challenges over and over and you won't need to keep sending cookies for the browser to use.

This also speeds up the requests since it won't have to launch a new browser instance for every request.

Parameter Notes
session Optional. The session ID that you want to be assigned to the instance. If isn't set a random UUID will be assigned.
proxy Optional, default disabled. Eg: "proxy": {"url": "http://127.0.0.1:8888"}. You must include the proxy schema in the URL: http://, socks4:// or socks5://. Authorization (username/password) is supported. Eg: "proxy": {"url": "http://127.0.0.1:8888", "username": "testuser", "password": "testpass"}

+ sessions.list

Returns a list of all the active sessions. More for debugging if you are curious to see how many sessions are running. You should always make sure to properly close each session when you are done using them as too many may slow your computer down.

Example response:

{
  "sessions": [
    "session_id_1",
    "session_id_2",
    "session_id_3..."
  ]
}

+ sessions.destroy

This will properly shutdown a browser instance and remove all files associated with it to free up resources for a new session. When you no longer need to use a session you should make sure to close it.

Parameter Notes
session The session ID that you want to be destroyed.

+ request.get

Parameter Notes
url Mandatory
session Optional. Will send the request from and existing browser instance. If one is not sent it will create a temporary instance that will be destroyed immediately after the request is completed.
session_ttl_minutes Optional. FlareSolverr will automatically rotate expired sessions based on the TTL provided in minutes.
maxTimeout Optional, default value 60000. Max timeout to solve the challenge in milliseconds.
cookies Optional. Will be used by the headless browser. Eg: "cookies": [{"name": "cookie1", "value": "value1"}, {"name": "cookie2", "value": "value2"}].
returnOnlyCookies Optional, default false. Only returns the cookies. Response data, headers and other parts of the response are removed.
proxy Optional, default disabled. Eg: "proxy": {"url": "http://127.0.0.1:8888"}. You must include the proxy schema in the URL: http://, socks4:// or socks5://. Authorization (username/password) is not supported. (When the session parameter is set, the proxy is ignored; a session specific proxy can be set in sessions.create.)

Warning If you want to use Cloudflare clearance cookie in your scripts, make sure you use the FlareSolverr User-Agent too. If they don't match you will see the challenge.

Example response from running the curl above:

{
    "solution": {
        "url": "https://www.google.com/?gws_rd=ssl",
        "status": 200,
        "headers": {
            "status": "200",
            "date": "Thu, 16 Jul 2020 04:15:49 GMT",
            "expires": "-1",
            "cache-control": "private, max-age=0",
            "content-type": "text/html; charset=UTF-8",
            "strict-transport-security": "max-age=31536000",
            "p3p": "CP=\"This is not a P3P policy! See g.co/p3phelp for more info.\"",
            "content-encoding": "br",
            "server": "gws",
            "content-length": "61587",
            "x-xss-protection": "0",
            "x-frame-options": "SAMEORIGIN",
            "set-cookie": "1P_JAR=2020-07-16-04; expires=Sat..."
        },
        "response":"<!DOCTYPE html>...",
        "cookies": [
            {
                "name": "NID",
                "value": "204=QE3Ocq15XalczqjuDy52HeseG3zAZuJzID3R57...",
                "domain": ".google.com",
                "path": "/",
                "expires": 1610684149.307722,
                "size": 178,
                "httpOnly": true,
                "secure": true,
                "session": false,
                "sameSite": "None"
            },
            {
                "name": "1P_JAR",
                "value": "2020-07-16-04",
                "domain": ".google.com",
                "path": "/",
                "expires": 1597464949.307626,
                "size": 19,
                "httpOnly": false,
                "secure": true,
                "session": false,
                "sameSite": "None"
            }
        ],
        "userAgent": "Windows NT 10.0; Win64; x64) AppleWebKit/5..."
    },
    "status": "ok",
    "message": "",
    "startTimestamp": 1594872947467,
    "endTimestamp": 1594872949617,
    "version": "1.0.0"
}

+ request.post

This is the same as request.get but it takes one more param:

Parameter Notes
postData Must be a string with application/x-www-form-urlencoded. Eg: a=b&c=d

Environment variables

Name Default Notes
LOG_LEVEL info Verbosity of the logging. Use LOG_LEVEL=debug for more information.
LOG_HTML false Only for debugging. If true all HTML that passes through the proxy will be logged to the console in debug level.
CAPTCHA_SOLVER none Captcha solving method. It is used when a captcha is encountered. See the Captcha Solvers section.
TZ UTC Timezone used in the logs and the web browser. Example: TZ=Europe/London.
LANG none Language used in the web browser. Example: LANG=en_GB.
HEADLESS true Only for debugging. To run the web browser in headless mode or visible.
BROWSER_TIMEOUT 40000 If you are experiencing errors/timeouts because your system is slow, you can try to increase this value. Remember to increase the maxTimeout parameter too.
TEST_URL https://www.google.com FlareSolverr makes a request on start to make sure the web browser is working. You can change that URL if it is blocked in your country.
PORT 8191 Listening port. You don't need to change this if you are running on Docker.
HOST 0.0.0.0 Listening interface. You don't need to change this if you are running on Docker.
PROMETHEUS_ENABLED false Enable Prometheus exporter. See the Prometheus section below.
PROMETHEUS_PORT 8192 Listening port for Prometheus exporter. See the Prometheus section below.

Environment variables are set differently depending on the operating system. Some examples:

  • Docker: Take a look at the Docker section in this document. Environment variables can be set in the docker-compose.yml file or in the Docker CLI command.
  • Linux: Run export LOG_LEVEL=debug and then run flaresolverr in the same shell.
  • Windows: Open cmd.exe, run set LOG_LEVEL=debug and then run flaresolverr.exe in the same shell.

Prometheus exporter

The Prometheus exporter for FlareSolverr is disabled by default. It can be enabled with the environment variable PROMETHEUS_ENABLED. If you are using Docker make sure you expose the PROMETHEUS_PORT.

Example metrics:

# HELP flaresolverr_request_total Total requests with result
# TYPE flaresolverr_request_total counter
flaresolverr_request_total{domain="nowsecure.nl",result="solved"} 1.0
# HELP flaresolverr_request_created Total requests with result
# TYPE flaresolverr_request_created gauge
flaresolverr_request_created{domain="nowsecure.nl",result="solved"} 1.690141657157109e+09
# HELP flaresolverr_request_duration Request duration in seconds
# TYPE flaresolverr_request_duration histogram
flaresolverr_request_duration_bucket{domain="nowsecure.nl",le="0.0"} 0.0
flaresolverr_request_duration_bucket{domain="nowsecure.nl",le="10.0"} 1.0
flaresolverr_request_duration_bucket{domain="nowsecure.nl",le="25.0"} 1.0
flaresolverr_request_duration_bucket{domain="nowsecure.nl",le="50.0"} 1.0
flaresolverr_request_duration_bucket{domain="nowsecure.nl",le="+Inf"} 1.0
flaresolverr_request_duration_count{domain="nowsecure.nl"} 1.0
flaresolverr_request_duration_sum{domain="nowsecure.nl"} 5.858
# HELP flaresolverr_request_duration_created Request duration in seconds
# TYPE flaresolverr_request_duration_created gauge
flaresolverr_request_duration_created{domain="nowsecure.nl"} 1.6901416571570296e+09

Captcha Solvers

Warning At this time none of the captcha solvers work. You can check the status in the open issues. Any help is welcome.

Sometimes CloudFlare not only gives mathematical computations and browser tests, sometimes they also require the user to solve a captcha. If this is the case, FlareSolverr will return the error Captcha detected but no automatic solver is configured.

FlareSolverr can be customized to solve the CAPTCHA automatically by setting the environment variable CAPTCHA_SOLVER to the file name of one of the adapters inside the /captcha directory.

Related projects

flaresolverr's People

Contributors

21hsmw avatar abeloin avatar arias800 avatar asthowen avatar athorcis avatar bilditup1 avatar dependabot[bot] avatar drsdavidsoft avatar dx37 avatar francisco-lafe avatar garfield69 avatar haroldm avatar hashworks avatar ilike2burnthing avatar joshdi avatar martinomensio avatar nabikaz avatar ngosang avatar nyamisty avatar rawandahmad698 avatar rpatterson avatar simonfr avatar tadasgedgaudas avatar termonio avatar thags avatar txtsd avatar xewdy444 avatar yi-ge avatar zacharyhampton avatar zax2002 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

flaresolverr's Issues

container issue on launch

Hey guys,

i just pulled the container and i've this on logs when it starts

npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /home/node/cloudproxy/package.json
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/home/node/cloudproxy/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent 
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/node/.npm/_logs/2020-12-14T01_58_45_963Z-debug.log
npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /home/node/cloudproxy/package.json
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/home/node/cloudproxy/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent 
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/node/.npm/_logs/2020-12-14T01_58_58_364Z-debug.log

Something went wrong with the latest build i guess

Increase timeout ?

Hi,
I use an old Raspberry Pi 3 and max time out is 60s. And everytime I have errror.
Is that possible to increase this in my container ?

Thanks a lot

Set captcha provider and options with json request

Given we can implement many captcha solver, having the possibility to choose with which solver you want to solve the request you're asking it to solve would be a great addition.

curl -L -X POST 'http://flaresolverr:8191/v1' \
-H 'Content-Type: application/json' \
--data-raw '{
  "cmd": "request.get",
  "url":"https://patrickhlauke.github.io/recaptcha/",
  "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleW...",
  "maxTimeout": 60000,
  "solver-provider": { 
    "name": "xxxxxx", 
    "options": {
      "apikey": "yyyyy",
      "otheroptionoptionnal": "zzzzz"
     }
  }
  "headers": {
    "X-Test": "Testing 123..."
  }
}'

setting solver-provider in the json with the name of the actual captcha solver existing, overriding the ENV var if defined and defining optional options, like api key and possibly other stuff

EXAMPLE

My first example would be in jackett to integrate a menu with captcha providers for flaresolverr (restricting jackett to flaresolverr ? or having a SELECT Menu with Solver APP like Flaresolverr) and then configuring said providers, the need to provide providers capabilities would then be great between app and solver to automagically propose settings (best case scenario)

Easy case, would just be type your provider name, and APIKEY in an input and that's it to POC, or basic implementation

Then Jackett could try 1st provider, if no result move to do res with second provider, etc.. etc...
You could in jackett set your provider priorities too

Thanks.

Only chrome is guaranteed to work

Instruction on how to enable debug and html trace

Follow the instructions from this wiki page

Environment

FlareSolverr Version:

Docker: [yes]

OS:

Last Working FlareSolverr Version: FlareSolverr v1.2.0

Are you using a proxy or VPN? [no]

Using Captcha Solver: [yes]

If using captcha solver, which one:
unknown

Description

Call of FlareSolver since JackettUI

Logged Error Messages

An error occurred while testing this indexer
Exception (yggtorrent): FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Timed out after 30000 ms while trying to connect to the browser! Only Chrome at revision r756035 is guaranteed to work.: FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Timed out after 30000 ms while trying to connect to the browser! Only Chrome at revision r756035 is guaranteed to work.

Not a bug (some help if is posible)

Hello, how can use this lib to bypass cloudflare?
I have a scriptp for generate traffic, vote etc, and we need to bypass cloudflare
here is a exemple of my script

`//fixed
is posible to use your lib to solve my solution? if yes, how

Error on YGGTorrent

Hi,
I'm trying to use FlareSolverr to connect to yggtorrent.si but obtain an error !
curl -L -X POST 'http://localhost:8191/v1' -H 'Content-Type: application/json' --data-raw '{ "url":"https://www.yggtorrent.si/", "userAgent": "Mozilla/5.0 (X11; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0", "maxTimeout": 10000 }'
Return this error :
{"status":"error","message":"Maximum timeout reached. maxTimeout=10000 (ms)","startTimestamp":1607254565415,"endTimestamp":1607254576646,"version":"1.0.0"}

Any idea to solve my problem ? Thx

Exception: The cookies provided by FlareSolverr are not valid

Jackett - Docker v0.17.50-ls9 amd64
FlareSolver - Docker v1.2.0 amd64

When not using a proxy/VPN in Jackett's settings, this affects:

  • cpasbien - region blocked
  • Epizod

Indexers which are working when not using a proxy/VPN:

  • dhmy
  • EXT
  • HDDolby
  • MejorTorrent
  • MyPornClub
  • ProStyleX
  • TorrentKitty
  • TorrentProject
  • YGGcookie

Indexers which get Exception (INDEXER): FlareSolverr was able to process the request, but a captcha was detected. Message: Captcha detected but no automatic solver is configured. (so would probably work if I set one up) when not using proxy/VPN:

However, the only indexers which work when using a proxy/VPN in Jackett's settings are:

  • EXT
  • HDDolby

The rest show that the cookies aren't valid, or that a captcha was detected. Any way to extend the proxy settings for Jackett to FlareSolver, or add similar environment variables to FlareSolver?

Error on Yggtorrent ARM64 only

Please use the search bar at the top of the page and make sure you are not creating an already submitted issue.
Check closed issues as well, because your issue may have already been fixed.

Instruction on how to enable debug and html trace

Follow the instructions from this wiki page

Environment

FlareSolverr Version: 1.2.0

Docker: yes

OS: Ubuntu ARM64

Last Working FlareSolverr Version: first time use

Are you using a proxy or VPN? no

Using Captcha Solver: no

If using captcha solver, which one:

Description

When I check Yggtorrent url with curl or jackett, it give me this error but not with Google. It only crash on ARM64 (haven't tested ARM32) and not on x86.

Logged Error Messages

2020-12-16T21:57:02.879Z INFO REQ-2 Cloudflare detected 2020-12-16T21:57:19.465Z DEBUG REQ-2 Waiting for Cloudflare challenge... /home/node/flaresolverr/node_modules/puppeteer/lib/Page.js:209 this.emit('error', new Error('Page crashed!')); ^ Error: Page crashed! at Page._onTargetCrashed (/home/node/flaresolverr/node_modules/puppeteer/lib/Page.js:209:28) at CDPSession.<anonymous> (/home/node/flaresolverr/node_modules/puppeteer/lib/Page.js:129:57) at CDPSession.emit (node:events:329:20) at CDPSession._onMessage (/home/node/flaresolverr/node_modules/puppeteer/lib/Connection.js:166:18) at Connection._onMessage (/home/node/flaresolverr/node_modules/puppeteer/lib/Connection.js:83:25) at WebSocket.<anonymous> (/home/node/flaresolverr/node_modules/puppeteer/lib/WebSocketTransport.js:25:32) at WebSocket.onMessage (/home/node/flaresolverr/node_modules/ws/lib/event-target.js:132:16) at WebSocket.emit (node:events:329:20) at Receiver.receiverOnMessage (/home/node/flaresolverr/node_modules/ws/lib/websocket.js:825:20) at Receiver.emit (node:events:329:20) Emitted 'error' event on Page instance at: at Page._onTargetCrashed (/home/node/flaresolverr/node_modules/puppeteer/lib/Page.js:209:14) at CDPSession.<anonymous> (/home/node/flaresolverr/node_modules/puppeteer/lib/Page.js:129:57) [... lines matching original stack trace ...] at Receiver.emit (node:events:329:20) npm ERR! code 1 npm ERR! path /home/node/flaresolverr npm ERR! command failed npm ERR! command sh -c node ./dist/index.js npm ERR! A complete log of this run can be found in: npm ERR! /home/node/.npm/_logs/2020-12-16T21_58_04_029Z-debug.log

Screenshots

https://i.imgur.com/7syQvjW.png

Support for other captcha solvers

Hello,

There's quite a lot of services that provides captcha solvers for a small fee, and are fairly reliable.
Current captcha solvers are either manual, or relying on random image selection, which is very unreliable. Users that want the most automated and autonomous systems might be interested in using other captcha solvers.

One example is anti-captcha. Do you think it would be possible to integrate it in FlareSolverr ? That would be a nice addition, IMHO.

Thanks for your work :)

FlareSolverrSharp.Exceptions.FlareSolverrException: Chaptcha detected!

Now I'm getting this error:

2020-06-22 03:36:30 	Error 	

{0} Jackett.Common.IndexerException: Exception (bteye): Chaptcha detected!
 ---> FlareSolverrSharp.Exceptions.FlareSolverrException: Chaptcha detected!
   at FlareSolverrSharp.Solvers.FlareSolverr.<>c__DisplayClass5_0.<<Solve>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at FlareSolverrSharp.Utilities.SemaphoreLocker.LockAsync[T](Func`1 worker)
   at FlareSolverrSharp.Solvers.FlareSolverr.Solve(HttpRequestMessage request)
   at FlareSolverrSharp.ClearanceHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Jackett.Common.Utils.Clients.HttpWebClient2NetCore.Run(WebRequest webRequest) in /home/vsts/work/1/s/src/Jackett.Common/Utils/Clients/HttpWebClient2NetCore.cs:line 230
   at Jackett.Common.Utils.Clients.WebClient.GetString(WebRequest request) in /home/vsts/work/1/s/src/Jackett.Common/Utils/Clients/WebClient.cs:line 114
   at Jackett.Common.Indexers.BaseWebIndexer.RequestStringWithCookies(String url, String cookieOverride, String referer, Dictionary`2 headers) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 443
   at Jackett.Common.Indexers.CardigannIndexer.PerformQuery(TorznabQuery query) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 1341
   at Jackett.Common.Indexers.BaseIndexer.ResultsForQuery(TorznabQuery query) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 325
   --- End of inner exception stack trace ---
   at Jackett.Common.Indexers.BaseIndexer.ResultsForQuery(TorznabQuery query) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 345
   at Jackett.Common.Indexers.BaseWebIndexer.ResultsForQuery(TorznabQuery query) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 804
   at Jackett.Server.Controllers.ResultsController.Results(ApiSearch requestt) in /home/vsts/work/1/s/src/Jackett.Server/Controllers/ResultsController.cs:line 224

Is this intended? Because I see this in readme:

The current implementation is not able to bypass Cloudflare because they are detecting the headless browser.

Add issue template

  • Include minimum information. FlareSolverr version, URL, traces, OS...
  • Instructions to enable html log and debug traces => Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36

FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Failed to launch the browser process!

Please use the search bar at the top of the page and make sure you are not creating an already submitted issue.
Check closed issues as well, because your issue may have already been fixed.

Please read our Contributing Guidelines before submitting your issue to ensure a prompt response to your bug.

Environment

OS: Ubuntu Server

.Net Runtime:

.Net Version:

Jackett Version: v0.17.159

Last Working Jackett Version: v0.17.159

Are you using a proxy or VPN? [no]

Logged Error Messages

2020-12-26T23:34:19.955Z INFO REQ-1 Incoming request: POST /v1
2020-12-26T23:34:19.957Z INFO REQ-1 Params: {"maxTimeout":60000,"cmd":"request.get","url":"https://www2.yggtorrent.si/new_search/search?category=all&name=&description=&file=&uploader=&sub_category=&do=search&order=desc&sort=publish_date","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"}
Error: Failed to launch the browser process!
/home/alex/FlareSolverr/node_modules/puppeteer/.local-chromium/linux-756035/chrome-linux/chrome: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory

TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/master/docs/troubleshooting.md

at onClose (/home/alex/FlareSolverr/node_modules/puppeteer/lib/launcher/BrowserRunner.js:159:20)
at Interface.helper_1.helper.addEventListener (/home/alex/FlareSolverr/node_modules/puppeteer/lib/launcher/BrowserRunner.js:149:65)
at Interface.emit (events.js:203:15)
at Interface.close (readline.js:397:8)
at Socket.onend (readline.js:173:10)
at Socket.emit (events.js:203:15)
at endReadableNT (_stream_readable.js:1145:12)
at process._tickCallback (internal/process/next_tick.js:63:19)

2020-12-26T23:34:20.006Z ERROR REQ-1 Failed to launch the browser process!
/home/alex/FlareSolverr/node_modules/puppeteer/.local-chromium/linux-756035/chrome-linux/chrome: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory

TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/master/docs/troubleshooting.md

Captcha detected

Hi, this is a great project. I was actually looking into the same direction after CF changed their challenge in June but could not get it work with puppeteer headless chrome. Your solution works fine but after a while I get captcha detected errors. Any idea what might be causing this? In python based CF solvers trigger of captcha used to be caused by problematic TLSv1.0 ciphers that had to be removed from the urllib3 default cipher suite. Could this be similar with puppeteer?

Thanks,

Jx-

docker error : npm ERR! command sh -c node ./dist/index.js

Hello,

Happy new year !

I've launched flaresolverr with docker. But I have this error :

 
npm ERR! path /home/node/flaresolverr
npm ERR! command failed
npm ERR! signal SIGTERM
npm ERR! command sh -c node ./dist/index.js
See logs in .....

The logs are :

21 timing command:run-script Completed in 36952ms
22 timing command:start Completed in 37001ms
23 verbose stack Error: command failed
23 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/@npmcli/promise-spawn/index.js:64:27)
23 verbose stack     at ChildProcess.emit (node:events:329:20)
23 verbose stack     at maybeClose (node:internal/child_process:1055:16)
23 verbose stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:288:5)
24 verbose pkgid [email protected]

admin@jarvis:~$ curl -L -X POST 'http://192.168.0.200:8191/v1' -H 'Content-Type: application/json' --data-raw '{
"cmd": "request.get",
"url":"http://www.google.com/",
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleW...",
"maxTimeout": 60000,
"headers": {
"X-Test": "Testing 123..."
}
}'
curl: (7) Failed to connect to 192.168.0.200 port 8191: Connection refused

Could you help me please ?

Thank you

Memory Leak / CPU Leak when calling specific websites

Please use the search bar at the top of the page and make sure you are not creating an already submitted issue.
Check closed issues as well, because your issue may have already been fixed.

Instruction on how to enable debug and html trace

Follow the instructions from this wiki page

Environment

FlareSolverr Version:
Latest

Docker: [yes/no]
yes

OS:
Centos 8.2

Last Working FlareSolverr Version:
None

Are you using a proxy or VPN? [yes/no]
no

Using Captcha Solver: [yse/no]
no

If using captcha solver, which one:

Description

[List steps to reproduce the error and details on what happens and what you expected to happen]

    static void Main(string[] args)
    {
        MainAsync().Wait();
    }

    static async Task MainAsync()
    {
        var handler = new ClearanceHandler("http://-snip-:8191/")
        {
            UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36",
            MaxTimeout = 60000,
        };

        var client = new HttpClient(handler);
        var content = await client.GetStringAsync("http://cfuam.audiograb.net/hit");
        Console.WriteLine(content);
    }

Logged Error Messages

[Place any relevant error messages you noticed from the logs here.]

[Make sure you attach the full logs with your personal information removed in case we need more information]

Screenshots

[Place any screenshots of the issue here if needed]

Screenshot_1

Help Install Yggtorent

Hello,
I installed the module for Jacket with the following method
On Debian10

  • Root / apt install nodejs npm
  • Root / copie fichier FlareSolverr-master .zip Unzzip
  • Root / docker build -t flaresolverr:latest .
  • User / docker run --restart=always --name flaresolverr -p 8191:8191 -d flaresolverr:latest
    Test : {"status":"ok","message":"","startTimestamp":1606038247135,"endTimestamp":1606038260071,"version":"1.0.0","solution":{"url":"https://www.google.com/?gws_rdTYPE html><html itemscope="" itemtype=\http://schema.org/WebPage\

Add API url in jakett : http://192.168.100.***:8191/
Test YggTorent : An error occurred while testing this indexer
Exception (yggcookie): Maximum timeout reached. maxTimeout=60000 (ms): Maximum timeout reached. maxTimeout=60000 (ms)
Click here to open an issue on GitHub for this indexer.

I have the same message for normal Yggtorent

User-Agent detected by CouldFlare (Docker ARM)

FlareSolverr 1.2.0
Docker ARM32

The default User-Agent in ARM is Mozilla/5.0 (X11; Linux armv7l) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.113 Safari/537.36
This User-Agent is detected by CouldFlare.

Steps to reproduce:
Fails:

{
    "cmd": "request.get",
    "url": "https://yggtorrent.si/",
    "maxTimeout": 60000
}

Works:

{
    "cmd": "request.get",
    "url": "https://yggtorrent.si/",
    "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36",
    "maxTimeout": 60000
}

I think we should use the Windows User-Agent when the user doesn't set it.

Configure CI/CD to test Docker images

We are supporting 3 architectures and sometimes Chrome/Chromium is not available in Alpine repositories.
Just add some test to be sure we are not breaking Docker Images when we update Alpine base image.

Publish release in GitHub repository too

We already have GitHub Actions to add tags and publish Docker images.

We should publish releases in this repository too with changelog + source code. In the future include Windows/MacOS installers too.

This allows the users to subscribe to new releases.

We can add a badge in the readme similar to this to show the latest version.
NuGet

Resource temporarily unavailable (flaresolverr:8191) (Need help please)

Environment

FlareSolverr Version:

Docker: yes

OS: Ubuntu

Last Working FlareSolverr Version: latest

Are you using a proxy or VPN? no

Using Captcha Solver: yes

If using captcha solver, which one: hcaptcha-solver

Description

I followed instructions to add flaresolver to my docker-compose.yml and jackett then. However when i try to add yggtorrent indexer, it says the following error message. This might appear simple for you but im learning so i still have some basic difficulties in that areas...Please be tolerant :D
Thank you ! 🙏

Jackett Error Messages

Error connecting to FlareSolverr server: System.Net.Http.HttpRequestException: Resource temporarily unavailable (flaresolverr:8191) ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.g__WaitForConnectWithCancellation|283_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.DefaultConnectAsync(SocketsHttpConnectionContext context, CancellationToken cancellationToken) at System.Net.Http.ConnectHelper.ConnectAsync(Func3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.ConnectAsync(Func3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean asy

Screenshots

[Place any screenshots of the issue here if needed]

Docker-compose.yml :

flaresolverr:
# DockerHub mirror flaresolverr/flaresolverr:latest
image: ghcr.io/flaresolverr/flaresolverr:latest
container_name: flaresolverr
environment:
# Used to change the verbosity of the logging
- LOG_LEVEL=debug
- LOG_HTML=true
# Enables hcaptcha-solver => https://github.com/JimmyLaurent/hcaptcha-solver
- CAPTCHA_SOLVER=hcaptcha-solver
# Enables CaptchaHarvester => https://github.com/NoahCardoza/CaptchaHarvester
#- CAPTCHA_SOLVER=harvester
#- HARVESTER_ENDPOINT=https://127.0.0.1:5000/token
ports:
- 8191:8191
restart: unless-stopped

Use global browser instance, keep cookies, delay exit

Hello,

to improve performance and resource usage, is it possible to:

  • use one browser instance for all requests (only max one instance open at all times)
  • keep existing cookies for some time (so subsequent requests to the same site do not require a new challenge at FlareSolverr level - without setting cookies in the caller)
  • delaying browser exit for a few minutes, to make the above possible

ERROR REQ-1 Could not find browser revision latest

Ubuntu 20.04

2020-06-22T08:30:24.191Z INFO REQ-1 Incoming request: POST /v1
2020-06-22T08:30:24.205Z INFO REQ-1 Params: {"method":"GET","url":"https://www.bteye.org/q/test","userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:77.0) Gecko/20100101 Firefox/77.0","maxTimeout":60000}
2020-06-22T08:30:24.269Z ERROR REQ-1 Could not find browser revision latest. Run "npm install" or "yarn install" to download a browser binary.

2020-06-22 01:30:24 Error

{0} Jackett.Common.IndexerException: Exception (bteye): Could not find browser revision latest. Run "npm install" or "yarn install" to download a browser binary.
---> FlareSolverrSharp.Exceptions.FlareSolverrException: Could not find browser revision latest. Run "npm install" or "yarn install" to download a browser binary.
at FlareSolverrSharp.Solvers.FlareSolverr.<>c__DisplayClass5_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at FlareSolverrSharp.Utilities.SemaphoreLocker.LockAsync[T](Func1 worker) at FlareSolverrSharp.Solvers.FlareSolverr.Solve(HttpRequestMessage request) at FlareSolverrSharp.ClearanceHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Jackett.Common.Utils.Clients.HttpWebClient2NetCore.Run(WebRequest webRequest) in /home/vsts/work/1/s/src/Jackett.Common/Utils/Clients/HttpWebClient2NetCore.cs:line 230
at Jackett.Common.Utils.Clients.WebClient.GetString(WebRequest request) in /home/vsts/work/1/s/src/Jackett.Common/Utils/Clients/WebClient.cs:line 114
at Jackett.Common.Indexers.BaseWebIndexer.RequestStringWithCookies(String url, String cookieOverride, String referer, Dictionary`2 headers) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 443
at Jackett.Common.Indexers.CardigannIndexer.PerformQuery(TorznabQuery query) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 1341
at Jackett.Common.Indexers.BaseIndexer.ResultsForQuery(TorznabQuery query) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 325
--- End of inner exception stack trace ---
at Jackett.Common.Indexers.BaseIndexer.ResultsForQuery(TorznabQuery query) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 345
at Jackett.Common.Indexers.BaseWebIndexer.ResultsForQuery(TorznabQuery query) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 804
at Jackett.Server.Controllers.ResultsController.Results(ApiSearch requestt) in /home/vsts/work/1/s/src/Jackett.Server/Controllers/ResultsController.cs:line 224

Improve logging further

Include a logging option INFO or DEBUG maybe in each captcha provider in the form of (not really sure of the code)

import log from '../log'
...
...
export default async function solve ............................. : Promise<string> {
  log.info(`Solving your Challenge using provider xxxxxx`);
...
...
}

That way we can be sure the solver function is being used to return the chalenge

Docker Compose Overrides or .env variables

Environment

FlareSolverr Version: 1.2.2

Docker: [yes/no] yes

OS: Ubuntu 20.04 x64

Last Working FlareSolverr Version: 1.2.2

Are you using a proxy or VPN? [yes/no] yes

Using Captcha Solver: [yes/no] yes

If using captcha solver, which one: hcaptcha-solver

Description

Can we please add a local folder that is ignored by GIT but allows us to override variables in the docker-compose.yml file?
Other projects I use do this by adding a folder /local/compose with override files and by .env file with variable overrides

for example: .env file:

FLARESOLVERR_HOST=192.168.1.40
FLARESOLVERR_PORT=8191
COMPOSE_FILE=docker-compose.yml:local/compose/memory-settings.yml

The docker-compose.yml file can then have optional variable lookups if they are overriden by the .env file. This will help with updating our dockers but while still keeping our custom settings.

- FLARESOLVERR_HOST=${FLARESOLVERR_HOST:-0.0.0.0}
- FLARESOLVERR_PORT=${FLARESOLVERR_PORT:-8191}

Support for Death by Captcha

Basic Stuff

  • Implement methods that don't use proxy until #26 solved

  • Manage APIKey/credentials

  • hcaptcha cloudflare tests

  • googlecaptcha tests

  • microsoft funcaptcha tests

Better Stuff

  • Could be great if we could send it via the request, so it could be set on the program doing the request
    Might be not doable until something else is done on Flaresolverr to manage this

Documentation

Need to create an account to start working on this later

Refs

Task of #24

Docker Image - Issues with NPM Install

Environment

FlareSolverr Version: 1.2.2

Docker: [yes/no] yes

OS: Ubuntu 20.04 LTS x64

Are you using a proxy or VPN? [yes/no] yes

Using Captcha Solver: [yes/no] yes

If using captcha solver, which one: hcaptcha-solver

Description

FlareSolverr doesnt seem to install all of the packages correctly when using it via docker.

If I login to the shell, docker exec -ti flaresolverr sh, and run npm install from /home/node/flaresolverr the process fails and here is my error log:

cat /home/node/.npm/_logs/2021-01-10T05_32_58_868Z-debug.log
0 verbose cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'run', 'build' ]
1 info using [email protected]
2 info using [email protected]
3 timing config:load:defaults Completed in 1ms
4 timing config:load:file:/usr/local/lib/node_modules/npm/npmrc Completed in 1ms
5 timing config:load:builtin Completed in 1ms
6 timing config:load:cli Completed in 1ms
7 timing config:load:env Completed in 1ms
8 timing config:load:file:/home/node/flaresolverr/.npmrc Completed in 0ms
9 timing config:load:project Completed in 0ms
10 timing config:load:file:/home/node/.npmrc Completed in 0ms
11 timing config:load:user Completed in 1ms
12 timing config:load:file:/usr/local/etc/npmrc Completed in 0ms
13 timing config:load:global Completed in 0ms
14 timing config:load:cafile Completed in 0ms
15 timing config:load:validate Completed in 0ms
16 timing config:load:setUserAgent Completed in 1ms
17 timing config:load:setEnvs Completed in 0ms
18 timing config:load Completed in 6ms
19 verbose npm-session 77da7c1530376504
20 timing npm:load Completed in 15ms
21 timing command:run-script Completed in 165ms
22 verbose stack Error: command failed
22 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/@npmcli/promise-spawn/index.js:64:27)
22 verbose stack     at ChildProcess.emit (node:events:329:20)
22 verbose stack     at maybeClose (node:internal/child_process:1055:16)
22 verbose stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:288:5)
23 verbose pkgid [email protected]
24 verbose cwd /home/node/flaresolverr
25 verbose Linux 5.8.0-36-generic
26 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "build"
27 verbose node v15.2.1
28 verbose npm  v7.0.8
29 error code 1
30 error path /home/node/flaresolverr
31 error command failed
32 error command sh -c tsc
33 verbose exit 1

Support for anti-Captcha.com

Basic Stuff

  • Implement methods that don't use proxy until #26 solved

  • Manage APIKey

  • hcaptcha cloudflare tests

  • googlecaptcha tests

  • microsoft funcaptcha tests

Better Stuff

  • Could be great if we could send it via the request, so it could be set on the program doing the request
    Might be not doable until something else is done on Flaresolverr to manage this

Documentation

Refs

Task of #24

Forking to https://github.com/xfouloux/FlareSolverr branch feature/anticaptcha

FlareSolverr logo

We need a logo but I don't have the skills.

  • Cloud + circular arrows can be fine

Problem of token for captcha solver

Environment

FlareSolverr Version: 1.2.1

Docker: yes

OS: Rasperry Pi 3 - Raspbian

Last Working FlareSolverr Version: 1.2.1

Are you using a proxy or VPN? no

Using Captcha Solver: yes

If using captcha solver, which one: I tried hcaptcha and captcha harveister

Description

Hi @ilike2burnthing and @ngosang
My principal problem is to use FlareSolverr with YGG on Jackett.
I installed a crontab to update my YGG Torznab feed every 30 minutes. (Because I don't use Radarr and Sonarr) Sometimes it's ok but not everytime.

I've installed Flaresolverr with the official stack (without # for hcaptcha or harvester to test 1 or the other)

---
version: "2.1"
services:
  flaresolverr:
    # DockerHub mirror flaresolverr/flaresolverr:latest
    image: ghcr.io/flaresolverr/flaresolverr:latest
    container_name: flaresolverr
    environment:
      # Used to change the verbosity of the logging
      - LOG_LEVEL=info
      # Enables hcaptcha-solver => https://github.com/JimmyLaurent/hcaptcha-solver
      #- CAPTCHA_SOLVER=hcaptcha-solver
      # Enables CaptchaHarvester => https://github.com/NoahCardoza/CaptchaHarvester
      #- CAPTCHA_SOLVER=harvester
      #- HARVESTER_ENDPOINT=https://127.0.0.1:5000/token
    ports:
      - 8191:8191
    restart: unless-stopped 

Logged Error Messages

Without captcha solver :

Jackett.Common.IndexerException: Exception (yggtorrent): FlareSolverr was able to process the request, but a captcha was detected. Message: Captcha detected but no automatic solver is configured.
 ---> System.Exception: Parse error
 ---> FlareSolverrSharp.Exceptions.FlareSolverrException: FlareSolverr was able to process the request, but a captcha was detected. Message: Captcha detected but no automatic solver is configured.
   at FlareSolverrSharp.Solvers.FlareSolverr.<>c__DisplayClass5_0.<<Solve>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at FlareSolverrSharp.Utilities.SemaphoreLocker.LockAsync[T](Func`1 worker)
   at FlareSolverrSharp.Solvers.FlareSolverr.Solve(HttpRequestMessage request)
   at FlareSolverrSharp.ClearanceHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
   at Jackett.Common.Utils.Clients.HttpWebClient2.Run(WebRequest webRequest) in /home/vsts/work/1/s/src/Jackett.Common/Utils/Clients/HttpWebClient2.cs:line 170
   at Jackett.Common.Utils.Clients.WebClient.GetResultAsync(WebRequest request) in /home/vsts/work/1/s/src/Jackett.Common/Utils/Clients/WebClient.cs:line 207
   at Jackett.Common.Indexers.BaseWebIndexer.RequestWithCookiesAsync(String url, String cookieOverride, RequestType method, String referer, IEnumerable`1 data, Dictionary`2 headers, String rawbody, Nullable`1 emulateBrowser) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 496
   at Jackett.Common.Indexers.CardigannIndexer.GetConfigurationForSetup(Boolean automaticlogin) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 913
   at Jackett.Common.Indexers.CardigannIndexer.DoLogin() in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 514
   at Jackett.Common.Indexers.CardigannIndexer.PerformQuery(TorznabQuery query) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 1290
   --- End of inner exception stack trace ---
   at Jackett.Common.Indexers.BaseWebIndexer.OnParseError(String results, Exception ex) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 682
   at Jackett.Common.Indexers.CardigannIndexer.PerformQuery(TorznabQuery query) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 1620
   at Jackett.Common.Indexers.BaseIndexer.ResultsForQuery(TorznabQuery query, Boolean isMetaIndexer) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 390
   --- End of inner exception stack trace ---
   at Jackett.Common.Indexers.BaseIndexer.ResultsForQuery(TorznabQuery query, Boolean isMetaIndexer) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 390
   at Jackett.Common.Indexers.BaseWebIndexer.ResultsForQuery(TorznabQuery query, Boolean isMetaIndexer) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 659
   at Jackett.Server.Controllers.ResultsController.Torznab(TorznabRequest request) in /home/vsts/work/1/s/src/Jackett.Server/Controllers/ResultsController.cs:line 431

With hcaptcha


Exception in GetConfigurationForSetup (yggtorrent): FlareSolverrSharp.Exceptions.FlareSolverrException: FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Token solver failed to return a token.
   at FlareSolverrSharp.Solvers.FlareSolverr.<>c__DisplayClass5_0.<<Solve>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at FlareSolverrSharp.Utilities.SemaphoreLocker.LockAsync[T](Func`1 worker)
   at FlareSolverrSharp.Solvers.FlareSolverr.Solve(HttpRequestMessage request)
   at FlareSolverrSharp.ClearanceHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
   at Jackett.Common.Utils.Clients.HttpWebClient2.Run(WebRequest webRequest) in /home/vsts/work/1/s/src/Jackett.Common/Utils/Clients/HttpWebClient2.cs:line 170
   at Jackett.Common.Utils.Clients.WebClient.GetResultAsync(WebRequest request) in /home/vsts/work/1/s/src/Jackett.Common/Utils/Clients/WebClient.cs:line 184
   at Jackett.Common.Indexers.BaseWebIndexer.RequestWithCookiesAsync(String url, String cookieOverride, RequestType method, String referer, IEnumerable`1 data, Dictionary`2 headers, String rawbody, Nullable`1 emulateBrowser) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 492
   at Jackett.Common.Indexers.CardigannIndexer.GetConfigurationForSetup(Boolean automaticlogin) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 843 at Jackett.Common.Indexers.CardigannIndexer.GetConfigurationForSetup() in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 822

With captcha harvester :

Jackett.Common.IndexerException: Exception (yggtorrent): FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Unable to process browser request. Error: TypeError: Cannot read property 'statusCode' of undefined
 ---> System.Exception: Parse error
 ---> FlareSolverrSharp.Exceptions.FlareSolverrException: FlareSolverr was unable to process the request, please check FlareSolverr logs. Message: Unable to process browser request. Error: TypeError: Cannot read property 'statusCode' of undefined
   at FlareSolverrSharp.Solvers.FlareSolverr.<>c__DisplayClass5_0.<<Solve>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at FlareSolverrSharp.Utilities.SemaphoreLocker.LockAsync[T](Func`1 worker)
   at FlareSolverrSharp.Solvers.FlareSolverr.Solve(HttpRequestMessage request)
   at FlareSolverrSharp.ClearanceHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
   at Jackett.Common.Utils.Clients.HttpWebClient2.Run(WebRequest webRequest) in /home/vsts/work/1/s/src/Jackett.Common/Utils/Clients/HttpWebClient2.cs:line 170
   at Jackett.Common.Utils.Clients.WebClient.GetResultAsync(WebRequest request) in /home/vsts/work/1/s/src/Jackett.Common/Utils/Clients/WebClient.cs:line 207
   at Jackett.Common.Indexers.BaseWebIndexer.RequestWithCookiesAsync(String url, String cookieOverride, RequestType method, String referer, IEnumerable`1 data, Dictionary`2 headers, String rawbody, Nullable`1 emulateBrowser) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 496
   at Jackett.Common.Indexers.CardigannIndexer.GetConfigurationForSetup(Boolean automaticlogin) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 843
   at Jackett.Common.Indexers.CardigannIndexer.DoLogin() in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 514
   at Jackett.Common.Indexers.CardigannIndexer.PerformQuery(TorznabQuery query) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 1290
   --- End of inner exception stack trace ---
   at Jackett.Common.Indexers.BaseWebIndexer.OnParseError(String results, Exception ex) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 682
   at Jackett.Common.Indexers.CardigannIndexer.PerformQuery(TorznabQuery query) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 1620
   at Jackett.Common.Indexers.BaseIndexer.ResultsForQuery(TorznabQuery query, Boolean isMetaIndexer) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 390
   --- End of inner exception stack trace ---
   at Jackett.Common.Indexers.BaseIndexer.ResultsForQuery(TorznabQuery query, Boolean isMetaIndexer) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 390
   at Jackett.Common.Indexers.BaseWebIndexer.ResultsForQuery(TorznabQuery query, Boolean isMetaIndexer) in /home/vsts/work/1/s/src/Jackett.Common/Indexers/BaseIndexer.cs:line 659
   at Jackett.Common.Services.IndexerManagerService.TestIndexer(String name) in /home/vsts/work/1/s/src/Jackett.Common/Services/IndexerManagerService.cs:line 302
   at Jackett.Server.Controllers.IndexerApiController.Test() in /home/vsts/work/1/s/src/Jackett.Server/Controllers/IndexerApiController.cs:line 132
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Jackett.Server.Middleware.CustomExceptionHandler.Invoke(HttpContext httpContext) in /home/vsts/work/1/s/src/Jackett.Server/Middleware/CustomExceptionHandler.cs:line 61

Can you please help ?

Thanks a lot !

The cookies provided by FlareSolverr are not valid

Environment

FlareSolverr Version: 1.2.0

Docker: Yes

OS: Ubuntu 20.04

Are you using a proxy or VPN? No

Using Captcha Solver: Yes

If using captcha solver, which one: hcaptcha-solver

Description

I use Flaresolverr for the tracker yggcookie on jackett. Reading the HTML and Debug log show me that it detects correctly the CLoudfare challenge but Chronium of FlareSolverr have not cookie setting enabled. Same for Javascript. See the log below.

<h1 data-translate="turn_on_js" style="color:#bd2426;">Please turn JavaScript on and reload the page.</h1>

<div id="no-cookie-warning" class="cookie-warning" data-translate="turn_on_cookies" style="display:none">
     <p data-translate="turn_on_cookies" style="color:#bd2426;">Please enable Cookies and reload the page.</p>

Logged Error Messages

Error output from Jackett
Exception (yggcookie): The cookies provided by FlareSolverr are not valid: The cookies provided by FlareSolverr are not valid

Output logs from Flaresolverr :

2020-12-17T10:09:15.764Z INFO REQ-1 Cloudflare detected
2020-12-17T10:09:15.902Z DEBUG REQ-1 <!DOCTYPE html><html lang="en-US"><head>
  <meta charset="UTF-8">
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1">
  <meta name="robots" content="noindex, nofollow">
  <meta name="viewport" content="width=device-width,initial-scale=1">
  <title>Just a moment...</title>
  <style type="text/css">
    html, body {width: 100%; height: 100%; margin: 0; padding: 0;}
    body {background-color: #ffffff; color: #000000; font-family:-apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, "Helvetica Neue",Arial, sans-serif; font-size: 16px; line-height: 1.7em;-webkit-font-smoothing: antialiased;}
    h1 { text-align: center; font-weight:700; margin: 16px 0; font-size: 32px; color:#000000; line-height: 1.25;}
    p {font-size: 20px; font-weight: 400; margin: 8px 0;}
    p, .attribution, {text-align: center;}
    #spinner {margin: 0 auto 30px auto; display: block;}
    .attribution {margin-top: 32px;}
    @keyframes fader     { 0% {opacity: 0.2;} 50% {opacity: 1.0;} 100% {opacity: 0.2;} }
    @-webkit-keyframes fader { 0% {opacity: 0.2;} 50% {opacity: 1.0;} 100% {opacity: 0.2;} }
    #cf-bubbles > .bubbles { animation: fader 1.6s infinite;}
    #cf-bubbles > .bubbles:nth-child(2) { animation-delay: .2s;}
    #cf-bubbles > .bubbles:nth-child(3) { animation-delay: .4s;}
    .bubbles { background-color: #f58220; width:20px; height: 20px; margin:2px; border-radius:100%; display:inline-block; }
    a { color: #2c7cb0; text-decoration: none; -moz-transition: color 0.15s ease; -o-transition: color 0.15s ease; -webkit-transition: color 0.15s ease; transition: color 0.15s ease; }
    a:hover{color: #f4a15d}
    .attribution{font-size: 16px; line-height: 1.5;}
    .ray_id{display: block; margin-top: 8px;}
    #cf-wrapper #challenge-form { padding-top:25px; padding-bottom:25px; }
    #cf-hcaptcha-container { text-align:center;}
    #cf-hcaptcha-container iframe { display: inline-block;}
  </style>

    <meta http-equiv="refresh" content="12">
<script type="text/javascript">
  //<![CDATA[
  (function(){
    

    window._cf_chl_enter = function(){window._cf_chl_opt.p=1};
    
    var a = function() {try{return !!window.addEventListener} catch(e) {return !1} },
    b = function(b, c) {a() ? document.addEventListener("DOMContentLoaded", b, c) : document.attachEvent("onreadystatechange", b)};
    b(function(){
      var cookiesEnabled=(navigator.cookieEnabled)? true : false;
      var cookieSupportInfix=cookiesEnabled?'/nocookie':'/cookie';
      var a = document.getElementById('cf-content');a.style.display = 'block';
      var isIE = /(MSIE|Trident\/|Edge\/)/i.test(window.navigator.userAgent);
      var trkjs = isIE ? new Image() : document.createElement('img');
      trkjs.setAttribute("src", "/cdn-cgi/images/trace/jschal/js"+cookieSupportInfix+"/transparent.gif?ray=602fd8f7eced0b4f");
      trkjs.id = "trk_jschal_js";
      trkjs.setAttribute("alt", "");
      document.body.appendChild(trkjs);
      
      var cpo = document.createElement('script');
      cpo.type = 'text/javascript';
      cpo.src = "/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1";
      var done = false;
      cpo.onload = cpo.onreadystatechange = function() {
        if (!done && (!this.readyState || this.readyState === "loaded" || this.readyState === "complete")) {
          done = true;
          cpo.onload = cpo.onreadystatechange = null;
          window._cf_chl_enter()
        }
      };
      document.getElementsByTagName('head')[0].appendChild(cpo);
    
    }, false);
  })();
  //]]>
</script>


<script type="text/javascript" src="/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1"></script></head>
<body>
  <table width="100%" height="100%" cellpadding="20">
    <tbody><tr>
      <td align="center" valign="middle">
          <div class="cf-browser-verification cf-im-under-attack">
  <noscript>
    <h1 data-translate="turn_on_js" style="color:#bd2426;">Please turn JavaScript on and reload the page.</h1>
  </noscript>
  <div id="cf-content" style="display: block;">
    <a href="https://thing-dot.com/bearded.php?cfm=8694" style="position: absolute; top: -250px; left: -250px;"></a>
    <div id="cf-bubbles">
      <div class="bubbles"></div>
      <div class="bubbles"></div>
      <div class="bubbles"></div>
    </div>
    <h1><span data-translate="checking_browser">Checking your browser before accessing</span> yggtorrent.si.</h1>
    
    <div id="no-cookie-warning" class="cookie-warning" data-translate="turn_on_cookies" style="display:none">
      <p data-translate="turn_on_cookies" style="color:#bd2426;">Please enable Cookies and reload the page.</p>
    </div>
    <p data-translate="process_is_automatic">This process is automatic. Your browser will redirect to your requested content shortly.</p>
    <p data-translate="allow_5_secs">Please allow up to 5 seconds…</p>
  </div>
   
  <form class="challenge-form" id="challenge-form" action="/engine/search?category=all&amp;name=&amp;description=&amp;file=&amp;uploader=&amp;sub_category=&amp;do=search&amp;order=desc&amp;sort=publish_date&amp;__cf_chl_jschl_tk__=XXXXXX" method="POST" enctype="application/x-www-form-urlencoded">
    <input type="hidden" name="XXXXXX" 
	<input type="hidden" value="XXXXXX" id="jschl-vc" name="XXXXXX">
    <!-- <input type="hidden" value="" id="XXXXXX" name="XXXXXX"/> -->
    <input type="hidden" name="pass" value="XXXXXX">
    <input type="hidden" id="jschl-answer" name="jschl_answer">
  </form>
  
  <div id="trk_jschal_nojs" style="background-image:url('/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=XXXXXX')"> </div>
</div>

          
          <div class="attribution">
            DDoS protection by <a rel="noopener noreferrer" href="https://www.cloudflare.com/5xx-error-landing/" target="_blank">Cloudflare</a>
            <br>
            <span class="ray_id">Ray ID: <code>XXXXXX</code></span>
          </div>
      </td>
     
    </tr>
  </tbody></table>


<img src="/cdn-cgi/images/trace/jschal/js/nocookie/transparent.gif?ray=XXXXXX" id="trk_jschal_js" alt=""></body></html>
2020-12-17T10:09:15.907Z DEBUG REQ-1 Waiting for Cloudflare challenge...
2020-12-17T10:09:22.061Z DEBUG REQ-1 Validating HTML code...
2020-12-17T10:09:22.197Z INFO REQ-1 Successful response in 9.684 s

How to run flaresolverr in background & auto start like jackett?

Please use the search bar at the top of the page and make sure you are not creating an already submitted issue.
Check closed issues as well, because your issue may have already been fixed.

Environment

FlareSolverr Version:

Docker: [yes/no]

OS: windows 10

Last Working FlareSolverr Version:

Are you using a proxy or VPN? [no]

Using Captcha Solver: [not yet]

If using captcha solver, which one:

Description

can we run flaresolverr in start up & background process? if possible, give us tutorial
without having run flaresolverr.exe

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.