searxng / searxng

SearXNG is a free internet metasearch engine which aggregates results from various search services and databases. Users are neither tracked nor profiled.

---

---

Privacy-respecting, hackable metasearch engine

Searx.space lists ready-to-use running instances.

A user, admin and developer handbook is available on the homepage.

---

Contact

Ask questions or just chat about SearXNG on

IRC

#searxng on libera.chat which is bridged to Matrix.

Matrix

#searxng:matrix.org

Setup

• A well maintained Docker image, also built for ARM64 and ARM/v7 architectures.
• Alternatively there are up to date installation scripts.
• For individual setup consult our detailed Step by step instructions.
• To fine-tune your instance, take a look at the Administrator documentation.

Translations

Help translate SearXNG at Weblate

Contributing

Are you a developer? Have a look at our development quickstart guide, it's very easy to contribute. Additionally we have a developer documentation.

Codespaces

You can contribute from your browser using GitHub Codespaces:

• Fork the repository
• Click on the <> Code green button
• Click on the Codespaces tab instead of Local
• Click on Create codespace on master
• VSCode is going to start in the browser
• Wait for git pull && make install to appear and then disappear
• You have 120 hours per month (see also your list of existing Codespaces)
• You can start SearXNG using make run in the terminal or by pressing Ctrl+Shift+B

searxng's Issues

manage / Makefile: PYLINT_FILES PR conflicts

The PYLINT_FILES variable conflicts when it is modified:

• a PR based on a master commit
• the PR changes PYLINT_FILES
• a new commit on master changes PYLINT_FILES
• now there is a conflict

Would it make sense

• to flag files with a tag inside their contents. For example # manage: enable=pylint
• then git grep can find them.
  ?

So

PYLINT_TRIGGER = "# manage: enable=pylint"
PYLINT_FILES = $(grep -l -r --include \*.py "${PYLINT_TRIGGER}" searx searx_utils)

could replace

searxng/manage

Lines 22 to 44 in 0603b04

	PYLINT_FILES=(
	searx/preferences.py
	searx/testing.py
	searx/engines/gigablast.py
	searx/engines/deviantart.py
	searx/engines/digg.py
	searx/engines/google.py
	searx/engines/google_news.py
	searx/engines/google_videos.py
	searx/engines/google_images.py
	searx/engines/mediathekviewweb.py
	searx/engines/meilisearch.py
	searx/engines/solidtorrents.py
	searx/engines/solr.py
	searx/engines/springer.py
	searx/engines/google_scholar.py
	searx/engines/yahoo_news.py
	searx/engines/apkmirror.py
	searx/engines/artic.py
	searx/engines/core.py
	searx_extra/update/update_external_bangs.py
	searx/metrics/__init__.py
	)

About pylint configuration

Here a command to get some statistics:
$ git grep "pylint: disable=" | cut -f2 -d\# | cut -f2 -d\= | tr , '\n' | sed 's/^ *//' | sort | uniq -c | sort -nr

• cut -f2 -d\#: the comment at the end of line (can be broken)
• cut -f2 -d\= : after pylint: disable=
• tr , '\n' : split coma separated values into multi lines
• sed 's/^ *//': trim spaces

Result:

     37 missing-function-docstring
     13 no-member
      9 invalid-name
      9 global-statement
      9 assigning-non-slot
      7 undefined-variable
      7 missing-module-docstring
      7 broad-except
      6 NOQA 
      6 attribute-defined-outside-init
      4 unused-import
      4 E1101
      3 protected-access
      3 no-self-use
      3 consider-using-with
      2 W0102
      2 unused-argument
      2 too-many-arguments
      2 C0116
      2 access-member-before-definition
      1 wrong-import-position
      1 W0223  
      1 useless-object-inheritance
      1 too-many-boolean-expressions
      1 stop-iteration-return
      1 see https://github.com/encode/httpx/blob/e05a5372eb6172287458b37447c30f650047e1b8/httpx/_transports/default.py
      1 R0903
      1 pointless-string-statement
      1 no-name-in-module
      1 missing-class-docstring
      1 inconsistent-return-statements
      1 import-outside-toplevel)
      1 from https://github.com/encode/httpcore/blob/4b662b5c42378a61e54d673b4c949420102379f5/httpcore/_backends/asyncio.py
      1 E0611
      1 E0401 C0413
      1 E0401
      1 C0301
      1 arguments-differ

.pylintrc to update (?)

• missing-function-docstring: 17 occurrences for searx.engines.* . Currently the pattern # pylint: disable=missing-function-docstring at the top of the file doesn't help in my opinion. Documentation about the request and response functions doesn't help. But documentation about custom functions can help. It is only my opinion.
• unused-import: all related to import _fetch_supported_languages, supported_languages_url: I would disabled unused-import for searx.engines.* or find a way to tell pylint that _fetch_supported_languages, supported_languages_url are actually used.
• global-statement: do we really need a reminder about global variable usage?

actual issue:

• invalid-name is actually not required in searx.engines.* (7 occurrences / 9).
• no-member: related to preferences.py (see constructor)
• assigning-non-slot: all related to request.something = , actually it should be g.something =.

manage script issue:

• undefined-variable is not required since there is PYLINT_ADDITIONAL_BUILTINS_FOR_ENGINES, but for some reason it is ignored: it is a bug.

Remove themes

Currently there are two themes: oscar and simple.

It is possible to choose at runtime which themes to use.

This issue suggests:

• to remove this choice to keep only one theme.
• keep the settings static_path and templates_path, so it is possible to another runtime at setup time, not runtime.

Why:

• the plugins and themes conflict each other:
  • looking at the source code of plugins https://github.com/searxng/searxng/tree/master/searx/static/plugins/js
  • everything is wired for the oscar: the plugins except jquery + the css classes from oscar
• it duplicates the maintenance: for example #136 despite its title, it is not exactly a copy of the simple layout to oscar (the "Special Queries" is new and should be back ported).
• we can focus one theme and make it right.

[EDIT] Remove the themes but the styles: light, dark (and we add "auto" using the browser settings).

Keep the user preferences consistent even if settings.yml change.

Is your feature request related to a problem? Please describe.
Currently the preferences is a diff to settings.yml : if an new engine is enabled by default in settings.yml, it will be enabled whatever the user preferences.

Describe the solution you'd like
I expect the engine to remain disabled

A technically solution: the disabled_engines and enabled_engines cookies are replaced by a engines cookie, which list all enabled engines (even if there are enabled by default). Since this list is exhaustive, it makes the preferences consistent even if settings.yml changes.

A possible format:
<category1>,<engine name1>,<engine name2>|<category2>,<engine name1>,<engine name3>....

For example:
general,google,bing|images,google images

As an alternative it is possible to store the engine shortcut instead of the name.

Of course it breaks the compatibility with existing cookies preferences: we can add some code that migrate the existing cookies to the new format for a time.

Describe alternatives you've considered

Additional context

searxng/searx/preferences.py

Lines 282 to 291 in 4f0d232

	def _post_init(self):
	super()._post_init()
	transformed_choices = []
	for engine_name, engine in self.choices.items(): # pylint: disable=no-member,access-member-before-definition
	for category in engine.categories:
	transformed_choice = dict()
	transformed_choice['default_on'] = not engine.disabled
	transformed_choice['id'] = '{}__{}'.format(engine_name, category)
	transformed_choices.append(transformed_choice)
	self.choices = transformed_choices

searxng/searx/preferences.py

Lines 454 to 455 in 4f0d232

	self.engines.parse_cookie((input_data.get('disabled_engines', ''),
	input_data.get('enabled_engines', '')))

searxng/searx/preferences.py

Lines 236 to 241 in 4f0d232

	def parse_cookie(self, data): # pylint: disable=missing-function-docstring
	# pylint: disable=attribute-defined-outside-init
	if data[DISABLED] != '':
	self.disabled = set(data[DISABLED].split(','))
	if data[ENABLED] != '':
	self.enabled = set(data[ENABLED].split(','))

searxng/searx/preferences.py

Lines 264 to 269 in 4f0d232

	def get_disabled(self): # pylint: disable=missing-function-docstring
	disabled = self.disabled
	for choice in self.choices: # pylint: disable=no-member
	if not choice['default_on'] and choice['id'] not in self.enabled:
	disabled.add(choice['id'])
	return self.transform_values(disabled)

The enabled_engines cookie seems ignored:

searxng/searx/webadapter.py

Line 216 in 4f0d232

disabled_engines = preferences.engines.get_disabled()

searxng/searx/webadapter.py

Lines 154 to 160 in 4f0d232

	def get_engineref_from_category_list(category_list: List[str], disabled_engines: List[str]) -> List[EngineRef]:
	result = []
	for categ in category_list:
	result.extend(EngineRef(engine.name, categ)
	for engine in categories[categ]
	if (engine.name, categ) not in disabled_engines)
	return result

---

Reading

searxng/searx/webadapter.py

Line 18 in 6c01145

def validate_engineref_list(engineref_list: List[EngineRef], preferences: Preferences)\

We can imagine that Preferences class provides engines to return all available engines:

• enabled by the user
• with a valid token

It can also provides a categories property.

So the webadapter.py, autocomplete.py can rely on a list of enabled engines instead of dealing with exceptions.

BTW: autocomplete.py doesn't check for the token validity.

Error in translation license

https://weblate.bubu1.eu/projects/searxng/searxng/#information says "GPLv3-only", whereas
many files in the source code repo are "or any later version"

searxng/searx/version.py

Line 6 in 6c01145

(at your option) any later version.

Ostensibly that means (?) current translations are also GPLv3+.
In any event, going or-later for everything causes fewer problems in that it is compatible, and has the most flexibility down the road.

remove of bashisms from shell scripts

Most of our shell scripts are written for bash.

I would suggest to write your scripts on dash shell [2]. Dash is mostly compliant to POSIX standard and pre installed on tiny (docker) images (d6c321b).

I add this issue to monitor the "remove of bashisms", comments and remarks are welcome. Further read:

• "How to make bash scripts work in dash" [1]
• "Dash shell Vs Bash shell" [2]
• "POSIX Shell Command Language" [3]

searxng/Makefile

Lines 56 to 68 in e0af67f

	test.shell:
	$(Q)shellcheck -x -s dash \
	dockerfiles/docker-entrypoint.sh
	$(Q)shellcheck -x -s bash \
	utils/brand.env \
	./manage \
	utils/lib.sh \
	utils/filtron.sh \
	utils/searx.sh \
	utils/morty.sh \
	utils/lxc.sh \
	utils/lxc-searx.env \
	.config.sh

[1] https://mywiki.wooledge.org/Bashism
[2] https://raviminnikantitechie.wordpress.com/2018/06/07/dash-shell-and-differences-with-bash/
[3] https://raviminnikantitechie.wordpress.com/2018/05/03/posix-shell-command-language/

docker environment variable INSTANCE_NAME not work due to wrong default instance name assumption

We substitute instance name with environment variable INSTANCE_NAME here

searxng/dockerfiles/docker-entrypoint.sh

Lines 70 to 74 in 332e3a2

	sed -i -e "s|base_url : False|base_url : ${BASE_URL}|g" \
	-e "s/instance_name : \"searx\"/instance_name : \"${INSTANCE_NAME}\"/g" \
	-e "s/autocomplete : \"\"/autocomplete : \"${AUTOCOMPLETE}\"/g" \
	-e "s/ultrasecretkey/$(openssl rand -hex 32)/g" \
	"${CONF}"

, which assumes the default configuration has line containing instance_name : "searx" . This is not true for searxng

searxng/searx/settings.yml

Line 3 in 332e3a2

instance_name : "searxng" # displayed name

. I think it is very brittle to use sed to substitute yaml file. Can we use something like yq to update settings.yml. This allows users to update more configurations in a more general and more reliable way.

Support anaconda.org

Working URL to the engine
https://anaconda.org/search?q={search-term}

Why do you want to add this engine?
One can use conda to search for Python packages similar to PyPI.

Search for much more python packages than PyPI
Features of this engine

Search Python packages without using the CLI version of conda

How can Searx fetch the information from this engine?

Additional context

Maybe include this search engine in the IT and/or General section.

Musixmatch request for "searxing" song lyrics

Working URL to the engine
https://www.musixmatch.com/search/%s

Why do you want to add this engine?
It can help get lyrics for songs (Google and Bing have these already but it doesn't seem to work on SearXNG

Features of this engine

How can Searx fetch the information from this engine?

Can be placed in the General category

oscar's logicodev-dark (less) compiled into logicodev.css and vice versa?

I had a look at oscar's gruntfile.js and was wondering why dark theme (logicodev-dark/oscar.less) is compiled into light CSS (css/logicodev.css) and vice versa .. does anyone know the reason for .. or did I missed something?

searxng/searx/static/themes/oscar/gruntfile.js

Lines 99 to 100 in fe3831d

	"css/logicodev.css": "src/less/logicodev-dark/oscar.less",
	"css/logicodev-dark.css": "src/less/logicodev/oscar.less"

searx_extra/update/update_languages.py : crash on peertube engine

The URL https://peer.tube/api/v1/videos/languages doesn't exist anymore:

searxng/searx/engines/peertube.py

Lines 24 to 25 in ee83c99

	base_url = "https://peer.tube"
	supported_languages_url = base_url + "/api/v1/videos/languages"

See also https://framagit.org/framasoft/peertube/search-index/-/tree/master/server/controllers/api

---

See https://framagit.org/framasoft/peertube/search-index/-/blob/master/client/src/views/Search.vue#L644-667

Discussion: change category order / engine categories

Category order

Change the default category order to something more usual:

1. General
2. Images
3. Videos
4. News
5. Map
6. Music
7. IT
8. Science
9. Files
10. Social media

Application category

Perhaps create an application category:

• apk mirror
• fdroid
• google play apps
• openrepos

Currently, these engines are in the Files category.

Video category

Remove bittorrent engines:

• 1337x
• btdigg
• kickass
• nyaa
• piratebay
• tokyotoshokan
• torrentz
• yggtorrent

Music category

Remove bittorrent engines:

• btdigg
• kickass
• nyaa
• piratebay
• tokyotoshokan
• torrentz
• yggtorrent

[Feature Request] include Google's “in this site” feature on search results

Bug: duckduckgo engine

Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG

How did you install SearXNG?

no

What happened?

ddg engine isn't working

How To Reproduce

any search

Expected behavior

error message

Screenshots & Logs

Additional context

Technical report

Error

• Error: httpx.TimeoutException
• Percentage: 0
• Parameters: (None, None, None)
• File name: searx/engines/duckduckgo.py:93
• Function: response
• Code: get(url_ping, headers=headers_ping)

Error

• Error: timeout
• Percentage: 0
• Parameters: ()
• File name: searx/search/processors/abstract.py:99
• Function: handle_exception
• Code: count_error(self.engine_name, exception_or_message)

Error

• Error: httpx.ConnectTimeout
• Percentage: 0
• Parameters: (None, None, 'html.duckduckgo.com')
• File name: searx/search/processors/online.py:99
• Function: _send_http_request
• Code: response = req(params['url'], **request_args)

Error

• Error: searx.exceptions.SearxEngineAccessDeniedException
• Percentage: 0
• Parameters: ('HTTP error 403',)
• File name: searx/search/processors/online.py:99
• Function: _send_http_request
• Code: response = req(params['url'], **request_args)

Remove timeout per engine

Currently there is one time per engine:

• either the default one: outgoing.request_timeout
• either the one specified in the engine settings.

Scenario:

• an user queries an engine with a bang
• --> timeout: the user don't have any answer, just timeout
• the user retries
• --> engine suspended
• the user retries
• --> may be an answer

In the end 3 user requests have been sent and 2 HTTP requests to the engine.

If the user is administrator of the instance, the documented solution is:

• to stop the application
• change the settings
• restart

.... or to use a workaround: send the request with another engine with a high timeout: since the global timeout is the max of all timeout it "works".

So the issue suggests to remove the timeout per engine and let the user decides how long to wait.

It can be:

• either a global user timeout
• either a timeout per category.

that the user sets in the preferences page.

settings.yml would contain only a max timeout (something like 10 seconds or 20)

Also see this undocumented feature: searx/searx#1640

It can be shown in the preferences with a vertical red line:

in this example:

• the timeout column has been removed (the last column is the reliability)
• we can see that according to the previous response time, engines won't timeout expect startpage in some rare occasions.

add cloudflare connecting ip to self info plugin

Is your feature request related to a problem? Please describe.

it doesn't show the correct ip behind cloudflare

Describe the solution you'd like

add Cf-Connecting-Ip header to the list of headers that the self info plugin uses when the query is ip

Describe alternatives you've considered

there is another header True-Client-IP in cloudflare enterprise plan but the Cf-Connecting-Ip header is always active

Additional context

none required

Make HTML templates more CSP compliant

What happened?

We have a lot of not CSP compliant style='...' attributes (and element) in our HTML templates

This single style element is not CSP compliant:

searxng/searx/templates/oscar/base.html

Lines 39 to 44 in d36adfa

	<style type="text/css">
	.tab-content > .active_if_nojs, .active_if_nojs {display: block !important; visibility: visible !important;}
	.margin_top_if_nojs {margin-top: 20px;}
	.hide_if_nojs {display: none !important;overflow: hidden !important;}
	.disabled_if_nojs {pointer-events: none; cursor: default; text-decoration: line-through;}
	</style>

But there is also a lot of style='...' attributes usage ...

update:: with patches from this PR #81 applied, the following remain ...

./oscar/preferences.html:25:<td class="{{ label }}" style="padding: 2px">{{- "" -}}
./oscar/preferences.html:29:        <span style="width: calc(max(2px, 100%*{{ (stats[engine_name].time / max_rate95)|round(3) }}))" class="stacked-bar-chart-median"></span>{{- "" -}}
./oscar/preferences.html:30:        <span style="width: calc(100%*{{ ((stats[engine_name].rate80 - stats[engine_name].time) / max_rate95)|round(3) }})" class="stacked-bar-chart-rate80"></span>{{- "" -}}
./oscar/preferences.html:31:        <span style="width: calc(100%*{{ ((stats[engine_name].rate95 - stats[engine_name].rate80) / max_rate95)|round(3) }})" class="stacked-bar-chart-rate95"></span>{{- "" -}}
./oscar/preferences.html:284:                                    <th scope="col" style="width: 10rem">{{ _("Selected language") }}</th>
./oscar/preferences.html:285:                                    <th scope="col" style="width: 10rem">{{ _("SafeSearch") }}</th>
./oscar/preferences.html:286:                                    <th scope="col" style="width: 10rem">{{ _("Time range") }}</th>
./oscar/preferences.html:288:                                    <th scope="col" class="text-right"  style="width: 7rem">{{ _("Max time") }}</th>
./oscar/preferences.html:289:                                    <th scope="col" class="text-right" style="width: 7rem">{{ _("Reliablity") }}</th>
./oscar/result_templates/default.html:19:        <img src="{{ image_proxify(result.img_src or result.thumbnail) }}" title="{{ result.title|striptags }}" style="width: auto; max-height: 60px; min-height: 60px;" class="col-xs-2 col-sm-4 col-md-4 result-content">
./oscar/result_templates/map.html:64:        <div style="height:300px; width:100%; margin: 10px 0;" id="osm-map-{{ index }}"></div>
./oscar/result_templates/files.html:24:<img src="{{ image_proxify(result.img_src) }}" alt="{{ result.title|striptags }}" title="{{ result.title|striptags }}" style="width: auto; max-height: 60px; min-height: 60px;" class="col-xs-2 col-sm-4 col-md-4 result-content">
./simple/preferences.html:39:<td class="{{ label }}" style="padding: 2px; width: 13rem;">{{- "" -}}
./simple/preferences.html:43:        <span style="width: calc(max(2px, 100%*{{ (stats[engine_name].time / max_rate95)|round(3) }}))" class="stacked-bar-chart-median"></span>{{- "" -}}
./simple/preferences.html:44:        <span style="width: calc(100%*{{ ((stats[engine_name].rate80 - stats[engine_name].time) / max_rate95)|round(3) }})" class="stacked-bar-chart-rate80"></span>{{- "" -}}
./simple/preferences.html:45:        <span style="width: calc(100%*{{ ((stats[engine_name].rate95 - stats[engine_name].rate80) / max_rate95)|round(3) }})" class="stacked-bar-chart-rate95"></span>{{- "" -}}
./simple/preferences.html:77:    <div class="engine-tooltip" style="right: 12rem;" role="tooltip" id="{{engine_name}}_reliability">
./simple/preferences.html:258:     <p class="text-muted" style="margin:20px 0;">
./simple/result_templates/map.html:65:    <div id="result-map-{{ index }}" class="invisible"><div id="osm-map-{{ index }}" style="height:300px; width:100%; margin: 10px 0;" ></div></div>

Related: 70cc196

Pagination is broken for Google Images engine

Version of Searx, commit number if you are using on master branch and stipulate if you forked Searx
Commit number: 056550d523ddf4e06821957e13623fc322cf4675

How did you install Searx?
Manually with python -m searx.webapp

What happened?
The Google Images engine doesn't have multiple pages, it only has one page. In contrary to Bing images which have multiple pages and the button "next page" is available.

How To Reproduce

1. !goi test
2. No "next page" button.

Expected behavior
Have multiple pages with "next page" button available.

Screenshots & Logs

Google images	Bing images

Additional context

[discussion] about yaml files: syntax, indentation, lint ...

indentation

The suggested syntax for YAML files is to use 2 spaces for indentation. We use sometimes 2 and sometimes 4 spaces, even in the same file.

• For the future I would suggest to normalize YAML files to have 2 spaces for indentation.

markup

We use PyYAML which supports YAML 1.1 syntax refcard

lint

It might be worth to have a look at a lint tool for our CI?

• https://github.com/adrienverge/yamllint

Add ability to disable RSS/JSON/CSV

Is your feature request related to a problem? Please describe.
Bots are the only users of this feature on a public instance, and they abuse it too much that the engines rate limit pretty quickly the IP address of the instance.

Describe the solution you'd like
I would like to disable the results in RSS/JSON/CSV format.

Describe alternatives you've considered
Using filtron in order to block the use of the RSS/JSON/CSV.

Additional context

Bug: invidious engine

Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG

How did you install SearXNG?

What happened?

How To Reproduce

Expected behavior

Screenshots & Logs

Additional context

Technical report

Error

• Error: searx.exceptions.SearxEngineAccessDeniedException
• Percentage: 10
• Parameters: ('HTTP error 403',)
• File name: searx/search/processors/online.py:99
• Function: _send_http_request
• Code: response = req(params['url'], **request_args)

Checker

• paging: thumbnail URL is invalid,

I was using searx.be instance

Move functions from utils/lib.sh to utils/lib_install.sh

Some functions from utils/lib.sh are related to the installation scripts ( utils/searx.sh, utils/morty.sh, utils/filtron.sh, utils/lxc.sh ).

To quote some, but not all:

• cache_download
• install_template
• go_is_available
• install_go
• service_account_is_available
• drop_service_account
• interactive_shell
• systemd_install_service
• systemd_remove_service
• systemd_activate_service
• systemd_deactivate_service
• systemd_restart_service
• nginx_distro_setup
• install_nginx
• nginx_is_installed
• nginx_reload
• nginx_install_app
• nginx_include_apps_enabled
• nginx_remove_app
• nginx_enable_app
• nginx_dissable_app
• apache_distro_setup
• install_apache
• apache_is_installed
• apache_reload
• apache_install_site
• apache_remove_site
• apache_enable_site
• apache_dissable_site

Would it make sense to move them to utils/lib_install.sh since all installation scripts have these pattern:

# shellcheck source=utils/lib.sh
source "$(dirname "${BASH_SOURCE[0]}")/lib.sh"
# shellcheck source=utils/lib_install.sh
source "${REPO_ROOT}/utils/lib_install.sh"

But instead of a huge utils/lib_install.sh file, perhaps it makes sense to create utils/lib_apache.sh, lib_nginx.sh etc... ?

No functional change, but utils/lib.sh could focus one thing: a helper for ./manage and other scripts.

Add ability to remove "proxy" button

Is your feature request related to a problem? Please describe.
I don't want people to use the proxy ability to browse on the web using my IPs, but I still want to proxy the resources like the images.

Describe the solution you'd like
The ability to disable the "proxy" button next to each result.

Describe alternatives you've considered
Override the macros file of each theme.

Additional context

UnicodeEncodeError for preferences url

Version of Searx

$ git log -1
commit f3e56836d6481f9807ac232b71c584c0535de783 (HEAD -> master, origin/master, origin/HEAD)
Merge: f4da4baf 0ef6aa51
Author: Alexandre Flament <[email protected]>
Date:   Mon Jun 21 20:12:07 2021 +0200

    Merge pull request #165 from return42/patch-google
    
    improve & document google engine

How did you install Searx?
Executing the python file without docker.

What happened?
I tried to use the preferences url but it didn't work.

How To Reproduce
Visiting url should work without 500-ing.

Expected behavior
Using the preferences URL from https://searx.be/preferences should work

Screenshots & Logs

uwsgi[3504]: ERROR:searx.webapp:Exception on / [GET]
uwsgi[3504]: Traceback (most recent call last):
uwsgi[3504]:   File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app
uwsgi[3504]:     response = self.full_dispatch_request()
uwsgi[3504]:   File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request
uwsgi[3504]:     rv = self.handle_user_exception(e)
uwsgi[3504]:   File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception
uwsgi[3504]:     reraise(exc_type, exc_value, tb)
uwsgi[3504]:   File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise
uwsgi[3504]:     raise value
uwsgi[3504]:   File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 1948, in full_dispatch_request
uwsgi[3504]:     rv = self.preprocess_request()
uwsgi[3504]:   File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 2242, in preprocess_request
uwsgi[3504]:     rv = func()
uwsgi[3504]:   File "/usr/local/searx/searx-src/searx/webapp.py", line 541, in pre_request
uwsgi[3504]:     preferences.parse_encoded_data(request.form['preferences'])
uwsgi[3504]:   File "/usr/local/searx/searx-src/searx/preferences.py", line 442, in parse_encoded_data
uwsgi[3504]:     for x, y in parse_qs(decoded_data).items():
uwsgi[3504]:   File "/usr/lib/python3.9/urllib/parse.py", line 701, in parse_qs
uwsgi[3504]:     pairs = parse_qsl(qs, keep_blank_values, strict_parsing,
uwsgi[3504]:   File "/usr/lib/python3.9/urllib/parse.py", line 775, in parse_qsl
uwsgi[3504]:     value = _coerce_result(value)
uwsgi[3504]:   File "/usr/lib/python3.9/urllib/parse.py", line 108, in _encode_result
uwsgi[3504]:     return obj.encode(encoding, errors)
uwsgi[3504]: UnicodeEncodeError: 'ascii' codec can't encode character '\u0142' in position 1045: ordinal not in range(128)

Additional context
I don't own searx.be but actually hosted my own private SearxNG instance. However the issue is present in searx.be as well

It's possible to get Google results even when getting blocked by Google Recaptcha.

On the mobile UI of Google Search, the button More results is not affected by Google rate limiting and I can still do requests while I'm actively blocked by the original Google search. You can even fetch the results from the first page by modifying the request.

The results are given in strange raw data, but you can extract the needed HTML code, which seems to be the same code as the one from Google search if you fake the user agent to a desktop browser.

RAW response data: https://gist.github.com/unixfox/0cb7eebd3add42bfcbf42ea29a063b89#file-raw-txt

HTML manually parsed from the RAW response data: https://gist.github.com/unixfox/0cb7eebd3add42bfcbf42ea29a063b89#file-beautifier-html

Here is how to do requests:

• URL: https://www.google.com/search?vet=12ahUKEwjE4O6xoajxAhWL_KQKHVCLBKoQxK8CegQIAhAG..i&ved=2ahUKEwjE4O6xoajxAhWL_KQKHVCLBKoQqq4CegQIAhAI&yv=3&q=test&prmd=vmin&ei=c0fQYITbBIv5kwXQlpLQCg&start=0&sa=N&asearch=arc&async=arc_id:srp_510,ffilt:all,ve_name:MoreResultsContainer,next_id:srp_5,use_ac:true,_id:arc-srp_510,_pms:qs,_fmt:pc
• Headers:
  • user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0
  • accept: */*
  • sec-fetch-site: same-origin
  • sec-fetch-mode: cors
  • sec-fetch-dest: empty
  • referer: https://www.google.com/
  • accept-encoding: gzip, deflate
  • accept-language: en-US,en;q=0.9

The query parameters are very similar to the ones from the original Google search. Not all query parameters are required, and some can be omitted. That's also the case of the headers.

@return42 Any idea how we could implement that in Searx?

settings.yml: remove locales setting

This issue suggest to remove this part of settings.yml:

searxng/searx/settings.yml

Lines 1659 to 1704 in ed0db4d

	locales:
	en: English
	ar: العَرَبِيَّة (Arabic)
	bg: Български (Bulgarian)
	bo: བོད་སྐད་ (Tibetian)
	ca: Català (Catalan)
	cs: Čeština (Czech)
	cy: Cymraeg (Welsh)
	da: Dansk (Danish)
	de: Deutsch (German)
	el_GR: Ελληνικά (Greek_Greece)
	eo: Esperanto (Esperanto)
	es: Español (Spanish)
	et: Eesti (Estonian)
	eu: Euskara (Basque)
	fa_IR: (fārsī) فارسى (Persian)
	fi: Suomi (Finnish)
	fil: Wikang Filipino (Filipino)
	fr: Français (French)
	gl: Galego (Galician)
	he: עברית (Hebrew)
	hr: Hrvatski (Croatian)
	hu: Magyar (Hungarian)
	ia: Interlingua (Interlingua)
	it: Italiano (Italian)
	ja: 日本語 (Japanese)
	lt: Lietuvių (Lithuanian)
	nl: Nederlands (Dutch)
	nl_BE: Vlaams (Dutch_Belgium)
	oc: Lenga D'òc (Occitan)
	pl: Polski (Polish)
	pt: Português (Portuguese)
	pt_BR: Português (Portuguese_Brazil)
	ro: Română (Romanian)
	ru: Русский (Russian)
	sk: Slovenčina (Slovak)
	sl: Slovenski (Slovene)
	sr: српски (Serbian)
	sv: Svenska (Swedish)
	te: తెలుగు (telugu)
	ta: தமிழ் (Tamil)
	tr: Türkçe (Turkish)
	uk: українська мова (Ukrainian)
	vi: tiếng việt (Vietnamese)
	zh: 中文 (Chinese)
	zh_TW: 國語 (Taiwanese Mandarin)

it is possible to:

• scan the existing language codes (scan the directory searx/translations, most probably Babel has a feature to do that too)
• rely on Babel to get the native and English names:
  • babel.Locale.parse(f'{language_code}').display_name
  • babel.Locale.parse(f'{language_code}').english_name

ping @MarcAbonce

engine: openstreetmap.py

searxng/searx/engines/openstreetmap.py

Lines 25 to 26 in 9a383d1

	base_url = 'https://nominatim.openstreetmap.org/'
	search_string = 'search/{query}?format=json&polygon_geojson=1&addressdetails=1'

could be
https://nominatim.openstreetmap.org/search.php?{query}&polygon_geojson=1&format=jsonv2&addressdetails=1&extratags=1&dedupe=1

see : https://nominatim.org/release-docs/latest/api/Search/

Note:

• It seems class has been replaced by category.
• I don't see address29 in the results
• each result contains an extratags entry which provides:
  • wikidata link
  • wikipedia link
  • opening_hours

-->

• It is possible to remove call to https://overpass-api.de from the browser (also )
• Since nominatim is fast it is possible to send an additional query to get the images from wikidata

-->

In less than 800ms per request, usually faster, the results can contain:

• the wikipedia & wikidata links
• opening hours (see https://wiki.openstreetmap.org/wiki/Key:opening_hours#Python )
• a thumbnail

unit test reads settings from /etc/searx

The unit test should not read settings from a settings.yml file located in /etc/searx

[ru-RU] Correct the Russian language in the settings

"(Учтите, что указанные пользовательские настройки в URL адресе, могут привести к их утечке на посещаемых сайтах из поисковой выдачи.)"

Bug: google images engine

Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG
30e4a2a

How did you install SearXNG?
docker, it happens on searx.be too.

What happened?
The engine crashes.

How To Reproduce
Search for anything using the engine, for example: !goi anything

Expected behavior
To get the results

Screenshots & Logs

Additional context
Perhaps related to #175

Technical report

Error

• Error: IndexError
• Percentage: 5
• Parameters: ()
• File name: searx/engines/google_images.py:93
• Function: scrap_img_by_id
• Code: url_line = _script[i + 1]

Simple theme tasks

From:

• issue 1025 in searx
• #162

General : UI

• replace LESS variables by CSS variables, so the user can choose the line height, dark / light easily (?)
• footer at the botton of the page: see dalf@df0914e
• center or move the results to the right.
• redo color theme and just overall bigger UI elements; round corners and so on (right now its a little blunt)
• add dark and black theme

see https://joe-crick.github.io/pure-css-components/

General : UX

• infinite_scroll plugin settings
• opening pages in a new tab settings
• do the RTL style
• engine crash warning : add a close button
• infoboxes : some properties value should be a link to another search ( Programming language :Python --> be able to click on Python to search that on searx)

Search

• update HTML input attributes: see https://github.com/benbusby/whoogle-search/blob/main/app/templates/search.html
• add safe search right to Language combobox and Time combobox
• apply #1226 to Simple theme ( searx/searx#1226 )

On mobile

• do not set the focus on the search input text : the keyboard hides the results.
• make the search input smaller when without focus to give space for the logo and the preferences button. With focus, make it full width wide as now.
• embedded videos on mobile : too large, outside the screen
• selected category on mobile : scroll to the first selected category.

Image category

• give the user the choice to view the page a image is from instead of the actual image source

Reduce the network usage

• create a small bandwidth version using javascript (use JS to add languages choices, cached links, categories)

one logger per engine declared in settings.yml ?

Should a logger be created automatically for each engine in searx.engines.__init__ ?

For example:

searxng/searx/engines/xpath.py

Line 28 in 8283ce3

logger = logger.getChild('XPath engine')

Is it interesting to have one logger per engine declared in settings.yml ?

Downside: it adds one more magical variable, that IDE / editors won't know. Solution: use a factory logger = searx.engines.get_logger().

Move from transifex to Weblate

Over the years, transifex has become proprietary crap.

We should move to a self-hosted weblate [1] instance I think.

Related-to: searx/searx#268 (comment)
Discussed-at: searx/searx#2552

[1] https://github.com/WeblateOrg/weblate

Static files: build process

Result of make themes is a mess to deal with: it creates conflicts on all pull requests, if the master branch modified the built files. So two or more pull requests about the themes will create git conflicts.

A solution:

• commit only the source code.
• ask the administrators to build the themes, but it requires a lot of more dependencies

Another solution can a submodule with the built files, but I expect difficulties to synchronize everything especially when there are different branches.

Here a proposal based on the principale used in the previous pull requests:

• the last commit in a branch contains the built files
• add some scripts to manage this commit

Note: "source files" and "build files" refer to the static files only.

Make targets

static

build static files (may be rename "themes")

• call searx_extra/update/update_pygments.py
• call "themes"

static.clean

It is actually "git restore", but as if the built files are not commited, so "git clean" (and in the same way than "make clean")

• git restore [built files]

static.git.drop.commit

• drop the last commit if it is made by "static.git.commit"

static.git.branch.check

Check the current branch:

• exit code 0 if there is neither commit with a built file, neither a commit with modified source file.
• exit code 1 if one of the commits of branch contains a mix of static built file and source file.
• exit code 2 if there is modified source file.
• exit code 3 if there are some uncommited built file.

static.git.commit

build & commit static files (update commit or new commit)

• call "static.git.branch.check"
• if exit code is 0 or 1: stop
• if exit code is 3:
  • call "static.clean"
• if exit code is 2 or 3:
  • call "static.git.commit.drop"
  • call "static"
  • commit built files (only built files nothing else)

static.git.restore

after "git add", make sure there is not a mix a built & source files

• check if the staged files contains a mixed of source files and built files
• if yes
  • git restore --staged [built files]
  • display a warning to the user to call "make static.git.commit"

make static.git.resolve.conflict

• it should be the only files to conflict (no other files), otherwise stop the hook.
• git checkout --theirs [built files] (keep our built files in the branch, it will dropped anyway).
• git add [built files]
• git rebase --continue

Git hooks

Enabled by default, can be disabled globally with a configuration file.

post-rebase hook

if some built files conflict:

• call "make static.git.resolve.conflict"
• ask the user to call "make static.commit" (that will drop the built commit).

pre-commit hook

• call "static.git.restore"
• display a warning to call "make static.commit"

Use cases

With git hooks

• Commit change in the source files (static files):

git add .  # all files including built files
git commit  # the commit actually ignore the built files with a warning
make static.commit  # build and commit the built files

• Rebase with change of the built files in master

git pull --rebase origin master  # the last 
make static.commit

Without git hooks

• Commit change in the source files (static files):

git add .
make static.git.restore  # The CI will fail without it
git commit
make static.commit

• Rebase with change of the built files in master

git pull --rebase origin master
make static.git.resolve.conflict
make static.commit

CI

Fail if there are some modified source files without the built files?

---

Related to https://github.com/searxng/searxng/wiki/Milestones#milestone-19---data-upgrade--build-process

Customizable Vim Bindings

Would you consider adding the option to customize the keybindings of the vim plugin? That plugin is fantastic but the lack of any sort of customizability is unfortunate and would be a great addition.

"define:" processors

Is your feature request related to a problem? Please describe.

Currently, the currency engine is triggered by this regex (and code in the get_params method)

searxng/searx/search/processors/online_currency.py

Line 13 in 5253df9

parser_re = re.compile('.*?(\\d+(?:\\.\\d+)?) ([^.0-9]+) (?:in|to) ([^.0-9]+)', re.I)

Some engines could be triggered by define: prefix in the query.

Describe the solution you'd like

wikidata can help. See example below, click on the ▶️ button on the left to get the results.

Some example using wikidata:

• define: +34 to look for country dialing code (the query actually for any identifier that is +34)
• define: 38000 to look for any id that is 38000, it can be useful to get a postal code for example. There are 83 results, so here I don't know if request should be define: 38000, postal code (a syntax must be found)
• define: dog: to get a dictionary feature (and possible translation: line 23 and 24, change fr to another language)

Also the wiktionary engine could be trigger by this prefix.

Describe alternatives you've considered
May be a define category ?

Additional context

[question] Add support for multiple architectures

I just want to leave some links here, what else we could do to support multiple architectures:

• There is a PR searx/searx#2490 fixing filtron.sh and morty.sh to support multiple architectures
• There is a searx/searx#2680 (comment) worth to read // update: done in #22)
• LXC: Here is what I will try to test: https://askubuntu.com/a/1223581 .. also using qemu-arm-static and binfmt_misc .. the charm is, that (I hope) I can integrate it in searx's LXC suite.

what else should we consider?

Refactor searx.webapp.preferences

searx.webapp.preferences contains duplicate code from searx.metrics.get_engines_stats :

In searx.webapp.preferences:

preferences	get_engines_stats
rate80	total_p80
rate95	total_p95
warn_timeout	N/A
supports_selected_language	N/A
result_count	result_count

[question] How to launch Searx with the new way?

I used to be able to do python searx/webapp.py but now it doesn't work anymore:

Traceback (most recent call last):
  File "/tmp/searxng/searx/webapp.py", line 58, in <module>
    from searx import logger
ModuleNotFoundError: No module named 'searx'

How do I launch it with the new way?

dependabot should not be active in forks

Is your feature request related to a problem? Please describe.

The dependabot creates PRs and noise in forks where it is not needed or wanted. E.g. in my fork the issues from dependabot are unwanted and it is not only mine where this is useless

Describe the solution you'd like

Enable github/dependabot.yml only for the origin https://github.com/searxng/searxng/

Describe alternatives you've considered

It seems we are not alone with the problem and a solution is not shine at nearer horizon ...

• dependabot/dependabot-core#2804 (comment)
• dependabot/dependabot-core#2198

For the interim we could deactivate dependabot and update manually.

I think we have to act here to wait until dependabot/dependabot-core#2804 is fixed, until this is fixed because each fork has a master branch that we cannot change after the fork is created / these dependabots from forks will run infinite.

searxng rpi container won't start

Docker image version
1.0.0-495-30e4a2a2

Platform
RPi3+

Docker run command
docker run --detach \ --name searxng \ --restart unless-stopped \ -p 8881:8080 \ -v "${PWD}/searx:/etc/searx" \ -e "INSTANCE_NAME=searxng" \ searxng/searxng

Problem
won't start and keeps looping

Add ability to configure the time for every suspended_time

Is your feature request related to a problem? Please describe.
I've 120 proxies on my Searx instance, with a "retries" of 5 on the Google engine, Searx doesn't know if all of my proxies are non-functional. With the current automatic suspend system Searx is suspending the Google engine even if the next search could work due to the huge amount of proxies available.

Describe the solution you'd like
I would like to be able to modifying the time for every suspended_time available here: https://github.com/searxng/searxng/blob/master/searx/network/raise_for_httperror.py

Describe alternatives you've considered
Overriding the suspended time in the python files.

/preferences: use simple layout for the oscar theme

Suggestion: change how the options are displayed in the /preferences page in the oscar theme :

• General
  • Default categories
  • Search languages
  • Autocomplete
  • SafeSearch
  • Open Access DOI rewrite (plugin)
  • Open Access DOI resolver
• Engines
• User Interface
  • Interface language
  • Themes
  • Results on new tabs
  • Infinite scroll (plugin)
  • Search on category select (plugin)
  • Hotkeys (plugins)
• Privacy
  • HTTP method
  • Image proxy
  • Tracker URL remover (plugin)
• Cookies
• Query syntax (new tab, a mix between documentation and auto documentation from the current settings)
  • Documentation about the query syntax (bang, external bangs, ...)
  • Answers (list of configured answers)
  • Special engines: currency, translations (list the engines that ).
  • External bangs

Manage HTTP headers

Currently, the default HTTP headers are set in settings.yml

Another version in searx-docker includes more headers.

• See https://owasp.org/www-project-secure-headers/
  • "Best Practices" tab
  • X-XSS-Protection: it is recommended to set the header as X-XSS-Protection: 0

So currently:

• Content-Security-Policy is set in searx-docker but otherwise except if the administrator look at different locations.
• Administrators who do not update their settings.yml, won't have a secure setup.
• Some other administrator will set theirs HTTP headers in the reverse proxy once for all.

Looking at owasp.org, the HTTP headers need regular maintenance.

---

This issue suggests to define the default headers directly a place that is keep up to date: the code, so the configuration is not spread into different locations.

An option can let the administrator override them if requires.

About Content-Security-Policy:

• a default can be set at first, see Whoogle
• in a second time, the engines can returns a list of external website to include in the CSP header.

[example] /etc/searx/settings.yml

To share admin experience I want to leave my actual /etc/searx/settings.yml setup here.

My searx setup has a weight on

• safe-search
• german-speaking user base
  • pure EN engines in categories 'general' and 'news' are disabled by default
  • german-speaking content like the one from mediathekviewweb are placed in category 'general'

last update: 2021/06/13

use_default_settings: true

general:
  debug: false  # Debug mode, only for development
  instance_name: "SearXNG@darmarIT"  # displayed name

search:
  # Filter results. 0: None, 1: Moderate, 2: Strict
  safe_search: 2
  # Existing autocomplete backends: "dbpedia", "duckduckgo", "google",
  # "startpage", "swisscows", "qwant", "wikipedia" - leave blank to turn it off
  # by default.
  autocomplete: "duckduckgo"
  # Default search language - leave blank to detect from browser information or
  # use codes from 'languages.py'
  default_lang: ""

server:
  port: 8888
  # address to listen on
  bind_address: "127.0.0.1"
  secret_key: "ultrasecretkey"  # change this!
  # Set custom base_url. Possible values:
  #   false or "https://your.custom.host/location/"
  base_url: https://darmarit.org/searx
  # Proxying image results through searx
  image_proxy: false

engines:

  - name: duckduckgo
    disabled: false

  - name: fdroid
    disabled: false

  - name: apk mirror
    disabled: false

  - name: springer nature
    engine: springer
    shortcut: springer
    categories: science
    # get your API key from: https://dev.springernature.com/signup
    api_key: ""
    disabled: false
    timeout: 6.0

  - name: mediathekviewweb
    engine: mediathekviewweb
    shortcut: mvw
    categories: general

  # bing is broken
  #
  - name: bing
    disabled: true

  # core.ac response very slow (inital most often 10sec.)
  #
  - name: core.ac.uk
    engine: core
    categories: science
    shortcut: cor
    api_key: ''
    timeout: 6

  # duden has very often HTTP errors
  #
  - name: duden
    disabled: true

  # yahoo news are only english
  #
  - name: yahoo news
    disabled: true

  # yahoo language support does not work
  #
  - name: yahoo
    disabled: true

  # reddit does not support laguages
  #
  - name: reddit
    timeout: 3

Bug: engine "archive is" reports HTTP 404 / Not found

Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG

1970d28

Technical report

Error

• Error: httpx.HTTPStatusError
• Percentage: 100
• Parameters: ('404', 'Not Found', 'archive.is')
• File name: searx/search/processors/online.py:99
• Function: _send_http_request
• Code: response = req(params['url'], **request_args)

Bug: qwant engine

Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG

current master

How did you install SearXNG?

Installation scripts

What happened?

Various exceptions in the qwant engine

How To Reproduce

https://darmarit.org/searx/stats?engine=qwant

Technical report

Error

• Error: ValueError
• Percentage: 0
• Parameters: ()
• File name: searx/engines/qwant.py:159
• Function: response
• Code: pub_date = datetime.fromtimestamp(item['date'], None)

Error

• Error: TypeError
• Percentage: 0
• Parameters: ('an integer is required (got type NoneType)',)
• File name: searx/engines/qwant.py:196
• Function: response
• Code: pub_date = datetime.fromtimestamp(item['date'])

Error

• Error: httpx.ConnectTimeout
• Percentage: 0
• Parameters: (None, None, 'api.qwant.com')
• File name: searx/search/processors/online.py:99
• Function: _send_http_request
• Code: response = req(params['url'], **request_args)

Error

• Error: TypeError
• Percentage: 0
• Parameters: ('unsupported type for timedelta seconds component: NoneType',)
• File name: searx/engines/qwant.py:195
• Function: response
• Code: length = timedelta(seconds=item['duration'])

Error

• Error: searx.exceptions.SearxEngineAPIException
• Percentage: 0
• Parameters: ('API error::locale must be a string,locale must be one of the following values: en_gb, en_ie, en_us, en_ca, en_in, en_my, en_au, en_nz, cy_gb, gd_gb, de_de, de_ch, de_at, fr_fr, br_fr, fr_be, fr_ch, fr_ca, fr_ad, fc_ca, ec_ca, co_fr, es_es, es_ar, es_cl, es_co, es_mx, es_pe, es_ad, ca_es, ca_ad, ca_fr, eu_es, eu_fr, it_it, it_ch, pt_br, pt_pt, pt_ad, nl_be, nl_nl, pl_pl, zh_hk, zh_cn, fi_fi, bg_bg, et_ee, hu_hu, da_dk, nb_no, sv_se, ko_kr, th_th, cs_cz, ro_ro, el_gr',)
• File name: searx/engines/qwant.py:114
• Function: response
• Code: raise SearxEngineAPIException('API error::' + msg)

Error

• Error: KeyError
• Percentage: 0
• Parameters: ()
• File name: searx/engines/qwant.py:147
• Function: response
• Code: title = item['title']

Bug: google news engine

Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG

7ef33c9

How did you install SearXNG?
docker

How To Reproduce
!google_news time crashes whatever the search language is

Technical report

Error

• Error: KeyError
• Percentage: 100
• Parameters: ()
• File name: searx/engines/google_news.py:97
• Function: request
• Code: if lang_info['hl'] == 'en':

Checker

• simple: KeyError ,
• lang_fr: KeyError ,
• lang_en: KeyError ,

Bug: dogpile engine

Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG

• https://darmarit.org/searx
• https://searx.zackptg5.com/ commit a898888

How did you install SearXNG?
N/A

What happened?

How To Reproduce
See above.

Expected behavior
To get the result or disable the engine.

Additional context

According to the logs it seems the engine redirects to http://omblockedips.com (not https). It shows the Recaptcha challenge.
So I suggest to add omblockedips.com to

searxng/searx/network/raise_for_httperror.py

Line 54 in 4f09c7b

def raise_for_captcha(resp):

Technical report

Error

• Error: httpx.UnsupportedProtocol
• Percentage: 100
• Parameters: (None, None, 'omblockedips.com')
• File name: searx/search/processors/online.py:99
• Function: _send_http_request
• Code: response = req(params['url'], **request_args)

Checker

• simple: httpx.UnsupportedProtocol ,
• paging: httpx.UnsupportedProtocol ,

Static files: move source to a dedicated directory.

Since it is a topic in itself, here a new issue :

From #137 (comment)

In first order we should separate source code from the ./searx/static folder.

I would see:

• src/static
• src/searx
• src/searx/static <-- built files

But a lot of stuff expect searx source code in the searx directory.

So If the source folder for the static files is static/src or just static.

static (or static/src)
	themes
		oscar
			img           # searx/static/themes/oscar/css/images AND searx/static/themes/oscar/css/img
			js            # searx/static/themes/oscar/src/js
			less          # searx/static/themes/oscar/src/less
			package.json
			gruntfile.js
			README.rst
		simple
			js            # searx/static/themes/simple/js/searx_head AND searx/static/themes/simple/js/searx_src
			less          # searx/static/themes/simple/less
			...           # incomplete because a similar PR to https://github.com/searx/searx/pull/2346 must be done
			gruntfile.js
			package.json