summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2021-03-11Merge pull request #2643 from return42/fix-makefileAlexandre Flament
[fix] make targets engines.languages and useragents.update
2021-03-08Merge pull request #2640 from return42/fix-yahoo-newsAlexandre Flament
[fix] rewrite Yahoo-News engine
2021-03-08[fix] make targets engines.languages and useragents.updateMarkus Heiser
Since [PR 2600] is merged the update scripts for languages and useragent has been moved to folder: searx_extra/update/ [PR 2600] https://github.com/searx/searx/pull/2600 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-08[fix] rewrite Yahoo-News engineMarkus Heiser
Many things have been changed since last review of this engine. This patch fix xpath selectors, implements suggestion and is a complete review / rewrite of the engine. Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-03-06Merge pull request #2615 from searx/engine-dataNoémi Ványi
Add ability to send engine data to subsequent requests
2021-03-06[enh] add ability to send engine data to subsequent requestsAdam Tauber
2021-03-06Merge pull request #2631 from searx/update_data_update_languages.pyAlexandre Flament
Update searx.data - update_languages.py
2021-03-05Merge pull request #2634 from return42/fedoraAlexandre Flament
[mod] LXC switch to Fedora 33 / Fedora 31 reached its EOL
2021-03-05Merge pull request #2635 from return42/einitAlexandre Flament
[mod] don't dump traceback of SearxEngineResponseException on init
2021-03-05[mod] don't dump traceback of SearxEngineResponseException on initMarkus Heiser
When initing engines a "SearxEngineResponseException" is logged very verbose, including full traceback information: ERROR:searx.engines:yggtorrent engine: Fail to initialize Traceback (most recent call last): File "share/searx/searx/engines/__init__.py", line 293, in engine_init init_fn(get_engine_from_settings(engine_name)) File "share/searx/searx/engines/yggtorrent.py", line 42, in init resp = http_get(url, allow_redirects=False) File "share/searx/searx/poolrequests.py", line 197, in get return request('get', url, **kwargs) File "share/searx/searx/poolrequests.py", line 190, in request raise_for_httperror(response) File "share/searx/searx/raise_for_httperror.py", line 60, in raise_for_httperror raise_for_captcha(resp) File "share/searx/searx/raise_for_httperror.py", line 43, in raise_for_captcha raise_for_cloudflare_captcha(resp) File "share/searx/searx/raise_for_httperror.py", line 30, in raise_for_cloudflare_captcha raise SearxEngineCaptchaException(message='Cloudflare CAPTCHA', suspended_time=3600 * 24 * 15) searx.exceptions.SearxEngineCaptchaException: Cloudflare CAPTCHA, suspended_time=1296000 For SearxEngineResponseException this is not needed. Those types of exceptions can be a normal use case. E.g. for CAPTCHA errors like shown in the example above. It should be enough to log a warning for such issues: WARNING:searx.engines:yggtorrent engine: Fail to initialize // Cloudflare CAPTCHA, suspended_time=1296000 closes: #2612 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-05[mod] LXC switch to Fedora 33 / Fedora 31 reached its EOLMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-05Merge pull request #2632 from searx/update_data_update_wikidata_units.pyAlexandre Flament
Update searx.data - update_wikidata_units.py
2021-03-05Merge pull request #2630 from searx/update_data_update_ahmia_blacklist.pyAlexandre Flament
Update searx.data - update_ahmia_blacklist.py
2021-03-05Merge pull request #2633 from searx/update_data_update_currencies.pyAlexandre Flament
Update searx.data - update_currencies.py
2021-03-05Update searx.data - update_currencies.pydalf
2021-03-05Update searx.data - update_wikidata_units.pydalf
2021-03-05Update searx.data - update_languages.pydalf
2021-03-05Update searx.data - update_ahmia_blacklist.pydalf
2021-03-05Fix integration.ymlAlexandre Flament
Don't run twice the workflow on PR opened from the same repository.
2021-03-05Fix data-update.yml, run once a monthAlexandre Flament
2021-03-05Fix data-update.ymlAlexandre Flament
2021-03-05Update data-update.ymlAlexandre Flament
Allow to manually running the workflow See: * https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow * https://docs.github.com/en/actions/reference/events-that-trigger-workflows#workflow_dispatch
2021-03-05Update data-update.ymlAlexandre Flament
Remove cache
2021-03-05Merge pull request #2600 from dalf/searx-extraAlexandre Flament
Add searx_extra package
2021-03-04Add searx_extra packageAlexandre Flament
Split the utils directory into: * searx_extra contains update scripts, standalone_searx.py * utils contains the files to build and setup searx.
2021-03-04Merge pull request #2618 from thezeroalpha/masterAlexandre Flament
Fix security vulnerabilities in provided nginx configuration
2021-03-04Merge pull request #2623 from return42/fix-centos7Alexandre Flament
[fix] add package `which` to CentOS-7 boilerplate
2021-03-04Merge pull request #2619 from return42/drop-ubu1604Alexandre Flament
[mod] Drop Ubuntu 16.04 (Xenial Xerus) support
2021-03-04Merge pull request #2620 from return42/fix-gitAlexandre Flament
[fix] support git versions <v2.22
2021-03-04Use $host in nginx morty.conf templateAlex Balgavy
2021-03-04Merge pull request #2593 from dalf/update-autocompleteAlexandre Flament
Update autocomplete
2021-03-03Merge pull request #2621 from return42/fix-searx.shNoémi Ványi
[fix] utils/serax.sh create_pyenv() - drop duplicate 'pip install .'
2021-03-03[fix] add package 'which' to CentOS-7 boilerplateMarkus Heiser
Newer CentOS-7 images from https://images.linuxcontainers.org do no longer include the which command. Issue: $ sudo -H ./utils/lxc.sh cmd searx-centos7 ./utils/filtron.sh install all INFO: [searx-centos7] ./utils/filtron.sh install all ... Install Go in user's HOME ------------------------- download and install go binary .. ... -bash: line 1: which: command not found -->|ERROR - Go Installation not found in PATH!?! -bash: line 2: which: command not found Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-03[fix] utils/serax.sh create_pyenv() - drop duplicate 'pip install .'Markus Heiser
The wrong and unnecessary `pip install .` is executed in /usr/local/searx and is responsible for the error message: ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: /usr/local/searx The correct pip-install comes right after changing to `cd ${SEARX_SRC}`. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-03[fix] support git versions <v2.22Markus Heiser
LTS distros like Ubuntu 18.04 do not ship a up-to-date version of git.:: $ sudo -H ./utils/lxc.sh cmd searx-ubu1804 git --version ... git version 2.17.1 The option `--show-current` was added in git v2.22, the alternative to this option is:: git rev-parse --abbrev-ref HEAD Issue when using option `--show-current`:: [searx-ubu1804] Clone searx sources [searx-ubu1804] ------------------- [searx-ubu1804] [searx-ubu1804] error: unknown option `show-current' [searx-ubu1804] usage: git branch [<options>] [-r | -a] [--merged | --no-merged] .... Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-03[mod] Drop Ubuntu 16.04 (Xenial Xerus) supportMarkus Heiser
EOL of Ubuntu 16.04 (Xenial Xerus) is in April 2021 but we do not support 16.04 since a longer time. Issues are comming from dependencies (e.g. pip):: [searx-ubu1604] |searx| SyntaxError: invalid syntax [searx-ubu1604] |searx| Traceback (most recent call last): [searx-ubu1604] |searx| File "/usr/local/searx/searx-pyenv/bin/pip", line 7, in <module> [searx-ubu1604] |searx| from pip._internal.cli.main import main [searx-ubu1604] |searx| File "/usr/local/searx/searx-pyenv/lib/python3.5/site-packages/pip/_internal/cli/main.py", line 60 [searx-ubu1604] |searx| sys.stderr.write(f"ERROR: {exc}") [searx-ubu1604] |searx| ^ [searx-ubu1604] |searx| SyntaxError: invalid syntax Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-03Fix security vulnerabilities in suggested nginx configurationAlex Balgavy
The suggested configurations for nginx found in the documentation and templates lead to vulnerabilities allowing host spoofing [1] and path traversal [2], as reported by Gixy [3]. This commit fixes those issues. [1] https://github.com/yandex/gixy/blob/master/docs/en/plugins/hostspoofing.md [2] https://github.com/yandex/gixy/blob/master/docs/en/plugins/aliastraversal.md [3] https://github.com/yandex/gixy
2021-03-02Merge pull request #2610 from searx/dependabot/pip/master/pylint-2.7.2Alexandre Flament
Bump pylint from 2.6.2 to 2.7.2
2021-03-02Bump pylint from 2.6.2 to 2.7.2dependabot[bot]
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.6.2 to 2.7.2. - [Release notes](https://github.com/PyCQA/pylint/releases) - [Changelog](https://github.com/PyCQA/pylint/blob/master/ChangeLog) - [Commits](https://github.com/PyCQA/pylint/compare/pylint-2.6.2...pylint-2.7.2) Signed-off-by: dependabot[bot] <support@github.com>
2021-03-02Merge pull request #2611 from return42/google-scolarAlexandre Flament
[enh] google scholar - python implementation of the engine
2021-03-01[enh] autocomplete refactoring, autocomplete on external bangsAlexandre Flament
2021-03-01[enh] google scholar - python implementation of the engineMarkus Heiser
The old xpath configuration for google scholar did not work and is replaced by a python implementation. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-01[fix] answers: don't crash when the query is an empty stringAlexandre Flament
2021-03-01Merge pull request #2608 from return42/unittest2Alexandre Flament
[py2to3] use unittest from py3, remove unittest2 from py2
2021-03-01Merge pull request #2604 from searx/update_data_firefox_versionAlexandre Flament
Update searx.data - firefox_version
2021-03-01Merge pull request #2602 from MarcAbonce/fix-bing-fetch-languagesAlexandre Flament
Fix fetch_languages for Bing
2021-03-01Add Freesound engine (#2596)GazoilKerozen
Add freesound engine with player. Co-authored-by: Gazoil <maildeguzel@gmail.com>
2021-02-28[py2to3] use unittest from py3, remove unittest2 from py2Markus Heiser
- unittest2 is a backport of the new features added to the unittest testing framework in Python 2.7 - unittest2 was only needed in py2 and can be dropped now Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-02-26Merge pull request #2605 from searx/update_data_currenciesAlexandre Flament
Update searx.data - currencies
2021-02-26Merge pull request #2606 from searx/update_data_wikidata_unitsAlexandre Flament
Update searx.data - wikidata_units