summaryrefslogtreecommitdiff
path: root/searx/engines
AgeCommit message (Collapse)Author
2021-10-07[fix] engine torznab - categories, before join convert int to strMarkus Heiser
BTW add init() function and replace SearxEngineAPIException by ValueError. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-07[fix] engine torznab - marginal issues reported from lintersMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-07[mod] engines - add torznab WebAPIPaolo Basso
2021-10-07[mod] set 'engine.supported_languages' from the origin python moduleMarkus Heiser
The key of the dictionary 'searx.data.ENGINES_LANGUAGES' is the *engine name* configured in settings.xml. When multiple engines are configured to use the same origin engine (e.g. `engine: google`):: - name: google engine: google use_mobile_ui: false ... - name: google italian engine: google use_mobile_ui: false language: it ... - name: google mobile ui engine: google shortcut: gomui use_mobile_ui: true There exists no entry for ENGINES_LANGUAGES[engine.name] (e.g. `name: google mobile ui` or `name: google italian`). This issue can be solved by recreate the ENGINES_LANGUAGES:: make data.languages But this is nothing an SearXNG admin would like to do when just configuring additional engines, since this just doubles entries in ENGINES_LANGUAGES and BTW: `make data.languages` has various external requirements which might be not installed or not available, on a production host. With this patch, if engine.name fails, ENGINES_LANGUAGES[engine.engine] is used to get the engine.supported_languages (e.g. `google` for the engine named `google mobile`). For an engine, when there is `language: ...` in the YAML settings, the engine supports only one language, in this case engine.supported_languages should contains this value defined in settings.yml (e.g. `it` for the engine named `google italian`). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> Closes: https://github.com/searxng/searxng/issues/384
2021-10-05[mod] engines - IMDB: add thumbnailsAlexandre Flament
2021-10-03[mod] engines - add IMDB / Internet Movie DatabasePaul Alcock
Merged from @Guilvareux's commit [1] and slightly modfied / see [2]. [1] https://github.com/searx/searx/pull/2980/commits/f2f90071 [2] https://github.com/searx/searx/pull/2980
2021-10-01[mod] engine duckduckgo - update supported_languages_urlMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-01[mod] engine duckduckgo - use DuckDuckGo-LiteMarkus Heiser
Implement a scrapper for DuckDuckGo-Lite [1]. The existing DuckDuckGo [2] engine does not support paging. DuckDuckgo-Lite is much faster, less verbose and does have a paging option (reversed engineered from the input form of [1]). [1] https://lite.duckduckgo.com/lite [2] https://duckduckgo.com/ Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-29[fix] engine stackexchange - decode HTML entities in title & contentMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-28[mod] replace old stackoverflow engine by Stack Exchange API v2.3Markus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-28[mod] engines - add Stack Exchange API v2.3Markus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-25Merge pull request #333 from dalf/enh-engine-descriptionsAlexandre Flament
RFC: /preferences: display engine descriptions
2021-09-25[fix] openstreetmap engine: optmizer SPARQL queryAlexandre Flament
add hint:Query hint:optimizer "None". to the SPARQL query to keep the response time small. It tells the optimizer to follow the path from ?item to the different property values instead of the other way around. See https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/query_optimization#Property_paths
2021-09-24[fix] fix the about section of some enginesAlexandre Flament
2021-09-24[fix] openstreetmap engine: map "all" language to EnglishAlexandre Flament
2021-09-17[pylint] fix global-variable-not-assigned issuesMarkus Heiser
If there is no write access, there is no need for global. Remove global statement if there is no assignment. global-variable-not-assigned: Using global for names but no assignment is done Used when a variable is defined through the "global" statement but no assignment to this variable is done. In Pylint 2.11 the global-variable-not-assigned checker now catches global variables that are never reassigned in a local scope and catches (reassigned) functions [1][2] [1] https://pylint.pycqa.org/en/latest/whatsnew/2.11.html [2] https://github.com/PyCQA/pylint/issues/1375 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-14Merge pull request #297 from dalf/engine-logger-enhAlexandre Flament
debug mode: more readable logging
2021-09-13[fix] logger per engine: make .logger is always initializedAlexandre Flament
the openstreetmap engine imports code from the wikidata engine. before this commit, specific code make sure to copy the logger variable to the wikidata engine. with this commit searx.engines.load_engine makes sure the .logger is initialized. The implementation scans sys.modules for module name starting with searx.engines.
2021-09-11[mod] xpath engine: remove logging of the requested URLAlexandre Flament
2021-09-07[pylint] engines: drop no longer needed 'missing-function-docstring'Markus Heiser
Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914168470 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-07[fix] add 'categories' to PYLINT_ADDITIONAL_BUILTINS_FOR_ENGINESMarkus Heiser
androp no longer needed (see line 591 in 7b235a1):: # pylint: disable=undefined-variable Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914068609 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-07[fix] drop useless pylint: disable=undefined-variableMarkus Heiser
Since 7b235a1 (see line 591) it is no longer needed to disable 'undefined-variable' for names defined in:: PYLINT_ADDITIONAL_BUILTINS_FOR_ENGINES Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914068609 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-06[fix] fix openstreetmap engineAlexandre Flament
close #298 This is a workaround: inside engine code, any call to function in another engine can crash since the logger won't be initialized except if it is done explicitly.
2021-09-06[mod] one logger per engine - drop obsolete logger.getChildMarkus Heiser
Remove the no longer needed `logger = logger.getChild(...)` from engines. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-06[mod] one logger per engineMarkus Heiser
Suggested-by: @dalf in https://github.com/searxng/searxng/issues/98#issuecomment-849013518 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-04[fix] remove minimum length of content for XPath engineMarkus Heiser
Instead of raising an exception and therefore hiding all results of the engine. It make sense to remove that requirement in order to allow the implementation of search engines that do not always have a description. In fact some search engines that in 99% of the case have a description like Brave Search or Mojeek crash completely if they for some reason included a result with no description. To test this patch try Mojeek: !mjk xyz before and after the patch. Suggested-by: 0xhtml in https://github.com/searx/searx/discussions/2933 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-04[fix] Correct engine name in for RumbleAllen
2021-09-04[fix] Update about section of InvidiousAllen
Another website and new documentation
2021-08-31[pylint] Pylint 2.10 - fix use-list-literal & use-dict-literalMarkus Heiser
Pylint 2.10 added new default checks [1]: use-list-literal Emitted when list() is called with no arguments instead of using [] use-dict-literal Emitted when dict() is called with no arguments instead of using {} [1] https://pylint.pycqa.org/en/latest/whatsnew/2.10.html Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-08-21[enh] google: add filter=0 to Google engine for more resultsNoémi Ványi
backport from searx ( 23b3b56a06ef831af0a1b30a12c26ebd50e329bb )
2021-07-27[fix] Seznam engine - some XPath selectors has been changedSamuel Dudik
Merged from https://github.com/dudik/searx/commit/5a4207759 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-25Merge pull request #221 from dalf/fix-peertube_fetch_supported_languagesAlexandre Flament
[fix] peertube: update _fetch_supported_languages
2021-07-24[fix] qwant engine - prevent API locale exception on lang 'all'Markus Heiser
Has been reported in [1], error message:: Error Error: searx.exceptions.SearxEngineAPIException Percentage: 0 Parameters: ('API error::locale must be a string,locale must be one of the following values: en_gb, en_ie, en_us, en_ca, en_in, en_my, en_au, en_nz, cy_gb, gd_gb, de_de, de_ch, de_at, fr_fr, br_fr, fr_be, fr_ch, fr_ca, fr_ad, fc_ca, ec_ca, co_fr, es_es, es_ar, es_cl, es_co, es_mx, es_pe, es_ad, ca_es, ca_ad, ca_fr, eu_es, eu_fr, it_it, it_ch, pt_br, pt_pt, pt_ad, nl_be, nl_nl, pl_pl, zh_hk, zh_cn, fi_fi, bg_bg, et_ee, hu_hu, da_dk, nb_no, sv_se, ko_kr, th_th, cs_cz, ro_ro, el_gr',) File name: searx/engines/qwant.py:114 Function: response Code: raise SearxEngineAPIException('API error::' + msg) [1] https://github.com/searxng/searxng/issues/222 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-24[fix] qwant engine - prevent exception on date/time value is NoneMarkus Heiser
Has been reported in [1], error messages:: Error Error: ValueError Percentage: 0 Parameters: () File name: searx/engines/qwant.py:159 Function: response Code: pub_date = datetime.fromtimestamp(item['date'], None) Error Error: TypeError Percentage: 0 Parameters: ('an integer is required (got type NoneType)',) File name: searx/engines/qwant.py:196 Function: response Code: pub_date = datetime.fromtimestamp(item['date']) Fix timedelta from seconds to milliseconds [1], error message:: Error Error: TypeError Percentage: 0 Parameters: ('unsupported type for timedelta seconds component: NoneType',) File name: searx/engines/qwant.py:195 Function: response Code: length = timedelta(seconds=item['duration']) [1] https://github.com/searxng/searxng/issues/222 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-23[fix] peertube: update _fetch_supported_languagesAlexandre Flament
update the regex to match the changes in peertube source code fix "make data.languages"
2021-07-16Merge pull request #211 from MarcAbonce/onions_v3_fix_searxngAlexandre Flament
Update onion engines to v3
2021-07-16Merge pull request #207 from return42/mongodbAlexandre Flament
[enh] add mongodb offline engine
2021-07-16[mod] qwant engine: fix typos / minor changeAlexandre Flament
minor modification of commit 628b5703f3aeeed117772696f83efb344d6f337e (no functionnal change)
2021-07-16update onion engines to v3Marc Abonce Seguin
remove not_evil which has been down for a while now: https://old.reddit.com/r/onions/search/?q=not+evil&restrict_sr=on&t=year
2021-07-15[enh] add mongodb offline engineMarkus Heiser
Cherry-Pick: https://github.com/searx/searx/commit/198aad43 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-15[mod] improve video results of the qwant engineMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-15Merge pull request #205 from unixfox/patch-2Alexandre Flament
Add missing parameter for mobile UI search
2021-07-15Add missing parameter for mobile UI searchÉmilien Devos
2021-07-14[fix] Qwant engines - implement API v3 and add 'quant videos'Markus Heiser
The implementation uses the Qwant API (https://api.qwant.com/v3). The API is undocumented but can be reverse engineered by reading the network log of https://www.qwant.com/ queries. This implementation is used by different qwant engines in the settings.yml:: - name: qwant categories: general ... - name: qwant news categories: news ... - name: qwant images categories: images ... - name: qwant videos categories: videos ... Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-11[drop] engine torrentz: torrentz2.eu and torrentz2.is are offlineMarkus Heiser
[1] https://torrentfreak.com/torrentz2-eu-domain-suspended-by-registry-on-public-prosecutors-order-200628/ Suggested-by: @rasos https://github.com/searx/searx/issues/1875#issuecomment-877755872 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-10Fix google imagesÉmilien Devos
Proposed fix in https://github.com/searx/searx/pull/2115#issuecomment-876716010
2021-06-21[docs] add documentation from the sources of the google enginesMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-21[fix] google answers: normalize space of the answers.Markus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-21[mod] google engine: reduce mobile UI parameters to what is neededMarkus Heiser
Reverse engineering shows that not all of the parameters used by google's mobile UI (aka "more results" button) are needed [1]. [1] https://github.com/searxng/searxng/pull/160#issuecomment-865013625 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-21[mod] google: add "use_mobile_ui" parameter to use mobile endpoint.Alexandre Flament
disable by default, it has to be enabled in settings.yml related to #159
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-15 15:26:47 +0000