summaryrefslogtreecommitdiff
path: root/searx/engines
AgeCommit message (Collapse)Author
2021-08-31[pylint] Pylint 2.10 - fix use-list-literal & use-dict-literalMarkus Heiser
Pylint 2.10 added new default checks [1]: use-list-literal Emitted when list() is called with no arguments instead of using [] use-dict-literal Emitted when dict() is called with no arguments instead of using {} [1] https://pylint.pycqa.org/en/latest/whatsnew/2.10.html Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-08-21[enh] google: add filter=0 to Google engine for more resultsNoémi Ványi
backport from searx ( 23b3b56a06ef831af0a1b30a12c26ebd50e329bb )
2021-07-27[fix] Seznam engine - some XPath selectors has been changedSamuel Dudik
Merged from https://github.com/dudik/searx/commit/5a4207759 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-25Merge pull request #221 from dalf/fix-peertube_fetch_supported_languagesAlexandre Flament
[fix] peertube: update _fetch_supported_languages
2021-07-24[fix] qwant engine - prevent API locale exception on lang 'all'Markus Heiser
Has been reported in [1], error message:: Error Error: searx.exceptions.SearxEngineAPIException Percentage: 0 Parameters: ('API error::locale must be a string,locale must be one of the following values: en_gb, en_ie, en_us, en_ca, en_in, en_my, en_au, en_nz, cy_gb, gd_gb, de_de, de_ch, de_at, fr_fr, br_fr, fr_be, fr_ch, fr_ca, fr_ad, fc_ca, ec_ca, co_fr, es_es, es_ar, es_cl, es_co, es_mx, es_pe, es_ad, ca_es, ca_ad, ca_fr, eu_es, eu_fr, it_it, it_ch, pt_br, pt_pt, pt_ad, nl_be, nl_nl, pl_pl, zh_hk, zh_cn, fi_fi, bg_bg, et_ee, hu_hu, da_dk, nb_no, sv_se, ko_kr, th_th, cs_cz, ro_ro, el_gr',) File name: searx/engines/qwant.py:114 Function: response Code: raise SearxEngineAPIException('API error::' + msg) [1] https://github.com/searxng/searxng/issues/222 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-24[fix] qwant engine - prevent exception on date/time value is NoneMarkus Heiser
Has been reported in [1], error messages:: Error Error: ValueError Percentage: 0 Parameters: () File name: searx/engines/qwant.py:159 Function: response Code: pub_date = datetime.fromtimestamp(item['date'], None) Error Error: TypeError Percentage: 0 Parameters: ('an integer is required (got type NoneType)',) File name: searx/engines/qwant.py:196 Function: response Code: pub_date = datetime.fromtimestamp(item['date']) Fix timedelta from seconds to milliseconds [1], error message:: Error Error: TypeError Percentage: 0 Parameters: ('unsupported type for timedelta seconds component: NoneType',) File name: searx/engines/qwant.py:195 Function: response Code: length = timedelta(seconds=item['duration']) [1] https://github.com/searxng/searxng/issues/222 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-23[fix] peertube: update _fetch_supported_languagesAlexandre Flament
update the regex to match the changes in peertube source code fix "make data.languages"
2021-07-16Merge pull request #211 from MarcAbonce/onions_v3_fix_searxngAlexandre Flament
Update onion engines to v3
2021-07-16Merge pull request #207 from return42/mongodbAlexandre Flament
[enh] add mongodb offline engine
2021-07-16[mod] qwant engine: fix typos / minor changeAlexandre Flament
minor modification of commit 628b5703f3aeeed117772696f83efb344d6f337e (no functionnal change)
2021-07-16update onion engines to v3Marc Abonce Seguin
remove not_evil which has been down for a while now: https://old.reddit.com/r/onions/search/?q=not+evil&restrict_sr=on&t=year
2021-07-15[enh] add mongodb offline engineMarkus Heiser
Cherry-Pick: https://github.com/searx/searx/commit/198aad43 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-15[mod] improve video results of the qwant engineMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-15Merge pull request #205 from unixfox/patch-2Alexandre Flament
Add missing parameter for mobile UI search
2021-07-15Add missing parameter for mobile UI searchÉmilien Devos
2021-07-14[fix] Qwant engines - implement API v3 and add 'quant videos'Markus Heiser
The implementation uses the Qwant API (https://api.qwant.com/v3). The API is undocumented but can be reverse engineered by reading the network log of https://www.qwant.com/ queries. This implementation is used by different qwant engines in the settings.yml:: - name: qwant categories: general ... - name: qwant news categories: news ... - name: qwant images categories: images ... - name: qwant videos categories: videos ... Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-11[drop] engine torrentz: torrentz2.eu and torrentz2.is are offlineMarkus Heiser
[1] https://torrentfreak.com/torrentz2-eu-domain-suspended-by-registry-on-public-prosecutors-order-200628/ Suggested-by: @rasos https://github.com/searx/searx/issues/1875#issuecomment-877755872 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-10Fix google imagesÉmilien Devos
Proposed fix in https://github.com/searx/searx/pull/2115#issuecomment-876716010
2021-06-21[docs] add documentation from the sources of the google enginesMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-21[fix] google answers: normalize space of the answers.Markus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-21[mod] google engine: reduce mobile UI parameters to what is neededMarkus Heiser
Reverse engineering shows that not all of the parameters used by google's mobile UI (aka "more results" button) are needed [1]. [1] https://github.com/searxng/searxng/pull/160#issuecomment-865013625 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-21[mod] google: add "use_mobile_ui" parameter to use mobile endpoint.Alexandre Flament
disable by default, it has to be enabled in settings.yml related to #159
2021-06-18[fix] google news - send CONSENT Cookie to not be redirectedMarkus Heiser
In the EU there exists a "General Data Protection Regulation" [1] aka GDPR (BTW: very user friendly!) which requires consent to tracking. To get the consent from the user, google-news requests are redirected to confirm and get a CONSENT Cookie from https://consent.google.de/s?continue=... This patch adds a CONSENT Cookie to the google-news request to avoid redirection. The behavior of the CONTENTS cookies over all google engines seems similar but the pattern is not yet fully clear to me, here are some random samples from my analysis .. Using common google search from different domains:: google.com: CONSENT=YES+cb.{{date}}-14-p0.de+FX+816 google.de: CONSENT=YES+cb.{{date}}-14-p0.de+FX+333 google.fr: CONSENT=YES+srp.gws-{{date}}-0-RC2.fr+FX+826 When searching about videos (google-videos):: google.es: CONSENT=YES+srp.gws-{{date}}-0-RC2.es+FX+076 google.de: CONSENT=YES+srp.gws-{{date}}-0-RC2.de+FX+171 Google news has only one domain for all languages:: news.google.com: CONSENT=YES+cb.{{date}}-14-p0.de+FX+816 Using google-scholar search from different domains:: scholar.google.de: CONSENT=YES+cb.{{date}}-14-p0.de+FX+333 scholar.google.fr: does not use such a cookie / did not ask the user scholar.google.es: does not use such a cookie / did not ask the user Interim summary: Pattern is unclear and I won't apply the CONSENT cookie to all google engines. More experience is need before we generalize the CONSENT cookies over all google engines. Related: - e9a6ab401 [fix] youtube - send CONSENT Cookie to not be redirected - https://github.com/benbusby/whoogle-search/issues/311 - https://github.com/benbusby/whoogle-search/issues/243 [1] https://en.wikipedia.org/wiki/General_Data_Protection_Regulation Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-18[fix] google-news engine - KeyError: 'hl in requestMarkus Heiser
Since we added - 1c67b6aec [enh] google engine: supports "default language" there is a KeyError: 'hl in request,error pattern:: ERROR:searx.searx.search.processor.online:engine google news : exception : 'hl' Traceback (most recent call last): File "searx/search/processors/online.py", line 144, in search search_results = self._search_basic(query, params) File "searx/search/processors/online.py", line 118, in _search_basic self.engine.request(query, params) File "searx/engines/google_news.py", line 97, in request if lang_info['hl'] == 'en': KeyError: 'hl' Closes: https://github.com/searxng/searxng/issues/154 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-14[pylint] searx/engines/duckduckgo_definitions.pyMarkus Heiser
BTW: normalize indentations Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-11[fix] log messages from: google- images, news, scholar, videosMarkus Heiser
- HTTP header Accept-Language --> lang_info['headers']['Accept-Language'] - remove obsolete query_url log messages which is already logged by httpx._client:HTTP request Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-11[mod] google - get_lang_info add documentataion & commentsMarkus Heiser
BTW: remove obsolete log messages from google engine Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-10[enh] google engine: supports "default language"Alexandre Flament
Same behaviour behaviour than Whoogle [1]. Only the google engine with the "Default language" choice "(all)"" is changed by this patch. When searching for a locate place, the result are in the expect language, without missing results [2]: > When a language is not specified, the language interpretation is left up to > Google to decide how the search results should be delivered. The query parameters are copied from Whoogle. With the ``all`` language: - add parameter ``source=lnt`` - don't use parameter ``lr`` - don't add a ``Accept-Language`` HTTP header. The new signature of function ``get_lang_info()`` is: lang_info = get_lang_info(params, lang_list, custom_aliases, supported_any_language) Argument ``supported_any_language`` is True for google.py and False for the other google engines. With this patch the function now returns: - query parameters: ``lang_info['params']`` - HTTP headers: ``lang_info['headers']`` - and as before this patch: - ``lang_info['subdomain']`` - ``lang_info['country']`` - ``lang_info['language']`` [1] https://github.com/benbusby/whoogle-search [2] https://github.com/benbusby/whoogle-search/releases/tag/v0.5.4
2021-06-09[fix] openstreetmap - fix some minor whitespace & indentation issuesMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-09[enh] openstreetmap / map template: improve resultsAlexandre Flament
implements ideas described in #69 * update the engine * use wikidata * update map.html template
2021-06-08Merge pull request #97 from return42/drop-searx-adminMarkus Heiser
[docs] reorder blog articles
2021-06-04[fix] peertube fetch supported languagesAlexandre Flament
close #127
2021-06-04[fix] typo: online_dictionnary --> online_dictionaryMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04[docs] add engine "Demo Online Engine"Markus Heiser
This engine just exists for documentation purpose. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04[docs] add engine "Demo Offline Engine"Markus Heiser
This engine just exists for documentation purpose. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-02Merge pull request #124 from return42/searx-mergeAlexandre Flament
merge redis offline engine from searx
2021-06-02[fix] enigine redis - avoid error when the engine is loadedMarkus Heiser
Should be _redis_client to avoid an error when the engine is loaded. Suggested-by: @dalf https://github.com/searxng/searxng/pull/124#pullrequestreview-673885664 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-01[fix] sys.exit(1) when there is duplicate engine nameAlexandre Flament
2021-06-01[mod] searx.engines.load_engine return None instead of sys.exit(1)Markus Heiser
Loading an engine should not exit the application (*). Instead of exit, return None. (*) RuntimeError still exit the application: syntax error, etc... BTW: add documentation and normalize indentation (no functional change) Suggested-by: @dalf https://github.com/searxng/searxng/pull/116#issuecomment-851865627 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-01[mod] searx.engines.__init__: refactoringAlexandre Flament
2021-06-01[enh] add redis offline engine / https://redis.io/Adam Tauber
Slightly modified merge of commit [97269be6], [01a8a5814a] and [c8d2b5eb] from searx. [97269be6] https://github.com/searx/searx/commit/97269be6 [01a8a581] https://github.com/searx/searx/commit/01a8a581 [c8d2b5eb] https://github.com/searx/searx/commit/c8d2b5eb Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-01[mod] move all default settings into searx.settings_defaultsAlexandre Flament
2021-05-30[enh] add engine - Docker HubKyle Anthony Williams
Slightly modified merge of commit [1cb1d3ac] from searx [PR 2543]: This adds Docker Hub .. as a search engine .. the engine's favicon was downloaded from the Docker Hub website with wget and converted to a PNG with ImageMagick .. It supports the parsing of URLs, titles, content, published dates, and thumbnails of Docker images. [1cb1d3ac] https://github.com/searx/searx/pull/2543/commits/1cb1d3ac [PR 2543] https://github.com/searx/searx/pull/2543 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-29[mod] the bittorent search engines are available only in the files categoryAlexandre Flament
related to #101
2021-05-28[enh] add MySQL engineNoémi Ványi
Slightly modified merge of [c00a33fe] from searx. [c00a33fe] https://github.com/searx/searx/commit/c00a33feee0886dc2b42f4c7b43731210597e5b7 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-28[enh] add PostgreSQL engineNoémi Ványi
Slightly modified merge of [22079ff] from searx. [22079ff] https://github.com/searx/searx/commit/22079ffdefb07cf69486a232e823797e737014e9 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-27Merge pull request #93 from return42/genius-miscMarkus Heiser
Some minor Genius improvements
2021-05-27Merge pull request #103 from searxng/add-sqlite-engine2Markus Heiser
[enh] add offline engine for sqlite database
2021-05-27[enh] add offline engine for sqlite databaseAlexandre Flament
To test & demonstrate this implementation download: https://liste.mediathekview.de/filmliste-v2.db.bz2 and unpack into searx/data/filmliste-v2.db, in your settings.yml define a sqlite engine named "demo":: - name : demo engine : sqlite shortcut: demo categories: general result_template: default.html database : searx/data/filmliste-v2.db query_str : >- SELECT title || ' (' || time(duration, 'unixepoch') || ')' AS title, COALESCE( NULLIF(url_video_hd,''), NULLIF(url_video_sd,''), url_video) AS url, description AS content FROM film WHERE title LIKE :wildcard OR description LIKE :wildcard ORDER BY duration DESC disabled : False Query to test: "!demo concert" This is a rewrite of the implementation from commit [1] [1] searx/searx@8e90a21 Suggested-by: @virtadpt searx/searx#2808
2021-05-25[fix] unsplash engine - 'searx:result: invalid title:'Markus Heiser
- Use result 'alt_description' as title, if not given use default title 'unknown'. - Use result 'description' from unsplash as 'content' Fix error:: DEBUG:searx:result: invalid title: {..., 'title': None, 'content': '', 'engine': 'unsplash'} Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-14 03:20:30 +0000