From 6e5f22e5583cfc2a413e0afac66d3c5ea9f628b1 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Thu, 29 Sep 2022 20:54:46 +0200 Subject: [mod] replace engines_languages.json by engines_traits.json Implementations of the *traits* of the engines. Engine's traits are fetched from the origin engine and stored in a JSON file in the *data folder*. Most often traits are languages and region codes and their mapping from SearXNG's representation to the representation in the origin search engine. To load traits from the persistence:: searx.enginelib.traits.EngineTraitsMap.from_data() For new traits new properties can be added to the class:: searx.enginelib.traits.EngineTraits .. hint:: Implementation is downward compatible to the deprecated *supported_languages method* from the vintage implementation. The vintage code is tagged as *deprecated* an can be removed when all engines has been ported to the *traits method*. Signed-off-by: Markus Heiser --- docs/admin/engines/configured_engines.rst | 2 +- docs/admin/engines/settings.rst | 7 +++-- docs/conf.py | 5 ++++ docs/dev/engine_overview.rst | 42 +++++++++++++++++++++++---- docs/dev/searxng_extra/update.rst | 8 +++--- docs/src/searx.enginelib.rst | 17 +++++++++++ docs/src/searx.engines.rst | 8 +++--- docs/src/searx.search.processors.rst | 47 +++++++++++++++++++++++++++++++ 8 files changed, 119 insertions(+), 17 deletions(-) create mode 100644 docs/src/searx.enginelib.rst create mode 100644 docs/src/searx.search.processors.rst (limited to 'docs') diff --git a/docs/admin/engines/configured_engines.rst b/docs/admin/engines/configured_engines.rst index c7b6a1f52..fa1e5a4b0 100644 --- a/docs/admin/engines/configured_engines.rst +++ b/docs/admin/engines/configured_engines.rst @@ -42,7 +42,7 @@ Explanation of the :ref:`general engine configuration` shown in the table - Timeout - Weight - Paging - - Language + - Language, Region - Safe search - Time range diff --git a/docs/admin/engines/settings.rst b/docs/admin/engines/settings.rst index 099e449e0..0d9e14e57 100644 --- a/docs/admin/engines/settings.rst +++ b/docs/admin/engines/settings.rst @@ -569,10 +569,13 @@ engine is shown. Most of the options have a default value or even are optional. To disable by default the engine, but not deleting it. It will allow the user to manually activate it in the settings. +``inactive``: optional + Remove the engine from the settings (*disabled & removed*). + ``language`` : optional If you want to use another language for a specific engine, you can define it - by using the full ISO code of language and country, like ``fr_FR``, ``en_US``, - ``de_DE``. + by using the ISO code of language (and region), like ``fr``, ``en-US``, + ``de-DE``. ``tokens`` : optional A list of secret tokens to make this engine *private*, more details see diff --git a/docs/conf.py b/docs/conf.py index 8e0c3ab1b..1d71b7f8a 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -127,6 +127,10 @@ extensions = [ 'notfound.extension', # https://github.com/readthedocs/sphinx-notfound-page ] +autodoc_default_options = { + 'member-order': 'groupwise', +} + myst_enable_extensions = [ "replacements", "smartquotes" ] @@ -135,6 +139,7 @@ suppress_warnings = ['myst.domains'] intersphinx_mapping = { "python": ("https://docs.python.org/3/", None), + "babel" : ("https://babel.readthedocs.io/en/latest/", None), "flask": ("https://flask.palletsprojects.com/", None), "flask_babel": ("https://python-babel.github.io/flask-babel/", None), # "werkzeug": ("https://werkzeug.palletsprojects.com/", None), diff --git a/docs/dev/engine_overview.rst b/docs/dev/engine_overview.rst index 95ed267e2..930fd0813 100644 --- a/docs/dev/engine_overview.rst +++ b/docs/dev/engine_overview.rst @@ -54,6 +54,7 @@ Engine File - ``offline`` :ref:`[ref] ` - ``online_dictionary`` - ``online_currency`` + - ``online_url_search`` ======================= =========== ======================================================== .. _engine settings: @@ -131,8 +132,10 @@ Passed Arguments (request) These arguments can be used to construct the search query. Furthermore, parameters with default value can be redefined for special purposes. +.. _engine request online: -.. table:: If the ``engine_type`` is ``online`` +.. table:: If the ``engine_type`` is :py:obj:`online + ` :width: 100% ====================== ============== ======================================================================== @@ -149,12 +152,16 @@ parameters with default value can be redefined for special purposes. safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict) time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year`` pageno int current pagenumber - language str specific language code like ``'en_US'``, or ``'all'`` if unspecified + searxng_locale str SearXNG's locale selected by user. Specific language code like + ``'en'``, ``'en-US'``, or ``'all'`` if unspecified. ====================== ============== ======================================================================== -.. table:: If the ``engine_type`` is ``online_dictionary``, in addition to the - ``online`` arguments: +.. _engine request online_dictionary: + +.. table:: If the ``engine_type`` is :py:obj:`online_dictionary + `, + in addition to the :ref:`online ` arguments: :width: 100% ====================== ============== ======================================================================== @@ -165,8 +172,11 @@ parameters with default value can be redefined for special purposes. query str the text query without the languages ====================== ============== ======================================================================== -.. table:: If the ``engine_type`` is ``online_currency```, in addition to the - ``online`` arguments: +.. _engine request online_currency: + +.. table:: If the ``engine_type`` is :py:obj:`online_currency + `, + in addition to the :ref:`online ` arguments: :width: 100% ====================== ============== ======================================================================== @@ -179,6 +189,26 @@ parameters with default value can be redefined for special purposes. to_name str currency name ====================== ============== ======================================================================== +.. _engine request online_url_search: + +.. table:: If the ``engine_type`` is :py:obj:`online_url_search + `, + in addition to the :ref:`online ` arguments: + :width: 100% + + ====================== ============== ======================================================================== + argument type default-value, information + ====================== ============== ======================================================================== + search_url dict URLs from the search query: + + .. code:: python + + { + 'http': str, + 'ftp': str, + 'data:image': str + } + ====================== ============== ======================================================================== Specify Request --------------- diff --git a/docs/dev/searxng_extra/update.rst b/docs/dev/searxng_extra/update.rst index d05c81409..a125303e0 100644 --- a/docs/dev/searxng_extra/update.rst +++ b/docs/dev/searxng_extra/update.rst @@ -52,12 +52,12 @@ Scripts to update static data in :origin:`searx/data/` :members: -``update_languages.py`` -======================= +``update_engine_traits.py`` +=========================== -:origin:`[source] ` +:origin:`[source] ` -.. automodule:: searxng_extra.update.update_languages +.. automodule:: searxng_extra.update.update_engine_traits :members: diff --git a/docs/src/searx.enginelib.rst b/docs/src/searx.enginelib.rst new file mode 100644 index 000000000..651a04e68 --- /dev/null +++ b/docs/src/searx.enginelib.rst @@ -0,0 +1,17 @@ +.. _searx.enginelib: + +============ +Engine model +============ + +.. automodule:: searx.enginelib + :members: + +.. _searx.enginelib.traits: + +============= +Engine traits +============= + +.. automodule:: searx.enginelib.traits + :members: diff --git a/docs/src/searx.engines.rst b/docs/src/searx.engines.rst index 687fdb0b2..4ce96b27e 100644 --- a/docs/src/searx.engines.rst +++ b/docs/src/searx.engines.rst @@ -1,8 +1,8 @@ -.. _load_engines: +.. _searx.engines: -============ -Load Engines -============ +================= +SearXNG's engines +================= .. automodule:: searx.engines :members: diff --git a/docs/src/searx.search.processors.rst b/docs/src/searx.search.processors.rst new file mode 100644 index 000000000..390680657 --- /dev/null +++ b/docs/src/searx.search.processors.rst @@ -0,0 +1,47 @@ +.. _searx.search.processors: + +================= +Search processors +================= + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +Abstract processor class +======================== + +.. automodule:: searx.search.processors.abstract + :members: + +Offline processor +================= + +.. automodule:: searx.search.processors.offline + :members: + +Online processor +================ + +.. automodule:: searx.search.processors.online + :members: + +Online currency processor +========================= + +.. automodule:: searx.search.processors.online_currency + :members: + +Online Dictionary processor +=========================== + +.. automodule:: searx.search.processors.online_dictionary + :members: + +Online URL search processor +=========================== + +.. automodule:: searx.search.processors.online_url_search + :members: -- cgit v1.2.3 From a7fe22770a830cafa4d74a7d5e6ae848c18a9f75 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sun, 2 Oct 2022 23:52:11 +0200 Subject: [mod] Peertube: re-engineered & upgrade to data_type: traits_v1 - fetch_traits(): Fetch languages from peertube's search-index source code. [mod] Include migration of the request methode from 'supported_languages' to 'traits' (EngineTraits) object. [fix] old supported_languages_url is no longer valid since the sources has been moved to a different path. - fixed code to pass pylint - request(): complete re-implementation based on the API docs [1] - response(): complete re-implementation, adds serveral fields missed before - add source code documentation [1] https://docs.joinpeertube.org/api-rest-reference.html#tag/Search/operation/searchVideos Signed-off-by: Markus Heiser --- docs/src/searx.engines.peertube.rst | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 docs/src/searx.engines.peertube.rst (limited to 'docs') diff --git a/docs/src/searx.engines.peertube.rst b/docs/src/searx.engines.peertube.rst new file mode 100644 index 000000000..757d9c742 --- /dev/null +++ b/docs/src/searx.engines.peertube.rst @@ -0,0 +1,19 @@ +.. _peertube engines: + +================ +Peertube Engines +================ + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +.. _peertube video engine: + +Peertube Video +============== + +.. automodule:: searx.engines.peertube + :members: -- cgit v1.2.3 From 61383edb27c50c3316226f88c60837d4a3c4f540 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Mon, 3 Oct 2022 17:21:13 +0200 Subject: [mod] Startpage: fetch engine traits (data_type: supported_languages) Implements a fetch_traits function for the Startpage engine. .. note:: Does not include migration of the request methode from 'supported_languages' to 'traits' (EngineTraits) object! Signed-off-by: Markus Heiser --- docs/src/searx.engines.startpage.rst | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 docs/src/searx.engines.startpage.rst (limited to 'docs') diff --git a/docs/src/searx.engines.startpage.rst b/docs/src/searx.engines.startpage.rst new file mode 100644 index 000000000..23ec63837 --- /dev/null +++ b/docs/src/searx.engines.startpage.rst @@ -0,0 +1,18 @@ +.. _startpage engines: + +================= +Startpage engines +================= + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +.. automodule:: searx.engines.startpage + +Functions +========= + +.. autofunction:: searx.engines.startpage.fetch_traits +.. autofunction:: searx.engines.startpage.get_sc_code -- cgit v1.2.3 From c9cd376186d12d2d281e655d0b5539d1359fe148 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Mon, 10 Oct 2022 19:31:22 +0200 Subject: [mod] replace searx.languages by searx.sxng_locales With the language and region tags from the EngineTraitsMap the handling of SearXNG's tags of languages and regions has been normalized and is no longer a *mystery*. The "languages" became "locales" that are supported by babel and by this, the update_engine_traits.py can be simplified a lot. Other code places can be simplified as well, but these simplifications should (respectively can) only be done when none of the engines work with the deprecated EngineTraits.supported_languages interface anymore. This commit replaces searx.languages by searx.sxng_locales and fix the naming of some names from "language" to "locale" (e.g. language_codes --> sxng_locales). Signed-off-by: Markus Heiser --- docs/src/searx.locales.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+) (limited to 'docs') diff --git a/docs/src/searx.locales.rst b/docs/src/searx.locales.rst index 579247aff..2f13bfca1 100644 --- a/docs/src/searx.locales.rst +++ b/docs/src/searx.locales.rst @@ -4,5 +4,17 @@ Locales ======= +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + .. automodule:: searx.locales :members: + + +SearXNG's locale codes +====================== + +.. automodule:: searx.sxng_locales + :members: -- cgit v1.2.3 From e0a6ca96cc071c7b02fb024d34a8902e636d0653 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Thu, 27 Oct 2022 19:21:17 +0200 Subject: [doc] add a description of bing engines (web, news, video, images) Signed-off-by: Markus Heiser --- docs/src/searx.engines.bing.rst | 43 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 docs/src/searx.engines.bing.rst (limited to 'docs') diff --git a/docs/src/searx.engines.bing.rst b/docs/src/searx.engines.bing.rst new file mode 100644 index 000000000..6b7bba8f2 --- /dev/null +++ b/docs/src/searx.engines.bing.rst @@ -0,0 +1,43 @@ +.. _bing engines: + +============ +Bing Engines +============ + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +.. _bing web engine: + +Bing WEB +======== + +.. automodule:: searx.engines.bing + :members: + +.. _bing images engine: + +Bing Images +=========== + +.. automodule:: searx.engines.bing_images + :members: + +.. _bing videos engine: + +Bing Videos +=========== + +.. automodule:: searx.engines.bing_videos + :members: + +.. _bing news engine: + +Bing News +========= + +.. automodule:: searx.engines.bing_news + :members: -- cgit v1.2.3 From 858aa3e6043a5102aec1b05e94ef1d65059f8898 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Fri, 28 Oct 2022 19:12:59 +0200 Subject: [mod] wikipedia & wikidata: upgrade to data_type: traits_v1 BTW this fix an issue in wikipedia: SearXNG's locales zh-TW and zh-HK are now using language `zh-classical` from wikipedia (and not `zh`). Signed-off-by: Markus Heiser --- docs/src/searx.engines.wikipedia.rst | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 docs/src/searx.engines.wikipedia.rst (limited to 'docs') diff --git a/docs/src/searx.engines.wikipedia.rst b/docs/src/searx.engines.wikipedia.rst new file mode 100644 index 000000000..e644cd645 --- /dev/null +++ b/docs/src/searx.engines.wikipedia.rst @@ -0,0 +1,27 @@ +.. _wikimedia engines: + +========= +Wikimedia +========= + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +.. _wikipedia engine: + +Wikipedia +========= + +.. automodule:: searx.engines.wikipedia + :members: + +.. _wikidata engine: + +Wikidata +========= + +.. automodule:: searx.engines.wikidata + :members: -- cgit v1.2.3 From e9afc4f8ce2df0b02a9b3d8664b66307255b056e Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sun, 30 Oct 2022 11:23:20 +0100 Subject: [mod] Startpage: reversed engineered & upgrade to data_type: traits_v1 One reason for the often seen CAPTCHA of the Startpage requests are the incomplete requests SearXNG sends to startpage.com: this patch is a complete new implementation of the ``request()`` function, reversed engineered from the Startpage's search form. The new implementation: - use traits of data_type: traits_v1 and drop deprecated data_type: supported_languages - adds time-range support - adds save-search support - fix searxng/searxng/issues 1884 - fix searxng/searxng/issues 1081 --> improvements to avoid CAPTCHA In preparation for more categories (News, Images, Videos ..) from Startpage, the variable ``startpage_categ`` was set up. The default value is ``web`` and other categories from Startpage are not yet implemented. Signed-off-by: Markus Heiser --- docs/src/searx.engines.startpage.rst | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) (limited to 'docs') diff --git a/docs/src/searx.engines.startpage.rst b/docs/src/searx.engines.startpage.rst index 23ec63837..c885d8f1b 100644 --- a/docs/src/searx.engines.startpage.rst +++ b/docs/src/searx.engines.startpage.rst @@ -10,9 +10,4 @@ Startpage engines :backlinks: entry .. automodule:: searx.engines.startpage - -Functions -========= - -.. autofunction:: searx.engines.startpage.fetch_traits -.. autofunction:: searx.engines.startpage.get_sc_code + :members: -- cgit v1.2.3 From c80e82a855fd388c6080066da892b9723d6037c9 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sat, 5 Nov 2022 15:10:52 +0100 Subject: [mod] DuckDuckGo: reversed engineered & upgrade to data_type: traits_v1 Partial reverse engineering of the DuckDuckGo (DDG) engines including a improved language and region handling based on the enigne.traits_v1 data. - DDG Lite - DDG Instant Answer API - DDG Images - DDG Weather docs/src/searx.engine.duckduckgo.rst: Online documentation of the DDG engines (make docs.live) searx/data/engine_traits.json Add data type "traits_v1" generated by the fetch_traits() functions from: - "duckduckgo" (WEB), - "duckduckgo images" and - "duckduckgo weather" and remove data from obsolete data type "supported_languages". searx/autocomplete.py: Reversed engineered Autocomplete from DDG. Supports DDG's languages. searx/engines/duckduckgo.py: - fetch_traits(): Fetch languages & regions from DDG. - get_ddg_lang(): Get DDG's language identifier from SearXNG's locale. DDG defines its languages by region codes. DDG-Lite does not offer a language selection to the user, only a region can be selected by the user. - Cache ``vqd`` value: The vqd value depends on the query string and is needed for the follow up pages or the images loaded by a XMLHttpRequest (DDG images). The ``vqd`` value of a search term is stored for 10min in the redis DB. - DDG Lite engine: reversed engineered request method with improved Language and region support and better ``vqd`` handling. searx/engines/duckduckgo_definitions.py: DDG Instant Answer API The *instant answers* API does not support languages, or at least we could not find out how language support should work. It seems that most of the features are based on English terms. searx/engines/duckduckgo_images.py: DDG Images Reversed engineered request method. Improved language and region handling based on cookies and the enigne.traits_v1 data. Response: add image format to the result list searx/engines/duckduckgo_weather.py: DDG Weather Improved language and region handling based on cookies and the enigne.traits_v1 data. Signed-off-by: Markus Heiser --- docs/src/searx.engine.duckduckgo.rst | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 docs/src/searx.engine.duckduckgo.rst (limited to 'docs') diff --git a/docs/src/searx.engine.duckduckgo.rst b/docs/src/searx.engine.duckduckgo.rst new file mode 100644 index 000000000..1646d4984 --- /dev/null +++ b/docs/src/searx.engine.duckduckgo.rst @@ -0,0 +1,22 @@ +.. _duckduckgo engines: + +================= +DukcDukGo engines +================= + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +.. automodule:: searx.engines.duckduckgo + :members: + +.. automodule:: searx.engines.duckduckgo_images + :members: + +.. automodule:: searx.engines.duckduckgo_definitions + :members: + +.. automodule:: searx.engines.duckduckgo_weather + :members: -- cgit v1.2.3 From 249989955497cd048fa3312d115971282983b269 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sun, 4 Dec 2022 22:57:22 +0100 Subject: [mod] Google: reversed engineered & upgrade to data_type: traits_v1 Partial reverse engineering of the Google engines including a improved language and region handling based on the engine.traits_v1 data. When ever possible the implementations of the Google engines try to make use of the async REST APIs. The get_lang_info() has been generalized to a get_google_info() function / especially the region handling has been improved by adding the cr parameter. searx/data/engine_traits.json Add data type "traits_v1" generated by the fetch_traits() functions from: - Google (WEB), - Google images, - Google news, - Google scholar and - Google videos and remove data from obsolete data type "supported_languages". A traits.custom type that maps region codes to *supported_domains* is fetched from https://www.google.com/supported_domains searx/autocomplete.py: Reversed engineered autocomplete from Google WEB. Supports Google's languages and subdomains. The old API suggestqueries.google.com/complete has been replaced by the async REST API: https://{subdomain}/complete/search?{args} searx/engines/google.py Reverse engineering and extensive testing .. - fetch_traits(): Fetch languages & regions from Google properties. - always use the async REST API (formally known as 'use_mobile_ui') - use *supported_domains* from traits - improved the result list by fetching './/div[@data-content-feature]' and parsing the type of the various *content features* --> thumbnails are added searx/engines/google_images.py Reverse engineering and extensive testing .. - fetch_traits(): Fetch languages & regions from Google properties. - use *supported_domains* from traits - if exists, freshness_date is added to the result - issue 1864: result list has been improved a lot (due to the new cr parameter) searx/engines/google_news.py Reverse engineering and extensive testing .. - fetch_traits(): Fetch languages & regions from Google properties. *supported_domains* is not needed but a ceid list has been added. - different region handling compared to Google WEB - fixed for various languages & regions (due to the new ceid parameter) / avoid CONSENT page - Google News do no longer support time range - result list has been fixed: XPath of pub_date and pub_origin searx/engines/google_videos.py - fetch_traits(): Fetch languages & regions from Google properties. - use *supported_domains* from traits - add paging support - implement a async request ('asearch': 'arc' & 'async': 'use_ac:true,_fmt:html') - simplified code (thanks to '_fmt:html' request) - issue 1359: fixed xpath of video length data searx/engines/google_scholar.py - fetch_traits(): Fetch languages & regions from Google properties. - use *supported_domains* from traits - request(): include patents & citations - response(): fixed CAPTCHA detection (Scholar has its own CATCHA manager) - hardening XPath to iterate over results - fixed XPath of pub_type (has been change from gs_ct1 to gs_cgt2 class) - issue 1769 fixed: new request implementation is no longer incompatible Signed-off-by: Markus Heiser --- docs/src/searx.engines.google.rst | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) (limited to 'docs') diff --git a/docs/src/searx.engines.google.rst b/docs/src/searx.engines.google.rst index 2d10b5eea..9c15325f8 100644 --- a/docs/src/searx.engines.google.rst +++ b/docs/src/searx.engines.google.rst @@ -12,15 +12,21 @@ Google Engines .. _google API: -google API +Google API ========== .. _Query Parameter Definitions: https://developers.google.com/custom-search/docs/xml_results#WebSearch_Query_Parameter_Definitions +SearXNG's implementation of the Google API is mainly done in +:py:obj:`get_google_info `. + For detailed description of the *REST-full* API see: `Query Parameter -Definitions`_. Not all parameters can be appied and some engines are *special* -(e.g. :ref:`google news engine`). +Definitions`_. The linked API documentation can sometimes be helpful during +reverse engineering. However, we cannot use it in the freely accessible WEB +services; not all parameters can be applied and some engines are more *special* +than other (e.g. :ref:`google news engine`). + .. _google web engine: @@ -30,6 +36,13 @@ Google WEB .. automodule:: searx.engines.google :members: +.. _google autocomplete: + +Google Autocomplete +==================== + +.. autofunction:: searx.autocomplete.google_complete + .. _google images engine: Google Images @@ -53,3 +66,11 @@ Google News .. automodule:: searx.engines.google_news :members: + +.. _google scholar engine: + +Google Scholar +============== + +.. automodule:: searx.engines.google_scholar + :members: -- cgit v1.2.3 From 8a8c584fec2b8404b294ca5012d0ced5bc26d986 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sun, 25 Dec 2022 15:33:46 +0100 Subject: [mod] Dailymotion: improved request API & upgrade to data_type: traits_v1 - fetch_traits(): fetch locales (and languages) from dailymotion API - removed obsolete data-type "supported_languages" - add documentation - improved argument list of the HTTP request: - add argument: family_filter_map - add conditional argument: localization Don't add localization and country arguments if the user does select a language (:de, :en, ..) - improve code quality (mainly improve readability) Signed-off-by: Markus Heiser --- docs/src/searx.engine.dailymotion.rst | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 docs/src/searx.engine.dailymotion.rst (limited to 'docs') diff --git a/docs/src/searx.engine.dailymotion.rst b/docs/src/searx.engine.dailymotion.rst new file mode 100644 index 000000000..84348e2d0 --- /dev/null +++ b/docs/src/searx.engine.dailymotion.rst @@ -0,0 +1,8 @@ +.. _dailymotion engine: + +=========== +Dailymotion +=========== + +.. automodule:: searx.engines.dailymotion + :members: -- cgit v1.2.3 From 057e9bc1d1314e6f776274ea94cad75d05065c96 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sun, 1 Jan 2023 15:50:21 +0100 Subject: [mod] SepiaSearch: re-engineered & upgrade to data_type: traits_v1 - fetch_traits() SepiaSearch and Peertube are using identical languages. Replace module's dictionary `supported_languages` by `engine.traits.languages` (data_type: `traits_v1`). - fixed code to pass pylint - request(): add argument boostLanguages - response(): is replaced by peertube's video_response() function, which adds metadata from channel name, host & tags Signed-off-by: Markus Heiser --- docs/src/searx.engines.peertube.rst | 8 ++++++++ 1 file changed, 8 insertions(+) (limited to 'docs') diff --git a/docs/src/searx.engines.peertube.rst b/docs/src/searx.engines.peertube.rst index 757d9c742..8e1576ea0 100644 --- a/docs/src/searx.engines.peertube.rst +++ b/docs/src/searx.engines.peertube.rst @@ -17,3 +17,11 @@ Peertube Video .. automodule:: searx.engines.peertube :members: + +.. _sepiasearch engine: + +SepiaSearch +=========== + +.. automodule:: searx.engines.sepiasearch + :members: -- cgit v1.2.3 From 96a2eec3b5e20afda89d9e3a81b09ca1612dc186 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sat, 7 Jan 2023 16:04:19 +0100 Subject: [mod] Archlinux Wiki: improved request API & upgrade to data_type: traits_v1 re-implementation of the Archlinux Wiki: - fetch_traits(): fetch languages, wiki URLs and title arguments - add content field to the result list - add documentation Wikis from wiki.archlinux.fr, wiki.archlinux.ro, archtr.org/wiki do no longer exists (has been merged in the main wiki). Signed-off-by: Markus Heiser --- docs/src/searx.engine.archlinux.rst | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 docs/src/searx.engine.archlinux.rst (limited to 'docs') diff --git a/docs/src/searx.engine.archlinux.rst b/docs/src/searx.engine.archlinux.rst new file mode 100644 index 000000000..be48b1859 --- /dev/null +++ b/docs/src/searx.engine.archlinux.rst @@ -0,0 +1,9 @@ +.. _archlinux engine: + +========== +Arch Linux +========== + +.. automodule:: searx.engines.archlinux + :members: + -- cgit v1.2.3