From 11d6d80d8831459161ac08ad0587ab8775ffd3e3 Mon Sep 17 00:00:00 2001 From: Adam Tauber Date: Sat, 28 Nov 2015 19:26:45 +0100 Subject: [doc] engines overview added --- docs/dev/engine_overview.rst | 314 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 314 insertions(+) create mode 100644 docs/dev/engine_overview.rst (limited to 'docs/dev/engine_overview.rst') diff --git a/docs/dev/engine_overview.rst b/docs/dev/engine_overview.rst new file mode 100644 index 000000000..7f170e9d6 --- /dev/null +++ b/docs/dev/engine_overview.rst @@ -0,0 +1,314 @@ +Engine overview +=============== + + +searx is a `metasearch-engine `__, +so it is using different search engines to provide better results. + +Because there is no general search-api which can be used for every +search-engine, there must be build an adapter between searx and the +external search-engine. This adapters are stored in the folder +`*searx/engines* `__, +and this site is build to make an general documentation about this +engines + + +.. contents:: + :depth: 3 + +general engine configuration +---------------------------- + +It is required to tell searx what results can the engine provide. The +arguments can be inserted in the engine file, or in the settings file +(normally ``settings.yml``). The arguments in the settings file override +the one in the engine file. + +Really, it is for most options no difference if there are contained in +the engine-file or in the settings. But there is a standard where to +place specific arguments by default. + + +engine-file +~~~~~~~~~~~ + ++---------------------+-----------+-----------------------------------------+ +| argument | type | information | ++=====================+===========+=========================================+ +| categories | list | pages, in which the engine is working | ++---------------------+-----------+-----------------------------------------+ +| paging | boolean | support multible pages | ++---------------------+-----------+-----------------------------------------+ +| language\_support | boolean | support language choosing | ++---------------------+-----------+-----------------------------------------+ + +settings.yml +~~~~~~~~~~~~ + ++------------+----------+-----------------------------------------------+ +| argument | type | information | ++============+==========+===============================================+ +| name | string | name of search-engine | ++------------+----------+-----------------------------------------------+ +| engine | string | name of searx-engine (filename without .py) | ++------------+----------+-----------------------------------------------+ +| shortcut | string | shortcut of search-engine | ++------------+----------+-----------------------------------------------+ +| timeout | string | specific timeout for search-engine | ++------------+----------+-----------------------------------------------+ + +overrides +~~~~~~~~~ + +There are some options, with have default values in the engine, but are +often overwritten by the settings. If the option is assigned in the +engine-file with ``None`` it has to be redefined in the settings, +otherwise searx is not starting with that engine. + +The naming of that overrides can be wathever you want. But we recommend +the using of already used overrides if possible: + ++-----------------------+----------+--------------------------------------------------------------+ +| argument | type | information | ++=======================+==========+==============================================================+ +| base\_url | string | base-url, can be overwrite to use same engine on other url | ++-----------------------+----------+--------------------------------------------------------------+ +| number\_of\_results | int | maximum number of results per request | ++-----------------------+----------+--------------------------------------------------------------+ +| language | string | ISO code of language and country like en\_US | ++-----------------------+----------+--------------------------------------------------------------+ +| api\_key | string | api-key if required by engine | ++-----------------------+----------+--------------------------------------------------------------+ + +example-code +~~~~~~~~~~~~ + +.. code:: python + + # engine dependent config + categories = ['general'] + paging = True + language_support = True + +doing request +------------- + +To perform a search you have to specific at least a url on which the +request is performing + +passed arguments +~~~~~~~~~~~~~~~~ + +This arguments can be used to calculate the search-query. Furthermore, +some of that parameters are filled with default values which can be +changed for special purpose. + ++----------------------+------------+------------------------------------------------------------------------+ +| argument | type | default-value, informations | ++======================+============+========================================================================+ +| url | string | ``''`` | ++----------------------+------------+------------------------------------------------------------------------+ +| method | string | ``'GET'`` | ++----------------------+------------+------------------------------------------------------------------------+ +| headers | set | ``{}`` | ++----------------------+------------+------------------------------------------------------------------------+ +| data | set | ``{}`` | ++----------------------+------------+------------------------------------------------------------------------+ +| cookies | set | ``{}`` | ++----------------------+------------+------------------------------------------------------------------------+ +| verify | boolean | ``True`` | ++----------------------+------------+------------------------------------------------------------------------+ +| headers.User-Agent | string | a random User-Agent | ++----------------------+------------+------------------------------------------------------------------------+ +| category | string | current category, like ``'general'`` | ++----------------------+------------+------------------------------------------------------------------------+ +| started | datetime | current date-time | ++----------------------+------------+------------------------------------------------------------------------+ +| pageno | int | current pagenumber | ++----------------------+------------+------------------------------------------------------------------------+ +| language | string | specific language code like ``'en_US'``, or ``'all'`` if unspecified | ++----------------------+------------+------------------------------------------------------------------------+ + +parsed arguments +~~~~~~~~~~~~~~~~ + +The function ``def request(query, params):`` is always returning the +``params`` variable back. Inside searx, the following paramters can be +used to specific a search-request: + ++------------+-----------+----------------------------------------------------------+ +| argument | type | information | ++============+===========+==========================================================+ +| url | string | requested url | ++------------+-----------+----------------------------------------------------------+ +| method | string | HTTP request methode | ++------------+-----------+----------------------------------------------------------+ +| headers | set | HTTP header informations | ++------------+-----------+----------------------------------------------------------+ +| data | set | HTTP data informations (parsed if ``method != 'GET'``) | ++------------+-----------+----------------------------------------------------------+ +| cookies | set | HTTP cookies | ++------------+-----------+----------------------------------------------------------+ +| verify | boolean | Performing SSL-Validity check | ++------------+-----------+----------------------------------------------------------+ + +example-code +~~~~~~~~~~~~ + +.. code:: python + + # search-url + base_url = 'https://example.com/' + search_string = 'search?{query}&page={page}' + + # do search-request + def request(query, params): + search_path = search_string.format( + query=urlencode({'q': query}), + page=params['pageno']) + + params['url'] = base_url + search_path + + return params + +returning results +----------------- + +Searx has the possiblity to return results in different media-types. +Currently the following media-types are supported: + +- default +- images +- videos +- torrent +- map + +to set another media-type as default, you must set the parameter +``template`` to the required type. + +default +~~~~~~~ + ++--------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| result-parameter | information | ++====================+====================================================================================================================================+ +| url | string, which is representing the url of the result | ++--------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| title | string, which is representing the title of the result | ++--------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| content | string, which is giving a general result-text | ++--------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| publishedDate | `datetime.datetime `__, represent when the result is published | ++--------------------+------------------------------------------------------------------------------------------------------------------------------------+ + +images +~~~~~~ + +to use this template, the parameter + ++--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| result-parameter | information | ++====================+===========================================================================================================================================================+ +| template | is set to ``images.html`` | ++--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| url | string, which is representing the url to the result site | ++--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| title | string, which is representing the title of the result *(partly implemented)* | ++--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| content | *(partly implemented)* | ++--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| publishedDate | `datetime.datetime `__, represent when the result is published *(partly implemented)* | ++--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| img\_src | string, which is representing the url to the result image | ++--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| thumbnail\_src | string, which is representing the url to a small-preview image | ++--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ + +videos +~~~~~~ + ++--------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| result-parameter | information | ++====================+====================================================================================================================================+ +| template | is set to ``videos.html`` | ++--------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| url | string, which is representing the url of the result | ++--------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| title | string, which is representing the title of the result | ++--------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| content | *(not implemented yet)* | ++--------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| publishedDate | `datetime.datetime `__, represent when the result is published | ++--------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| thumbnail | string, which is representing the url to a small-preview image | ++--------------------+------------------------------------------------------------------------------------------------------------------------------------+ + +torrent +~~~~~~~ + ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ +| result-parameter | information | ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ +| template | is set to ```torrent.html``` | ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ +| url | string, which is representing the url of the result | ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ +| title | string, which is representing the title of the result | ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ +| content | string, which is giving a general result-text | ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ +| publishedDate | [datetime.datetime](https://docs.python.org/2/library/datetime.html#datetime-objects), represent when the result is published _(not implemented yet)_ | ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ +| seed | int, number of seeder | ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ +| leech | int, number of leecher | ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ +| filesize | int, size of file in bytes | ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ +| files | int, number of files | ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ +| magnetlink | string, which is the [magnetlink](https://en.wikipedia.org/wiki/Magnet_URI_scheme) of the result | ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ +| torrentfile | string, which is the torrentfile of the result | ++------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ + + +map +~~~ + ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| result-parameter | information | ++=========================+====================================================================================================================================+ +| url | string, which is representing the url of the result | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| title | string, which is representing the title of the result | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| content | string, which is giving a general result-text | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| publishedDate | `datetime.datetime `__, represent when the result is published | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| latitude | latitude of result (in decimal format) | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| longitude | longitude of result (in decimal format) | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| boundingbox | boundingbox of result (array of 4. values ``[lat-min, lat-max, lon-min, lon-max]``) | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| geojson | geojson of result (http://geojson.org) | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| osm.type | type of osm-object (if OSM-Result) | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| osm.id | id of osm-object (if OSM-Result) | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| address.name | name of object | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| address.road | street adress of object | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| address.house\_number | house number of object | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| address.locality | city, place of object | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| address.postcode | postcode of object | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ +| address.country | country of object | ++-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + -- cgit v1.2.3 From ae69a58089cc195d211cd195c79cd60bb49a474f Mon Sep 17 00:00:00 2001 From: Noemi Vanyi Date: Tue, 1 Dec 2015 15:32:06 +0100 Subject: [doc] grammar & format fixes in engine overview --- docs/dev/engine_overview.rst | 185 +++++++++++++++++++++---------------------- 1 file changed, 91 insertions(+), 94 deletions(-) (limited to 'docs/dev/engine_overview.rst') diff --git a/docs/dev/engine_overview.rst b/docs/dev/engine_overview.rst index 7f170e9d6..05a54b641 100644 --- a/docs/dev/engine_overview.rst +++ b/docs/dev/engine_overview.rst @@ -3,14 +3,13 @@ Engine overview searx is a `metasearch-engine `__, -so it is using different search engines to provide better results. +so it uses different search engines to provide better results. -Because there is no general search-api which can be used for every -search-engine, there must be build an adapter between searx and the -external search-engine. This adapters are stored in the folder -`*searx/engines* `__, -and this site is build to make an general documentation about this -engines +Because there is no general search API which could be used for every +search engine, an adapter has to be built between searx and the +external search engines. Adapters are stored under the folder +`searx/engines +`__. .. contents:: @@ -19,17 +18,16 @@ engines general engine configuration ---------------------------- -It is required to tell searx what results can the engine provide. The -arguments can be inserted in the engine file, or in the settings file +It is required to tell searx the type of results the engine provides. The +arguments can be set in the engine file or in the settings file (normally ``settings.yml``). The arguments in the settings file override -the one in the engine file. +the ones in the engine file. -Really, it is for most options no difference if there are contained in -the engine-file or in the settings. But there is a standard where to -place specific arguments by default. +It does not matter if an options is stored in the engine file or in the +settings. However, the standard way is the following: -engine-file +engine file ~~~~~~~~~~~ +---------------------+-----------+-----------------------------------------+ @@ -60,27 +58,27 @@ settings.yml overrides ~~~~~~~~~ -There are some options, with have default values in the engine, but are -often overwritten by the settings. If the option is assigned in the -engine-file with ``None`` it has to be redefined in the settings, -otherwise searx is not starting with that engine. - -The naming of that overrides can be wathever you want. But we recommend -the using of already used overrides if possible: - -+-----------------------+----------+--------------------------------------------------------------+ -| argument | type | information | -+=======================+==========+==============================================================+ -| base\_url | string | base-url, can be overwrite to use same engine on other url | -+-----------------------+----------+--------------------------------------------------------------+ -| number\_of\_results | int | maximum number of results per request | -+-----------------------+----------+--------------------------------------------------------------+ -| language | string | ISO code of language and country like en\_US | -+-----------------------+----------+--------------------------------------------------------------+ -| api\_key | string | api-key if required by engine | -+-----------------------+----------+--------------------------------------------------------------+ - -example-code +A few of the options have default values in the engine, but are +often overwritten by the settings. If ``None`` is assigned to an option +in the engine file, it has to be redefined in the settings, +otherwise searx will not start with that engine. + +The naming of that overrides is arbitrary. But the recommended +overrides are the following: + ++-----------------------+----------+----------------------------------------------------------------+ +| argument | type | information | ++=======================+==========+================================================================+ +| base\_url | string | base-url, can be overwritten to use same engine on other URL | ++-----------------------+----------+----------------------------------------------------------------+ +| number\_of\_results | int | maximum number of results per request | ++-----------------------+----------+----------------------------------------------------------------+ +| language | string | ISO code of language and country like en\_US | ++-----------------------+----------+----------------------------------------------------------------+ +| api\_key | string | api-key if required by engine | ++-----------------------+----------+----------------------------------------------------------------+ + +example code ~~~~~~~~~~~~ .. code:: python @@ -90,21 +88,20 @@ example-code paging = True language_support = True -doing request -------------- +making a request +---------------- -To perform a search you have to specific at least a url on which the -request is performing +To perform a search an URL have to be specified. In addition to +specifying an URL, arguments can be passed to the query. passed arguments ~~~~~~~~~~~~~~~~ -This arguments can be used to calculate the search-query. Furthermore, -some of that parameters are filled with default values which can be -changed for special purpose. +These arguments can be used to construct the search query. Furthermore, +parameters with default value can be redefined for special purposes. +----------------------+------------+------------------------------------------------------------------------+ -| argument | type | default-value, informations | +| argument | type | default-value, information | +======================+============+========================================================================+ | url | string | ``''`` | +----------------------+------------+------------------------------------------------------------------------+ @@ -132,27 +129,27 @@ changed for special purpose. parsed arguments ~~~~~~~~~~~~~~~~ -The function ``def request(query, params):`` is always returning the -``params`` variable back. Inside searx, the following paramters can be -used to specific a search-request: - -+------------+-----------+----------------------------------------------------------+ -| argument | type | information | -+============+===========+==========================================================+ -| url | string | requested url | -+------------+-----------+----------------------------------------------------------+ -| method | string | HTTP request methode | -+------------+-----------+----------------------------------------------------------+ -| headers | set | HTTP header informations | -+------------+-----------+----------------------------------------------------------+ -| data | set | HTTP data informations (parsed if ``method != 'GET'``) | -+------------+-----------+----------------------------------------------------------+ -| cookies | set | HTTP cookies | -+------------+-----------+----------------------------------------------------------+ -| verify | boolean | Performing SSL-Validity check | -+------------+-----------+----------------------------------------------------------+ - -example-code +The function ``def request(query, params):`` always returns the +``params`` variable. Inside searx, the following paramters can be +used to specify a search request: + ++------------+-----------+---------------------------------------------------------+ +| argument | type | information | ++============+===========+=========================================================+ +| url | string | requested url | ++------------+-----------+---------------------------------------------------------+ +| method | string | HTTP request method | ++------------+-----------+---------------------------------------------------------+ +| headers | set | HTTP header information | ++------------+-----------+---------------------------------------------------------+ +| data | set | HTTP data information (parsed if ``method != 'GET'``) | ++------------+-----------+---------------------------------------------------------+ +| cookies | set | HTTP cookies | ++------------+-----------+---------------------------------------------------------+ +| verify | boolean | Performing SSL-Validity check | ++------------+-----------+---------------------------------------------------------+ + +example code ~~~~~~~~~~~~ .. code:: python @@ -171,10 +168,10 @@ example-code return params -returning results ------------------ +returned results +---------------- -Searx has the possiblity to return results in different media-types. +Searx is able to return results of different media-types. Currently the following media-types are supported: - default @@ -183,8 +180,8 @@ Currently the following media-types are supported: - torrent - map -to set another media-type as default, you must set the parameter -``template`` to the required type. +To set another media-type as default, the parameter +``template`` must be set to the desired type. default ~~~~~~~ @@ -246,31 +243,31 @@ videos torrent ~~~~~~~ -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ -| result-parameter | information | -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ -| template | is set to ```torrent.html``` | -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ -| url | string, which is representing the url of the result | -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ -| title | string, which is representing the title of the result | -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ -| content | string, which is giving a general result-text | -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ -| publishedDate | [datetime.datetime](https://docs.python.org/2/library/datetime.html#datetime-objects), represent when the result is published _(not implemented yet)_ | -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ -| seed | int, number of seeder | -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ -| leech | int, number of leecher | -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ -| filesize | int, size of file in bytes | -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ -| files | int, number of files | -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ -| magnetlink | string, which is the [magnetlink](https://en.wikipedia.org/wiki/Magnet_URI_scheme) of the result | -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ -| torrentfile | string, which is the torrentfile of the result | -+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------+ ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| result-parameter | information | ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| template | is set to ```torrent.html``` | ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| url | string, which is representing the url of the result | ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| title | string, which is representing the title of the result | ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| content | string, which is giving a general result-text | ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| publishedDate | `datetime.datetime `__, represent when the result is published *(not implemented yet)* | ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| seed | int, number of seeder | ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| leech | int, number of leecher | ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| filesize | int, size of file in bytes | ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| files | int, number of files | ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| magnetlink | string, which is the `magnetlink `__ of the result | ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ +| torrentfile | string, which is the torrentfile of the result | ++------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ map -- cgit v1.2.3 From 240802003ee98e93e084788665445d81a9623c7d Mon Sep 17 00:00:00 2001 From: Noemi Vanyi Date: Thu, 10 Dec 2015 21:38:39 +0100 Subject: [doc] fix small mistakes in engine overview --- docs/dev/engine_overview.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'docs/dev/engine_overview.rst') diff --git a/docs/dev/engine_overview.rst b/docs/dev/engine_overview.rst index 05a54b641..186cc0f94 100644 --- a/docs/dev/engine_overview.rst +++ b/docs/dev/engine_overview.rst @@ -23,7 +23,7 @@ arguments can be set in the engine file or in the settings file (normally ``settings.yml``). The arguments in the settings file override the ones in the engine file. -It does not matter if an options is stored in the engine file or in the +It does not matter if an option is stored in the engine file or in the settings. However, the standard way is the following: @@ -63,7 +63,7 @@ often overwritten by the settings. If ``None`` is assigned to an option in the engine file, it has to be redefined in the settings, otherwise searx will not start with that engine. -The naming of that overrides is arbitrary. But the recommended +The naming of overrides is arbitrary. But the recommended overrides are the following: +-----------------------+----------+----------------------------------------------------------------+ -- cgit v1.2.3 From c2cf46092e7e805587fde191ea9c620ee2bcb158 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?No=C3=A9mi=20V=C3=A1nyi?= Date: Fri, 4 Nov 2016 18:38:07 +0100 Subject: docs: fix typos based on #653 --- docs/dev/engine_overview.rst | 226 ++++++++++++++++++++++--------------------- 1 file changed, 114 insertions(+), 112 deletions(-) (limited to 'docs/dev/engine_overview.rst') diff --git a/docs/dev/engine_overview.rst b/docs/dev/engine_overview.rst index 186cc0f94..da7ee2710 100644 --- a/docs/dev/engine_overview.rst +++ b/docs/dev/engine_overview.rst @@ -30,15 +30,17 @@ settings. However, the standard way is the following: engine file ~~~~~~~~~~~ -+---------------------+-----------+-----------------------------------------+ -| argument | type | information | -+=====================+===========+=========================================+ -| categories | list | pages, in which the engine is working | -+---------------------+-----------+-----------------------------------------+ -| paging | boolean | support multible pages | -+---------------------+-----------+-----------------------------------------+ -| language\_support | boolean | support language choosing | -+---------------------+-----------+-----------------------------------------+ ++----------------------+-----------+-----------------------------------------+ +| argument | type | information | ++======================+===========+=========================================+ +| categories | list | pages, in which the engine is working | ++----------------------+-----------+-----------------------------------------+ +| paging | boolean | support multible pages | ++----------------------+-----------+-----------------------------------------+ +| language\_support | boolean | support language choosing | ++----------------------+-----------+-----------------------------------------+ +| time\_range\_support | boolean | support search time range | ++----------------------+-----------+-----------------------------------------+ settings.yml ~~~~~~~~~~~~ @@ -186,126 +188,126 @@ To set another media-type as default, the parameter default ~~~~~~~ -+--------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| result-parameter | information | -+====================+====================================================================================================================================+ -| url | string, which is representing the url of the result | -+--------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| title | string, which is representing the title of the result | -+--------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| content | string, which is giving a general result-text | -+--------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| publishedDate | `datetime.datetime `__, represent when the result is published | -+--------------------+------------------------------------------------------------------------------------------------------------------------------------+ ++--------------------+---------------------------------------------------------------------------------------------------------------+ +| result-parameter | information | ++====================+===============================================================================================================+ +| url | string, url of the result | ++--------------------+---------------------------------------------------------------------------------------------------------------+ +| title | string, title of the result | ++--------------------+---------------------------------------------------------------------------------------------------------------+ +| content | string, general result-text | ++--------------------+---------------------------------------------------------------------------------------------------------------+ +| publishedDate | `datetime.datetime `__, time of publish | ++--------------------+---------------------------------------------------------------------------------------------------------------+ images ~~~~~~ to use this template, the parameter -+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| result-parameter | information | -+====================+===========================================================================================================================================================+ -| template | is set to ``images.html`` | -+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| url | string, which is representing the url to the result site | -+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| title | string, which is representing the title of the result *(partly implemented)* | -+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| content | *(partly implemented)* | -+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| publishedDate | `datetime.datetime `__, represent when the result is published *(partly implemented)* | -+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| img\_src | string, which is representing the url to the result image | -+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| thumbnail\_src | string, which is representing the url to a small-preview image | -+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ ++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| result-parameter | information | ++====================+=======================================================================================================================================+ +| template | is set to ``images.html`` | ++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| url | string, url to the result site | ++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| title | string, title of the result *(partly implemented)* | ++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| content | *(partly implemented)* | ++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| publishedDate | `datetime.datetime `__, time of publish *(partly implemented)* | ++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| img\_src | string, url to the result image | ++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| thumbnail\_src | string, url to a small-preview image | ++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ videos ~~~~~~ -+--------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| result-parameter | information | -+====================+====================================================================================================================================+ -| template | is set to ``videos.html`` | -+--------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| url | string, which is representing the url of the result | -+--------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| title | string, which is representing the title of the result | -+--------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| content | *(not implemented yet)* | -+--------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| publishedDate | `datetime.datetime `__, represent when the result is published | -+--------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| thumbnail | string, which is representing the url to a small-preview image | -+--------------------+------------------------------------------------------------------------------------------------------------------------------------+ ++--------------------+--------------------------------------------------------------------------------------------------------------+ +| result-parameter | information | ++====================+==============================================================================================================+ +| template | is set to ``videos.html`` | ++--------------------+--------------------------------------------------------------------------------------------------------------+ +| url | string, url of the result | ++--------------------+--------------------------------------------------------------------------------------------------------------+ +| title | string, title of the result | ++--------------------+--------------------------------------------------------------------------------------------------------------+ +| content | *(not implemented yet)* | ++--------------------+--------------------------------------------------------------------------------------------------------------+ +| publishedDate | `datetime.datetime `__, time of publish | ++--------------------+--------------------------------------------------------------------------------------------------------------+ +| thumbnail | string, url to a small-preview image | ++--------------------+--------------------------------------------------------------------------------------------------------------+ torrent ~~~~~~~ -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| result-parameter | information | -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| template | is set to ```torrent.html``` | -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| url | string, which is representing the url of the result | -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| title | string, which is representing the title of the result | -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| content | string, which is giving a general result-text | -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| publishedDate | `datetime.datetime `__, represent when the result is published *(not implemented yet)* | -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| seed | int, number of seeder | -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| leech | int, number of leecher | -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| filesize | int, size of file in bytes | -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| files | int, number of files | -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| magnetlink | string, which is the `magnetlink `__ of the result | -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ -| torrentfile | string, which is the torrentfile of the result | -+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+ ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| result-parameter | information | ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| template | is set to ``torrent.html`` | ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| url | string, url of the result | ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| title | string, title of the result | ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| content | string, general result-text | ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| publishedDate | `datetime.datetime `__, time of publish *(not implemented yet)* | ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| seed | int, number of seeder | ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| leech | int, number of leecher | ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| filesize | int, size of file in bytes | ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| files | int, number of files | ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| magnetlink | string, `magnetlink `__ of the result | ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +| torrentfile | string, torrentfile of the result | ++------------------+---------------------------------------------------------------------------------------------------------------------------------------+ map ~~~ -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| result-parameter | information | -+=========================+====================================================================================================================================+ -| url | string, which is representing the url of the result | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| title | string, which is representing the title of the result | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| content | string, which is giving a general result-text | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| publishedDate | `datetime.datetime `__, represent when the result is published | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| latitude | latitude of result (in decimal format) | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| longitude | longitude of result (in decimal format) | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| boundingbox | boundingbox of result (array of 4. values ``[lat-min, lat-max, lon-min, lon-max]``) | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| geojson | geojson of result (http://geojson.org) | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| osm.type | type of osm-object (if OSM-Result) | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| osm.id | id of osm-object (if OSM-Result) | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| address.name | name of object | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| address.road | street adress of object | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| address.house\_number | house number of object | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| address.locality | city, place of object | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| address.postcode | postcode of object | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ -| address.country | country of object | -+-------------------------+------------------------------------------------------------------------------------------------------------------------------------+ ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| result-parameter | information | ++=========================+==============================================================================================================+ +| url | string, url of the result | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| title | string, title of the result | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| content | string, general result-text | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| publishedDate | `datetime.datetime `__, time of publish | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| latitude | latitude of result (in decimal format) | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| longitude | longitude of result (in decimal format) | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| boundingbox | boundingbox of result (array of 4. values ``[lat-min, lat-max, lon-min, lon-max]``) | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| geojson | geojson of result (http://geojson.org) | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| osm.type | type of osm-object (if OSM-Result) | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| osm.id | id of osm-object (if OSM-Result) | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| address.name | name of object | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| address.road | street name of object | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| address.house\_number | house number of object | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| address.locality | city, place of object | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| address.postcode | postcode of object | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ +| address.country | country of object | ++-------------------------+--------------------------------------------------------------------------------------------------------------+ -- cgit v1.2.3 From 50f25193c17707157c905f455b1d3a27c3edc001 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?No=C3=A9mi=20V=C3=A1nyi?= Date: Sat, 5 Nov 2016 16:22:29 +0100 Subject: engine overview fix torrent table --- docs/dev/engine_overview.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/dev/engine_overview.rst') diff --git a/docs/dev/engine_overview.rst b/docs/dev/engine_overview.rst index da7ee2710..d57ca5b38 100644 --- a/docs/dev/engine_overview.rst +++ b/docs/dev/engine_overview.rst @@ -247,7 +247,7 @@ torrent +------------------+---------------------------------------------------------------------------------------------------------------------------------------+ | result-parameter | information | -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ ++==================+=======================================================================================================================================+ | template | is set to ``torrent.html`` | +------------------+---------------------------------------------------------------------------------------------------------------------------------------+ | url | string, url of the result | -- cgit v1.2.3 From 021e1da4c9a8eca7293ad62608cc55da5dfde73f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?No=C3=A9mi=20V=C3=A1nyi?= Date: Wed, 23 Oct 2019 13:06:19 +0200 Subject: add post about introducing offline engines --- docs/dev/engine_overview.rst | 2 ++ 1 file changed, 2 insertions(+) (limited to 'docs/dev/engine_overview.rst') diff --git a/docs/dev/engine_overview.rst b/docs/dev/engine_overview.rst index d57ca5b38..a6867b5d0 100644 --- a/docs/dev/engine_overview.rst +++ b/docs/dev/engine_overview.rst @@ -41,6 +41,8 @@ engine file +----------------------+-----------+-----------------------------------------+ | time\_range\_support | boolean | support search time range | +----------------------+-----------+-----------------------------------------+ +| offline | boolean | engine runs offline | ++----------------------+-----------+-----------------------------------------+ settings.yml ~~~~~~~~~~~~ -- cgit v1.2.3 From af2cae6d1d71859e867a9fbc5da604ef6a898794 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Wed, 4 Dec 2019 16:48:36 +0100 Subject: doc: make use of sphinx.ext.extlinks & sphinx.ext.intersphinx - add sphinx extensions - patch documentation to make use of These modules help to simplify the reST markup of external references. BTW it helps to write more readable reST and form custom brands. Signed-off-by: Markus Heiser --- docs/dev/engine_overview.rst | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) (limited to 'docs/dev/engine_overview.rst') diff --git a/docs/dev/engine_overview.rst b/docs/dev/engine_overview.rst index a6867b5d0..937ca8880 100644 --- a/docs/dev/engine_overview.rst +++ b/docs/dev/engine_overview.rst @@ -8,8 +8,7 @@ so it uses different search engines to provide better results. Because there is no general search API which could be used for every search engine, an adapter has to be built between searx and the external search engines. Adapters are stored under the folder -`searx/engines -`__. +:origin:`searx/engines`. .. contents:: @@ -199,7 +198,7 @@ default +--------------------+---------------------------------------------------------------------------------------------------------------+ | content | string, general result-text | +--------------------+---------------------------------------------------------------------------------------------------------------+ -| publishedDate | `datetime.datetime `__, time of publish | +| publishedDate | :py:class:`datetime.datetime`, time of publish | +--------------------+---------------------------------------------------------------------------------------------------------------+ images @@ -218,7 +217,7 @@ to use this template, the parameter +--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ | content | *(partly implemented)* | +--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| publishedDate | `datetime.datetime `__, time of publish *(partly implemented)* | +| publishedDate | :py:class:`datetime.datetime`, time of publish *(partly implemented)* | +--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ | img\_src | string, url to the result image | +--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ @@ -239,7 +238,7 @@ videos +--------------------+--------------------------------------------------------------------------------------------------------------+ | content | *(not implemented yet)* | +--------------------+--------------------------------------------------------------------------------------------------------------+ -| publishedDate | `datetime.datetime `__, time of publish | +| publishedDate | :py:class:`datetime.datetime`, time of publish | +--------------------+--------------------------------------------------------------------------------------------------------------+ | thumbnail | string, url to a small-preview image | +--------------------+--------------------------------------------------------------------------------------------------------------+ @@ -258,7 +257,7 @@ torrent +------------------+---------------------------------------------------------------------------------------------------------------------------------------+ | content | string, general result-text | +------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| publishedDate | `datetime.datetime `__, time of publish *(not implemented yet)* | +| publishedDate | :py:class:`datetime.datetime`, time of publish *(not implemented yet)* | +------------------+---------------------------------------------------------------------------------------------------------------------------------------+ | seed | int, number of seeder | +------------------+---------------------------------------------------------------------------------------------------------------------------------------+ @@ -286,7 +285,7 @@ map +-------------------------+--------------------------------------------------------------------------------------------------------------+ | content | string, general result-text | +-------------------------+--------------------------------------------------------------------------------------------------------------+ -| publishedDate | `datetime.datetime `__, time of publish | +| publishedDate | :py:class:`datetime.datetime`, time of publish | +-------------------------+--------------------------------------------------------------------------------------------------------------+ | latitude | latitude of result (in decimal format) | +-------------------------+--------------------------------------------------------------------------------------------------------------+ -- cgit v1.2.3 From e9fff4fde6d7a8bec3fae087d2afe1fce2145f22 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Thu, 12 Dec 2019 19:20:56 +0100 Subject: doc: proofread of the all reST sources (no content change) Normalize reST sources with best practice and KISS in mind. to name a few points: - simplify reST tables - make use of ``literal`` markup for monospace rendering - fix code-blocks for better rendering in HTML - normalize section header markup - limit all lines to a maximum of 79 characters - add option -H to the sudo command used in code blocks - drop useless indentation of lists - ... [1] https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html Signed-off-by: Markus Heiser --- docs/dev/engine_overview.rst | 453 +++++++++++++++++++------------------------ 1 file changed, 202 insertions(+), 251 deletions(-) (limited to 'docs/dev/engine_overview.rst') diff --git a/docs/dev/engine_overview.rst b/docs/dev/engine_overview.rst index 937ca8880..92405dc64 100644 --- a/docs/dev/engine_overview.rst +++ b/docs/dev/engine_overview.rst @@ -1,21 +1,25 @@ + +.. _engines-dev: + +=============== Engine overview =============== +.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine -searx is a `metasearch-engine `__, -so it uses different search engines to provide better results. - -Because there is no general search API which could be used for every -search engine, an adapter has to be built between searx and the -external search engines. Adapters are stored under the folder -:origin:`searx/engines`. +searx is a metasearch-engine_, so it uses different search engines to provide +better results. +Because there is no general search API which could be used for every search +engine, an adapter has to be built between searx and the external search +engines. Adapters are stored under the folder :origin:`searx/engines`. .. contents:: :depth: 3 + :backlinks: entry general engine configuration ----------------------------- +============================ It is required to tell searx the type of results the engine provides. The arguments can be set in the engine file or in the settings file @@ -23,292 +27,239 @@ arguments can be set in the engine file or in the settings file the ones in the engine file. It does not matter if an option is stored in the engine file or in the -settings. However, the standard way is the following: +settings. However, the standard way is the following: engine file -~~~~~~~~~~~ - -+----------------------+-----------+-----------------------------------------+ -| argument | type | information | -+======================+===========+=========================================+ -| categories | list | pages, in which the engine is working | -+----------------------+-----------+-----------------------------------------+ -| paging | boolean | support multible pages | -+----------------------+-----------+-----------------------------------------+ -| language\_support | boolean | support language choosing | -+----------------------+-----------+-----------------------------------------+ -| time\_range\_support | boolean | support search time range | -+----------------------+-----------+-----------------------------------------+ -| offline | boolean | engine runs offline | -+----------------------+-----------+-----------------------------------------+ +----------- + +======================= =========== =========================================== +argument type information +======================= =========== =========================================== +categories list pages, in which the engine is working +paging boolean support multible pages +language_support boolean support language choosing +time_range_support boolean support search time range +offline boolean engine runs offline +======================= =========== =========================================== + settings.yml -~~~~~~~~~~~~ - -+------------+----------+-----------------------------------------------+ -| argument | type | information | -+============+==========+===============================================+ -| name | string | name of search-engine | -+------------+----------+-----------------------------------------------+ -| engine | string | name of searx-engine (filename without .py) | -+------------+----------+-----------------------------------------------+ -| shortcut | string | shortcut of search-engine | -+------------+----------+-----------------------------------------------+ -| timeout | string | specific timeout for search-engine | -+------------+----------+-----------------------------------------------+ +------------ + +======================= =========== =========================================== +argument type information +======================= =========== =========================================== +name string name of search-engine +engine string name of searx-engine + (filename without ``.py``) +shortcut string shortcut of search-engine +timeout string specific timeout for search-engine +======================= =========== =========================================== + overrides -~~~~~~~~~ - -A few of the options have default values in the engine, but are -often overwritten by the settings. If ``None`` is assigned to an option -in the engine file, it has to be redefined in the settings, -otherwise searx will not start with that engine. - -The naming of overrides is arbitrary. But the recommended -overrides are the following: - -+-----------------------+----------+----------------------------------------------------------------+ -| argument | type | information | -+=======================+==========+================================================================+ -| base\_url | string | base-url, can be overwritten to use same engine on other URL | -+-----------------------+----------+----------------------------------------------------------------+ -| number\_of\_results | int | maximum number of results per request | -+-----------------------+----------+----------------------------------------------------------------+ -| language | string | ISO code of language and country like en\_US | -+-----------------------+----------+----------------------------------------------------------------+ -| api\_key | string | api-key if required by engine | -+-----------------------+----------+----------------------------------------------------------------+ +--------- + +A few of the options have default values in the engine, but are often +overwritten by the settings. If ``None`` is assigned to an option in the engine +file, it has to be redefined in the settings, otherwise searx will not start +with that engine. + +The naming of overrides is arbitrary. But the recommended overrides are the +following: + +======================= =========== =========================================== +argument type information +======================= =========== =========================================== +base_url string base-url, can be overwritten to use same + engine on other URL +number_of_results int maximum number of results per request +language string ISO code of language and country like en_US +api_key string api-key if required by engine +======================= =========== =========================================== example code -~~~~~~~~~~~~ +------------ .. code:: python - # engine dependent config - categories = ['general'] - paging = True - language_support = True + # engine dependent config + categories = ['general'] + paging = True + language_support = True + making a request ----------------- +================ -To perform a search an URL have to be specified. In addition to -specifying an URL, arguments can be passed to the query. +To perform a search an URL have to be specified. In addition to specifying an +URL, arguments can be passed to the query. passed arguments -~~~~~~~~~~~~~~~~ +---------------- -These arguments can be used to construct the search query. Furthermore, +These arguments can be used to construct the search query. Furthermore, parameters with default value can be redefined for special purposes. -+----------------------+------------+------------------------------------------------------------------------+ -| argument | type | default-value, information | -+======================+============+========================================================================+ -| url | string | ``''`` | -+----------------------+------------+------------------------------------------------------------------------+ -| method | string | ``'GET'`` | -+----------------------+------------+------------------------------------------------------------------------+ -| headers | set | ``{}`` | -+----------------------+------------+------------------------------------------------------------------------+ -| data | set | ``{}`` | -+----------------------+------------+------------------------------------------------------------------------+ -| cookies | set | ``{}`` | -+----------------------+------------+------------------------------------------------------------------------+ -| verify | boolean | ``True`` | -+----------------------+------------+------------------------------------------------------------------------+ -| headers.User-Agent | string | a random User-Agent | -+----------------------+------------+------------------------------------------------------------------------+ -| category | string | current category, like ``'general'`` | -+----------------------+------------+------------------------------------------------------------------------+ -| started | datetime | current date-time | -+----------------------+------------+------------------------------------------------------------------------+ -| pageno | int | current pagenumber | -+----------------------+------------+------------------------------------------------------------------------+ -| language | string | specific language code like ``'en_US'``, or ``'all'`` if unspecified | -+----------------------+------------+------------------------------------------------------------------------+ +====================== ============ ======================================================================== +argument type default-value, information +====================== ============ ======================================================================== +url string ``''`` +method string ``'GET'`` +headers set ``{}`` +data set ``{}`` +cookies set ``{}`` +verify boolean ``True`` +headers.User-Agent string a random User-Agent +category string current category, like ``'general'`` +started datetime current date-time +pageno int current pagenumber +language string specific language code like ``'en_US'``, or ``'all'`` if unspecified +====================== ============ ======================================================================== parsed arguments -~~~~~~~~~~~~~~~~ - -The function ``def request(query, params):`` always returns the -``params`` variable. Inside searx, the following paramters can be -used to specify a search request: - -+------------+-----------+---------------------------------------------------------+ -| argument | type | information | -+============+===========+=========================================================+ -| url | string | requested url | -+------------+-----------+---------------------------------------------------------+ -| method | string | HTTP request method | -+------------+-----------+---------------------------------------------------------+ -| headers | set | HTTP header information | -+------------+-----------+---------------------------------------------------------+ -| data | set | HTTP data information (parsed if ``method != 'GET'``) | -+------------+-----------+---------------------------------------------------------+ -| cookies | set | HTTP cookies | -+------------+-----------+---------------------------------------------------------+ -| verify | boolean | Performing SSL-Validity check | -+------------+-----------+---------------------------------------------------------+ +---------------- + +The function ``def request(query, params):`` always returns the ``params`` +variable. Inside searx, the following paramters can be used to specify a search +request: + +============ =========== ========================================================= +argument type information +============ =========== ========================================================= +url string requested url +method string HTTP request method +headers set HTTP header information +data set HTTP data information (parsed if ``method != 'GET'``) +cookies set HTTP cookies +verify boolean Performing SSL-Validity check +============ =========== ========================================================= + example code -~~~~~~~~~~~~ +------------ .. code:: python - # search-url - base_url = 'https://example.com/' - search_string = 'search?{query}&page={page}' + # search-url + base_url = 'https://example.com/' + search_string = 'search?{query}&page={page}' - # do search-request - def request(query, params): - search_path = search_string.format( - query=urlencode({'q': query}), - page=params['pageno']) + # do search-request + def request(query, params): + search_path = search_string.format( + query=urlencode({'q': query}), + page=params['pageno']) - params['url'] = base_url + search_path + params['url'] = base_url + search_path + + return params - return params returned results ----------------- +================ -Searx is able to return results of different media-types. -Currently the following media-types are supported: +Searx is able to return results of different media-types. Currently the +following media-types are supported: -- default -- images -- videos -- torrent -- map +- default_ +- images_ +- videos_ +- torrent_ +- map_ -To set another media-type as default, the parameter -``template`` must be set to the desired type. +To set another media-type as default, the parameter ``template`` must be set to +the desired type. default -~~~~~~~ - -+--------------------+---------------------------------------------------------------------------------------------------------------+ -| result-parameter | information | -+====================+===============================================================================================================+ -| url | string, url of the result | -+--------------------+---------------------------------------------------------------------------------------------------------------+ -| title | string, title of the result | -+--------------------+---------------------------------------------------------------------------------------------------------------+ -| content | string, general result-text | -+--------------------+---------------------------------------------------------------------------------------------------------------+ -| publishedDate | :py:class:`datetime.datetime`, time of publish | -+--------------------+---------------------------------------------------------------------------------------------------------------+ +------- + +========================= ===================================================== +result-parameter information +========================= ===================================================== +url string, url of the result +title string, title of the result +content string, general result-text +publishedDate :py:class:`datetime.datetime`, time of publish +========================= ===================================================== images -~~~~~~ - -to use this template, the parameter - -+--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| result-parameter | information | -+====================+=======================================================================================================================================+ -| template | is set to ``images.html`` | -+--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| url | string, url to the result site | -+--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| title | string, title of the result *(partly implemented)* | -+--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| content | *(partly implemented)* | -+--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| publishedDate | :py:class:`datetime.datetime`, time of publish *(partly implemented)* | -+--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| img\_src | string, url to the result image | -+--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| thumbnail\_src | string, url to a small-preview image | -+--------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +------ + +To use this template, the parameter: + +========================= ===================================================== +result-parameter information +========================= ===================================================== +template is set to ``images.html`` +url string, url to the result site +title string, title of the result *(partly implemented)* +content *(partly implemented)* +publishedDate :py:class:`datetime.datetime`, + time of publish *(partly implemented)* +img\_src string, url to the result image +thumbnail\_src string, url to a small-preview image +========================= ===================================================== videos -~~~~~~ - -+--------------------+--------------------------------------------------------------------------------------------------------------+ -| result-parameter | information | -+====================+==============================================================================================================+ -| template | is set to ``videos.html`` | -+--------------------+--------------------------------------------------------------------------------------------------------------+ -| url | string, url of the result | -+--------------------+--------------------------------------------------------------------------------------------------------------+ -| title | string, title of the result | -+--------------------+--------------------------------------------------------------------------------------------------------------+ -| content | *(not implemented yet)* | -+--------------------+--------------------------------------------------------------------------------------------------------------+ -| publishedDate | :py:class:`datetime.datetime`, time of publish | -+--------------------+--------------------------------------------------------------------------------------------------------------+ -| thumbnail | string, url to a small-preview image | -+--------------------+--------------------------------------------------------------------------------------------------------------+ +------ + +========================= ===================================================== +result-parameter information +========================= ===================================================== +template is set to ``videos.html`` +url string, url of the result +title string, title of the result +content *(not implemented yet)* +publishedDate :py:class:`datetime.datetime`, time of publish +thumbnail string, url to a small-preview image +========================= ===================================================== torrent -~~~~~~~ - -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| result-parameter | information | -+==================+=======================================================================================================================================+ -| template | is set to ``torrent.html`` | -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| url | string, url of the result | -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| title | string, title of the result | -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| content | string, general result-text | -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| publishedDate | :py:class:`datetime.datetime`, time of publish *(not implemented yet)* | -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| seed | int, number of seeder | -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| leech | int, number of leecher | -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| filesize | int, size of file in bytes | -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| files | int, number of files | -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| magnetlink | string, `magnetlink `__ of the result | -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ -| torrentfile | string, torrentfile of the result | -+------------------+---------------------------------------------------------------------------------------------------------------------------------------+ +------- + +.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme + +========================= ===================================================== +result-parameter information +========================= ===================================================== +template is set to ``torrent.html`` +url string, url of the result +title string, title of the result +content string, general result-text +publishedDate :py:class:`datetime.datetime`, + time of publish *(not implemented yet)* +seed int, number of seeder +leech int, number of leecher +filesize int, size of file in bytes +files int, number of files +magnetlink string, magnetlink_ of the result +torrentfile string, torrentfile of the result +========================= ===================================================== map -~~~ - -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| result-parameter | information | -+=========================+==============================================================================================================+ -| url | string, url of the result | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| title | string, title of the result | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| content | string, general result-text | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| publishedDate | :py:class:`datetime.datetime`, time of publish | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| latitude | latitude of result (in decimal format) | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| longitude | longitude of result (in decimal format) | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| boundingbox | boundingbox of result (array of 4. values ``[lat-min, lat-max, lon-min, lon-max]``) | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| geojson | geojson of result (http://geojson.org) | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| osm.type | type of osm-object (if OSM-Result) | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| osm.id | id of osm-object (if OSM-Result) | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| address.name | name of object | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| address.road | street name of object | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| address.house\_number | house number of object | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| address.locality | city, place of object | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| address.postcode | postcode of object | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ -| address.country | country of object | -+-------------------------+--------------------------------------------------------------------------------------------------------------+ - +--- + +========================= ===================================================== +result-parameter information +========================= ===================================================== +url string, url of the result +title string, title of the result +content string, general result-text +publishedDate :py:class:`datetime.datetime`, time of publish +latitude latitude of result (in decimal format) +longitude longitude of result (in decimal format) +boundingbox boundingbox of result (array of 4. values + ``[lat-min, lat-max, lon-min, lon-max]``) +geojson geojson of result (http://geojson.org) +osm.type type of osm-object (if OSM-Result) +osm.id id of osm-object (if OSM-Result) +address.name name of object +address.road street name of object +address.house_number house number of object +address.locality city, place of object +address.postcode postcode of object +address.country country of object +========================= ===================================================== -- cgit v1.2.3