diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/admin/installation-searx.rst | 2 | ||||
| -rw-r--r-- | docs/admin/settings.rst | 64 | ||||
| -rw-r--r-- | docs/blog/index.rst | 1 | ||||
| -rw-r--r-- | docs/blog/search-indexer-engines.rst | 114 |
4 files changed, 167 insertions, 14 deletions
diff --git a/docs/admin/installation-searx.rst b/docs/admin/installation-searx.rst index 512a185a7..adea0166f 100644 --- a/docs/admin/installation-searx.rst +++ b/docs/admin/installation-searx.rst @@ -61,7 +61,7 @@ from the login (*~/.profile*): .. tip:: - Open a second terminal for the configuration tasks and left the ``(searx)$`` + Open a second terminal for the configuration tasks and leave the ``(searx)$`` terminal open for the tasks below. diff --git a/docs/admin/settings.rst b/docs/admin/settings.rst index 7cf055dbf..622218279 100644 --- a/docs/admin/settings.rst +++ b/docs/admin/settings.rst @@ -130,14 +130,12 @@ Global Settings request_timeout : 2.0 # default timeout in seconds, can be override by engine # max_request_timeout: 10.0 # the maximum timeout in seconds useragent_suffix : "" # informations like an email address to the administrator - pool_connections : 100 # Number of different hosts - pool_maxsize : 10 # Number of simultaneous requests by host + pool_connections : 100 # Maximum number of allowable connections, or None for no limits. The default is 100. + pool_maxsize : 10 # Number of allowable keep-alive connections, or None to always allow. The default is 10. + enable_http2: True # See https://www.python-httpx.org/http2/ # uncomment below section if you want to use a proxy # proxies: - # http: - # - http://proxy1:8080 - # - http://proxy2:8080 - # https: + # all://: # - http://proxy1:8080 # - http://proxy2:8080 # uncomment below section only if you have more than one network interface @@ -145,6 +143,7 @@ Global Settings # source_ips: # - 1.1.1.1 # - 1.1.1.2 + # - fe80::/126 ``request_timeout`` : @@ -157,20 +156,46 @@ Global Settings Suffix to the user-agent searx uses to send requests to others engines. If an engine wish to block you, a contact info here may be useful to avoid that. -.. _requests proxies: https://requests.readthedocs.io/en/latest/user/advanced/#proxies -.. _PySocks: https://pypi.org/project/PySocks/ +``keepalive_expiry``: + Number of seconds to keep a connection in the pool. By default 5.0 seconds. + +.. _httpx proxies: https://www.python-httpx.org/advanced/#http-proxying ``proxies`` : - Define one or more proxies you wish to use, see `requests proxies`_. + Define one or more proxies you wish to use, see `httpx proxies`_. If there are more than one proxy for one protocol (http, https), requests to the engines are distributed in a round-robin fashion. - - Proxy: `see <https://2.python-requests.org/en/latest/user/advanced/#proxies>`__. - - SOCKS proxies are also supported: `see <https://2.python-requests.org/en/latest/user/advanced/#socks>`__ - ``source_ips`` : If you use multiple network interfaces, define from which IP the requests must - be made. This parameter is ignored when ``proxies`` is set. + be made. Example: + + * ``0.0.0.0`` any local IPv4 address. + * ``::`` any local IPv6 address. + * ``192.168.0.1`` + * ``[ 192.168.0.1, 192.168.0.2 ]`` these two specific IP addresses + * ``fe80::60a2:1691:e5a2:ee1f`` + * ``fe80::60a2:1691:e5a2:ee1f/126`` all IP addresses in this network. + * ``[ 192.168.0.1, fe80::/126 ]`` + +``retries`` : + Number of retry in case of an HTTP error. + On each retry, searx uses an different proxy and source ip. + +``retry_on_http_error`` : + Retry request on some HTTP status code. + + Example: + + * ``true`` : on HTTP status code between 400 and 599. + * ``403`` : on HTTP status code 403. + * ``[403, 429]``: on HTTP status code 403 and 429. + +``enable_http2`` : + Enable by default. Set to ``False`` to disable HTTP/2. + +``max_redirects`` : + 30 by default. Maximum redirect before it is an error. ``locales:`` @@ -216,6 +241,13 @@ Engine settings api_key : 'apikey' disabled : True language : en_US + #enable_http: False + #enable_http2: False + #retries: 1 + #retry_on_http_error: True # or 403 or [404, 429] + #max_connections: 100 + #max_keepalive_connections: 10 + #keepalive_expiry: 5.0 #proxies: # http: # - http://proxy1:8080 @@ -270,6 +302,12 @@ Engine settings ``display_error_messages`` : default ``True`` When an engine returns an error, the message is displayed on the user interface. +``network``: optional + Use the network configuration from another engine. + In addition, there are two default networks: + * ``ipv4`` set ``local_addresses`` to ``0.0.0.0`` (use only IPv4 local addresses) + * ``ipv6`` set ``local_addresses`` to ``::`` (use only IPv6 local addresses) + .. note:: A few more options are possible, but they are pretty specific to some diff --git a/docs/blog/index.rst b/docs/blog/index.rst index 689739a58..8651cef69 100644 --- a/docs/blog/index.rst +++ b/docs/blog/index.rst @@ -12,3 +12,4 @@ Blog intro-offline private-engines command-line-engines + search-indexer-engines diff --git a/docs/blog/search-indexer-engines.rst b/docs/blog/search-indexer-engines.rst new file mode 100644 index 000000000..ca4dd3c88 --- /dev/null +++ b/docs/blog/search-indexer-engines.rst @@ -0,0 +1,114 @@ +=============================== +Query your local search engines +=============================== + +From now on, searx lets you to query your locally running search engines. The following +ones are supported now: + +* `Elasticsearch`_ +* `Meilisearch`_ +* `Solr`_ + +All of the engines above are added to ``settings.yml`` just commented out, as you have to +``base_url`` for all them. + +Please note that if you are not using HTTPS to access these engines, you have to enable +HTTP requests by setting ``enable_http`` to ``True``. + +Futhermore, if you do not want to expose these engines on a public instance, you can +still add them and limit the access by setting ``tokens`` as described in the `blog post about +private engines`_. + +Configuring searx for search engines +==================================== + +Each search engine is powerful, capable of full-text search. + +Elasticsearch +------------- + +Elasticsearch supports numerous ways to query the data it is storing. At the moment +the engine supports the most popular search methods: ``match``, ``simple_query_string``, ``term`` and ``terms``. + +If none of the methods fit your use case, you can select ``custom`` query type and provide the JSON payload +searx has to submit to Elasticsearch in ``custom_query_json``. + +The following is an example configuration for an Elasticsearch instance with authentication +configured to read from ``my-index`` index. + +.. code:: yaml + + - name : elasticsearch + shortcut : es + engine : elasticsearch + base_url : http://localhost:9200 + username : elastic + password : changeme + index : my-index + query_type : match + enable_http : True + + +Meilisearch +----------- + +This search engine is aimed at individuals and small companies. It is designed for +small-scale (less than 10 million documents) data collections. E.g. it is great for storing +web pages you have visited and searching in the contents later. + +The engine supports faceted search, so you can search in a subset of documents of the collection. +Futhermore, you can search in Meilisearch instances that require authentication by setting ``auth_token``. + +Here is a simple example to query a Meilisearch instance: + +.. code:: yaml + + - name : meilisearch + engine : meilisearch + shortcut: mes + base_url : http://localhost:7700 + index : my-index + enable_http: True + + +Solr +---- + +Solr is a popular search engine based on Lucene, just like Elasticsearch. +But instead of searching in indices, you can search in collections. + +This is an example configuration for searching in the collection ``my-collection`` and get +the results in ascending order. + +.. code:: yaml + + - name : solr + engine : solr + shortcut : slr + base_url : http://localhost:8983 + collection : my-collection + sort : asc + enable_http : True + + +Next steps +========== + +The next step is to add support for various SQL databases. + +Acknowledgement +=============== + +This development was sponsored by `Search and Discovery Fund`_ of `NLnet Foundation`_ . + +.. _blog post about private engines: private-engines.html#private-engines +.. _Elasticsearch: https://www.elastic.co/elasticsearch/ +.. _Meilisearch: https://www.meilisearch.com/ +.. _Solr: https://solr.apache.org/ +.. _Search and Discovery Fund: https://nlnet.nl/discovery +.. _NLnet Foundation: https://nlnet.nl/ + + +| Happy hacking. +| kvch // 2021.04.07 23:16 + |