diff options
Diffstat (limited to 'docs/admin')
| -rw-r--r-- | docs/admin/arch_public.dot | 8 | ||||
| -rw-r--r-- | docs/admin/architecture.rst | 14 | ||||
| -rw-r--r-- | docs/admin/buildhosts.rst | 74 | ||||
| -rw-r--r-- | docs/admin/engines.rst | 4 | ||||
| -rw-r--r-- | docs/admin/filtron.rst | 247 | ||||
| -rw-r--r-- | docs/admin/index.rst | 9 | ||||
| -rw-r--r-- | docs/admin/installation-apache.rst | 514 | ||||
| -rw-r--r-- | docs/admin/installation-docker.rst | 60 | ||||
| -rw-r--r-- | docs/admin/installation-nginx.rst | 381 | ||||
| -rw-r--r-- | docs/admin/installation-searx.rst | 92 | ||||
| -rw-r--r-- | docs/admin/installation-uwsgi.rst | 150 | ||||
| -rw-r--r-- | docs/admin/installation.rst | 357 | ||||
| -rw-r--r-- | docs/admin/morty.rst | 7 | ||||
| -rw-r--r-- | docs/admin/settings.rst | 27 | ||||
| -rw-r--r-- | docs/admin/update-searx.rst | 23 |
15 files changed, 1515 insertions, 452 deletions
diff --git a/docs/admin/arch_public.dot b/docs/admin/arch_public.dot index a46c96de3..5018225c3 100644 --- a/docs/admin/arch_public.dot +++ b/docs/admin/arch_public.dot @@ -4,11 +4,11 @@ digraph G { edge [fontname="Sans"]; browser [label="Browser", shape=Mdiamond]; - rp [label="Reverse Proxy", href="url to configure reverse proxy"]; - filtron [label="Filtron", href="https://github.com/asciimoo/filtron"]; - morty [label="Morty", href="https://github.com/asciimoo/morty"]; + rp [label="Reverse Proxy", href="https://asciimoo.github.io/searx/utils/filtron.sh.html#public-reverse-proxy"]; + filtron [label="Filtron", href="https://asciimoo.github.io/searx/utils/filtron.sh.html"]; + morty [label="Morty", href="https://asciimoo.github.io/searx/utils/morty.sh.html"]; static [label="Static files", href="url to configure static files"]; - uwsgi [label="uwsgi", href="url to configure uwsgi"] + uwsgi [label="uwsgi", href="https://asciimoo.github.io/searx/utils/searx.sh.html"] searx1 [label="Searx #1"]; searx2 [label="Searx #2"]; searx3 [label="Searx #3"]; diff --git a/docs/admin/architecture.rst b/docs/admin/architecture.rst index 7064a294b..464e765eb 100644 --- a/docs/admin/architecture.rst +++ b/docs/admin/architecture.rst @@ -4,17 +4,21 @@ Architecture ============ -.. sidebar:: Needs work! +.. sidebar:: Further reading - This article needs some work / Searx is a collaborative effort. If you have - any contribution, feel welcome to send us your :pull:`PR <../pulls>`, see - :ref:`how to contribute`. + - Reverse Proxy: :ref:`Apache <apache searx site>` & :ref:`nginx <nginx searx + site>` + - Filtron: :ref:`searx filtron` + - Morty: :ref:`searx morty` + - uWSGI: :ref:`searx uwsgi` + - Searx: :ref:`installation basic` Herein you will find some hints and suggestions about typical architectures of searx infrastructures. We start with a contribution from :pull:`@dalf <1776#issuecomment-567917320>`. -It shows a *reference* setup for public searx instances. +It shows a *reference* setup for public searx instances which can build up and +maintained by the scripts from our :ref:`toolboxing`. .. _arch public: diff --git a/docs/admin/buildhosts.rst b/docs/admin/buildhosts.rst index 5260da033..a727d25b9 100644 --- a/docs/admin/buildhosts.rst +++ b/docs/admin/buildhosts.rst @@ -9,8 +9,27 @@ Buildhosts If you have any contribution send us your :pull:`PR <../pulls>`, see :ref:`how to contribute`. +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + To get best results from build, its recommend to install additional packages -on build hosts. +on build hosts (see :ref:`searx.sh`).:: + + sudo -H ./utils/searx.sh install buildhost + +This will install packages needed by searx: + +.. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START distro-packages + :end-before: END distro-packages + +and packages needed to build docuemtation and run tests: + +.. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START build-packages + :end-before: END build-packages .. _docs build: @@ -35,8 +54,17 @@ processing additional packages are needed. The XeTeX_ needed not only for PDF creation, its also needed for :ref:`math` when HTML output is build. To be able to do :ref:`sphinx:math-support` without CDNs, the math are rendered -as images (``sphinx.ext.imgmath`` extension). If your docs build (``make -docs``) shows warnings like this:: +as images (``sphinx.ext.imgmath`` extension). + +Here is the extract from the :origin:`docs/conf.py` file, setting math renderer +to ``imgmath``: + +.. literalinclude:: ../conf.py + :language: python + :start-after: # sphinx.ext.imgmath setup + :end-before: # sphinx.ext.imgmath setup END + +If your docs build (``make docs``) shows warnings like this:: WARNING: dot(1) not found, for better output quality install \ graphviz from http://www.graphviz.org @@ -47,8 +75,6 @@ docs``) shows warnings like this:: you need to install additional packages on your build host, to get better HTML output. -.. _system requirements: - .. tabs:: .. group-tab:: Ubuntu / debian @@ -92,12 +118,38 @@ For PDF output you also need: $ sudo dnf install \ texlive-collection-fontsrecommended texlive-collection-latex \ - dejavu-sans-fonts dejavu-serif-fonts dejavu-sans-mono-fonts + dejavu-sans-fonts dejavu-serif-fonts dejavu-sans-mono-fonts \ + ImageMagick -.. _system requirements END: +.. _sh lint: -.. literalinclude:: ../conf.py - :language: python - :start-after: # sphinx.ext.imgmath setup - :end-before: # sphinx.ext.imgmath setup END +Lint shell scripts +================== + +.. _ShellCheck: https://github.com/koalaman/shellcheck + +To lint shell scripts, we use ShellCheck_ - A shell script static analysis tool. + +.. SNIP sh lint requirements + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code-block:: sh + + $ sudo apt install shellcheck + + .. group-tab:: Arch Linux + + .. code-block:: sh + + $ sudo pacman -S shellcheck + + .. group-tab:: Fedora / RHEL + + .. code-block:: sh + + $ sudo dnf install ShellCheck +.. SNAP sh lint requirements diff --git a/docs/admin/engines.rst b/docs/admin/engines.rst index 4b78c8c2b..4d1872dfc 100644 --- a/docs/admin/engines.rst +++ b/docs/admin/engines.rst @@ -26,6 +26,8 @@ Safe search **SS** Weigth **W** ------------- ----------- --------------------------------- Disabled **D** +------------- ----------- --------------------------------- +Show errors **DE** ============= =========== ================================= Configuration defaults (at built time): @@ -51,6 +53,7 @@ Configuration defaults (at built time): - O - W - D + - DE {% for name, mod in engines.items() %} @@ -67,5 +70,6 @@ Configuration defaults (at built time): - {{(mod.offline and "y") or ""}} - {{mod.weight or 1 }} - {{(mod.disabled and "y") or ""}} + - {{(mod.display_error_messages and "y") or ""}} {% endfor %} diff --git a/docs/admin/filtron.rst b/docs/admin/filtron.rst index 07dcb9bc5..503a4d51d 100644 --- a/docs/admin/filtron.rst +++ b/docs/admin/filtron.rst @@ -1,18 +1,51 @@ + +.. _searx filtron: + ========================== How to protect an instance ========================== -Searx depens on external search services. To avoid the abuse of these services +.. sidebar:: further reading + + - :ref:`filtron.sh` + - :ref:`nginx searx site` + + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +.. _filtron: https://github.com/asciimoo/filtron + +Searx depends on external search services. To avoid the abuse of these services it is advised to limit the number of requests processed by searx. -An application firewall, ``filtron`` solves exactly this problem. Information -on how to install it can be found at the `project page of filtron -<https://github.com/asciimoo/filtron>`__. +An application firewall, filtron_ solves exactly this problem. Filtron is just +a middleware between your web server (nginx, apache, ...) and searx, we describe +such infratructures in chapter: :ref:`architecture`. +filtron & go +============ + +.. _Go: https://golang.org/ +.. _filtron README: https://github.com/asciimoo/filtron/blob/master/README.md + +Filtron needs Go_ installed. If Go_ is preinstalled, filtron_ is simply +installed by ``go get`` package management (see `filtron README`_). If you use +filtron as middleware, a more isolated setup is recommended. To simplify such +an installation and the maintenance of, use our script :ref:`filtron.sh`. + +.. _Sample configuration of filtron: + Sample configuration of filtron =============================== +.. sidebar:: Tooling box + + - :origin:`/etc/filtron/rules.json <utils/templates/etc/filtron/rules.json>` + An example configuration can be find below. This configuration limits the access of: @@ -24,105 +57,104 @@ of: .. code:: json - [{ - "name":"search request", - "filters":[ - "Param:q", - "Path=^(/|/search)$" - ], - "interval":"<time-interval-in-sec (int)>", - "limit":"<max-request-number-in-interval (int)>", - "subrules":[ - { - "name":"roboagent limit", - "interval":"<time-interval-in-sec (int)>", - "limit":"<max-request-number-in-interval (int)>", - "filters":[ - "Header:User-Agent=(curl|cURL|Wget|python-requests|Scrapy|FeedFetcher|Go-http-client)" - ], - "actions":[ - { - "name":"block", - "params":{ - "message":"Rate limit exceeded" - } - } - ] - }, - { - "name":"botlimit", - "limit":0, - "stop":true, - "filters":[ - "Header:User-Agent=(Googlebot|bingbot|Baiduspider|yacybot|YandexMobileBot|YandexBot|Yahoo! Slurp|MJ12bot|AhrefsBot|archive.org_bot|msnbot|MJ12bot|SeznamBot|linkdexbot|Netvibes|SMTBot|zgrab|James BOT)" - ], - "actions":[ - { - "name":"block", - "params":{ - "message":"Rate limit exceeded" - } - } - ] - }, - { - "name":"IP limit", - "interval":"<time-interval-in-sec (int)>", - "limit":"<max-request-number-in-interval (int)>", - "stop":true, - "aggregations":[ - "Header:X-Forwarded-For" - ], - "actions":[ - { - "name":"block", - "params":{ - "message":"Rate limit exceeded" - } - } - ] - }, - { - "name":"rss/json limit", - "interval":"<time-interval-in-sec (int)>", - "limit":"<max-request-number-in-interval (int)>", - "stop":true, - "filters":[ - "Param:format=(csv|json|rss)" - ], - "actions":[ - { - "name":"block", - "params":{ - "message":"Rate limit exceeded" - } - } - ] - }, - { - "name":"useragent limit", - "interval":"<time-interval-in-sec (int)>", - "limit":"<max-request-number-in-interval (int)>", - "aggregations":[ - "Header:User-Agent" + [ + { + "name": "search request", + "filters": [ + "Param:q", + "Path=^(/|/search)$" ], - "actions":[ - { - "name":"block", - "params":{ - "message":"Rate limit exceeded" - } - } + "interval": "<time-interval-in-sec (int)>", + "limit": "<max-request-number-in-interval (int)>", + "subrules": [ + { + "name": "missing Accept-Language", + "filters": ["!Header:Accept-Language"], + "limit": "<max-request-number-in-interval (int)>", + "stop": true, + "actions": [ + {"name":"log"}, + {"name": "block", + "params": {"message": "Rate limit exceeded"}} + ] + }, + { + "name": "suspiciously Connection=close header", + "filters": ["Header:Connection=close"], + "limit": "<max-request-number-in-interval (int)>", + "stop": true, + "actions": [ + {"name":"log"}, + {"name": "block", + "params": {"message": "Rate limit exceeded"}} + ] + }, + { + "name": "IP limit", + "interval": "<time-interval-in-sec (int)>", + "limit": "<max-request-number-in-interval (int)>", + "stop": true, + "aggregations": [ + "Header:X-Forwarded-For" + ], + "actions": [ + { "name": "log"}, + { "name": "block", + "params": { + "message": "Rate limit exceeded" + } + } + ] + }, + { + "name": "rss/json limit", + "filters": [ + "Param:format=(csv|json|rss)" + ], + "interval": "<time-interval-in-sec (int)>", + "limit": "<max-request-number-in-interval (int)>", + "stop": true, + "actions": [ + { "name": "log"}, + { "name": "block", + "params": { + "message": "Rate limit exceeded" + } + } + ] + }, + { + "name": "useragent limit", + "interval": "<time-interval-in-sec (int)>", + "limit": "<max-request-number-in-interval (int)>", + "aggregations": [ + "Header:User-Agent" + ], + "actions": [ + { "name": "log"}, + { "name": "block", + "params": { + "message": "Rate limit exceeded" + } + } + ] + } ] - } - ] - }] + } + ] +.. _filtron route request: Route request through filtron ============================= +.. sidebar:: further reading + + - :ref:`filtron.sh overview` + - :ref:`installation nginx` + - :ref:`installation apache` + Filtron can be started using the following command: .. code:: sh @@ -136,13 +168,24 @@ Use it along with ``nginx`` with the following example configuration. .. code:: nginx - location / { - proxy_set_header Host $http_host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Scheme $scheme; - proxy_pass http://127.0.0.1:4004/; + # https://example.org/searx + + location /searx { + proxy_pass http://127.0.0.1:4004/; + + proxy_set_header Host $http_host; + proxy_set_header Connection $http_connection; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_set_header X-Script-Name /searx; } + location /searx/static { + /usr/local/searx/searx-src/searx/static; + } + + Requests are coming from port 4004 going through filtron and then forwarded to -port 8888 where a searx is being run. +port 8888 where a searx is being run. For a complete setup see: :ref:`nginx +searx site`. diff --git a/docs/admin/index.rst b/docs/admin/index.rst index b3c7f5119..c708c4ffa 100644 --- a/docs/admin/index.rst +++ b/docs/admin/index.rst @@ -3,9 +3,16 @@ Administrator documentation =========================== .. toctree:: - :maxdepth: 1 + :maxdepth: 2 + :caption: Contents installation + installation-searx + installation-uwsgi + installation-nginx + installation-apache + installation-docker + update-searx settings api architecture diff --git a/docs/admin/installation-apache.rst b/docs/admin/installation-apache.rst new file mode 100644 index 000000000..217e57718 --- /dev/null +++ b/docs/admin/installation-apache.rst @@ -0,0 +1,514 @@ +.. _installation apache: + +=================== +Install with apache +=================== + +.. _Apache: https://httpd.apache.org/ +.. _Apache Debian: + https://cwiki.apache.org/confluence/display/HTTPD/DistrosDefaultLayout#DistrosDefaultLayout-Debian,Ubuntu(Apachehttpd2.x): +.. _README.Debian: + https://salsa.debian.org/apache-team/apache2/raw/master/debian/apache2.README.Debian +.. _Apache Arch Linux: + https://wiki.archlinux.org/index.php/Apache_HTTP_Server +.. _Apache Fedora: + https://docs.fedoraproject.org/en-US/quick-docs/getting-started-with-apache-http-server/index.html +.. _Apache directives: + https://httpd.apache.org/docs/trunk/mod/directives.html +.. _Getting Started: + https://httpd.apache.org/docs/current/en/getting-started.html +.. _Terms Used to Describe Directives: + https://httpd.apache.org/docs/current/en/mod/directive-dict.html +.. _Configuration Files: + https://httpd.apache.org/docs/current/en/configuring.html +.. _ProxyPreserveHost: https://httpd.apache.org/docs/trunk/mod/mod_proxy.html#proxypreservehost +.. _LoadModule: + https://httpd.apache.org/docs/2.4/mod/mod_so.html#loadmodule +.. _DocumentRoot: + https://httpd.apache.org/docs/trunk/mod/core.html#documentroot +.. _Location: + https://httpd.apache.org/docs/trunk/mod/core.html#location +.. _uWSGI Apache support: + https://uwsgi-docs.readthedocs.io/en/latest/Apache.html +.. _mod_proxy_uwsgi: + https://uwsgi-docs.readthedocs.io/en/latest/Apache.html#mod-proxy-uwsgi + +.. sidebar:: further read + + - `Apache Arch Linux`_ + - `Apache Debian`_ and `README.Debian`_ + - `Apache Fedora`_ + - `Apache directives`_ + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +---- + +**Install** :ref:`apache searx site` using :ref:`filtron.sh <filtron.sh overview>` + +.. code:: bash + + $ sudo -H ./utils/filtron.sh apache install + +**Install** :ref:`apache searx site` using :ref:`morty.sh <morty.sh overview>` + +.. code:: bash + + $ sudo -H ./utils/morty.sh apache install + +---- + +The apache HTTP server +====================== + +If Apache_ is not installed, install it now. If apache_ is new to you, the +`Getting Started`_, `Configuration Files`_ and `Terms Used to Describe +Directives`_ documentation gives first orientation. There is also a list of +`Apache directives`_ *to keep in the pocket*. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H apt-get install apache2 + + .. group-tab:: Arch Linux + + .. code:: sh + + sudo -H pacman -S apache + sudo -H systemctl enable httpd + sudo -H systemctl start http + + .. group-tab:: Fedora / RHEL + + .. code:: sh + + sudo -H dnf install httpd + sudo -H systemctl enable httpd + sudo -H systemctl start httpd + +Now at http://localhost you should see any kind of *Welcome* or *Test* page. +How this default intro site is configured, depends on the linux distribution +(compare `Apache directives`_). + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + less /etc/apache2/sites-enabled/000-default.conf + + In this file, there is a line setting the `DocumentRoot`_ directive: + + .. code:: apache + + DocumentRoot /var/www/html + + And the *welcome* page is the HTML file at ``/var/www/html/index.html``. + + .. group-tab:: Arch Linux + + .. code:: sh + + less /etc/httpd/conf/httpd.conf + + In this file, there is a line setting the `DocumentRoot`_ directive: + + .. code:: apache + + DocumentRoot "/srv/http" + <Directory "/srv/http"> + Options Indexes FollowSymLinks + AllowOverride None + Require all granted + </Directory> + + The *welcome* page of Arch Linux is a page showing directory located at + ``DocumentRoot``. This is *directory* page is generated by the Module + `mod_autoindex <https://httpd.apache.org/docs/2.4/mod/mod_autoindex.html>`_: + + .. code:: apache + + LoadModule autoindex_module modules/mod_autoindex.so + ... + Include conf/extra/httpd-autoindex.conf + + .. group-tab:: Fedora / RHEL + + .. code:: sh + + less /etc/httpd/conf/httpd.conf + + In this file, there is a line setting the ``DocumentRoot`` directive: + + .. code:: apache + + DocumentRoot "/var/www/html" + ... + <Directory "/var/www"> + AllowOverride None + # Allow open access: + Require all granted + </Directory> + + On fresh installations, the ``/var/www`` is empty and the *default + welcome page* is shown, the configuration is located at:: + + less /etc/httpd/conf.d/welcome.conf + +.. _apache searx site: + +Apache Reverse Proxy +==================== + +.. sidebar:: public to the internet? + + If your searx instance is public, stop here and first install :ref:`filtron + reverse proxy <filtron.sh>` and :ref:`result proxy morty <morty.sh>`, see + :ref:`installation scripts`. If already done, follow setup: *searx via + filtron plus morty*. + +To setup a Apache revers proxy you have to enable the *headers* and *proxy* +modules and create a `Location`_ configuration for the searx site. In most +distributions you have to un-comment the lines in the main configuration file, +except in :ref:`The Debian Layout`. + +To pass the HTTP HOST header +With ProxyPreserveHost_ the incoming Host HTTP request header is passed to the +proxied host. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + In the Apache setup, enable headers and proxy modules: + + .. code:: sh + + sudo -H a2enmod headers + sudo -H a2enmod proxy + sudo -H a2enmod proxy_http + + In :ref:`The Debian Layout` you create a ``searx.conf`` with the + ``<Location /searx >`` directive and save this file in the *sites + available* folder at ``/etc/apache2/sites-available``. To enable the + ``searx.conf`` use :man:`a2ensite`: + + .. code:: sh + + sudo -H a2ensite searx.conf + + .. group-tab:: Arch Linux + + In the ``/etc/httpd/conf/httpd.conf`` file, activate headers and proxy + modules (LoadModule_): + + .. code:: apache + + FIXME needs test + + LoadModule headers_module modules/mod_headers.so + LoadModule proxy_module modules/mod_proxy.so + LoadModule proxy_http_module modules/mod_proxy_http.so + + .. group-tab:: Fedora / RHEL + + In the ``/etc/httpd/conf/httpd.conf`` file, activate headers and proxy + modules (LoadModule_): + + .. code:: apache + + FIXME needs test + + LoadModule headers_module modules/mod_headers.so + LoadModule proxy_module modules/mod_proxy.so + LoadModule proxy_http_module modules/mod_proxy_http.so + +.. tabs:: + + .. group-tab:: searx via filtron plus morty + + Use this setup, if your instance is public to the internet, compare + figure: :ref:`architecture <arch public>` and :ref:`installation scripts`. + + 1. Configure a reverse proxy for :ref:`filtron <filtron.sh>`, listening on + *localhost 4004* (:ref:`filtron route request`): + + .. code:: apache + + <Location /searx > + + # SetEnvIf Request_URI "/searx" dontlog + # CustomLog /dev/null combined env=dontlog + + Require all granted + + Order deny,allow + Deny from all + #Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + ProxyPreserveHost On + ProxyPass http://127.0.0.1:4004 + RequestHeader set X-Script-Name /searx + + </Location> + + 2. Configure reverse proxy for :ref:`morty <searx morty>`, listening on + *localhost 3000* + + .. code:: apache + + ProxyPreserveHost On + + <Location /morty > + + # SetEnvIf Request_URI "/morty" dontlog + # CustomLog /dev/null combined env=dontlog + + Require all granted + + Order deny,allow + Deny from all + #Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + ProxyPass http://127.0.0.1:3000 + RequestHeader set X-Script-Name /morty + + </Location> + + Note that reverse proxy advised to be used in case of single-user or + low-traffic instances. For a fully result proxification add :ref:`morty's + <searx morty>` **public URL** to your :origin:`searx/settings.yml`: + + .. code:: yaml + + result_proxy: + # replace example.org with your server's public name + url : https://example.org/morty + + server: + image_proxy : True + +uWSGI support +============= + +Be warned, with this setup, your instance isn't :ref:`protected <searx +filtron>`, nevertheless it is good enough for intranet usage. In modern Linux +distributions, the `mod_proxy_uwsgi`_ is compiled into the *normal* apache +package and you need to install only the :ref:`uWSGI <searx uwsgi>` package: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H apt-get install uwsgi + + # Ubuntu =< 18.04 + sudo -H apt-get install libapache2-mod-proxy-uwsgi + + .. group-tab:: Arch Linux + + .. code:: sh + + sudo -H pacman -S uwsgi + + .. group-tab:: Fedora / RHEL + + .. code:: sh + + sudo -H dnf install uwsgi + +The next example shows a configuration using the `uWSGI Apache support`_ via +unix sockets and `mod_proxy_uwsgi`_. + +For socket communication, you have to activate ``socket = +/run/uwsgi/app/searx/socket`` and comment out the ``http = 127.0.0.1:8888`` +configuration in your :ref:`uwsgi ini file <uwsgi configuration>`. If not +already exists, create a folder for the unix sockets, which can be used by the +searx account (see :ref:`create searx user`): + +.. code:: bash + + sudo -H mkdir -p /run/uwsgi/app/searx/ + sudo -H chown -R searx:searx /run/uwsgi/app/searx/ + +If the server is public; to limit access to your intranet replace ``Allow from +all`` directive and replace ``192.168.0.0/16`` with your subnet IP/class. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: apache + + LoadModule headers_module /usr/lib/apache2/mod_headers.so + LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so + LoadModule proxy_uwsgi_module /usr/lib/apache2/modules/mod_proxy_uwsgi.so + + # SetEnvIf Request_URI /searx dontlog + # CustomLog /dev/null combined env=dontlog + + <Location /searx> + + Require all granted + Order deny,allow + Deny from all + # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + ProxyPreserveHost On + ProxyPass unix:/run/uwsgi/app/searx/socket|uwsgi://uwsgi-uds-searx/ + + </Location> + + .. group-tab:: Arch Linux + + .. code:: apache + + FIXME needs test + + LoadModule proxy_module modules/mod_proxy.so + LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so + + # SetEnvIf Request_URI /searx dontlog + # CustomLog /dev/null combined env=dontlog + + <Location /searx> + + Require all granted + Order deny,allow + Deny from all + # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + ProxyPreserveHost On + ProxyPass unix:/run/uwsgi/app/searx/socket|uwsgi://uwsgi-uds-searx/ + + </Location> + + .. group-tab:: Fedora / RHEL + + .. code:: apache + + FIXME needs test + + LoadModule proxy_module modules/mod_proxy.so + LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so + <IfModule proxy_uwsgi_module> + + # SetEnvIf Request_URI /searx dontlog + # CustomLog /dev/null combined env=dontlog + + <Location /searx> + + Require all granted + Order deny,allow + Deny from all + # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + ProxyPreserveHost On + ProxyPass unix:/run/uwsgi/app/searx/socket|uwsgi://uwsgi-uds-searx/ + + </Location> + + </IfModule> + + .. group-tab:: old mod_wsgi + + We show this only for historical reasons, DON'T USE `mod_uwsgi + <https://uwsgi-docs.readthedocs.io/en/latest/Apache.html#mod-uwsgi>`_. + ANYMORE! + + .. code:: apache + + <IfModule mod_uwsgi.c> + + # SetEnvIf Request_URI "/searx" dontlog + # CustomLog /dev/null combined env=dontlog + + <Location /searx > + + Require all granted + + Options FollowSymLinks Indexes + SetHandler uwsgi-handler + uWSGISocket /run/uwsgi/app/searx/socket + + Order deny,allow + Deny from all + # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + </Location> + + </IfModule> + +.. _restart apache: + +Restart service +=============== + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H systemctl restart apache2 + sudo -H service uwsgi restart searx + + .. group-tab:: Arch Linux + + .. code:: sh + + sudo -H systemctl restart httpd + sudo -H systemctl restart uwsgi@searx + + .. group-tab:: Fedora / RHEL + + .. code:: sh + + sudo -H systemctl restart httpd + sudo -H touch /etc/uwsgi.d/searx.ini + + +disable logs +============ + +For better privacy you can disable Apache logs. In the examples above activate +one of the lines and `restart apache`_:: + + + # SetEnvIf Request_URI "/searx" dontlog + # CustomLog /dev/null combined env=dontlog + +The ``CustomLog`` directive disable logs for the whole (virtual) server, use it +when the URL of the service does not have a path component (``/searx``) / is +located at root (``/``). + +.. _The Debian Layout: + +The Debian Layout +================= + +Be aware that the Debian layout is quite different from the standard Apache +configuration. For details look at the README.Debian_ +(``/usr/share/doc/apache2/README.Debian.gz``). Some commands you should know on +Debian: + +* :man:`apache2ctl`: Apache HTTP server control interface +* :man:`a2enmod`, :man:`a2dismod`: switch on/off modules +* :man:`a2enconf`, :man:`a2disconf`: switch on/off configurations +* :man:`a2ensite`, :man:`a2dissite`: switch on/off sites diff --git a/docs/admin/installation-docker.rst b/docs/admin/installation-docker.rst new file mode 100644 index 000000000..b30cdcbd0 --- /dev/null +++ b/docs/admin/installation-docker.rst @@ -0,0 +1,60 @@ +.. _installation docker: + +=================== +Docker installation +=================== + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +---- + +Docker image searx/searx +======================== + + +The docker image is `searx/searx <https://hub.docker.com/r/searx/searx>`_ (based on `github.com/searx/searx <https://github.com/searx/searx>`_). + +Make sure you have `installed Docker <https://docs.docker.com/get-docker/>`_. For instance, you can deploy a local instance: + +.. code:: sh + + export PORT=80 + docker pull searx/searx + docker run --rm -d -v ${PWD}/searx:/etc/searx -p $PORT:8080 -e BASE_URL=http://localhost:$PORT/ searx/searx + +Go to ``http://localhost:$PORT``. + +Inside ``${PWD}/searx``, you will find ``settings.yml`` and ``uwsgi.ini``. +You can modify these files according to your needs and restart the Docker image. + + +Command line +------------ + + +.. code:: sh + + docker run --rm -it searx/searx -h + +.. program-output:: ../dockerfiles/docker-entrypoint.sh help + + +Build the image +--------------- + +It's also possible to build searx from the embedded Dockerfile. + +.. code:: sh + + git clone https://github.com/asciimoo/searx.git + cd searx + make docker + + +Public instance +=============== + +If you intend to create a public instance using Docker, see https://github.com/searx/searx-docker diff --git a/docs/admin/installation-nginx.rst b/docs/admin/installation-nginx.rst new file mode 100644 index 000000000..5e32d9684 --- /dev/null +++ b/docs/admin/installation-nginx.rst @@ -0,0 +1,381 @@ +.. _installation nginx: + +================== +Install with nginx +================== + +.. _nginx: + https://docs.nginx.com/nginx/admin-guide/ +.. _nginx server configuration: + https://docs.nginx.com/nginx/admin-guide/web-server/web-server/#setting-up-virtual-servers +.. _nginx beginners guide: + http://nginx.org/en/docs/beginners_guide.html +.. _Getting Started wiki: + https://www.nginx.com/resources/wiki/start/ +.. _uWSGI support from nginx: + https://uwsgi-docs.readthedocs.io/en/latest/Nginx.html +.. _uwsgi_params: + https://uwsgi-docs.readthedocs.io/en/latest/Nginx.html#configuring-nginx +.. _SCRIPT_NAME: + https://werkzeug.palletsprojects.com/en/1.0.x/wsgi/#werkzeug.wsgi.get_script_name + +.. sidebar:: further reading + + - nginx_ + - `nginx beginners guide`_ + - `nginx server configuration`_ + - `Getting Started wiki`_ + - `uWSGI support from nginx`_ + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +---- + +**Install** :ref:`nginx searx site` using :ref:`filtron.sh <filtron.sh overview>` + +.. code:: bash + + $ sudo -H ./utils/filtron.sh nginx install + +**Install** :ref:`nginx searx site` using :ref:`morty.sh <morty.sh overview>` + +.. code:: bash + + $ sudo -H ./utils/morty.sh nginx install + +---- + + +The nginx HTTP server +===================== + +If nginx_ is not installed (uwsgi will not work with the package nginx-light), +install it now. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H apt-get install nginx + + .. group-tab:: Arch Linux + + .. code-block:: sh + + sudo -H pacman -S nginx-mainline + sudo -H systemctl enable nginx + sudo -H systemctl start nginx + + .. group-tab:: Fedora / RHEL + + .. code-block:: sh + + sudo -H dnf install nginx + sudo -H systemctl enable nginx + sudo -H systemctl start nginx + +Now at http://localhost you should see a *Welcome to nginx!* page, on Fedora you +see a *Fedora Webserver - Test Page*. The test page comes from the default +`nginx server configuration`_. How this default intro site is configured, +depends on the linux distribution: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + less /etc/nginx/nginx.conf + + there is a line including site configurations from: + + .. code:: nginx + + include /etc/nginx/sites-enabled/*; + + .. group-tab:: Arch Linux + + .. code-block:: sh + + less /etc/nginx/nginx.conf + + in there is a configuration section named ``server``: + + .. code-block:: nginx + + server { + listen 80; + server_name localhost; + # ... + } + + .. group-tab:: Fedora / RHEL + + .. code-block:: sh + + less /etc/nginx/nginx.conf + + there is a line including site configurations from: + + .. code:: nginx + + include /etc/nginx/conf.d/*.conf; + +.. _nginx searx site: + +A nginx searx site +================== + +.. sidebar:: public to the internet? + + If your searx instance is public, stop here and first install :ref:`filtron + reverse proxy <filtron.sh>` and :ref:`result proxy morty <morty.sh>`, see + :ref:`installation scripts`. If already done, follow setup: *searx via + filtron plus morty*. + +Now you have to create a configuration for the searx site. If nginx_ is new to +you, the `nginx beginners guide`_ is a good starting point and the `Getting +Started wiki`_ is always a good resource *to keep in the pocket*. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + Create configuration at ``/etc/nginx/sites-available/searx`` and place a + symlink to sites-enabled: + + .. code:: sh + + sudo -H ln -s /etc/nginx/sites-available/searx /etc/nginx/sites-enabled/searx + + .. group-tab:: Arch Linux + + In the ``/etc/nginx/nginx.conf`` file, replace the configuration section + named ``server``. + + .. group-tab:: Fedora / RHEL + + Create configuration at ``/etc/nginx/conf.d/searx`` and place a + symlink to sites-enabled: + +.. tabs:: + + .. group-tab:: searx via filtron plus morty + + Use this setup, if your instance is public to the internet, compare + figure: :ref:`architecture <arch public>` and :ref:`installation scripts`. + + 1. Configure a reverse proxy for :ref:`filtron <filtron.sh>`, listening on + *localhost 4004* (:ref:`filtron route request`): + + .. code:: nginx + + # https://example.org/searx + + location /searx { + proxy_pass http://127.0.0.1:4004/; + + proxy_set_header Host $http_host; + proxy_set_header Connection $http_connection; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_set_header X-Script-Name /searx; + } + + location /searx/static { + /usr/local/searx/searx-src/searx/static; + } + + + 2. Configure reverse proxy for :ref:`morty <searx morty>`, listening on + *localhost 3000*: + + .. code:: nginx + + # https://example.org/morty + + location /morty { + proxy_pass http://127.0.0.1:3000/; + + proxy_set_header Host $http_host; + proxy_set_header Connection $http_connection; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + } + + Note that reverse proxy advised to be used in case of single-user or + low-traffic instances. For a fully result proxification add :ref:`morty's + <searx morty>` **public URL** to your :origin:`searx/settings.yml`: + + .. code:: yaml + + result_proxy: + # replace example.org with your server's public name + url : https://example.org/morty + + server: + image_proxy : True + + + .. group-tab:: proxy or uWSGI + + Be warned, with this setup, your instance isn't :ref:`protected <searx + filtron>`. Nevertheless it is good enough for intranet usage and it is a + excellent example of; *how different services can be set up*. The next + example shows a reverse proxy configuration wrapping the :ref:`searx-uWSGI + application <uwsgi configuration>`, listening on ``http = + 127.0.0.1:8888``. + + .. code:: nginx + + # https://hostname.local/ + + location / { + proxy_pass http://127.0.0.1:8888; + + proxy_set_header Host $host; + proxy_set_header Connection $http_connection; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_buffering off; + } + + Alternatively you can use the `uWSGI support from nginx`_ via unix + sockets. For socket communication, you have to activate ``socket = + /run/uwsgi/app/searx/socket`` and comment out the ``http = + 127.0.0.1:8888`` configuration in your :ref:`uwsgi ini file <uwsgi + configuration>`. + + The example shows a nginx virtual ``server`` configuration, listening on + port 80 (IPv4 and IPv6 http://[::]:80). The uWSGI app is configured at + location ``/`` by importing the `uwsgi_params`_ and passing requests to + the uWSGI socket (``uwsgi_pass``). The ``server``\'s root points to the + :ref:`searx-src clone <searx-src>` and wraps directly the + :origin:`searx/static/` content at ``location /static``. + + .. code:: nginx + + server { + # replace hostname.local with your server's name + server_name hostname.local; + + listen 80; + listen [::]:80; + + location / { + include uwsgi_params; + uwsgi_pass unix:/run/uwsgi/app/searx/socket; + } + + root /usr/local/searx/searx-src/searx; + location /static { } + } + + If not already exists, create a folder for the unix sockets, which can be + used by the searx account: + + .. code:: bash + + mkdir -p /run/uwsgi/app/searx/ + sudo -H chown -R searx:searx /run/uwsgi/app/searx/ + + .. group-tab:: \.\. at subdir URL + + Be warned, with these setups, your instance isn't :ref:`protected <searx + filtron>`. The examples are just here to demonstrate how to export the + searx application from a subdirectory URL ``https://example.org/searx/``. + + .. code:: nginx + + # https://hostname.local/searx + + location /searx { + proxy_pass http://127.0.0.1:8888; + + proxy_set_header Host $host; + proxy_set_header Connection $http_connection; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_set_header X-Script-Name /searx; + proxy_buffering off; + } + + location /searx/static { + alias /usr/local/searx/searx-src/searx/static; + } + + The ``X-Script-Name /searx`` is needed by the searx implementation to + calculate relative URLs correct. The next example shows a uWSGI + configuration. Since there are no HTTP headers in a (u)WSGI protocol, the + value is shipped via the SCRIPT_NAME_ in the WSGI environment. + + .. code:: nginx + + # https://hostname.local/searx + + location /searx { + uwsgi_param SCRIPT_NAME /searx; + include uwsgi_params; + uwsgi_pass unix:/run/uwsgi/app/searx/socket; + } + + location /searx/static { + alias /usr/local/searx/searx-src/searx; + } + + For searx to work correctly the ``base_url`` must be set in the + :origin:`searx/settings.yml`. + + .. code:: yaml + + server: + # replace example.org with your server's public name + base_url : https://example.org/searx/ + + +Restart service: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H systemctl restart nginx + sudo -H service uwsgi restart searx + + .. group-tab:: Arch Linux + + .. code:: sh + + sudo -H systemctl restart nginx + sudo -H systemctl restart uwsgi@searx + + .. group-tab:: Fedora + + .. code:: sh + + sudo -H systemctl restart nginx + sudo -H touch /etc/uwsgi.d/searx.ini + + +Disable logs +============ + +For better privacy you can disable nginx logs in ``/etc/nginx/nginx.conf``. + +.. code:: nginx + + http { + # ... + access_log /dev/null; + error_log /dev/null; + # ... + } diff --git a/docs/admin/installation-searx.rst b/docs/admin/installation-searx.rst new file mode 100644 index 000000000..f1d486021 --- /dev/null +++ b/docs/admin/installation-searx.rst @@ -0,0 +1,92 @@ +.. _installation basic: + +========================= +Step by step installation +========================= + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +Step by step installation with virtualenv. For Ubuntu, be sure to have enable +universe repository. + +.. _install packages: + +Install packages +================ + +.. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START distro-packages + :end-before: END distro-packages + +.. hint:: + + This installs also the packages needed by :ref:`searx uwsgi` + +.. _create searx user: + +Create user +=========== + +.. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START create user + :end-before: END create user + +.. _searx-src: + +install searx & dependencies +============================ + +Start a interactive shell from new created user and clone searx: + +.. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START clone searx + :end-before: END clone searx + +In the same shell create *virtualenv*: + +.. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START create virtualenv + :end-before: END create virtualenv + +To install searx's dependencies, exit the searx *bash* session you opened above +and restart a new. Before install, first check if your *virualenv* was sourced +from the login (*~/.profile*): + +.. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START manage.sh update_packages + :end-before: END manage.sh update_packages + +.. tip:: + + Open a second terminal for the configuration tasks and left the ``(searx)$`` + terminal open for the tasks below. + +Configuration +============== + +Create a copy of the :origin:`searx/settings.yml` configuration file in system's +*/etc* folder. Configure like shown below -- replace ``searx@\$(uname -n)`` with +a name of your choice -- *and/or* edit ``/etc/searx/settings.yml`` if necessary. + +.. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START searx config + :end-before: END searx config + +Check +===== + +To check your searx setup, optional enable debugging and start the *webapp*. +Searx looks at the exported environment ``$SEARX_SETTINGS_PATH`` for a +configuration file. + +.. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START check searx installation + :end-before: END check searx installation + +If everything works fine, hit ``[CTRL-C]`` to stop the *webapp* and disable the +debug option in ``settings.yml``. You can now exit searx user bash (enter exit +command twice). At this point searx is not demonized; uwsgi allows this. + diff --git a/docs/admin/installation-uwsgi.rst b/docs/admin/installation-uwsgi.rst new file mode 100644 index 000000000..7b482975d --- /dev/null +++ b/docs/admin/installation-uwsgi.rst @@ -0,0 +1,150 @@ +.. _searx uwsgi: + +===== +uwsgi +===== + +.. sidebar:: further reading + + - `systemd.unit`_ + - `uWSGI Emperor`_ + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +.. _systemd.unit: https://www.freedesktop.org/software/systemd/man/systemd.unit.html +.. _One service per app in systemd: + https://uwsgi-docs.readthedocs.io/en/latest/Systemd.html#one-service-per-app-in-systemd +.. _uWSGI Emperor: + https://uwsgi-docs.readthedocs.io/en/latest/Emperor.html +.. _uwsgi ini file: + https://uwsgi-docs.readthedocs.io/en/latest/Configuration.html#ini-files +.. _systemd unit template: + http://0pointer.de/blog/projects/instances.html + + +Origin uWSGI +============ + +How uWSGI is implemented by distributors is different. uWSGI itself +recommend two methods + +`systemd.unit`_ template files as described here `One service per app in systemd`_. + + There is one `systemd unit template`_ and one `uwsgi ini file`_ per uWSGI-app + placed at dedicated locations. Take archlinux and a searx.ini as example:: + + unit template --> /usr/lib/systemd/system/uwsgi@.service + uwsgi ini files --> /etc/uwsgi/searx.ini + + The searx app can be maintained as know from common systemd units:: + + systemctl enable uwsgi@searx + systemctl start uwsgi@searx + systemctl restart uwsgi@searx + systemctl stop uwsgi@searx + +The `uWSGI Emperor`_ mode which fits for maintaining a large range of uwsgi apps. + + The Emperor mode is a special uWSGI instance that will monitor specific + events. The Emperor mode (service) is started by a (common, not template) + systemd unit. The Emperor service will scan specific directories for `uwsgi + ini file`_\s (also know as *vassals*). If a *vassal* is added, removed or the + timestamp is modified, a corresponding action takes place: a new uWSGI + instance is started, reload or stopped. Take Fedora and a searx.ini as + example:: + + to start a new searx instance create --> /etc/uwsgi.d/searx.ini + to reload the instance edit timestamp --> touch /etc/uwsgi.d/searx.ini + to stop instance remove ini --> rm /etc/uwsgi.d/searx.ini + +Distributors +============ + +The `uWSGI Emperor`_ mode and `systemd unit template`_ is what the distributors +mostly offer their users, even if they differ in the way they implement both +modes and their defaults. Another point they might differ is the packaging of +plugins (if so, compare :ref:`install packages`) and what the default python +interpreter is (python2 vs. python3). + +Fedora starts a Emperor by default, while archlinux does not start any uwsgi +service by default. Worth to know; debian (ubuntu) follow a complete different +approach. *debian*: your are familiar with the apache infrastructure? .. they +do similar for the uWSGI infrastructure (with less comfort), the folders are:: + + /etc/uwsgi/apps-available/ + /etc/uwsgi/apps-enabled/ + +The `uwsgi ini file`_ is enabled by a symbolic link:: + + ln -s /etc/uwsgi/apps-available/searx.ini /etc/uwsgi/apps-enabled/ + +From debian's documentation (``/usr/share/doc/uwsgi/README.Debian.gz``): You +could control specific instance(s) by issuing:: + + service uwsgi <command> <confname> <confname> ... + + sudo -H service uwsgi start searx + sudo -H service uwsgi stop searx + +My experience is, that this command is a bit buggy. + +.. _uwsgi configuration: + +Alltogether +=========== + +Create the configuration ini-file according to your distribution (see below) and +restart the uwsgi application. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START searx uwsgi-description ubuntu-20.04 + :end-before: END searx uwsgi-description ubuntu-20.04 + + .. hotfix: a bug group-tab need this comment + + .. group-tab:: Arch Linux + + .. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START searx uwsgi-description arch + :end-before: END searx uwsgi-description arch + + .. hotfix: a bug group-tab need this comment + + .. group-tab:: Fedora / RHEL + + .. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START searx uwsgi-description fedora + :end-before: END searx uwsgi-description fedora + + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START searx uwsgi-appini ubuntu-20.04 + :end-before: END searx uwsgi-appini ubuntu-20.04 + + .. hotfix: a bug group-tab need this comment + + .. group-tab:: Arch Linux + + .. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START searx uwsgi-appini arch + :end-before: END searx uwsgi-appini arch + + .. hotfix: a bug group-tab need this comment + + .. group-tab:: Fedora / RHEL + + .. kernel-include:: $DOCS_BUILD/includes/searx.rst + :start-after: START searx uwsgi-appini fedora + :end-before: END searx uwsgi-appini fedora diff --git a/docs/admin/installation.rst b/docs/admin/installation.rst index 15800fc01..167c300fe 100644 --- a/docs/admin/installation.rst +++ b/docs/admin/installation.rst @@ -4,346 +4,63 @@ Installation ============ -.. contents:: - :depth: 3 +*You're spoilt for choice*, choose your preferred method of installation. -Basic installation -================== +- :ref:`installation docker` +- :ref:`installation scripts` +- :ref:`installation basic` -Step by step installation for Debian/Ubuntu with virtualenv. For Ubuntu, be sure -to have enable universe repository. +The :ref:`installation basic` is good enough for intranet usage and it is a +excellent illustration of *how a searx instance is build up*. If you place your +instance public to the internet you should really consider to install a +:ref:`filtron reverse proxy <filtron.sh>` and for privacy a :ref:`result proxy +<morty.sh>` is mandatory. -Install packages: +Therefore, if you do not have any special preferences, its recommend to use the +:ref:`installation docker` or the `Installation scripts`_ from our :ref:`tooling +box <toolboxing>` as described below. -.. code:: sh +.. _installation scripts: - $ sudo -H apt-get install \ - git build-essential libxslt-dev \ - python-dev python-virtualenv python-babel \ - zlib1g-dev libffi-dev libssl-dev +Installation scripts +==================== -Install searx: +.. sidebar:: Update OS first! -.. code:: sh + To avoid unwanted side effects, update your OS before installing searx. - cd /usr/local - sudo -H git clone https://github.com/asciimoo/searx.git - sudo -H useradd searx -d /usr/local/searx - sudo -H chown searx:searx -R /usr/local/searx +The following will install a setup as shown in :ref:`architecture`. First you +need to get a clone. The clone is only needed for the installation procedure +and some maintenance tasks (alternatively you can create your own fork). -Install dependencies in a virtualenv: +.. code:: bash -.. code:: sh + $ cd ~/Downloads + $ git clone https://github.com/asciimoo/searx searx + $ cd searx - cd /usr/local/searx - sudo -H -u searx -i +**Install** :ref:`searx service <searx.sh>` -.. code:: sh +This installs searx as described in :ref:`installation basic`. - (searx)$ virtualenv searx-ve - (searx)$ . ./searx-ve/bin/activate - (searx)$ ./manage.sh update_packages +.. code:: bash -Configuration -============== + $ sudo -H ./utils/searx.sh install all -.. code:: sh +**Install** :ref:`filtron reverse proxy <filtron.sh>` - sed -i -e "s/ultrasecretkey/`openssl rand -hex 16`/g" searx/settings.yml +.. code:: bash -Edit searx/settings.yml if necessary. + $ sudo -H ./utils/filtron.sh install all -Check -===== +**Install** :ref:`result proxy <morty.sh>` -Start searx: +.. code:: bash -.. code:: sh + $ sudo -H ./utils/morty.sh install all - python searx/webapp.py +If all services are running fine, you can add it to your HTTP server: -Go to http://localhost:8888 +- :ref:`installation apache` +- :ref:`installation nginx` -If everything works fine, disable the debug option in settings.yml: - -.. code:: sh - - sed -i -e "s/debug : True/debug : False/g" searx/settings.yml - -At this point searx is not demonized ; uwsgi allows this. - -You can exit the virtualenv and the searx user bash (enter exit command -twice). - -uwsgi -===== - -Install packages: - -.. code:: sh - - sudo -H apt-get install \ - uwsgi uwsgi-plugin-python - -Create the configuration file ``/etc/uwsgi/apps-available/searx.ini`` with this -content: - -.. code:: ini - - [uwsgi] - # Who will run the code - uid = searx - gid = searx - - # disable logging for privacy - disable-logging = true - - # Number of workers (usually CPU count) - workers = 4 - - # The right granted on the created socket - chmod-socket = 666 - - # Plugin to use and interpretor config - single-interpreter = true - master = true - plugin = python - lazy-apps = true - enable-threads = true - - # Module to import - module = searx.webapp - - # Support running the module from a webserver subdirectory. - route-run = fixpathinfo: - - # Virtualenv and python path - virtualenv = /usr/local/searx/searx-ve/ - pythonpath = /usr/local/searx/ - chdir = /usr/local/searx/searx/ - -Activate the uwsgi application and restart: - -.. code:: sh - - cd /etc/uwsgi/apps-enabled - ln -s ../apps-available/searx.ini - /etc/init.d/uwsgi restart - -Web server -========== - -with nginx ----------- - -If nginx is not installed (uwsgi will not work with the package -nginx-light): - -.. code:: sh - - sudo -H apt-get install nginx - -Hosted at / -~~~~~~~~~~~ - -Create the configuration file ``/etc/nginx/sites-available/searx`` with this -content: - -.. code:: nginx - - server { - listen 80; - server_name searx.example.com; - root /usr/local/searx/searx; - - location /static { - } - - location / { - include uwsgi_params; - uwsgi_pass unix:/run/uwsgi/app/searx/socket; - } - } - -Create a symlink to sites-enabled: - -.. code:: sh - - sudo -H ln -s /etc/nginx/sites-available/searx /etc/nginx/sites-enabled/searx - -Restart service: - -.. code:: sh - - sudo -H service nginx restart - sudo -H service uwsgi restart - -from subdirectory URL (/searx) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Add this configuration in the server config file -``/etc/nginx/sites-enabled/default``: - -.. code:: nginx - - location /searx/static { - alias /usr/local/searx/searx/static; - } - - location /searx { - uwsgi_param SCRIPT_NAME /searx; - include uwsgi_params; - uwsgi_pass unix:/run/uwsgi/app/searx/socket; - } - - -**OR** using reverse proxy (Please, note that reverse proxy advised to be used -in case of single-user or low-traffic instances.) - -.. code:: nginx - - location /searx/static { - alias /usr/local/searx/searx/static; - } - - location /searx { - proxy_pass http://127.0.0.1:8888; - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Scheme $scheme; - proxy_set_header X-Script-Name /searx; - proxy_buffering off; - } - - -Enable ``base_url`` in ``searx/settings.yml`` - -.. code:: yaml - - base_url : http://your.domain.tld/searx/ - -Restart service: - -.. code:: sh - - sudo -H service nginx restart - sudo -H service uwsgi restart - -disable logs -^^^^^^^^^^^^ - -for better privacy you can disable nginx logs about searx. - -how to proceed: below ``uwsgi_pass`` in ``/etc/nginx/sites-available/default`` -add: - -.. code:: nginx - - access_log /dev/null; - error_log /dev/null; - -Restart service: - -.. code:: sh - - sudo -H service nginx restart - -with apache ------------ - -Add wsgi mod: - -.. code:: sh - - sudo -H apt-get install libapache2-mod-uwsgi - sudo -H a2enmod uwsgi - -Add this configuration in the file ``/etc/apache2/apache2.conf``: - -.. code:: apache - - <Location /> - Options FollowSymLinks Indexes - SetHandler uwsgi-handler - uWSGISocket /run/uwsgi/app/searx/socket - </Location> - -Note that if your instance of searx is not at the root, you should change -``<Location />`` by the location of your instance, like ``<Location /searx>``. - -Restart Apache: - -.. code:: sh - - sudo -H /etc/init.d/apache2 restart - -disable logs -~~~~~~~~~~~~ - -For better privacy you can disable Apache logs. - -.. warning:: - - You can only disable logs for the whole (virtual) server not for a specific - path. - -Go back to ``/etc/apache2/apache2.conf`` and above ``<Location />`` add: - -.. code:: apache - - CustomLog /dev/null combined - -Restart Apache: - -.. code:: sh - - sudo -H /etc/init.d/apache2 restart - -How to update -============= - -.. code:: sh - - cd /usr/local/searx - sudo -H -u searx -i - -.. code:: sh - - (searx)$ . ./searx-ve/bin/activate - (searx)$ git stash - (searx)$ git pull origin master - (searx)$ git stash apply - (searx)$ ./manage.sh update_packages - -.. code:: sh - - sudo -H service uwsgi restart - -Docker -====== - -Make sure you have installed Docker. For instance, you can deploy searx like this: - -.. code:: sh - - docker pull wonderfall/searx - docker run -d --name searx -p $PORT:8888 wonderfall/searx - -Go to ``http://localhost:$PORT``. - -See https://hub.docker.com/r/wonderfall/searx/ for more informations. It's also -possible to build searx from the embedded Dockerfile. - -.. code:: sh - - git clone https://github.com/asciimoo/searx.git - cd searx - docker build -t whatever/searx . - -References -========== - -* https://about.okhin.fr/posts/Searx/ with some additions - -* How to: `Setup searx in a couple of hours with a free SSL certificate - <https://www.reddit.com/r/privacytoolsIO/comments/366kvn/how_to_setup_your_own_privacy_respecting_search/>`__ diff --git a/docs/admin/morty.rst b/docs/admin/morty.rst index 7d7b34492..2858fde3c 100644 --- a/docs/admin/morty.rst +++ b/docs/admin/morty.rst @@ -1,7 +1,14 @@ + +.. _searx morty: + ========================= How to setup result proxy ========================= +.. sidebar:: further reading + + - :ref:`morty.sh` + .. _morty: https://github.com/asciimoo/morty .. _morty's README: https://github.com/asciimoo/morty diff --git a/docs/admin/settings.rst b/docs/admin/settings.rst index 0bfdcc6cb..17dee4da8 100644 --- a/docs/admin/settings.rst +++ b/docs/admin/settings.rst @@ -4,11 +4,17 @@ ``settings.yml`` ================ +This page describe the options possibilities of the :origin:`searx/settings.yml` +file. + .. sidebar:: Further reading .. - :ref:`search API` -This page describe the options possibilities of the settings.yml file. +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry .. _settings global: @@ -70,7 +76,7 @@ Global Settings messages in the browser too, so this must be deactivated in production. ``request_timeout`` : - Global timeout of the requests made to others engines in seconds. A bigger + Global timeout of the requests made to others engines in seconds. A bigger timeout will allow to wait for answers from slow engines, but in consequence will slow searx reactivity (the result page may take the time specified in the timeout to load) @@ -83,7 +89,7 @@ Global Settings blank. ``default_theme`` : - Name of the theme you want to use by default on you searx instance. + Name of the theme you want to use by default on your searx instance. ``useragent_suffix`` : Suffix to the user-agent searx uses to send requests to others engines. If an @@ -93,12 +99,12 @@ Global Settings Allow your instance of searx of being able to proxy images. Uses memory space. ``default_locale`` : - Aearx interface language. If blank, the locale is detected by using the + Searx interface language. If blank, the locale is detected by using the browser language. If it doesn't work, or you are deploying a language specific instance of searx, a locale can be defined using an ISO language code, like ``fr``, ``en``, ``de``. -.. _requests proxies: http://docs.python-requests.org/en/latest/user/advanced/#proxies +.. _requests proxies: http://requests.readthedocs.io/en/latest/user/advanced/#proxies .. _PR SOCKS support: https://github.com/kennethreitz/requests/pull/478 ``outgoing_proxies`` : @@ -106,7 +112,7 @@ Global Settings not supported / see `PR SOCKS support`. ``source_ips`` : - If you use multiple nework interfaces, define from which IP the requests must + If you use multiple network interfaces, define from which IP the requests must be made. ``locales`` : @@ -135,7 +141,7 @@ Engine settings language : en_US ``name`` : - Name that will be used accross searx to define this engine. In settings, on + Name that will be used across searx to define this engine. In settings, on the result page... ``engine`` : @@ -146,7 +152,7 @@ Engine settings Code used to execute bang requests (in this case using ``!bi`` or ``?bi``) ``base_url`` : optional - Part of the URL that should be stable accross every request. Can be useful to + Part of the URL that should be stable across every request. Can be useful to use multiple sites using only one engine, or updating the site URL without touching at the code. @@ -164,7 +170,7 @@ Engine settings is described in the file. ``disabled`` : optional - To disable by default the engine, but not deleting it. It will allow the user + To disable by default the engine, but not deleting it. It will allow the user to manually activate it in the settings. ``language`` : optional @@ -175,6 +181,9 @@ Engine settings ``weigth`` : default ``1`` Weighting of the results of this engine. +``display_error_messages`` : default ``True`` + When an engine returns an error, the message is displayed on the user interface. + .. note:: A few more options are possible, but they are pretty specific to some diff --git a/docs/admin/update-searx.rst b/docs/admin/update-searx.rst new file mode 100644 index 000000000..c74c9568a --- /dev/null +++ b/docs/admin/update-searx.rst @@ -0,0 +1,23 @@ +.. _update searx: + +============= +How to update +============= + +.. code:: sh + + sudo -H -u searx -i + (searx)$ git stash + (searx)$ git pull origin master + (searx)$ git stash apply + (searx)$ ./manage.sh update_packages + +Restart uwsgi: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H systemctl restart uwsgi |