From bdf392093af92da7d8fc4e32b68ff0716700522d Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Wed, 8 Jan 2020 18:09:36 +0100 Subject: build requirements: add a shell script static analysis tool ShellCheck: https://github.com/koalaman/shellcheck Signed-off-by: Markus Heiser --- docs/admin/buildhosts.rst | 50 ++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 9 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/buildhosts.rst b/docs/admin/buildhosts.rst index 5260da033..c1582cef2 100644 --- a/docs/admin/buildhosts.rst +++ b/docs/admin/buildhosts.rst @@ -35,8 +35,17 @@ processing additional packages are needed. The XeTeX_ needed not only for PDF creation, its also needed for :ref:`math` when HTML output is build. To be able to do :ref:`sphinx:math-support` without CDNs, the math are rendered -as images (``sphinx.ext.imgmath`` extension). If your docs build (``make -docs``) shows warnings like this:: +as images (``sphinx.ext.imgmath`` extension). + +Here is the extract from the :origin:`docs/conf.py` file, setting math renderer +to ``imgmath``: + +.. literalinclude:: ../conf.py + :language: python + :start-after: # sphinx.ext.imgmath setup + :end-before: # sphinx.ext.imgmath setup END + +If your docs build (``make docs``) shows warnings like this:: WARNING: dot(1) not found, for better output quality install \ graphviz from http://www.graphviz.org @@ -47,8 +56,6 @@ docs``) shows warnings like this:: you need to install additional packages on your build host, to get better HTML output. -.. _system requirements: - .. tabs:: .. group-tab:: Ubuntu / debian @@ -94,10 +101,35 @@ For PDF output you also need: texlive-collection-fontsrecommended texlive-collection-latex \ dejavu-sans-fonts dejavu-serif-fonts dejavu-sans-mono-fonts -.. _system requirements END: +.. _sh lint: -.. literalinclude:: ../conf.py - :language: python - :start-after: # sphinx.ext.imgmath setup - :end-before: # sphinx.ext.imgmath setup END +Lint shell scripts +================== + +.. _ShellCheck: https://github.com/koalaman/shellcheck + +To lint shell scripts, we use ShellCheck_ - A shell script static analysis tool. + +.. SNIP sh lint requirements + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code-block:: sh + + $ sudo apt install shellcheck + + .. group-tab:: Arch Linux + + .. code-block:: sh + + $ sudo pacman -S shellcheck + + .. group-tab:: Fedora / RHEL + + .. code-block:: sh + + $ sudo dnf install ShellCheck +.. SNAP sh lint requirements -- cgit v1.2.3 From 39feb141bc8361915b3d80a976852b38851e0419 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sat, 11 Jan 2020 12:50:40 +0100 Subject: docs(admin): add description of the utils/filtron.sh script Signed-off-by: Markus Heiser --- docs/admin/filtron.rst | 34 +++++++++++++++++++++++++++++++--- 1 file changed, 31 insertions(+), 3 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/filtron.rst b/docs/admin/filtron.rst index 07dcb9bc5..009ca1c29 100644 --- a/docs/admin/filtron.rst +++ b/docs/admin/filtron.rst @@ -2,12 +2,40 @@ How to protect an instance ========================== +.. _filtron: https://github.com/asciimoo/filtron + Searx depens on external search services. To avoid the abuse of these services it is advised to limit the number of requests processed by searx. -An application firewall, ``filtron`` solves exactly this problem. Information -on how to install it can be found at the `project page of filtron -`__. +An application firewall, filtron_ solves exactly this problem. Filtron is just +a middleware between your web server (nginx, apache, ...) and searx. + + +filtron & go +============ + +.. _Go: https://golang.org/ +.. _filtron README: https://github.com/asciimoo/filtron/blob/master/README.md + +Filtron needs Go_ installed. If Go_ is preinstalled, filtron_ is simply +installed by ``go get`` package management (see `filtron README`_). If you use +filtron as middleware, a more isolated setup is recommended. + +#. Create a separated user account (``filtron``). +#. Download and install Go_ binary in users $HOME (``~filtron``). +#. Install filtron with the package management of Go_ (``go get -v -u + github.com/asciimoo/filtron``) +#. Setup a proper rule configuration :origin:`[ref] + ` (``/etc/filtron/rules.json``). +#. Setup a systemd service unit :origin:`[ref] + ` + (``/lib/systemd/system/filtron.service``). + +To simplify such a installation and the maintenance of; use our script +``utils/filtron.sh``: + +.. program-output:: ../utils/filtron.sh --help + :ellipsis: 0,5 Sample configuration of filtron -- cgit v1.2.3 From fe64274fb0405c47b3ba9039ab4e5b1440370a4d Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sun, 12 Jan 2020 15:53:35 +0100 Subject: docs: filtron.sh add remark about init systems Signed-off-by: Markus Heiser --- docs/admin/filtron.rst | 7 +++++++ 1 file changed, 7 insertions(+) (limited to 'docs/admin') diff --git a/docs/admin/filtron.rst b/docs/admin/filtron.rst index 009ca1c29..e8a2bfb15 100644 --- a/docs/admin/filtron.rst +++ b/docs/admin/filtron.rst @@ -17,6 +17,13 @@ filtron & go .. _Go: https://golang.org/ .. _filtron README: https://github.com/asciimoo/filtron/blob/master/README.md + +.. sidebar:: init system + + ATM the ``filtron.sh`` supports only systemd init process used by debian, + ubuntu and many other dists. If you have a working init.d file to start/stop + filtron service, please contribute. + Filtron needs Go_ installed. If Go_ is preinstalled, filtron_ is simply installed by ``go get`` package management (see `filtron README`_). If you use filtron as middleware, a more isolated setup is recommended. -- cgit v1.2.3 From 94ac560dcc85e2e75d3b14706a65eb9488643eb4 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Mon, 3 Feb 2020 13:25:51 +0100 Subject: doc: add descriptions of our tooling box in ./utils Initial started with the scripts for searx, filtron and morty installations. Signed-off-by: Markus Heiser --- docs/admin/filtron.rst | 36 +++++++++++------------------------- docs/admin/morty.rst | 3 +++ 2 files changed, 14 insertions(+), 25 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/filtron.rst b/docs/admin/filtron.rst index e8a2bfb15..332c27203 100644 --- a/docs/admin/filtron.rst +++ b/docs/admin/filtron.rst @@ -1,14 +1,22 @@ + +.. _searx_filtron: + ========================== How to protect an instance ========================== +.. sidebar:: further reading + + - :ref:`filtron.sh` + .. _filtron: https://github.com/asciimoo/filtron Searx depens on external search services. To avoid the abuse of these services it is advised to limit the number of requests processed by searx. An application firewall, filtron_ solves exactly this problem. Filtron is just -a middleware between your web server (nginx, apache, ...) and searx. +a middleware between your web server (nginx, apache, ...) and searx, we describe +such infratructures in chapter: :ref:`architecture`. filtron & go @@ -17,32 +25,10 @@ filtron & go .. _Go: https://golang.org/ .. _filtron README: https://github.com/asciimoo/filtron/blob/master/README.md - -.. sidebar:: init system - - ATM the ``filtron.sh`` supports only systemd init process used by debian, - ubuntu and many other dists. If you have a working init.d file to start/stop - filtron service, please contribute. - Filtron needs Go_ installed. If Go_ is preinstalled, filtron_ is simply installed by ``go get`` package management (see `filtron README`_). If you use -filtron as middleware, a more isolated setup is recommended. - -#. Create a separated user account (``filtron``). -#. Download and install Go_ binary in users $HOME (``~filtron``). -#. Install filtron with the package management of Go_ (``go get -v -u - github.com/asciimoo/filtron``) -#. Setup a proper rule configuration :origin:`[ref] - ` (``/etc/filtron/rules.json``). -#. Setup a systemd service unit :origin:`[ref] - ` - (``/lib/systemd/system/filtron.service``). - -To simplify such a installation and the maintenance of; use our script -``utils/filtron.sh``: - -.. program-output:: ../utils/filtron.sh --help - :ellipsis: 0,5 +filtron as middleware, a more isolated setup is recommended. To simplify such +an installation and the maintenance of, use our script :ref:`filtron.sh`. Sample configuration of filtron diff --git a/docs/admin/morty.rst b/docs/admin/morty.rst index 7d7b34492..9af9b6ae9 100644 --- a/docs/admin/morty.rst +++ b/docs/admin/morty.rst @@ -1,3 +1,6 @@ + +.. _searx_morty: + ========================= How to setup result proxy ========================= -- cgit v1.2.3 From 9278f0fb45bab21247085a709c0e4bef743752db Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Tue, 4 Feb 2020 13:13:17 +0100 Subject: docs: add some documentation about the tooling box ./utils/* Signed-off-by: Markus Heiser --- docs/admin/arch_public.dot | 8 ++++---- docs/admin/architecture.rst | 12 +++++++----- docs/admin/installation.rst | 6 ++++++ 3 files changed, 17 insertions(+), 9 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/arch_public.dot b/docs/admin/arch_public.dot index a46c96de3..5018225c3 100644 --- a/docs/admin/arch_public.dot +++ b/docs/admin/arch_public.dot @@ -4,11 +4,11 @@ digraph G { edge [fontname="Sans"]; browser [label="Browser", shape=Mdiamond]; - rp [label="Reverse Proxy", href="url to configure reverse proxy"]; - filtron [label="Filtron", href="https://github.com/asciimoo/filtron"]; - morty [label="Morty", href="https://github.com/asciimoo/morty"]; + rp [label="Reverse Proxy", href="https://asciimoo.github.io/searx/utils/filtron.sh.html#public-reverse-proxy"]; + filtron [label="Filtron", href="https://asciimoo.github.io/searx/utils/filtron.sh.html"]; + morty [label="Morty", href="https://asciimoo.github.io/searx/utils/morty.sh.html"]; static [label="Static files", href="url to configure static files"]; - uwsgi [label="uwsgi", href="url to configure uwsgi"] + uwsgi [label="uwsgi", href="https://asciimoo.github.io/searx/utils/searx.sh.html"] searx1 [label="Searx #1"]; searx2 [label="Searx #2"]; searx3 [label="Searx #3"]; diff --git a/docs/admin/architecture.rst b/docs/admin/architecture.rst index 7064a294b..fbc3bf483 100644 --- a/docs/admin/architecture.rst +++ b/docs/admin/architecture.rst @@ -4,17 +4,19 @@ Architecture ============ -.. sidebar:: Needs work! +.. sidebar:: Tooling box - This article needs some work / Searx is a collaborative effort. If you have - any contribution, feel welcome to send us your :pull:`PR <../pulls>`, see - :ref:`how to contribute`. + - :ref:`searx & uwsgi ` + - :ref:`filtron ` + - :ref:`reverse proxy` + - :ref:`morty ` Herein you will find some hints and suggestions about typical architectures of searx infrastructures. We start with a contribution from :pull:`@dalf <1776#issuecomment-567917320>`. -It shows a *reference* setup for public searx instances. +It shows a *reference* setup for public searx instances which can build up and +maintained by the scripts from our :ref:`toolboxing`. .. _arch public: diff --git a/docs/admin/installation.rst b/docs/admin/installation.rst index 15800fc01..bff3e44d9 100644 --- a/docs/admin/installation.rst +++ b/docs/admin/installation.rst @@ -7,9 +7,15 @@ Installation .. contents:: :depth: 3 +.. _installation basic: + Basic installation ================== +.. sidebar:: further reading + + - :ref:`searx.sh` + Step by step installation for Debian/Ubuntu with virtualenv. For Ubuntu, be sure to have enable universe repository. -- cgit v1.2.3 From eedd63ccd504099836d9f528d02daffa6bc826ad Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Tue, 4 Feb 2020 16:42:13 +0100 Subject: docs: revision of the installation instructions Signed-off-by: Markus Heiser --- docs/admin/buildhosts.rst | 9 +- docs/admin/filtron.rst | 5 + docs/admin/index.rst | 5 +- docs/admin/installation-apache.rst | 94 +++++++++++ docs/admin/installation-nginx.rst | 141 ++++++++++++++++ docs/admin/installation.rst | 324 ++++++++++++------------------------- docs/admin/settings.rst | 8 +- 7 files changed, 361 insertions(+), 225 deletions(-) create mode 100644 docs/admin/installation-apache.rst create mode 100644 docs/admin/installation-nginx.rst (limited to 'docs/admin') diff --git a/docs/admin/buildhosts.rst b/docs/admin/buildhosts.rst index c1582cef2..27784764f 100644 --- a/docs/admin/buildhosts.rst +++ b/docs/admin/buildhosts.rst @@ -4,13 +4,18 @@ Buildhosts ========== +To get best results from build, its recommend to install additional packages +on build hosts. + .. sidebar:: This article needs some work If you have any contribution send us your :pull:`PR <../pulls>`, see :ref:`how to contribute`. -To get best results from build, its recommend to install additional packages -on build hosts. +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry .. _docs build: diff --git a/docs/admin/filtron.rst b/docs/admin/filtron.rst index 332c27203..205269973 100644 --- a/docs/admin/filtron.rst +++ b/docs/admin/filtron.rst @@ -9,6 +9,11 @@ How to protect an instance - :ref:`filtron.sh` +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + .. _filtron: https://github.com/asciimoo/filtron Searx depens on external search services. To avoid the abuse of these services diff --git a/docs/admin/index.rst b/docs/admin/index.rst index b3c7f5119..4190bf24b 100644 --- a/docs/admin/index.rst +++ b/docs/admin/index.rst @@ -3,9 +3,12 @@ Administrator documentation =========================== .. toctree:: - :maxdepth: 1 + :maxdepth: 2 + :caption: Contents installation + installation-nginx + installation-apache settings api architecture diff --git a/docs/admin/installation-apache.rst b/docs/admin/installation-apache.rst new file mode 100644 index 000000000..7f92e47d5 --- /dev/null +++ b/docs/admin/installation-apache.rst @@ -0,0 +1,94 @@ +.. _installation apache: + +=================== +Install with apache +=================== + +.. sidebar:: public HTTP servers + + On public searx instances use an application firewall (:ref:`filtron + `). + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +Add wsgi mod +============ + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H apt-get install libapache2-mod-uwsgi + sudo -H a2enmod uwsgi + +Add this configuration in the file ``/etc/apache2/apache2.conf``. To limit +acces to your intranet replace ``Allow from all`` directive and replace +``192.168.0.0/16`` with your subnet IP/class. + +.. _inranet apache site: + +Note that if your instance of searx is not at the root, you should change +```` by the location of your instance, like ````: + +.. code:: apache + + # CustomLog /dev/null combined + + + + + + Options FollowSymLinks Indexes + SetHandler uwsgi-handler + uWSGISocket /run/uwsgi/app/searx/socket + + Order deny,allow + Deny from all + # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + + + + +Enable apache mod_uwsgi and restart apache: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + a2enmod uwsgi + sudo -H systemctl restart apache2 + +disable logs +============ + +For better privacy you can disable Apache logs. Go back to +``/etc/apache2/apache2.conf`` :ref:`[example] ` and above +```` activate directive: + +.. code:: apache + + CustomLog /dev/null combined + +Restart apache: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H systemctl restart apache2 + +.. warning:: + + You can only disable logs for the whole (virtual) server not for a specific + path. diff --git a/docs/admin/installation-nginx.rst b/docs/admin/installation-nginx.rst new file mode 100644 index 000000000..37d3e7532 --- /dev/null +++ b/docs/admin/installation-nginx.rst @@ -0,0 +1,141 @@ +.. _installation nginx: + +================== +Install with nginx +================== + +.. sidebar:: public HTTP servers + + On public searx instances use an application firewall (:ref:`filtron + `). + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +If nginx is not installed (uwsgi will not work with the package +nginx-light): + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H apt-get install nginx + +Hosted at ``/`` +=============== + +Create the configuration file ``/etc/nginx/sites-available/searx`` with this +content: + +.. code:: nginx + + server { + listen 80; + server_name searx.example.com; + root /usr/local/searx/searx; + + location /static { + } + + location / { + include uwsgi_params; + uwsgi_pass unix:/run/uwsgi/app/searx/socket; + } + } + +Create a symlink to sites-enabled: + +.. code:: sh + + sudo -H ln -s /etc/nginx/sites-available/searx /etc/nginx/sites-enabled/searx + +Restart service: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H systemctl restart nginx + sudo -H systemctl restart uwsgi + +from subdirectory URL (``/searx``) +================================== + +Add this configuration in the server config file +``/etc/nginx/sites-enabled/default``: + +.. code:: nginx + + location /searx/static { + alias /usr/local/searx/searx/static; + } + + location /searx { + uwsgi_param SCRIPT_NAME /searx; + include uwsgi_params; + uwsgi_pass unix:/run/uwsgi/app/searx/socket; + } + + +**OR** using reverse proxy (Please, note that reverse proxy advised to be used +in case of single-user or low-traffic instances.) + +.. code:: nginx + + location /searx/static { + alias /usr/local/searx/searx/static; + } + + location /searx { + proxy_pass http://127.0.0.1:8888; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_set_header X-Script-Name /searx; + proxy_buffering off; + } + +Enable ``base_url`` in ``searx/settings.yml`` + +.. code:: yaml + + base_url : http://your.domain.tld/searx/ + +Restart service: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H systemctl restart nginx + sudo -H systemctl restart uwsgi + + +disable logs +============ + +For better privacy you can disable nginx logs about searx. How to proceed: +below ``uwsgi_pass`` in ``/etc/nginx/sites-available/default`` add: + +.. code:: nginx + + access_log /dev/null; + error_log /dev/null; + +Restart service: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H systemctl restart nginx diff --git a/docs/admin/installation.rst b/docs/admin/installation.rst index bff3e44d9..a901fee1c 100644 --- a/docs/admin/installation.rst +++ b/docs/admin/installation.rst @@ -4,50 +4,64 @@ Installation ============ -.. contents:: - :depth: 3 +.. sidebar:: Searx server setup + + - :ref:`installation nginx` + - :ref:`installation apache` + + If you do not have any special preferences, it is recommend to use + :ref:`searx.sh`. + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry .. _installation basic: Basic installation ================== -.. sidebar:: further reading +Step by step installation with virtualenv. For Ubuntu, be sure to have enable +universe repository. - - :ref:`searx.sh` +Install packages: -Step by step installation for Debian/Ubuntu with virtualenv. For Ubuntu, be sure -to have enable universe repository. +.. tabs:: -Install packages: + .. group-tab:: Ubuntu / debian -.. code:: sh + .. code-block:: sh - $ sudo -H apt-get install \ - git build-essential libxslt-dev \ - python-dev python-virtualenv python-babel \ - zlib1g-dev libffi-dev libssl-dev + $ sudo -H apt-get install \ + git build-essential + libxslt-dev python3-dev python3-babel \ + zlib1g-dev libffi-dev libssl-dev Install searx: .. code:: sh - cd /usr/local - sudo -H git clone https://github.com/asciimoo/searx.git - sudo -H useradd searx -d /usr/local/searx + sudo -H useradd searx --system --disabled-password -d /usr/local/searx + sudo -H usermod -a -G shadow $SERVICE_USER + cd /usr/local/searx + sudo -H git clone https://github.com/asciimoo/searx.git searx-src sudo -H chown searx:searx -R /usr/local/searx -Install dependencies in a virtualenv: +Install virtualenv: .. code:: sh - cd /usr/local/searx sudo -H -u searx -i + (searx)$ python3 -m venv searx-pyenv + (searx)$ echo 'source ~/searx-pyenv/bin/activate' > ~/.profile + +Exit the searx bash and restart a new to install the searx dependencies: .. code:: sh - (searx)$ virtualenv searx-ve - (searx)$ . ./searx-ve/bin/activate + sudo -H -u searx -i + (searx)$ cd searx-src (searx)$ ./manage.sh update_packages Configuration @@ -55,7 +69,9 @@ Configuration .. code:: sh - sed -i -e "s/ultrasecretkey/`openssl rand -hex 16`/g" searx/settings.yml + sudo -H -u searx -i + (searx)$ cd searx-src + (searx)$ sed -i -e "s/ultrasecretkey/`openssl rand -hex 16`/g" searx/settings.yml Edit searx/settings.yml if necessary. @@ -66,7 +82,9 @@ Start searx: .. code:: sh - python searx/webapp.py + sudo -H -u searx -i + (searx)$ cd searx-src + (searx)$ python3 searx/webapp.py Go to http://localhost:8888 @@ -76,254 +94,118 @@ If everything works fine, disable the debug option in settings.yml: sed -i -e "s/debug : True/debug : False/g" searx/settings.yml -At this point searx is not demonized ; uwsgi allows this. - -You can exit the virtualenv and the searx user bash (enter exit command -twice). +At this point searx is not demonized ; uwsgi allows this. You can exit the +virtualenv and the searx user bash (enter exit command twice). uwsgi ===== Install packages: -.. code:: sh - - sudo -H apt-get install \ - uwsgi uwsgi-plugin-python - -Create the configuration file ``/etc/uwsgi/apps-available/searx.ini`` with this -content: - -.. code:: ini - - [uwsgi] - # Who will run the code - uid = searx - gid = searx - - # disable logging for privacy - disable-logging = true - - # Number of workers (usually CPU count) - workers = 4 - - # The right granted on the created socket - chmod-socket = 666 +.. tabs:: - # Plugin to use and interpretor config - single-interpreter = true - master = true - plugin = python - lazy-apps = true - enable-threads = true + .. group-tab:: Ubuntu / debian - # Module to import - module = searx.webapp + .. code-block:: bash - # Support running the module from a webserver subdirectory. - route-run = fixpathinfo: + sudo -H apt-get install uwsgi uwsgi-plugin-python3 - # Virtualenv and python path - virtualenv = /usr/local/searx/searx-ve/ - pythonpath = /usr/local/searx/ - chdir = /usr/local/searx/searx/ - -Activate the uwsgi application and restart: - -.. code:: sh - - cd /etc/uwsgi/apps-enabled - ln -s ../apps-available/searx.ini - /etc/init.d/uwsgi restart - -Web server -========== - -with nginx ----------- - -If nginx is not installed (uwsgi will not work with the package -nginx-light): - -.. code:: sh - - sudo -H apt-get install nginx - -Hosted at / -~~~~~~~~~~~ - -Create the configuration file ``/etc/nginx/sites-available/searx`` with this +Create the configuration file ``/etc/uwsgi/apps-available/searx.ini`` with this content: -.. code:: nginx - - server { - listen 80; - server_name searx.example.com; - root /usr/local/searx/searx; - - location /static { - } - - location / { - include uwsgi_params; - uwsgi_pass unix:/run/uwsgi/app/searx/socket; - } - } - -Create a symlink to sites-enabled: - -.. code:: sh - - sudo -H ln -s /etc/nginx/sites-available/searx /etc/nginx/sites-enabled/searx - -Restart service: - -.. code:: sh - - sudo -H service nginx restart - sudo -H service uwsgi restart - -from subdirectory URL (/searx) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Add this configuration in the server config file -``/etc/nginx/sites-enabled/default``: - -.. code:: nginx - - location /searx/static { - alias /usr/local/searx/searx/static; - } - - location /searx { - uwsgi_param SCRIPT_NAME /searx; - include uwsgi_params; - uwsgi_pass unix:/run/uwsgi/app/searx/socket; - } - - -**OR** using reverse proxy (Please, note that reverse proxy advised to be used -in case of single-user or low-traffic instances.) - -.. code:: nginx - - location /searx/static { - alias /usr/local/searx/searx/static; - } - - location /searx { - proxy_pass http://127.0.0.1:8888; - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Scheme $scheme; - proxy_set_header X-Script-Name /searx; - proxy_buffering off; - } - - -Enable ``base_url`` in ``searx/settings.yml`` - -.. code:: yaml - - base_url : http://your.domain.tld/searx/ +.. code:: ini -Restart service: + [uwsgi] -.. code:: sh + # uWSGI core + # ---------- + # + # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#uwsgi-core - sudo -H service nginx restart - sudo -H service uwsgi restart + # Who will run the code + uid = searx + gid = searx -disable logs -^^^^^^^^^^^^ + # chdir to specified directory before apps loading + chdir = /usr/local/searx/searx-src/searx -for better privacy you can disable nginx logs about searx. + # disable logging for privacy + disable-logging = true -how to proceed: below ``uwsgi_pass`` in ``/etc/nginx/sites-available/default`` -add: + # The right granted on the created socket + chmod-socket = 666 -.. code:: nginx + # Plugin to use and interpretor config + single-interpreter = true - access_log /dev/null; - error_log /dev/null; + # enable master process + master = true -Restart service: + # load apps in each worker instead of the master + lazy-apps = true -.. code:: sh + # load uWSGI plugins + plugin = python3,http - sudo -H service nginx restart + # By default the Python plugin does not initialize the GIL. This means your + # app-generated threads will not run. If you need threads, remember to enable + # them with enable-threads. Running uWSGI in multithreading mode (with the + # threads options) will automatically enable threading support. This *strange* + # default behaviour is for performance reasons. + enable-threads = true -with apache ------------ + # plugin: python + # -------------- + # + # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-python -Add wsgi mod: + # load a WSGI module + module = searx.webapp -.. code:: sh + # set PYTHONHOME/virtualenv + virtualenv = /usr/local/searx/searx-pyenv - sudo -H apt-get install libapache2-mod-uwsgi - sudo -H a2enmod uwsgi + # add directory (or glob) to pythonpath + pythonpath = /usr/local/searx/searx-src -Add this configuration in the file ``/etc/apache2/apache2.conf``: -.. code:: apache + # plugin http + # ----------- + # + # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-http - - Options FollowSymLinks Indexes - SetHandler uwsgi-handler - uWSGISocket /run/uwsgi/app/searx/socket - + # Native HTTP support: https://uwsgi-docs.readthedocs.io/en/latest/HTTP.html + http = 127.0.0.1:8888 -Note that if your instance of searx is not at the root, you should change -```` by the location of your instance, like ````. - -Restart Apache: +Activate the uwsgi application and restart: .. code:: sh - sudo -H /etc/init.d/apache2 restart - -disable logs -~~~~~~~~~~~~ - -For better privacy you can disable Apache logs. - -.. warning:: - - You can only disable logs for the whole (virtual) server not for a specific - path. - -Go back to ``/etc/apache2/apache2.conf`` and above ```` add: - -.. code:: apache - - CustomLog /dev/null combined - -Restart Apache: - -.. code:: sh + cd /etc/uwsgi/apps-enabled + ln -s ../apps-available/searx.ini + /etc/init.d/uwsgi restart - sudo -H /etc/init.d/apache2 restart How to update ============= .. code:: sh - cd /usr/local/searx sudo -H -u searx -i - -.. code:: sh - - (searx)$ . ./searx-ve/bin/activate (searx)$ git stash (searx)$ git pull origin master (searx)$ git stash apply (searx)$ ./manage.sh update_packages -.. code:: sh +Restart uwsgi: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh - sudo -H service uwsgi restart + sudo -H systemctl restart uwsgi Docker ====== diff --git a/docs/admin/settings.rst b/docs/admin/settings.rst index 0bfdcc6cb..59a406a28 100644 --- a/docs/admin/settings.rst +++ b/docs/admin/settings.rst @@ -4,11 +4,17 @@ ``settings.yml`` ================ +This page describe the options possibilities of the :origin:`searx/settings.yml` +file. + .. sidebar:: Further reading .. - :ref:`search API` -This page describe the options possibilities of the settings.yml file. +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry .. _settings global: -- cgit v1.2.3 From 2f40f61f83afb34f0c4b95019a53050a504ce43a Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Tue, 4 Feb 2020 17:59:58 +0100 Subject: /etc/filtron/rules.json: normalize rules from docs & tooling box Signed-off-by: Markus Heiser --- docs/admin/filtron.rst | 192 +++++++++++++++++++++++++------------------------ 1 file changed, 98 insertions(+), 94 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/filtron.rst b/docs/admin/filtron.rst index 205269973..2bc663411 100644 --- a/docs/admin/filtron.rst +++ b/docs/admin/filtron.rst @@ -39,6 +39,10 @@ an installation and the maintenance of, use our script :ref:`filtron.sh`. Sample configuration of filtron =============================== +.. sidebar:: Tooling box + + - :origin:`/etc/filtron/rules.json ` + An example configuration can be find below. This configuration limits the access of: @@ -50,100 +54,100 @@ of: .. code:: json - [{ - "name":"search request", - "filters":[ - "Param:q", - "Path=^(/|/search)$" - ], - "interval":"", - "limit":"", - "subrules":[ - { - "name":"roboagent limit", - "interval":"", - "limit":"", - "filters":[ - "Header:User-Agent=(curl|cURL|Wget|python-requests|Scrapy|FeedFetcher|Go-http-client)" - ], - "actions":[ - { - "name":"block", - "params":{ - "message":"Rate limit exceeded" - } - } - ] - }, - { - "name":"botlimit", - "limit":0, - "stop":true, - "filters":[ - "Header:User-Agent=(Googlebot|bingbot|Baiduspider|yacybot|YandexMobileBot|YandexBot|Yahoo! Slurp|MJ12bot|AhrefsBot|archive.org_bot|msnbot|MJ12bot|SeznamBot|linkdexbot|Netvibes|SMTBot|zgrab|James BOT)" - ], - "actions":[ - { - "name":"block", - "params":{ - "message":"Rate limit exceeded" - } - } - ] - }, - { - "name":"IP limit", - "interval":"", - "limit":"", - "stop":true, - "aggregations":[ - "Header:X-Forwarded-For" - ], - "actions":[ - { - "name":"block", - "params":{ - "message":"Rate limit exceeded" - } - } - ] - }, - { - "name":"rss/json limit", - "interval":"", - "limit":"", - "stop":true, - "filters":[ - "Param:format=(csv|json|rss)" - ], - "actions":[ - { - "name":"block", - "params":{ - "message":"Rate limit exceeded" - } - } - ] - }, - { - "name":"useragent limit", - "interval":"", - "limit":"", - "aggregations":[ - "Header:User-Agent" - ], - "actions":[ - { - "name":"block", - "params":{ - "message":"Rate limit exceeded" - } - } - ] - } - ] - }] - + [ + { "name": "search request", + "filters": [ + "Param:q", + "Path=^(/|/search)$" + ], + "interval": "", + "limit": "", + "subrules": [ + { + "name": "roboagent limit", + "interval": "", + "limit": "", + "filters": [ + "Header:User-Agent=(curl|cURL|Wget|python-requests|Scrapy|FeedFetcher|Go-http-client)" + ], + "actions": [ + { "name": "log"}, + { "name": "block", + "params": { + "message": "Rate limit exceeded" + } + } + ] + }, + { + "name": "botlimit", + "limit": 0, + "stop": true, + "filters": [ + "Header:User-Agent=(Googlebot|bingbot|Baiduspider|yacybot|YandexMobileBot|YandexBot|Yahoo! Slurp|MJ12bot|AhrefsBot|archive.org_bot|msnbot|MJ12bot|SeznamBot|linkdexbot|Netvibes|SMTBot|zgrab|James BOT)" + ], + "actions": [ + { "name": "log"}, + { "name": "block", + "params": { + "message": "Rate limit exceeded" + } + } + ] + }, + { + "name": "IP limit", + "interval": "", + "limit": "", + "stop": true, + "aggregations": [ + "Header:X-Forwarded-For" + ], + "actions": [ + { "name": "log"}, + { "name": "block", + "params": { + "message": "Rate limit exceeded" + } + } + ] + }, + { + "name": "rss/json limit", + "interval": "", + "limit": "", + "stop": true, + "filters": [ + "Param:format=(csv|json|rss)" + ], + "actions": [ + { "name": "log"}, + { "name": "block", + "params": { + "message": "Rate limit exceeded" + } + } + ] + }, + { + "name": "useragent limit", + "interval": "", + "limit": "", + "aggregations": [ + "Header:User-Agent" + ], + "actions": [ + { "name": "log"}, + { "name": "block", + "params": { + "message": "Rate limit exceeded" + } + } + ] + } + ] + } + ] Route request through filtron -- cgit v1.2.3 From ed4cb4f1603dc519aa42cc626874f23d2e9db84e Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sat, 8 Feb 2020 13:24:08 +0100 Subject: tooling box: varius fix from tests --- docs/admin/installation.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/admin') diff --git a/docs/admin/installation.rst b/docs/admin/installation.rst index a901fee1c..67358e996 100644 --- a/docs/admin/installation.rst +++ b/docs/admin/installation.rst @@ -43,7 +43,7 @@ Install searx: .. code:: sh sudo -H useradd searx --system --disabled-password -d /usr/local/searx - sudo -H usermod -a -G shadow $SERVICE_USER + sudo -H usermod -a -G shadow searx cd /usr/local/searx sudo -H git clone https://github.com/asciimoo/searx.git searx-src sudo -H chown searx:searx -R /usr/local/searx -- cgit v1.2.3 From 0bfc61dbe9a7b54b1053f9c92f9ff04d8ff7e1bc Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sun, 16 Feb 2020 22:29:06 +0100 Subject: tooling box: misc fixes from lxc tests Tested by: sudo -H ./utils/lxc.sh build containers sudo -H ./utils/lxc.sh cmd /share/searx/utils/searx.sh install all sudo -H ./utils/lxc.sh cmd /share/searx/utils/filtron.sh install all sudo -H ./utils/lxc.sh cmd /share/searx/utils/morty.sh install all Signed-off-by: Markus Heiser --- docs/admin/installation.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/installation.rst b/docs/admin/installation.rst index 67358e996..a4c7eb8b3 100644 --- a/docs/admin/installation.rst +++ b/docs/admin/installation.rst @@ -33,9 +33,9 @@ Install packages: .. code-block:: sh - $ sudo -H apt-get install \ + $ sudo -H apt-get install -m \ git build-essential - libxslt-dev python3-dev python3-babel \ + libxslt-dev python3-dev python3-babel python3-venv \ zlib1g-dev libffi-dev libssl-dev Install searx: -- cgit v1.2.3 From c3e4753ce951f759844db13d98f9ad5b226f84b6 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Mon, 2 Mar 2020 19:00:19 +0100 Subject: docs: generic documentation from the installation scripts Signed-off-by: Markus Heiser --- docs/admin/index.rst | 4 + docs/admin/installation-docker.rst | 28 +++++ docs/admin/installation-searx.rst | 85 +++++++++++++ docs/admin/installation-uwsgi.rst | 82 ++++++++++++ docs/admin/installation.rst | 247 ++++++------------------------------- docs/admin/update-searx.rst | 23 ++++ 6 files changed, 260 insertions(+), 209 deletions(-) create mode 100644 docs/admin/installation-docker.rst create mode 100644 docs/admin/installation-searx.rst create mode 100644 docs/admin/installation-uwsgi.rst create mode 100644 docs/admin/update-searx.rst (limited to 'docs/admin') diff --git a/docs/admin/index.rst b/docs/admin/index.rst index 4190bf24b..c708c4ffa 100644 --- a/docs/admin/index.rst +++ b/docs/admin/index.rst @@ -7,8 +7,12 @@ Administrator documentation :caption: Contents installation + installation-searx + installation-uwsgi installation-nginx installation-apache + installation-docker + update-searx settings api architecture diff --git a/docs/admin/installation-docker.rst b/docs/admin/installation-docker.rst new file mode 100644 index 000000000..340e66319 --- /dev/null +++ b/docs/admin/installation-docker.rst @@ -0,0 +1,28 @@ +.. _installation docker: + +=================== +Docker installation +=================== + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +Make sure you have installed Docker. For instance, you can deploy searx like this: + +.. code:: sh + + docker pull wonderfall/searx + docker run -d --name searx -p $PORT:8888 wonderfall/searx + +Go to ``http://localhost:$PORT``. + +See https://hub.docker.com/r/wonderfall/searx/ for more informations. It's also +possible to build searx from the embedded Dockerfile. + +.. code:: sh + + git clone https://github.com/asciimoo/searx.git + cd searx + docker build -t whatever/searx . diff --git a/docs/admin/installation-searx.rst b/docs/admin/installation-searx.rst new file mode 100644 index 000000000..11018bb3b --- /dev/null +++ b/docs/admin/installation-searx.rst @@ -0,0 +1,85 @@ +.. _installation basic: + +========================= +Step by step installation +========================= + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + +Step by step installation with virtualenv. For Ubuntu, be sure to have enable +universe repository. + +Install packages +================ + +.. include:: ../../build/docs/includes/searx.rst + :start-after: START distro-packages + :end-before: END distro-packages + +.. hint:: + + This installs also the packages needed by :ref:`searx uwsgi` + +Create user +=========== + +.. include:: ../../build/docs/includes/searx.rst + :start-after: START create user + :end-before: END create user + +Start a interactive shell from new created user and clone searx: + +.. include:: ../../build/docs/includes/searx.rst + :start-after: START clone searx + :end-before: END clone searx + +In the same shell create *virtualenv*: + +.. include:: ../../build/docs/includes/searx.rst + :start-after: START create virtualenv + :end-before: END create virtualenv + +To install searx's dependencies, exit the searx *bash* session you opened above +and restart a new. Before install, first check if your *virualenv* was sourced +from the login (*~/.profile*): + +.. include:: ../../build/docs/includes/searx.rst + :start-after: START manage.sh update_packages + :end-before: END manage.sh update_packages + +.. tip:: + + Open a second terminal for the configuration tasks and left the ``(searx)$`` + terminal open for the tasks below. + + +Configuration +============== + +Create a copy of the :origin:`searx/settings.yml` configuration file in system's +*/etc* folder. Configure like shown below -- replace ``searx@\$(uname -n)`` with +a name of your choice -- *and/or* edit ``/etc/searx/settings.yml`` if necessary. + +.. include:: ../../build/docs/includes/searx.rst + :start-after: START searx config + :end-before: END searx config + + +Check +===== + +To check your searx setup, optional enable debugging and start the *webapp*. +Searx looks at the exported environment ``$SEARX_SETTINGS_PATH`` for a +configuration file. + +.. include:: ../../build/docs/includes/searx.rst + :start-after: START check searx installation + :end-before: END check searx installation + +If everything works fine, hit ``[CTRL-C]`` to stop the *webapp* and disable the +debug option in ``settings.yml``. You can now exit searx user bash (enter exit +command twice). At this point searx is not demonized; uwsgi allows this. + diff --git a/docs/admin/installation-uwsgi.rst b/docs/admin/installation-uwsgi.rst new file mode 100644 index 000000000..0ee4800ad --- /dev/null +++ b/docs/admin/installation-uwsgi.rst @@ -0,0 +1,82 @@ +.. _searx uwsgi: + +===== +uwsgi +===== + +Create the configuration file ``/etc/uwsgi/apps-available/searx.ini`` with this +content: + +.. code:: ini + + [uwsgi] + + # uWSGI core + # ---------- + # + # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#uwsgi-core + + # Who will run the code + uid = searx + gid = searx + + # chdir to specified directory before apps loading + chdir = /usr/local/searx/searx-src/searx + + # disable logging for privacy + disable-logging = true + + # The right granted on the created socket + chmod-socket = 666 + + # Plugin to use and interpretor config + single-interpreter = true + + # enable master process + master = true + + # load apps in each worker instead of the master + lazy-apps = true + + # load uWSGI plugins + plugin = python3,http + + # By default the Python plugin does not initialize the GIL. This means your + # app-generated threads will not run. If you need threads, remember to enable + # them with enable-threads. Running uWSGI in multithreading mode (with the + # threads options) will automatically enable threading support. This *strange* + # default behaviour is for performance reasons. + enable-threads = true + + # plugin: python + # -------------- + # + # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-python + + # load a WSGI module + module = searx.webapp + + # set PYTHONHOME/virtualenv + virtualenv = /usr/local/searx/searx-pyenv + + # add directory (or glob) to pythonpath + pythonpath = /usr/local/searx/searx-src + + + # plugin http + # ----------- + # + # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-http + + # Native HTTP support: https://uwsgi-docs.readthedocs.io/en/latest/HTTP.html + http = 127.0.0.1:8888 + +Activate the uwsgi application and restart: + +.. code:: sh + + cd /etc/uwsgi/apps-enabled + ln -s ../apps-available/searx.ini + /etc/init.d/uwsgi restart + + diff --git a/docs/admin/installation.rst b/docs/admin/installation.rst index a4c7eb8b3..330a51fd1 100644 --- a/docs/admin/installation.rst +++ b/docs/admin/installation.rst @@ -4,234 +4,63 @@ Installation ============ -.. sidebar:: Searx server setup +*You're spoilt for choice*, choose your preferred method of installation. - - :ref:`installation nginx` - - :ref:`installation apache` +- :ref:`installation docker` +- `Installation scripts`_ +- :ref:`installation basic` - If you do not have any special preferences, it is recommend to use - :ref:`searx.sh`. +The :ref:`installation basic` is good enough for intranet usage and it is a +excellent illustration of *how a searx instance is build up*. If you place your +instance public to the internet you should really consider to install a +:ref:`filtron reverse proxy ` and for privacy a :ref:`result proxy +` is mandatory. -.. contents:: Contents - :depth: 2 - :local: - :backlinks: entry +Therefore, if you do not have any special preferences, its recommend to use the +:ref:`installation docker` or the `Installation scripts`_ from our :ref:`tooling +box ` as described below. -.. _installation basic: -Basic installation -================== +Installation scripts +==================== -Step by step installation with virtualenv. For Ubuntu, be sure to have enable -universe repository. +The following will install a setup as shown in :ref:`architecture`. First you +need to get a clone. The clone is only needed for the installation procedure +and some maintenance tasks (alternatively you can create your own fork). -Install packages: +.. code:: bash -.. tabs:: + $ cd ~/Download + $ git clone https://github.com/asciimoo/searx searx + $ cd searx - .. group-tab:: Ubuntu / debian +.. hint:: - .. code-block:: sh + The *tooling box* is not yet merged into `asciimoo/searx master + `_. As long as PR is not merged, you need + to merge the PR into your local clone (see below). The discussion takes + place in :pull:`1803`. To merge the :pull:`1803` in your local branch use: - $ sudo -H apt-get install -m \ - git build-essential - libxslt-dev python3-dev python3-babel python3-venv \ - zlib1g-dev libffi-dev libssl-dev + .. code:: bash -Install searx: + $ git pull origin refs/pull/1803/head -.. code:: sh +**Install** :ref:`searx service ` - sudo -H useradd searx --system --disabled-password -d /usr/local/searx - sudo -H usermod -a -G shadow searx - cd /usr/local/searx - sudo -H git clone https://github.com/asciimoo/searx.git searx-src - sudo -H chown searx:searx -R /usr/local/searx +This installs searx as described in :ref:`installation basic`. -Install virtualenv: +.. code:: bash -.. code:: sh + $ sudo -H ./utils/searx.sh install all - sudo -H -u searx -i - (searx)$ python3 -m venv searx-pyenv - (searx)$ echo 'source ~/searx-pyenv/bin/activate' > ~/.profile +**Install** :ref:`filtron reverse proxy ` -Exit the searx bash and restart a new to install the searx dependencies: +.. code:: bash -.. code:: sh + $ sudo -H ./utils/filtron.sh install all - sudo -H -u searx -i - (searx)$ cd searx-src - (searx)$ ./manage.sh update_packages +**Install** :ref:`result proxy ` -Configuration -============== +.. code:: bash -.. code:: sh - - sudo -H -u searx -i - (searx)$ cd searx-src - (searx)$ sed -i -e "s/ultrasecretkey/`openssl rand -hex 16`/g" searx/settings.yml - -Edit searx/settings.yml if necessary. - -Check -===== - -Start searx: - -.. code:: sh - - sudo -H -u searx -i - (searx)$ cd searx-src - (searx)$ python3 searx/webapp.py - -Go to http://localhost:8888 - -If everything works fine, disable the debug option in settings.yml: - -.. code:: sh - - sed -i -e "s/debug : True/debug : False/g" searx/settings.yml - -At this point searx is not demonized ; uwsgi allows this. You can exit the -virtualenv and the searx user bash (enter exit command twice). - -uwsgi -===== - -Install packages: - -.. tabs:: - - .. group-tab:: Ubuntu / debian - - .. code-block:: bash - - sudo -H apt-get install uwsgi uwsgi-plugin-python3 - -Create the configuration file ``/etc/uwsgi/apps-available/searx.ini`` with this -content: - -.. code:: ini - - [uwsgi] - - # uWSGI core - # ---------- - # - # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#uwsgi-core - - # Who will run the code - uid = searx - gid = searx - - # chdir to specified directory before apps loading - chdir = /usr/local/searx/searx-src/searx - - # disable logging for privacy - disable-logging = true - - # The right granted on the created socket - chmod-socket = 666 - - # Plugin to use and interpretor config - single-interpreter = true - - # enable master process - master = true - - # load apps in each worker instead of the master - lazy-apps = true - - # load uWSGI plugins - plugin = python3,http - - # By default the Python plugin does not initialize the GIL. This means your - # app-generated threads will not run. If you need threads, remember to enable - # them with enable-threads. Running uWSGI in multithreading mode (with the - # threads options) will automatically enable threading support. This *strange* - # default behaviour is for performance reasons. - enable-threads = true - - # plugin: python - # -------------- - # - # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-python - - # load a WSGI module - module = searx.webapp - - # set PYTHONHOME/virtualenv - virtualenv = /usr/local/searx/searx-pyenv - - # add directory (or glob) to pythonpath - pythonpath = /usr/local/searx/searx-src - - - # plugin http - # ----------- - # - # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-http - - # Native HTTP support: https://uwsgi-docs.readthedocs.io/en/latest/HTTP.html - http = 127.0.0.1:8888 - -Activate the uwsgi application and restart: - -.. code:: sh - - cd /etc/uwsgi/apps-enabled - ln -s ../apps-available/searx.ini - /etc/init.d/uwsgi restart - - -How to update -============= - -.. code:: sh - - sudo -H -u searx -i - (searx)$ git stash - (searx)$ git pull origin master - (searx)$ git stash apply - (searx)$ ./manage.sh update_packages - -Restart uwsgi: - -.. tabs:: - - .. group-tab:: Ubuntu / debian - - .. code:: sh - - sudo -H systemctl restart uwsgi - -Docker -====== - -Make sure you have installed Docker. For instance, you can deploy searx like this: - -.. code:: sh - - docker pull wonderfall/searx - docker run -d --name searx -p $PORT:8888 wonderfall/searx - -Go to ``http://localhost:$PORT``. - -See https://hub.docker.com/r/wonderfall/searx/ for more informations. It's also -possible to build searx from the embedded Dockerfile. - -.. code:: sh - - git clone https://github.com/asciimoo/searx.git - cd searx - docker build -t whatever/searx . - -References -========== - -* https://about.okhin.fr/posts/Searx/ with some additions - -* How to: `Setup searx in a couple of hours with a free SSL certificate - `__ + $ sudo -H ./utils/morty.sh install all diff --git a/docs/admin/update-searx.rst b/docs/admin/update-searx.rst new file mode 100644 index 000000000..c74c9568a --- /dev/null +++ b/docs/admin/update-searx.rst @@ -0,0 +1,23 @@ +.. _update searx: + +============= +How to update +============= + +.. code:: sh + + sudo -H -u searx -i + (searx)$ git stash + (searx)$ git pull origin master + (searx)$ git stash apply + (searx)$ ./manage.sh update_packages + +Restart uwsgi: + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H systemctl restart uwsgi -- cgit v1.2.3 From 3cb7daedb7c5c93a98dfefa191c161f31a1347c6 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Tue, 3 Mar 2020 19:57:15 +0100 Subject: docs: generic doocumentation for searx-uwsgi Signed-off-by: Markus Heiser --- docs/admin/installation-searx.rst | 5 ++- docs/admin/installation-uwsgi.rst | 88 ++++++++++++++------------------------- 2 files changed, 34 insertions(+), 59 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/installation-searx.rst b/docs/admin/installation-searx.rst index 11018bb3b..76449beb6 100644 --- a/docs/admin/installation-searx.rst +++ b/docs/admin/installation-searx.rst @@ -30,6 +30,9 @@ Create user :start-after: START create user :end-before: END create user +install searx & dependencies +============================ + Start a interactive shell from new created user and clone searx: .. include:: ../../build/docs/includes/searx.rst @@ -55,7 +58,6 @@ from the login (*~/.profile*): Open a second terminal for the configuration tasks and left the ``(searx)$`` terminal open for the tasks below. - Configuration ============== @@ -67,7 +69,6 @@ a name of your choice -- *and/or* edit ``/etc/searx/settings.yml`` if necessary. :start-after: START searx config :end-before: END searx config - Check ===== diff --git a/docs/admin/installation-uwsgi.rst b/docs/admin/installation-uwsgi.rst index 0ee4800ad..f9161679f 100644 --- a/docs/admin/installation-uwsgi.rst +++ b/docs/admin/installation-uwsgi.rst @@ -4,79 +4,53 @@ uwsgi ===== -Create the configuration file ``/etc/uwsgi/apps-available/searx.ini`` with this -content: +Create the configuration ini-file according to your distribution (see below) and +restart the uwsgi application. -.. code:: ini +.. tabs:: - [uwsgi] + .. group-tab:: Ubuntu / debian - # uWSGI core - # ---------- - # - # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#uwsgi-core + .. literalinclude:: ../../build/docs/includes/searx.rst + :start-after: START searx uwsgi-description ubuntu-20.04 + :end-before: END searx uwsgi-description ubuntu-20.04 - # Who will run the code - uid = searx - gid = searx - # chdir to specified directory before apps loading - chdir = /usr/local/searx/searx-src/searx + .. group-tab:: Arch Linux - # disable logging for privacy - disable-logging = true + .. literalinclude:: ../../build/docs/includes/searx.rst + :start-after: START searx uwsgi-description arch + :end-before: END searx uwsgi-description arch - # The right granted on the created socket - chmod-socket = 666 - # Plugin to use and interpretor config - single-interpreter = true + .. group-tab:: Fedora / RHEL - # enable master process - master = true + .. literalinclude:: ../../build/docs/includes/searx.rst + :start-after: START searx uwsgi-description fedora + :end-before: END searx uwsgi-description fedora - # load apps in each worker instead of the master - lazy-apps = true - # load uWSGI plugins - plugin = python3,http +.. tabs:: - # By default the Python plugin does not initialize the GIL. This means your - # app-generated threads will not run. If you need threads, remember to enable - # them with enable-threads. Running uWSGI in multithreading mode (with the - # threads options) will automatically enable threading support. This *strange* - # default behaviour is for performance reasons. - enable-threads = true + .. group-tab:: Ubuntu / debian - # plugin: python - # -------------- - # - # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-python + .. literalinclude:: ../../build/docs/includes/searx.rst + :language: ini + :start-after: START searx uwsgi-appini ubuntu-20.04 + :end-before: END searx uwsgi-appini ubuntu-20.04 - # load a WSGI module - module = searx.webapp + .. group-tab:: Arch Linux - # set PYTHONHOME/virtualenv - virtualenv = /usr/local/searx/searx-pyenv + .. literalinclude:: ../../build/docs/includes/searx.rst + :language: ini + :start-after: START searx uwsgi-appini arch + :end-before: END searx uwsgi-appini arch - # add directory (or glob) to pythonpath - pythonpath = /usr/local/searx/searx-src + .. group-tab:: Fedora / RHEL - - # plugin http - # ----------- - # - # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-http - - # Native HTTP support: https://uwsgi-docs.readthedocs.io/en/latest/HTTP.html - http = 127.0.0.1:8888 - -Activate the uwsgi application and restart: - -.. code:: sh - - cd /etc/uwsgi/apps-enabled - ln -s ../apps-available/searx.ini - /etc/init.d/uwsgi restart + .. literalinclude:: ../../build/docs/includes/searx.rst + :language: ini + :start-after: START searx uwsgi-appini fedora + :end-before: END searx uwsgi-appini fedora -- cgit v1.2.3 From 3aff2c19d16619d949dc52216b29ef637d1a44a1 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Wed, 4 Mar 2020 12:22:20 +0100 Subject: docs: complement uwsgi documentation for all distros Signed-off-by: Markus Heiser --- docs/admin/installation-searx.rst | 2 + docs/admin/installation-uwsgi.rst | 91 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 93 insertions(+) (limited to 'docs/admin') diff --git a/docs/admin/installation-searx.rst b/docs/admin/installation-searx.rst index 76449beb6..88562f3e3 100644 --- a/docs/admin/installation-searx.rst +++ b/docs/admin/installation-searx.rst @@ -12,6 +12,8 @@ Step by step installation Step by step installation with virtualenv. For Ubuntu, be sure to have enable universe repository. +.. _install packages: + Install packages ================ diff --git a/docs/admin/installation-uwsgi.rst b/docs/admin/installation-uwsgi.rst index f9161679f..7996bf937 100644 --- a/docs/admin/installation-uwsgi.rst +++ b/docs/admin/installation-uwsgi.rst @@ -4,6 +4,97 @@ uwsgi ===== +.. sidebar:: further read + + - `systemd.unit`_ + - `uWSGI Emperor`_ + +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +.. _systemd.unit: https://www.freedesktop.org/software/systemd/man/systemd.unit.html +.. _One service per app in systemd: + https://uwsgi-docs.readthedocs.io/en/latest/Systemd.html#one-service-per-app-in-systemd +.. _uWSGI Emperor: + https://uwsgi-docs.readthedocs.io/en/latest/Emperor.html +.. _uwsgi ini file: + https://uwsgi-docs.readthedocs.io/en/latest/Configuration.html#ini-files +.. _systemd unit template: + http://0pointer.de/blog/projects/instances.html + + +Origin uWSGI +============ + +How uWSGI is implemented by distributors is different. uWSGI itself +recommend two methods + +`systemd.unit`_ template files as described here `One service per app in systemd`_. + + There is one `systemd unit template`_ and one `uwsgi ini file`_ per uWSGI-app + placed at dedicated locations. Take archlinux and a searx.ini as example:: + + unit template --> /usr/lib/systemd/system/uwsgi@.service + uwsgi ini files --> /etc/uwsgi/searx.ini + + The searx app can be maintained as know from common systemd units:: + + systemctl enable uwsgi@searx + systemctl start uwsgi@searx + systemctl restart uwsgi@searx + systemctl stop uwsgi@searx + +The `uWSGI Emperor`_ mode which fits for maintaining a large range of uwsgi apps. + + The Emperor mode is a special uWSGI instance that will monitor specific + events. The Emperor mode (service) is started by a (common, not template) + systemd unit. The Emperor service will scan specific directories for `uwsgi + ini file`_\s (also know as *vassals*). If a *vassal* is added, removed or the + timestamp is modified, a corresponding action takes place: a new uWSGI + instance is started, reload or stopped. Take Fedora and a searx.ini as + example:: + + to start a new searx instance create --> /etc/uwsgi.d/searx.ini + to reload the instance edit timestamp --> touch /etc/uwsgi.d/searx.ini + to stop instance remove ini --> rm /etc/uwsgi.d/searx.ini + +Distributors +============ + +The `uWSGI Emperor`_ mode and `systemd unit template`_ is what the distributors +mostly offer their users, even if they differ in the way they implement both +modes and their defaults. Another point they might differ is the packaging of +plugins (if so, compare :ref:`install packages`) and what the default python +interpreter is (python2 vs. python3). + +Fedora starts a Emperor by default, while archlinux does not start any uwsgi +service by default. Worth to know; debian (ubuntu) follow a complete different +approach. *debian*: your are familiar with the apache infrastructure? .. they +do similar for the uWSGI infrastructure (with less comfort), the folders are:: + + /etc/uwsgi/apps-available/ + /etc/uwsgi/apps-enabled/ + +The `uwsgi ini file`_ is enabled by a symbolic link:: + + ln -s /etc/uwsgi/apps-available/searx.ini /etc/uwsgi/apps-enabled/ + +From debian's documentation (``/usr/share/doc/uwsgi/README.Debian.gz``): You +could control specific instance(s) by issuing:: + + service uwsgi ... + + sudo -H service uwsgi start searx + sudo -H service uwsgi stop searx + +My experience is, that this command is a bit buggy. + +Alltogether +=========== + Create the configuration ini-file according to your distribution (see below) and restart the uwsgi application. -- cgit v1.2.3 From cbc08fdc26e96bf2cb02b76a30be095f5f60df9f Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Wed, 4 Mar 2020 19:56:40 +0100 Subject: docs: describe uwsgi setup of all suported distributions Signed-off-by: Markus Heiser --- docs/admin/installation-apache.rst | 7 +- docs/admin/installation-nginx.rst | 277 +++++++++++++++++++++++++++---------- docs/admin/installation.rst | 3 +- 3 files changed, 213 insertions(+), 74 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/installation-apache.rst b/docs/admin/installation-apache.rst index 7f92e47d5..8c6228380 100644 --- a/docs/admin/installation-apache.rst +++ b/docs/admin/installation-apache.rst @@ -4,10 +4,11 @@ Install with apache =================== -.. sidebar:: public HTTP servers +.. sidebar:: public to the internet? - On public searx instances use an application firewall (:ref:`filtron - `). + If your searx instance is public, stop here and first install :ref:`filtron + reverse proxy ` and :ref:`result proxy morty `, see + :ref:`installation scripts`. .. contents:: Contents :depth: 2 diff --git a/docs/admin/installation-nginx.rst b/docs/admin/installation-nginx.rst index 37d3e7532..6cd815d1d 100644 --- a/docs/admin/installation-nginx.rst +++ b/docs/admin/installation-nginx.rst @@ -4,18 +4,27 @@ Install with nginx ================== -.. sidebar:: public HTTP servers - - On public searx instances use an application firewall (:ref:`filtron - `). +.. _nginx: + https://docs.nginx.com/nginx/admin-guide/ +.. _nginx server configuration: + https://docs.nginx.com/nginx/admin-guide/web-server/web-server/#setting-up-virtual-servers +.. _nginx beginners guide: + http://nginx.org/en/docs/beginners_guide.html +.. _Getting Started wiki: + https://www.nginx.com/resources/wiki/start/ .. contents:: Contents :depth: 2 :local: :backlinks: entry -If nginx is not installed (uwsgi will not work with the package -nginx-light): + +The nginx HTTP server +===================== + +If nginx_ is not installed (uwsgi will not work with the package nginx-light) +install it now. + .. tabs:: @@ -25,87 +34,208 @@ nginx-light): sudo -H apt-get install nginx -Hosted at ``/`` -=============== + .. group-tab:: Arch Linux -Create the configuration file ``/etc/nginx/sites-available/searx`` with this -content: + .. code-block:: sh -.. code:: nginx + sudo -H pacman -S nginx-mainline + sudo -H systemctl enable nginx + sudo -H systemctl start nginx - server { - listen 80; - server_name searx.example.com; - root /usr/local/searx/searx; + .. group-tab:: Fedora / RHEL - location /static { - } + .. code-block:: sh - location / { - include uwsgi_params; - uwsgi_pass unix:/run/uwsgi/app/searx/socket; - } - } + sudo -H dnf install nginx + sudo -H systemctl enable nginx + sudo -H systemctl start nginx -Create a symlink to sites-enabled: +Now at http://localhost you should see a *Welcome to nginx!* page, on Fedora you +see a *Fedora Webserver - Test Page*. The test page comes from the default +`nginx server configuration`_: -.. code:: sh +.. tabs:: - sudo -H ln -s /etc/nginx/sites-available/searx /etc/nginx/sites-enabled/searx + .. group-tab:: Ubuntu / debian -Restart service: + .. code:: sh + + less /etc/nginx/nginx.conf + + there is a line including site configurations from: + + .. code:: nginx + + include /etc/nginx/sites-enabled/*; + + .. group-tab:: Arch Linux + + .. code-block:: sh + + less /etc/nginx/nginx.conf + + in there is a configuration section named ``server``: + + .. code-block:: nginx + + server { + listen 80; + server_name localhost; + # ... + } + + .. group-tab:: Fedora / RHEL + + .. code-block:: sh + + less /etc/nginx/nginx.conf + + there is a line including site configurations from: + + .. code:: nginx + + include /etc/nginx/conf.d/*.conf; + +.. _nginx searx site: + +A searx site +============ + +.. sidebar:: public to the internet? + + If your searx instance is public, stop here and first install :ref:`filtron + reverse proxy ` and :ref:`result proxy morty `, see + :ref:`installation scripts`. + +Now you have to create a configuration for the searx site. If nginx_ is new to +you, the `nginx beginners guide`_ is a good starting point and the `Getting +Started wiki`_ is always a good resource *to keep in the pocket*. .. tabs:: .. group-tab:: Ubuntu / debian + Create configuration at ``/etc/nginx/sites-available/searx`` and place a + symlink to sites-enabled: + .. code:: sh - sudo -H systemctl restart nginx - sudo -H systemctl restart uwsgi + sudo -H ln -s /etc/nginx/sites-available/searx /etc/nginx/sites-enabled/searx -from subdirectory URL (``/searx``) -================================== + .. group-tab:: Arch Linux -Add this configuration in the server config file -``/etc/nginx/sites-enabled/default``: + In the ``/etc/nginx/nginx.conf`` file, replace the configuration section + named ``server``. -.. code:: nginx + .. group-tab:: Fedora / RHEL - location /searx/static { - alias /usr/local/searx/searx/static; - } + Create configuration at ``/etc/nginx/conf.d/searx`` and place a + symlink to sites-enabled: - location /searx { - uwsgi_param SCRIPT_NAME /searx; - include uwsgi_params; - uwsgi_pass unix:/run/uwsgi/app/searx/socket; - } +.. tabs:: -**OR** using reverse proxy (Please, note that reverse proxy advised to be used -in case of single-user or low-traffic instances.) + .. group-tab:: filtron at ``/`` & ``/morty`` -.. code:: nginx + Use this setup, if your instance is public to the internet: - location /searx/static { - alias /usr/local/searx/searx/static; - } + .. code:: nginx - location /searx { - proxy_pass http://127.0.0.1:8888; - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Scheme $scheme; - proxy_set_header X-Script-Name /searx; - proxy_buffering off; - } + location / { + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_pass http://127.0.0.1:4004/; + } -Enable ``base_url`` in ``searx/settings.yml`` + .. code:: nginx -.. code:: yaml + location /morty { + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_pass http://127.0.0.1:3000/; + } + + For a fully result proxification add :ref:`morty's ` public + URL to your :origin:`searx/settings.yml`: + + .. code:: yaml + + result_proxy: + # replace searx.example.com with your server's public name + url : http://searx.example.com/ + + + .. group-tab:: searx at ``/`` + + Use this setup only, if your instance is **NOT** public to the internet: + + .. code:: nginx + + server { + listen 80; + listen [::]:80; + + # replace searx.example.com with your server's public name + server_name searx.example.com; + + root /usr/local/searx/searx; + + location /static { + } + + location / { + include uwsgi_params; + uwsgi_pass unix:/run/uwsgi/app/searx/socket; + } + } + + .. group-tab:: searx at ``/searx`` + + Use this setup only, if your instance is **NOT** public to the internet: + + .. code:: nginx + + location /searx/static { + alias /usr/local/searx/searx/static; + } + + location /searx { + uwsgi_param SCRIPT_NAME /searx; + include uwsgi_params; + uwsgi_pass unix:/run/uwsgi/app/searx/socket; + } + + + **OR** using reverse proxy. Please, note that reverse proxy advised to be + used in case of single-user or low-traffic instances. + + .. code:: nginx + + location /searx/static { + alias /usr/local/searx/searx/static; + } + + location /searx { + proxy_pass http://127.0.0.1:8888; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_set_header X-Script-Name /searx; + proxy_buffering off; + } + + Enable ``base_url`` in :origin:`searx/settings.yml` + + .. code:: yaml + + server: + # replace searx.example.com with your server's public name + base_url : http://searx.example.com/searx/ - base_url : http://your.domain.tld/searx/ Restart service: @@ -118,24 +248,31 @@ Restart service: sudo -H systemctl restart nginx sudo -H systemctl restart uwsgi + .. group-tab:: Arch Linux -disable logs -============ + .. code:: sh -For better privacy you can disable nginx logs about searx. How to proceed: -below ``uwsgi_pass`` in ``/etc/nginx/sites-available/default`` add: + sudo -H systemctl restart nginx + sudo -H systemctl restart uwsgi -.. code:: nginx + .. group-tab:: Fedora - access_log /dev/null; - error_log /dev/null; + .. code:: sh -Restart service: + sudo -H systemctl restart nginx + sudo -H systemctl restart uwsgi -.. tabs:: - .. group-tab:: Ubuntu / debian +Disable logs +============ - .. code:: sh +For better privacy you can disable nginx logs in ``/etc/nginx/nginx.conf``. - sudo -H systemctl restart nginx +.. code:: nginx + + http { + # ... + access_log /dev/null; + error_log /dev/null; + # ... + } diff --git a/docs/admin/installation.rst b/docs/admin/installation.rst index 330a51fd1..25d7f0ea9 100644 --- a/docs/admin/installation.rst +++ b/docs/admin/installation.rst @@ -7,7 +7,7 @@ Installation *You're spoilt for choice*, choose your preferred method of installation. - :ref:`installation docker` -- `Installation scripts`_ +- :ref:`installation scripts` - :ref:`installation basic` The :ref:`installation basic` is good enough for intranet usage and it is a @@ -20,6 +20,7 @@ Therefore, if you do not have any special preferences, its recommend to use the :ref:`installation docker` or the `Installation scripts`_ from our :ref:`tooling box ` as described below. +.. _installation scripts: Installation scripts ==================== -- cgit v1.2.3 From 387c6a77691fec514704bdf178b9ab94ad4abb40 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Fri, 6 Mar 2020 14:47:00 +0100 Subject: docs: improve description of uwsgi & ngingx setup Signed-off-by: Markus Heiser --- docs/admin/filtron.rst | 20 +++-- docs/admin/installation-nginx.rst | 149 ++++++++++++++++++++++++++------------ docs/admin/installation-searx.rst | 2 + docs/admin/installation-uwsgi.rst | 2 + docs/admin/morty.rst | 2 +- 5 files changed, 121 insertions(+), 54 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/filtron.rst b/docs/admin/filtron.rst index 2bc663411..785b02261 100644 --- a/docs/admin/filtron.rst +++ b/docs/admin/filtron.rst @@ -1,5 +1,5 @@ -.. _searx_filtron: +.. _searx filtron: ========================== How to protect an instance @@ -8,6 +8,8 @@ How to protect an instance .. sidebar:: further reading - :ref:`filtron.sh` + - :ref:`nginx searx site` + .. contents:: Contents :depth: 2 @@ -150,6 +152,8 @@ of: ] +.. _filtron route request: + Route request through filtron ============================= @@ -167,12 +171,14 @@ Use it along with ``nginx`` with the following example configuration. .. code:: nginx location / { - proxy_set_header Host $http_host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Scheme $scheme; - proxy_pass http://127.0.0.1:4004/; + proxy_pass http://127.0.0.1:4004/; + + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; } Requests are coming from port 4004 going through filtron and then forwarded to -port 8888 where a searx is being run. +port 8888 where a searx is being run. For a complete setup see: :ref:`nginx +searx site`. diff --git a/docs/admin/installation-nginx.rst b/docs/admin/installation-nginx.rst index 6cd815d1d..0b3ea022f 100644 --- a/docs/admin/installation-nginx.rst +++ b/docs/admin/installation-nginx.rst @@ -12,6 +12,12 @@ Install with nginx http://nginx.org/en/docs/beginners_guide.html .. _Getting Started wiki: https://www.nginx.com/resources/wiki/start/ +.. _uWSGI support from nginx: + https://uwsgi-docs.readthedocs.io/en/latest/Nginx.html +.. _uwsgi_params: + https://uwsgi-docs.readthedocs.io/en/latest/Nginx.html#configuring-nginx +.. _SCRIPT_NAME: + https://werkzeug.palletsprojects.com/en/1.0.x/wsgi/#werkzeug.wsgi.get_script_name .. contents:: Contents :depth: 2 @@ -98,8 +104,8 @@ see a *Fedora Webserver - Test Page*. The test page comes from the default .. _nginx searx site: -A searx site -============ +A nginx searx site +================== .. sidebar:: public to the internet? @@ -134,33 +140,42 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. tabs:: + .. group-tab:: searx via filtron plus morty - .. group-tab:: filtron at ``/`` & ``/morty`` - - Use this setup, if your instance is public to the internet: + Use this setup, if your instance is public to the internet, compare + figure: :ref:`architecture `. Configure a reverse proxy for + :ref:`filtron `, listening on *localhost 4004* (:ref:`filtron + route request`): .. code:: nginx location / { - proxy_set_header Host $http_host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Scheme $scheme; proxy_pass http://127.0.0.1:4004/; + + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; } + + Configure reverse proxy for :ref:`morty `, listening on + *localhost 3000*: + .. code:: nginx location /morty { - proxy_set_header Host $http_host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Scheme $scheme; proxy_pass http://127.0.0.1:3000/; + + proxy_set_header Host $http_host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; } - For a fully result proxification add :ref:`morty's ` public - URL to your :origin:`searx/settings.yml`: + Note that reverse proxy advised to be used in case of single-user or + low-traffic instances. For a fully result proxification add :ref:`morty's + ` **public URL** to your :origin:`searx/settings.yml`: .. code:: yaml @@ -169,66 +184,108 @@ Started wiki`_ is always a good resource *to keep in the pocket*. url : http://searx.example.com/ - .. group-tab:: searx at ``/`` + .. group-tab:: proxy or uWSGI - Use this setup only, if your instance is **NOT** public to the internet: + Be warned, with this setup, your Instance isn't :ref:`protected `. Nevertheless it is good enough for intranet usage and it is a + excellent example of; *how different services can be set up*. The next + example shows a reverse proxy configuration wrapping the :ref:`searx-uWSGI + application `, listening on ``http = + 127.0.0.1:8888``. .. code:: nginx - server { - listen 80; - listen [::]:80; + location / { + proxy_pass http://127.0.0.1:8888; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_set_header X-Script-Name /searx; + proxy_buffering off; + } + + Alternatively you can use the `uWSGI support from nginx`_ via unix + sockets. For socket communication, you have to activate ``socket = + /run/uwsgi/app/searx/socket`` and comment out the ``http = + 127.0.0.1:8888`` configuration in your :ref:`uwsgi ini file `. + + The example shows a nginx virtual ``server`` configuration, listening on + port 80 (IPv4 and IPv6 http://[::]:80). The uWSGI app is configured at + location ``/`` by importing the `uwsgi_params`_ and passing requests to + the uWSGI socket (``uwsgi_pass``). The ``server``\'s root points to the + :ref:`searx-src clone ` and wraps directly the + :origin:`searx/static/` content at ``location /static``. + + .. code:: nginx + + server { # replace searx.example.com with your server's public name server_name searx.example.com; - root /usr/local/searx/searx; - - location /static { - } + listen 80; + listen [::]:80; location / { include uwsgi_params; uwsgi_pass unix:/run/uwsgi/app/searx/socket; } + + root /usr/local/searx/searx-src/searx; + location /static { } } - .. group-tab:: searx at ``/searx`` + If not already exists, create a folder for the unix sockets, which can be + used by the searx account: - Use this setup only, if your instance is **NOT** public to the internet: + .. code:: bash - .. code:: nginx + mkdir -p /run/uwsgi/app/searx/ + sudo -H chown -R searx:searx /run/uwsgi/app/searx/ - location /searx/static { - alias /usr/local/searx/searx/static; - } + .. group-tab:: subdirectory URL + + Be warned, with these setups, your Instance isn't :ref:`protected `. The examples are just here to demonstrate how to export the + searx application from a subdirectory URL + http://searx.example.com/searx/\. + + .. code:: nginx location /searx { - uwsgi_param SCRIPT_NAME /searx; - include uwsgi_params; - uwsgi_pass unix:/run/uwsgi/app/searx/socket; + proxy_pass http://127.0.0.1:8888; + + proxy_set_header Host $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_set_header X-Script-Name /searx; + proxy_buffering off; } + location /searx/static { + alias /usr/local/searx/searx-src/searx/static; + } - **OR** using reverse proxy. Please, note that reverse proxy advised to be - used in case of single-user or low-traffic instances. + The ``X-Script-Name /searx`` is needed by the searx implementation to + calculate relative URLs correct. The next example shows a uWSGI + configuration. Since there are no HTTP headers in a (u)WSGI protocol, the + value is shipped via the SCRIPT_NAME_ in the WSGI environment. .. code:: nginx location /searx/static { - alias /usr/local/searx/searx/static; + alias /usr/local/searx/searx-src/searx; } location /searx { - proxy_pass http://127.0.0.1:8888; - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Scheme $scheme; - proxy_set_header X-Script-Name /searx; - proxy_buffering off; + uwsgi_param SCRIPT_NAME /searx; + include uwsgi_params; + uwsgi_pass unix:/run/uwsgi/app/searx/socket; } - Enable ``base_url`` in :origin:`searx/settings.yml` + For searx to work correctly the ``base_url`` must be set in the + :origin:`searx/settings.yml`. .. code:: yaml @@ -246,21 +303,21 @@ Restart service: .. code:: sh sudo -H systemctl restart nginx - sudo -H systemctl restart uwsgi + sudo -H service uwsgi restart searx .. group-tab:: Arch Linux .. code:: sh sudo -H systemctl restart nginx - sudo -H systemctl restart uwsgi + sudo -H systemctl restart uwsgi@searx .. group-tab:: Fedora .. code:: sh sudo -H systemctl restart nginx - sudo -H systemctl restart uwsgi + sudo -H touch /etc/uwsgi.d/searx.ini Disable logs diff --git a/docs/admin/installation-searx.rst b/docs/admin/installation-searx.rst index 88562f3e3..91891d85c 100644 --- a/docs/admin/installation-searx.rst +++ b/docs/admin/installation-searx.rst @@ -32,6 +32,8 @@ Create user :start-after: START create user :end-before: END create user +.. _searx-src: + install searx & dependencies ============================ diff --git a/docs/admin/installation-uwsgi.rst b/docs/admin/installation-uwsgi.rst index 7996bf937..67a8e127a 100644 --- a/docs/admin/installation-uwsgi.rst +++ b/docs/admin/installation-uwsgi.rst @@ -92,6 +92,8 @@ could control specific instance(s) by issuing:: My experience is, that this command is a bit buggy. +.. _uwsgi configuration: + Alltogether =========== diff --git a/docs/admin/morty.rst b/docs/admin/morty.rst index 9af9b6ae9..48ff5b9c4 100644 --- a/docs/admin/morty.rst +++ b/docs/admin/morty.rst @@ -1,5 +1,5 @@ -.. _searx_morty: +.. _searx morty: ========================= How to setup result proxy -- cgit v1.2.3 From 857a1458b18476e18f1df9e338ba270c4ff2e35d Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Fri, 6 Mar 2020 15:04:12 +0100 Subject: docs: for example URLs use iana's example.org Signed-off-by: Markus Heiser --- docs/admin/installation-nginx.rst | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/installation-nginx.rst b/docs/admin/installation-nginx.rst index 0b3ea022f..264bdacda 100644 --- a/docs/admin/installation-nginx.rst +++ b/docs/admin/installation-nginx.rst @@ -180,8 +180,8 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. code:: yaml result_proxy: - # replace searx.example.com with your server's public name - url : http://searx.example.com/ + # replace example.org with your server's public name + url : https://example.org/ .. group-tab:: proxy or uWSGI @@ -221,8 +221,8 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. code:: nginx server { - # replace searx.example.com with your server's public name - server_name searx.example.com; + # replace example.org with your server's public name + server_name example.org; listen 80; listen [::]:80; @@ -248,8 +248,7 @@ Started wiki`_ is always a good resource *to keep in the pocket*. Be warned, with these setups, your Instance isn't :ref:`protected `. The examples are just here to demonstrate how to export the - searx application from a subdirectory URL - http://searx.example.com/searx/\. + searx application from a subdirectory URL ``https://example.org/searx/``. .. code:: nginx @@ -290,8 +289,8 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. code:: yaml server: - # replace searx.example.com with your server's public name - base_url : http://searx.example.com/searx/ + # replace example.org with your server's public name + base_url : https://example.org/searx/ Restart service: -- cgit v1.2.3 From c15337850e64562c376e5de57d8809a3a05b8a5d Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Fri, 6 Mar 2020 22:06:19 +0100 Subject: fix: minor typos Signed-off-by: Markus Heiser --- docs/admin/filtron.rst | 2 +- docs/admin/installation-nginx.rst | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/filtron.rst b/docs/admin/filtron.rst index 785b02261..970ce25d2 100644 --- a/docs/admin/filtron.rst +++ b/docs/admin/filtron.rst @@ -18,7 +18,7 @@ How to protect an instance .. _filtron: https://github.com/asciimoo/filtron -Searx depens on external search services. To avoid the abuse of these services +Searx depends on external search services. To avoid the abuse of these services it is advised to limit the number of requests processed by searx. An application firewall, filtron_ solves exactly this problem. Filtron is just diff --git a/docs/admin/installation-nginx.rst b/docs/admin/installation-nginx.rst index 264bdacda..2097d7daf 100644 --- a/docs/admin/installation-nginx.rst +++ b/docs/admin/installation-nginx.rst @@ -186,7 +186,7 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. group-tab:: proxy or uWSGI - Be warned, with this setup, your Instance isn't :ref:`protected `. Nevertheless it is good enough for intranet usage and it is a excellent example of; *how different services can be set up*. The next example shows a reverse proxy configuration wrapping the :ref:`searx-uWSGI @@ -246,7 +246,7 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. group-tab:: subdirectory URL - Be warned, with these setups, your Instance isn't :ref:`protected `. The examples are just here to demonstrate how to export the searx application from a subdirectory URL ``https://example.org/searx/``. -- cgit v1.2.3 From 80f7b658047a3541697ef5ae1aae897817b3f43c Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Mon, 9 Mar 2020 01:37:26 +0100 Subject: searx.sh: add buildhost cmd, installs OS packages for builds Signed-off-by: Markus Heiser --- docs/admin/buildhosts.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'docs/admin') diff --git a/docs/admin/buildhosts.rst b/docs/admin/buildhosts.rst index 27784764f..a6820aa61 100644 --- a/docs/admin/buildhosts.rst +++ b/docs/admin/buildhosts.rst @@ -104,7 +104,8 @@ For PDF output you also need: $ sudo dnf install \ texlive-collection-fontsrecommended texlive-collection-latex \ - dejavu-sans-fonts dejavu-serif-fonts dejavu-sans-mono-fonts + dejavu-sans-fonts dejavu-serif-fonts dejavu-sans-mono-fonts \ + ImageMagick .. _sh lint: -- cgit v1.2.3 From f27f6c6c5d372b257dfe671ec26eb73f6e12ed28 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Tue, 31 Mar 2020 18:25:40 +0200 Subject: docs: use make and uitls/searx.sh in containers Signed-off-by: Markus Heiser --- docs/admin/buildhosts.rst | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/buildhosts.rst b/docs/admin/buildhosts.rst index a6820aa61..04698d8d2 100644 --- a/docs/admin/buildhosts.rst +++ b/docs/admin/buildhosts.rst @@ -4,9 +4,6 @@ Buildhosts ========== -To get best results from build, its recommend to install additional packages -on build hosts. - .. sidebar:: This article needs some work If you have any contribution send us your :pull:`PR <../pulls>`, see @@ -17,6 +14,23 @@ on build hosts. :local: :backlinks: entry +To get best results from build, its recommend to install additional packages +on build hosts (see :ref:`searx.sh`).:: + + sudo -H ./utils/searx.sh buildhost + +This will install packages needed by searx: + +.. include:: ../../build/docs/includes/searx.rst + :start-after: START distro-packages + :end-before: END distro-packages + +and packages needed to build docuemtation and run tests: + +.. include:: ../../build/docs/includes/searx.rst + :start-after: START build-packages + :end-before: END build-packages + .. _docs build: Build docs -- cgit v1.2.3 From 853be4e11cd8a82b6b08f04db1717329907a5aaf Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Fri, 3 Apr 2020 20:24:40 +0200 Subject: utils/filtron.sh: add generic documentation Signed-off-by: Markus Heiser --- docs/admin/filtron.rst | 1 + 1 file changed, 1 insertion(+) (limited to 'docs/admin') diff --git a/docs/admin/filtron.rst b/docs/admin/filtron.rst index 970ce25d2..9615cd935 100644 --- a/docs/admin/filtron.rst +++ b/docs/admin/filtron.rst @@ -37,6 +37,7 @@ installed by ``go get`` package management (see `filtron README`_). If you use filtron as middleware, a more isolated setup is recommended. To simplify such an installation and the maintenance of, use our script :ref:`filtron.sh`. +.. _Sample configuration of filtron: Sample configuration of filtron =============================== -- cgit v1.2.3 From 84daf9999c9b52bf0b938725755a871aa359c62c Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sat, 4 Apr 2020 08:36:48 +0200 Subject: docs: remove PR #1803 hints Signed-off-by: Markus Heiser --- docs/admin/installation.rst | 11 ----------- 1 file changed, 11 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/installation.rst b/docs/admin/installation.rst index 25d7f0ea9..ac17c326a 100644 --- a/docs/admin/installation.rst +++ b/docs/admin/installation.rst @@ -35,17 +35,6 @@ and some maintenance tasks (alternatively you can create your own fork). $ git clone https://github.com/asciimoo/searx searx $ cd searx -.. hint:: - - The *tooling box* is not yet merged into `asciimoo/searx master - `_. As long as PR is not merged, you need - to merge the PR into your local clone (see below). The discussion takes - place in :pull:`1803`. To merge the :pull:`1803` in your local branch use: - - .. code:: bash - - $ git pull origin refs/pull/1803/head - **Install** :ref:`searx service ` This installs searx as described in :ref:`installation basic`. -- cgit v1.2.3 From eb0d4646d818fe12032379aae2fcd8b5bdb6467e Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Mon, 6 Apr 2020 17:59:06 +0200 Subject: docs: rework of chapter "Install with apache" BTW: normalize installation-nginx.rst --- docs/admin/installation-apache.rst | 462 +++++++++++++++++++++++++++++++++---- docs/admin/installation-nginx.rst | 27 ++- docs/admin/installation.rst | 6 + 3 files changed, 443 insertions(+), 52 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/installation-apache.rst b/docs/admin/installation-apache.rst index 8c6228380..da551c3a9 100644 --- a/docs/admin/installation-apache.rst +++ b/docs/admin/installation-apache.rst @@ -4,19 +4,303 @@ Install with apache =================== -.. sidebar:: public to the internet? - - If your searx instance is public, stop here and first install :ref:`filtron - reverse proxy ` and :ref:`result proxy morty `, see - :ref:`installation scripts`. +.. _Apache: https://httpd.apache.org/ +.. _Apache Debian: + https://cwiki.apache.org/confluence/display/HTTPD/DistrosDefaultLayout#DistrosDefaultLayout-Debian,Ubuntu(Apachehttpd2.x): +.. _README.Debian: + https://salsa.debian.org/apache-team/apache2/raw/master/debian/apache2.README.Debian +.. _Apache Arch Linux: + https://wiki.archlinux.org/index.php/Apache_HTTP_Server +.. _Apache Fedora: + https://docs.fedoraproject.org/en-US/quick-docs/getting-started-with-apache-http-server/index.html +.. _Apache directives: + https://httpd.apache.org/docs/trunk/mod/directives.html +.. _Getting Started: + https://httpd.apache.org/docs/current/en/getting-started.html +.. _Terms Used to Describe Directives: + https://httpd.apache.org/docs/current/en/mod/directive-dict.html +.. _Configuration Files: + https://httpd.apache.org/docs/current/en/configuring.html +.. _ProxyPreserveHost: https://httpd.apache.org/docs/trunk/mod/mod_proxy.html#proxypreservehost +.. _LoadModule: + https://httpd.apache.org/docs/2.4/mod/mod_so.html#loadmodule +.. _DocumentRoot: + https://httpd.apache.org/docs/trunk/mod/core.html#documentroot +.. _Location: + https://httpd.apache.org/docs/trunk/mod/core.html#location +.. _uWSGI Apache support: + https://uwsgi-docs.readthedocs.io/en/latest/Apache.html +.. _apache uwsgi: + https://uwsgi-docs.readthedocs.io/en/latest/Apache.html#mod-proxy-uwsgi +.. _mod_proxy_uwsgi: + https://uwsgi-docs.readthedocs.io/en/latest/Apache.html#mod-proxy-uwsgi + +.. sidebar:: further read + + - `Apache Arch Linux`_ + - `Apache Debian`_ and `README.Debian`_ + - `Apache Fedora`_ + - `Apache directives`_ .. contents:: Contents :depth: 2 :local: :backlinks: entry -Add wsgi mod -============ +The apache HTTP server +====================== + +If Apache_ is not installed, install it now. If apache_ is new to you, the +`Getting Started`_, `Configuration Files`_ and `Terms Used to Describe +Directives`_ documentation gives first orientation. There is also a list of +`Apache directives`_ *to keep in the pocket*. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + sudo -H apt-get install apache2 + + .. group-tab:: Arch Linux + + .. code:: sh + + sudo -H pacman -S apache + sudo -H systemctl enable httpd + sudo -H systemctl start http + + .. group-tab:: Fedora / RHEL + + .. code:: sh + + sudo -H dnf install httpd + sudo -H systemctl enable httpd + sudo -H systemctl start httpd + +Now at http://localhost you should see any kind of *Welcome* or *Test* page. +How this default intro site is configured, depends on the linux distribution +(compare `Apache directives`_). + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + .. code:: sh + + less /etc/apache2/sites-enabled/000-default.conf + + In this file, there is a line setting the `DocumentRoot`_ directive: + + .. code:: apache + + DocumentRoot /var/www/html + + And the *welcome* page is the HTML file at ``/var/www/html/index.html``. + + .. group-tab:: Arch Linux + + .. code:: sh + + less /etc/httpd/conf/httpd.conf + + In this file, there is a line setting the `DocumentRoot`_ directive: + + .. code:: apache + + DocumentRoot "/srv/http" + + Options Indexes FollowSymLinks + AllowOverride None + Require all granted + + + The *welcome* page of Arch Linux is a page showing directory located at + ``DocumentRoot``. This is *directory* page is generated by the Module + `mod_autoindex `_: + + .. code:: apache + + LoadModule autoindex_module modules/mod_autoindex.so + ... + Include conf/extra/httpd-autoindex.conf + + .. group-tab:: Fedora / RHEL + + .. code:: sh + + less /etc/httpd/conf/httpd.conf + + In this file, there is a line setting the ``DocumentRoot`` directive: + + .. code:: apache + + DocumentRoot "/var/www/html" + ... + + AllowOverride None + # Allow open access: + Require all granted + + + On fresh installations, the ``/var/www`` is empty and the *default + welcome page* is shown, the configuration is located at:: + + less /etc/httpd/conf.d/welcome.conf + +.. _The Debian Layout: + +The Debian Layout +================= + +Be aware that the Debian layout is quite different from the standard Apache +configuration. For details look at the README.Debian_ +(``/usr/share/doc/apache2/README.Debian.gz``). Some commands you should know on +Debian: + +* :man:`apache2ctl`: Apache HTTP server control interface +* :man:`a2enmod`, :man:`a2dismod`: switch on/off modules +* :man:`a2enconf`, :man:`a2disconf`: switch on/off configurations +* :man:`a2ensite`, :man:`a2dissite`: switch on/off sites + + +.. _apache searx site: + +Apache Reverse Proxy +==================== + +.. sidebar:: public to the internet? + + If your searx instance is public, stop here and first install :ref:`filtron + reverse proxy ` and :ref:`result proxy morty `, see + :ref:`installation scripts`. If already done, follow setup: *searx via + filtron plus morty*. + +To setup a Apache revers proxy you have to enable the *headers* and *proxy* +modules and create a `Location`_ configuration for the searx site. In most +distributions you have to uncomment the lines in the main configuration file, +except in the :ref:`The Debian Layout`. + +.. tabs:: + + .. group-tab:: Ubuntu / debian + + In the Apache setup, enable headers and proxy modules: + + .. code:: sh + + sudo -H a2enmod headers + sudo -H a2enmod proxy + sudo -H a2enmod proxy_http + + In :ref:`The Debian Layout` you create a ``searx.conf`` with the + ```` directive and save this file in the *sites + available* folder at ``/etc/apache2/sites-available``. To enable the + ``searx.conf`` use :man:`a2ensite`: + + .. code:: sh + + sudo -H a2ensite searx.conf + + .. group-tab:: Arch Linux + + In the ``/etc/httpd/conf/httpd.conf`` file, activate headers and proxy + modules (LoadModule_): + + .. code:: apache + + LoadModule headers_module modules/mod_headers.so + LoadModule proxy_module modules/mod_proxy.so + LoadModule proxy_http_module modules/mod_proxy_http.so + + .. group-tab:: Fedora / RHEL + + In the ``/etc/httpd/conf/httpd.conf`` file, activate headers and proxy + modules (LoadModule_): + + .. code:: apache + + LoadModule headers_module modules/mod_headers.so + LoadModule proxy_module modules/mod_proxy.so + LoadModule proxy_http_module modules/mod_proxy_http.so + +.. tabs:: + + .. group-tab:: searx via filtron plus morty + + Use this setup, if your instance is public to the internet, compare + figure: :ref:`architecture ` and :ref:`installation scripts`. + + 1. Configure a reverse proxy for :ref:`filtron `, listening on + *localhost 4004* (:ref:`filtron route request`): + + .. code:: apache + + + + # SetEnvIf Request_URI "/searx" dontlog + # CustomLog /dev/null combined env=dontlog + + Require all granted + + Order deny,allow + Deny from all + #Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + ProxyPreserveHost On + ProxyPass http://127.0.0.1:4004 + RequestHeader set X-Script-Name /searx + + + + 2. Configure reverse proxy for :ref:`morty `, listening on + *localhost 3000* (FYI: ``ProxyPreserveHost On`` is already set, see + above): + + .. code:: apache + + ProxyPreserveHost On + + + + # SetEnvIf Request_URI "/morty" dontlog + # CustomLog /dev/null combined env=dontlog + + Require all granted + + Order deny,allow + Deny from all + #Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + ProxyPass http://127.0.0.1:3000 + RequestHeader set X-Script-Name /morty + + + + Note that reverse proxy advised to be used in case of single-user or + low-traffic instances. For a fully result proxification add :ref:`morty's + ` **public URL** to your :origin:`searx/settings.yml`: + + .. code:: yaml + + result_proxy: + # replace example.org with your server's public name + url : https://example.org/morty + + server: + image_proxy : True + +uWSGI support +============= + +Be warned, with this setup, your instance isn't :ref:`protected `. Nevertheless it is good enough for intranet usage and it +demonstrates: *how different the uwsgi support is, depending on the +distribution*. To enable :ref:`uWSGI ` support you need to install +the apache `apache uwsgi`_ support: .. tabs:: @@ -27,59 +311,130 @@ Add wsgi mod sudo -H apt-get install libapache2-mod-uwsgi sudo -H a2enmod uwsgi -Add this configuration in the file ``/etc/apache2/apache2.conf``. To limit -acces to your intranet replace ``Allow from all`` directive and replace -``192.168.0.0/16`` with your subnet IP/class. + .. group-tab:: Arch Linux + + .. code:: sh + + sudo -H pacman -S uwsgi -.. _inranet apache site: + In the ``/etc/httpd/conf/httpd.conf`` file, activate headers and proxy + modules (LoadModule_): -Note that if your instance of searx is not at the root, you should change -```` by the location of your instance, like ````: + .. code:: apache -.. code:: apache + LoadModule proxy_module modules/mod_proxy.so + LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so - # CustomLog /dev/null combined + .. group-tab:: Fedora / RHEL - + .. code:: sh - + sudo -H dnf install uwsgi + FIXME: enable uwsgi in apache - Options FollowSymLinks Indexes - SetHandler uwsgi-handler - uWSGISocket /run/uwsgi/app/searx/socket +The next example shows a configuration using the `uWSGI Apache support`_ via +unix sockets. For socket communication, you have to activate ``socket = +/run/uwsgi/app/searx/socket`` and comment out the ``http = 127.0.0.1:8888`` +configuration in your :ref:`uwsgi ini file `. - Order deny,allow - Deny from all - # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 - Allow from all +If not already exists, create a folder for the unix sockets, which can be +used by the searx account: - +.. code:: bash - + sudo -H mkdir -p /run/uwsgi/app/searx/ + sudo -H chown -R searx:searx /run/uwsgi/app/searx/ -Enable apache mod_uwsgi and restart apache: +To limit acces to your intranet replace ``Allow from all`` directive and replace +``192.168.0.0/16`` with your subnet IP/class. .. tabs:: .. group-tab:: Ubuntu / debian - .. code:: sh + Debian uses the (old) `mod_uwsgi + `_. - a2enmod uwsgi - sudo -H systemctl restart apache2 + .. code:: apache -disable logs -============ + + + # SetEnvIf Request_URI "/searx" dontlog + # CustomLog /dev/null combined env=dontlog + + + + Require all granted + + Options FollowSymLinks Indexes + SetHandler uwsgi-handler + uWSGISocket /run/uwsgi/app/searx/socket + + Order deny,allow + Deny from all + # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + + + + + .. group-tab:: Arch Linux + + Arch Linux uses the (recommend) `mod_proxy_uwsgi`_. + + .. code:: apache + + + + # SetEnvIf Request_URI /searx dontlog + # CustomLog /dev/null combined env=dontlog + + + + Require all granted + Order deny,allow + Deny from all + # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + ProxyPreserveHost On + ProxyPass unix:/run/uwsgi/app/searx/socket|uwsgi://uwsgi-uds-searx/ + + + + + + .. group-tab:: Fedora / RHEL + + RHEL uses the (recommend) `mod_proxy_uwsgi`_. -For better privacy you can disable Apache logs. Go back to -``/etc/apache2/apache2.conf`` :ref:`[example] ` and above -```` activate directive: + .. code:: apache -.. code:: apache + - CustomLog /dev/null combined + # SetEnvIf Request_URI /searx dontlog + # CustomLog /dev/null combined env=dontlog -Restart apache: + + + Require all granted + Order deny,allow + Deny from all + # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + ProxyPreserveHost On + ProxyPass unix:/run/uwsgi/app/searx/socket|uwsgi://uwsgi-uds-searx/ + + + + + +.. _restart apache: + +Restart service +=============== .. tabs:: @@ -88,8 +443,33 @@ Restart apache: .. code:: sh sudo -H systemctl restart apache2 + sudo -H service uwsgi restart searx + + .. group-tab:: Arch Linux + + .. code:: sh + + sudo -H systemctl restart httpd + sudo -H systemctl restart uwsgi@searx + + .. group-tab:: Fedora / RHEL + + .. code:: sh + + sudo -H systemctl restart httpd + sudo -H touch /etc/uwsgi.d/searx.ini + + +disable logs +============ + +For better privacy you can disable Apache logs. In the examples above activate +one of the lines and `restart apache`_:: + -.. warning:: + # SetEnvIf Request_URI "/searx" dontlog + # CustomLog /dev/null combined env=dontlog - You can only disable logs for the whole (virtual) server not for a specific - path. +The ``CustomLog`` directive disable logs for the whole (virtual) server, use it +when the URL of the service does not have a path component (``/searx``) / is +located at root (``/``). diff --git a/docs/admin/installation-nginx.rst b/docs/admin/installation-nginx.rst index 2097d7daf..b02c555f6 100644 --- a/docs/admin/installation-nginx.rst +++ b/docs/admin/installation-nginx.rst @@ -28,10 +28,9 @@ Install with nginx The nginx HTTP server ===================== -If nginx_ is not installed (uwsgi will not work with the package nginx-light) +If nginx_ is not installed (uwsgi will not work with the package nginx-light), install it now. - .. tabs:: .. group-tab:: Ubuntu / debian @@ -58,7 +57,8 @@ install it now. Now at http://localhost you should see a *Welcome to nginx!* page, on Fedora you see a *Fedora Webserver - Test Page*. The test page comes from the default -`nginx server configuration`_: +`nginx server configuration`_. How this default intro site is configured, +depends on the linux distribution: .. tabs:: @@ -111,7 +111,8 @@ A nginx searx site If your searx instance is public, stop here and first install :ref:`filtron reverse proxy ` and :ref:`result proxy morty `, see - :ref:`installation scripts`. + :ref:`installation scripts`. If already done, follow setup: *searx via + filtron plus morty*. Now you have to create a configuration for the searx site. If nginx_ is new to you, the `nginx beginners guide`_ is a good starting point and the `Getting @@ -143,9 +144,10 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. group-tab:: searx via filtron plus morty Use this setup, if your instance is public to the internet, compare - figure: :ref:`architecture `. Configure a reverse proxy for - :ref:`filtron `, listening on *localhost 4004* (:ref:`filtron - route request`): + figure: :ref:`architecture ` and :ref:`installation scripts`. + + 1. Configure a reverse proxy for :ref:`filtron `, listening on + *localhost 4004* (:ref:`filtron route request`): .. code:: nginx @@ -159,8 +161,8 @@ Started wiki`_ is always a good resource *to keep in the pocket*. } - Configure reverse proxy for :ref:`morty `, listening on - *localhost 3000*: + 2. Configure reverse proxy for :ref:`morty `, listening on + *localhost 3000*: .. code:: nginx @@ -181,7 +183,10 @@ Started wiki`_ is always a good resource *to keep in the pocket*. result_proxy: # replace example.org with your server's public name - url : https://example.org/ + url : https://example.org/morty + + server: + image_proxy : True .. group-tab:: proxy or uWSGI @@ -244,7 +249,7 @@ Started wiki`_ is always a good resource *to keep in the pocket*. mkdir -p /run/uwsgi/app/searx/ sudo -H chown -R searx:searx /run/uwsgi/app/searx/ - .. group-tab:: subdirectory URL + .. group-tab:: proxy at subdir URL Be warned, with these setups, your instance isn't :ref:`protected `. The examples are just here to demonstrate how to export the diff --git a/docs/admin/installation.rst b/docs/admin/installation.rst index ac17c326a..81fc4d3e2 100644 --- a/docs/admin/installation.rst +++ b/docs/admin/installation.rst @@ -54,3 +54,9 @@ This installs searx as described in :ref:`installation basic`. .. code:: bash $ sudo -H ./utils/morty.sh install all + +If all services are running fine, you can add it to your HTTP server: + +- :ref:`installation apache` +- :ref:`installation nginx` + -- cgit v1.2.3 From ee39a098acb2386abd5382de5c9476cc4ffe2e03 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Tue, 7 Apr 2020 18:31:51 +0200 Subject: apache: normalize installation (docs and script)s over all distros Signed-off-by: Markus Heiser --- docs/admin/installation-apache.rst | 169 +++++++++++++++++++++---------------- docs/admin/installation-searx.rst | 2 + 2 files changed, 98 insertions(+), 73 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/installation-apache.rst b/docs/admin/installation-apache.rst index da551c3a9..0b6cc38df 100644 --- a/docs/admin/installation-apache.rst +++ b/docs/admin/installation-apache.rst @@ -30,8 +30,6 @@ Install with apache https://httpd.apache.org/docs/trunk/mod/core.html#location .. _uWSGI Apache support: https://uwsgi-docs.readthedocs.io/en/latest/Apache.html -.. _apache uwsgi: - https://uwsgi-docs.readthedocs.io/en/latest/Apache.html#mod-proxy-uwsgi .. _mod_proxy_uwsgi: https://uwsgi-docs.readthedocs.io/en/latest/Apache.html#mod-proxy-uwsgi @@ -149,22 +147,6 @@ How this default intro site is configured, depends on the linux distribution less /etc/httpd/conf.d/welcome.conf -.. _The Debian Layout: - -The Debian Layout -================= - -Be aware that the Debian layout is quite different from the standard Apache -configuration. For details look at the README.Debian_ -(``/usr/share/doc/apache2/README.Debian.gz``). Some commands you should know on -Debian: - -* :man:`apache2ctl`: Apache HTTP server control interface -* :man:`a2enmod`, :man:`a2dismod`: switch on/off modules -* :man:`a2enconf`, :man:`a2disconf`: switch on/off configurations -* :man:`a2ensite`, :man:`a2dissite`: switch on/off sites - - .. _apache searx site: Apache Reverse Proxy @@ -179,8 +161,12 @@ Apache Reverse Proxy To setup a Apache revers proxy you have to enable the *headers* and *proxy* modules and create a `Location`_ configuration for the searx site. In most -distributions you have to uncomment the lines in the main configuration file, -except in the :ref:`The Debian Layout`. +distributions you have to un-comment the lines in the main configuration file, +except in :ref:`The Debian Layout`. + +To pass the HTTP HOST header +With ProxyPreserveHost_ the incoming Host HTTP request header is passed to the +proxied host. .. tabs:: @@ -210,6 +196,8 @@ except in the :ref:`The Debian Layout`. .. code:: apache + FIXME needs test + LoadModule headers_module modules/mod_headers.so LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_http_module modules/mod_proxy_http.so @@ -221,7 +209,9 @@ except in the :ref:`The Debian Layout`. .. code:: apache - LoadModule headers_module modules/mod_headers.so + FIXME needs test + + LoadModule headers_module modules/mod_headers.so LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_http_module modules/mod_proxy_http.so @@ -256,8 +246,7 @@ except in the :ref:`The Debian Layout`. 2. Configure reverse proxy for :ref:`morty `, listening on - *localhost 3000* (FYI: ``ProxyPreserveHost On`` is already set, see - above): + *localhost 3000* .. code:: apache @@ -297,10 +286,9 @@ uWSGI support ============= Be warned, with this setup, your instance isn't :ref:`protected `. Nevertheless it is good enough for intranet usage and it -demonstrates: *how different the uwsgi support is, depending on the -distribution*. To enable :ref:`uWSGI ` support you need to install -the apache `apache uwsgi`_ support: +filtron>`, nevertheless it is good enough for intranet usage. In modern Linux +distributions, the `mod_proxy_uwsgi`_ is compiled into the *normal* apache +package and you need to install only the :ref:`uWSGI ` package: .. tabs:: @@ -308,8 +296,10 @@ the apache `apache uwsgi`_ support: .. code:: sh - sudo -H apt-get install libapache2-mod-uwsgi - sudo -H a2enmod uwsgi + sudo -H apt-get install uwsgi + + # Ubuntu =< 18.04 + sudo -H apt-get install libapache2-mod-proxy-uwsgi .. group-tab:: Arch Linux @@ -317,74 +307,88 @@ the apache `apache uwsgi`_ support: sudo -H pacman -S uwsgi - In the ``/etc/httpd/conf/httpd.conf`` file, activate headers and proxy - modules (LoadModule_): - - .. code:: apache - - LoadModule proxy_module modules/mod_proxy.so - LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so - .. group-tab:: Fedora / RHEL .. code:: sh sudo -H dnf install uwsgi - FIXME: enable uwsgi in apache The next example shows a configuration using the `uWSGI Apache support`_ via -unix sockets. For socket communication, you have to activate ``socket = -/run/uwsgi/app/searx/socket`` and comment out the ``http = 127.0.0.1:8888`` -configuration in your :ref:`uwsgi ini file `. +unix sockets and `mod_proxy_uwsgi`_. -If not already exists, create a folder for the unix sockets, which can be -used by the searx account: +For socket communication, you have to activate ``socket = +/run/uwsgi/app/searx/socket`` and comment out the ``http = 127.0.0.1:8888`` +configuration in your :ref:`uwsgi ini file `. If not +already exists, create a folder for the unix sockets, which can be used by the +searx account (see :ref:`create searx user`): .. code:: bash sudo -H mkdir -p /run/uwsgi/app/searx/ sudo -H chown -R searx:searx /run/uwsgi/app/searx/ -To limit acces to your intranet replace ``Allow from all`` directive and replace -``192.168.0.0/16`` with your subnet IP/class. +If the server is public; to limit access to your intranet replace ``Allow from +all`` directive and replace ``192.168.0.0/16`` with your subnet IP/class. .. tabs:: .. group-tab:: Ubuntu / debian - Debian uses the (old) `mod_uwsgi - `_. - .. code:: apache - + LoadModule headers_module /usr/lib/apache2/mod_headers.so + LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so + LoadModule proxy_uwsgi_module /usr/lib/apache2/modules/mod_proxy_uwsgi.so - # SetEnvIf Request_URI "/searx" dontlog - # CustomLog /dev/null combined env=dontlog + # SetEnvIf Request_URI /searx dontlog + # CustomLog /dev/null combined env=dontlog - + - Require all granted + Require all granted + Order deny,allow + Deny from all + # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all - Options FollowSymLinks Indexes - SetHandler uwsgi-handler - uWSGISocket /run/uwsgi/app/searx/socket + ProxyPreserveHost On + ProxyPass unix:/run/uwsgi/app/searx/socket|uwsgi://uwsgi-uds-searx/ - Order deny,allow - Deny from all - # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 - Allow from all + - + .. group-tab:: Arch Linux - + .. code:: apache - .. group-tab:: Arch Linux + FIXME needs test + + LoadModule proxy_module modules/mod_proxy.so + LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so + + # SetEnvIf Request_URI /searx dontlog + # CustomLog /dev/null combined env=dontlog + + + + Require all granted + Order deny,allow + Deny from all + # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 + Allow from all + + ProxyPreserveHost On + ProxyPass unix:/run/uwsgi/app/searx/socket|uwsgi://uwsgi-uds-searx/ - Arch Linux uses the (recommend) `mod_proxy_uwsgi`_. + + + .. group-tab:: Fedora / RHEL .. code:: apache + FIXME needs test + + LoadModule proxy_module modules/mod_proxy.so + LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so # SetEnvIf Request_URI /searx dontlog @@ -405,29 +409,33 @@ To limit acces to your intranet replace ``Allow from all`` directive and replace - .. group-tab:: Fedora / RHEL + .. group-tab:: old mod_wsgi - RHEL uses the (recommend) `mod_proxy_uwsgi`_. + We show this only for historical reasons, DON'T USE `mod_uwsgi + `_. + ANYMORE! .. code:: apache - + - # SetEnvIf Request_URI /searx dontlog + # SetEnvIf Request_URI "/searx" dontlog # CustomLog /dev/null combined env=dontlog - + Require all granted + + Options FollowSymLinks Indexes + SetHandler uwsgi-handler + uWSGISocket /run/uwsgi/app/searx/socket + Order deny,allow Deny from all # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1 Allow from all - ProxyPreserveHost On - ProxyPass unix:/run/uwsgi/app/searx/socket|uwsgi://uwsgi-uds-searx/ - - + @@ -473,3 +481,18 @@ one of the lines and `restart apache`_:: The ``CustomLog`` directive disable logs for the whole (virtual) server, use it when the URL of the service does not have a path component (``/searx``) / is located at root (``/``). + +.. _The Debian Layout: + +The Debian Layout +================= + +Be aware that the Debian layout is quite different from the standard Apache +configuration. For details look at the README.Debian_ +(``/usr/share/doc/apache2/README.Debian.gz``). Some commands you should know on +Debian: + +* :man:`apache2ctl`: Apache HTTP server control interface +* :man:`a2enmod`, :man:`a2dismod`: switch on/off modules +* :man:`a2enconf`, :man:`a2disconf`: switch on/off configurations +* :man:`a2ensite`, :man:`a2dissite`: switch on/off sites diff --git a/docs/admin/installation-searx.rst b/docs/admin/installation-searx.rst index 91891d85c..d5c0063ee 100644 --- a/docs/admin/installation-searx.rst +++ b/docs/admin/installation-searx.rst @@ -25,6 +25,8 @@ Install packages This installs also the packages needed by :ref:`searx uwsgi` +.. _create searx user: + Create user =========== -- cgit v1.2.3 From ebde9d10fd84de74ea99385dae4d95bcf932bcf3 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Thu, 9 Apr 2020 15:04:23 +0200 Subject: docs: proofreading of all the installation topics Signed-off-by: Markus Heiser --- docs/admin/architecture.rst | 14 ++++++++------ docs/admin/installation-apache.rst | 16 ++++++++++++++++ docs/admin/installation-nginx.rst | 8 ++++++++ docs/admin/installation-uwsgi.rst | 2 +- docs/admin/installation.rst | 2 +- docs/admin/morty.rst | 4 ++++ 6 files changed, 38 insertions(+), 8 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/architecture.rst b/docs/admin/architecture.rst index fbc3bf483..464e765eb 100644 --- a/docs/admin/architecture.rst +++ b/docs/admin/architecture.rst @@ -4,12 +4,14 @@ Architecture ============ -.. sidebar:: Tooling box - - - :ref:`searx & uwsgi ` - - :ref:`filtron ` - - :ref:`reverse proxy` - - :ref:`morty ` +.. sidebar:: Further reading + + - Reverse Proxy: :ref:`Apache ` & :ref:`nginx ` + - Filtron: :ref:`searx filtron` + - Morty: :ref:`searx morty` + - uWSGI: :ref:`searx uwsgi` + - Searx: :ref:`installation basic` Herein you will find some hints and suggestions about typical architectures of searx infrastructures. diff --git a/docs/admin/installation-apache.rst b/docs/admin/installation-apache.rst index 0b6cc38df..217e57718 100644 --- a/docs/admin/installation-apache.rst +++ b/docs/admin/installation-apache.rst @@ -45,6 +45,22 @@ Install with apache :local: :backlinks: entry +---- + +**Install** :ref:`apache searx site` using :ref:`filtron.sh ` + +.. code:: bash + + $ sudo -H ./utils/filtron.sh apache install + +**Install** :ref:`apache searx site` using :ref:`morty.sh ` + +.. code:: bash + + $ sudo -H ./utils/morty.sh apache install + +---- + The apache HTTP server ====================== diff --git a/docs/admin/installation-nginx.rst b/docs/admin/installation-nginx.rst index b02c555f6..e62c60df6 100644 --- a/docs/admin/installation-nginx.rst +++ b/docs/admin/installation-nginx.rst @@ -19,6 +19,14 @@ Install with nginx .. _SCRIPT_NAME: https://werkzeug.palletsprojects.com/en/1.0.x/wsgi/#werkzeug.wsgi.get_script_name +.. sidebar:: further reading + + - nginx_ + - `nginx beginners guide`_ + - `nginx server configuration`_ + - `Getting Started wiki`_ + - `uWSGI support from nginx`_ + .. contents:: Contents :depth: 2 :local: diff --git a/docs/admin/installation-uwsgi.rst b/docs/admin/installation-uwsgi.rst index 67a8e127a..72498c0bf 100644 --- a/docs/admin/installation-uwsgi.rst +++ b/docs/admin/installation-uwsgi.rst @@ -4,7 +4,7 @@ uwsgi ===== -.. sidebar:: further read +.. sidebar:: further reading - `systemd.unit`_ - `uWSGI Emperor`_ diff --git a/docs/admin/installation.rst b/docs/admin/installation.rst index 81fc4d3e2..bfc0742de 100644 --- a/docs/admin/installation.rst +++ b/docs/admin/installation.rst @@ -31,7 +31,7 @@ and some maintenance tasks (alternatively you can create your own fork). .. code:: bash - $ cd ~/Download + $ cd ~/Downloads $ git clone https://github.com/asciimoo/searx searx $ cd searx diff --git a/docs/admin/morty.rst b/docs/admin/morty.rst index 48ff5b9c4..2858fde3c 100644 --- a/docs/admin/morty.rst +++ b/docs/admin/morty.rst @@ -5,6 +5,10 @@ How to setup result proxy ========================= +.. sidebar:: further reading + + - :ref:`morty.sh` + .. _morty: https://github.com/asciimoo/morty .. _morty's README: https://github.com/asciimoo/morty -- cgit v1.2.3 From 58d5da8b57c5aeab92f551e8d175be67537c351c Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sat, 11 Apr 2020 13:19:11 +0200 Subject: nginx: normalize installation (docs and script)s over all distros This is the revision of the documentation about the varous nginx installation variants. It also implements the nginx installation scripts for morty and filtron. Signed-off-by: Markus Heiser --- docs/admin/filtron.rst | 17 ++++++- docs/admin/installation-nginx.rst | 102 ++++++++++++++++++++++---------------- 2 files changed, 76 insertions(+), 43 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/filtron.rst b/docs/admin/filtron.rst index 9615cd935..8bf641649 100644 --- a/docs/admin/filtron.rst +++ b/docs/admin/filtron.rst @@ -158,6 +158,12 @@ of: Route request through filtron ============================= +.. sidebar:: further reading + + - :ref:`filtron.sh overview` + - :ref:`installation nginx` + - :ref:`installation apache` + Filtron can be started using the following command: .. code:: sh @@ -171,15 +177,24 @@ Use it along with ``nginx`` with the following example configuration. .. code:: nginx - location / { + # https://example.org/searx + + location /searx { proxy_pass http://127.0.0.1:4004/; proxy_set_header Host $http_host; + proxy_set_header Connection $http_connection; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Scheme $scheme; + proxy_set_header X-Script-Name /searx; } + location /searx/static { + /usr/local/searx/searx-src/searx/static; + } + + Requests are coming from port 4004 going through filtron and then forwarded to port 8888 where a searx is being run. For a complete setup see: :ref:`nginx searx site`. diff --git a/docs/admin/installation-nginx.rst b/docs/admin/installation-nginx.rst index e62c60df6..4ecc54066 100644 --- a/docs/admin/installation-nginx.rst +++ b/docs/admin/installation-nginx.rst @@ -159,14 +159,22 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. code:: nginx - location / { - proxy_pass http://127.0.0.1:4004/; + # https://example.org/searx - proxy_set_header Host $http_host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Scheme $scheme; - } + location /searx { + proxy_pass http://127.0.0.1:4004/; + + proxy_set_header Host $http_host; + proxy_set_header Connection $http_connection; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_set_header X-Script-Name /searx; + } + + location /searx/static { + /usr/local/searx/searx-src/searx/static; + } 2. Configure reverse proxy for :ref:`morty `, listening on @@ -174,10 +182,13 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. code:: nginx - location /morty { + # https://example.org/morty + + location /morty { proxy_pass http://127.0.0.1:3000/; proxy_set_header Host $http_host; + proxy_set_header Connection $http_connection; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Scheme $scheme; @@ -197,7 +208,7 @@ Started wiki`_ is always a good resource *to keep in the pocket*. image_proxy : True - .. group-tab:: proxy or uWSGI + .. group-tab:: proxy or uWSGI Be warned, with this setup, your instance isn't :ref:`protected `. Nevertheless it is good enough for intranet usage and it is a @@ -208,15 +219,17 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. code:: nginx - location / { - proxy_pass http://127.0.0.1:8888; + # https://hostname.local/ - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Scheme $scheme; - proxy_set_header X-Script-Name /searx; - proxy_buffering off; - } + location / { + proxy_pass http://127.0.0.1:8888; + + proxy_set_header Host $host; + proxy_set_header Connection $http_connection; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_buffering off; + } Alternatively you can use the `uWSGI support from nginx`_ via unix sockets. For socket communication, you have to activate ``socket = @@ -234,8 +247,8 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. code:: nginx server { - # replace example.org with your server's public name - server_name example.org; + # replace hostname.local with your server's name + server_name hostname.local; listen 80; listen [::]:80; @@ -245,7 +258,7 @@ Started wiki`_ is always a good resource *to keep in the pocket*. uwsgi_pass unix:/run/uwsgi/app/searx/socket; } - root /usr/local/searx/searx-src/searx; + root /usr/local/searx/searx-src/searx; location /static { } } @@ -254,10 +267,10 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. code:: bash - mkdir -p /run/uwsgi/app/searx/ - sudo -H chown -R searx:searx /run/uwsgi/app/searx/ + mkdir -p /run/uwsgi/app/searx/ + sudo -H chown -R searx:searx /run/uwsgi/app/searx/ - .. group-tab:: proxy at subdir URL + .. group-tab:: \.\. at subdir URL Be warned, with these setups, your instance isn't :ref:`protected `. The examples are just here to demonstrate how to export the @@ -265,19 +278,22 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. code:: nginx - location /searx { - proxy_pass http://127.0.0.1:8888; + # https://hostname.local/searx - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Scheme $scheme; - proxy_set_header X-Script-Name /searx; - proxy_buffering off; - } + location /searx { + proxy_pass http://127.0.0.1:8888; + + proxy_set_header Host $host; + proxy_set_header Connection $http_connection; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Scheme $scheme; + proxy_set_header X-Script-Name /searx; + proxy_buffering off; + } - location /searx/static { - alias /usr/local/searx/searx-src/searx/static; - } + location /searx/static { + alias /usr/local/searx/searx-src/searx/static; + } The ``X-Script-Name /searx`` is needed by the searx implementation to calculate relative URLs correct. The next example shows a uWSGI @@ -286,15 +302,17 @@ Started wiki`_ is always a good resource *to keep in the pocket*. .. code:: nginx - location /searx/static { - alias /usr/local/searx/searx-src/searx; - } + # https://hostname.local/searx + + location /searx { + uwsgi_param SCRIPT_NAME /searx; + include uwsgi_params; + uwsgi_pass unix:/run/uwsgi/app/searx/socket; + } - location /searx { - uwsgi_param SCRIPT_NAME /searx; - include uwsgi_params; - uwsgi_pass unix:/run/uwsgi/app/searx/socket; - } + location /searx/static { + alias /usr/local/searx/searx-src/searx; + } For searx to work correctly the ``base_url`` must be set in the :origin:`searx/settings.yml`. -- cgit v1.2.3 From 30c0a0fb641f2248829a3ccc3fd6abe58a55c77f Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Mon, 13 Apr 2020 13:00:03 +0200 Subject: doc: add missing command lines to nginx docs Signed-off-by: Markus Heiser --- docs/admin/installation-nginx.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) (limited to 'docs/admin') diff --git a/docs/admin/installation-nginx.rst b/docs/admin/installation-nginx.rst index 4ecc54066..5e32d9684 100644 --- a/docs/admin/installation-nginx.rst +++ b/docs/admin/installation-nginx.rst @@ -32,6 +32,22 @@ Install with nginx :local: :backlinks: entry +---- + +**Install** :ref:`nginx searx site` using :ref:`filtron.sh ` + +.. code:: bash + + $ sudo -H ./utils/filtron.sh nginx install + +**Install** :ref:`nginx searx site` using :ref:`morty.sh ` + +.. code:: bash + + $ sudo -H ./utils/morty.sh nginx install + +---- + The nginx HTTP server ===================== -- cgit v1.2.3 From 26a3a7d52359075bb96b39b26ed60430f2c667b9 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Tue, 28 Apr 2020 14:44:28 +0200 Subject: [docs] installation: add note to update OS before install searx see: https://github.com/asciimoo/searx/pull/1803#issuecomment-618456661 Signed-off-by: Markus Heiser --- docs/admin/installation.rst | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'docs/admin') diff --git a/docs/admin/installation.rst b/docs/admin/installation.rst index bfc0742de..167c300fe 100644 --- a/docs/admin/installation.rst +++ b/docs/admin/installation.rst @@ -25,6 +25,10 @@ box ` as described below. Installation scripts ==================== +.. sidebar:: Update OS first! + + To avoid unwanted side effects, update your OS before installing searx. + The following will install a setup as shown in :ref:`architecture`. First you need to get a clone. The clone is only needed for the installation procedure and some maintenance tasks (alternatively you can create your own fork). -- cgit v1.2.3 From 6ff20cef7385d6827042f00d295f51bbc213afab Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Thu, 18 Jun 2020 18:31:46 +0200 Subject: [fix] indentation of filtron's rules (json) Signed-off-by: Markus Heiser --- docs/admin/filtron.rst | 179 +++++++++++++++++++++++-------------------------- 1 file changed, 85 insertions(+), 94 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/filtron.rst b/docs/admin/filtron.rst index 8bf641649..93e430b1f 100644 --- a/docs/admin/filtron.rst +++ b/docs/admin/filtron.rst @@ -57,100 +57,91 @@ of: .. code:: json - [ - { "name": "search request", - "filters": [ - "Param:q", - "Path=^(/|/search)$" - ], - "interval": "", - "limit": "", - "subrules": [ - { - "name": "roboagent limit", - "interval": "", - "limit": "", - "filters": [ - "Header:User-Agent=(curl|cURL|Wget|python-requests|Scrapy|FeedFetcher|Go-http-client)" - ], - "actions": [ - { "name": "log"}, - { "name": "block", - "params": { - "message": "Rate limit exceeded" - } - } - ] - }, - { - "name": "botlimit", - "limit": 0, - "stop": true, - "filters": [ - "Header:User-Agent=(Googlebot|bingbot|Baiduspider|yacybot|YandexMobileBot|YandexBot|Yahoo! Slurp|MJ12bot|AhrefsBot|archive.org_bot|msnbot|MJ12bot|SeznamBot|linkdexbot|Netvibes|SMTBot|zgrab|James BOT)" - ], - "actions": [ - { "name": "log"}, - { "name": "block", - "params": { - "message": "Rate limit exceeded" - } - } - ] - }, - { - "name": "IP limit", - "interval": "", - "limit": "", - "stop": true, - "aggregations": [ - "Header:X-Forwarded-For" - ], - "actions": [ - { "name": "log"}, - { "name": "block", - "params": { - "message": "Rate limit exceeded" - } - } - ] - }, - { - "name": "rss/json limit", - "interval": "", - "limit": "", - "stop": true, - "filters": [ - "Param:format=(csv|json|rss)" - ], - "actions": [ - { "name": "log"}, - { "name": "block", - "params": { - "message": "Rate limit exceeded" - } - } - ] - }, - { - "name": "useragent limit", - "interval": "", - "limit": "", - "aggregations": [ - "Header:User-Agent" - ], - "actions": [ - { "name": "log"}, - { "name": "block", - "params": { - "message": "Rate limit exceeded" - } - } - ] - } - ] - } - ] + [ + { + "name": "search request", + "filters": [ + "Param:q", + "Path=^(/|/search)$" + ], + "interval": "" + "limit": "", + "subrules": [ + { + "name": "missing Accept-Language", + "filters": ["!Header:Accept-Language"], + "limit": "", + "stop": true, + "actions": [ + {"name":"log"}, + {"name": "block", + "params": {"message": "Rate limit exceeded"}} + ] + }, + { + "name": "suspiciously Connection=close header", + "filters": ["Header:Connection=close"], + "limit": "", + "stop": true, + "actions": [ + {"name":"log"}, + {"name": "block", + "params": {"message": "Rate limit exceeded"}} + ] + }, + { + "name": "IP limit", + "interval": "" + "limit": "", + "stop": true, + "aggregations": [ + "Header:X-Forwarded-For" + ], + "actions": [ + { "name": "log"}, + { "name": "block", + "params": { + "message": "Rate limit exceeded" + } + } + ] + }, + { + "name": "rss/json limit", + "filters": [ + "Param:format=(csv|json|rss)" + ], + "interval": "" + "limit": "", + "stop": true, + "actions": [ + { "name": "log"}, + { "name": "block", + "params": { + "message": "Rate limit exceeded" + } + } + ] + }, + { + "name": "useragent limit", + "interval": "" + "limit": "", + "aggregations": [ + "Header:User-Agent" + ], + "actions": [ + { "name": "log"}, + { "name": "block", + "params": { + "message": "Rate limit exceeded" + } + } + ] + } + ] + } + ] .. _filtron route request: -- cgit v1.2.3 From 4f7a6c09dd1b799a53a25166b49d23adceccea7d Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Thu, 18 Jun 2020 18:39:56 +0200 Subject: [fix] buildhosts documentation Signed-off-by: Markus Heiser --- docs/admin/buildhosts.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/admin') diff --git a/docs/admin/buildhosts.rst b/docs/admin/buildhosts.rst index 04698d8d2..b1e787398 100644 --- a/docs/admin/buildhosts.rst +++ b/docs/admin/buildhosts.rst @@ -17,7 +17,7 @@ Buildhosts To get best results from build, its recommend to install additional packages on build hosts (see :ref:`searx.sh`).:: - sudo -H ./utils/searx.sh buildhost + sudo -H ./utils/searx.sh install buildhost This will install packages needed by searx: -- cgit v1.2.3 From bfef2c3b65aea8dd3485e0959e068d460577a841 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Thu, 18 Jun 2020 18:52:45 +0200 Subject: [fix] buildprocess of docs in LXC (containers) The $DOCS_BUILD variable (like all BUILD variables) in the lxc environment:: sudo -H ./utils/lxc.sh cmd searx-archlinux make docs is different from running build process in the HOST:: make docs with kernel-include directive we can use the environment variables in the reST documents to address the correct file location of the include. Signed-off-by: Markus Heiser --- docs/admin/buildhosts.rst | 4 ++-- docs/admin/installation-searx.rst | 14 +++++++------- docs/admin/installation-uwsgi.rst | 18 +++++++++--------- 3 files changed, 18 insertions(+), 18 deletions(-) (limited to 'docs/admin') diff --git a/docs/admin/buildhosts.rst b/docs/admin/buildhosts.rst index b1e787398..a727d25b9 100644 --- a/docs/admin/buildhosts.rst +++ b/docs/admin/buildhosts.rst @@ -21,13 +21,13 @@ on build hosts (see :ref:`searx.sh`).:: This will install packages needed by searx: -.. include:: ../../build/docs/includes/searx.rst +.. kernel-include:: $DOCS_BUILD/includes/searx.rst :start-after: START distro-packages :end-before: END distro-packages and packages needed to build docuemtation and run tests: -.. include:: ../../build/docs/includes/searx.rst +.. kernel-include:: $DOCS_BUILD/includes/searx.rst :start-after: START build-packages :end-before: END build-packages diff --git a/docs/admin/installation-searx.rst b/docs/admin/installation-searx.rst index d5c0063ee..f1d486021 100644 --- a/docs/admin/installation-searx.rst +++ b/docs/admin/installation-searx.rst @@ -17,7 +17,7 @@ universe repository. Install packages ================ -.. include:: ../../build/docs/includes/searx.rst +.. kernel-include:: $DOCS_BUILD/includes/searx.rst :start-after: START distro-packages :end-before: END distro-packages @@ -30,7 +30,7 @@ Install packages Create user =========== -.. include:: ../../build/docs/includes/searx.rst +.. kernel-include:: $DOCS_BUILD/includes/searx.rst :start-after: START create user :end-before: END create user @@ -41,13 +41,13 @@ install searx & dependencies Start a interactive shell from new created user and clone searx: -.. include:: ../../build/docs/includes/searx.rst +.. kernel-include:: $DOCS_BUILD/includes/searx.rst :start-after: START clone searx :end-before: END clone searx In the same shell create *virtualenv*: -.. include:: ../../build/docs/includes/searx.rst +.. kernel-include:: $DOCS_BUILD/includes/searx.rst :start-after: START create virtualenv :end-before: END create virtualenv @@ -55,7 +55,7 @@ To install searx's dependencies, exit the searx *bash* session you opened above and restart a new. Before install, first check if your *virualenv* was sourced from the login (*~/.profile*): -.. include:: ../../build/docs/includes/searx.rst +.. kernel-include:: $DOCS_BUILD/includes/searx.rst :start-after: START manage.sh update_packages :end-before: END manage.sh update_packages @@ -71,7 +71,7 @@ Create a copy of the :origin:`searx/settings.yml` configuration file in system's */etc* folder. Configure like shown below -- replace ``searx@\$(uname -n)`` with a name of your choice -- *and/or* edit ``/etc/searx/settings.yml`` if necessary. -.. include:: ../../build/docs/includes/searx.rst +.. kernel-include:: $DOCS_BUILD/includes/searx.rst :start-after: START searx config :end-before: END searx config @@ -82,7 +82,7 @@ To check your searx setup, optional enable debugging and start the *webapp*. Searx looks at the exported environment ``$SEARX_SETTINGS_PATH`` for a configuration file. -.. include:: ../../build/docs/includes/searx.rst +.. kernel-include:: $DOCS_BUILD/includes/searx.rst :start-after: START check searx installation :end-before: END check searx installation diff --git a/docs/admin/installation-uwsgi.rst b/docs/admin/installation-uwsgi.rst index 72498c0bf..ac4c463b9 100644 --- a/docs/admin/installation-uwsgi.rst +++ b/docs/admin/installation-uwsgi.rst @@ -104,21 +104,21 @@ restart the uwsgi application. .. group-tab:: Ubuntu / debian - .. literalinclude:: ../../build/docs/includes/searx.rst + .. kernel-include:: $DOCS_BUILD/includes/searx.rst :start-after: START searx uwsgi-description ubuntu-20.04 :end-before: END searx uwsgi-description ubuntu-20.04 .. group-tab:: Arch Linux - .. literalinclude:: ../../build/docs/includes/searx.rst + .. kernel-include:: $DOCS_BUILD/includes/searx.rst :start-after: START searx uwsgi-description arch :end-before: END searx uwsgi-description arch .. group-tab:: Fedora / RHEL - .. literalinclude:: ../../build/docs/includes/searx.rst + .. kernel-include:: $DOCS_BUILD/includes/searx.rst :start-after: START searx uwsgi-description fedora :end-before: END searx uwsgi-description fedora @@ -127,22 +127,22 @@ restart the uwsgi application. .. group-tab:: Ubuntu / debian - .. literalinclude:: ../../build/docs/includes/searx.rst - :language: ini + .. kernel-include:: $DOCS_BUILD/includes/searx.rst + :code: ini :start-after: START searx uwsgi-appini ubuntu-20.04 :end-before: END searx uwsgi-appini ubuntu-20.04 .. group-tab:: Arch Linux - .. literalinclude:: ../../build/docs/includes/searx.rst - :language: ini + .. kernel-include:: $DOCS_BUILD/includes/searx.rst + :code: ini :start-after: START searx uwsgi-appini arch :end-before: END searx uwsgi-appini arch .. group-tab:: Fedora / RHEL - .. literalinclude:: ../../build/docs/includes/searx.rst - :language: ini + .. kernel-include:: $DOCS_BUILD/includes/searx.rst + :code: ini :start-after: START searx uwsgi-appini fedora :end-before: END searx uwsgi-appini fedora -- cgit v1.2.3