summaryrefslogtreecommitdiff
path: root/docs/dev
diff options
context:
space:
mode:
authorMarkus Heiser <markus.heiser@darmarit.de>2025-07-16 15:47:36 +0200
committerMarkus Heiser <markus.heiser@darmarIT.de>2025-07-28 12:53:26 +0200
commitae0fcf3a4208401f5dc0296c29227a55f63163da (patch)
tree3f3612b982c1180064e944e1a83060f0161af57e /docs/dev
parent54a2b553f46df6dc98a4b8410788e35165e0fa3e (diff)
[chore] overdue maintenance of shell scripts
Removes obsolete scripts and fix various leftovers. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Diffstat (limited to 'docs/dev')
-rw-r--r--docs/dev/index.rst1
-rw-r--r--docs/dev/lxcdev.rst437
2 files changed, 0 insertions, 438 deletions
diff --git a/docs/dev/index.rst b/docs/dev/index.rst
index 6d601d166..82670284b 100644
--- a/docs/dev/index.rst
+++ b/docs/dev/index.rst
@@ -17,7 +17,6 @@ Developer documentation
plugins/index
answerers/index
translation
- lxcdev
makefile
reST
searxng_extra/index
diff --git a/docs/dev/lxcdev.rst b/docs/dev/lxcdev.rst
deleted file mode 100644
index 22c16ff0d..000000000
--- a/docs/dev/lxcdev.rst
+++ /dev/null
@@ -1,437 +0,0 @@
-.. _lxcdev:
-
-==============================
-Developing in Linux Containers
-==============================
-
-.. _LXC: https://linuxcontainers.org/lxc/introduction/
-
-In this article we will show, how you can make use of Linux Containers (LXC_) in
-*distributed and heterogeneous development cycles* (TL;DR; jump to the
-:ref:`lxcdev summary`).
-
-.. sidebar:: Audience
-
- This blog post is written for experienced admins and developers. Readers
- should have a serious meaning about the terms: *distributed*, *merge* and
- *linux container*.
-
- **hint**
-
- If you have issues with the internet connectivity of your containers read
- section :ref:`internet connectivity docker`.
-
-
-.. contents::
- :depth: 2
- :local:
- :backlinks: entry
-
-
-Motivation
-==========
-
-Most often in our development cycle, we edit the sources and run some test
-and/or builds by using ``make`` :ref:`[ref] <makefile>` before we commit. This
-cycle is simple and perfect but might fail in some aspects we should not
-overlook.
-
- **The environment in which we run all our development processes matters!**
-
-The :ref:`makefile` and the :ref:`make install` encapsulate a lot for us, but
-these tools do not have access to all prerequisites. For example, there may
-have dependencies on packages that are installed on developer's desktop, but
-usually are not preinstalled on a server or client system. Another example is;
-settings have been made to the software on developer's desktop that would never
-be set on a *production* system.
-
- **Linux Containers are isolate environments**, we use them to not mix up all
- the prerequisites from various projects on developer's desktop.
-
-The scripts from :ref:`searx_utils` can divide in those to install and maintain
-software
-
-- :ref:`searxng.sh`
-
-and the script
-
-- :ref:`lxc.sh`
-
-with we can scale our installation, maintenance or even development tasks over a
-stack of isolated containers / what we call the:
-
-- :ref:`searxng lxc suite`
-
-.. _lxcdev install searxng:
-
-Gentlemen, start your engines!
-==============================
-
-.. _LXD: https://linuxcontainers.org/lxd/introduction/
-.. _archlinux: https://www.archlinux.org/
-
-Before you can start with containers, you need to install and initiate LXD_
-once:
-
-.. tabs::
-
- .. group-tab:: desktop (HOST)
-
- .. code:: bash
-
- $ snap install lxd
- $ lxd init --auto
-
-And you need to clone from origin or if you have your own fork, clone from your
-fork:
-
-.. tabs::
-
- .. group-tab:: desktop (HOST)
-
- .. code:: bash
-
- $ cd ~/Downloads
- $ git clone https://github.com/searxng/searxng.git searxng
- $ cd searxng
-
-.. sidebar:: The ``searxng-archlinux`` container
-
- is the base of all our exercises here.
-
-The :ref:`lxc-searxng.env` consists of several images, see ``export
-LXC_SUITE=(...`` near by :origin:`utils/lxc-searxng.env#L19`.
-For this blog post we exercise on a archlinux_ image. The container of this
-image is named ``searxng-archlinux``.
-
-Lets build the container, but be sure that this container does not already
-exists, so first lets remove possible old one:
-
-.. tabs::
-
- .. group-tab:: desktop (HOST)
-
- .. code:: bash
-
- $ sudo -H ./utils/lxc.sh remove searxng-archlinux
- $ sudo -H ./utils/lxc.sh build searxng-archlinux
-
-
-.. sidebar:: further read
-
- - :ref:`lxc.sh install suite`
- - :ref:`installation nginx`
-
-To install the complete :ref:`SearXNG suite <searxng lxc suite>` and the HTTP
-proxy :ref:`installation nginx` into the archlinux container run:
-
-.. tabs::
-
- .. group-tab:: desktop (HOST)
-
- .. code:: bash
-
- $ sudo -H ./utils/lxc.sh install suite searxng-archlinux
- $ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
- $ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL
- ...
- [searxng-archlinux] SEARXNG_URL : http://n.n.n.140/searxng
-
-.. sidebar:: Fully functional SearXNG suite
-
- From here on you have a fully functional SearXNG suite (including a
- :ref:`valkey db`).
-
-In such a SearXNG suite admins can maintain and access the debug log of the
-services quite easy.
-
-In the example above the SearXNG instance in the container is wrapped to
-``http://n.n.n.140/searxng`` to the HOST system. Note, on your HOST system, the
-IP of your ``searxng-archlinux`` container is different to this example. To
-test the instance in the container from outside of the container, in your WEB
-browser on your desktop just open the URL reported in your installation
-
-.. _working in containers:
-
-In containers, work as usual
-============================
-
-Usually you open a root-bash using ``sudo -H bash``. In case of LXC containers
-open the root-bash in the container is done by the ``./utils/lxc.sh cmd
-searxng-archlinux`` command:
-
-.. tabs::
-
- .. group-tab:: desktop (HOST)
-
- .. code:: bash
-
- $ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
- INFO: [searxng-archlinux] bash
- [root@searxng-archlinux SearXNG]$
-
-The prompt ``[root@searxng-archlinux ...]`` signals, that you are the root user
-in the container (GUEST). To debug the running SearXNG instance use:
-
-.. tabs::
-
- .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
-
- .. code:: bash
-
- $ ./utils/searxng.sh instance inspect
- ...
- use [CTRL-C] to stop monitoring the log
- ...
-
- .. group-tab:: desktop (HOST)
-
- .. code:: bash
-
- $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance inspect
- ...
- use [CTRL-C] to stop monitoring the log
- ...
-
-
-Back in the browser on your desktop open the service http://n.n.n.140/searxng
-and run your application tests while the debug log is shown in the terminal from
-above. You can stop monitoring using ``CTRL-C``, this also disables the *"debug
-option"* in SearXNG's settings file and restarts the SearXNG uwsgi application.
-
-Another point we have to notice is that the service :ref:`SearXNG <searxng.sh>`
-runs under dedicated system user account with the same name (compare
-:ref:`create searxng user`). To get a login shell from these accounts, simply
-call:
-
-.. tabs::
-
- .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
-
- .. code:: bash
-
- $ ./utils/searxng.sh instance cmd bash -l
- (searx-pyenv) [searxng@searxng-archlinux ~]$ pwd
- /usr/local/searxng
-
- .. group-tab:: desktop (HOST)
-
- .. code:: bash
-
- $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance cmd bash -l
- INFO: [searxng-archlinux] ./utils/searxng.sh instance cmd bash -l
- (searx-pyenv) [searxng@searxng-archlinux ~]$ pwd
- /usr/local/searxng
-
-The prompt ``[searxng@searxng-archlinux]`` signals that you are logged in as system
-user ``searxng`` in the ``searxng-archlinux`` container and the python *virtualenv*
-``(searxng-pyenv)`` environment is activated.
-
-
-Wrap production into developer suite
-====================================
-
-In this section we will see how to change the *"Fully functional SearXNG suite"*
-from a LXC container (which is quite ready for production) into a developer
-suite. For this, we have to keep an eye on the :ref:`installation basic`:
-
-- SearXNG setup in: ``/etc/searxng/settings.yml``
-- SearXNG user's home: ``/usr/local/searxng``
-- virtualenv in: ``/usr/local/searxng/searxng-pyenv``
-- SearXNG software in: ``/usr/local/searxng/searxng-src``
-
-With the use of the :ref:`searxng.sh` the SearXNG service was installed as
-:ref:`uWSGI application <searxng uwsgi>`. To maintain this service, we can use
-``systemctl`` (compare :ref:`uWSGI maintenance`).
-
-.. tabs::
-
- .. group-tab:: uwsgi@searxng
-
- .. code:: bash
-
- $ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl stop uwsgi@searxng
-
-With the command above, we stopped the SearXNG uWSGI-App in the archlinux
-container.
-
-The uWSGI-App for the archlinux distros is configured in
-:origin:`utils/templates/etc/uwsgi/apps-archlinux/searxng.ini`, from where at
-least you should attend the settings of ``uid``, ``chdir``, ``env`` and
-``http``::
-
- env = SEARXNG_SETTINGS_PATH=/etc/searxng/settings.yml
- http = 127.0.0.1:8888
-
- chdir = /usr/local/searxng/searxng-src/searx
- virtualenv = /usr/local/searxng/searxng-pyenv
- pythonpath = /usr/local/searxng/searxng-src
-
-If you have read the :ref:`Good to know` you remember, that each container
-shares the root folder of the repository and the command ``utils/lxc.sh cmd``
-handles relative path names **transparent**.
-
-To wrap the SearXNG installation in the container into a developer one, we
-simple have to create a symlink to the **transparent** repository from the
-desktop. Now lets replace the repository at ``searxng-src`` in the container
-with the working tree from outside of the container:
-
-.. tabs::
-
- .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
-
- .. code:: bash
-
- $ mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
- $ ln -s /share/SearXNG/ /usr/local/searxng/searxng-src
-
- .. group-tab:: desktop (HOST)
-
- .. code:: bash
-
- $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
- mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
-
- $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
- ln -s /share/SearXNG/ /usr/local/searxng/searxng-src
-
-Now we can develop as usual in the working tree of our desktop system. Every
-time the software was changed, you have to restart the SearXNG service (in the
-container):
-
-.. tabs::
-
- .. group-tab:: uwsgi@searxng
-
- .. code:: bash
-
- $ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl restart uwsgi@searxng
-
-
-Remember: :ref:`working in containers` .. here are just some examples from my
-daily usage:
-
-To *inspect* the SearXNG instance (already described above):
-
-.. tabs::
-
- .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
-
- .. code:: bash
-
- $ ./utils/searxng.sh inspect service
-
- .. group-tab:: desktop (HOST)
-
- .. code:: bash
-
- $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh inspect service
-
-Run :ref:`makefile`, e.g. to test inside the container:
-
-.. tabs::
-
- .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
-
- .. code:: bash
-
- $ make test
-
- .. group-tab:: desktop (HOST)
-
- .. code:: bash
-
- $ sudo -H ./utils/lxc.sh cmd searxng-archlinux make test
-
-
-
-To install all prerequisites needed for a :ref:`buildhosts`:
-
-.. tabs::
-
- .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
-
- .. code:: bash
-
- $ ./utils/searxng.sh install buildhost
-
- .. group-tab:: desktop (HOST)
-
- .. code:: bash
-
- $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh install buildhost
-
-
-To build the docs on a buildhost :ref:`buildhosts`:
-
-.. tabs::
-
- .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
-
- .. code:: bash
-
- $ make docs.html
-
- .. group-tab:: desktop (HOST)
-
- .. code:: bash
-
- $ sudo -H ./utils/lxc.sh cmd searxng-archlinux make docs.html
-
-
-.. _lxcdev summary:
-
-Summary
-=======
-
-We build up a fully functional SearXNG suite in a archlinux container:
-
-.. code:: bash
-
- $ sudo -H ./utils/lxc.sh build searxng-archlinux
- $ sudo -H ./utils/lxc.sh install suite searxng-archlinux
- ...
- Developer install? (wraps source from HOST into the running instance) [YES/no]
-
-To wrap the suite into a developer one answer ``YES`` (or press Enter).
-
-.. code:: text
-
- link SearXNG's sources to: /share/SearXNG
- =========================================
-
- mv -f "/usr/local/searxng/searxng-src" "/usr/local/searxng/searxng-src.backup"
- ln -s "/share/SearXNG" "/usr/local/searxng/searxng-src"
- ls -ld /usr/local/searxng/searxng-src
- |searxng| lrwxrwxrwx 1 searxng searxng ... /usr/local/searxng/searxng-src -> /share/SearXNG
-
-On code modification the instance has to be restarted (see :ref:`uWSGI
-maintenance`):
-
-.. code:: bash
-
- $ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl restart uwsgi@searxng
-
-To access HTTP from the desktop we installed nginx for the services inside the
-container:
-
-.. code:: bash
-
- $ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
-
-To get information about the SearxNG suite in the archlinux container we can
-use:
-
-.. code:: text
-
- $ sudo -H ./utils/lxc.sh show suite searxng-archlinux
- [searxng-archlinux] INFO: (eth0) docs-live: http:///n.n.n.140:8080/
- [searxng-archlinux] INFO: (eth0) IPv6: http://[fd42:555b:2af9:e121:216:3eff:fe5b:1744]
- [searxng-archlinux] uWSGI:
- [searxng-archlinux] SEARXNG_UWSGI_SOCKET : /usr/local/searxng/run/socket
- [searxng-archlinux] environment /usr/local/searxng/searxng-src/utils/brand.env:
- [searxng-archlinux] GIT_URL : https://github.com/searxng/searxng
- [searxng-archlinux] GIT_BRANCH : master
- [searxng-archlinux] SEARXNG_URL : http:///n.n.n.140/searxng
- [searxng-archlinux] SEARXNG_PORT : 8888
- [searxng-archlinux] SEARXNG_BIND_ADDRESS : 127.0.0.1