| Age | Commit message (Collapse) | Author |
|---|
|
Docker buildx outputs the following error:
variable expansion is not supported for --from, define a new stage with FROM
using ARG from global scope as a workaround.
Also force BuildKit extension to be installed, legacy build is no longer
supported.
Closes https://github.com/searxng/searxng/issues/5219
|
|
* [enh] container: reproducible layers
We are not aiming for reproducibility compliance, but we look to make most
builder layers reproducible without caching at least for a short period of time
(until the builder's base image changes or the child dependencies of a
requirements.txt package are updated).
This feature is only available on Podman.
This targets https://github.com/searxng/searxng/pull/5086 main goal.
* [fix] misc: apply suggestions
Suggested: https://github.com/searxng/searxng/pull/5222#discussion_r2364630496
Suggested: https://github.com/searxng/searxng/pull/5222#discussion_r2364630511
* [enh] container: prevent useless layer
|
|
This commit replaces `pip` in container builds with `uv` pip compat
with a 1:1 parity. The only thing that changes is the installation speed of the
wheels, which seems to be considerably faster, although I haven't been able to
properly quantify this yet.
uv also gives us more tools to manage the cache. We can revert the prior cache
changes in `container.yml` as we won't have duplicated wheels anymore.
|
|
Building the container currently does not work properly.
When rebuilding several times with `make container`, `version_frozen.py`
is recreated, which wouldn't be an issue if the file’s timestamp was constant.
Now, when creating `version_frozen.py`, it will have the same timestamp as the
commit when it was created. (`version_frozen.py` is moved to a dedicated layer).
Reusing "builder" cache when building "dist" could be slow
(CD reports 2 seconds, but locally I've seen it take up to 10 seconds),
so the Dockerfile is now split and we save a couple steps
by importing the "builder" image directly.
The last changes made it possible to remove the layer cache in "builder",
since the overhead is now greater than building the layers from scratch.
Until now, all "dist" layers were squashed into a single layer,
which in most cases is a good idea
(except for storage/delivery pricing/overhead), but in our case,
since we manage the entire pipeline, we can ignore this
and share layers between builds.
This means (for example) that if we change files unrelated to the container
in several consecutive commits (documentation changes), we don't have to push
the entire image to registry, but only the different layers
(`version_frozen.py` in this example).
The same applies when pulling, as only the layers that have changed
compared to the local layers will be downloaded (that's the theory,
we'll see if this works as expected or if we need to tweak something else).
|
|
With this change, the "latest" tag will be visually higher (on registry tag list). Right now, it appears under the "DOCKER_TAG" manifest tag, which can be confusing.
|
|
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
* [mod] container: replace uWSGI with Granian
The configuration in Granian is handled with ENVs, much more convenient and practical for updating. The settings have been tested for over two months in a production instance, being usable on small to somewhat large instances without having to modify anything.
It also removes the patch functions and ENVs abstraction from the entrypoint, this makes it possible to run the container with immutable configuration.
In some setups, It may be desired to have the volumes/files under a specific uid/gid (other than searxng:searxng), if the entrypoint has root permissions it will chown automatically on every start, which may not be desired. Explicitly setting the new ENV `FORCE_OWNERSHIP=false` will prevent ownership from being modified.
No manual migration is necessary **unless** the user has changed the default uWSGI configuration or has a very specific setup.
Closes https://github.com/searxng/searxng/issues/4894
Closes https://github.com/searxng/searxng/issues/4818
Closes https://github.com/searxng/searxng/issues/4802
Supersedes https://github.com/searxng/searxng/pull/4596
Related https://github.com/searxng/searxng/discussions/4479
* [mod] docs: add container/granian
All container documentation has been recreated.
A new documentation page has been created for Granian.
* [enh] misc: apply suggestions
Minor documentation changes.
Suggested https://github.com/searxng/searxng/pull/4820#discussion_r2134539259
Suggested https://github.com/searxng/searxng/pull/4820#discussion_r2134538610
Suggested https://github.com/searxng/searxng/pull/4820#discussion_r2134827964
Suggested https://github.com/searxng/searxng/pull/4820#discussion_r2134544300
Suggested https://github.com/searxng/searxng/pull/4820#discussion_r2149387388
---------
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Co-authored-by: Ivan Gabaldon <igabaldon@inetol.net>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
This is a poorly designed instruction, which is hardcoded and cannot be easily modified or maintained on a rolling release sw like ours. This *should* be set in the SearXNG Docker Compose template, not in the image itself.
The OCI format is now used since we no longer have the HEALTHCHECK on the Dockerfile.
Closes https://github.com/searxng/searxng/issues/4906
Closes https://github.com/searxng/searxng/issues/4722
|
|
I'm not too pleased to reverse this, but issues like https://github.com/searxng/searxng/issues/4792 have not been foreseen, and we can't just turn away. It has become apparent over the last weeks that there are still quite a few people with an incompatible CPU or having SearXNG on some random VM provider who can't (or won't) modify the configuration of their machines to expose the features needed for x86_64v2 march.
As I don't want to trash the work with apko and base images, I thought about trying building Alpine again now that we have all the container related workflow refactored.
There will still be the discussion of whether to use musl and its drawbacks, but right now I don't know any other alternatives.
The nice part of this is that both Dockerfiles (mainline and legacy) can now be unified under the same umbrella again.
Closes https://github.com/searxng/searxng/issues/4792
Closes https://github.com/searxng/searxng/issues/4753
|
|
That entrypoint is prone to screw things up, especially with permission handling. The new script handles initialization better and fixes some issues like delayed settings update via ENVs and timestamp overwriting, also adjusts what should be copied into the container.
Related https://github.com/searxng/searxng/pull/4721#issuecomment-2850272129
|
|
Allows to push the manifests to other registries, this allows to push both docker.io and ghcr.io registries.
|
|
Currently, we have 1100~ cache images uploaded to GHCR that weigh more than 300 MB each (most of them are layers from the second phase of the Dockerfile that were uploaded by mistake, read below). To avoid problems, I have set up a new job in a new workflow to be run weekly purging all images older than 1 week, but leaving always the 100 most recent ones.
Only the builder images should be uploaded to cache, the actual behaviour not only slows down the time for building the container, but also wastes lots of space by saving large and useless layers to GHCR that will never be used again.
|
|
Suggested-by: @return42 https://github.com/searxng/searxng/pull/4764#discussion_r2083571202
|
|
Suggested-by: @return42 https://github.com/searxng/searxng/pull/4764#discussion_r2083564489
|
|
container.yml will run after integration.yml COMPLETES successfully and in master branch.
Style changes, cleanup and improved integration with CI by leveraging the use of
shared cache between all workflows.
* Podman is now supported to build the container images (Docker also received a refactor, merging both build and buildx)
* Container images are being built by Buildah instead of Docker BuildKit.
* Container images are tested before release.
* Splitting "modern" (amd64 & arm64) and "legacy" (armv7) arches on different Dockerfiles allowing future optimizations.
|
