summaryrefslogtreecommitdiff
path: root/Makefile
AgeCommit message (Collapse)Author
2025-03-01[fix] wikidata: increase wikidata queries timeoutMarkus Heiser
The big queries for initializing and updating the currencies take longer than the default of the wikidata engine, which is only 3sec. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-02-28[web-client] integrate the migrated theme in the github CIMarkus Heiser
- upgrade to ubuntu-24.04 to get NodeJS v20 - remove DEBUG (V=1) environment / in a ViteJS env the DEBUG environment is reserved for interactive debugging tasks (not for verbose build messages). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-02-28[web-client] integrate the migrated theme in the development workflowsMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-01-28[refactor] typification of SearXNG (initial) / result items (part 1)Markus Heiser
Typification of SearXNG ======================= This patch introduces the typing of the results. The why and how is described in the documentation, please generate the documentation .. $ make docs.clean docs.live and read the following articles in the "Developer documentation": - result types --> http://0.0.0.0:8000/dev/result_types/index.html The result types are available from the `searx.result_types` module. The following have been implemented so far: - base result type: `searx.result_type.Result` --> http://0.0.0.0:8000/dev/result_types/base_result.html - answer results --> http://0.0.0.0:8000/dev/result_types/answer.html including the type for translations (inspired by #3925). For all other types (which still need to be set up in subsequent PRs), template documentation has been created for the transition period. Doc of the fields used in Templates =================================== The template documentation is the basis for the typing and is the first complete documentation of the results (needed for engine development). It is the "working paper" (the plan) with which further typifications can be implemented in subsequent PRs. - https://github.com/searxng/searxng/issues/357 Answer Templates ================ With the new (sub) types for `Answer`, the templates for the answers have also been revised, `Translation` are now displayed with collapsible entries (inspired by #3925). !en-de dog Plugins & Answerer ================== The implementation for `Plugin` and `Answer` has been revised, see documentation: - Plugin: http://0.0.0.0:8000/dev/plugins/index.html - Answerer: http://0.0.0.0:8000/dev/answerers/index.html With `AnswerStorage` and `AnswerStorage` to manage those items (in follow up PRs, `ArticleStorage`, `InfoStorage` and .. will be implemented) Autocomplete ============ The autocompletion had a bug where the results from `Answer` had not been shown in the past. To test activate autocompletion and try search terms for which we have answerers - statistics: type `min 1 2 3` .. in the completion list you should find an entry like `[de] min(1, 2, 3) = 1` - random: type `random uuid` .. in the completion list, the first item is a random UUID Extended Types ============== SearXNG extends e.g. the request and response types of flask and httpx, a module has been set up for type extensions: - Extended Types --> http://0.0.0.0:8000/dev/extended_types.html Unit-Tests ========== The unit tests have been completely revised. In the previous implementation, the runtime (the global variables such as `searx.settings`) was not initialized before each test, so the runtime environment with which a test ran was always determined by the tests that ran before it. This was also the reason why we sometimes had to observe non-deterministic errors in the tests in the past: - https://github.com/searxng/searxng/issues/2988 is one example for the Runtime issues, with non-deterministic behavior .. - https://github.com/searxng/searxng/pull/3650 - https://github.com/searxng/searxng/pull/3654 - https://github.com/searxng/searxng/pull/3642#issuecomment-2226884469 - https://github.com/searxng/searxng/pull/3746#issuecomment-2300965005 Why msgspec.Struct ================== We have already discussed typing based on e.g. `TypeDict` or `dataclass` in the past: - https://github.com/searxng/searxng/pull/1562/files - https://gist.github.com/dalf/972eb05e7a9bee161487132a7de244d2 - https://github.com/searxng/searxng/pull/1412/files - https://github.com/searxng/searxng/pull/1356 In my opinion, TypeDict is unsuitable because the objects are still dictionaries and not instances of classes / the `dataclass` are classes but ... The `msgspec.Struct` combine the advantages of typing, runtime behaviour and also offer the option of (fast) serializing (incl. type check) the objects. Currently not possible but conceivable with `msgspec`: Outsourcing the engines into separate processes, what possibilities this opens up in the future is left to the imagination! Internally, we have already defined that it is desirable to decouple the development of the engines from the development of the SearXNG core / The serialization of the `Result` objects is a prerequisite for this. HINT: The threads listed above were the template for this PR, even though the implementation here is based on msgspec. They should also be an inspiration for the following PRs of typification, as the models and implementations can provide a good direction. Why just one commit? ==================== I tried to create several (thematically separated) commits, but gave up at some point ... there are too many things to tackle at once / The comprehensibility of the commits would not be improved by a thematic separation. On the contrary, we would have to make multiple changes at the same places and the goal of a change would be vaguely recognizable in the fog of the commits. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-01-16[fix] build workflows of the themesMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-03Fix make search.checkerAlexandre Flament
2024-02-20[fix] sort RTL_LOCALES before written into locales.jsonMarkus Heiser
To avoid unnecessary changes to the file, the list should be sorted before it is written to the file. You can test it by calling multiple times:: make data.locales and searx/data/locales.json should be unchanged. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-02-16[fix] remove twine from requirements-devMarkus Heiser
SearXNG is a rolling release / we do not deploy packages on PyPi Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-01-09[mod] get rid of ./utils/brand.env and its workflowMarkus Heiser
All the environments defined in ./utils/brand.env are generated on the fly, so there is no longer a need to define the brand environment in this file and all the workflows to handle this file. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-06[doc] improve documentation of make targets and ./manage scriptMarkus Heiser
BTW force modularization of the ./mange script into sub modules: - utils/lib_sxng_data.sh - utils/lib_sxng_node.sh - utils/lib_sxng_static.sh - utils/lib_sxng_test.sh - utils/lib_sxng_themes.sh Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-04-05[fix] ./manage: replace data.languages by data.traitsMarkus Heiser
Follow up of https://github.com/searxng/searxng/pull/2269 Related: - https://github.com/searxng/searxng/issues/2314 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-11[mod] install developer tools by: make node.env.devMarkus Heiser
- rename ``./manage node.env.devtools`` to ``./manage node.env.dev`` - add ``make node.env.dev`` - remove local developer tools by ``make node.clean`` Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-30[clean up] drop obsolete searx, filtron and morty install scriptsMarkus Heiser
Since ./utils/searxng.sh is implemented, the old installation procedures from filtron, morty and searx can be removed. For users who want to upgrade, the procedures for removing old installations have still been retained. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-30[utils/searxng.sh] implement new script to install SearXNGMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-04-30[theme] peel out oscar from SearXNG developmentMarkus Heiser
This is the first step of removing oscar theme Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-04-23[mod] add test.pybabel to cover issues in translation tasksMarkus Heiser
We need to cover issue related to pybabel/translation tasks. By example there was an issue [1] (fixed [2]) when upgrading jinja2 from 3.0.3 to 3.1.0 [3] that has not been covered by the CI tests. [1] https://github.com/searxng/searxng/runs/5688624325?check_suite_focus=true#step:6:348 [2] https://github.com/searxng/searxng/pull/1011 [3] https://github.com/searxng/searxng/pull/1008 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-23[mod] add test.pyright to test & ci.test targetsMartin Fischer
Since we currently have many type checking errors, we for now only test with typeCheckingMode: off which makes pyright only check files that contain a comment: # pyright: basic to enable basic type checking, or # pyright: strict to enable strict type checking.
2022-01-23[mod] manage test.pyright: a commandline to run pyright testsMarkus Heiser
This patch implements the command and Makefile target:: ./manage test.pyright make test.pyright Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-17[mod] CI target test.rst: test reST markup of README.rstMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-11Merge pull request #686 from return42/lib_redisMarkus Heiser
Add redis DB and connector
2022-01-04Merge pull request #664 from return42/themes.liveMarkus Heiser
[mod] manage: add themes.live command (rebuild on modification)
2022-01-03[mod] test.shell: add utils/lib_redis.sh to shellcheck procedureMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-03[fix] Makefile: mixed tab & space indentationMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-29[mod] live build of a theme: LIVE_THEME=simple make runMarkus Heiser
Environment variable to get live builds while modifying CSS & JS of a theme:: LIVE_THEME=simple make run Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-27[mod] replace pycodestyle by blackAlexandre Flament
"make test.black" checks for the code style "make format.python" format the python code
2021-12-14[simple theme] activate - SearXNG JavaScript Style GuideMarkus Heiser
Add the 'SearXNG JavaScript Style Guide' as one quality gate in the build chain of the simple theme:: make themes.simple make themes.simple.test In the github CI the `themes.all` target enforce a `themes.simple`. BTW: Remove 'jshint' left overs from 0ee316f3d Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-18Merge pull request #508 from return42/arm64Markus Heiser
[mod] Tools to install and maintain golang binaries & packages.
2021-11-17[mod] Makefile - remove targets marked as deprecatedMarkus Heiser
Remove targets marked as deprecated and convert selected to short hands [1]. [1] https://github.com/searxng/searxng/pull/511#issuecomment-971324265 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-16[mod] Tools to install and maintain NVM versions manager for Node.jsMarkus Heiser
[1] https://github.com/nvm-sh/nvm Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-16[mod] Tools to install and maintain golang [1] binaries & packages.Markus Heiser
[1] https://golang.org/doc/devel/release#policy A simple *helloworld* test with user 'my_user' : sudo -H adduser my_user ./manage go.golang go1.17.3 my_user ./manage go.install github.com/go-training/helloworld@latest my_user ./manage go.bash my_user $ helloword Hello World!! Don't forget to remove 'my_user': sudo -H deluser --remove-home my_user Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-02SearXNG: environment variablesAlexandre Flament
2021-10-01SearXNG: .github, searxng-dockerAlexandre Flament
2021-09-10[mod] debug mode: use coloredlogs packageAlexandre Flament
in debug mode, add some padding to make the output more readable in procution mode, add the timestamp
2021-08-16[translations] ./manage rename shell functions to integrate weblateMarkus Heiser
Functions implemented to run *Weblate* workflows should use prefix 'weblate.': - babel.setup.translations.worktree --> weblate.translations.worktree Create git worktree ${TRANSLATIONS_WORKTREE} and checkout branch 'translations' from Weblate's counterpart (weblate) of the SearXNG (origin):: remote weblate https://weblate.bubu1.eu/git/searxng/searxng/ - babel.weblate.to.translations --> weblate.to.translations Update 'translations' branch of SearXNG (origin) with last additions from Weblate. - babel.translations.to.master --> weblate.translations.commit Update 'translations' branch of SearXNG (origin) with last additions from Weblate. Copy the changes to the master branch, compile translations and create a commit in the local branch (master) - babel.master.to.translations --> weblate.push.translations Push *translation changes* from SearXNG (origin) to Weblate's counterpart (weblate). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-08-07[translations] web integrationAlexandre Flament
* make babel.translations.to.master: pull weblate updates * make babel.master.to.translations: push .pot and .po files to weblate
2021-08-02[mod] manage - implement babel commands, drop update_translations.shMarkus Heiser
In ./manage implement babel.*: - extract : extract messages from source files and generate POT file - update : update existing message catalogs from POT file - compile : compile translation catalogs into binary MO files Replace searx_extra/update/update_translations.sh by command: - ci.babel.update Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-21[mod] load .config.sh from installation (utils/lib_install.sh)Markus Heiser
**new** utils/lib_install.sh: Used to initialize installation procedures - Modified source_dot_config function that - loads .config.sh from an existing installation (at SEARX_SRC). - initialize **SEARX_SRC_INIT_FILES** - functions like: - install_log_searx_instance() - install_searx_get_state() **modified** utils/searx.sh - obsolete environment SEARX_INSTANCE_NAME has been replaced by string 'SearXNG'. **modified** utils/filtron.sh, utils/morty.sh, utils/searx.sh - source utils/lib_install.sh - normalize logging of environment variables using new function install_log_searx_instance() **modified** utils/lib.sh - fix marginal typos **Installation scripts** The utils/lib_install.sh is sourced by the installations scripts: - utils/searx.sh - utils/morty.sh - utils/filtron.sh If '${SEARX_SRC}/.config.sh' exists, the modified source_dot_config() function loads this configuration (instead of './.config.sh'). **SEARX_SRC_INIT_FILES** Array of file names to sync into a installation at $SEARX_SRC. The file names are relative to the $REPO_ROOT. Set by function init_SEARX_SRC_INIT_FILES(). Most often theses are files like: - .config.sh - searx/settings.yml - utils/brand.env - ... Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-03[mod] add Makfile target lxc.clean which deletes ./lxc-envMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-26[mod] move functions from utils/manage_static.sh to ./manage scriptMarkus Heiser
The functions: - static.build.commit - static.build.commit.drop - static.build.restore are imported into the ./manage script. To avoid name collisions some variables and fucntions has been renamed by adding the prefix *static_*. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-26[mod] utils/manage_static.sh: add script to shellcheckMarkus Heiser
- add script to shellcheck, - fix error messages from shellcheck and - moved global variables to local variables (lower case) No functional change! Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-26[mod] add make targets to manage the build files in the /searx/static directoryAlexandre Flament
The idea is to avoid conflict when there are differents branches with changes are made on the static files. A solution is to ask the administrators to build the files from the sources, but it requires to install the npm dependencies. So the solution in this commit keep the sources and the build files in the same git repository. In one branch, the modification of the source (*.less, *.js) are commited without the built files. The built files are commited in a uniq commit, with a commit message "Static build" In case of merge or rebase, this commit can be dropped. New make targets: * static.build.commit.drop: drop the last "Static build". The command checks that there are only build files in the commit. * static.build.commit : call "make static.build.commit.drop" call "make themes.all" commit the files * static.git.restore.staged: git restore --staged <build files> * static.git.restore: git restore <build files> Related to https://github.com/searxng/searxng/issues/137
2021-06-22[fix] Makefile: remove themes.bootstrapAlexandre Flament
2021-06-22[fix] pygments.less - remove pyenv_OK conditionMarkus Heiser
BTW: add pygments.less to the help message Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-09[data.all] add searx_extra/update/update_osm_keys_tags.pyMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05[enh] add test.yamllint - lint yaml filesMarkus Heiser
Usage:: make test.yamllint ./manage test.yamllint test.yamllint is also added to the `test` makefile target. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-17[mod] docker-entrypoint.sh - add script to make test.sh (shellcheck)Markus Heiser
Add script docker-entrypoint.sh to shellcheck and try to _simplify_ and _normalize_ some parts: - fix issues reported by shellcheck - don't mix tab and space indent - command 'help' replaced by '-h': ./dockerfiles/docker-entrypoint.sh -h - replace printf in help() by 'cat <<EOF' Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-28[docker] multiarch support: linux/amd64,linux/arm64,linux/arm/v7Alexandre Flament
make docker.buildx : build and push multiarch build. (it can't be only build) use buildx with the --cache-from and --cache-to options to cache the layers (only the last built is cached)
2021-04-23[fix] Makefile fix target 'test' & 'test.sh'Markus Heiser
Renamed 'test.sh' to 'test.shell' seems to solve this issue on my ubu204:: $ make test.sh ... TEST robot ... INSTALL gecko.driver ... Running 7 tests cat test.sh >test cat: test.sh: Datei oder Verzeichnis nicht gefunden make: *** [<eingebaut>: test] Fehler 1 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-23[mod] replace makefile boilerplate by 'manage' scriptMarkus Heiser
Replaces the make targets with the bash scripts Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-22Merge pull request #15 from return42/add-springerAlexandre Flament
Add a search engine for Springer Nature
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-03 21:15:16 +0000