| Age | Commit message (Collapse) | Author |
|---|
|
- pyright configuration [1]_
- stub files: types-lxml [2]_
- addition of various type hints
- enable use of new type system features on older Python versions [3]_
- ``.tool-versions`` - set python to lowest version we support (3.10.18) [4]_:
Older versions typically lack some typing features found in newer Python
versions. Therefore, for local type checking (before commit), it is necessary
to use the older Python interpreter.
.. [1] https://docs.basedpyright.com/v1.20.0/configuration/config-files/
.. [2] https://pypi.org/project/types-lxml/
.. [3] https://typing-extensions.readthedocs.io/en/latest/#
.. [4] https://mise.jdx.dev/configuration.html#tool-versions
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Format: reST
|
|
Repology_ monitors a huge number of package repositories and other sources
comparing packages versions across them and gathering other information.
Repology_ shows you in which repositories a given project is packaged, which
version is the latest and which needs updating, who maintains the package, and
other related information.
.. _Repology: https://repology.org/docs/about
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
Format: reST
|
|
This patch adds a new result type: Code
- Python class: searx/result_types/code.py
- Jinja template: searx/templates/simple/result_templates/code.html
- CSS (less) client/simple/src/less/result_types/code.less
Signed-of-by: Markus Heiser <markus.heiser@darmarIT.de>
|
|
The plugin uses the ``GeoLocation`` class, which is already implemented in the
context of weather forecasts, to determine the time zone. The ``DateTime`` class
is used for the localized display of date and time.
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
|
|
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Format: reST
|
|
TL;DR; For all the issues that comes with HTTP POST I recommend instance
maintainers to switch to GET and lock the property in the preferences:
```yaml
server:
method: GET
preferences:
lock:
- method
```
We don't want this in the defaults of the SearXNG distributions for the pros vs
cons listed in this discussion:
- https://github.com/searxng/searxng/pull/3619
|
|
- Adds a new engine `searx/engines/openalex.py` that integrates the OpenAlex
Works API to return scientific paper results using the `paper.html` template.
- Uses the official API (no auth required); supports OpenAlex polite pool via `mailto`.
- Adds developer docs at `docs/dev/engines/online/openalex.rst`.
OpenAlex API reference: https://docs.openalex.org/how-to-use-the-api/api-overview
|
|
This patch adds GitHub Code Search [1] engine to allow querying the codebases.
Template code.html is changed to allow passthrough of strip and highlighting
options.
Engine Searchcode is adjusted to pass filename and not rely on hardcoded
extensions.
GitHub search code API does not return the exact code line indices, this
implementation assigns the code arbitrary numbers starting from 1
(effectively relabeling the code).
The API allows for unauth calls, and the default engine settings default to
that, although the calls are heavily rate limited.
The 'text' lexer is the default pygments lexer when parsing fails.
[1] https://docs.github.com/en/rest/search/search?apiVersion=2022-11-28#search-code
Co-authored-by: Markus Heiser <markus.heiser@darmarIT.de>
|
|
TypeScript is a superset of JavaScript, converting the entire theme to
TypeScript allows us to receive much more feedback on possible issues made in
package updates or our own typos, furthermore, it allows to transpile properly
to lower specs. This PR couldn't be done in smaller commits, a lot of work
needed to make everything *work properly*:
- A browser baseline has been set that requires minimum **Chromium 93, Firefox
92 and Safari 15** (proper visuals/operation on older browser versions is not
guaranteed)
- LightningCSS now handles minification and prefix creation for CSS.
- All hardcoded polyfills and support for previous browser baseline versions
have been removed.
- Convert codebase to TypeScript.
- Convert IIFE to ESM, handling globals with IIFE is cumbersome, ESM is the
standard for virtually any use of JS nowadays.
- Vite now builds the theme without the need for `vite-plugin-static-copy`.
- `searxng.ready` now accepts an array of conditions for the callback to be
executed.
- Replace `leaflet` with `ol` as there were some issues with proper Vite
bundling.
- Merged `head` with `main` script, as head was too small now.
- Add `assertElement` to properly check the existence of critical DOM elements.
- `searxng.on` renamed to `searxng.listen` with some handling improvements.
|
|
Related:
- https://github.com/searxng/searxng/pull/5037#issuecomment-3166851578
|
|
To get an API key follow instructions at [1].
[1] https://about.marginalia-search.com/article/api/
Related (historical ordered):
- https://github.com/searxng/searxng/issues/1620
- https://github.com/searxng/searxng/issues/1673
- https://github.com/searxng/searxng/pull/1627
- https://github.com/searxng/searxng/pull/2489
Closes:
- https://github.com/searxng/searxng/issues/3034
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Building the container currently does not work properly.
When rebuilding several times with `make container`, `version_frozen.py`
is recreated, which wouldn't be an issue if the file’s timestamp was constant.
Now, when creating `version_frozen.py`, it will have the same timestamp as the
commit when it was created. (`version_frozen.py` is moved to a dedicated layer).
Reusing "builder" cache when building "dist" could be slow
(CD reports 2 seconds, but locally I've seen it take up to 10 seconds),
so the Dockerfile is now split and we save a couple steps
by importing the "builder" image directly.
The last changes made it possible to remove the layer cache in "builder",
since the overhead is now greater than building the layers from scratch.
Until now, all "dist" layers were squashed into a single layer,
which in most cases is a good idea
(except for storage/delivery pricing/overhead), but in our case,
since we manage the entire pipeline, we can ignore this
and share layers between builds.
This means (for example) that if we change files unrelated to the container
in several consecutive commits (documentation changes), we don't have to push
the entire image to registry, but only the different layers
(`version_frozen.py` in this example).
The same applies when pulling, as only the layers that have changed
compared to the local layers will be downloaded (that's the theory,
we'll see if this works as expected or if we need to tweak something else).
|
|
Implement rules and functions to format shell scripts:
$ make format.shell
or alternatively to format all source code:
$ make format
The formatter `shfmt` reads the rules from the editorconfig[1]
If any EditorConfig files are found, they will be used to apply formatting
options. If any parser or printer flags are given to the tool, no
EditorConfig files will be used.
[1] https://github.com/patrickvane/shfmt?tab=readme-ov-file#description
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Removes obsolete scripts and fix various leftovers.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
In customizing it should be decided which plugin modules should be loaded and
which should not.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Cache busting has caused serious problems for users in the past, here are two
examples:
- https://github.com/searxng/searxng/issues/4419
- https://github.com/searxng/searxng/issues/4481
And it makes development and deployment significantly more complex because it
binds the client side to the server side:
- https://github.com/searxng/searxng/pull/4466
In the light of a decoupled development of the WEB clients from the server side:
- https://github.com/searxng/searxng/pull/4988
is it appropriate to abandon this feature. In fact, it has been ineffective
since #4436 anyway.
However, the benefit has always been questionable, since at best only a few kB
of data are saved (at least in the context of an image_proxy, the effect is below
the detection limit). Ultimately, the client is responsible for caching.
Related: https://github.com/searxng/searxng/issues?q=label%3A%22clear%20browser%20cache%22
Closes: https://github.com/searxng/searxng/pull/4466
Closes: https://github.com/searxng/searxng/issues/1326
Closes: https://github.com/searxng/searxng/issues/964
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
This patch migrates from `redis==5.2.1` [1] to `valkey==6.1.0` [2].
The migration to valkey is necessary because the company behind Redis has decided
to abandon the open source license. After experiencing a drop in user numbers,
they now want to run it under a dual license again. But this move demonstrates
once again how unreliable the company is and how it treats open source
developers.
To review first, read the docs::
$ make docs.live
Follow the instructions to remove redis:
- http://0.0.0.0:8000/admin/settings/settings_redis.html
Config and install a local valkey DB:
- http://0.0.0.0:8000/admin/settings/settings_valkey.html
[1] https://pypi.org/project/redis/
[2] https://pypi.org/project/valkey/
Co-authored-by: HLFH <gaspard@dhautefeuille.eu>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
* [mod] container: replace uWSGI with Granian
The configuration in Granian is handled with ENVs, much more convenient and practical for updating. The settings have been tested for over two months in a production instance, being usable on small to somewhat large instances without having to modify anything.
It also removes the patch functions and ENVs abstraction from the entrypoint, this makes it possible to run the container with immutable configuration.
In some setups, It may be desired to have the volumes/files under a specific uid/gid (other than searxng:searxng), if the entrypoint has root permissions it will chown automatically on every start, which may not be desired. Explicitly setting the new ENV `FORCE_OWNERSHIP=false` will prevent ownership from being modified.
No manual migration is necessary **unless** the user has changed the default uWSGI configuration or has a very specific setup.
Closes https://github.com/searxng/searxng/issues/4894
Closes https://github.com/searxng/searxng/issues/4818
Closes https://github.com/searxng/searxng/issues/4802
Supersedes https://github.com/searxng/searxng/pull/4596
Related https://github.com/searxng/searxng/discussions/4479
* [mod] docs: add container/granian
All container documentation has been recreated.
A new documentation page has been created for Granian.
* [enh] misc: apply suggestions
Minor documentation changes.
Suggested https://github.com/searxng/searxng/pull/4820#discussion_r2134539259
Suggested https://github.com/searxng/searxng/pull/4820#discussion_r2134538610
Suggested https://github.com/searxng/searxng/pull/4820#discussion_r2134827964
Suggested https://github.com/searxng/searxng/pull/4820#discussion_r2134544300
Suggested https://github.com/searxng/searxng/pull/4820#discussion_r2149387388
---------
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Co-authored-by: Ivan Gabaldon <igabaldon@inetol.net>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
This is one of various PR to refactor the simple theme internally.
Replace eslint tool with Biome. I have been using this for quite some time, and
it will help us to have more consistent and valid code without extending on
other third party plugins.
Removes unused dependencies.
|
|
Changes:
- Undo deletions of the autocompleter in settings and logic
- Add fixed autocomplete function in autocomplete.py
Related:
- https://github.com/searxng/searxng/issues/4334
|
|
The new Docker entrypoint.sh script implemented in PR:
- https://github.com/searxng/searxng/pull/4793
does no longer have a `-h` option [1]. When building the `make docs` a warming
is shown::
WARNING: Unexpected return code 2 from command Command(command=('../container/entrypoint.sh', '-h') .. (output='../container/entrypoint.sh: 152: SEARXNG_VERSION: parameter not set')
[1] https://github.com/searxng/searxng/pull/4793/files#diff-694a402a03e8de5aa227b1c0294ffdc072b6bac09b4dcbe144dc7d97d4e07159L35
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Tube Archivist [1] is a self-hosted project which archives youtube videos on
your own local server. This engine connects with Tube Archivist's search API to
allow searching from SearXNG into your own hosted videos.
[1] https://www.tubearchivist.com/
Signed-off-by: Robert M. Clabough <robert@claobugh.tech>
Co-authored-by: Bnyro <bnyro@tutanota.com>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
The types necessary for weather information such as GeoLocation, DateTime,
Temperature,Pressure, WindSpeed, RelativeHumidity, Compass (wind direction) and
symbols for the weather have been implemented.
There are unit conversions and translations for weather property labels.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Refactor Naver engine (Web, News, Images, Videos, Autocomplete)
- ref: https://search.naver.com/
- lang: `ko`
- Wikidata: https://www.wikidata.org/wiki/Q485639
Co-authored-by: Bnyro <bnyro@tutanota.com>
|
|
That entrypoint is prone to screw things up, especially with permission handling. The new script handles initialization better and fixes some issues like delayed settings update via ENVs and timestamp overwriting, also adjusts what should be copied into the container.
Related https://github.com/searxng/searxng/pull/4721#issuecomment-2850272129
|
|
Everyone should have already switched from legacy methods
|
|
Chapter on the purpose of (git) commits
The commits and their messages are elementary for the traceability of changes
and are unfortunately still too often given too little attention.
It therefore seems necessary to dedicate a chapter to this topic in the context
of development.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
container.yml will run after integration.yml COMPLETES successfully and in master branch.
Style changes, cleanup and improved integration with CI by leveraging the use of
shared cache between all workflows.
* Podman is now supported to build the container images (Docker also received a refactor, merging both build and buildx)
* Container images are being built by Buildah instead of Docker BuildKit.
* Container images are tested before release.
* Splitting "modern" (amd64 & arm64) and "legacy" (armv7) arches on different Dockerfiles allowing future optimizations.
|
|
|
|
HTTP Fetch Metadata Request Headers [1][2] are used to detect bot requests. Bots
with invalid *Fetch Metadata* will be redirected to the intro (`index`) page.
[1] https://www.w3.org/TR/fetch-metadata/
[2] https://developer.mozilla.org/en-US/docs/Glossary/Fetch_metadata_request_header
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
|
|
Reported:
- https://github.com/inetol-infrastructure/searxng-container/issues/5
Related:
- https://github.com/searxng/searxng/issues/4405#issuecomment-2692352352
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
The language settings page in the documentation reference languages.py which was
removed in commit c9cd376
Closes: https://github.com/searxng/searxng/issues/4690
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
* filtering ChinaSo-News results by source, option ``chinaso_news_source``
* add ChinaSo engine to the online docs https://docs.searxng.org/dev/engines/online/chinaso.html
* fix SearXNG categories in the settings.yml
* deactivate ChinaSo engines ``inactive: true`` until [1] is fixed
* configure network of the ChinaSo engines
[1] https://github.com/searxng/searxng/issues/4694
Signed-off-by: @BrandonStudio
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Re-writes the Mullvad Leta integration to work with the new breaking changes.
Mullvad Leta is a search engine proxy. Currently Leta only offers text search
results not image, news or any other types of search result. Leta acts as a
proxy to Google and Brave search results.
- Remove docstring comments regarding requiring the use of Mullvad VPN, which is
no longer a hard requirement.
- configured two engines: ``mullvadleta`` (uses google) and
``mullvadleta brave`` (uses brave)
- since leta may not provide up-to-date search results, both search engines are
disabled by default.
.. hint::
Leta caches each search for up to 30 days. For example, if you use search
terms like ``news``, contrary to your intention you'll get very old results!
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
Signed-off-by: Grant Lanham <contact@grantlanham.com>
|
|
|
|
A local development server can be launched by one of these command lines::
$ flask --app searx.webapp run
$ python -m searx.webapp
The different ways of starting the server should lead to the same result, which
is generally the case. However, if the modules are reloaded after code
changes (reload option), it must be avoided that the application is initialized
twice at startup. We have already discussed this in 2022 [1][2].
Further information on this topic can be found in [3][4][5].
To test a bash in the ./local environment was started and the follwing commands
had been executed::
$ ./manage pyenv.cmd bash --norc --noprofile
(py3) SEARXNG_DEBUG=1 flask --app searx.webapp run --reload
(py3) SEARXNG_DEBUG=1 python -m searx.webapp
Since the generic parts of the docs also initialize the app to generate doc from
it, the build of the docs was also tested::
$ make docs.clean docs.live
[1] https://github.com/searxng/searxng/pull/1656#issuecomment-1214198941
[2] https://github.com/searxng/searxng/pull/1616#issuecomment-1206137468
[3] https://flask.palletsprojects.com/en/stable/api/#flask.Flask.run
[4] https://github.com/pallets/flask/issues/5307#issuecomment-1774646119
[5] https://stackoverflow.com/a/25504196
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
privacy by design section
|
|
|
|
|
|
of the tool" -> "it might be because the tool"
|
|
|
|
|
|
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Related:
- https://github.com/searxng/searxng/issues/4634
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
BTW: fix a bug with sys.path: repo-root (not util) needs to added to generate
autodoc from scripts in ./searxng_extra
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Co-authored-by: Bnyro <bnyro@tutanota.com>
|
|
|
