summaryrefslogtreecommitdiff
path: root/searx/data/__init__.py
AgeCommit message (Collapse)Author
2026-01-11[fix] google: switch to using GSA for iPhone useragentHEADmastermg95
2025-09-26[upd] pypi: Bump black from 24.3.0 to 25.9.0 (#5251)Markus Heiser
In 25.1.0 [2] an old bug has been fixed: "Docstring formatting does not apply to module docstrings" [3]. [1] https://github.com/psf/black/blob/main/CHANGES.md#2590 [2] https://github.com/psf/black/blob/main/CHANGES.md#2510 [3] https://github.com/psf/black/issues/4094 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-03[mod] addition of various type hints / tbcMarkus Heiser
- pyright configuration [1]_ - stub files: types-lxml [2]_ - addition of various type hints - enable use of new type system features on older Python versions [3]_ - ``.tool-versions`` - set python to lowest version we support (3.10.18) [4]_: Older versions typically lack some typing features found in newer Python versions. Therefore, for local type checking (before commit), it is necessary to use the older Python interpreter. .. [1] https://docs.basedpyright.com/v1.20.0/configuration/config-files/ .. [2] https://pypi.org/project/types-lxml/ .. [3] https://typing-extensions.readthedocs.io/en/latest/# .. [4] https://mise.jdx.dev/configuration.html#tool-versions Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> Format: reST
2025-06-23[mod] data: implement a simple tracker URL (SQL) databaseMarkus Heiser
On demand, the tracker data is loaded directly into the cache, so that the maintenance of this data via PRs is no longer necessary. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-06-23[feat] tracker url plugin: use ClearURL tracking param listBnyro
2025-05-25[mod] data: implement a simple currencies (SQL) database (#4836)Markus Heiser
To reduce the memory footprint, this patch no longer loads the JSON data completely into memory. Instead, there is an SQL database based on `ExpireCacheSQLite`. The class CurrenciesDB is a simple DB application that encapsulates the DB (queries and initialization) and provides convenient methods like `name_to_iso4217` and `iso4217_to_name`. Related: - https://github.com/searxng/searxng/discussions/1892 - https://github.com/searxng/searxng/pull/3458#issuecomment-2900807671 - https://github.com/searxng/searxng/pull/4650 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-05-22[mod] searx.data: lazy load of the data objects (databases) (#4834)Markus Heiser
In the previous implementation, all databases were loaded into memory when importing the searx.data package, regardless of whether they were ever needed. Regardless of this, it is an antipattern to load entire databases into memory when importing a package or module; databases should be loaded when needed. Lazy loading is a first step toward improving memory usage and also improves performance when setting up the runtime environment. Building on this, subsequent PRs will be able to further optimize memory behavior, e.g., by using a real database application such as the one already available via searx.cache.ExpireCache Related: - https://github.com/searxng/searxng/discussions/1892 - https://github.com/searxng/searxng/pull/3458 - https://github.com/searxng/searxng/pull/4650 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-11[mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTIONMarkus Heiser
In the past, some files were tested with the standard profile, others with a profile in which most of the messages were switched off ... some files were not checked at all. - ``PYLINT_SEARXNG_DISABLE_OPTION`` has been abolished - the distinction ``# lint: pylint`` is no longer necessary - the pylint tasks have been reduced from three to two 1. ./searx/engines -> lint engines with additional builtins 2. ./searx ./searxng_extra ./tests -> lint all other python files Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-02-20[mod] reduce memory footprint by not calling babel.Locale.parse at runtimeAlexandre Flament
babel.Locale.parse loads more than 60MB in RAM. The only purpose is to get: LOCALE_NAMES - searx.data.LOCALES["LOCALE_NAMES"] RTL_LOCALES - searx.data.LOCALES["RTL_LOCALES"] This commit calls babel.Locale.parse when the translations are update from weblate and stored in:: searx/data/locales.json This file can be build by:: ./manage data.locales By store these variables in searx.data when the translations are updated we save round about 65MB (usually 4 worker = 260MB of RAM saved. Suggested-by: https://github.com/searxng/searxng/discussions/2633#discussioncomment-8490494 Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24[mod] replace engines_languages.json by engines_traits.jsonMarkus Heiser
Implementations of the *traits* of the engines. Engine's traits are fetched from the origin engine and stored in a JSON file in the *data folder*. Most often traits are languages and region codes and their mapping from SearXNG's representation to the representation in the origin search engine. To load traits from the persistence:: searx.enginelib.traits.EngineTraitsMap.from_data() For new traits new properties can be added to the class:: searx.enginelib.traits.EngineTraits .. hint:: Implementation is downward compatible to the deprecated *supported_languages method* from the vintage implementation. The vintage code is tagged as *deprecated* an can be removed when all engines has been ported to the *traits method*. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-29[mod] remove deprecate codeAlexandre Flament
remove code to support Python 3.5 and Python 3.6
2021-12-27[format.python] initial formatting of the python codeMarkus Heiser
This patch was generated by black [1]:: make format.python [1] https://github.com/psf/black Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-02SearXNG: searxng_extraAlexandre Flament
2021-09-24[mod] add /engine_descriptions.json endpointAlexandre Flament
returns engine descriptions (JSON): * key: engine name * value: description in the user locale, use English description as a fallback
2021-06-09[pylint] searx/data/__init__.pyMarkus Heiser
BTW: add doc strings and moved __all__ to the top [1] [1] https://www.python.org/dev/peps/pep-0008/#module-level-dunder-names Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-09[fix] searx/data/__init__.py: rename __init__ as __all__Alexandre Flament
2021-06-09[enh] openstreetmap / map template: improve resultsAlexandre Flament
implements ideas described in #69 * update the engine * use wikidata * update map.html template
2021-02-24[mod] add utils/fetch_external_bangs.pyAlexandre Flament
Based on duckduckgo bangs Store bangs on a trie to allow autocomplete (not in this commit)
2021-01-12[mod] drop Python 3.5 supportAlexandre Flament
2020-10-28[mod] add external_urls.json and wikidata_units.jsonAlexandre Flament
2020-10-27[mod] ahmia_filter.py: minor changesAlexandre Flament
- use result['parsed_url'] - load ahmia_blacklist.txt in searx.datae
2020-10-07[mod] Add searx.data moduleAlexandre Flament
Instead of loading the data/*.json in different location, load these files in the new searx.data module.
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-03 17:03:58 +0000