summaryrefslogtreecommitdiff
path: root/searx/data/currencies.py
AgeCommit message (Collapse)Author
2025-09-28[mod] ExpireCacheSQLite - implement .setmany() for bulk loadingMarkus Heiser
[1] https://github.com/searxng/searxng/issues/5223#issuecomment-3328597147 Suggested-by: Ivan G <igabaldon@inetol.net> [1] Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-28[mod] debug-log the bootload of the CURRENCIES caches (CurrenciesDB)Markus Heiser
The time for the bootload is measured and recorded. To have an eye on the bootload time is motivated by: In [1] it can be seen that the bootload of the CURRENCIES cache takes about 2/3 of the time required to initialize SearXNG:: 6.068 <module> searx/webapp.py:1 └─ 6.068 wrapper pyinstrument/context_manager.py:52 ├─ 5.538 init searx/webapp.py:1373 │ ├─ 4.822 initialize searx/search/__init__.py:34 │ │ ├─ 4.631 ProcessorMap.init searx/search/processors/__init__.py:47 │ │ │ ├─ 4.607 OnlineCurrencyProcessor.initialize searx/search/processors/online_currency.py:55 │ │ │ │ └─ 4.607 CurrenciesDB.init searx/data/currencies.py:25 │ │ │ │ ├─ 4.601 CurrenciesDB.load searx/data/currencies.py:34 │ │ │ │ │ ├─ 4.572 ExpireCacheSQLite.set searx/cache.py:334 In the example, the CurrenciesDB.init call takes 4.6 seconds... on my laptop, CurrenciesDB.init takes only 0.7 seconds. The absolute numerical values depend on external conditions, where I already find 4-5 seconds very long. Test:: $ rm /tmp/sxng_cache_*.db* $ make run 2>&1 | grep "searx.data.CURRENCIES" DEBUG searx.data : init searx.data.CURRENCIES DEBUG searx.data : init searx.data.CURRENCIES added 9089 items in 0.7623255252838135 sec. [1] https://github.com/searxng/searxng/issues/5223#issuecomment-3323083411 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-18[mod] addition of various type hints / engine processorsMarkus Heiser
Continuation of #5147 .. typification of the engine processors. BTW: - removed obsolete engine property https_support - fixed & improved currency_convert - engine instances can now implement a engine.setup method [#5147] https://github.com/searxng/searxng/pull/5147 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-03[mod] drop: from __future__ import annotationsMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-07-09[fix] prevent multiple, parallel initializations of tables in the cache DB ↵Markus Heiser
(#4991) Depending on the respective runtime behavior, it could happen that the initial loading of the DB tables in the cache was performed multiple times and in parallel. The concurrent accesses then led to the `sqlite3.OperationalError: database is locked` exception as in #4951. Since this problem depends significantly on the runtimes (e.g., how long it takes to retrieve the content for a table), this error could not be observed in all installations. Closes: https://github.com/searxng/searxng/issues/4951 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-05-29[fix] plugins: currency processor bugMarkus Heiser
Fix C&P typo from https://github.com/searxng/searxng/pull/4836 Closes: https://github.com/searxng/searxng/issues/4861
2025-05-25[mod] data: implement a simple currencies (SQL) database (#4836)Markus Heiser
To reduce the memory footprint, this patch no longer loads the JSON data completely into memory. Instead, there is an SQL database based on `ExpireCacheSQLite`. The class CurrenciesDB is a simple DB application that encapsulates the DB (queries and initialization) and provides convenient methods like `name_to_iso4217` and `iso4217_to_name`. Related: - https://github.com/searxng/searxng/discussions/1892 - https://github.com/searxng/searxng/pull/3458#issuecomment-2900807671 - https://github.com/searxng/searxng/pull/4650 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-03 19:21:53 +0000