[fix] make data.traits - partial revert of commit 30a8204

The entries in the catalog of search languages are build up from the "Engine
Traits" [1] and which entries are included in the catalog is controlled qby two
threshold values [2].

If possible, the values should ensure that no languages or regions disappear
from the catalog of search languages [3].

The threshold values should have been adjusted in commit 30a8204:

- ``min_eng_per_region = 18``
- ``min_eng_per_lang = 22``

Because the threshold values were not adjusted, many entries were missing in the
search language catalog.  This bug has been fixed with this patch: the threshold
values have been adjusted and the catalog of search languages has been completed
again.

[1] https://docs.searxng.org/dev/engines/enginelib.html#module-searx.enginelib.traits
[2] https://github.com/searxng/searxng/blob/96a6e3dcb2283fa7ad9db4172a00582073a166d7/searxng_extra/update/update_engine_traits.py#L104-L105
[3] https://github.com/searxng/searxng/blob/master/searx/sxng_locales.py

Closes: https://github.com/searxng/searxng/issues/4519
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

1 files changed, 15 insertions, 1 deletions

diff --git a/searx/enginelib/traits.py b/searx/enginelib/traits.py
index cab6557dd..8bb4e0554 100644
--- a/searx/enginelib/traits.py
+++ b/searx/enginelib/traits.py
@@ -10,6 +10,8 @@ used.
 """
 
 from __future__ import annotations
+
+import os
 import json
 import dataclasses
 import types
@@ -219,8 +221,20 @@ class EngineTraitsMap(Dict[str, EngineTraits]):
 
         for engine_name in names:
             engine = engines.engines[engine_name]
+            traits = None
+
+            # pylint: disable=broad-exception-caught
+            try:
+                traits = EngineTraits.fetch_traits(engine)
+            except Exception as exc:
+                log("FATAL: while fetch_traits %s: %s" % (engine_name, exc))
+                if os.environ.get('FORCE', '').lower() not in ['on', 'true', '1']:
+                    raise
+                v = ENGINE_TRAITS.get(engine_name)
+                if v:
+                    log("FORCE: re-use old values from fetch_traits - ENGINE_TRAITS[%s]" % engine_name)
+                    traits = EngineTraits(**v)
 
-            traits = EngineTraits.fetch_traits(engine)
             if traits is not None:
                 log("%-20s: SearXNG languages --> %s " % (engine_name, len(traits.languages)))
                 log("%-20s: SearXNG regions   --> %s" % (engine_name, len(traits.regions)))