summaryrefslogtreecommitdiff
path: root/searx/engines
AgeCommit message (Collapse)Author
2025-11-06[enh] py: drop deps (#5407)Ivan Gabaldon
The difference between decompression with brotli or gzip in HTML files is negligible for 3 MB of compiled binary package. Introduced in https://github.com/searxng/searxng/commit/eaa694fb7d0e47b943bc6d6edb6cb6a40ab2d85e Closes https://github.com/searxng/searxng/security/code-scanning/276 Closes https://github.com/searxng/searxng/security/dependabot/37
2025-11-06[fix] yandex engine: capture captcha from header instead of url path (#5417)Aadniz
Yandex engine will return parsing error instead of informing that a CAPTCHA was found. It is confusing for the admin and the users (#5415). This patch fixes an issue where the CAPTCHA response from Yandex wouldn't be detected, resulting in `ParserError` when trying to parse the response to DOM. In this fix, I replaced the url condition and instead is checking if the `x-yandex-captcha` header is set, and is equal to `captcha`. Alternatively, maybe something like `resp.headers.get('Location', '').startswith("https://yandex.com/showcaptcha")` could be done instead. Lastly, setting `params['allow_redirects'] = True` can also work, but this will waste an extra request. Just let me know. Closes: https://github.com/searxng/searxng/issues/5415
2025-11-05[feat] sourcehut engine: implement as custom module, fix user agentBnyro
SourceHut uses a foss bot protection tool called `go-away` (which I can recommend BTW). It blocks common crawler user agents, such as the standard Firefox user agent. Hence, we're now using our custom SearXNG user agent to clarify we're not a crawler. Closes: https://github.com/searxng/searxng/issues/5270 Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2025-11-05[fix] engine ahmia blacklist, arch linux: use proper searxng user agent ↵Bnyro
including version (#5414)
2025-11-03[fix] qwant engine: order query parameters to prevent 403 forbidden (#5410)Aadniz
2025-10-28[fix] deviantart engine: pagination match change (#5384)Aadniz
Pagination currently does not work for deviantart, resulting in the same page being shown when going to the next page in SearXNG.
2025-10-27[fix] qwant engine: set header Accept-Language to bypass bot detection (#5382)Aadniz
Set HTTP header Accept-Language [1] for the Qwant engine. Qwant does not seem to work on any SearXNG instance right now, and this is a fix for this issue. During testing, it seems like setting the Accept-Language gives more success for bypassing bot detection (tested with a few ~20 searches). [1] https://docs.searxng.org/dev/engines/enginelib.html#searx.enginelib.Engine.send_accept_language_header
2025-10-27[fix] deviantart engine: does not return any results (#5383)Aadniz
2025-10-26[fix] startpage engine: properly display CAPTCHA if redirect page is seen ↵Aadniz
(#5380) Fixes an issue where startpage engine would display parsing error (`json.decoder.JSONDecodeError`) when returning CAPTCHA redirect page. The fix simply checks if response header has `Location` set, and if it starts with `https://www.startpage.com/sp/captcha`, it will raise a CAPTCHA exception before trying to parse the data.
2025-10-25[fix] engine qwant - return forbidden instead of showing parse error (#5377)Aadniz
2025-10-20[mod] typification of SearXNG: add new result type FileMarkus Heiser
This PR adds a new result type: File Python class: searx/result_types/file.py Jinja template: searx/templates/simple/result_templates/file.html CSS (less) client/simple/src/less/result_types/file.less Class 'File' (singular) replaces template 'files.html' (plural). The renaming was carried out because there is only one file (singular) in a result. Not to be confused with the category 'files' where in multiple results can exist. As mentioned in issue [1], the class '.category-files' was removed from the CSS and the stylesheet was adopted in result_types/file.less (there based on the templates and no longer based on the category). [1] https://github.com/searxng/searxng/issues/5198 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-18[mod] engine: reuters - REST-API for Reuter's thumbnail, height:80Markus Heiser
The size of the full-size images from ``thumbnail.url`` is usually several MB. By reducing the full-size image to 80 pixels, the data size for a thumb is reduced from MB to a few KB. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-18[fix] reuters: crash on empty results pages & date parsingBnyro
1. On empty result list, return empty EngineResults (#5330) 2. Use ``dateutil.parser`` to avoid ``ValueError``: ERROR searx.engines.reuters : exception : Invalid isoformat string: '2022-06-08T16:07:54Z' File "searx/engines/reuters.py", line 91, in response publishedDate=datetime.fromisoformat(result["display_time"]), ValueError: Invalid isoformat string: '2022-06-08T16:07:54Z' Closes: https://github.com/searxng/searxng/issues/5330 Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-18[fix] engine mullvadleta - ignore HTTP 403 & 429 responseMarkus Heiser
It doesn't matter if you're using Mullvad's VPN and a proper browser, you'll still get blocked for specific searches [1] with a 403 or 429 HTTP status code. Mullvad only blocks the search request and doesn't prevent you from doing more searches. The logic should handle the blocked requests (403, 429), but not put the engine on a cooldown. [1] https://leta.mullvad.net/search?q=site%3Afoo+bar&engine=brave Closes: https://github.com/searxng/searxng/issues/5328 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-17[mod] bing engine: follow redirects (#5324)Bnyro
Apparently, in China, Bing redirects from `www.bing.com` to `cn.bing.com`. So in order to make Bing work for chinese users by default, we have to follow that redirect. related: https://github.com/searxng/searxng/issues/5243
2025-10-13[feat] engines: add Azure resources engine (#5235)Tommaso Colella
Adds a new engine `searx/engines/azure.py` to search cloud resources on Azure. A lot of enterprise users have to deal with Azure Public Cloud. This helps them easily search for cloud resources without logging in to the Portal first How to test this PR locally? You should create an App Registration on Azure Entra Id with Reader access on the resources you want to search for. You should create a Secret for the App Registration. After that, you should set up appropriate values in the `settings.yml` file [1]:: - name: azure engine: azure ... azure_tenant_id: "your_tenant_id" azure_client_id: "your_client_id" azure_client_secret: "your_client_secret" azure_token_expiration_seconds: 5000 [1] https://github.com/searxng/searxng/pull/5235#issuecomment-3397664928 Co-authored-by: Bnyro <bnyro@tutanota.com> Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-13[fix] pinterest: crash when there's no link & show image resolution + ↵Bnyro
uploader name (#5314) closes #5231
2025-10-09[fix] startpage engine - SafeSearch works in reverse (#5290)Markus Heiser
The Name of the option is *disable_family_filter* -> we have to reverse the meaning of the ascending safe-search filter level. Closes: https://github.com/searxng/searxng/issues/5287 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-06[fix] google scholar - detect CAPTCHA (HTTP redirects) (#5268)Markus Heiser
In the case of .. response, for example, an HTTP 302 is returned by Google Scholar:: Our systems have detected unusual traffic from your computer network. Please try again later. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-03[fix] engine - cppreference has no longer a search function (#5273)Markus Heiser
cppreference has replaced its search (``mwiki/index.php?title=``) with a DDG search. The engine was first introduced in SearXNG with PR-3274 [1], and even back then the mediawiki proved to be incompatible, which is why the API could not be used at the time. Now there isn't even a dedicated search function anymore.. I think the cppreference project suffers from a lack of maintenance. [1] https://github.com/searxng/searxng/pull/3247 Closes: https://github.com/searxng/searxng/issues/5271 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-01[fix] searx/results.py - TypeError: object of type 'NoneType' has no len()Markus Heiser
In some engines, under certain circumstances, the content field can also have the value ``None``; in these cases, a length check results in an exception:: File "/usr/local/searxng/searx/results.py", line 360, in merge_two_main_results if len(other.content) > len(origin.content): ^^^^^^^^^^^^^^^^^^ TypeError: object of type 'NoneType' has no len() [1] https://github.com/searxng/searxng/issues/5250#issuecomment-3352863488 Reported-by: @scross01 [1] Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-01[fix] JSON format: serialization of the result-typesMarkus Heiser
The ``JSONEncoder`` (``format="json"``) must perform a conversion to the built-in types for the ``msgspec.Struct``:: if isinstance(o, msgspec.Struct): return msgspec.to_builtins(o) The result types are already of type ``msgspec.Struct``, so they can be converted into built-in types. The field types (in the result type) that were not yet of type ``msgspec.Struct`` have been converted to:: searx.weather.GeoLocation@dataclass -> msgspec.Struct searx.weather.DateTime -> msgspec.Struct searx.weather.Temperature -> msgspec.Struct searx.weather.PressureUnits -> msgspec.Struct searx.weather.WindSpeed -> msgspec.Struct searx.weather.RelativeHumidity -> msgspec.Struct searx.weather.Compass -> msgspec.Struct BTW: Wherever it seemed sensible, the typing was also modernized in the modified files. Closes: https://github.com/searxng/searxng/issues/5250 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-28[fix] openstreetmap: fix CURRENCIES.iso4217_to_nameMarkus Heiser
This patch is a leftover from PR-5204 [1]. [1] https://github.com/searxng/searxng/pull/5204 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-26[upd] pypi: Bump black from 24.3.0 to 25.9.0 (#5251)Markus Heiser
In 25.1.0 [2] an old bug has been fixed: "Docstring formatting does not apply to module docstrings" [3]. [1] https://github.com/psf/black/blob/main/CHANGES.md#2590 [2] https://github.com/psf/black/blob/main/CHANGES.md#2510 [3] https://github.com/psf/black/issues/4094 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] demo engines: smaller improvementMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] ADS engine: revision of the engine (Paper result)Markus Heiser
Revision of the Astrophysics Data System (ADS) engine / use of the result type Paper as well as other typifications. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] Z-Library engine: revision of the engine (Paper result)Markus Heiser
Revision of the engine / use of the result type Paper as well as other typifications. The engine has been placed on inactive because no service is currently available, or at least not known in the SearXNG community [1] [1] https://github.com/searxng/searxng/issues/3610 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] Springer Nature engine: revision of the engine (Paper result)Markus Heiser
Revision of the engine / use of the result type Paper as well as other typifications. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] Semantic Scholar engine: revision of the engine (Paper result)Markus Heiser
Revision of the engine / use of the result type Paper as well as other typifications. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] PubMed engine: revision of the engine (Paper result)Markus Heiser
Revision of the engine / use of the result type Paper as well as other typifications. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] Open Library engine: revision of the engine (Paper result)Markus Heiser
Revision of the engine / use of the result type Paper as well as other typifications. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] OpenAlex engine: revision of the engine (Paper result)Markus Heiser
Revision of the engine / use of the result type Paper as well as other typifications. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] Google Scholar engine: revision of the engine (Paper result)Markus Heiser
Revision of the engine / use of the result type Paper as well as other typifications. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] Crossref engine: revision of the engine (Paper result)Markus Heiser
Revision of the engine / use of the result type Paper as well as other typifications. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] CORE engine: revision of the engine (Paper result)Markus Heiser
Revision of the engine / use of the result type Paper as well as other typifications. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] Anna's Archive engine: revision of the engine (Paper result)Markus Heiser
Revision of the engine / use of the result type Paper as well as other typifications. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20[mod] arXiv engine: revision of the engine (Paper result)Markus Heiser
Revision of the engine / use of the result type Paper as well as other typifications. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-18[mod] demo engines: smaller corrections and improvementsMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-18[mod] addition of various type hints / engine processorsMarkus Heiser
Continuation of #5147 .. typification of the engine processors. BTW: - removed obsolete engine property https_support - fixed & improved currency_convert - engine instances can now implement a engine.setup method [#5147] https://github.com/searxng/searxng/pull/5147 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-10[feat] engines yacy & piped: enable individual configuration of URLs (#5195)Markus Heiser
With this change it is possible with individual engines (yacy & piped) to configure individual URLs. Related: - https://github.com/searxng/searxng/issues/4869#issuecomment-327335928 - https://github.com/searxng/searxng/pull/3472/files#r1595586019 - https://github.com/searxng/searxng/issues/3428#issuecomment-2102142530 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-09[feat] naver engine: add video embedsAustin-Olacsi
2025-09-03[mod] drop: from __future__ import annotationsMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-03[mod] addition of various type hints / tbcMarkus Heiser
- pyright configuration [1]_ - stub files: types-lxml [2]_ - addition of various type hints - enable use of new type system features on older Python versions [3]_ - ``.tool-versions`` - set python to lowest version we support (3.10.18) [4]_: Older versions typically lack some typing features found in newer Python versions. Therefore, for local type checking (before commit), it is necessary to use the older Python interpreter. .. [1] https://docs.basedpyright.com/v1.20.0/configuration/config-files/ .. [2] https://pypi.org/project/types-lxml/ .. [3] https://typing-extensions.readthedocs.io/en/latest/# .. [4] https://mise.jdx.dev/configuration.html#tool-versions Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> Format: reST
2025-09-03[fix} engine chinaso - parse_images `ImageInfo` key error (#5175)Butui Hu
Signed-off-by: Butui Hu <hot123tea123@gmail.com>
2025-09-01[feat] engines: add repology.org engine for linux packages (#5103)Bnyro
Repology_ monitors a huge number of package repositories and other sources comparing packages versions across them and gathering other information. Repology_ shows you in which repositories a given project is packaged, which version is the latest and which needs updating, who maintains the package, and other related information. .. _Repology: https://repology.org/docs/about Co-authored-by: Markus Heiser <markus.heiser@darmarit.de> Format: reST
2025-09-01[fix] chinaoso: add random `uid` to cookie (#5173)Butui Hu
Signed-off-by: Butui Hu <hot123tea123@gmail.com>
2025-09-01[mod] typification of SearXNG: add new result type CodeMarkus Heiser
This patch adds a new result type: Code - Python class: searx/result_types/code.py - Jinja template: searx/templates/simple/result_templates/code.html - CSS (less) client/simple/src/less/result_types/code.less Signed-of-by: Markus Heiser <markus.heiser@darmarIT.de>
2025-08-28[fix] annas archive: engine broken due to site HTML changesBnyro
Apparently the layout of https://annas-archive.org has changed, making changes necessary. The issue has been reported in #5146, see there for more details. - closes #5146
2025-08-24[feat] engines: add OpenAlex Works engine (#5102)muthukumaran R
- Adds a new engine `searx/engines/openalex.py` that integrates the OpenAlex Works API to return scientific paper results using the `paper.html` template. - Uses the official API (no auth required); supports OpenAlex polite pool via `mailto`. - Adds developer docs at `docs/dev/engines/online/openalex.rst`. OpenAlex API reference: https://docs.openalex.org/how-to-use-the-api/api-overview
2025-08-20[fix] selfhst icons: icon list url invalid, set to activeBnyro
- the previous CDN icon list url no longer works - a list of all icons is mirrored to the JSDelivr CDN however - there's no reason to set the engine to inactive now that we use public CDNs
generated by cgit v1.2.3 (git 2.25.1) at 2026-04-03 15:45:18 +0000