| Age | Commit message (Collapse) | Author |
|---|
|
This PR adds a new result type: File
Python class: searx/result_types/file.py
Jinja template: searx/templates/simple/result_templates/file.html
CSS (less) client/simple/src/less/result_types/file.less
Class 'File' (singular) replaces template 'files.html' (plural). The renaming
was carried out because there is only one file (singular) in a result. Not to be
confused with the category 'files' where in multiple results can exist.
As mentioned in issue [1], the class '.category-files' was removed from the CSS
and the stylesheet was adopted in result_types/file.less (there based on the
templates and no longer based on the category).
[1] https://github.com/searxng/searxng/issues/5198
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
The size of the full-size images from ``thumbnail.url`` is usually several
MB. By reducing the full-size image to 80 pixels, the data size for a thumb is
reduced from MB to a few KB.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
1. On empty result list, return empty EngineResults (#5330)
2. Use ``dateutil.parser`` to avoid ``ValueError``:
ERROR searx.engines.reuters : exception : Invalid isoformat string: '2022-06-08T16:07:54Z'
File "searx/engines/reuters.py", line 91, in response
publishedDate=datetime.fromisoformat(result["display_time"]),
ValueError: Invalid isoformat string: '2022-06-08T16:07:54Z'
Closes: https://github.com/searxng/searxng/issues/5330
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
It doesn't matter if you're using Mullvad's VPN and a proper browser, you'll
still get blocked for specific searches [1] with a 403 or 429 HTTP status code.
Mullvad only blocks the search request and doesn't prevent you from doing more
searches.
The logic should handle the blocked requests (403, 429), but not put the engine
on a cooldown.
[1] https://leta.mullvad.net/search?q=site%3Afoo+bar&engine=brave
Closes: https://github.com/searxng/searxng/issues/5328
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Apparently, in China, Bing redirects from `www.bing.com` to `cn.bing.com`.
So in order to make Bing work for chinese users by default, we have to follow that redirect.
related: https://github.com/searxng/searxng/issues/5243
|
|
Adds a new engine `searx/engines/azure.py` to search cloud resources on Azure.
A lot of enterprise users have to deal with Azure Public Cloud. This helps them
easily search for cloud resources without logging in to the Portal first
How to test this PR locally?
You should create an App Registration on Azure Entra Id with Reader access on
the resources you want to search for. You should create a Secret for the App
Registration. After that, you should set up appropriate values in the
`settings.yml` file [1]::
- name: azure
engine: azure
...
azure_tenant_id: "your_tenant_id"
azure_client_id: "your_client_id"
azure_client_secret: "your_client_secret"
azure_token_expiration_seconds: 5000
[1] https://github.com/searxng/searxng/pull/5235#issuecomment-3397664928
Co-authored-by: Bnyro <bnyro@tutanota.com>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
uploader name (#5314)
closes #5231
|
|
The Name of the option is *disable_family_filter* -> we have to reverse the
meaning of the ascending safe-search filter level.
Closes: https://github.com/searxng/searxng/issues/5287
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
In the case of .. response, for example, an HTTP 302 is returned by Google
Scholar::
Our systems have detected unusual traffic from your computer
network. Please try again later.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
cppreference has replaced its search (``mwiki/index.php?title=``) with a DDG
search.
The engine was first introduced in SearXNG with PR-3274 [1], and even back then
the mediawiki proved to be incompatible, which is why the API could not be used
at the time. Now there isn't even a dedicated search function anymore.. I think
the cppreference project suffers from a lack of maintenance.
[1] https://github.com/searxng/searxng/pull/3247
Closes: https://github.com/searxng/searxng/issues/5271
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
In some engines, under certain circumstances, the content field can also have
the value ``None``; in these cases, a length check results in an exception::
File "/usr/local/searxng/searx/results.py", line 360, in merge_two_main_results
if len(other.content) > len(origin.content):
^^^^^^^^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()
[1] https://github.com/searxng/searxng/issues/5250#issuecomment-3352863488
Reported-by: @scross01 [1]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
The ``JSONEncoder`` (``format="json"``) must perform a conversion to the
built-in types for the ``msgspec.Struct``::
if isinstance(o, msgspec.Struct):
return msgspec.to_builtins(o)
The result types are already of type ``msgspec.Struct``, so they can be
converted into built-in types.
The field types (in the result type) that were not yet of type ``msgspec.Struct``
have been converted to::
searx.weather.GeoLocation@dataclass -> msgspec.Struct
searx.weather.DateTime -> msgspec.Struct
searx.weather.Temperature -> msgspec.Struct
searx.weather.PressureUnits -> msgspec.Struct
searx.weather.WindSpeed -> msgspec.Struct
searx.weather.RelativeHumidity -> msgspec.Struct
searx.weather.Compass -> msgspec.Struct
BTW: Wherever it seemed sensible, the typing was also modernized in the modified
files.
Closes: https://github.com/searxng/searxng/issues/5250
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
This patch is a leftover from PR-5204 [1].
[1] https://github.com/searxng/searxng/pull/5204
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
In 25.1.0 [2] an old bug has been fixed: "Docstring formatting does not apply to
module docstrings" [3].
[1] https://github.com/psf/black/blob/main/CHANGES.md#2590
[2] https://github.com/psf/black/blob/main/CHANGES.md#2510
[3] https://github.com/psf/black/issues/4094
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Revision of the Astrophysics Data System (ADS) engine / use of the result type
Paper as well as other typifications.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Revision of the engine / use of the result type Paper as well as other
typifications.
The engine has been placed on inactive because no service is currently
available, or at least not known in the SearXNG community [1]
[1] https://github.com/searxng/searxng/issues/3610
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Revision of the engine / use of the result type Paper as well as other
typifications.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Revision of the engine / use of the result type Paper as well as other
typifications.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Revision of the engine / use of the result type Paper as well as other
typifications.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Revision of the engine / use of the result type Paper as well as other
typifications.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Revision of the engine / use of the result type Paper as well as other
typifications.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Revision of the engine / use of the result type Paper as well as other
typifications.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Revision of the engine / use of the result type Paper as well as other
typifications.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Revision of the engine / use of the result type Paper as well as other
typifications.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Revision of the engine / use of the result type Paper as well as other
typifications.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Revision of the engine / use of the result type Paper as well as other
typifications.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Continuation of #5147 .. typification of the engine processors.
BTW:
- removed obsolete engine property https_support
- fixed & improved currency_convert
- engine instances can now implement a engine.setup method
[#5147] https://github.com/searxng/searxng/pull/5147
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
With this change it is possible with individual engines (yacy & piped)
to configure individual URLs.
Related:
- https://github.com/searxng/searxng/issues/4869#issuecomment-327335928
- https://github.com/searxng/searxng/pull/3472/files#r1595586019
- https://github.com/searxng/searxng/issues/3428#issuecomment-2102142530
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
|
|
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
- pyright configuration [1]_
- stub files: types-lxml [2]_
- addition of various type hints
- enable use of new type system features on older Python versions [3]_
- ``.tool-versions`` - set python to lowest version we support (3.10.18) [4]_:
Older versions typically lack some typing features found in newer Python
versions. Therefore, for local type checking (before commit), it is necessary
to use the older Python interpreter.
.. [1] https://docs.basedpyright.com/v1.20.0/configuration/config-files/
.. [2] https://pypi.org/project/types-lxml/
.. [3] https://typing-extensions.readthedocs.io/en/latest/#
.. [4] https://mise.jdx.dev/configuration.html#tool-versions
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Format: reST
|
|
Signed-off-by: Butui Hu <hot123tea123@gmail.com>
|
|
Repology_ monitors a huge number of package repositories and other sources
comparing packages versions across them and gathering other information.
Repology_ shows you in which repositories a given project is packaged, which
version is the latest and which needs updating, who maintains the package, and
other related information.
.. _Repology: https://repology.org/docs/about
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
Format: reST
|
|
Signed-off-by: Butui Hu <hot123tea123@gmail.com>
|
|
This patch adds a new result type: Code
- Python class: searx/result_types/code.py
- Jinja template: searx/templates/simple/result_templates/code.html
- CSS (less) client/simple/src/less/result_types/code.less
Signed-of-by: Markus Heiser <markus.heiser@darmarIT.de>
|
|
Apparently the layout of https://annas-archive.org has changed, making changes necessary.
The issue has been reported in #5146, see there for more details.
- closes #5146
|
|
- Adds a new engine `searx/engines/openalex.py` that integrates the OpenAlex
Works API to return scientific paper results using the `paper.html` template.
- Uses the official API (no auth required); supports OpenAlex polite pool via `mailto`.
- Adds developer docs at `docs/dev/engines/online/openalex.rst`.
OpenAlex API reference: https://docs.openalex.org/how-to-use-the-api/api-overview
|
|
- the previous CDN icon list url no longer works
- a list of all icons is mirrored to the JSDelivr CDN however
- there's no reason to set the engine to inactive now that we use public CDNs
|
|
This patch adds GitHub Code Search [1] engine to allow querying the codebases.
Template code.html is changed to allow passthrough of strip and highlighting
options.
Engine Searchcode is adjusted to pass filename and not rely on hardcoded
extensions.
GitHub search code API does not return the exact code line indices, this
implementation assigns the code arbitrary numbers starting from 1
(effectively relabeling the code).
The API allows for unauth calls, and the default engine settings default to
that, although the calls are heavily rate limited.
The 'text' lexer is the default pygments lexer when parsing fails.
[1] https://docs.github.com/en/rest/search/search?apiVersion=2022-11-28#search-code
Co-authored-by: Markus Heiser <markus.heiser@darmarIT.de>
|
|
pyrightconfig.json :
for the paths searx, searxng_extra and tests, individual rules were
defined (for example, in test fewer / different rules are needed than in the
searx package
searx/engines/__builtins__.pyi :
The builtin types that are added to the global namespace of a module by the
intended monkey patching of the engine modules / replaces the previous
filtering of the stdout using grep.
test.pyright_modified (utils/lib_sxng_test.sh) :
static type check of local modified files not yet commited
make test :
prerequisite 'test.pyright' has been replaced by 'test.pyright_modified'
searx/engines/__init__.py, searx/enginelib/__init__.py :
First, minimal typifications that were considered necessary.
|
|
FIxes publishedDate format in reuters engine to encompass ISO 8601 times both with and without milliseconds.
Why is this change important?
Previously, the engine would sometimes fail saying:
2025-08-12 21:13:23,091 ERROR:searx.engines.reuters: exception : time data '2024-04-15T19:08:30.833Z' does not match format '%Y-%m-%dT%H:%M:%SZ'
Traceback (most recent call last):
...
File "/usr/local/searxng/searx/engines/reuters.py", line 87, in response
publishedDate=datetime.strptime(result["display_time"], "%Y-%m-%dT%H:%M:%SZ"),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
Note that most queries seem to work with Reuters, but there are some results that have the additional milliseconds and fail. Regardless, the change is backwards compatible as both the formats (with and without the ms) should now parse correctly.
|
|
Add Baidu Captcha detection to reduce `JSONDecodeError` error
Baidu will redirect to `wappass.baidu.com` and return a captcha challenge.
Current behavior will get the data from `wappass.baidu.com` then return a
`json.decoder.JSONDecodeError` error.
|
|
|
|
To get an API key follow instructions at [1].
[1] https://about.marginalia-search.com/article/api/
Related (historical ordered):
- https://github.com/searxng/searxng/issues/1620
- https://github.com/searxng/searxng/issues/1673
- https://github.com/searxng/searxng/pull/1627
- https://github.com/searxng/searxng/pull/2489
Closes:
- https://github.com/searxng/searxng/issues/3034
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Sometimes, there's only an `adaptivestreaming` field in `streams`, which
is usually an m3u8 file. That's however not supported by the video player
of any browser, so we can't use and must build a different url instead.
|
|
- not 100% sure about the condition code mapping, there are no real matches for most of the codes from Apple WeatherKit to the weather codes we have in SearXNG
- related: https://github.com/searxng/searxng/issues/4885
|
|
The app and website were taken offline in December 2024, with the latter
pointing to KitchenAid's US website. [1]
[1] https://en.wikipedia.org/wiki/Yummly
Closes: https://github.com/searxng/searxng/issues/5079
|
|
* [fix] CI task "update_engine_traits.py" fails
To catch all problems with an HTTP request, the more general class
``httpx.HTTPError`` must be caught, for your test use::
$ ./manage dev.env
$ python ./searxng_extra/update/update_engine_traits.py
Closes: https://github.com/searxng/searxng/issues/5068
* [data] update searx.data - update_engine_traits.py
|
