summaryrefslogtreecommitdiff
path: root/searx/network/client.py
AgeCommit message (Collapse)Author
2025-12-26[fix] searx.network: fix string concatenation of proxy error messageAadniz
2025-09-20[mod] py: remove uvloop (#5220)Ivan Gabaldon
We get some good stuff without uvloop, 13MB~ less of dependencies, 3 minutes of build time for armv7 saved, and we are one step closer to NT compatibility. Although it's true that theoretically the raw performance have worsened on network side (we only used uvloop for that), the latest cpython versions have been improving on asyncio performance.
2025-09-18[mod] addition of various type hints / engine processorsMarkus Heiser
Continuation of #5147 .. typification of the engine processors. BTW: - removed obsolete engine property https_support - fixed & improved currency_convert - engine instances can now implement a engine.setup method [#5147] https://github.com/searxng/searxng/pull/5147 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-03[mod] addition of various type hints / tbcMarkus Heiser
- pyright configuration [1]_ - stub files: types-lxml [2]_ - addition of various type hints - enable use of new type system features on older Python versions [3]_ - ``.tool-versions`` - set python to lowest version we support (3.10.18) [4]_: Older versions typically lack some typing features found in newer Python versions. Therefore, for local type checking (before commit), it is necessary to use the older Python interpreter. .. [1] https://docs.basedpyright.com/v1.20.0/configuration/config-files/ .. [2] https://pypi.org/project/types-lxml/ .. [3] https://typing-extensions.readthedocs.io/en/latest/# .. [4] https://mise.jdx.dev/configuration.html#tool-versions Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> Format: reST
2025-05-20[mod] upgrade to httpx 0.28.1 (#4674)Alexandre Flament
2024-07-13[mod] remove py 3.6 leftoversAllen
2024-03-11[mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTIONMarkus Heiser
In the past, some files were tested with the standard profile, others with a profile in which most of the messages were switched off ... some files were not checked at all. - ``PYLINT_SEARXNG_DISABLE_OPTION`` has been abolished - the distinction ``# lint: pylint`` is no longer necessary - the pylint tasks have been reduced from three to two 1. ./searx/engines -> lint engines with additional builtins 2. ./searx ./searxng_extra ./tests -> lint all other python files Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-27[mod] searx.network: memory optimizationAlexandre Flament
Avoid to create a SSLContext in AsyncHTTPTransportNoHttp See: * https://github.com/encode/httpx/blob/0f61aa58d66680c239ce43c8cdd453e7dc532bfc/httpx/_transports/default.py#L271 * https://github.com/encode/httpx/issues/2298
2023-08-21Bump httpx 0.21.2 from to 0.24.1Alexandre Flament
2023-03-19[mod] Shuffle httpx's default ciphers of a SSL context randomly.Markus Heiser
From the analyse of @9Ninety [1] we know that DDG (and may be other engines / I have startpage in mind) does some kind of TLS fingerprint to block bots. This patch shuffles the default ciphers from httpx to avoid a cipher profile that is known to httpx (and blocked by DDG). [1] https://github.com/searxng/searxng/issues/2246#issuecomment-1467895556 ---- From `What Is TLS Fingerprint and How to Bypass It`_ > When implementing TLS fingerprinting, servers can't operate based on a > locked-in whitelist database of fingerprints. New fingerprints appear > when web clients or TLS libraries release new versions. So, they have to > live off a blocklist database instead. > ... > It's safe to leave the first three as is but shuffle the remaining ciphers > and you can bypass the TLS fingerprint check. .. _What Is TLS Fingerprint and How to Bypass It: https://www.zenrows.com/blog/what-is-tls-fingerprint#how-to-bypass-tls-fingerprinting Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> Closes: https://github.com/searxng/searxng/issues/2246
2022-10-14searx.network: add "verify" option to the networksAlexandre Flament
Each network can define a verify option: * false to disable certificate verification * a path to existing certificate. SearXNG uses SSL_CERT_FILE and SSL_CERT_DIR when they are defined see https://www.python-httpx.org/environment_variables/#ssl_cert_file
2022-01-17[typing] add type hints for dictionariesMartin Fischer
2022-01-05[mod] upgrade httpx 0.21.2Alexandre Flament
httpx 0.21.2 and httpcore 0.14.4 fix multiple issues: * https://github.com/encode/httpx/releases/tag/0.21.2 * https://github.com/encode/httpcore/releases/tag/0.14.4 so most of the workarounds in searx.network have been removed.
2021-12-27[format.python] initial formatting of the python codeMarkus Heiser
This patch was generated by black [1]:: make format.python [1] https://github.com/psf/black Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-28[fix] searx.network.stream: fix memory leakAlexandre Flament
2021-09-17Merge pull request #261 from dalf/upgrade_httpxAlexandre Flament
[upd] upgrade httpx 0.19.0
2021-09-17[pylint] fix global-variable-not-assigned issuesMarkus Heiser
If there is no write access, there is no need for global. Remove global statement if there is no assignment. global-variable-not-assigned: Using global for names but no assignment is done Used when a variable is defined through the "global" statement but no assignment to this variable is done. In Pylint 2.11 the global-variable-not-assigned checker now catches global variables that are never reassigned in a local scope and catches (reassigned) functions [1][2] [1] https://pylint.pycqa.org/en/latest/whatsnew/2.11.html [2] https://github.com/PyCQA/pylint/issues/1375 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-17[mod] searx.network: remove redundant codeAlexandre Flament
searx.client.new_client: the proxies parameter is a dictonnary, and the protocol (key of the dictionnary) is already normalized (see usage of searx.network.network.PROXY_PATTERN_MAPPING)
2021-09-17[upd] upgrade httpx 0.19.0Alexandre Flament
adjust searx.network module to the new internal API see https://github.com/encode/httpx/pull/1522
2021-09-11[mod] debug mode: log HTTP requests with network nameAlexandre Flament
For example wikipedia requests use the logger name "searx.network.wikipedia" Log is disable when searx_debug is False
2021-09-07[pylint] searx: drop no longer needed 'missing-function-docstring'Markus Heiser
Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914168470 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-24[coding-style] searx/network/client.py - normalized indentationsMarkus Heiser
No functional change! Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-24[pylint] searx/network/client.py & add global (TRANSPORT_KWARGS)Markus Heiser
No functional change! - fix messages from pylint - add ``global TRANSPORT_KWARGS`` - normalized python_socks imports Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-05[mod] searx.network.client: the same configuration reuses the same ↵Alexandre Flament
ssl.SSLContext before there was one ssl.SSLContext per client. see https://github.com/encode/httpx/issues/978
2021-04-27[fix] searx.network: fix rare cases where LOOP is NoneAlexandre Flament
* searx.network.client.LOOP is initialized in a thread * searx.network.__init__ imports LOOP which may happen before the thread has initialized LOOP This commit adds a new function "searx.network.client.get_loop()" to fix this issue
2021-04-12[httpx] replace searx.poolrequests by searx.networkAlexandre Flament
settings.yml: * outgoing.networks: * can contains network definition * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections, keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time) * local_addresses can be "192.168.0.1/24" (it supports IPv6) * support_ipv4 & support_ipv6: both True by default see https://github.com/searx/searx/pull/1034 * each engine can define a "network" section: * either a full network description * either reference an existing network * all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-03 22:15:43 +0000