summaryrefslogtreecommitdiff
path: root/searx/search/processors/online.py
AgeCommit message (Collapse)Author
2023-02-10[mod] make python code pylint 2.16.1 compliantMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-10-14searx.network: add "verify" option to the networksAlexandre Flament
Each network can define a verify option: * false to disable certificate verification * a path to existing certificate. SearXNG uses SSL_CERT_FILE and SSL_CERT_DIR when they are defined see https://www.python-httpx.org/environment_variables/#ssl_cert_file
2022-09-27[fix] typos / reported by @kianmeng in searx PR-3366Markus Heiser
[PR-3366] https://github.com/searx/searx/pull/3366 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-08-01[mod] add 'Accept-Language' HTTP header to online processoresMarkus Heiser
Most engines that support languages (and regions) use the Accept-Language from the WEB browser to build a response that fits to the language (and region). - add new engine option: send_accept_language_header Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-06Better explanation for the use of use_mobile_uiÉmilien Devos
2022-06-11notify the user that use_mobile_ui parameter existEmilien Devos
2021-12-27[format.python] initial formatting of the python codeMarkus Heiser
This patch was generated by black [1]:: make format.python [1] https://github.com/psf/black Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-27[format.python] disable py code formatting for some hunks of codeMarkus Heiser
Disable the python code formatting from python-black, where the readability of code suffers by formatting. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-10[mod] searx.metrics & searx.search: use the engine loggersAlexandre Flament
metrics & processors use the engine logger
2021-05-05[mod] multithreading only in searx.search.* packagesAlexandre Flament
it prepares the new architecture change, everything about multithreading in moved in the searx.search.* packages previously the call to the "init" function of the engines was done in searx.engines: * the network was not set (request not sent using the defined proxy) * it requires to monkey patch the code to avoid HTTP requests during the tests
2021-04-27[lint] pylint searx/search/processors files / BTW add some doc-stringsMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-27[mod] processors: show identical error messages on /search and /statsAlexandre Flament
2021-04-21[mod] metrics: add secondary parameterAlexandre Flament
Some error won't stop the engine: * additional HTTP redirects for example * some invalid results secondary=True allows to flag these errors as not important.
2021-04-21[enh] rewrite and enhance metricsAlexandre Flament
2021-04-21[mod] refactoring: processorsAlexandre Flament
Report to the user suspended engines. searx.search.processor.abstract: * manages suspend time (per network). * reports suspended time to the ResultContainer (method extend_container_if_suspended) * adds the results to the ResultContainer (method extend_container) * handles exceptions (method handle_exception)
2021-04-12[httpx] replace searx.poolrequests by searx.networkAlexandre Flament
settings.yml: * outgoing.networks: * can contains network definition * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections, keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time) * local_addresses can be "192.168.0.1/24" (it supports IPv6) * support_ipv4 & support_ipv6: both True by default see https://github.com/searx/searx/pull/1034 * each engine can define a "network" section: * either a full network description * either reference an existing network * all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-04-10[enh] replace requests by httpxAlexandre Flament
2021-03-08[mod] by default allow only HTTPS, not HTTPAlexandre Flament
Related to https://github.com/searx/searx/pull/2373
2021-02-12[fix] duckduckgo engine: "!ddg !g" do not redirect to googleAlexandre Flament
* searx understand "!ddg !g time" as : send "!g time" to DDG * !g a DDG bang for Google: DDG return a HTTP redirect to Google This commit adds a the allows_redirect param not to follow HTTP redirect. The DDG engine returns a empty result as before without HTTP redirect.
2021-02-09Fix: activate raise_for_error by defaultAlexandre Flament
Fix commit d703119d3a313a406482b121ee94c6afee3bc307 : Some engines need to parse the HTTP error but raise_for_error is always set to False in the "request" function.
2021-01-19[fix] checker: minor fix about language detectionAlexandre Flament
2021-01-12[mod] checker: minor adjustements on the default testsAlexandre Flament
the query "time" is convinient because most of the search engine will return some results, but some engines in the general category will return documentation about the HTML tags <time> or <input type="time">
2021-01-12[enh] add checkerAlexandre Flament
2020-12-17[fix] fix of PR #2225Alexandre Flament
2020-12-17[mod] split searx.search into different processorsAlexandre Flament
see searx.search.processors.abstract.EngineProcessor First the method searx call the get_params method. If the return value is not None, then the searx call the method search.