summaryrefslogtreecommitdiff
path: root/searx/search/processors
AgeCommit message (Collapse)Author
2023-02-10[mod] make python code pylint 2.16.1 compliantMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-10-14searx.network: add "verify" option to the networksAlexandre Flament
Each network can define a verify option: * false to disable certificate verification * a path to existing certificate. SearXNG uses SSL_CERT_FILE and SSL_CERT_DIR when they are defined see https://www.python-httpx.org/environment_variables/#ssl_cert_file
2022-09-27[fix] typos / reported by @kianmeng in searx PR-3366Markus Heiser
[PR-3366] https://github.com/searx/searx/pull/3366 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-08-01[mod] add 'Accept-Language' HTTP header to online processoresMarkus Heiser
Most engines that support languages (and regions) use the Accept-Language from the WEB browser to build a response that fits to the language (and region). - add new engine option: send_accept_language_header Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-08-01[doc] add description of method EngineProcessor.get_params()Markus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-07Merge pull request #1443 from return42/fix-online_dictionaryMarkus Heiser
[fix] online_dictionary: regular expression
2022-07-07[fix] online_dictionary: regular expressionMarkus Heiser
The query term of a engine-type `online_dictionary` can consist of more than one word. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-06Better explanation for the use of use_mobile_uiÉmilien Devos
2022-06-11notify the user that use_mobile_ui parameter existEmilien Devos
2022-06-03[fix] prepare for pylint 2.14.0Markus Heiser
Remove issue reported by Pylint 2.14.0: - no-self-use: has been moved to optional extension [1] - The refactoring checker now also raises 'consider-using-generator' messages for max(), min() and sum(). [2] .pylintrc: - <option name>-hint has been removed since long, Pylint 2.14.0 raises an error on invalid options - bad-continuation and bad-whitespace have been removed [3] [1] https://pylint.pycqa.org/en/latest/whatsnew/2/2.14/summary.html#removed-checkers [2] https://pylint.pycqa.org/en/latest/whatsnew/2/2.14/full.html#what-s-new-in-pylint-2-14-0 [2] https://pylint.pycqa.org/en/latest/whatsnew/2/2.6/summary.html#summary-release-highlights Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-30[enh] implement a OnlineUrlSearchProcessorMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-17[typing] add type hints for dictionariesMartin Fischer
2021-12-27[format.python] initial formatting of the python codeMarkus Heiser
This patch was generated by black [1]:: make format.python [1] https://github.com/psf/black Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-27[format.python] disable py code formatting for some hunks of codeMarkus Heiser
Disable the python code formatting from python-black, where the readability of code suffers by formatting. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-17[pylint] fix global-variable-not-assigned issuesMarkus Heiser
If there is no write access, there is no need for global. Remove global statement if there is no assignment. global-variable-not-assigned: Using global for names but no assignment is done Used when a variable is defined through the "global" statement but no assignment to this variable is done. In Pylint 2.11 the global-variable-not-assigned checker now catches global variables that are never reassigned in a local scope and catches (reassigned) functions [1][2] [1] https://pylint.pycqa.org/en/latest/whatsnew/2.11.html [2] https://github.com/PyCQA/pylint/issues/1375 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-10[mod] searx.metrics & searx.search: use the engine loggersAlexandre Flament
metrics & processors use the engine logger
2021-09-07[pylint] searx: drop no longer needed 'missing-function-docstring'Markus Heiser
Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914168470 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-03[pylint] prepare for pylint v2.9.3 / fix some (new) pylint issuesMarkus Heiser
Upgrade from pylint v2.8.3 to 2.9.3 raise some new issues:: searx/search/checker/__main__.py:37:26: R1732: Consider using 'with' for resource-allocating operations (consider-using-with) searx/search/checker/__main__.py:38:26: R1732: Consider using 'with' for resource-allocating operations (consider-using-with) searx/search/processors/__init__.py:20:0: R0402: Use 'from searx import engines' instead (consider-using-from-import) searx/preferences.py:182:19: C0207: Use data.split('-', maxsplit=1)[0] instead (use-maxsplit-arg) searx/preferences.py:506:15: R1733: Unnecessary dictionary index lookup, use 'user_setting' instead (unnecessary-dict-index-lookup) searx/webapp.py:436:0: C0206: Consider iterating with .items() (consider-using-dict-items) searx/webapp.py:950:4: C0206: Consider iterating with .items() (consider-using-dict-items) Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04[fix] typo: online_dictionnary --> online_dictionaryMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-05[mod] multithreading only in searx.search.* packagesAlexandre Flament
it prepares the new architecture change, everything about multithreading in moved in the searx.search.* packages previously the call to the "init" function of the engines was done in searx.engines: * the network was not set (request not sent using the defined proxy) * it requires to monkey patch the code to avoid HTTP requests during the tests
2021-04-27[lint] pylint searx/search/processors files / BTW add some doc-stringsMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-27[mod] processors: show identical error messages on /search and /statsAlexandre Flament
2021-04-21[mod] metrics: add secondary parameterAlexandre Flament
Some error won't stop the engine: * additional HTTP redirects for example * some invalid results secondary=True allows to flag these errors as not important.
2021-04-21[enh] rewrite and enhance metricsAlexandre Flament
2021-04-21[mod] refactoring: processorsAlexandre Flament
Report to the user suspended engines. searx.search.processor.abstract: * manages suspend time (per network). * reports suspended time to the ResultContainer (method extend_container_if_suspended) * adds the results to the ResultContainer (method extend_container) * handles exceptions (method handle_exception)
2021-04-12[httpx] replace searx.poolrequests by searx.networkAlexandre Flament
settings.yml: * outgoing.networks: * can contains network definition * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections, keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time) * local_addresses can be "192.168.0.1/24" (it supports IPv6) * support_ipv4 & support_ipv6: both True by default see https://github.com/searx/searx/pull/1034 * each engine can define a "network" section: * either a full network description * either reference an existing network * all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-04-10[enh] replace requests by httpxAlexandre Flament
2021-03-25[fix] checker: various bug fixesAlexandre Flament
* initialize engine_data (youtube engine) * don't crash if an engine don't set result['url']
2021-03-08[mod] by default allow only HTTPS, not HTTPAlexandre Flament
Related to https://github.com/searx/searx/pull/2373
2021-02-23[mod] update currencies.json and fetch_currencies.pyAlexandre Flament
use a sparql request on wikidata to get the list of currencies. currencies.json contains the translation for all supported searx languages. Supersede #993
2021-02-12[fix] duckduckgo engine: "!ddg !g" do not redirect to googleAlexandre Flament
* searx understand "!ddg !g time" as : send "!g time" to DDG * !g a DDG bang for Google: DDG return a HTTP redirect to Google This commit adds a the allows_redirect param not to follow HTTP redirect. The DDG engine returns a empty result as before without HTTP redirect.
2021-02-09Fix: activate raise_for_error by defaultAlexandre Flament
Fix commit d703119d3a313a406482b121ee94c6afee3bc307 : Some engines need to parse the HTTP error but raise_for_error is always set to False in the "request" function.
2021-01-19[fix] checker: minor fix about language detectionAlexandre Flament
2021-01-18[fix] checker: fix engine statisticsAlexandre Flament
Without this commit, the URL /stats/errors shows percentage above 100% after the checker has run.
2021-01-12[mod] checker: minor adjustements on the default testsAlexandre Flament
the query "time" is convinient because most of the search engine will return some results, but some engines in the general category will return documentation about the HTML tags <time> or <input type="time">
2021-01-12[enh] add checkerAlexandre Flament
2020-12-17[fix] fix of PR #2225Alexandre Flament
2020-12-17[mod] split searx.search into different processorsAlexandre Flament
see searx.search.processors.abstract.EngineProcessor First the method searx call the get_params method. If the return value is not None, then the searx call the method search.
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-12 03:00:49 +0000