summaryrefslogtreecommitdiff
path: root/searx/engines/duckduckgo.py
AgeCommit message (Collapse)Author
2023-09-05[fix] engine - duckduckgo_images / determination of vqd value incorrectMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-25[fix] engine & network issues / documentation and type annotationsMarkus Heiser
This patch fixes some quirks and issues related to the engines and the network. Each engine has its own network and this network was broken for the following engines[1]: - archlinux - bing - dailymotion - duckduckgo - google - peertube - startpage - wikipedia Since the files have been touched anyway, the type annotaions of the engine modules has also been completed so that error messages from the type checker are no longer reported. Related and (partial) fixed issue: - [1] https://github.com/searxng/searxng/issues/762#issuecomment-1605323861 - [2] https://github.com/searxng/searxng/issues/2513 - [3] https://github.com/searxng/searxng/issues/2515 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-12[fix] engine ddg: minor change in the API of ddgMarkus Heiser
Closes: https://github.com/searxng/searxng/issues/2419 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-04-03[fix] engine ddg: quote !bangs in a request send to ddgMarkus Heiser
Closes: https://github.com/searxng/searxng/issues/392 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24[mod] DuckDuckGo: reversed engineered & upgrade to data_type: traits_v1Markus Heiser
Partial reverse engineering of the DuckDuckGo (DDG) engines including a improved language and region handling based on the enigne.traits_v1 data. - DDG Lite - DDG Instant Answer API - DDG Images - DDG Weather docs/src/searx.engine.duckduckgo.rst: Online documentation of the DDG engines (make docs.live) searx/data/engine_traits.json Add data type "traits_v1" generated by the fetch_traits() functions from: - "duckduckgo" (WEB), - "duckduckgo images" and - "duckduckgo weather" and remove data from obsolete data type "supported_languages". searx/autocomplete.py: Reversed engineered Autocomplete from DDG. Supports DDG's languages. searx/engines/duckduckgo.py: - fetch_traits(): Fetch languages & regions from DDG. - get_ddg_lang(): Get DDG's language identifier from SearXNG's locale. DDG defines its languages by region codes. DDG-Lite does not offer a language selection to the user, only a region can be selected by the user. - Cache ``vqd`` value: The vqd value depends on the query string and is needed for the follow up pages or the images loaded by a XMLHttpRequest (DDG images). The ``vqd`` value of a search term is stored for 10min in the redis DB. - DDG Lite engine: reversed engineered request method with improved Language and region support and better ``vqd`` handling. searx/engines/duckduckgo_definitions.py: DDG Instant Answer API The *instant answers* API does not support languages, or at least we could not find out how language support should work. It seems that most of the features are based on English terms. searx/engines/duckduckgo_images.py: DDG Images Reversed engineered request method. Improved language and region handling based on cookies and the enigne.traits_v1 data. Response: add image format to the result list searx/engines/duckduckgo_weather.py: DDG Weather Improved language and region handling based on cookies and the enigne.traits_v1 data. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24[mod] DuckDuckGo: fetch engine traits (data_type: supported_languages)Markus Heiser
Implements a fetch_traits function for the DuckDuckGo engines. .. note:: Does not include migration of the request methode from 'supported_languages' to 'traits' (EngineTraits) object! Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-03[fix-2146] set different HTTP Referer header to DuckDuckGo requestsMarkus Heiser
For what ever reasons, ddg-lite don't like the Referer https://lite.duckduckgo.com/ In an interactive session in the WEB browser the the Reverer has exactly this value, but ddg-lite don't like this value when the request is build up by SearXNG. The new value is: https://google.com/ What fakes a user comes from a google link. Related: https://github.com/searxng/searxng/pull/2081 Closes: https://github.com/searxng/searxng/issues/2146 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-01-06Add HTTP Referer header to DuckDuckGo requestsRudis Muiznieks
closes #2080
2022-12-22Fix: add trailing slash to duckduckgo urlRudis Muiznieks
Close #1854
2022-08-01[mod] add 'Accept-Language' HTTP header to online processoresMarkus Heiser
Most engines that support languages (and regions) use the Accept-Language from the WEB browser to build a response that fits to the language (and region). - add new engine option: send_accept_language_header Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-05[enh] add more categoriesMartin Fischer
2021-12-27[format.python] initial formatting of the python codeMarkus Heiser
This patch was generated by black [1]:: make format.python [1] https://github.com/psf/black Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-01[mod] engine duckduckgo - update supported_languages_urlMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-01[mod] engine duckduckgo - use DuckDuckGo-LiteMarkus Heiser
Implement a scrapper for DuckDuckGo-Lite [1]. The existing DuckDuckGo [2] engine does not support paging. DuckDuckgo-Lite is much faster, less verbose and does have a paging option (reversed engineered from the input form of [1]). [1] https://lite.duckduckgo.com/lite [2] https://duckduckgo.com/ Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-12[httpx] replace searx.poolrequests by searx.networkAlexandre Flament
settings.yml: * outgoing.networks: * can contains network definition * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections, keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time) * local_addresses can be "192.168.0.1/24" (it supports IPv6) * support_ipv4 & support_ipv6: both True by default see https://github.com/searx/searx/pull/1034 * each engine can define a "network" section: * either a full network description * either reference an existing network * all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-03-25[enh] add year filter to duckduckgoAdam Tauber
2021-02-12[fix] duckduckgo engine: "!ddg !g" do not redirect to googleAlexandre Flament
* searx understand "!ddg !g time" as : send "!g time" to DDG * !g a DDG bang for Google: DDG return a HTTP redirect to Google This commit adds a the allows_redirect param not to follow HTTP redirect. The DDG engine returns a empty result as before without HTTP redirect.
2021-02-09[mod] duckduckgo engine: better support of the language preferenceAlexandre Flament
After the main request, send a second to https://duckduckgo.com/t/sl_h See https://github.com/searx/searx/issues/2259
2021-02-01[mod] dynamically set language_support variableAlexandre Flament
The language_support variable is set to True by default, and set to False in only 5 engines. Except the documentation and the /config URL, this variable is not used. This commit remove the variable definition in the engines, and set value according to supported_languages length: False when the length is 0, True otherwise. Close #2485
2021-01-14[enh] engines: add about variableAlexandre Flament
move meta information from comment to the about variable so the preferences, the documentation can show these information
2020-12-01[mod] pylint: numerous minor code fixesAlexandre Flament
2020-11-14[mod] remove unused importAlexandre Flament
use from searx.engines.duckduckgo import _fetch_supported_languages, supported_languages_url # NOQA so it is possible to easily remove all unused import using autoflake: autoflake --in-place --recursive --remove-all-unused-imports searx tests
2020-10-09[fix] fix duckduckgo engineAlexandre Flament
- remove paging support: a "vqd" parameter is required between each request. This parameter is uniq for each request - update the URL (no redirect), use the POST method - language support: works if there is no more than request per minute, otherwise it is ignored !
2020-10-02[mod] move extract_text, extract_url to searx.utilsAlexandre Flament
2020-09-10Drop Python 2 (1/n): remove unicode string and url_utilsDalf
2020-06-13add correction support for duckduckgoGordon Quad
2019-11-15[mod] speed optimizationDalf
compile XPath only once avoid redundant call to urlparse get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
2019-10-14[fix] pep8Adam Tauber
2019-10-14[fix] duckduckgo paging - closes #1677Adam Tauber
2019-01-07Revert "remove 'all' option from search languages"Noémi Ványi
This reverts commit 4d1770398a6af8902e75c0bd885781584d39e796.
2018-03-27refactor engine's search language handlingMarc Abonce Seguin
Add match_language function in utils to match any user given language code with a list of engine's supported languages. Also add language_aliases dict on each engine to translate standard language codes into the custom codes used by the engine.
2018-03-20fix fetch_langauges to be more accurateMarc Abonce Seguin
Add languages supported by either all default general engines or 10 engines.
2018-03-14[fix] change duckduckgo url to avoid error responseMarc Abonce Seguin
2017-12-06remove 'all' option from search languagesmarc
2017-10-10update engines_languages.json and languages.pymarc
Also, fix fetch_languages.py so it can run on python3.
2017-07-21[fix] use poolrequests in ddg enginesAdam Tauber
2017-07-10[fix] duckduckgo images doesn't fail with countryless languagemarc
2017-05-27[fix] fix duckduckgo's offsetmarc
First page now starts with 0 offset, rather than starting on the 30th result. DuckDuckGo returns 30 results on each page.
2017-05-23add duckduckgo images enginemarc
2017-05-15[enh] py3 compatibilityAdam Tauber
2017-02-01[fix] fix paging of duckduckgoNoémi Ványi
2016-12-15tests for _fetch_supported_languages in enginesmarc
and refactor method to make it testable without making requests
2016-12-13tests for language support in enginesmarc
2016-12-13[mod] fetch supported languages for several enginesmarc
utils/fetch_languages.py gets languages supported by each engine and generates engines_languages.json with each engine's supported language.
2016-12-13Add language support for more engines.marc
2016-12-13[enh] add supported_languages on engines and auto-generate languages.pymarc
2016-12-11add year filter to engines with time range support && testsNoémi Ványi
Following engines does not support "Last year": * Bing News * DeviantArt * DuckDuckGo * Yahoo * YouTube (noapi)
2016-07-26[fix] time range detectionAdam Tauber
2016-07-25add time range search for duckduckgoNoemi Vanyi
2016-07-14[fix] guess country code if not specified in duckduckgo enginemarc
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-30 13:30:55 +0000