summaryrefslogtreecommitdiff
path: root/searx/engines
AgeCommit message (Collapse)Author
2021-04-21[mod] refactoring: processorsAlexandre Flament
Report to the user suspended engines. searx.search.processor.abstract: * manages suspend time (per network). * reports suspended time to the ResultContainer (method extend_container_if_suspended) * adds the results to the ResultContainer (method extend_container) * handles exceptions (method handle_exception)
2021-04-19Merge remote-tracking branch 'searx/master'Alexandre Flament
2021-04-16Merge pull request #2736 from plague-doctor/sjpNoémi Ványi
Add new engine: SJP - Słownik języka polskiego
2021-04-16Merge pull request #2759 from ypid/fix/typoNoémi Ványi
Fix grammar mistake in debug log output
2021-04-16Code refactoring.Plague Doctor
2021-04-15[fix] xpath expressions to grap all items from bandcamp's responseMarkus Heiser
I also found some items missing a thumbnail and I used text_extract for content and title, to remove unneeded whitespaces. BTW: added bandcamp's favicon Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-15[feat] add bandcamp engineKyle Anthony Williams
2021-04-12[httpx] replace searx.poolrequests by searx.networkAlexandre Flament
settings.yml: * outgoing.networks: * can contains network definition * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections, keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time) * local_addresses can be "192.168.0.1/24" (it supports IPv6) * support_ipv4 & support_ipv6: both True by default see https://github.com/searx/searx/pull/1034 * each engine can define a "network" section: * either a full network description * either reference an existing network * all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-04-11Fix grammar mistake in debug log outputRobin Schneider
2021-04-10[enh] replace requests by httpxAlexandre Flament
2021-04-09Fix conflictsPlague Doctor
2021-04-08Add new engine: SJPPlague Doctor
2021-04-08Add new engine: Wordnik.comPlague Doctor
2021-04-06Add Meilisearch engineNoémi Ványi
Website: https://www.meilisearch.com/
2021-03-27Merge pull request #2681 from dalf/fix-wikipedia-titleAlexandre Flament
[fix] wikipedia: remove HTML from the title
2021-03-27Remove duplicated key from dict in Semantic ScholarNoémi Ványi
2021-03-27fix Semantic Scholar engineNoémi Ványi
2021-03-27[fix] update seznam engine to be compatible with the new websiteAdam Tauber
2021-03-25[fix] wikipedia: remove HTML from the titleAlexandre Flament
fr.wikipedia.org (and it seems not other wikipedia websites), adds HTML to api_result['displayTitle']. (Search for '!wp :fr Braid' for example) The commit uses api_result['title']
2021-03-25[fix] make ina engine compatible with the new response jsonAdam Tauber
2021-03-25[enh] add year filter to duckduckgoAdam Tauber
2021-03-24[fix] remove debug codeAdam Tauber
2021-03-21[mod] soundcloud: faster initializationAlexandre Flament
The get_cliend_id() function: * fetches https://soundcloud.com * then fetches each referenced javascript URL to get the client id. This commit fetches the javascript URLs in the reverse order: the client id is in the last javascript URL.
2021-03-15[fix] remove debug codeAdam Tauber
2021-03-15fix Microsoft Academic engineNoémi Ványi
2021-03-15[fix] update youtube_noapi pagingAdam Tauber
2021-03-15Merge pull request #2652 from kvch/solr-engineAdam Tauber
Add Apache Solr engine
2021-03-15Merge pull request #2655 from return42/fix-importsAlexandre Flament
[fix] remove unused import from yahoo-news engine
2021-03-14[fix] remove unused import from yahoo-news engineMarkus Heiser
Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-03-14[drop] Acgsou engine - www.acgsou.com no longer existsMarkus Heiser
- https://www.acgsou.com/ acgsou.com is redirected to 36dm.club - @rinpatch do not plan on maintaining the engine [1] [1] https://github.com/searx/searx/pull/1283#issuecomment-798783585 Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-03-13Add Solr engineNoémi Ványi
2021-03-12Merge pull request #2626 from mikeri/solidtorrentsAlexandre Flament
Add Solid Torrents engine
2021-03-12Merge pull request #2641 from dalf/disable_http_by_defaultAlexandre Flament
[mod] by default allow only HTTPS, not HTTP
2021-03-09[fix] APKMirror engine - update xpath selectors and fix img_srcMarkus Heiser
BTW: make the code slightly more readable Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-08[fix] rewrite Yahoo-News engineMarkus Heiser
Many things have been changed since last review of this engine. This patch fix xpath selectors, implements suggestion and is a complete review / rewrite of the engine. Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-03-08[mod] by default allow only HTTPS, not HTTPAlexandre Flament
Related to https://github.com/searx/searx/pull/2373
2021-03-07Add Solid Torrents engineMichael Ilsaas
2021-03-06[enh] add ability to send engine data to subsequent requestsAdam Tauber
2021-03-05[mod] don't dump traceback of SearxEngineResponseException on initMarkus Heiser
When initing engines a "SearxEngineResponseException" is logged very verbose, including full traceback information: ERROR:searx.engines:yggtorrent engine: Fail to initialize Traceback (most recent call last): File "share/searx/searx/engines/__init__.py", line 293, in engine_init init_fn(get_engine_from_settings(engine_name)) File "share/searx/searx/engines/yggtorrent.py", line 42, in init resp = http_get(url, allow_redirects=False) File "share/searx/searx/poolrequests.py", line 197, in get return request('get', url, **kwargs) File "share/searx/searx/poolrequests.py", line 190, in request raise_for_httperror(response) File "share/searx/searx/raise_for_httperror.py", line 60, in raise_for_httperror raise_for_captcha(resp) File "share/searx/searx/raise_for_httperror.py", line 43, in raise_for_captcha raise_for_cloudflare_captcha(resp) File "share/searx/searx/raise_for_httperror.py", line 30, in raise_for_cloudflare_captcha raise SearxEngineCaptchaException(message='Cloudflare CAPTCHA', suspended_time=3600 * 24 * 15) searx.exceptions.SearxEngineCaptchaException: Cloudflare CAPTCHA, suspended_time=1296000 For SearxEngineResponseException this is not needed. Those types of exceptions can be a normal use case. E.g. for CAPTCHA errors like shown in the example above. It should be enough to log a warning for such issues: WARNING:searx.engines:yggtorrent engine: Fail to initialize // Cloudflare CAPTCHA, suspended_time=1296000 closes: #2612 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-01[enh] google scholar - python implementation of the engineMarkus Heiser
The old xpath configuration for google scholar did not work and is replaced by a python implementation. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-01Merge pull request #2602 from MarcAbonce/fix-bing-fetch-languagesAlexandre Flament
Fix fetch_languages for Bing
2021-03-01Add Freesound engine (#2596)GazoilKerozen
Add freesound engine with player. Co-authored-by: Gazoil <maildeguzel@gmail.com>
2021-02-25remove articles number from engines_languages.jsonMarc Abonce Seguin
2021-02-25fix fetch_languages for bingMarc Abonce Seguin
Bing has a list of regions that it supports and some of these regions may have more than one possible language. In some cases, like Switzerland, these languages are always shown as options, so there is no issue. But in other cases, like Andorra, Bing will only show one language at the time, either the region's default or the request's language if the latter is supported by that region. For example, if the HTTP request is in French, Andorra will appear as fr-AD but if the same page is requested in any other language Andorra will appear as ca-AD. This is specially a problem when Bing assumes that the request is in English because it overrides enough language codes to make several major languages like Arabic dissappear from the languages.py file. To avoid that issue, I set the Accept-Language header to a language that's only supported in one region to hopefully avoid these overrides.
2021-02-22Fix paging of Bing ImagesNoémi Ványi
2021-02-20Added rumble.com video search engine. TODO video embedding.datagram1
Update rumble.py some lines too long. Disable Rumble engine disabled : True PEP8 fix change line spacing
2021-02-16Merge pull request #2573 from unixfox/yggtorrentAlexandre Flament
update yggtorrent url + add it back
2021-02-15fix yggtorrent url + add it backEmilien Devos
2021-02-13Improve peertube searchingThorben Günther
At the moment videos without a description are not shown - setting default content to "" fixes this. Another current bug is that thumbnails are not displayed. This is caused by a double slash in the url. For this every trailing slash is now stripped (for backwards compatibility) and the API response is correctly parsed.
2021-02-12Merge pull request #2566 from dalf/remove-yandexAlexandre Flament
[remove] yandex engine
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-22 08:37:47 +0000