summaryrefslogtreecommitdiff
path: root/searx/engines
AgeCommit message (Collapse)Author
2021-04-22Merge pull request #15 from return42/add-springerAlexandre Flament
Add a search engine for Springer Nature
2021-04-22[enh] Add Springer Nature enginePierre Chevalier
Springer Nature is a global publisher dedicated to providing service to research community [1] with official API [2]. To test this PR, first get your API key following this page: https://dev.springernature.com/signup In searx/engines/springer.py at line 24, add this API key. I left my own key, commented out in the line aboce. Feel free to use it, if needed. [1] https://www.springernature.com/ [2] https://dev.springernature.com/
2021-04-22[enh] add engine using API from "The Art Institute of Chicago"habsinn
2021-04-22[fix] youtube - send CONSENT Cookie to not be redirectedMarkus Heiser
In the EU there exists a "General Data Protection Regulation" [1] aka GDPR (BTW: very user friendly!) which requires consent to tracking. To get the consent from the user, youtube requests are redirected to confirm and get a CONSENT Cookie from https://consent.youtube.com This patch adds a CONSENT Cookie to the youtube request to avoid redirection. [1] https://en.wikipedia.org/wiki/General_Data_Protection_Regulation Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> Reported-by: https://github.com/searx/searx/issues/2774
2021-04-22Merge pull request #7 from searxng/metricsAlexandre Flament
Metrics
2021-04-21[fix] searxng fix: sjp engineAlexandre Flament
2021-04-21[enh] rewrite and enhance metricsAlexandre Flament
2021-04-21[mod] refactoring: processorsAlexandre Flament
Report to the user suspended engines. searx.search.processor.abstract: * manages suspend time (per network). * reports suspended time to the ResultContainer (method extend_container_if_suspended) * adds the results to the ResultContainer (method extend_container) * handles exceptions (method handle_exception)
2021-04-19Merge remote-tracking branch 'searx/master'Alexandre Flament
2021-04-16Merge pull request #2736 from plague-doctor/sjpNoémi Ványi
Add new engine: SJP - Słownik języka polskiego
2021-04-16Merge pull request #2759 from ypid/fix/typoNoémi Ványi
Fix grammar mistake in debug log output
2021-04-16Code refactoring.Plague Doctor
2021-04-15[fix] xpath expressions to grap all items from bandcamp's responseMarkus Heiser
I also found some items missing a thumbnail and I used text_extract for content and title, to remove unneeded whitespaces. BTW: added bandcamp's favicon Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-15[feat] add bandcamp engineKyle Anthony Williams
2021-04-12[httpx] replace searx.poolrequests by searx.networkAlexandre Flament
settings.yml: * outgoing.networks: * can contains network definition * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections, keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time) * local_addresses can be "192.168.0.1/24" (it supports IPv6) * support_ipv4 & support_ipv6: both True by default see https://github.com/searx/searx/pull/1034 * each engine can define a "network" section: * either a full network description * either reference an existing network * all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-04-11Fix grammar mistake in debug log outputRobin Schneider
2021-04-10[enh] replace requests by httpxAlexandre Flament
2021-04-09Fix conflictsPlague Doctor
2021-04-08Add new engine: SJPPlague Doctor
2021-04-08Add new engine: Wordnik.comPlague Doctor
2021-04-06Add Meilisearch engineNoémi Ványi
Website: https://www.meilisearch.com/
2021-03-27Merge pull request #2681 from dalf/fix-wikipedia-titleAlexandre Flament
[fix] wikipedia: remove HTML from the title
2021-03-27Remove duplicated key from dict in Semantic ScholarNoémi Ványi
2021-03-27fix Semantic Scholar engineNoémi Ványi
2021-03-27[fix] update seznam engine to be compatible with the new websiteAdam Tauber
2021-03-25[fix] wikipedia: remove HTML from the titleAlexandre Flament
fr.wikipedia.org (and it seems not other wikipedia websites), adds HTML to api_result['displayTitle']. (Search for '!wp :fr Braid' for example) The commit uses api_result['title']
2021-03-25[fix] make ina engine compatible with the new response jsonAdam Tauber
2021-03-25[enh] add year filter to duckduckgoAdam Tauber
2021-03-24[fix] remove debug codeAdam Tauber
2021-03-21[mod] soundcloud: faster initializationAlexandre Flament
The get_cliend_id() function: * fetches https://soundcloud.com * then fetches each referenced javascript URL to get the client id. This commit fetches the javascript URLs in the reverse order: the client id is in the last javascript URL.
2021-03-15[fix] remove debug codeAdam Tauber
2021-03-15fix Microsoft Academic engineNoémi Ványi
2021-03-15[fix] update youtube_noapi pagingAdam Tauber
2021-03-15Merge pull request #2652 from kvch/solr-engineAdam Tauber
Add Apache Solr engine
2021-03-15Merge pull request #2655 from return42/fix-importsAlexandre Flament
[fix] remove unused import from yahoo-news engine
2021-03-14[fix] remove unused import from yahoo-news engineMarkus Heiser
Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-03-14[drop] Acgsou engine - www.acgsou.com no longer existsMarkus Heiser
- https://www.acgsou.com/ acgsou.com is redirected to 36dm.club - @rinpatch do not plan on maintaining the engine [1] [1] https://github.com/searx/searx/pull/1283#issuecomment-798783585 Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-03-13Add Solr engineNoémi Ványi
2021-03-12Merge pull request #2626 from mikeri/solidtorrentsAlexandre Flament
Add Solid Torrents engine
2021-03-12Merge pull request #2641 from dalf/disable_http_by_defaultAlexandre Flament
[mod] by default allow only HTTPS, not HTTP
2021-03-09[fix] APKMirror engine - update xpath selectors and fix img_srcMarkus Heiser
BTW: make the code slightly more readable Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-08[fix] rewrite Yahoo-News engineMarkus Heiser
Many things have been changed since last review of this engine. This patch fix xpath selectors, implements suggestion and is a complete review / rewrite of the engine. Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-03-08[mod] by default allow only HTTPS, not HTTPAlexandre Flament
Related to https://github.com/searx/searx/pull/2373
2021-03-07Add Solid Torrents engineMichael Ilsaas
2021-03-06[enh] add ability to send engine data to subsequent requestsAdam Tauber
2021-03-05[mod] don't dump traceback of SearxEngineResponseException on initMarkus Heiser
When initing engines a "SearxEngineResponseException" is logged very verbose, including full traceback information: ERROR:searx.engines:yggtorrent engine: Fail to initialize Traceback (most recent call last): File "share/searx/searx/engines/__init__.py", line 293, in engine_init init_fn(get_engine_from_settings(engine_name)) File "share/searx/searx/engines/yggtorrent.py", line 42, in init resp = http_get(url, allow_redirects=False) File "share/searx/searx/poolrequests.py", line 197, in get return request('get', url, **kwargs) File "share/searx/searx/poolrequests.py", line 190, in request raise_for_httperror(response) File "share/searx/searx/raise_for_httperror.py", line 60, in raise_for_httperror raise_for_captcha(resp) File "share/searx/searx/raise_for_httperror.py", line 43, in raise_for_captcha raise_for_cloudflare_captcha(resp) File "share/searx/searx/raise_for_httperror.py", line 30, in raise_for_cloudflare_captcha raise SearxEngineCaptchaException(message='Cloudflare CAPTCHA', suspended_time=3600 * 24 * 15) searx.exceptions.SearxEngineCaptchaException: Cloudflare CAPTCHA, suspended_time=1296000 For SearxEngineResponseException this is not needed. Those types of exceptions can be a normal use case. E.g. for CAPTCHA errors like shown in the example above. It should be enough to log a warning for such issues: WARNING:searx.engines:yggtorrent engine: Fail to initialize // Cloudflare CAPTCHA, suspended_time=1296000 closes: #2612 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-01[enh] google scholar - python implementation of the engineMarkus Heiser
The old xpath configuration for google scholar did not work and is replaced by a python implementation. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-01Merge pull request #2602 from MarcAbonce/fix-bing-fetch-languagesAlexandre Flament
Fix fetch_languages for Bing
2021-03-01Add Freesound engine (#2596)GazoilKerozen
Add freesound engine with player. Co-authored-by: Gazoil <maildeguzel@gmail.com>
2021-02-25remove articles number from engines_languages.jsonMarc Abonce Seguin
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-19 22:15:32 +0000