summaryrefslogtreecommitdiff
path: root/searx/engines
AgeCommit message (Collapse)Author
2022-02-21[fix] genius: add player an avoid exceptional programmingMarkus Heiser
Add player: - The players are just playing 30sec from the title. Some of the player will be blocked because of a cross-origin request and some players will link to apple when you press the play button. Avoid exceptions and (and BTW improve results) - ERROR searx.engines.genius : list index out of range Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-20Merge pull request #910 from tiekoetter/fix-909Markus Heiser
[fix] google images engine: Fix 'scrap_img_by_id' function
2022-02-19[mod] add artwork to mixcloud & soundcloud enginesMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-19[fix] google images engine: Fix 'scrap_img_by_id' functionMarkus Heiser
The 'scrap_img_by_id' function didn't return any longer anything useful. This fix allows the google images engine to present the full source image instead of only the thumbnail. The function scrap_img_by_id() is rpelaced by a fully rewrite to parse image URLs by a regular expression. The new function parse_urls_img_from_js(dom) returns a mapping of data-id to image URL. Closes: https://github.com/searxng/searxng/issues/909 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-19Merge pull request #900 from return42/fix-883Alexandre Flament
[fix] bandcamp: fix itemtype (album|track) and exceptions
2022-02-19[mod] mediathekviewweb engine: add iframe_src and use videos templateMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-18[fix] bandcamp: fix itemtype (album|track) and exceptionsMarkus Heiser
BTW: polish implementation and show tracklist for albums Closes: https://github.com/searxng/searxng/issues/883 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-18[mod] templates: rename field for <iframe> URL to iframe_srcMarkus Heiser
Rename result field data_src to iframe_src Suggested-by: @dalf https://github.com/searxng/searxng/pull/882#issuecomment-1037997402 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-13[mod] result_templates/default.html replace embedded HTML by data_src audio_srcMarkus Heiser
Embedded HTML breaks SearXNG architecture. To modularize, HTML is generated in the templates (oscar & simple) and result parameter 'embedded' is replaced by 'data_src' (and 'audio_src'), an URL for embedded content (<iframe>). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-13[mod] result_templates/videos.html: replace embedded HTML by data_srcMarkus Heiser
Embedded HTML breaks SearXNG architecture. To modularize, HTML is generated in the templates (oscar & simple) and result parameter 'embedded' is replaced by 'data_src', an URL for embedded content (<iframe>). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-09Update the XPath for fetching the Google resultsÉmilien Devos
2022-02-07[fix] openstreatmap: load thumbnail from uploads.wikimedia.orgMarkus Heiser
Openstreatmap images are now loaded from uploads.wikimedia.org instead of commons.wikimedia.org to prevent redirects. With `image_proxy` enabled images from commons.wikimedia.org cant be loaded since they are redirected. We already discussed this issue [875] and @tiekoetter fixed this issue in PR [878]. Related-to: - [875] https://github.com/searxng/searxng/issues/875 - [878] https://github.com/searxng/searxng/pull/878 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-07[pylint] searx/engines/wikidata.py (no functional change)Markus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-07[fix] wikidata: URL decoding and file extension handlingLéon Tiekötter
Add '.png' to the second img_src_name if it has the extension '.svg'. Use urllib.parse.unquote for URL decoding.
2022-02-06[fix] wikidata engine: select image with higher (not lower) priorityMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-06wikidata: load thumbnail instead of full imageLéon Tiekötter
2022-02-06[fix] wikidata info box imagesLéon Tiekötter
Wikidata info box images are now loaded from uploads.wikimedia.org instead of commons.wikimedia.org to prevent redirects Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04[pylint] engines/invidious.pyMarkus Heiser
Fix remarks from pylint and remove usless comments Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04[fix] invidious engine: store random base_url in paramMarkus Heiser
Two different threads ( = two different user queries) can call the request function in a row and then the response function. The namespace will be same since this is the same engine. To keep exactly the same value ``base_url`` must be stored in params and then retrieve using ``resp.search_params["base_url"]``. Suggested-by: @dalf https://github.com/searxng/searxng/pull/862#discussion_r799324861 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04[fix] solidtorrents engine: store random bas_url in paramMarkus Heiser
Two different threads ( = two different user queries) can call the request function in a row and then the response function. The namespace will be same since this is the same engine. To keep exactly the same value ``base_url`` must be stored in params and then retrieve using ``resp.search_params["base_url"]``. Suggested-by: @dalf https://github.com/searxng/searxng/pull/862#discussion_r799324861 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04[mod] solidtorrents engine: add metadata & torrentfileMarkus Heiser
BTW: define min_len in eval_xpath_list of 'stats' list Suggested-by: @dalf https://github.com/searxng/searxng/pull/862#pullrequestreview-872910744 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04[fix] solidtorrents engine: use get_torrent_size from searx.utilsMarkus Heiser
Suggested-by: @dalf https://github.com/searxng/searxng/pull/862#pullrequestreview-872858489 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-04[fix] solidtorrents engine: JSON API no longer existsMarkus Heiser
The API endpoint, we where using does not exist anymore. This patch is a rewrite that parses the HTML page. Related: https://github.com/paulgoio/searxng/issues/17 Closes: https://github.com/searxng/searxng/issues/858 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-01[pylint] engines/currency_convert.pyMarkus Heiser
Fix remarks from pylint Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-31[fix] engines description - currency_convert.pyMarkus Heiser
Currency engine has DuckDuckGo metadata In the engine selector of the preferences window, the currency search engine has the same metadata and wikidata url as duckduckgo, I'd assume there should be a difference of some sort there clarifying what source the currency uses or, if it's a duckduckgo service, at least clarifying that it's a currency service by duck duck go. Closes: https://github.com/searxng/searxng/issues/787 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-31Merge pull request #475 from return42/tineyeMarkus Heiser
[enh] engine - add Tineye reverse image search
2022-01-30[mod] tineye engine: minor changesAlexandre Flament
* remove "disable: false" in settings.yml * use the json() method from httpx.Response (faster character encoding detection)
2022-01-30[fix] 1x engineLéon Tiekötter
1x changed the XML result layout.
2022-01-30[mod] tineye engine: set engine_type to 'online_url_search'Markus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-28[fix] ina engineAlexandre Flament
based on https://gitlab.e.foundation/e/cloud/my-spot/-/commit/a45408e8e2ced90d3f268e09ef01808234072c8b
2022-01-28[mod] tineye - add some documentationMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-28[enh] engine - add Tineye reverse image searchAllen
Other optional parameter .. `&sort=crawl_date` can be appended to search_string to sort results by date. `&domain=example.org` can be implemented to search_string to get results from just one domain. Public instances could get relatively fast timed-out for 3600s. -- Merged from @allendema's commit [1] and slightly modfied / see [2]. Related-to: [1] https://github.com/allendema/searx/commit/455b2b4460cd830ac1f7e62e824040e2fe648de9 Related-to: [2] https://github.com/searx/searx/pull/3040 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-27Allow 'using_tor_proxy' to be set for each engine individuallyLéon Tiekötter
Check 'using_tor_proxy' for each engine individually instead of checking globally [fix] searx.network: update _rdns test to the last httpx version Co-authored-by: Alexandre Flament <alex@al-f.net>
2022-01-18[fix] googel engine - "some results are invalids: invalid content"Markus Heiser
Fix google issues listet in the `/stats?engine=google` and message:: some results are invalids: invalid content The log is:: DEBUG searx : result: invalid content: {'url': 'https://de.wikipedia.org/wiki/Foo', 'title': 'Foo - Wikipedia', 'content': None, 'engine': 'google'} WARNING searx.engines.google : ErrorContext('searx/search/processors/abstract.py', 111, 'result_container.extend(self.engine_name, search_results)', None, 'some results are invalids: invalid content', ()) True Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-18[fix] google engine: remove adds and fix mobile_ui selectorMarkus Heiser
1. Fix issue reported in comment [1] 2. Fix XPath selector for the response of google's mobile UI, reported in comment [2] [1] https://github.com/searxng/searxng/pull/777#issuecomment-1015121322 [2] https://github.com/searxng/searxng/pull/777#issuecomment-1015236238 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-17Update XPath for Google engineÉmilien Devos
2022-01-16Merge pull request #695 from return42/fix-spAlexandre Flament
[fix] startpage engine / modified API
2022-01-15[fix] startpage: workaround to use the startpage networkAlexandre Flament
workaround for the issue #762
2022-01-11[mod] engine mysql_server: make port configurableMarkus Heiser
Cherry piked from https://github.com/searx/searx/commit/82ac634070 Suggested-by: https://github.com/searx/searx/issues/3117 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10[mod] starpage engine: add comment about Startpage's FFox add-onMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10[fix] startpage engine: fetch CAPTCHA & issues related to PR-695Markus Heiser
In case of CAPTCHA raise a SearxEngineCaptchaException and suspend for 7 days. When get_sc_code() fails raise a SearxEngineResponseException and suspend for 7 days. [1] https://github.com/searxng/searxng/pull/695 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10[fix] Get an actual `sc` argument from startpage's home page.Markus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10[pylint] Startpage engineMarkus Heiser
Fix remarks from pylint Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10[fix] startpage engine - avoid captchaMarkus Heiser
Startpage has introduced new anti-scraping measures that make SearXNG instances run into captchas: 1. some arguments has been removed and a new `sc` has been added. 2. search path changed from `do/search` to `sp/search` 3. POST request is no longer needed Closes: https://github.com/searxng/searxng/issues/692 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10[fix] add default for "about" engine propertyMartin Fischer
Fixes #732.
2022-01-07[fix] ccengine engine - avoid unwanted redirectsMarkus Heiser
api.openverse.engineering is a little picky and wants to have a trailing slash in the path: /v1/images? -->/ v1/images/? otherwise it redirects, here is the debug log: DEBUG searx.network.openverse : HTTP Request: GET https://api.openverse.engineering/v1/images?&page=1&page_size=20&format=json&q=foo "HTTP/2 301 Moved Permanently" (text/html; charset=utf-8) DEBUG searx.network.openverse : HTTP Request: GET https://api.openverse.engineering/v1/images/?&page=1&page_size=20&format=json&q=foo "HTTP/2 200 OK" (application/json) WARNING searx.engines.openverse : ErrorContext('searx/search/processors/online.py', 105, 'count_error(', None, '1 redirects, maximum: 0', ('200', 'OK', 'api.openverse.engineering')) True Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07[fix] Rename ccengine engine to openverseLéon Tiekötter
The CC engine was merged with WordPress and renamed to Openverse Source: https://wordpress.org/news/2021/05/welcome-to-openverse/
2022-01-07[fix] ccengine engineLéon Tiekötter
Change domain to api.openverse.engineering
2022-01-07Merge pull request #709 from return42/drop-etoolsMarkus Heiser
[fix] drop etools engine module
2022-01-07[fix] drop etools engine moduleMarkus Heiser
The implementation of the etools engine is poor. No date-range support, no language support and it is broken by a CAPTCHA. etools is a metasearch engine, the major search engines it supports (google, bing, wikipedia, Yahoo) are already available in SeaarXNG. While etools does support several engines we currently don't support directly, support for them should be added directly to SearXNG if there is demand. In practice: in SearXNG the worse etools results will be mixed with good results from other engines we have (as long as there is no captcha). At best case, what we win with etools is in e.g. results from de.ask.com in a query from a german request .. in all other cases worse results are bubble up in SearXNG's result list. [1] https://github.com/searxng/searxng/issues/696#issuecomment-1005855499 Closes: https://github.com/searxng/searxng/issues/696 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-22 10:53:05 +0000