summaryrefslogtreecommitdiff
path: root/searx/engines
AgeCommit message (Collapse)Author
2022-08-01[mod] add 'Accept-Language' HTTP header to online processoresMarkus Heiser
Most engines that support languages (and regions) use the Accept-Language from the WEB browser to build a response that fits to the language (and region). - add new engine option: send_accept_language_header Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-30[fix] pyright repported errorsAlexandre Flament
The errors make pyright usage useless since a new error won't be seen [1]. [1] https://github.com/searxng/searxng/pull/1569 ``` searx/compat.py:11:27 - error: Expression of type "Type[cached_property[_T@cached_property]]" cannot be assigned to declared type "Type[cached_property]" "Type[cached_property[_T@cached_property]]" is incompatible with "Type[cached_property]" Type "Type[cached_property[_T@cached_property]]" cannot be assigned to type "Type[cached_property]" (reportGeneralTypeIssues) searx/utils.py:69:36 - error: Expression of type "None" cannot be assigned to parameter of type "str" Type "None" cannot be assigned to type "str" (reportGeneralTypeIssues) searx/utils.py:573:85 - error: Expression of type "None" cannot be assigned to parameter of type "int" Type "None" cannot be assigned to type "int" (reportGeneralTypeIssues) searx/webapp.py:1306:22 - error: Argument of type "str" cannot be assigned to parameter "__a" of type "BytesPath" in function "join" Type "str" cannot be assigned to type "BytesPath" "str" is incompatible with "bytes" "str" is incompatible with protocol "PathLike[bytes]" "__fspath__" is not present (reportGeneralTypeIssues) searx/webapp.py:1306:68 - error: Argument of type "Literal['themes']" cannot be assigned to parameter "paths" of type "BytesPath" in function "join" Type "Literal['themes']" cannot be assigned to type "BytesPath" "Literal['themes']" is incompatible with "bytes" "Literal['themes']" is incompatible with protocol "PathLike[bytes]" "__fspath__" is not present (reportGeneralTypeIssues) searx/webapp.py:1306:78 - error: Argument of type "str | Any | None" cannot be assigned to parameter "paths" of type "BytesPath" in function "join" Type "str | Any | None" cannot be assigned to type "BytesPath" Type "str" cannot be assigned to type "BytesPath" "str" is incompatible with "bytes" "str" is incompatible with protocol "PathLike[bytes]" "__fspath__" is not present (reportGeneralTypeIssues) searx/webapp.py:1306:85 - error: Argument of type "Literal['img']" cannot be assigned to parameter "paths" of type "BytesPath" in function "join" Type "Literal['img']" cannot be assigned to type "BytesPath" "Literal['img']" is incompatible with "bytes" "Literal['img']" is incompatible with protocol "PathLike[bytes]" "__fspath__" is not present (reportGeneralTypeIssues) searx/engines/mongodb.py:8:6 - warning: Import "pymongo" could not be resolved (reportMissingImports) searx/engines/mysql_server.py:9:8 - warning: Import "mysql.connector" could not be resolved (reportMissingImports) searx/engines/postgresql.py:9:8 - warning: Import "psycopg2" could not be resolved from source (reportMissingModuleSource) searx/engines/xpath.py:187:28 - warning: "categories" is not defined (reportUndefinedVariable) searx/search/__init__.py:184:82 - warning: "flask" is not defined (reportUndefinedVariable) searx/search/checker/background.py:19:26 - error: Type of "schedule" is partially unknown Type of "schedule" is "(delay: Any, func: Any, *args: Any) -> Literal[True]" (reportUnknownVariableType) searx/shared/__init__.py:8:12 - warning: Import "uwsgi" could not be resolved (reportMissingImports) searx/shared/shared_uwsgi.py:5:8 - warning: Import "uwsgi" could not be resolved (reportMissingImports) ```
2022-07-26Revert "Quick fix for google engine for EU countries"Markus Heiser
This reverts commit 747cf1a246df587aeb3b6b175c315ef0b9612dc4.
2022-07-26[fix] google engine: results XPathLéon Tiekötter
2022-07-25Quick fix for google engine for EU countriesÉmilien Devos
This revert part of the commit of https://github.com/searxng/searxng/commit/5fb2071cb2248c0f0ada7affb0c47f841ddbf102
2022-07-25[fix] demo_online.py - fixed typoMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-25[fix] google & youtube - set EU consent cookieEmilien Devos
This change the previous bypass method for Google consent using ``ucbcb=1`` (6face215b8) to accept the consent using ``CONSENT=YES+``. The youtube_noapi and google have a similar API, at least for the consent[1]. Get CONSENT cookie from google reguest:: curl -i "https://www.google.com/search?q=time&tbm=isch" \ -A "Mozilla/5.0 (X11; Linux i686; rv:102.0) Gecko/20100101 Firefox/102.0" \ | grep -i consent ... location: https://consent.google.com/m?continue=https://www.google.com/search?q%3Dtime%26tbm%3Disch&gl=DE&m=0&pc=irp&uxe=eomtm&hl=en-US&src=1 set-cookie: CONSENT=PENDING+936; expires=Wed, 24-Jul-2024 11:26:20 GMT; path=/; domain=.google.com; Secure ... PENDING & YES [2]: Google change the way for consent about YouTube cookies agreement in EU countries. Instead of showing a popup in the website, YouTube redirects the user to a new webpage at consent.youtube.com domain ... Fix for this is to put a cookie CONSENT with YES+ value for every YouTube request [1] https://github.com/iv-org/invidious/pull/2207 [2] https://github.com/TeamNewPipe/NewPipeExtractor/issues/592 Closes: https://github.com/searxng/searxng/issues/1432
2022-07-24[fix] sjp engine - convert enginename to a latin1 compliance nameMarkus Heiser
The engine name is not only a *name* its also a identifier that is used in logs, HTTP headers and more. Unicode characters in the name of an engine could cause various issues. Closes: https://github.com/searxng/searxng/issues/1544 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-24[fix] emojipedia - update XPath to be relativejames-still
2022-07-23[fix] engine tineye: handle 422 response of not supported img formatMarkus Heiser
Closes: https://github.com/searxng/searxng/issues/1449 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-15Merge pull request #1475 from return42/EmojipediaMarkus Heiser
[mod] Add engine for Emojipedia
2022-07-15[mod] Add engine for EmojipediaJay
Emojipedia is an emoji reference website which documents the meaning and common usage of emoji characters in the Unicode Standard. It is owned by Zedge since 2021. Emojipedia is a voting member of The Unicode Consortium.[1] Cherry picked from @james-still [2[3] and slightly modified to fit SearXNG's quality gates. [1] https://en.wikipedia.org/wiki/Emojipedia [2] https://github.com/james-still/searx/commit/2fc01eb20f8de5f9cac492dcdfb817a6f0636580 [3] https://github.com/searx/searx/pull/3278
2022-07-10Merge pull request #1219 from dalf/follow_bing_redirectAlexandre Flament
bing.py: remove redirection links
2022-07-09bypass google consent with ucbcb=1Emilien Devos
2022-07-08bing.py: resolve bing.com/ck/a redirectionsAlexandre Flament
add a new function searx.network.multi_requests to send multiple HTTP requests at once
2022-07-05[fix] engine z-zlibrary https URLMarkus Heiser
before this patch: DEBUG searx.engines.z-library : using base_url: https:https://de1lib.org with this patch URL is fixed to: DEBUG searx.engines.z-library : using base_url: https://de1lib.org Closes: https://github.com/searxng/searxng/issues/1435 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-04[mod] Adds Lingva translate engineta
Add the lingva engine (which grabs data from google translate). Results from Lingva are added to the infobox results.
2022-07-02[fix] engines/openstreetmap.py typo: user_langage --> user_languageMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-02openstreetmap engine: return the localized named.Alexandre Flament
For example: display "Tokyo" instead of "東京都" when the language is English.
2022-06-18[fix] google play apps engine: implement engines/google_play_apps.pyta
2022-06-18Fix: use sys.modules.copy() to avoid RuntimeErrorAlexandre Flament
use sys.modules.copy() to avoid "RuntimeError: dictionary changed size during iteration" see https://github.com/python/cpython/issues/89516 and https://docs.python.org/3.10/library/sys.html#sys.modules close https://github.com/searxng/searxng/issues/1342
2022-06-12Merge pull request #1308 from allendema/add-yep-com-jsonAlexandre Flament
[enh] Add yep.com via json_engine
2022-06-11[enh] Initial no paging support for Yep.comAllen
Upstream example query: https://yep.com/web?q=test https://yep.com/about
2022-06-11Wikidata engine: ignore dummy entitiesAlexandre Flament
Close #641
2022-06-11Wikidata engine: minor change of the SPARQL requestAlexandre Flament
The engine can be slow especially when the query won't return any answer. See https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual/MWAPI#Find_articles_in_Wikipedia_speaking_about_cheese_and_see_which_Wikibase_items_they_correspond_to Related to #1290
2022-06-03[fix] prepare for pylint 2.14.0Markus Heiser
Remove issue reported by Pylint 2.14.0: - no-self-use: has been moved to optional extension [1] - The refactoring checker now also raises 'consider-using-generator' messages for max(), min() and sum(). [2] .pylintrc: - <option name>-hint has been removed since long, Pylint 2.14.0 raises an error on invalid options - bad-continuation and bad-whitespace have been removed [3] [1] https://pylint.pycqa.org/en/latest/whatsnew/2/2.14/summary.html#removed-checkers [2] https://pylint.pycqa.org/en/latest/whatsnew/2/2.14/full.html#what-s-new-in-pylint-2-14-0 [2] https://pylint.pycqa.org/en/latest/whatsnew/2/2.6/summary.html#summary-release-highlights Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-06-02[enh] Initial Petalsearch Images supportAllen
Upstream example query: https://petalsearch.com/search?query=test&channel=image&ps=50&pn=1&region=de-de&ss_mode=off&ss_type=normal Depending on locale it will internally use some/all results from other engines. See: https://seirdy.one/posts/2021/03/10/search-engines-with-own-indexes/#general-indexing-search-engines
2022-05-10Reflect the real world parameter from settings.ymlÉmilien Devos
2022-05-07[format.python] based on bugfix in 9ed626130Markus Heiser
2022-04-22[fix] FutureWarning from lxmlcapric98
Just in case if content is None, the original code will skip extract_text(), and just append the None value to 'content'. So just add allow_none=True, and this will return None without raising a ValueError in extract_text().
2022-04-17Merge pull request #1101 from allendema/pass-cookies-from-settingsAlexandre Flament
[enh] Allow passing headers/cookies from settings.yml
2022-04-17[fix[ Update only cookies/headersAllen
2022-04-17[lint] Remove whitespaceAllen
From GH GUI
2022-04-16[mod] Pass desired ebay domain in settings Allen
https://www.ebay.de https://www.ebay.com htttps://www.ebay.es etc
2022-04-16[enh] Allow passing headers/cookies from settings.ymlAllen
Example: - engine: xpath - search_url: example.org - headers: {'example_header': 'example_header'} - cookies: {'safesearch': 'off'}
2022-04-16Merge pull request #1071 from return42/fix-lang-dailymotionAlexandre Flament
[fix] dailymotion engine: filter by language & country
2022-04-16dailymotion: send valid value for the language parameterAlexandre Flament
2022-04-16[fix] dailymotion engine: filter by language & countryMarkus Heiser
- fix the issue of fetching more the 7000 *languages* - improve the request function and filter by language & country - implement time_range_support & safesearch - add more fields to the response from dailymotion (allow_embed, length) - better clean up of HTML tags in the 'content' field. This is more or less a complete rework based on the '/videos' API from [1]. This patch cleans up the language list in SearXNG that has been polluted by the ISO-639-3 2 and 3 letter codes from dailymotion languages which have never been used. [1] https://developers.dailymotion.com/tools/ Closes: https://github.com/searxng/searxng/issues/1065 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-04-12change "Wolfram|Alpha" to "Wolfram Alpha" in search resultsJabster28
2022-04-09Merge pull request #1030 from austinhuang0131/masterAlexandre Flament
(feat) add jisho.org
2022-04-09jisho : code refactoringAlexandre Flament
2022-04-01(fix) satisfy the linter, and btw reduce timeoutAustin Huang
2022-04-01update jisho.py according to suggestionsAustin Huang
2022-03-31bing engine: _fetch_supported_languages: don't use the language code as a ↵Alexandre FLAMENT
country ref #1029
2022-03-31(feat) add jisho.orgAustin Huang
Closes #1016
2022-03-19fix startpage: update XPath in _fetch_supported_languagesAlexandre Flament
2022-03-07[fix] engine mediathekviewweb: replace http links by httpsMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-03-05[fix] engine: Semantic Scholar (Science) // rework & fixMarkus Heiser
Closes: https://github.com/searxng/searxng/issues/939 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-21Merge pull request #913 from return42/add-artworkMarkus Heiser
[mod] add artwork to mixcloud & soundcloud engines
2022-02-21[mod] bandcamp & genius: in result set img_src instead thumbnailMarkus Heiser
Suggested-by: @dalf https://github.com/searxng/searxng/pull/900#issuecomment-1046009057 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-23 02:01:05 +0000