summaryrefslogtreecommitdiff
path: root/searx/plugins
AgeCommit message (Collapse)Author
2023-04-30[mod] limiter: block requests from PetalBotMarkus Heiser
Block requests from PetalBlock. Normally robots.txt is enough to stop PetalBlock from making requests [1]. However, if SearXNG is offered below a path (example.org/search), then the robots.txt is not available in the root paths of the domain / subdomain. [1] https://webmaster.petalsearch.com/site/petalbot Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-04-13[mod] limiter: block unmaintained Farside instancesMarkus Heiser
Since [bb3a01f8] has been merged to the Farside project, Farside instances do no longer need to send requests to SearXNG instances [1]. There are some old unmaintained Farside instances on the web that continue to query SearXNG instances --> we can safely block their requests. [1] https://github.com/benbusby/farside/issues/95 [bb3a01f8] https://github.com/benbusby/farside/commit/bb3a01f8 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-04-03[fix] limiter: never block a /healthz requestMarkus Heiser
Related: https://github.com/searxng/searxng/issues/2310#issuecomment-1494417531 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-04-01[mod] limiter: minor improvementsMarkus Heiser
- requests without HTTP header 'Connection' or missing 'User-Agent' will be blocked by the limiter - re_bot is related to 'User-Agent' and has been renamed to block_user_agent Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-04-01[mod] plugin limiter: improve the log messagesMarkus Heiser
In debug mode more detailed logging is needed to evaluate if an access should have been blocked by the limiter. BTW: remove duplicate code checking bot signature ``re_bot.match(user_agent)`` Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-17Add "Auto-detected" as a language.Alexandre Flament
When the user choose "Auto-detected", the choice remains on the following queries. The detected language is displayed. For example "Auto-detected (en)": * the next query language is going to be auto detected * for the current query, the detected language is English. This replace the autodetect_search_language plugin.
2023-02-17[fix] typo in searx/plugins/tor_check.pyMarkus Heiser
Related: https://github.com/searxng/searxng/pull/2189 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-14[fix] spelling: Tor, SearXNGAllan Nordhøy
2022-12-16Merge branch 'master' into fasttextArtikusHG
2022-12-16Replace langdetect with fasttextArtikusHG
2022-12-14Merge pull request #1954 from dalf/fix.redis.init.2Alexandre Flament
[fix] follow up of PR-1856
2022-12-11[mod] improve 'Autodetect search language' pluginMarkus Heiser
- Add documentation to the plugin - Harmonize FastText language model with SearXNG's language model Reosurces:: import fasttext # --> +10 MB fasttext.load_model(str(data_dir / 'lid.176.ftz')) # --> +4MB Suggested-by: @dalf - To speed up and simplify the deployment use fasttext-wheel instead of fasttext - Building numpy on the Alpine Linux of docker-images takes ages --> install py3-numpy from Alpines package manager (apk) - Alpine Linux on docker-images (musl libc) do not support fasttext-wheel (gnu libc) --> patch Dockerfile and build from fastetxt: sed -i s/fasttext-wheel/fasttext/ requirements.txt Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-12-10[mod] new plugin: Autodetect search languageArtikusHG
2022-12-10move searx.shared.redisdb to searx.redisdbAlexandre Flament
2022-11-05The checker requires RedisAlexandre Flament
Remove the abstraction in searx.shared.SharedDict. Implement a basic and dedicated scheduler for the checker using a Redis script.
2022-09-27[fix] typos / reported by @kianmeng in searx PR-3366Markus Heiser
[PR-3366] https://github.com/searx/searx/pull/3366 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-09-23oa_doi_rewrite: add the doi to the result when it is found.Alexandre FLAMENT
Currentty, when oa_doi_rewrite find a DOI in the result URL, it replace the URL. In this commit, the plugin adds the key "doi" to the result, so the paper.html can show it.
2022-08-25[mod] limiter plugin: Accept-Encoding handlingLéon Tiekötter
Only raise "suspicious Accept-Encoding" when both "gzip" and "deflate" are missing from Accept-Encoding. Prevent Browsers which only implement one compression solution from being blocked by the limiter plugin. Example Browser which is currently blocked: Lynx Browser (https://lynx.invisible-island.net)
2022-07-19[mod] add tor_check plugin - convenient tor checking trough searxngSolirs
2022-07-05fix: return body on limiter block so there is not just a blank pagemrpaulblack
2022-06-06Merge pull request #1047 from return42/redis-libAlexandre Flament
Add a redis library to generalize DB functions we need in SearXNG.
2022-06-03[fix] prepare for pylint 2.14.0Markus Heiser
Remove issue reported by Pylint 2.14.0: - no-self-use: has been moved to optional extension [1] - The refactoring checker now also raises 'consider-using-generator' messages for max(), min() and sum(). [2] .pylintrc: - <option name>-hint has been removed since long, Pylint 2.14.0 raises an error on invalid options - bad-continuation and bad-whitespace have been removed [3] [1] https://pylint.pycqa.org/en/latest/whatsnew/2/2.14/summary.html#removed-checkers [2] https://pylint.pycqa.org/en/latest/whatsnew/2/2.14/full.html#what-s-new-in-pylint-2-14-0 [2] https://pylint.pycqa.org/en/latest/whatsnew/2/2.6/summary.html#summary-release-highlights Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-05-30[mod] plugins/limiter.py - use searx.redislib.incr_sliding_windowMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-05-10Fix typoÉmilien Devos
2022-05-07theme: remove __common__Alexandre Flament
2022-03-25[doc] add some documentation about the limiter plugin (and redis)Markus Heiser
Requested-by: https://github.com/searxng/searxng/discussions/993#discussioncomment-2396914 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-20[mod] infinite_scroll as preferenceAlexandre Flament
* oscar theme: code from searx/plugins/infinite_scroll.py * simple theme: new implementation Co-authored-by: Markus Heiser <markus.heiser@darmarIT.de>
2022-02-18Merge pull request #899 from dalf/limiter_updateAlexandre Flament
[limiter] update
2022-02-18[mod] templates: rename field for <iframe> URL to iframe_srcMarkus Heiser
Rename result field data_src to iframe_src Suggested-by: @dalf https://github.com/searxng/searxng/pull/882#issuecomment-1037997402 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-18[fix] hostname_replace.py: don't stop replace URL in fieldsMarkus Heiser
This is a rewrite of the hostname_replace.py that: - don't stop to replace URL in fields ('data_src', 'audio_src') if there isn't a 'parsed_url', - adds a comment about keep or remove a result from the result list - adds a loop over ['data_src', 'audio_src'] instead of doubling code lines Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-17[limiter] updateAlexandre Flament
2022-02-13[mod] result_templates/default.html replace embedded HTML by data_src audio_srcMarkus Heiser
Embedded HTML breaks SearXNG architecture. To modularize, HTML is generated in the templates (oscar & simple) and result parameter 'embedded' is replaced by 'data_src' (and 'audio_src'), an URL for embedded content (<iframe>). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-13[mod] hostname_replace: replace hostnames in result's data_src paramMarkus Heiser
To test you need to redirect embeded videos (e.g.) from youtube to a invidios instance. Search for videos using engine `!youtube lebowski`. The result URLs and the embeded videos should link to the invidios instance. Here is an example of such a `hostname_replace` configuration:: hostname_replace: # youtube --> Invidious '(.*\.)?youtube-nocookie\.com': 'invidio.xamh.de' '(.*\.)?youtube\.com$': 'invidio.xamh.de' '(.*\.)?invidious\.snopyta\.org$': 'invidio.xamh.de' '(.*\.)?vid\.puffyan\.us': 'invidio.xamh.de' '(.*\.)?invidious\.kavin\.rocks$': 'invidio.xamh.de' '(.*\.)?inv\.riverside\.rocks$': 'invidio.xamh.de' Closes: https://github.com/searxng/searxng/issues/873 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-02-12plugin limiter: check only /image_proxy and /searchAlexandre Flament
also adjust the number of req/time
2022-02-02[enh] limiter pluginAlexandre Flament
can replace filtron: * rate limite the number of request per IP and per (IP, User-Agent) * block some bots use Redis data stored in Redis never contains the IP addresses, only HMAC using the secret_key Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-17[typing] add optional attrs to PluginMartin Fischer
2022-01-06[refactor] add type hints & remove Setting._post_initMartin Fischer
Previously the Setting classes used a horrible _post_init hack that prevented proper type checking.
2021-12-27[format.python] initial formatting of the python codeMarkus Heiser
This patch was generated by black [1]:: make format.python [1] https://github.com/psf/black Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-27[format.python] disable py code formatting for some hunks of codeMarkus Heiser
Disable the python code formatting from python-black, where the readability of code suffers by formatting. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-07[mod] searx.plugins.prepare_package_resources() - use generatorsMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-07[fix] searx.plugins.initialize() - don't miss module & module-nameMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-06plugins: refactor initializationAlexandre Flament
add a new function "init" call when the app starts. The function can: * return False to disable the plugin. * modify the Flask app.
2021-09-11[enh] add hostname_replace pluginAlexandre Flament
* backport of https://github.com/searx/searx/pull/2724 * allow to remove result if the replacement is the boolean value false
2021-09-09[mod] ahmia_filter: use on_result instead of post_searchAlexandre Flament
see commit 6c9ae7911e9639bc46cd53af215734b4bdb61ba9
2021-09-07[mod] oa_doi_rewrite plugin: get_doi_resolver: remove args parameterAlexandre Flament
doi_resolvers.get_value('preferences') already contains the value from request.args.get('doi_resolver')
2021-08-14[mod] plugins: minor changeAlexandre Flament
required attributes: display a different message when the attribute has the wrong type
2021-06-17[mod] oscar theme: /preferences : HTML detail order match visual tabsAlexandre Flament
First details about the general tab, then detail about UI tab, etc... No functionnal change
2021-06-17[mod] oscar theme /preferences: reorganize the preferencesAlexandre Flament
close #115
2021-06-01[mod] move all default settings into searx.settings_defaultsAlexandre Flament
2021-04-22[drop] plugin HTTPS rewrite - rule base is to old (update is to big)Markus Heiser
The usefulness of the _HTTPS rewrite_ plugin is questionable: - the 36 rule files have not been updated since 2015 [1] - actual there are 23760 rule files in the https-everywhere repo [2] For the first, we can remove this plugin. For a complete new implementation, it might be good to know that there is a project "https-everywhere : Privacy for Pythons" [3] related: https://github.com/return42/searx-next/issues/8 [1] https://github.com/return42/searx-next/tree/d187a1d/searx/plugins/https_rules [2] https://github.com/EFForg/https-everywhere/tree/master/src/chrome/content/rules [3] https://github.com/jayvdb/https-everywhere-py Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-10 08:58:24 +0000