summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2021-02-10Merge pull request #2556 from dalf/fix-apk-mirrorAlexandre Flament
[fix] fix apk_mirror engine
2021-02-10Merge pull request #2558 from dalf/remove-google-play-musicAlexandre Flament
[upd] remove google_play_music engine
2021-02-09[upd] remove google_play_music engineAlexandre Flament
Google Play Music has been replaced by Youtube music.
2021-02-09[fix) fix apk_mirror engineAlexandre Flament
2021-02-09Merge pull request #2555 from dalf/fix-github-data-updateAlexandre Flament
[fix] fix github action data-update.yml
2021-02-09[fix] fix github action data-update.ymlAlexandre Flament
2021-02-09Merge pull request #2542 from kvch/fix-naver-engineAlexandre Flament
Fix XPATHs in Naver engine
2021-02-09Merge pull request #2554 from MarcAbonce/zh-variants-in-wikipediaAlexandre Flament
Add support for Chinese variants in Wikipedia
2021-02-09Merge pull request #2553 from danielhones/improve-results-highlighting-updatedAlexandre Flament
Ignore double-quotes when highlighting query parts
2021-02-08Updated webutils.highlight_content to ignore double-quotes when highlighting ↵Daniel Hones
query parts
2021-02-08add support for Chinese variants in WikipediaMarc Abonce Seguin
2021-02-07Fix naver engineNoémi Ványi
Closes #2540
2021-02-07Merge pull request #2538 from return42/drop-metagerNoémi Ványi
[drop] metager - xpath engine won't work anymore
2021-02-07[drop] metager - xpath engine won't work anymoreMarkus Heiser
The new version of MetaGer needs to reload the reults (into a iframe) with a unique tag (see HTML response below). Implementing a dedicated metager-engine for searx makes no sense to me. The great days of MetaGer seems to be ended. I remember the good old days this project started in the 90's of the last century. But in the last few years it becomes more and more crap. As the name suggested, MetaGer was made for germans in the first place. They have added a english and spain translation but the i18n is very poor compared to what searx offers. It's a pity, lets drop MetaGer. This is the first response, the id (b82679980656899ba5a17ffd02a56846) is unique for each query: $ curl "https://metager.org/meta/meta.ger3?eingabe=foo&submit-query=&focus=web" <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <link rel="stylesheet" href="/index.css?id=b82679980656899ba5a17ffd02a56846"> <script src="/index.js?id=b82679980656899ba5a17ffd02a56846"></script> <title>foo - MetaGer</title> <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1" /> </head> <body> <iframe id="mg-framed" src="https://metager.org/meta/meta.ger3?eingabe=foo&amp;submit-query=&amp;focus=web&amp;mgv=b82679980656899ba5a17ffd02a56846" autofocus="true" onload="this.contentWindow.focus();"></iframe> </body> Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-02-07Merge pull request #2539 from OliveiraHermogenes/recoll/paged_jsonNoémi Ványi
[feat] recoll: add support for paging
2021-02-07[feat] recoll: paged json supportHermógenes Oliveira
2021-02-05Merge pull request #2531 from MarcAbonce/fix-browser-localeAlexandre Flament
[fix] Get correct locale with country from browser
2021-02-04[fix] get correct locale with country from browserMarc Abonce Seguin
Some of our interface locales include uppercase country codes, which are separated by `_` instead of the more common `-`. Also, a browser's `Accept-Language` header could be in lowercase. This commit attempts to normalize those cases so a browser's language+country codes can better match with our locales. This solution assumes that our UI locales have nothing more than language and optionally country. If we ever add a script specific locale like `zh-Hant-TW` this would have to change to accomodate that, but the idea would be pretty much the same as this fix.
2021-02-04Merge pull request #2528 from dalf/mod-ci-gh-pagesAlexandre Flament
[mod] CI: minor changes
2021-02-04Merge pull request #2533 from mrwormo/ccengineNoémi Ványi
[Engine] Add Creative Commons search engine
2021-02-04Add Creative Commons search enginemrwormo
2021-02-02Merge pull request #2530 from return42/fix-user-hbNoémi Ványi
[fix] make books/user.pdf
2021-02-02[fix] make books/user.pdfMarkus Heiser
Error: Configuration error: There is a programmable error in your configuration file: ... NameError: name 'DOCS_URL' is not defined make: *** [utils/makefile.sphinx:156: books/user.latex] Fehler 2 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-02-02[mod] CI: minor changesAlexandre Flament
* utils/makefile.python: travis-gh-pages renamed ci-gh-pages
2021-02-01Merge pull request #2500 from dalf/github-action-dataAlexandre Flament
[enh] every Sunday, call utils/fetch_*.py scripts and create a PR automatically
2021-02-01Merge pull request #2499 from dalf/remove-language-support-variableAlexandre Flament
[mod] dynamically set language_support variable
2021-02-01[mod] dynamically set language_support variableAlexandre Flament
The language_support variable is set to True by default, and set to False in only 5 engines. Except the documentation and the /config URL, this variable is not used. This commit remove the variable definition in the engines, and set value according to supported_languages length: False when the length is 0, True otherwise. Close #2485
2021-02-01Merge pull request #2514 from return42/fix-gh-pagesAlexandre Flament
[fix] Makefile target gh-pages & flatten history of branch gh.pages
2021-02-01Merge pull request #2517 from dalf/debug-ciAlexandre Flament
Update pyenv pyenvinstall Make targets
2021-02-01[hardening] github workflows - corrupted cacheMarkus Heiser
aka: ensure that 'make test' works as expected The cache contains a copy './local' which is - under some circumstance - corrupted. It is not possible to clear the cache [1] (see the top of the page). Ensure that 'make test' works as expected [2] even if - the python interpreter is missing - the virtualenv exists but pyyaml is missing To hardening when the workflow cache fails, this patch adds the new target 'travis.test' into the workflow. This target probes to import a python module 'yaml'. If this fails the virtualenv will be completely new build. [1] https://github.com/actions/cache/issues/2#issuecomment-673493515 [2] https://github.com/searx/searx/pull/2517#discussion_r567240235 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-02-01[fix] re-add 'pip-exe' target - partial revert 9b48ae47Markus Heiser
Target pip-exe is a prerequisite of the targets: - pyinstall - pyuninstall and was accidentally deleted in commit 9b48ae47. HINT: do not confuse pyinstall with penvinstall pyinstall & pyuninstall Installing into user's HOME using pip from OS, therefore the message is needed. pyenvinstall & pyenvuninstall Installing into virtualenv (./local) using pip which is provided by prerequisite 'pyenv' in the virtualenv. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-02-01[mod] more robust make pyenv / make pyenvinstallAlexandre Flament
"make pyenv" ensures that ./local/py3/bin/python is an executable
2021-02-01Merge pull request #2494 from return42/rm-fabfileAlexandre Flament
[fix] remove Fabric file
2021-01-29[fix] Makefile target gh-pages & flatten history of branch gh.pagesMarkus Heiser
1. This patch fixes error: rm -rf gh-pages/ make V=1 gh-pages make[1]: Leaving directory '/800GBPCIex4/share/searx' [ -d "gh-pages/.git" ] || git clone gh-pages fatal: repository 'gh-pages' does not exist 2. The gh-page build has been moved to ./build/gh-pages this also affects 'travis-gh-pages' 3. The gh-pages commit messages now includes a ref to the repository and commit 4. Since a gh-pages history has only the drawback that the reposetory grows fast, this patch also flattens the history: cd build/gh-pages/; git log --oneline bash: cd: build/gh-pages/: Datei oder Verzeichnis nicht gefunden 026126be (HEAD -> gh-pages, origin/gh-pages) make gh-pages: from https://github.com/return42/searx.git@71d66979c2935312e0aed7fc7c3cf6199fbe88a2 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-28Merge pull request #2482 from return42/fix-google-videoAlexandre Flament
[fix] revise of the google-Video engine
2021-01-28[fix] google: avoid unnecessary SearxEngineXPathException errorsMarkus Heiser
Avoid SearxEngineXPathException errors when parsing non valid results:: .//div[@class="yuRUbf"]//a/@href index 0 not found Traceback (most recent call last): File "./searx/engines/google.py", line 274, in response url = eval_xpath_getindex(result, href_xpath, 0) File "./searx/searx/utils.py", line 608, in eval_xpath_getindex raise SearxEngineXPathException(xpath_spec, 'index ' + str(index) + ' not found') searx.exceptions.SearxEngineXPathException: .//div[@class="yuRUbf"]//a/@href index 0 not found Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-28[mod] checker: add some additional testsMarkus Heiser
BTW: fix indentation by 2 spaces The additional tests has been commented out in the google engines to not release any CAPTCHA issues. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-28[fix] normalize the language & region aspects of all google enginesMarkus Heiser
BTW: make the engines ready for search.checker: - replace eval_xpath by eval_xpath_getindex and eval_xpath_list - google_images: remove outer try/except block Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-27Merge pull request #2479 from Tobi823/masterAlexandre Flament
Document workaround for using 2 languages simultaneously #1508
2021-01-27Merge pull request #2509 from return42/fix-morty-keyAlexandre Flament
[doc] improve admin-docs about result proxy (morty) configuration
2021-01-27[doc] improve admin-docs about result proxy (morty) configurationMarkus Heiser
[1] https://github.com/searx/searx/pull/1872#issuecomment-768107138 Suggested-by @dalf [1] Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-26[mod] add Makfile targets for search.checker.<engine_name>Markus Heiser
To check all engines: make search.checker To check a engine 'google news' replace space by underline: make search.checker.google_news To see HTTP requests and more use SEARX_DEBUG: make SEARX_DEBUG=1 search.checker.google_news To filter out HTTP redirects: make SEARX_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]" ... Engine google news Checking https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0 https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None -- https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0 https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None -- Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-24[mod] utils/fetch_languages.py: write files at the right locationAlexandre Flament
2021-01-24[enh] every monday, call utils/fetch_*.py scripts and create a PR automaticallyAlexandre Flament
2021-01-24[data] make engines.languagesMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-24[fix] google-videos: parse values for 'length' & 'author'Markus Heiser
The 'video.html' template from the 'oscar' design supports replacement for *author* and *length*. Google-videos does not have an author, alternatively the publisher info from is used for the *author*. Hint: these replacements are not supported by the 'simple' design. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-24[fix] revise of the google-Video engineMarkus Heiser
This revise is based on the methods developed in the revise of the google engine (see commit 410c2f9). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-24Merge pull request #2498 from dalf/minor-fix-google-newsAlexandre Flament
[fix] google_news: avoid one HTTP redirect except for the English results
2021-01-24Merge pull request #2497 from return42/fix-test.shAlexandre Flament
[fix] lxc.sh - SC2034: ubu2010_boilerplate appears unused.
2021-01-24[fix] google_news: avoid one HTTP redirect except for the English resultsAlexandre Flament
also add params['soft_max_redirects'] = 1 to avoid false error reporting in /stats/errors