summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2021-02-07Merge pull request #2539 from OliveiraHermogenes/recoll/paged_jsonNoémi Ványi
[feat] recoll: add support for paging
2021-02-07[feat] recoll: paged json supportHermógenes Oliveira
2021-02-05Merge pull request #2531 from MarcAbonce/fix-browser-localeAlexandre Flament
[fix] Get correct locale with country from browser
2021-02-04[fix] get correct locale with country from browserMarc Abonce Seguin
Some of our interface locales include uppercase country codes, which are separated by `_` instead of the more common `-`. Also, a browser's `Accept-Language` header could be in lowercase. This commit attempts to normalize those cases so a browser's language+country codes can better match with our locales. This solution assumes that our UI locales have nothing more than language and optionally country. If we ever add a script specific locale like `zh-Hant-TW` this would have to change to accomodate that, but the idea would be pretty much the same as this fix.
2021-02-04Merge pull request #2528 from dalf/mod-ci-gh-pagesAlexandre Flament
[mod] CI: minor changes
2021-02-04Merge pull request #2533 from mrwormo/ccengineNoémi Ványi
[Engine] Add Creative Commons search engine
2021-02-04Add Creative Commons search enginemrwormo
2021-02-02Merge pull request #2530 from return42/fix-user-hbNoémi Ványi
[fix] make books/user.pdf
2021-02-02[fix] make books/user.pdfMarkus Heiser
Error: Configuration error: There is a programmable error in your configuration file: ... NameError: name 'DOCS_URL' is not defined make: *** [utils/makefile.sphinx:156: books/user.latex] Fehler 2 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-02-02[mod] CI: minor changesAlexandre Flament
* utils/makefile.python: travis-gh-pages renamed ci-gh-pages
2021-02-01Merge pull request #2500 from dalf/github-action-dataAlexandre Flament
[enh] every Sunday, call utils/fetch_*.py scripts and create a PR automatically
2021-02-01Merge pull request #2499 from dalf/remove-language-support-variableAlexandre Flament
[mod] dynamically set language_support variable
2021-02-01[mod] dynamically set language_support variableAlexandre Flament
The language_support variable is set to True by default, and set to False in only 5 engines. Except the documentation and the /config URL, this variable is not used. This commit remove the variable definition in the engines, and set value according to supported_languages length: False when the length is 0, True otherwise. Close #2485
2021-02-01Merge pull request #2514 from return42/fix-gh-pagesAlexandre Flament
[fix] Makefile target gh-pages & flatten history of branch gh.pages
2021-02-01Merge pull request #2517 from dalf/debug-ciAlexandre Flament
Update pyenv pyenvinstall Make targets
2021-02-01[hardening] github workflows - corrupted cacheMarkus Heiser
aka: ensure that 'make test' works as expected The cache contains a copy './local' which is - under some circumstance - corrupted. It is not possible to clear the cache [1] (see the top of the page). Ensure that 'make test' works as expected [2] even if - the python interpreter is missing - the virtualenv exists but pyyaml is missing To hardening when the workflow cache fails, this patch adds the new target 'travis.test' into the workflow. This target probes to import a python module 'yaml'. If this fails the virtualenv will be completely new build. [1] https://github.com/actions/cache/issues/2#issuecomment-673493515 [2] https://github.com/searx/searx/pull/2517#discussion_r567240235 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-02-01[fix] re-add 'pip-exe' target - partial revert 9b48ae47Markus Heiser
Target pip-exe is a prerequisite of the targets: - pyinstall - pyuninstall and was accidentally deleted in commit 9b48ae47. HINT: do not confuse pyinstall with penvinstall pyinstall & pyuninstall Installing into user's HOME using pip from OS, therefore the message is needed. pyenvinstall & pyenvuninstall Installing into virtualenv (./local) using pip which is provided by prerequisite 'pyenv' in the virtualenv. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-02-01[mod] more robust make pyenv / make pyenvinstallAlexandre Flament
"make pyenv" ensures that ./local/py3/bin/python is an executable
2021-02-01Merge pull request #2494 from return42/rm-fabfileAlexandre Flament
[fix] remove Fabric file
2021-01-29[fix] Makefile target gh-pages & flatten history of branch gh.pagesMarkus Heiser
1. This patch fixes error: rm -rf gh-pages/ make V=1 gh-pages make[1]: Leaving directory '/800GBPCIex4/share/searx' [ -d "gh-pages/.git" ] || git clone gh-pages fatal: repository 'gh-pages' does not exist 2. The gh-page build has been moved to ./build/gh-pages this also affects 'travis-gh-pages' 3. The gh-pages commit messages now includes a ref to the repository and commit 4. Since a gh-pages history has only the drawback that the reposetory grows fast, this patch also flattens the history: cd build/gh-pages/; git log --oneline bash: cd: build/gh-pages/: Datei oder Verzeichnis nicht gefunden 026126be (HEAD -> gh-pages, origin/gh-pages) make gh-pages: from https://github.com/return42/searx.git@71d66979c2935312e0aed7fc7c3cf6199fbe88a2 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-28Merge pull request #2482 from return42/fix-google-videoAlexandre Flament
[fix] revise of the google-Video engine
2021-01-28[fix] google: avoid unnecessary SearxEngineXPathException errorsMarkus Heiser
Avoid SearxEngineXPathException errors when parsing non valid results:: .//div[@class="yuRUbf"]//a/@href index 0 not found Traceback (most recent call last): File "./searx/engines/google.py", line 274, in response url = eval_xpath_getindex(result, href_xpath, 0) File "./searx/searx/utils.py", line 608, in eval_xpath_getindex raise SearxEngineXPathException(xpath_spec, 'index ' + str(index) + ' not found') searx.exceptions.SearxEngineXPathException: .//div[@class="yuRUbf"]//a/@href index 0 not found Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-28[mod] checker: add some additional testsMarkus Heiser
BTW: fix indentation by 2 spaces The additional tests has been commented out in the google engines to not release any CAPTCHA issues. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-28[fix] normalize the language & region aspects of all google enginesMarkus Heiser
BTW: make the engines ready for search.checker: - replace eval_xpath by eval_xpath_getindex and eval_xpath_list - google_images: remove outer try/except block Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-27Merge pull request #2479 from Tobi823/masterAlexandre Flament
Document workaround for using 2 languages simultaneously #1508
2021-01-27Merge pull request #2509 from return42/fix-morty-keyAlexandre Flament
[doc] improve admin-docs about result proxy (morty) configuration
2021-01-27[doc] improve admin-docs about result proxy (morty) configurationMarkus Heiser
[1] https://github.com/searx/searx/pull/1872#issuecomment-768107138 Suggested-by @dalf [1] Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-26[mod] add Makfile targets for search.checker.<engine_name>Markus Heiser
To check all engines: make search.checker To check a engine 'google news' replace space by underline: make search.checker.google_news To see HTTP requests and more use SEARX_DEBUG: make SEARX_DEBUG=1 search.checker.google_news To filter out HTTP redirects: make SEARX_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]" ... Engine google news Checking https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0 https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None -- https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0 https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None -- Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-24[mod] utils/fetch_languages.py: write files at the right locationAlexandre Flament
2021-01-24[enh] every monday, call utils/fetch_*.py scripts and create a PR automaticallyAlexandre Flament
2021-01-24[data] make engines.languagesMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-24[fix] google-videos: parse values for 'length' & 'author'Markus Heiser
The 'video.html' template from the 'oscar' design supports replacement for *author* and *length*. Google-videos does not have an author, alternatively the publisher info from is used for the *author*. Hint: these replacements are not supported by the 'simple' design. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-24[fix] revise of the google-Video engineMarkus Heiser
This revise is based on the methods developed in the revise of the google engine (see commit 410c2f9). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-24Merge pull request #2498 from dalf/minor-fix-google-newsAlexandre Flament
[fix] google_news: avoid one HTTP redirect except for the English results
2021-01-24Merge pull request #2497 from return42/fix-test.shAlexandre Flament
[fix] lxc.sh - SC2034: ubu2010_boilerplate appears unused.
2021-01-24[fix] google_news: avoid one HTTP redirect except for the English resultsAlexandre Flament
also add params['soft_max_redirects'] = 1 to avoid false error reporting in /stats/errors
2021-01-24[fix] lxc.sh - SC2034: ubu2010_boilerplate appears unused.Markus Heiser
$ make test.sh In utils/lxc.sh line 42: ubu2010_boilerplate="$ubu1904_boilerplate" ^-----------------^ SC2034: ubu2010_boilerplate appears unused. Verify use (or export if used externally). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-23Merge pull request #2483 from return42/fix-google-newsAlexandre Flament
[fix] revise of the google-News engine
2021-01-23[fix] google-news: query uses locale without country tagMarkus Heiser
Wthout country-region tag google will redirect to correct the contry tag [1]: SEARX_DEBUG=1 searx-checker -v "google news" ... https://news.google.com:443 "GET /search?q=computer&hl=en... HTTP/1.1" 302 0 https://news.google.com:443 "GET /search?q=computer&hl=en-US&.... HTTP/1.1" 200 None ... [1] https://github.com/searx/searx/pull/2483#issuecomment-765600849 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-22[fix] revise of the google-news engineMarkus Heiser
This revise is based on the methods developed in the revise of the google engine (see commit 410c2f9). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-22[fix] remove Fabric fileMarkus Heiser
The fabfile.py has not been updated since 5 years. I also asked [1] if someone still use Fabric wtihout any response. Lets drop outdated Fabric file. [1] https://github.com/searx/searx/discussions/2400 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-20Merge pull request #2481 from dalf/mod-checkAdam Tauber
Mod check
2021-01-19[mod] checker: disable by defaultAlexandre Flament
2021-01-19[fix] checker: minor fix about language detectionAlexandre Flament
2021-01-19[mod] checker : replace pycld3 by langdetectAlexandre Flament
pycld3 requires the native library cld3 langdetect is a pure python package
2021-01-18Document workaround for using 2 languages simultaneously #1508Tobi823
2021-01-18Merge pull request #2476 from dalf/fix-error-recording-and-checkerAlexandre Flament
Fix error recording and checker
2021-01-18[fix] /stats/checker : remove the timestamp field when the checker is disabledAlexandre Flament
2021-01-18[fix] checker: fix engine statisticsAlexandre Flament
Without this commit, the URL /stats/errors shows percentage above 100% after the checker has run.
2021-01-17[fix] error_recorder: record code and lineno about the engineAlexandre Flament
since the PR #2225 , code and lineno were sometimes meaningless see /stats/errors
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-09 20:12:10 +0000