doc: move patches from /doc folder of branch gh-pages to master

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

7 files changed, 1160 insertions, 0 deletions

diff --git a/docs/dev/contribution_guide.rst b/docs/dev/contribution_guide.rst
new file mode 100644
index 000000000..65e587294
--- /dev/null
+++ b/docs/dev/contribution_guide.rst
@@ -0,0 +1,98 @@
+How to contribute
+-----------------
+
+Prime directives: Privacy, Hackability
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Searx has two prime directives, privacy-by-design and hackability. The
+hackability comes in three levels:
+
+-  support of search engines
+-  plugins to alter search behaviour
+-  hacking searx itself
+
+Note the lack of "world domination" among the directives.
+Searx has no intention of wide mass-adoption, rounded
+corners, etc. The prime directive "privacy" deserves a separate
+chapter, as it's quite uncommon unfortunately.
+
+Privacy-by-design
+^^^^^^^^^^^^^^^^^
+
+Searx was born out of the need for a privacy-respecting search tool
+which can be extended easily to maximize both its search and its
+privacy protecting capabilities.
+
+A few widely used features work differently or turned off by default or not implemented
+at all as a consequence of privacy-by-design.
+
+If a feature reduces the privacy preserving aspects of searx, it
+should be switched off by default or should not implemented at all.
+There are plenty of search engines already providing such features.
+If a feature reduces the protection of searx, users must be
+informed about the effect of choosing to enable it. Features
+that protect privacy but differ from the expectations of the
+user should also be explained.
+
+Also, if you think that something works weird with searx,
+it's might be because of the tool you use is designed in a way to interfere with
+the privacy respect. Submitting a bugreport to the vendor of the tool that
+misbehaves might be a good feedback to reconsider the disrespect to
+its customers (e.g. GET vs POST requests in various browsers).
+
+Remember the other prime directive of searx is to be hackable, so if the
+above privacy concerns do not fancy you, simply fork it.
+
+Happy hacking.
+
+Code
+~~~~
+
+In order to submit a patch, please follow the steps below:
+
+- Follow coding conventions.
+
+  - PEP8 standards apply, except the convention of line length
+
+  - Maximum line length is 120 characters
+
+- Check if your code breaks existing tests. If so, update the tests or fix your code.
+
+- If your code can be unit-tested, add unit tests.
+
+- Add yourself to the AUTHORS file.
+
+- Create a pull request.
+
+For more help on getting started with searx development, see :ref:`devquickstart`.
+
+
+Translation
+~~~~~~~~~~~
+
+Translation currently takes place on
+`transifex <https://transifex.com/projects/p/searx>`__.
+
+**Please, do not update translation files in the repo.**
+
+Documentation
+~~~~~~~~~~~~~
+
+The documentation is built using Sphinx. So in order to be able to generate the required
+files, you have to install it on your system. (It can be installed easily using pip.)
+
+1. Checkout the gh-pages branch.
+
+2. Edit the rst file you wish to update. Or create a new rst file and place it under the appropriate folder.
+
+3. Build the documentation using Sphinx.
+
+4. Add the updated and created files of these extension:
+
+   - .rst
+
+   - .html
+
+   - .txt
+
+6. Create a pull request.
diff --git a/docs/dev/engine_overview.rst b/docs/dev/engine_overview.rst
new file mode 100644
index 000000000..a6867b5d0
--- /dev/null
+++ b/docs/dev/engine_overview.rst
@@ -0,0 +1,315 @@
+Engine overview
+===============
+
+
+searx is a `metasearch-engine <https://en.wikipedia.org/wiki/Metasearch_engine>`__,
+so it uses different search engines to provide better results.
+
+Because there is no general search API which could be used for every
+search engine, an adapter has to be built between searx and the
+external search engines. Adapters are stored under the folder
+`searx/engines
+<https://github.com/asciimoo/searx/tree/master/searx/engines>`__.
+
+
+.. contents::
+   :depth: 3
+
+general engine configuration
+----------------------------
+
+It is required to tell searx the type of results the engine provides. The
+arguments can be set in the engine file or in the settings file
+(normally ``settings.yml``). The arguments in the settings file override
+the ones in the engine file.
+
+It does not matter if an option is stored in the engine file or in the
+settings. However, the standard way is the following:
+
+
+engine file
+~~~~~~~~~~~
+
++----------------------+-----------+-----------------------------------------+
+| argument             | type      | information                             |
++======================+===========+=========================================+
+| categories           | list      | pages, in which the engine is working   |
++----------------------+-----------+-----------------------------------------+
+| paging               | boolean   | support multible pages                  |
++----------------------+-----------+-----------------------------------------+
+| language\_support    | boolean   | support language choosing               |
++----------------------+-----------+-----------------------------------------+
+| time\_range\_support | boolean   | support search time range               |
++----------------------+-----------+-----------------------------------------+
+| offline              | boolean   | engine runs offline                     |
++----------------------+-----------+-----------------------------------------+
+
+settings.yml
+~~~~~~~~~~~~
+
++------------+----------+-----------------------------------------------+
+| argument   | type     | information                                   |
++============+==========+===============================================+
+| name       | string   | name of search-engine                         |
++------------+----------+-----------------------------------------------+
+| engine     | string   | name of searx-engine (filename without .py)   |
++------------+----------+-----------------------------------------------+
+| shortcut   | string   | shortcut of search-engine                     |
++------------+----------+-----------------------------------------------+
+| timeout    | string   | specific timeout for search-engine            |
++------------+----------+-----------------------------------------------+
+
+overrides
+~~~~~~~~~
+
+A few of the options have default values in the engine, but are
+often overwritten by the settings. If ``None`` is assigned to an option
+in the engine file, it has to be redefined in the settings,
+otherwise searx will not start with that engine.
+
+The naming of overrides is arbitrary. But the recommended
+overrides are the following:
+
++-----------------------+----------+----------------------------------------------------------------+
+| argument              | type     | information                                                    |
++=======================+==========+================================================================+
+| base\_url             | string   | base-url, can be overwritten to use same engine on other URL   |
++-----------------------+----------+----------------------------------------------------------------+
+| number\_of\_results   | int      | maximum number of results per request                          |
++-----------------------+----------+----------------------------------------------------------------+
+| language              | string   | ISO code of language and country like en\_US                   |
++-----------------------+----------+----------------------------------------------------------------+
+| api\_key              | string   | api-key if required by engine                                  |
++-----------------------+----------+----------------------------------------------------------------+
+
+example code
+~~~~~~~~~~~~
+
+.. code:: python
+
+    # engine dependent config
+    categories = ['general']
+    paging = True
+    language_support = True
+
+making a request
+----------------
+
+To perform a search an URL have to be specified. In addition to
+specifying an URL, arguments can be passed to the query.
+
+passed arguments
+~~~~~~~~~~~~~~~~
+
+These arguments can be used to construct the search query. Furthermore,
+parameters with default value can be redefined for special purposes.
+
++----------------------+------------+------------------------------------------------------------------------+
+| argument             | type       | default-value, information                                             |
++======================+============+========================================================================+
+| url                  | string     | ``''``                                                                 |
++----------------------+------------+------------------------------------------------------------------------+
+| method               | string     | ``'GET'``                                                              |
++----------------------+------------+------------------------------------------------------------------------+
+| headers              | set        | ``{}``                                                                 |
++----------------------+------------+------------------------------------------------------------------------+
+| data                 | set        | ``{}``                                                                 |
++----------------------+------------+------------------------------------------------------------------------+
+| cookies              | set        | ``{}``                                                                 |
++----------------------+------------+------------------------------------------------------------------------+
+| verify               | boolean    | ``True``                                                               |
++----------------------+------------+------------------------------------------------------------------------+
+| headers.User-Agent   | string     | a random User-Agent                                                    |
++----------------------+------------+------------------------------------------------------------------------+
+| category             | string     | current category, like ``'general'``                                   |
++----------------------+------------+------------------------------------------------------------------------+
+| started              | datetime   | current date-time                                                      |
++----------------------+------------+------------------------------------------------------------------------+
+| pageno               | int        | current pagenumber                                                     |
++----------------------+------------+------------------------------------------------------------------------+
+| language             | string     | specific language code like ``'en_US'``, or ``'all'`` if unspecified   |
++----------------------+------------+------------------------------------------------------------------------+
+
+parsed arguments
+~~~~~~~~~~~~~~~~
+
+The function ``def request(query, params):`` always returns the
+``params`` variable. Inside searx, the following paramters can be
+used to specify a search request:
+
++------------+-----------+---------------------------------------------------------+
+| argument   | type      | information                                             |
++============+===========+=========================================================+
+| url        | string    | requested url                                           |
++------------+-----------+---------------------------------------------------------+
+| method     | string    | HTTP request method                                     |
++------------+-----------+---------------------------------------------------------+
+| headers    | set       | HTTP header information                                 |
++------------+-----------+---------------------------------------------------------+
+| data       | set       | HTTP data information (parsed if ``method != 'GET'``)   |
++------------+-----------+---------------------------------------------------------+
+| cookies    | set       | HTTP cookies                                            |
++------------+-----------+---------------------------------------------------------+
+| verify     | boolean   | Performing SSL-Validity check                           |
++------------+-----------+---------------------------------------------------------+
+
+example code
+~~~~~~~~~~~~
+
+.. code:: python
+
+    # search-url
+    base_url = 'https://example.com/'
+    search_string = 'search?{query}&page={page}'
+
+    # do search-request
+    def request(query, params):
+        search_path = search_string.format(
+            query=urlencode({'q': query}),
+            page=params['pageno'])
+
+        params['url'] = base_url + search_path
+
+        return params
+
+returned results
+----------------
+
+Searx is able to return results of different media-types.
+Currently the following media-types are supported:
+
+-  default
+-  images
+-  videos
+-  torrent
+-  map
+
+To set another media-type as default, the parameter
+``template`` must be set to the desired type.
+
+default
+~~~~~~~
+
++--------------------+---------------------------------------------------------------------------------------------------------------+
+| result-parameter   | information                                                                                                   |
++====================+===============================================================================================================+
+| url                | string, url of the result                                                                                     |
++--------------------+---------------------------------------------------------------------------------------------------------------+
+| title              | string, title of the result                                                                                   |
++--------------------+---------------------------------------------------------------------------------------------------------------+
+| content            | string, general result-text                                                                                   |
++--------------------+---------------------------------------------------------------------------------------------------------------+
+| publishedDate      | `datetime.datetime <https://docs.python.org/2/library/datetime.html#datetime-objects>`__, time of publish     |
++--------------------+---------------------------------------------------------------------------------------------------------------+
+
+images
+~~~~~~
+
+to use this template, the parameter
+
++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| result-parameter   | information                                                                                                                           |
++====================+=======================================================================================================================================+
+| template           | is set to ``images.html``                                                                                                             |
++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| url                | string, url to the result site                                                                                                        |
++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| title              | string, title of the result *(partly implemented)*                                                                                    |
++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| content            | *(partly implemented)*                                                                                                                |
++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| publishedDate      | `datetime.datetime <https://docs.python.org/2/library/datetime.html#datetime-objects>`__, time of publish *(partly implemented)*      |
++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| img\_src           | string, url to the result image                                                                                                       |
++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| thumbnail\_src     | string, url to a small-preview image                                                                                                  |
++--------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+
+videos
+~~~~~~
+
++--------------------+--------------------------------------------------------------------------------------------------------------+
+| result-parameter   | information                                                                                                  |
++====================+==============================================================================================================+
+| template           | is set to ``videos.html``                                                                                    |
++--------------------+--------------------------------------------------------------------------------------------------------------+
+| url                | string, url of the result                                                                                    |
++--------------------+--------------------------------------------------------------------------------------------------------------+
+| title              | string, title of the result                                                                                  |
++--------------------+--------------------------------------------------------------------------------------------------------------+
+| content            | *(not implemented yet)*                                                                                      |
++--------------------+--------------------------------------------------------------------------------------------------------------+
+| publishedDate      | `datetime.datetime <https://docs.python.org/2/library/datetime.html#datetime-objects>`__, time of publish    |
++--------------------+--------------------------------------------------------------------------------------------------------------+
+| thumbnail          | string, url to a small-preview image                                                                         |
++--------------------+--------------------------------------------------------------------------------------------------------------+
+
+torrent
+~~~~~~~
+
++------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| result-parameter | information                                                                                                                           |
++==================+=======================================================================================================================================+
+| template         | is set to ``torrent.html``                                                                                                            |
++------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| url              | string, url of the result                                                                                                             |
++------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| title            | string, title of the result                                                                                                           |
++------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| content          | string, general result-text                                                                                                           |
++------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| publishedDate    | `datetime.datetime <https://docs.python.org/2/library/datetime.html#datetime-objects>`__, time of publish *(not implemented yet)*     |
++------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| seed             | int, number of seeder                                                                                                                 |
++------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| leech            | int, number of leecher                                                                                                                |
++------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| filesize         | int, size of file in bytes                                                                                                            |
++------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| files            | int, number of files                                                                                                                  |
++------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| magnetlink       | string, `magnetlink <https://en.wikipedia.org/wiki/Magnet_URI_scheme>`__ of the result                                                |
++------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+| torrentfile      | string, torrentfile of the result                                                                                                     |
++------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+
+
+map
+~~~
+
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| result-parameter        | information                                                                                                  |
++=========================+==============================================================================================================+
+| url                     | string, url of the result                                                                                    |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| title                   | string, title of the result                                                                                  |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| content                 | string, general result-text                                                                                  |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| publishedDate           | `datetime.datetime <https://docs.python.org/2/library/datetime.html#datetime-objects>`__, time of publish    |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| latitude                | latitude of result (in decimal format)                                                                       |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| longitude               | longitude of result (in decimal format)                                                                      |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| boundingbox             | boundingbox of result (array of 4. values ``[lat-min, lat-max, lon-min, lon-max]``)                          |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| geojson                 | geojson of result (http://geojson.org)                                                                       |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| osm.type                | type of osm-object (if OSM-Result)                                                                           |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| osm.id                  | id of osm-object (if OSM-Result)                                                                             |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| address.name            | name of object                                                                                               |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| address.road            | street name of object                                                                                        |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| address.house\_number   | house number of object                                                                                       |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| address.locality        | city, place of object                                                                                        |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| address.postcode        | postcode of object                                                                                           |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+| address.country         | country of object                                                                                            |
++-------------------------+--------------------------------------------------------------------------------------------------------------+
+
diff --git a/docs/dev/install/installation.rst b/docs/dev/install/installation.rst
new file mode 100644
index 000000000..50b33a324
--- /dev/null
+++ b/docs/dev/install/installation.rst
@@ -0,0 +1,329 @@
+.. _installation:
+
+Installation
+============
+
+.. contents::
+   :depth: 3
+
+Basic installation
+------------------
+
+Step by step installation for Debian/Ubuntu with virtualenv. For Ubuntu, be sure to have enable universe repository.
+
+Install packages:
+
+.. code:: sh
+
+    sudo apt-get install git build-essential libxslt-dev python-dev python-virtualenv python-babel zlib1g-dev libffi-dev libssl-dev
+
+Install searx:
+
+.. code:: sh
+
+    cd /usr/local
+    sudo git clone https://github.com/asciimoo/searx.git
+    sudo useradd searx -d /usr/local/searx
+    sudo chown searx:searx -R /usr/local/searx
+
+Install dependencies in a virtualenv:
+
+.. code:: sh
+
+    sudo -u searx -i
+    cd /usr/local/searx
+    virtualenv searx-ve
+    . ./searx-ve/bin/activate
+    ./manage.sh update_packages
+
+Configuration
+-------------
+
+.. code:: sh
+
+    sed -i -e "s/ultrasecretkey/`openssl rand -hex 16`/g" searx/settings.yml
+
+Edit searx/settings.yml if necessary.
+
+Check
+-----
+
+Start searx:
+
+.. code:: sh
+
+    python searx/webapp.py
+
+Go to http://localhost:8888
+
+If everything works fine, disable the debug option in settings.yml:
+
+.. code:: sh
+
+    sed -i -e "s/debug : True/debug : False/g" searx/settings.yml
+
+At this point searx is not demonized ; uwsgi allows this.
+
+You can exit the virtualenv and the searx user bash (enter exit command
+twice).
+
+uwsgi
+-----
+
+Install packages:
+
+.. code:: sh
+
+    sudo apt-get install uwsgi uwsgi-plugin-python
+
+Create the configuration file /etc/uwsgi/apps-available/searx.ini with
+this content:
+
+::
+
+    [uwsgi]
+    # Who will run the code
+    uid = searx
+    gid = searx
+
+    # disable logging for privacy
+    disable-logging = true
+
+    # Number of workers (usually CPU count)
+    workers = 4
+
+    # The right granted on the created socket
+    chmod-socket = 666
+
+    # Plugin to use and interpretor config
+    single-interpreter = true
+    master = true
+    plugin = python
+    lazy-apps = true
+    enable-threads = true
+
+    # Module to import
+    module = searx.webapp
+
+    # Virtualenv and python path
+    virtualenv = /usr/local/searx/searx-ve/
+    pythonpath = /usr/local/searx/
+    chdir = /usr/local/searx/searx/
+
+Activate the uwsgi application and restart:
+
+.. code:: sh
+
+    cd /etc/uwsgi/apps-enabled
+    ln -s ../apps-available/searx.ini
+    /etc/init.d/uwsgi restart
+
+Web server
+----------
+
+with nginx
+^^^^^^^^^^
+
+If nginx is not installed (uwsgi will not work with the package
+nginx-light):
+
+.. code:: sh
+
+    sudo apt-get install nginx
+
+Hosted at /
+"""""""""""
+
+Create the configuration file /etc/nginx/sites-available/searx with this
+content:
+
+.. code:: nginx
+
+    server {
+        listen 80;
+        server_name searx.example.com;
+        root /usr/local/searx;
+
+        location / {
+                include uwsgi_params;
+                uwsgi_pass unix:/run/uwsgi/app/searx/socket;
+        }
+    }
+
+Create a symlink to sites-enabled:
+
+.. code:: sh
+
+   sudo ln -s /etc/nginx/sites-available/searx /etc/nginx/sites-enabled/searx
+
+Restart service:
+
+.. code:: sh
+
+    sudo service nginx restart
+    sudo service uwsgi restart
+
+from subdirectory URL (/searx)
+""""""""""""""""""""""""""""""
+
+Add this configuration in the server config file
+/etc/nginx/sites-enabled/default:
+
+.. code:: nginx
+
+    location = /searx { rewrite ^ /searx/; }
+    location /searx {
+            try_files $uri @searx;
+    }
+    location @searx {
+            uwsgi_param SCRIPT_NAME /searx;
+            include uwsgi_params;
+            uwsgi_modifier1 30;
+            uwsgi_pass unix:/run/uwsgi/app/searx/socket;
+    }
+
+
+OR
+
+using reverse proxy
+(Please, note that reverse proxy advised to be used in case of single-user or low-traffic instances.)
+
+.. code:: nginx
+
+    location /searx {
+        proxy_pass http://127.0.0.1:8888;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Scheme $scheme;
+        proxy_set_header X-Script-Name /searx;
+        proxy_buffering off;
+    }
+
+
+Enable base\_url in searx/settings.yml
+
+::
+
+    base_url : http://your.domain.tld/searx/
+
+Restart service:
+
+.. code:: sh
+
+    sudo service nginx restart
+    sudo service uwsgi restart
+
+disable logs
+~~~~~~~~~~~~
+
+for better privacy you can disable nginx logs about searx.
+
+how to proceed: below ``uwsgi_pass`` in
+/etc/nginx/sites-available/default add
+
+::
+
+    access_log /dev/null;
+    error_log /dev/null;
+
+Restart service:
+
+.. code:: sh
+
+    sudo service nginx restart
+
+with apache
+^^^^^^^^^^^
+
+Add wsgi mod:
+
+.. code:: sh
+
+    sudo apt-get install libapache2-mod-uwsgi
+    sudo a2enmod uwsgi
+
+Add this configuration in the file /etc/apache2/apache2.conf:
+
+.. code:: apache
+
+    <Location />
+        Options FollowSymLinks Indexes
+        SetHandler uwsgi-handler
+        uWSGISocket /run/uwsgi/app/searx/socket
+    </Location>
+
+Note that if your instance of searx is not at the root, you should
+change ``<Location />`` by the location of your instance, like
+``<Location /searx>``.
+
+Restart Apache:
+
+.. code:: sh
+
+    sudo /etc/init.d/apache2 restart
+
+disable logs
+""""""""""""
+
+For better privacy you can disable Apache logs.
+
+WARNING: not tested
+
+WARNING: you can only disable logs for the whole (virtual) server not
+for a specific path.
+
+Go back to /etc/apache2/apache2.conf and above ``<Location />`` add:
+
+.. code:: apache
+
+    CustomLog /dev/null combined
+
+Restart Apache:
+
+.. code:: sh
+
+    sudo /etc/init.d/apache2 restart
+
+How to update
+-------------
+
+.. code:: sh
+
+    cd /usr/local/searx
+    sudo -u searx -i
+    . ./searx-ve/bin/activate
+    git stash
+    git pull origin master
+    git stash apply
+    ./manage.sh update_packages
+    sudo service uwsgi restart
+
+Docker
+------
+
+Make sure you have installed Docker. For instance, you can deploy searx like this:
+
+.. code:: sh
+
+    docker pull wonderfall/searx
+    docker run -d --name searx -p $PORT:8888 wonderfall/searx
+
+Go to http://localhost:$PORT.
+
+See https://hub.docker.com/r/wonderfall/searx/ for more informations.
+
+It's also possible to build searx from the embedded Dockerfile.
+
+.. code:: sh
+
+    git clone https://github.com/asciimoo/searx.git
+    cd searx
+    docker build -t whatever/searx .
+
+References
+==========
+
+    * https://about.okhin.fr/posts/Searx/ with some additions
+
+    * How to: `Setup searx in a couple of hours with a free SSL certificate <https://www.reddit.com/r/privacytoolsIO/comments/366kvn/how_to_setup_your_own_privacy_respecting_search/>`__
+
diff --git a/docs/dev/plugins.rst b/docs/dev/plugins.rst
new file mode 100644
index 000000000..1d77f8611
--- /dev/null
+++ b/docs/dev/plugins.rst
@@ -0,0 +1,45 @@
+Plugins
+-------
+
+Plugins can extend or replace functionality of various components of
+searx.
+
+Example plugin
+~~~~~~~~~~~~~~
+
+.. code:: python
+
+    name = 'Example plugin'
+    description = 'This plugin extends the suggestions with the word "example"'
+    default_on = False  # disabled by default
+
+    js_dependencies = tuple()  # optional, list of static js files
+    css_dependencies = tuple()  # optional, list of static css files
+
+
+    # attach callback to the post search hook
+    #  request: flask request object
+    #  ctx: the whole local context of the post search hook
+    def post_search(request, ctx):
+        ctx['search'].suggestions.add('example')
+        return True
+
+Plugin entry points
+~~~~~~~~~~~~~~~~~~~
+
+Entry points (hooks) define when a plugin runs. Right now only three hooks are implemented. So feel free to implement a hook if it fits the behaviour of your plugin.
+
+Pre search hook
+```````````````
+
+Runs BEFORE the search request. Function to implement: ``pre_search``
+
+Post search hook
+````````````````
+
+Runs AFTER the search request. Function to implement: ``post_search``
+
+Result hook
+```````````
+
+Runs when a new result is added to the result list. Function to implement: ``on_result``
diff --git a/docs/dev/quickstart.rst b/docs/dev/quickstart.rst
new file mode 100644
index 000000000..e7d4baf46
--- /dev/null
+++ b/docs/dev/quickstart.rst
@@ -0,0 +1,108 @@
+.. _devquickstart:
+
+Development Quickstart
+----------------------
+
+This quickstart guide gets your environment set up with searx. Furthermore, it gives a
+short introduction to the new manage.sh script.
+
+How to setup your development environment
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+First, clone the source code of searx to the desired folder. In this case the source
+is cloned to ~/myprojects/searx. Then create and activate the searx-ve
+virtualenv and install the required packages using manage.sh.
+
+.. code:: sh
+
+    cd ~/myprojects
+    git clone https://github.com/asciimoo/searx.git
+    cd searx
+    virtualenv searx-ve
+    . ./searx-ve/bin/activate
+    ./manage.sh update_dev_packages
+
+
+How to run tests
+~~~~~~~~~~~~~~~~
+
+Tests can be run using the manage.sh script.
+
+Following tests and checks are available:
+
+- Unit tests
+
+- Selenium tests
+
+- PEP8 validation
+
+- Unit test coverage check
+
+For example unit tests are run with the command below:
+
+.. code:: sh
+
+    ./manage.sh unit_tests
+
+For further test options, please consult the help of the manage.sh script.
+
+
+How to compile styles and javascript
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+How to build styles
+^^^^^^^^^^^^^^^^^^^
+
+Less is required to build the styles of searx. Less can be installed using either NodeJS or Apt.
+
+.. code:: sh
+
+    sudo apt-get install nodejs
+    sudo npm install -g less
+
+
+OR
+
+.. code:: sh
+
+    sudo apt-get install node-less
+
+After satisfying the requirements styles can be build using manage.sh
+
+.. code:: sh
+
+    ./manage.sh styles
+
+
+How to build the source of the oscar theme
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Grunt must be installed in order to build the javascript sources. It depends on NodeJS, so first
+Node has to be installed.
+
+.. code:: sh
+
+    sudo apt-get install nodejs
+    sudo npm install -g grunt-cli
+
+After installing grunt, the files can be built using the following command: 
+
+.. code:: sh
+
+    ./manage.sh grunt_build
+
+
+
+Tips for debugging/development
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. Turn on debug logging
+    Whether you are working on a new engine or trying to eliminate a bug, it is always a good idea
+    to turn on debug logging. When debug logging is enabled a stack trace appears,
+    instead of the cryptic ``Internal Server Error`` message. It can be turned on by setting
+    ``debug: False`` to ``debug: True`` in settings.yml.
+
+2. Run ``./manage.sh tests`` before creating a PR.
+    Failing build on Travis is common because of PEP8 checks. So a new commit must be created
+    containing these format fixes. This phase can be skipped if ``./manage.sh tests`` is run
+    locally before creating a PR.
diff --git a/docs/dev/search_api.rst b/docs/dev/search_api.rst
new file mode 100644
index 000000000..ea3b0501a
--- /dev/null
+++ b/docs/dev/search_api.rst
@@ -0,0 +1,204 @@
+Search API
+==========
+
+The search supports both ``GET`` and ``POST``.
+
+Furthermore, two enpoints ``/`` and ``/search`` are available for querying.
+
+
+``GET /``
+
+``GET /search``
+
+Parameters
+~~~~~~~~~~
+
+.. code:: sh
+
+    q
+
+The search query. This string is passed to external search services.
+Thus, searx supports syntax of each search service. For example, ``site:github.com searx`` is a valid
+query for Google. However, if simply the query above is passed to any search engine which does not filter its
+results based on this syntax, you might not get the results you wanted.
+
+
+See more at :doc:`/user/search_syntax`
+
+Required.
+
+.. code:: sh
+
+    categories
+
+Comma separated list, specifies the active search categories
+
+Optional.
+
+.. code:: sh
+
+    engines
+
+Comma separated list, specifies the active search engines.
+
+Optional.
+
+.. code:: sh
+
+    lang
+
+Code of the language.
+
+Optional.
+
+Default: ``all``
+
+.. code:: sh
+
+    pageno
+
+Search page number.
+
+Optional.
+
+Default: ``1``
+
+.. code:: sh
+
+    time_range
+
+Time range of search for engines which support it. See if an engine supports time range search in the preferences page of an instance.
+
+Optional.
+
+Possible: ``day``, ``month``, ``year``
+
+.. code:: sh
+
+    format
+
+Output format of results.
+
+Optional.
+
+Possible: ``json``, ``csv``, ``rss``
+
+.. code:: sh
+
+    results_on_new_tab
+
+Open search results on new tab.
+
+Optional.
+
+Default: ``0``
+
+Possible: ``0``, ``1``
+
+.. code:: sh
+
+    image_proxy
+
+Proxy image results through searx.
+
+Optional.
+
+Default: ``False``
+
+Possible: ``True``, ``False``
+
+.. code:: sh
+
+    autocomplete
+
+Service which completes words as you type.
+
+Optional.
+
+Default: empty
+
+Possible: ``google``, ``dbpedia``, ``duckduckgo``, ``startpage``, ``wikipedia``
+
+.. code:: sh
+
+    safesearch
+
+Filter search results of engines which support safe search. See if an engine supports safe search in the preferences page of an instance.
+
+Optional.
+
+Default: ``None``
+
+Possible: ``0``, ``1``, ``None``
+
+.. code:: sh
+
+    theme
+
+Theme of instance.
+
+Optional.
+
+Default: ``oscar``
+
+Possible: ``oscar``, ``simple``, ``legacy``, ``pix-art``, ``courgette``
+
+Please note, available themes depend on an instance. It is possible that an instance administrator deleted, created or renamed themes on his/her instance. See the available options in the preferences page of the instance.
+
+.. code:: sh
+
+    oscar-style
+
+Style of Oscar theme. It is only parsed if the theme of an instance is ``oscar``.
+
+Optional.
+
+Default: ``logicodev``
+
+Possible: ``pointhi``, ``logicodev``
+
+Please note, available styles depend on an instance. It is possible that an instance administrator deleted, created or renamed styles on his/her instance. See the available options in the preferences page of the instance.
+
+.. code:: sh
+
+    enabled_plugins
+
+List of enabled plugins.
+
+Optional.
+
+Default: ``HTTPS_rewrite``, ``Self_Informations``, ``Search_on_category_select``, ``Tracker_URL_remover``
+
+Possible: ``DOAI_rewrite``, ``HTTPS_rewrite``, ``Infinite_scroll``, ``Vim-like_hotkeys``, ``Self_Informations``, ``Tracker_URL_remover``, ``Search_on_category_select``
+
+.. code:: sh
+
+    disabled_plugins
+
+List of disabled plugins.
+
+Optional.
+
+Default: ``DOAI_rewrite``, ``Infinite_scroll``, ``Vim-like_hotkeys``
+
+Possible: ``DOAI_rewrite``, ``HTTPS_rewrite``, ``Infinite_scroll``, ``Vim-like_hotkeys``, ``Self_Informations``, ``Tracker_URL_remover``, ``Search_on_category_select``
+
+.. code:: sh
+
+    enabled_engines
+
+List of enabled engines.
+
+Optional.
+
+Possible:  all engines
+
+.. code:: sh
+
+    disabled_engines
+
+List of disabled engines.
+
+Optional.
+
+Possible: all engines
diff --git a/docs/dev/translation.rst b/docs/dev/translation.rst
new file mode 100644
index 000000000..ab4dd98cb
--- /dev/null
+++ b/docs/dev/translation.rst
@@ -0,0 +1,61 @@
+Translation
+===========
+
+Requirements
+------------
+
+ * Transifex account
+
+ * Installed CLI tool of Transifex
+
+Init Transifex project
+----------------------
+
+After installing ``transifex`` using pip, run the following command to initialize the project.
+
+.. code:: shell
+
+    tx init # Transifex instance: https://www.transifex.com/asciimoo/searx/
+
+
+After ``$HOME/.transifexrc`` is created, get a Transifex API key and insert it into the configuration file.
+
+Create a configuration file for ``tx`` named ``$HOME/.tx/config``.
+
+.. code:: shell
+
+    [main]
+    host = https://www.transifex.com
+    [searx.messagespo]
+    file_filter = searx/translations/<lang>/LC_MESSAGES/messages.po
+    source_file = messages.pot
+    source_lang = en
+    type = PO
+
+
+Then run ``tx set``:
+
+.. code:: shell
+
+    tx set --auto-local -r searx.messagespo 'searx/translations/<lang>/LC_MESSAGES/messages.po' \
+    --source-lang en --type PO --source-file messages.pot --execute
+
+
+Update translations
+-------------------
+
+To retrieve the latest translations, pull it from Transifex.
+
+.. code:: shell
+
+    tx pull -a
+
+Then check the new languages. If strings translated are not enough, delete those folders, because
+those should not be compiled. Call the command below to compile the ``.po`` files.
+
+.. code:: shell
+
+    pybabel compile -d searx/translations
+
+
+After the compilation is finished commit the ``.po`` and ``.mo`` files and create a PR.