README()

Introduction

This document aims to provide information about Elektra’s website-backend feature. website-backend offers a RESTful server to search, store and convert configuration snippets. It also offers authentication methods to restrict access to manipulative methods. A detailed description of the implemented API can be found in the API description /doc/api_blueprints/snippet-sharing.apib.

Stored are snippet and user entries, both below different paths. Each entry consists of multiple keys (+ meta keys) belonging together, while the root key matching a certain schema (regex) is used to identify an entry.

The REST service operates on user-defined repositories (paths). This paths are not meant to be used in other tools or applications, as the service caches all data in-memory as well. Changes to the repositories can therefore result in unexpected and undefined behavior, both for the service as well as the stored data.

The reason why a special repository is used to store and retrieve entries is that opening the system to everyone by providing a public interface to the whole key database would result in a security leak. For a REST API with such an interface, another tool will be published.

Run and Configure
To run the website-backend we need to find out where it has been installed to. This can be done by running the command kdb list-tools which will output a list of installed tools and something like External tools are located in /usr/local/lib/elektra/tool_exec. With this path we can run the service like cd /usr/local/lib/elektra/tool_exec/ && sh run-website-backend. An alternative is to make use of the kdb tool and run kdb run-website-backend.

The REST service can also be configured. All configuration is read from Elektra’s key database at start-time of the service. Further details and configuration options are listed below.

To stop the service, run sh stop-website-backend in the directory where the start script is located or kdb stop-website-backend from anywhere.

  Configuration
The service stores all its configuration below /sw/elektra/websitebackend/#0/current, which is split into two parts:

•   one for CppCMS below /sw/elektra/websitebackend/#0/current/cppcms and

•   one for the service itself below /sw/elektra/websitebackend/#0/current/backend

All configuration options for CppCMS are listed on their Website http://cppcms.com/wikipp/en/page/cppcms_1x_config. The JSON configuration explained on the website can be translated into Elektra keys easily. For the values true and false, the strings "true" and "false" can be used.

The following example configuration in CppCMS style

    {
      "service": {
        "api": "http",
        "port": 8080,
        "ip": "0.0.0.0"
      },
      "security": {
        "display_error_messages": true
      }
    }

can be realized within Elektra like

    kdb set /sw/elektra/websitebackend/#0/current/cppcms/service/api "http"
    kdb set /sw/elektra/websitebackend/#0/current/cppcms/service/port 8080
    kdb set /sw/elektra/websitebackend/#0/current/cppcms/service/ip "0.0.0.0"
    kdb set /sw/elektra/websitebackend/#0/current/cppcms/security/display_error_messages "true"

Simply set the desired settings as keys in the key database and you are done!

The service itself offers quite some configuration options as well. In detail, the options (without the base key /sw/elektra/websitebackend/#0/) are:

    [current]
    mountpoint = website-backend.ini
    [current/backend/api/description/raw]
    check/type = string
    description = A link to the blueprint describing the API.
    default = https://master.libelektra.org/doc/api_blueprints/snippet-sharing.apib
    [current/backend/api/description/html]
    check/type = string
    description = A link to the compiled blueprint describing the API.
    default = https://doc.libelektra.org/restapi/0.9.1/snippet-sharing.html
    [current/backend/jwt/encryption/secret]
    check/type = string
    description = A secret string used to encrypt session tokens (JWT).
    example = al3h120d8a_19s
    [current/backend/jwt/validity]
    check/type = long
    description = The number of seconds a JWT is valid from its creation.
    example = 3600
    default = 7200
    [current/backend/kdb/path/configs]
    check/type = string
    description = The root path being used to store configuration snippet entries, must start with a namespace.
    example = user/website-backend/configs
    default = dir/configs
    [current/backend/kdb/path/users]
    check/type = string
    description = The root path being used to store user entries, must start with a namespace.
    example = user/website-backend/users
    default = dir/users
    [current/backend/output/default/entry/sort]
    check/enum = 'asc', 'desc'
    description = The default sort direction being used for requests against configuration snippet entry resources.
    default = 'asc'
    [current/backend/output/default/entry/sortby]
    check/enum = 'key', 'organization', 'application', 'scope', 'slug', 'title', 'author', 'created_at'
    description = The default sort criteria being used for requests against configuration snippet entry resources.
    default = 'key'
    [current/backend/output/default/entry/filterby]
    check/enum = 'all', 'key', 'title', 'description', 'author', 'tags'
    description = The default filter criteria being used for requests against configuration snippet entry resources.
    default = 'all'
    [current/backend/output/default/user/sort]
    check/enum  = 'asc', 'desc'
    description = The default sort direction being used for requests against user entry resources.
    default = 'asc'
    [current/backend/output/default/user/sortby]
    check/enum = 'username', 'email', 'created_at'
    description = The default sort criteria being used for requests against user entry resources.
    default = 'username'
    [current/backend/output/default/user/filterby]
    check/enum = 'all', 'username', 'email'
    description = The default filter criteria being used for requests against user entry resources.
    default = 'all'
    [current/backend/permissions/entry/create]
    check/type = long
    check/type/min = 0
    check/type/max = 100
    description = The required rank a user needs to be able to create new configuration snippet entries.
    default = 10
    [current/backend/permissions/entry/edit]
    check/type = long
    check/type/min = 0
    check/type/max = 100
    description = The required rank a user needs to be able to edit any configuration snippet entry (also from other users).
    default = 50
    [current/backend/permissions/entry/delete]
    check/type = long
    check/type/min = 0
    check/type/max = 100
    description = The required rank a user needs to be able to delete any configuration snippet entry (also from other users).
    default = 50
    [current/backend/permissions/user/view]
    check/type = long
    check/type/min = 0
    check/type/max = 100
    description = The required rank a user needs to be able to view account details of other users.
    default = 100
    [current/backend/permissions/user/edit]
    check/type = long
    check/type/min = 0
    check/type/max = 100
    description = The required rank a user needs to be able to edit account details of other users.
    default = 100
    [current/backend/permissions/user/delete]
    check/type = long
    check/type/min = 0
    check/type/max = 100
    description = The required rank a user needs to be able to delete user accounts of other users.
    default = 100
    [current/backend/permissions/rank/default]
    check/type = long
    check/type/min = 0
    check/type/max = 100
    description = The default rank new users will receive on registration.
    default = 10
    [current/cppcms/service/api]
    check/enum = 'fastcgi', 'scgi', 'http'
    description = This options specifies the API the CppCMS application communicates with client or web server.
    [current/cppcms/service/ip]
    check/type = string
    description = This option defines the IPv4/IPv6 IP the application should listen on. By default it listens on "127.0.0.1".
    default = 127.0.0.1
    [current/cppcms/service/port]
    check/type = long
    description = This option defines the port the application should listen on, default is 8080.
    default = 8080
    [current/cppcms/service/socket]
    check/type = string
    description = This option defines the Unix domain socket that the application should listen on. Only use this if IP and port are not set.
    [current/cppcms/service/worker_threads]
    check/type = long
    description = The number of worker threads per process. Default is 5 * number of CPUs. For example quad core it would be 20 threads.
    [current/cppcms/service/worker_processes]
    check/type = long
    description = The number of forked worker processes. This option is relevant only for POSIX platforms. Values: 0 means no fork is executed, default; 1 means that one child process is forked and the parent supervises and and would restart if in case of crash; >1 several processes are forked and try to accept incoming connections.
    default = 0
    [current/cppcms/service/backlog]
    check/type = long
    description = The second parameter to listen() system call, by default it is twice of size of service.worker_threads. It is good idea to set it to high values if you experience problems in connecting to server.
    [current/cppcms/service/applications_pool_size]
    check/type = long
    description = User application objects are generally cached in special pool for future faster reuse, this parameter defines maximal number of applications that can be cached there. By default it is twice of size of service.worker_threads.
    [current/cppcms/service/disable_xpowered_by]
    check/type = boolean
    description = By default CppCMS sends X-Powered-By: CppCMS/X.Y.Z handler in response, this can be disabled by setting this parameter to true.
    default = 0
    [current/cppcms/service/output_buffer_size]
    check/type = long
    description = The default size of the output buffer that is used for caching output stream.
    default = 16384
    [current/cppcms/service/generate_http_headers]
    check/type = boolean
    description = Send the HTTP headers in response rather then CGI ones. Useful for broken SCGI connectors like isapi_scgi.
    default = 0
    [current/cppcms/security/content_length_limit]
    check/type = long
    description = The maximal size of POST data in KB.
    default = 1024
    [current/cppcms/security/multipart_form_data_limit]
    check/type = long
    description = The maximal size of multipart/form_data POST in KB (i.e. maximal allowed upload size).
    default = 65536
    [current/cppcms/security/file_in_memory_limit]
    check/type = long
    description = When files are uploaded for efficiency, small files are generally stored in memory and big ones are saved in files. This is the limit on the file size to be stored in memory in bytes.
    default = 128
    [current/cppcms/security/uploads_path]
    check/type = string
    description = The location of temporary upload files. By default they are saved in the temporary directory defined by TEMP or TMP environment variable, or if they undefined it would use /tmp as a path for temporary files.
    [current/cppcms/security/display_error_message]
    check/type = boolean
    description = When the exception is thrown by user application and this parameter set to true its message what() would be displayed in 500 Internal Server error page, it is useful for debugging. However it should never be used in production environment.
    default = 0
    [current/cppcms/daemon/enable]
    check/type = boolean
    description = Create daemon process - fork off and become session leader.
    default = 1
    [current/cppcms/daemon/lock]
    check/type = string
    description = File name for lock file for this daemon. This file contains the process ID of the daemon that allows you to kill it.
    default = /run/elektra-website-backend.pid
    [current/cppcms/daemon/user]
    check/type = string
    description = The unprivileged user that this daemon should run under. It is recommended to use this option if the service is started with root privileges.
    [current/cppcms/daemon/group]
    check/type = string
    description = The unprivileged group that this daemon should run under. It is recommended to use this option if the service is started with root privileges.
    [current/cppcms/daemon/chroot]
    check/type = string
    description = Chroot to specific directory - extra security option that limits an access to specific tree.
    [current/cppcms/daemon/fdlimit]
    check/type = long
    description = Set maximal number of open file descriptors, it is useful for applications that handle many simulations connections.
    [current/cppcms/http/script]
    check/type = string
    description = The name of script that the application runs on. Actually it is what the SCRIPT_NAME CGI variable should be. If you using HTTP backend you need to specify one. The script name is matched against requested URL and if it matches its beginning it is used for dispatch application.
    default = /
    [current/cppcms/http/timeout]
    check/type = long
    description = The number of seconds to keep the idle connection alive, i.e. the connection that is blocking on read or on write other the connection that is waiting for client side disconnect using cppcms::http::context::async_on_peer_reset().
    default = 30
    [current/cppcms/fastcgi/concurrency_hint]
    check/type = long
    description = Special setting for concurrency ability of FastCGI server that may be queried by some web servers. Default is the total number of threads application uses (in all started processes).

  Configure as Service
To configure the website-backend as service, it is possible to use systemd on most systems.

1.  Create a new service file with the following command (and make sure the paths of ExecStart match your installation of Elektra):

    cat > /etc/systemd/system/website-backend.service << EOF
    [Unit]
    Description=Start the REST backend for sharing of configuration snippets
    Requires=network.target
    After=network.target
    [Service]
    Type=simple
    Restart=always
    ExecStart=kdb run-website-backend
    ExecStop=kdb stop-website-backend
    [Install]
    WantedBy=multi-user.target
    EOF

2.  Reload the configuration of systemctl with systemctl daemon-reload.

3.  Enable the website-backend service with systemctl enable website-backend.service, a symbolic link should be created.

4.  Make sure the service is enabled with systemctl is-enabled website-backend.service.

5.  Restart the website-backend service with systemctl restart website-backend.service. If everything went fine, the service should be reachable and systemctl status website-backend.service should print information about the running service (PID, etc).

Compiling and Installation

  Dependencies
In order to compile and use the new website-backend there are a few dependencies which must be installed.

•   CppCMS version 1.0.0 or higher

•   Boost version 1.45 or higher

•   LibJWT version 1.5 or higher

•   OpenSSL

An extensive tutorial describing the installation and configuration can be found here /doc/tutorials/snippet-sharing-rest-service.md.

  Compiling
Compile Elektra as normal as per the COMPILE document https://master.libelektra.org/doc/COMPILE.md, but make sure to include the website-backend tool using the -DTOOLS flag.

For instance: -DTOOLS=ALL or -DTOOLS=website-backend

  Installing
You can now install Elektra as you normally would or as described in the install documentation https://master.libelektra.org/doc/INSTALL.md.

Implementation Notes and Hints for Front-Ends
The here described tool offers an API which can be consumed by either a command line tool like cURL or a custom frontend. In the following some hints for frontend implementations will be given.

  Usability
The API validates all inputs, but does not respond always with exact error messages. Normally error messages contain a general hint on what input was wrong (e.g. 'the username has to be 3-20 signs long, contain only letters, digits and dashes'), but not what particular constraint was wrong for the last input (e.g. that the input was only 2 instead of 3 signs long). This limitation comes from the usage of regex patterns instead of atomic comparisons during validation.

In terms of usability this is sufficient, but not the best possible. Therefore it would be advisable to implement live-validation for frontends with more granularity. Information about allowed input formats can be found in the API description https://master.libelektra.org/doc/api_blueprints/snippet-sharing.apib.

Benchmarks
The service has been benchmarked against a MySQL solution, for further details see benchmarks readme benchmarks/README.md.