Database

xbat uses MongoDB for storing user, job and configuration data while QuestDB is used for time-series data. The xbat.service must be running in order to access the databases for intial configuration.

The Mongo database is hosted within the docker infrastructure (xbat-mongodb). Data is persistently stored at /var/lib/xbat/mongodb/ on the host and then mounted into the container. The corresponding mongod.conf is created on installation at /etc/xbat/ and does not require further modification.

Attach to xbat-mongodb container to start the configuration:

podman exec -it xbat-mongodb /bin/bash

Follow the steps below to create the necessary accounts for authentication. All accounts for authentication must be created in database admin

mongo
use admin

Create an admin account to manage MongoDB. This account is different from the admin account used to access xbat.

db.createUser({
    user: "admin",
    pwd: passwordPrompt(),
    roles: [
        { role: "userAdminAnyDatabase", db: "admin" },
        "readWriteAnyDatabase"
    ]
});

Exit mongo with CTRL+D. From now on access to MongoDB requires authentication. Exit the mongo tool and log back in with

mongo --authenticationDatabase=admin --username admin

Afterwards add a user called xbat. These credentials will be used by the xbat services to access the database and must be entered in the /etc/xbat/xbat.conf file under the [mongodb] section (user and password).

use admin
db.createUser(
    {
     user: "xbat",
     pwd: passwordPrompt(),
     roles: [ { role: "readWrite", db: "xbat" }]
    }
)

Authentication of all users is handled by OAuth and requires creating the admin user and client on initial setup. Switch to the xbat database with the following command:

use xbat

Create the xbat admin account. The default password is admin and can be changed via the user interface. All other users will be imported from the User Management System upon their first login.

db.users.insert({
    user_name: "admin",
    user_type: "admin",
    password: "*4ACFE3202A5FF5CF467898FC58AAB1D615029441"
});

Create the frontend and swagger clients for admin. All other user clients will be created automatically.

db.clients.insert({
    name: "admin",
    client_id: "wf_admin",
    redirect_uris: "",
    default_scopes:
        "benchmarks_r benchmarks_w configurations_r configurations_w projects_r projects_w settings_r settings_w users_r users_w user_self_r user_self_w benchmarks_dr benchmarks_dw"
});

db.clients.insert({
    name: "admin",
    client_id: "admin",
    redirect_uris:
        "https://localhost:7000/api/v1/ui/oauth2-redirect.html https://127.0.0.1:7000/api/v1/ui/oauth2-redirect.html",
    default_scopes:
        "benchmarks_r benchmarks_w configurations_r configurations_w projects_r projects_w settings_r settings_w users_r users_w user_self_r user_self_w benchmarks_dr benchmarks_dw"
});

The xbat daemon (xbatd) also authenticates with the backend via the REST-API. Create a client for xbatd by generating a client secret (typically a random string with length between 32 and 64).

openssl rand -base64 32

Insert the xbatd client into the database.

db.clients.insert({
    name: "xbatd",
    client_id: "xbatd",
    client_secret: "<CLIENT_SECRET>",
    redirect_uris: "",
    default_scopes: "benchmarks_dr benchmarks_dw"
});

Exit the mongo shell with CTRL+D and set the credentials of the xbat user in the /etc/xbat/xbat.conf file under the [mongodb] section.

QuestDB is hosted within the docker infrastructure (xbat-questdb). Data is persistently stored at /var/lib/xbat/questdb/ on the host and then mounted into the container. The corresponding questdb-log.conf and questdb.conf are created on installation at /etc/xbat/. The latter requires to set user and password for the PGWire and HTTP interface.

# for backend
pg.password=<password>
pg.user=<user>

# for xbatd
http.user=<user>
http.password=<password>

Afterwards set the credentials at /etc/xbat/xbat.conf for [questdb] accordingly. The user and password are equivalent to pg.user and pg.password in the questdb.conf. Credentials for api_user and api_password must match http.user and http.password.

QuestDB and Valkey require adjusting sysctl settings to prevent unpredictable behaviour and out-of-memory exceptions (see here).

Modify /etc/sysctl.conf to change limits.

# add/set open file limit (QuestDB)
fs.file-max=1048576

# add/set max virtual memory areas limit (QuestDB)
vm.max_map_count=1048576

# enable memory overcommitment (Valkey)
vm.overcommit_memory=1

Afterwards apply the changes with sysctl -p.