⚙ Server Configuration and Provisioning
Settings
General
ReductStore uses environment variables for configuration. Here is a list of available settings:
| Name | Default | Description |
|---|---|---|
RS_LOG_LEVEL | INFO | Logging level: TRACE, DEBUG, INFO, WARNING, ERROR. A user can filter logs by path e.g. INFO,reductstore/api=DEBUG |
RS_HOST | 0.0.0.0 | Listening IP address |
RS_PORT | 8383 | Listening port |
RS_API_BASE_PATH | / | Prefix for all URLs of requests |
RS_PUBLIC_URL | http(s)://[RS_HOST]:[RS_PORT][RS_API_BASE_PATH] | Public URL of the instance. Used for query links. |
RS_DATA_PATH | /data | Path to a folder where the storage stores the data |
RS_API_TOKEN | If set, the storage uses token authorization | |
RS_CERT_PATH | Path to an SSL certificate. If unset, the storage uses HTTP instead of HTTPS | |
RS_CERT_KEY_PATH | Path to the private key of the desired SSL certificate. Should be set with RS_CERT_PATH | |
RS_LICENSE_PATH | Path to the license file. Required for commercial usage. See Pricing for more details. | |
RS_CORS_ALLOW_ORIGIN | Sets the Access-Control-Allow-Origin header. Use * to allow all origins or list specific origins separated by commas. | |
RS_EXT_PATH | If set, it activates the experimental extension API. The path should be a folder with extension binaries. |
For Snap Users
If you use snap, you can configure the database by using the snap set command:
snap set reductstore log-level=DEBUG
This command changes the log level to DEBUG and restarts the database. You can check the current configuration with the snap get reductstoret command:
snap get reductstore
Key Value
api-base /
api-token
cert-key-path
cert-path
data-path /var/snap/reductstore/common
host 0.0.0.0
log-level DEBUG
port 8383
By default, the database uses the /var/snap/reductstore/common folder to store data. Due to snap restrictions, you can't use root filesystem folders.
We recommend that you connect the snap to the removable-media interface and store the data in a folder outside the snap in /mnt/* or /media/* folders:
snap connect reductstore:removable-media
mkdir /mnt/data
snap set reductstore data-path=/mnt/data
Removing the snap will remove all data stored in the default data path. If you want to keep the data, you need to move it to another location before removing the snap.
Or use the snap connect reductstore:removable-media command to connect the snap to the removable-media interface and store the data in a folder outside the snap.
Remote Backend Settings
ReductStore supports using a remote object storage as a backend to store data. You can configure the remote backend using the following environment variables:
| Name | Default | Description |
|---|---|---|
RS_REMOTE_BACKEND_TYPE | File system | If it is set to S3, the storage uses an S3-compatible object storage as a backend |
RS_REMOTE_ENDPOINT | URL of the S3-compatible object storage endpoint. It's ignored for the AWS S3 service. | |
RS_REMOTE_REGION | Region of the S3-compatible object storage. It's ignored for on-premises storage like MinIO. | |
RS_REMOTE_ACCESS_KEY | Access key for the S3-compatible object storage. | |
RS_REMOTE_SECRET_KEY | Secret key for the S3-compatible object storage. | |
RS_REMOTE_BUCKET | Name of the bucket to store data. The bucket must be created before starting ReductStore. | |
RS_REMOTE_CACHE_PATH | Path to a folder where the storage caches data before uploading it to the remote backend. | |
RS_REMOTE_CACHE_SIZE | 1GB | Maximum size of the cache folder. When the limit is reached, the oldest data is removed. |
RS_REMOTE_SYNC_INTERVAL | 0.1, 60 for S3 | Synchronization interval with the remote backend in seconds. |
RS_REMOTE_DEFAULT_STORAGE_CLASS | STANDARD | Default storage class for data blocks. See the list of supported S3 storage classes. |
By default, ReductStore uses the local file system to store data. To use a remote backend, set the RS_REMOTE_BACKEND_TYPE variable to S3 and configure the other variables accordingly.
When the remote backend is enabled, ReductStore caches data in the folder specified by the RS_REMOTE_CACHE_PATH variable, thereby reducing the number of requests made to the remote backend.
The RS_DATA_PATH variable is ignored when the remote backend is enabled.
Supported S3 Storage Classes
A user can specify default storage class for data blocks stored in the S3-compatible object storage using the RS_REMOTE_DEFAULT_STORAGE_CLASS environment variable.
The following storage classes are supported:
| Name | Description |
|---|---|
STANDARD | Standard storage class. Suitable for frequently accessed data. |
STANDARD_IA | Standard-Infrequent Access storage class. Suitable for data that is infrequently accessed but requires rapid access when needed. |
INTELLIGENT_TIERING | Intelligent-Tiering storage class. Automatically moves data to the most cost-effective access tier based on usage patterns. |
ONEZONE_IA | One Zone-Infrequent Access storage class. Suitable for data that is infrequently accessed and stored in a single availability zone. |
EXPRESS_ONEZONE | Express-One Zone storage class. Suitable for data that requires low latency access and is stored in a single availability zone. |
GLACIER_IR | Glacier Instant Retrieval storage class. Suitable for data that is rarely accessed and requires immediate access when needed. |
GLACIER | Glacier storage class. Suitable for data that is rarely accessed and can be retrieved within a few hours. |
DEEP_ARCHIVE | Deep Archive storage class. Suitable for data that is rarely accessed and can be retrieved within 12 hours. |
OUTPOSTS | Outposts storage class. Suitable for data that needs to be stored on AWS Outposts. |
For more information about S3 storage classes, see the Amazon S3 Storage Classes documentation.
I/O Settings
In addition to general settings, you can configure I/O settings to optimize communication over HTTP with the storage engine. The following table describes the available environment variables:
| Name | Default | Description |
|---|---|---|
RS_IO_BATCH_MAX_SIZE | 8MB | Maximum size of a batch of records sent to the client. |
RS_IO_BATCH_MAX_RECORDS | 85 | Maximum number of records in a batch sent and received from the client. |
RS_IO_BATCH_MAX_METADATA_SIZE | 8KB | Maximum size of metadata in a batch of records sent and received from the client. |
RS_IO_BATCH_TIMEOUT | 5s | Maximum time for a batch of records to be prepared and sent to the client. If the batch is not full, it will be sent after the timeout. |
RS_IO_BATCH_RECORD_TIMEOUT | 1s | Maximum time to wait for a record to be added to a batch. If the record is not added, the unfinished batch will be sent to the client. |
RS_IO_OPERATION_TIMEOUT | 60s | Maximum time for an I/O operation (e.g., read or write). If the operation takes longer, it will be aborted. |
Most of the I/O settings are related to batching and specify the size of the batch, the number of records in the batch, and the timeout for preparing and sending the batch on the server side.
However, if a client is using the HTTP/1.1 protocol, the RS_IO_BATCH_MAX_METADATA_SIZE and RS_IO_BATCH_MAX_RECORDS settings are used to limit the size of the headers in the HTTP request from the client.
This means that the client can't send more records in a single batch than specified by the RS_IO_BATCH_MAX_RECORDS and the size of the headers in the request can't exceed the value of the RS_IO_BATCH_MAX_METADATA_SIZE setting.
Read more about batching in the HTTP API Reference.
Lock File Settings
ReductStore uses a lock file to prevent multiple instances from accessing the same data folder. The lock file is created in the data folder specified by the RS_DATA_PATH variable, using the same storage backend as the data. You can configure the lock file settings using the following environment variables:
| Name | Default | Description |
|---|---|---|
RS_LOCK_FILE_ENABLED | false | If set to false, the storage will not use a lock file. |
RS_LOCK_FILE_TIMEOUT | 10s | Timeout for acquiring the lock file. If the lock file is not acquired within the timeout, the storage will exit. When set to 0, the storage will wait indefinitely. |
RS_LOCK_FILE_FAILURE_ACTION | abort | Action to take if the lock file can't be acquired. It can be abort or proceed. If set to abort, the storage will exit. If set to proceed, the storage will overwrite the lock file and continue. |
Enable the lock file if using a shared data folder or remote backend to prevent data corruption in blue-green deployments or high availability setups.
Replication Settings
ReductStore supports data replication between different instances. You can configure replication settings using the following environment variables:
| Name | Default | Description |
|---|---|---|
RS_REPLICATION_TIMEOUT | 5s | Timeout for attempts to reconnect to the target server in seconds. |
RS_REPLICATION_LOG_SIZE | 1000000 | Maximum number of pending records in the replication log. The oldest records are overwritten when the limit is reached. |
Provisioning
ReductStore provides an HTTP API to create and configure resources such as buckets, access tokens or replication tasks. However, if you are following an Infrastructure as Code (IaC) approach, you may want to provision resources at the deployment stage and ensure that they can't be modified or deleted using the HTTP API.
Example
Here is an example of provisioning two buckets and a token to access them:
RS_BUCKET_A_NAME=bucket-1
RS_BUCKET_A_QUOTA_TYPE=FIFO
RS_BUCKET_A_QUOTA_SIZE=1Tb
RS_BUCKET_B_NAME=bucket-2
RS_TOKEN_A_NAME=token
RS_TOKEN_A_VALUE=somesecret
RS_TOKEN_A_READ=bucket-1,bucket-2
As you can see, each resource has a type and an ID. The type is BUCKET for buckets and TOKEN for tokens. The ID is a unique identifier for the resource. You can use any string value for the ID. It is only used to group resources of the same type.
Below you will find the available settings for each resource type.
Bucket Provisioning
You can provision buckets by setting environment variables. The following table lists the available settings:
| Name | Default | Description |
|---|---|---|
RS_BUCKET_<ID>_NAME | Provisioned bucket name (required) | |
RS_BUCKET_<ID>_QUOTA_TYPE | NONE | It can be NONE, FIFO or HARD. |
RS_BUCKET_<ID>_QUOTA_SIZE | 0 | Size of quota to start removing old data e.g. 1 KB, 10.4 MB etc. |
RS_BUCKET_<ID>_MAX_BLOCK_SIZE | 64Mb | Maximal block size for batched records |
RS_BUCKET_<ID>_MAX_BLOCK_RECORDS | 1024 | Maximal number of batched records in a block |
For more information about the bucket settings, see the Buckets Guide.
Token Provisioning
You can provision tokens by setting environment variables. The following table lists the available settings:
| Name | Default | Description |
|---|---|---|
RS_TOKEN_<ID>_NAME | Provisioned token name (required) | |
RS_TOKEN_<ID>_VALUE | Provisioned value of token (required) | |
RS_TOKEN_<ID>_FULL_ACCESS | false | Full access permission |
RS_TOKEN_<ID>_READ | List of buckets for reading separate by a comma. | |
RS_TOKEN_<ID>_WRITE | List of buckets for writing separate by a comma. |
For more information about the token settings, see the Access Control Guide.
Replication Provisioning
You can provision replication tasks by setting environment variables. The following table lists the available settings:
| Name | Default | Description |
|---|---|---|
RS_REPLICATION_<ID>_NAME | Provisioned replication task name (required) | |
RS_REPLICATION_<ID>_SRC_BUCKET | Source bucket name (required) | |
RS_REPLICATION_<ID>_DST_BUCKET | Remote bucket name (required). It must be created before replication | |
RS_REPLICATION_<ID>_DST_HOST | URL of destination instance (required). For example, https://play.reduct.store or http://localhost:8383 | |
RS_REPLICATION_<ID>_DST_TOKEN | Token for destination instance | |
RS_REPLICATION_<ID>_ENTRIES | List of entries to replicate. Separate entries with a comma. If the list is empty, all entries will be replicated. | |
RS_REPLICATION_<ID>_WHEN | Condition for replication. If not set, all entries will be replicated. |
For more information about the replication settings, see the Data Replication Guide.