Load and stress tests
TruBudget uses load and stress tests to ensure the system's stability. Some tests are triggered automatically after a major release, others run on each push to the remote repository or get triggered manually. The automated testing strategy allows to regularly check the system minimal requirements and adjust them as needed. Furthermore, developer get detailed information about the system allowing to search for flaws i.e. bottlenecks.
There are three types of load/stress tests:
- Stress test: Classic stress to bring the core part of the system (API/Blockchain) to their limits. Although the system capacities are entirely used, the system should not crash nor be unusable
- Smoke Tests: Smoke test are regular load-test, configured for minimal load. They are used for sanity checks.
- Stability Test: The whole system and it's interconnecting services such as Email Service and Storage Service are tested. This test, in terms of requests and users per second, is set up similarly to the stress test.
TruBudget uses k6 to define and run load and stability tests. Tests can be run both locally and using Docker. We highly recommend using docker since the provided script spins up a complete test setup including Grafana and Influx.
However, if you choose to run the tests locally, k6 needs to be installed. An optional Grafana and Influx instance are not required, but recommended in order to analyze the data generated by the tests.
Per default, the test script will run a smoke test against your local TruBudget instance. However, if you want to test a remote instance, the target URL can be changed in api/loadtest/k6-tests/script.js by modifying the BASE_URL constant.
The test type can be changed in the provided script (api/loadtest/runTests.sh) by modifying the --config parameter. Valid options are:
smoke.config.json: runs a smoke-test (1 virtual user)load.config.json: runs a load-test (100 virtual user)stress.config.json: runs a stress-test (400 virtual user)
Getting started
- Navigate to
api/loadtest - Configure the test as described above and add execution rights to
runTests.sh - Run
./runTests.sh
Visualizing
To visualize the generated data from the tests via Grafana:
- Open Grafana
- Navigate to Dashboard
- In the left sidebar, click the + button > Import
- Paste the Dashboard ID: 2587 or 11837
- Click Load
- Select the Influx data-source
- Click Import
Grafana has a bunch of pre-configured dashboards for k6 we recommend using one of the two predefined dashboards (2587 or 11837) mentioned above. However, if you want to browse the available dashboards, you can do it here. Also, feel free to contribute to the Grafana community by creating your own dashboard and publishing it to the Grafana Hub.
Minimum requirements
TruBudget can be deployed in different ways. To give your users the best experience, we tested TruBudget with different configurations. The minimum requirements aim to provide a working environment with 50 to 100 parallel user. The recommended environment is capable of handling 100 - 400 parallel user (using Kubernetes).
If you plan to deploy TruBudget on a single VM with following services: Blockchain, API and Frontend, we recommend the following for minimum setup:
| Minimum | |
|---|---|
| CPU† | 2 vCPU |
| RAM | 8 GB |
| Storage | 10 GB |
If you plan to have more than 100 parallel users, we strongly recommend using Kubernetes. TruBudget load and stress tests are run against the minimum and recommend system requirements in a Kubernetes environment to ensure the correctness of the data below.
Blockchain:
| Minimum | Recommended | |
|---|---|---|
| CPU† | 1 vCPU | 2 vCPU |
| RAM | 4 GB | 8 GB |
| Storage | 1 GB | 1* GB |
Frontend:
| Minimum | Recommended | |
|---|---|---|
| CPU† | 0.5 vCPU | 1 vCPU |
| RAM | 1 GB | 2 GB |
| Storage | 1 GB | 1 GB |
API:
| Minimum | Recommended | |
|---|---|---|
| CPU† | 0.5 vCPU | 1 vCPU |
| RAM | 2 GB | 8 GB |
| Storage | 1 GB | 1 GB |
Export-Service:
| Minimum | Recommended | |
|---|---|---|
| CPU† | 0.5 vCPU | 0.5 vCPU |
| RAM | 1 GB | 1 GB |
| Storage | 1 GB | 1 GB |
Email-Service:
| Minimum | Recommended | |
|---|---|---|
| CPU† | 0.5 vCPU | 0.5 vCPU |
| RAM | 1 GB | 1 GB |
| Storage | 1 GB | 1 GB |
Storage-Service:
| Minimum | Recommended | |
|---|---|---|
| CPU† | 1 vCPU | 2 vCPU |
| RAM | 1 GB | 2 GB |
| Storage | 1 GB | 50* GB |
† Depending on your or your provider's infrastructure, the strength of one virtual CPU might vary. The above values are based on the Intel® Xeon® Platinum 8272CL processor (second generation Intel® Xeon® Scalable processors), Intel® Xeon® 8171M 2.1GHz (Skylake), Intel® Xeon® E5-2673 v4 2.3 GHz (Broadwell), or the Intel® Xeon® E5-2673 v3 2.4 GHz (Haswell) processors.
* Assuming you are using off-chain-storage.