Skip to main content

Setup

This document describes how to set up your environment to start developing and debugging the TruBudget application. If you want to contribute to TruBudget please check out the guidelines in the CONTRIBUTING.md.

Table of Contents#

Software components#

Working on Windows#

If you are using a Windows operating system, we suggest using the Windows Subsystem for Linux, for the installation please read this guide. Because our setup was tested on WSL 1 and not on WSL 2, we suggest using WSL 1. However, if you want to use WSL 2, you can try by following these steps.

Docker/Docker-Compose#

The easiest way to setup the application on your machine is via Docker and Docker-Compose. To install these components, please follow the official documentation:

Node.js / npm#

Follow the official instructions on how to setup node.js/npm.

TypeScript#

If you are using a global installation of TypeScript, please make sure you have a version starting from 4.0.2

Project Setup#

Clone Repository#

Clone the repository from Github:

  • SSH:

Follow the instructions on how to setup your SSH-connection

git clone https://github.com/openkfw/TruBudget.git
  • HTTPS:
git clone https://github.com/openkfw/TruBudget.git

### IDE

For coding, we recommend the use of Visual Studio Code, which you can find here.

To ensure that the code is formatted properly we recommend following extensions for VS Code:

- aaron-bond.better-comments

- CoenraadS.bracket-pair-colorizer

- dbaeumer.vscode-eslint †

- EditorConfig.EditorConfig

- esbenp.prettier-vscode

- pmneo.tsimporter

- rbbit.typescript-hero

- xabikos.JavaScriptSnippets

- esbenp.prettier-vscode

note

† Have a look here to learn more about how to set up es-lint for validating TypeScript in VSCode

Chrome Developer Tools#

If you are testing and debugging in Google Chrome, we recommend the following extensions:

Git-Secrets#

awslabs/git-secrets is a tool to scan a repository for secrets, credentials or any unwanted text. We recommend to install git-secrets and setup git-hooks to prevent committing secrets. The setup-script(.githooks/setupGitSecrets.sh) edits your local git-config located in .git. Changes made by setupGitSecrets.sh:

  • change hooks-path to .githooks
  • adds git-secrets-patterns listed in .githooks/git-secrets-patterns

How to install git-secrets?#

Follow the instructions here or use homebrew

brew install git-secrets

How to setup?#

Setup git-secrets by execute the shell script in the .githooks folder

cd .githooksbash setupGitSecrets.sh

Environment variables#

All projects in TruBudget (blockchain, api, frontend, etc) have a docker-compose file that can be used to start the project with. In order to start the projects, some environment variables must be set. In order to make this easier, there are some files containing the environment variables called .env_example in each project directory. To use these environemnt variables, simply copy the .env.example file and rename it to .env.

caution

Do NOT use other additional ways to set environment variables like inline env-variables or env-variables defined by 'export'. Why? - Because these env-variables will overwrite each which makes it very hard to find mistakes.

Development Setup#

If you want to start developing on Trubudget, you need to setup the application locally. This guide tells you how to start the blockchain, start the API, load up some test data and start the frontend.

Dockerized Application (recommended)#

This is the fastest way you can run all services needed for development. Everything is started in one command:

  1. in root directory execute: bash scripts/development/start-dev.sh --slim or bash scripts/development/start-dev.sh --full

This script sets the .env - file in the development directory automatically. No further steps needed.

Following services are dockerized with hot-reloading:

  • Blockchain (alpha node)
  • API (alpha API)
  • Provisioning (feeds application with dummy data)
  • Frontend
  • Excel export
  • Email notification
  • Email Database
  • Storage Service
  • Minio Database (external document storage)

If the script is started with --slim, only the Blockchain, API, Provisioning and Frontend will be started. For further details how to use the script, print out its help section with bash scripts/development/start-dev.sh --help.

It takes some time to build and run at the first launch. After that, source codes of all services are live reloaded. That means any change in ./src folder is reflected in respective container automatically.

The frontend should be availaible as usual at http://localhost:3000

Docker Compose ensures that services are communicating and have correct environment variables set. Docker Compose puts all services in the same network and exposes needed ports.

You can inspect each container individually: docker logs --follow CONTAINER. Where CONTAINER represents name of the service that can be found in the .yml file.

note

Hot reloading with docker requires a lot of processing power (due to docker volumes). If you experience huge delays while hot-reloading, you should start the services you need by yourself without docker.

TruBudget Services#

Blockchain#

The blockchain works as data layer for the Trubudget application. We suggest reading the README.md file in the /blockchain to have a better idea on how the blockchain layer works in TruBudget. Also, take a look at the multichain website to see how the multichain works.

We start by creating an instance of the blockchain.

First, navigate to the /blockchain folder, install all the npm packages and then start the blockchain via a shellscript:

  1. Navigate to the ./blockchain folder and start your favourite shell.
cd blockchain

  1. Start docker if it is not running already

  2. Install node-modules

npm install
  1. Create a blockchain instance
bash startDev.sh
Developing on Windows:#

If you are developing on Windows, we recommend using the Git Bash to run the startDev.sh script.

API#

The API takes care of the communication between the blockchain and the frontend and basically serves as backend. The API is developed using Node.js, with Fastify, Axios, Joi The proper setting of environment variables is important so the API can communicate with the blockchain correctly. Details regarding the environment variables can be found here.

The API is structured into layers (application layer, service layer, domain layer) and each of these layers has its own language. There is a README file in each of the layers, where the structure of the API and the current layer is described in detail. You can start by checking out this link to get more details.

To start the api open your favorite shell, navigate to your api folder and follow these instructions:

cd ../api
  1. Set environment variables
  • Rename the .env_example file to .env and set following environment variables accordingly.
ORGANIZATION=ACMECorpMULTICHAIN_RPC_HOST=127.0.0.1MULTICHAIN_RPC_PORT=8000PORT=8080ROOT_SECRET='root-secret'ORGANIZATION_VAULT_SECRET="secret"MULTICHAIN_RPC_PASSWORD=s750SiJnj50yIrmwxPnEdSzpfGlTAHzhaUwgqKeb0G1j
  1. Install node-modules
  • (Linux/Mac) Depending on your machine configuration, it might be necessary to install autoconf and automake
brew install autoconfbrew install automake
  • Terminal Mac/Git Bash
npm install
  1. Start the api

On Linux/Mac:

npm run dev

On Windows: Unfortunately hot reloading using nodemon is currently not working properly on Windows. It's therefore recommended to open two open 2 separate terminal windows. Navigate to the api folder and set the environment variables as described above. Make sure that the above environment variables are set in both terminal windows. In the first terminal window run

tsc --watch

In the second window run

npm install -g nodemonnodemon dist

to start the api.

The API-Documentation should then be available at http://localhost:8080/api/documentation/index.html

Using Postman#

One tool that can be used for REST calls is called Postman. There is a collection of API calls ready to be imported into Postman.

Frontend#

After initializing the blockchain and api, we can now start the frontend to visualize the data. For developing the frontend we use React, along with the Redux-Saga library and the Material-UI framework. You can check out the README.md for more details about the frontend.

Navigate to the frontend folder:

cd ../frontend
  1. Set environment variables
  • Rename the .env_example file to .env and set following environment variables accordingly.
PORT=3000

Additionally you can connect the frontend to the optional excel export service, by adding the environment variables mentioned in the excel-export section of the README.md

  1. Install node-modules
npm install
  1. Start the frontend
npm start

The frontend should then be availaible at http://localhost:3000

caution

If you change the port of the api you may have to consider to change the proxy port in the package.json accordingly.

note

You do not need to run every project separately if you are developing on a single one. Just do following:

  1. go to a desired folder (e.g. /api)
  2. copy .env.example file and rename it to .env
  3. run bash startDev.sh in the folder to start dependent project(s)

Provisioning (Optional)#

This part will initiate the blockchain with test user and project data. The environment variables for this step differ slightly from the ones for the API, so please set them accordingly. You can skip this part if you don't want to have initial test data on your blockchain. Check out the README.md for more details.

cd ../provisioning
  1. Set environment variables
  • Rename the .env_example file to .env and set following environment variables accordingly.
API_PORT=8080ORGANIZATION=ACMECorpROOT_SECRET='root-secret'
  1. Install node-modules
npm install
  1. Start the provisioning
npm start

Excel-Export (Optional)#

There is a service that exports TruBudget data into an Excel sheet. For exporting we use the npm package exceljs. The service is a node package and needs to be started separately. More details regarding the excel-export service can be found in the README.md.

To start the service locally, follow these commands:

cd ../excel-export-service
  1. Set environment variables
  • Rename the .env_example file to .env and set following environment variables accordingly.
API_HOST="localhost"API_PORT=8080PORT=8888
  1. Install node-modules
npm install
  1. Start the service
npm start

The service is then available either on the host and port set by the environment variable or localhost:8888 by default.

note

in order to access the excel export service from the UI, you should start the frontend with some additional environment variables as mentioned in the excel-export section of the frontend README.md file.

note

Another way to start the excel export service is in a docker container. Using the docker-compose file in the excel-export folder will build the whole application including api, blockchain, frontend and excel-export service at the same time.

Email-Notification (Optional)#

The email notification service is responsible for saving/deleting email adresses per Trubudget user in a connected database. These email addresses are used to send configurable notifications to a connected SMTP server.

If you want to start this service or simply see more details regarding this feature you can check out the README.md file.

Storage-Service (Optional)#

The storage service is responsible for saving/accessing documents to Minio, an external storage server.

More details and how to enable the storage service with an external storage can be found in README.md file.

Tests#

End-to-end Tests#

Before checking in, you should always run the end-to-end test which explores / tests the whole functionality of the application. For end-to-end testing we use the testing framework Cypress. If you want to start all e2e-tests to check if your changes are not breaking any stuff we recommend the Docker Compose Setup. More details regarding the environment variables can be found in the README file.

Prerequisits#

Before running the tests you should make sure that the application is started (including the excel export service if needed) and that you first run the provisioning. In order for the backup_spec tests to pass you should also start the project with the same configurations (Organization and RPC Password). When running the e2e-tests locally, you have to make sure that the password used for authentication as root user in the test matches the one used in the project that is currently running, otherwise some tests can fail because of an "Authentication failed" error.

Setup#

note

If you are using WSL on Windows check out this setup to run cypress in WSL.

To setup a new TruBudget instance with Docker-Compose, run following command (recommended):

bash scripts/development/start-dev.sh --slim

If you do not want to use Docker, you can setup all service by yourself with npm start.

cd e2e-test

In the e2e-test folder you can run the following commands:

Before you run cypress, you need to specify the environment variables (such as Organization, URLs of fronted, api, excel export, email service) in the .env* file. If you have not .env* file, create one and copy the content from /e2e-test/.env_example* to /e2e-test/.env*. For more information see Environment variables

npm run cypress
or
npm run e2etest

or through the Cypress frontend under settings.

For further information see the README.md of /e2e-test

note

The .env file is needed.

Unit Tests#

Before checking in, you should always create unit tests for the implemented part. For unit testing we use the testing framework Mocha with Chai as assertion library.

Following command executes all spec.js files in the src folder:

cd apinpm run test

Following command executes all spec.js files in a specific folder (e.g. project) located in the src folder:

cd apinpm run test:here <folderName>

Following command executes a specific test group (e.g. "Updating a project") defined within spec.js files in a specific folder (e.g. project) located in the src folder. The test group is defined with the describe function of mocha.

cd apinpm run test:here <folderName> <describeString>

CI/CD#

The CI/CD is described here

Links#

Here you can find a summary of all the technologies used in TruBudget with links to their websites and documentation

Blockchain#

API#

Frontend#

Provisioning#

Excel-Export#

E2E-Tests#

Unit-Tests#