Server migration

🚧

Pre-requisite: CLI installed and ready to use

This guide assumes that you have installed and used the Budibase CLI before, if you have not then you can follow our guide on installing the CLI here.

From within the Budibase builder it is possible to export and import individual apps, however this really acts as a template of that app for other Budibase systems, rather than a full backup.

To properly backup a Budibase installation you need to be able to take a snapshot of everything within the self hosted install, the users, apps, permissions - everything. The Budibase CLI has a backups feature which can be used to do this, and it can be useful in two ways:

1. Performing a full backup of your self hosted install incase anything goes wrong, from this backup you can completely re-create the state of the system at that time by importing it onto a new Budibase installation.
2. Moving a Budibase installation from one system to another - while it is possible to do this by exporting and importing the volumes used by Budibase our CLI is agnostic of the underlying hosting method, whether its hosted with Kubernetes, Docker or Docker Compose, the CLI can be used to produce a full export of the installation - this can then be used to replicate your old installation on a new one.

Setting up

The first step to exporting via the CLI is informing it about all of your installations environment variables. The CLI will be connecting directly to your CouchDB and MinIO/S3 compatible object store services to extract the persistent data directly, to do this the CLI needs access to your environments access keys. There are three ways in which we can configure the CLI:

1. If you have used Docker Compose to install Budibase this is quite simple, you can use the .env file that is utilised by your composed installation to run the CLI. Please note if you are in a Windows/OSX environment dot files may be hidden.
2. If this is a one time operation then you can simply run the export/import commands in the CLI, if no environment variables are found it will ask a series of questions to determine the access keys it needs.

You can also build an environment file for use with the CLI for backups, it should look something like the following:

COUCH_DB_URL=http://<username>:<password>@localhost:10000/db/
MINIO_ACCESS_KEY=<object store access key>
MINIO_SECRET_KEY=<object store secret key>
MINIO_URL=http://localhost:10000
MAIN_PORT=10000

Exporting

The first step to creating a backup or moving your installation is the performing the export, this can be done with the following command:

budi backups --export --env <environment file>

If you do not have an environment file then you can provide the required access keys via questions asked in the CLI, this will look like:

budi backups --export
*** NOTE: use a .env file to load these parameters repeatedly ***
? MAIN_PORT 10000
? COUCH_DB_URL http://<username>:<password>@localhost:10000/db/
? MINIO_URL http://localhost:10000/
? MINIO_ACCESS_KEY <object store access key>
? MINIO_SECRET_KEY <object store secret key>
CouchDB Export
 ████████████████████████████████████████ 100% | ETA: 0s | 9/9
S3 Export
 ████████████████████████████████████████ 100% | ETA: 0s | 14/14
Generated export file - backup-2022-09-01T18:42:25.104Z.tar.gz

If you provide an environment file then the question step will be skipped and it will occur automatically, this can be useful for setting up a backup in an automated fashion. You can also provide a filename to export to (if you wanted to always have a latest file) via the command budi backups --export filename.tar.gz.

Importing

The import command is nearly identical to the export command, except it requires a filename be passed to the import argument to specify which backup to import. The command should be used in the following fashion:

budi backups --import <filename> --env <environment file>

The process for importing a backup generated by Budibase is very simple, it works in essentially the same way that the export process works, except you use environment variables for the environment you wish to import to. We currently only recommend one method for import, to a brand new installation. To do this follow these steps:

1. Make sure you have an identical environment variable file for your new environment.

• Docker Compose - make sure to copy the .env file from your old installation to the new one before creating it.
• Docker single image - either use Docker Compose to inject the environment variables, or make sure the same .env file that was accessible in the single images mounted directory is shared into the new installation.
• Kubernetes - make sure an identical chart is used to create the new installation.

2. Once your new installation is started, make sure your CLI environment file is correct for the new installation.
3. Finally, run the import command and wait for import to complete.

As with the export command you can run the import without an environment file, you will be asked the same set of questions to determine which installation to import to, this appears as follows:

budi backups --import <filename> 
*** NOTE: use a .env file to load these parameters repeatedly ***
? MAIN_PORT 10000
? COUCH_DB_URL http://<username>:<password>@localhost:10000/db/
? MINIO_URL http://localhost:10000/
? MINIO_ACCESS_KEY <object store access key>
? MINIO_SECRET_KEY <object store secret key>
CouchDB Import
 ████████████████████████████████████████ 100% | ETA: 0s | 9/9
MinIO Import
 ████████████████████████████████████████ 100% | ETA: 0s | 5/5
Import complete

❗

Importing to an existing installation

While it is possible to import to an existing environment, its likely errors will occur if any information is duplicated - e.g. the same user already exists within it. In some situations the CLI will be able to resolve this, in others it will cause issues. If you wish to revert an install we recommend taking a backup, clearing down the installation and then importing the backup.

Notes

Please note that the CLI may take some time to run exports/imports if you've utilised internal tables with very large amounts of data (say 100K+ rows). Simply leave the CLI and let it run, it will eventually perform the export. This is not an intensive operation for your Budibase installation, it can be performed live.

If you have utilised any external datasources, for example Postgres, MySQL, MongoDB, DynamoDB etc - then you will need to backup these independently from Budibase. At the moment we cannot perform backups for anything stored externally to Budibase so please follow regular backup practice for all critical datasources.

When importing to a new installation, if any external access is utilised, e.g. datasources, automation actions, bash scripts etc - you will need to confirm that the new environment has access to all the same resources. The Budibase CLI will only backup the state of your CouchDB installation and your object store, if you have made changes to the containers or provided specific network access to some resources, you will need to make these same changes for the new installation.