×

☰ Table of Contents

TigerGraph Docs : GBAR - Graph Backup and Restore v1.1

Backup and Restore

Updated:

Introduction and Syntax

GBAR (Graph Backup And Restore), is an integrated tool for backing up and restoring the data and data dictionary (schema, loading jobs, and queries) of a single TigerGraph node. In Backup mode, it packs TigerGraph data and configuration information in a single file onto disk or a remote AWS S3 bucket. Multiple backup files can be archived. Later, you can use the Restore mode to rollback the system to any backup point. This tool can also be integrated easily with Linux cron to perform periodic backup jobs.

The current version of GBAR is intended for restoring the same machine that was backed up. For help with cloning a database (i.e., backing up machine A and restoring the database to machine B), please contact support@tigergraph.com .



Synopsis
Usage: gbar backup [options] -t <backup_tag> gbar restore [options] <backup_tag> gbar config gbar list Options: -h, --help Show this help message and exit -v Run with debug info dumped -vv Run with verbose debug info dumped -y Run without prompt -t BACKUP_TAG Tag for backup file, required on backup


The -y option forces GBAR to skip interactive prompt questions by selecting the default answer. There are currently five situations for prompts:

  • During backup, if GBAR calculates there is insufficient disk space to copy and then compress the graph data, it will ask: Do you want to continue?(y/N). The default answer is no.
  • At the start of restore, GBAR will always asks if it is okay to stop and reset the TigerGraph services: (y/N)? The default answer is yes.
  • During restore, if user does not provide the backup_tag with a full backup file name in command line, and there are multiple files matching that tag, it by default choose the latest, and will ask: Do you want to continue?(y/N) The default answer is yes.
  • During restore, if GBAR calculates there is insufficient disk space to copy the current graph data and then uncompress the archived data, it will ask: Do you want to continue?(y/N). The default answer is no.
  • After restore, old gstore data will be left on disk. GBAR needs your confirmation to remove it, and will ask: Do you want to continue removing it?(y/N). The default answer is no.

Config

gbar config

GBAR Config must be run before using GBAR backup/restore functionality. GBAR Config will open the following configuration template interactively in a text editor. Using the comments as a guide, edit the configuration file to set the configuration parameters according to your own needs.

Synopsis
# Configure file for GBAR # you can specify storage method as either local or s3, or both # Assign True if you want to store backup files on local disk # Assign False otherwise, in this case no need to set path store_local: False path: PATH_TO_BACKUP_REPOSITORY # Assign True if you want to store backup files on AWS S3 # Assign False otherwise, in this case no need to set AWS key and bucket store_s3: False aws_access_key_id: YOUR_ACCESS_KEY aws_secret_access_key: YOUR_SECRET_KEY bucket: YOUR_BUCKET_NAME # The maximum timeout value to wait for core modules(GPE/GSE) on backup. # As a roughly estimated number, # GPE & GSE backup throughoutput is about 2GB in one minute on HDD. # You can set this value according to your gstore size. # Interval string could be with format 1h2m3s, means 1 hour 2 minutes 3 seconds, # or 200m means 200 minutes. # You can set to 0 for endless waiting. backup_core_timeout: 5h

Backup

gbar backup -t <backup_tag>

The backup_tag acts like a filename prefix for the archive filename. The full name of the backup archive will be <backup_tag>-<timestamp>.tgz.

GBAR Backup performs a live backup, meaning that normal operations may continue while backup is in progress. When GBAR backup starts, it sends a request to GADMIN, which then requests the GPE and GSE to create snapshots of their data. Per the request, the GPE and GSE store their data under GBAR’s own working directory. GBAR also directly contacts the Dictionary and obtains a dump of its system configuration information. Besides, GBAR records TigerGraph system version. Then, GBAR compresses all the data and configuration information into a single file named <backup_tag>-<timestamp>.tgz. As the last step, GBAR copies that file to local storage or AWS S3, according to the Config settings, and removes all temporary files generated during backup.

The current version of GBAR Backup takes snapshots quickly to make it very likely that all the components (GPE, GSE, and Dictionary) are in a consistent state, but it does not fully guarantee consistency. It’s highly recommended when issuing the backup command, no active data update is in progress. A no-write time period of about 5 seconds is sufficient.

Backup does not save input message queues for REST++ or Kafka.

Restore

gbar restore <backup_tag>

Restore is an offline operation, requiring the data services to be temporarily shut down. The backup tag acts as a filename prefix. During restore, the user can provide either the tag (filename prefix) or the full filename with timestamp information in the name. When GBAR restore begins, it first searches for a backup file matching the backup_tag supplied in the command line. If multiple matching backup archives are found , GBAR will select the most recent one, and ask the user for confirmation to continue. Then it decompresses the backup file to a working directory. As the next step, GBAR will compare the Tigergraph system version in the backup archive with the current system's version, to make sure that backup archive is compatible with that current system. It will then shut down the TigerGraph servers (GSE, RESTPP, etc.) temporarily. Then, GBAR makes a copy of the current graph data, as a precaution. If GBAR estimates that there is not sufficient disk space for the copy, GBAR will display a warning and prompt the user to abort (unless the user has overridden the prompt with the -y option). Next, GBAR copies the backup graph data into the GPE and GSE and notifies the Dictionary to load the configuration data. When these actions are all done, GBAR will restart the TigerGraph servers.

The primary purpose of GBAR is to save snapshots of the data configuraton of a TigerGraph system, so that in the future the same system can be rolled back (restored) to one of the saved states.  A key assumption is that Backup and Restore are performed on the same machine, and that the file structure of the TigerGraph software has not changed. Specific requirements are listed below.

Restore Requirements and Limitations


Restore is supported if the TigerGraph system has had only minor version updates since the backup.

  • TigerGraph version numbers have the format X.Y[.Z], where X is the major version number and Y is the minor version number.
  • Restore is supported if the backup archive and the current system have the same major version number AND the current system has a minor version number that is greater than or equal to the backup archive minor version number.
  • Backup archives from a 0.8.x system cannot be Restored to a 1.x system.
  • Examples:

    Backup archive's system version current system version Restore is allowed?
    0.8 1.0 NO - Major versions differ
    1.1 1.1 YES - Major and minor versions are the same
    1.1 1.2 YES - Major versions are the same; current minor version > archived minor version
    1.1 1.0 NO - Major versions are the same; current minor version < archived minor version


Restore needs enough free space to accommodate both the old gstore and the gstore to be restored.

After restore, old gstore data will be left on disk by default. To remove the old data, either answer "Y" when Restore asks you, or remove it yourself after restore has completed and the system is running again.

List Backup Files

gbar list

This command lists all generated backup files in the storage place configured by the user. For each file, it shows the file’s full tag, file’s size in human readable format, and its creation time.

GBAR Detailed Example

The following example describes a real example, to show the actual commands, the expected output, and the amount of time and disk space used, for a given set of graph data. For this example, and Amazon EC2 instance was used, with the follwing specifications:

single instance with 32 CPU + 244GB memory + 2TB HDD.

Naturally, backup and restore time will vary depending on the hardware used.

GBAR Backup Operational Details

The flowchart below shows how GBAR processes a backup request.

To run a daily backup, we tell GBAR to backup with the tag name daily .

$ gbar backup -t daily
[SUMMARY] Retrieve TigerGraph system configuration...
[SUMMARY] Check TigerGraph system status...
[SUMMARY] Get TigerGraph version as 1.0
[SUMMARY] Issued snapshot command to GPE/GSE
[SUMMARY] Wait for GPE/GSE snapshot done...
[SUMMARY] GPE/GSE snapshot done in 37m11s
[SUMMARY] Backup DICT...
[SUMMARY] Compress backup data to daily-20171206031441.tgz...
[SUMMARY] Compress data done in 39m38s
[SUMMARY] Clean intermediate files...
[SUMMARY] Backup file daily-20171206031441.tgz size 64.4GB
[SUMMARY] Copy daily-20171206031441.tgz to local storage /home/tigergraph/backups...
[SUMMARY] Copy finished in 10m31s
Backup done in 1h37m43s.

The total backup process took about 1 hour and a half, and the generated archive is about 64GB. Dumping the dump GPE/GSE data to disk took 37 minutes. Compressing the files to a single portable backup archive took another 40 minutes.

GBAR Restore Operational Details

This flowchart shows GBAR runs a restore job.



To restore from a backup archive, tell GBAR the backup tag ( daily ).GBAR will choose the latest one by default. To select a specific archive to restore, provide GBAR with a full archive name, such as daily-20171206031441 . By default, restore will ask the user to approve at least two actions. If you want to pre-approve these actions, use the "-y" option. GBAR will make the default choice for you.

$ gbar restore daily
[SUMMARY] Retrieve TigerGraph system configuration...
GBAR restore needs to reset TigerGraph system.
Do you want to continue?(y/N):y
[SUMMARY] Multiple backup points found for tag daily, will pick up the latest.
Will restore from the latest one daily-20171206031441.
Do you want to continue?(y/N):y
[SUMMARY] Restore to latest one daily-20171206031441
[SUMMARY] Backup file daily-20171206031441.tgz size 64.4GB
[SUMMARY] Copy daily-20171206031441.tgz to GBAR work dir...
[SUMMARY] Copy finished in 4m13s
[SUMMARY] Decompress daily-20171206031441.tgz with size 64.4GB...
[SUMMARY] Decompress done in 13m23s
[SUMMARY] Backup data with version 1.0 applicable to 1.0 system.
[SUMMARY] Stop TigerGraph system...
[SUMMARY] Start GDICT...
[SUMMARY] Move aside old GPE data...
[SUMMARY] Move aside old GSE data...
[SUMMARY] Snapshot old GDICT data...
[SUMMARY] Restore GPE data...
[SUMMARY] Restore IDS data...
[SUMMARY] Restore DICT...
[SUMMARY] Reset TigerGraph system...
[SUMMARY] Start TigerGraph system...
[SUMMARY] Reinstall all GSQL query...
[SUMMARY] Recompile all loading job...
[SUMMARY] Running post restore jobs...
Restore done in 21m33s.
GPE/GSE old data still saved on disk, you can remove it after TigerGraph system stable, or remove it right now.
Do you want to continue removing it?(y/N):n
GPE/GSE old data saved as /home/tigergraph/tigergraph/gstore/0/part-20171206032413 and /home/tigergraph/tigergraph/gstore/0/<part_id>/ids-20171206032413, you need to remove them manually.

For our test,  GBAR spent about 20 minutes to finish the restore job. Most of the time (13 minutes) was spent decompressing the backup archive.

Note that after the restore is done, GBAR prompts you to make a choice whether to remove old data. Here we choose no, in that case, we will need to remove the old gstore files manually after, say, we verify that the restored system is functioning correctly.

Performance Summary

GStore size Backup file size Backup time Restore time
278GB 64GB 1.5 hour 20 mins