For any project, the data you store and use plays a key role. The safety and availability of the data are also the same. There may be events of natural disaster or man-made errors that make data inaccessible, thus disturbance in the availability of application to the end-user. Therefore we need a solution to backup data in a safe place and restore it whenever the need arises.
This article provides a solution to automate the entire import/export process using a shell script. The user just has to provide the project ID(s) to perform actions.
What are Google Cloud Firebase and Firestore?
Firebase is Google’s mobile application development platform that helps you build, improve, and grow your app. It provides developers with a variety of tools and services to help them develop quality apps, grow their user base, and earn profit. It is built on Google’s infrastructure.
Cloud Firestore is a scalable NoSQL database to store and sync data for client-and-server-side development.
Firestore comes with a managed export and import service to recover your data from accidental deletion and to export data for offline processing. Though this is a managed service to export/import the user has to perform all the commands with more inputs.
Firstore Export/Import Operations using Automation Script
Engineer’s Note: Here Export and Import are nothing but Backup and Restore.
Before going into the actual script, take a look at the Prerequisites:
Feature Flags or Toggles give application developers the ability to change the functionality of an application at runtime without having to compile the code. Using feature flags, developers can enable/disable various functionalities by introducing feature flags in their code and execute statements related to features based on the value of the feature toggle.
Two of the most important advantages of using feature flags are:
- You should install Cloud SDK on the environment where you want to run the script.
- You should also install Firebase CLI.
- Make sure you have created a GCS Bucket of your choice
- Set appropriate Object Life Cycle Role and Retention Policy to the GCS Bucket
There can be 3 possibilities of Backup/Restoring Firestore data.
- Backup only: In Backup only you’ll just Backup the data to a GCS Bucket.
- Restore only: In Restore only you have two options, the first one is to Restore from the Same Project and the second option is Restore from another project.
- Both Backup and Restore: In Both Backup and Restore, we will back up the data to a GCS Bucket of the same project and Restore into the other project.
The user has to choose among these three options after making sure he/she has fulfilled prerequisites.
Engineer’s Note: In each of the following scenarios, there may be multiple Backup folders in GCS Bucket to restore, before performing Restore operations the terminal asks the user to select a particular folder from listed folders, user can press enter if they have to restore the recent folder available.
Backup Only
Terminal prompts users to enter Project ID, then gcloud will be configured with that project ID and perform Firestore Backup operations. Users just have to enter Project ID, other things will be done by the script. You can pre-set the Backup Folder name in GCS Bucket.
Restore Only
We have two cases here
Restore from the same project
Terminal prompts users to enter Project ID, gcloud will be configured with entered Project ID and performs Firestore Restore Operations.
Restore from a different project
As we are doing different projects, the terminal asks the user to enter both Source and Destination Project ID’s.
In this case, we need to give access permissions of GCS Bucket to the service account of the project where you are Restoring into.
Firstly, the terminal sets gcloud to Source Project and permissions are set to GCS Bucket to be able to access from the destination project.
Then the gcloud is set to Destination Project ID and performs Restore operations.
Both Backup and Restore
This is also between different projects, the user has to enter both Source and Destination Project ID’s.
gcloud is set to Source Project ID and performs Backup operations and grants GCS Bucket access permissions to Destination Project then gcloud is set to Destination Project ID performs Restore operations.
Engineer’s Note: Project IDs have to be set correctly. Otherwise, the data would be copied to the wrong project and the entire application may not work properly.
Best Practices for Dealing with the GCS Bucket
It’s better to create a GCS Bucket in the same project of data to back up to avoid confusion.
There can be multiple folders of Backups depending upon the time frame you run the script, to avoid storing up multiple folders, set an appropriate Object LifeCycle Policy that automatically deletes folders after attaining the prescribed age.
There can also be human error that accidentally deletes folders without knowledge, to avoid this set a Retention Policy that retains folders for prescribed time periods.
You can see the status of the performed operations here else you can run this command to view detailed information Google Cloud Firestore operations list.
Wrapping up…
Performing regular Export and Import operations is important for any healthy infrastructure. However, doing this manually leaves room for error and also the possibility of missing an important backup. This script automates the process, giving developers and business owners greater peace of mind.
If you’re like to learn more about how you can leverage Firestore and automation within Google Cloud for your business, book a free consultation with D3V’s certified cloud engineers.