Importing And Exporting Firestore Operations Using Automation Script

Learn the solution to automate the entire import/export process using a shell script.

August 3, 2023 | Google Cloud | Technical |

Importing And Exporting Firestore Operations Using Automation Script

For any project, the data you store and use plays a key role. The safety and availability of the data are also the same. There may be events of natural disaster or man-made errors that make data inaccessible, thus disturbance in the availability of application to the end-user. Therefore we need a solution to backup data in a safe place and restore it whenever the need arises.

This article provides a solution to automate the entire import/export process using a shell script. The user just has to provide the project ID(s) to perform actions.

What are Google Cloud Firebase and Firestore?

Firebase is Google’s mobile application development platform that helps you build, improve, and grow your app. It provides developers with a variety of tools and services to help them develop quality apps, grow their user base, and earn profit. It is built on Google’s infrastructure.

Cloud Firestore is a scalable NoSQL database to store and sync data for client-and-server-side development.

Firestore comes with a managed export and import service to recover your data from accidental deletion and to export data for offline processing. Though this is a managed service to export/import the user has to perform all the commands with more inputs.

Firstore Export/Import Operations using Automation Script

Engineer’s Note: Here Export and Import are nothing but Backup and Restore.

Before going into the actual script, take a look at the Prerequisites:

Feature Flags or Toggles give application developers the ability to change the functionality of an application at runtime without having to compile the code. Using feature flags, developers can enable/disable various functionalities by introducing feature flags in their code and execute statements related to features based on the value of the feature toggle.

Two of the most important advantages of using feature flags are:

 

  • You should install Cloud SDK on the environment where you want to run the script.
  • You should also install Firebase CLI.
  • Make sure you have created a GCS Bucket of your choice
  • Set appropriate Object Life Cycle Role and Retention Policy to the GCS Bucket

 

There can be 3 possibilities of Backup/Restoring Firestore data.

 

  1. Backup only: In Backup only you’ll just Backup the data to a GCS Bucket.
  2. Restore only: In Restore only you have two options, the first one is to Restore from the Same Project and the second option is Restore from another project.
  3. Both Backup and Restore: In Both Backup and Restore, we will back up the data to a GCS Bucket of the same project and Restore into the other project.

 

The user has to choose among these three options after making sure he/she has fulfilled prerequisites.

Engineer’s Note: In each of the following scenarios, there may be multiple Backup folders in GCS Bucket to restore, before performing Restore operations the terminal asks the user to select a particular folder from listed folders, user can press enter if they have to restore the recent folder available.

 

  1. Backup Only

    Terminal prompts users to enter Project ID, then gcloud will be configured with that project ID and perform Firestore Backup operations. Users just have to enter Project ID, other things will be done by the script. You can pre-set the Backup Folder name in GCS Bucket.

  2. Restore Only

    We have two cases here

     

    • Restore from the same project

     

    Terminal prompts users to enter Project ID, gcloud will be configured with entered Project ID and performs Firestore Restore Operations.

     

    • Restore from a different project

     

    As we are doing different projects, the terminal asks the user to enter both Source and Destination Project ID’s.

    In this case, we need to give access permissions of GCS Bucket to the service account of the project where you are Restoring into.

    Firstly, the terminal sets gcloud to Source Project and permissions are set to GCS Bucket to be able to access from the destination project.

    Then the gcloud is set to Destination Project ID and performs Restore operations.

  3. Both Backup and Restore

    This is also between different projects, the user has to enter both Source and Destination Project ID’s.

    gcloud is set to Source Project ID and performs Backup operations and grants GCS Bucket access permissions to Destination Project then gcloud is set to Destination Project ID performs Restore operations.

 

Engineer’s Note: Project IDs have to be set correctly. Otherwise, the data would be copied to the wrong project and the entire application may not work properly.

Best Practices for Dealing with the GCS Bucket

It’s better to create a GCS Bucket in the same project of data to back up to avoid confusion.

There can be multiple folders of Backups depending upon the time frame you run the script, to avoid storing up multiple folders, set an appropriate Object LifeCycle Policy that automatically deletes folders after attaining the prescribed age.

There can also be human error that accidentally deletes folders without knowledge, to avoid this set a Retention Policy that retains folders for prescribed time periods.

You can see the status of the performed operations here else you can run this command to view detailed information Google Cloud Firestore operations list.

Wrapping up…

Performing regular Export and Import operations is important for any healthy infrastructure. However, doing this manually leaves room for error and also the possibility of missing an important backup. This script automates the process, giving developers and business owners greater peace of mind.

If you’re like to learn more about how you can leverage Firestore and automation within Google Cloud for your business, book a free consultation with D3V’s certified cloud engineers.

Author

Harsimran Singh Bedi

Daivik Dave

Software Engineer
Daivik has 2 years in modernizing applications on the Google Cloud. He holds a Master’s Degree in Computer Applications and is a Google Certified Professional Cloud Developer. Daivik is passionate about all things tech, movies, traveling, his pets, and video games.

Related Posts

What Our
Clients Are
Saying

Working with D3V was hands down one of the best experiences we’ve had with a vendor. After partnering, we realized right away how they differ from other development teams. They are genuinely interested in our business to understand what unique tech needs we have and how they can help us improve.

Lee ZimbelmanWe had an idea and D3V nailed it. Other vendors that we had worked with did not understand what we were trying to do – which was not the case with D3V. They worked with us through weekly meetings to create what is now the fastest and most accurate steel estimating software in the world. Could not have asked for anything better – what a Team!

We used D3V to help us launch our app. They built the front end using React and then pushed to native versions of iOS and Android. Our backend was using AWS and Google Firebase for messaging. They were knowledgeable, experienced, and efficient. We will continue to use them in the future and have recommended their services to others looking for outside guidance.

Constrained with time and budget, we were in search of an experienced technology partner who could navigate through the migration work quickly and effectively. With D3V, we found the right experts who exceeded our expectations and got the job done in no time.

Protecting our customers data & providing seamless service to our customers was our top priority, which came at a cost. We are very satisfied with the cost savings & operational efficiency that D3V has achieved by optimizing our current setup. We’re excited about future opportunities for improvements through deriving insights from our 400 million biomechanics data points.

Our experience with D3V was fantastic. Their team was a pleasure to work with, very knowledgeable, and explained everything to us very clearly and concisely. We are very happy with the outcome of this project!

Jared Formanr

Jared Forman

CEO & Co-Founder, OSMix Music

Lee Zimbelmanr

Lee Zimbelman

IT Director, BLI Rentals

Terry Thornbergr

Terry Thornberg

CEO, Fabsystems Inc.

David Brottonr

David Brotton

CEO & Founder, Squirrelit

Dr. A. Ason Okoruwar

Dr. A. Ason Okoruwa

President, Bedrock Real Property Services

Ryan Moodier

Ryan Moodie

Founder, DARI Motion

Schedule a call

Book a free technical consultation
with a certified expert.

Schedule Call

Get an estimate

Fill out our form to hear back with a project’s cost estimate. No meeting required.

Get Estimate

Get in touch

Send a message to D3V team.

Let’s Talk