Application Development

Serverless in action: building a simple backend with Cloud Firestore and Cloud Functions

October 12, 2018

Stefan Hogendoorn

Co-founder at Qlouder and Google Developer Expert

[Editor’s note: Qlouder is a long-time Google Cloud partner that specializes in application development, big data, machine learning and collaboration platforms. Based in the Netherlands, Qlouder is a big believer in serverless methodologies. In this blog series, they will show how to supercharge your Firebase applications with Google Cloud Platform’s (GCP) fully-managed services. In this first of four posts, they show how they approach building simple applications using Cloud Firestore and Cloud Functions. The next posts will take you through progressively more complex systems and design patterns for processing large data sets and incorporating legacy systems as a part of your serverless application architecture.]

Here at Qlouder, we’ve been using using Google Cloud and serverless computing for 10 years, starting with App Engine. We’ve seen serverless development from its earliest beginnings till the latest developments and we’re enthusiastic ambassadors of this method. Over the years, we’ve built dozens of serverless apps for our customers, and most recently, we used two Google Cloud services, Cloud Firestore and Cloud Functions, to completely rebuild our internal expense reporting system.

Why serverless

In this post, I’ll describe the serverless approach we took to building this system. But first, I’d like to talk about why I’m such a fan of serverless. If you’re a developer or manager, you’ve probably heard that serverless development can help you to create scalable and high-quality apps. But you’re probably wondering, how easy is it, really? Pretty easy, actually.

Why’s that, you ask? Serverless doesn’t mean you don’t use servers, it just means you focus on design and code, without having to worry about the moving parts—the services running on your app’s servers. Before serverless computing, you had to take care of the whole underlying IT infrastructure. Take setting up a server or virtual machine and maintaining it, for example. When your application grew or shrunk, you had to add/remove servers and when coding, you had to make sure to keep the server infrastructure in mind, instead of just thinking about the functions and user experience. With serverless development, you can start coding right away. You skip a lot of internal complexity, making it easier to get to market. And because you don’t have to create a whole fixed infrastructure, you can scale up and down according to use, so you only pay for what you use.

Erasing the borders between front-end and back-end

Actually, serverless has had a huge impact on the way the dev world is shaped. Because of serverless, the differences between front-end and back-end are disappearing. To do back-end development, there’s a lot less complexity to master. You don't need to know about distributed system design, load balancing, database sharding, or system maintenance. You only have to know what your customer wants and what kind of functions you need to use.

https://storage.googleapis.com/gweb-cloudblog-publish/images/front_end_back_end.max-700x700.png

Serverless allows developers to quickly adapt code to functional requirements and make adjustments to the database structures as needed without the separation of concern you often see in strict n-tier development.

Developing an expenses app

For this example I’ll show you an expenses app I built with my team at Qlouder. As a fast growing systems integrator, we need to implement systems for all sorts of things—from finance to HR and from knowledge sharing to expense reporting. Over the past few years we’d used off-the-shelf products, but with our changing needs we found those packaged products lagging behind. In particular, one of the systems causing havoc among our team was the expense system. With our background experience using Firebase and GCP, we thought ‘Hey, we can do this ourselves!’

https://storage.googleapis.com/gweb-cloudblog-publish/images/expenses_app.max-700x700.png

Focus on the user

Our first task was to pick a web front-end. At a high level, the purpose of the expensing app is to streamline and control a company’s expenses and budget. With versions for Web, iOS and Android, we wanted to put the the user experience and journey front and center, and wanted an expenses app that was simple and straightforward. For example, users should be able to put their expenses into the system and be informed of the progress via notifications.

A Polymer web front-end combined with Flutter, Google’s mobile app UI SDK, addresses those requirements from all perspectives. Polymer provides an attractive (and easy-to-use) interface and Flutter provides the mobile UI for both Android and iOS: you develop the app and Flutter’s framework gets it ready for mobile.

https://storage.googleapis.com/gweb-cloudblog-publish/images/qlouder.max-1600x1600.png

Example UIs built with Polymer (left) and Flutter (right)

Business logic. Cloud Functions lets you build a scalable back-end by simply adding business logic that connects your application’s front-end to your data repositories. Your code triggers on data changes and interfaces with other services, including Google’s machine learning solutions.

Our expenses app doesn’t have a lot of complicated processes: users file their expenses, then the finance team reviews and approves or declines them. That meant our expenses app can read and write directly to the database, instead of using a server as an endpoint.

There’s no need for an infrastructure to handle the communication between the database and application; the user interacts directly with the database through the Cloud Firestore SDK. Major bonus: this eliminates a lot of complexity and helps reduce security risks and overall development time!

Data. In our expenses app, we store the receipts on Cloud Storage for easy storage and retrieval. The expenses structured data, such as the amount, who registered the expense and the expense approval, needs to be stored as well. For that we use Google’s Cloud Firestore.

If you ever built an app using Firebase, you may be familiar with Realtime Database. It’s a magical cloud-hosted NoSQL database that synchronizes data in realtime to every connected client. Cloud Firestore is the latest database from Firebase and Google Cloud Platform that makes it even easier to store and query data from the application. A major added bonus for our mobile app is that Cloud Firestore also provides built-in syncing of data in case users are offline while submitting their expense reports!

Oh, and as a ‘little’ bonus, when you build your mobile app using Firebase services, you get Google Analytics for Firebase and Firebase Predictions. Combined, these tools give you detailed insights into how your users use your application and if they’ll continue using it.

Serverless economics. For our expenses app, the number of users varies in usage. Some people input their expenses right after they travel, some once a day, once a week or even just once a month.

Having a serverless backend guarantees speed and scalability: our app runs smoothly for any number of users, whether their use is intensive or not. And it also provides utility-like pay-as-you-go billing based on actual usage.

From a DevOps point of view the advantages are quite obvious as well; there’s no need for servers, virtual machines, or communication between them. Nor do you have to worry about uptime or patching your server OS. It’s all part of these fully managed services.

Whether you’re building native apps, web apps or you’re using Flutter, Cloud Functions and Cloud Firestore make it easy to build a solid, scalable application.

On that note: Cloud Functions and uploading receipts

Cloud Functions are designed to make it easy to process data. Functions are triggered by events in the database (Create, Update and Delete) or by external events such as an HTTP request or a write to Cloud Storage.

https://storage.googleapis.com/gweb-cloudblog-publish/images/cloud_functions.max-600x600.png

We use Cloud Functions for Firebase within the expenses app to process uploaded photos and receipts. The code is triggered the moment data is added to or updated in the database or when files are added to Cloud Storage. Cloud Functions optimizes size/resolution of uploaded photos. This is a perfect way to keep your app from becoming cluttered with large images, even after years of active usage—all in just a few lines of code!

Here’s some sample code from our expense reporting app to give you an idea:

/**
* When an image is uploaded, we generate a thumbnail using ImageMagick,
* save the thumbnail, and write its public URL to Firestore.
*/
exports.createImageThumbnail = functions.storage.object().onFinalize((object) => {
   // Exit if the image is already a thumbnail
   if (fileName.startsWith(THUMB_PREFIX)) {
       console.log('Already a Thumbnail.');
       return null;
   }

// Firestore reference
   const firestoreID = object.id.split('/')[2];

// Create a temp directory for the image file
   return mkdirp(tempLocalDir).then(() => {
       // Download file from the Cloud Storage bucket
       return file.download({destination: tempLocalFile});
   }).then(() => {
       // Generate a thumbnail using ImageMagick
       return spawn('convert', [tempLocalFile, '-thumbnail', `${THUMB_MAX_WIDTH}x${THUMB_MAX_HEIGHT}>`, tempLocalThumbFile], {capture: ['stdout', 'stderr']});
   }).then(() => {
       // Upload the thumbnail to Cloud Storage
       return bucket.upload(tempLocalThumbFile, { destination: thumbFilePath, metadata: metadata });
   }).then(() => {
       // Delete the local temp files
       fs.unlinkSync(tempLocalFile);
       fs.unlinkSync(tempLocalThumbFile);
       // Get the URLs for the thumbnail and original image
       const config = {
           action: 'read',
           expires: '03-01-2500'
       };
       return Promise.all([
           thumbFile.getSignedUrl(config),
           file.getSignedUrl(config)
       ]);
   }).then(results => {
       const imageFirestoreRef = firestore.doc(`expenses/${firestoreID}/images/${fileName}`);
       return imageFirestoreRef.set({
           name: fileName,
           bucket: object.bucket,
           fileUrl: fileUrl,
           filePath: filePath,
           thumbURL: thumbFileUrl,
           thumbPath: thumbFilePath,
           contentType: object.contentType,
           addedOn: new Date().getTime()
       });
   }).then(() => console.log('Thumbnail created.'));
});

You can also use a cloud function to easily export your data into Google Sheets. We use the following function to create an overview of all approved expenses and share it with our accountant for further administrative processing. (We could have also opted to create do an API call straight into the finance system but that didn’t fit the sprint.)

export const exportExpenses = functions.https.onRequest((request, response) => {
   exportService.getApprovedExpenses().then(expenses => {
       writeToSheet(expenses).then(res => {
           return response.status(200).send(res);
       }).catch(err => {
           return response.status(400).send(err);
       })
   });
});

writeToSheet = (expenses) => {
   return new Promise((resolve, reject) => {
       getAuthorizedClient().then((client) => {
           try {
               const sheets = google.sheets('v4');
               sheets.spreadsheets.values.append({
                   auth: client,
                   spreadsheetId: SHEET_ID,
                   range: 'A2:C2',
                   valueInputOption: "USER_ENTERED",
                   resource: {
                       values: expenses
                   }
               }, (err, response) => {
                   if (err) {
                       reject('The API returned an error: ' + err)
                   }
                   resolve(response)
               });
           } catch (e) {
               reject(e)
           }
       }).catch(() => reject('No client'));
   })
}

Other considerations

Individual Functions vs. API

For our expenses app, which has a limited number of features and a limited dataset, using Cloud Functions to consistently use the same trigger is a logical, lightweight solution. Of course, when your apps get more complicated, you should also consider writing your own REST APIs using Cloud Functions. A cloud function not only triggers on database or storage events but can also be triggered by HTTP requests. This allows you to write your own set of RESTful APIs.

Who does what? The authorization question

What about authorization? An essential part of the expenses app is who has access to what. Who can write records? Who can edit these records? And of course, who can make decisions about the input, because you want to separate regular employees from administrators from finance people.

This is another area where Cloud Firestore helps out, without having to write a lot of code. Cloud Firestore Authentications is a pre-made integrated solution that lets users log in with existing accounts. These can be email, Google or Facebook accounts. You determine the security rules per type of user, and Firebase Authentication arranges the distribution. In our expenses app we set a rule to allow every user to add expenses, edit those records and see their account history. Admins are able to edit all entries and review (approve/decline) expenses. And we made sure that the financial team has ‘read-only’ rights for ‘to pay’-expenses.

Airplane mode

Using a serverless application with a lot of connections to an online server does raise the question of whether it’s possible to use the app offline, without a lot of effort. In short: yes! When the app goes offline, Cloud Firestore takes over and syncs the user entries once the app is back online again. As a developer, you know how much back-end complexity this saves.

Summing things up

There are many ways to build an internal line-of-business application such as an expenses system. If you’re looking for simplicity, speed, security and scalability, serverless is your path. On the database end, Cloud Firestore provides everything you need: a separate front- and back-end, the infrastructure, authorization rules, data analyses, behavior prediction, and offline usage. And Cloud Functions makes it easy to build business logic. Using this method, it took us only three days to build the whole application!

We hope you enjoyed this post. In our next blog we’ll show you how to integrate complex data structures in external systems. We’ll also introduce more serverless computing possibilities through Polymer and complex Cloud Functions.

Posted in

https://storage.googleapis.com/gweb-cloudblog-publish/images/overwhelmed_persons_guide.max-700x700.png

Google Cloud

The overwhelmed person’s guide to Google Cloud: week of November 14

By Richard Seroter • 5-minute read

Partners

Arize, Vertex AI API: Evaluation workflows to accelerate generative app development and AI ROI

By Gabe Barcelos • 9-minute read

AI & Machine Learning

PyTorch/XLA 2.5: vLLM support and an improved developer experience

By Manfei Bai • 4-minute read

https://storage.googleapis.com/gweb-cloudblog-publish/images/DX.max-700x700.png