Every once in a while, an uncomfortably tight deadline appears. In this case, as part of a course titled Interactive Programming and the Dynamic Web, almost sounding like the title of a Harry Potter entry. The rules were simple. About two weeks of development at around 30% speed to deliver a project containing authentication, live data updates across users, persistence across sessions, error and loading state handling, and an enforced MVP project structure. The only novel part for me would be the live data updates, as standard REST and GraphQL queries had been my way to go to provide solutions so far, but this would require a timeframe so short my regular stack of mostly hand-crafted components and interface pieces would not suffice anymore. Additionally, I had teammates to give a solid footing, which includes making collaboration as pain-free as possible.
It was clear I’d have to thicken my stack.
Stack choices
Thankfully, tools that do just that are in no shortage. To cover the most vital aspect, live data updates and persistence, we elected to use the tried and true Firebase solution using the Firestore database. It provides a subscription based abstracted webhook solution that allows you to subscribe to documents (it’s a document based database) and update on changes, which is fantastic for this use case. The API is simple, and getting the basics up and running didn’t take long. What’s especially nice here is that Firebase handles authentication as well, with support for most use cases and enhanced by React Firebase Hooks if React is used. This was a breeze, and setting up the entire auth flow including login, register, reset password, email confirmation and even change of email was almost too easy.
To power this up even more, we combined it with Mantine and accompanying Mantine UI, a ready to go component framework for React that includes almost every UI component we could possibly need, while the UI package provided ready-made authentication designs. Normally, I dislike relying component libraries, especially for whole pages, as it kills my creativity, instead providing a quite sterile and uninspired look and feel. CSS is additionally not as hard as it used to be, and custom components can be built relatively fast for most cases. But I had to admit their case when time was this short. It looks okay, it handles a11y better than most people would handle it themselves, and gets your site up and running very quickly. The theming capabilites are restrictive but good enough to provide a recognisable vibe in a punch, while in this case providing a whole host of useful hooks out of the box as well. Overall we were very satisfied with what Mantine made possible in such a hurry, and the results speak for themselves on that front.
The gotchas of Firebase
But then it came to building the app functionality into something complete, and this is where we started questioning some of the choices made at the start. While Firebase solves the live updating data aspect, we failed to account for the multiple users part. Not that Firebase can’t handle users, it clearly can, but Firestore is a document-based database. Our idea for the app would require users to be put in groups accessible by invite code, with items belonging to each group added by users themselves. If it’s not obvious already this is a trademark case where a relational database excels, which became painfully obvious the further we went. There are a couple of methods available to simulate similar behavior in document based databases, but it always involves data duplication somehow, and keeping that data in sync, especially with webhooks involved, can pose a real threat. The issue is exarcerbated by Firestore’s preferred method of securing the API, namely Firestore Security Rules, which when written correctly handles database access restrictions and authentication. It gets complicated however, when data needs to be duplicated across collections, or updated differently but still in sync. This happens through the client, and without Firestore’s batch updates and transactions it would’ve been an insurmountable task for this project timeline. A better way to handle this would be Firestore’s version of serverless functions called Cloud Functions. The catch there being that they’ll bill you twice per request — first for the serverless function, then for the database read/write operation. It goes without saying that it scales quite terribly using that setup, so security rules is likely the way to go even considering the added complexity.
Additional APIs
In addition to request to our database, we also required API calls to a game database. We looked into options that could be called directly from the client to mitigate any additional backend setup on our end but failed in that search, with the best APIs not only enforcing API keys but also requiring a backend proxy before calling the API. A textbook case for serverless functions, in other words. We found that given our requirements, Amazon AWS provided it all, with a generous free tier for their lambda functions and a large community for support and documentation. It took about a day to spin up the AWS setup with a correctly configured proxy for the API in question, but when it was up and running it worked flawlessly and added a surprisingly small amount of delay on the request from the client. We opted for using Tanstack Query for these API calls due to its fantastic handling of caching and background updates, with easy to use hooks and revalidation triggers available. It's my go-to package for API requests in client side apps nowadays, with alternatives not quite as good in my opinion. This combination worked wonders, and with proper loading and error boundaries implemented, we could tick that box as well.
We did intend to utilize Tanstack Router as well due to its exceptional typesafety and search params API's, but ran into some beta issues that prevented fast enough implementation, so we resorted to falling back to the solid React Router which for our purposes still works well. In fact from what I can see, most of the criticism towards React Router seems to stem from the fact that some APIs were deprecated without viable alternatives, which is fair criticism but nothing preventing usage if those are not on the requirements list. I doubt similar mistakes will be repeated in that project.
An interesting hurdle we ran into that is worth mentioning was that the game database we used offered genre data on each game queried, as integers. To get information about what each integer represented, you had to query another endpoint, but only one ID at a time. This meant that in the simple scenario of offering a game search function with displayed genres of each result, you’d need so many requests your quota would be used up in a minute. This was interesting to us because the integer mappings was stable and not very plentiful, so they might as well have offered a simple JSON object of key value pairs for client side parsing of genre IDs. Since they did not, we wrote a quick python script that called the endpoint with every integer from 0-999 and formatted it all into just that, a JSON object. Turns out there were only 40 or so genres, so client side conversion was no problem at all, no API requests required. I suppose there is the possibilities that these are changed eventually, so ideally you’d run this script periodically, SSG fashion I imagine. Anything’s better than hundreds of API requests per search result though!
Conclusion
With that covered the app was completed, not a day too early. The key takeway from that experience was that rapid development is much easier with the right set of tools, but extra effort should be placed on picking the right set of tools from the start. This requires in-depth review of project requirements and problems one might face. Doing this right, for instance by picking the right type database for the type of application in question, will yield faster development speed and fewer unexpected obstacles down the line. I certainly won’t underestimate the impact of pre-development analysis from this point and onwards!