When you’re a developer, everyday is an adventure.
Let me tell you about something that seemed really simple initially and turned out to be surpringly tricky…
My initial need: displaying a warning message
Nuclia is providing an API to index and search unstructured data, and, believe or not, sometimes our platform is not stable.
Obviously we are monitoring our infrastucture and usually we can fix problems before they impact our customers, but still, shit happens, and if something goes wrong, the first thing we want to do is to notify our customers.
So we wanted to be able to write somewhere a short message explaining we have a problem, and have this message displayed to our users when they enter our dashboard app.
And here are my constraints:
- Make it easy to enable/disable/modify the warning message
- Do not involve our CI/CD pipeline (I do not want to rebuild/redeploy anything for a simple warning message)
- Do not rely on our backend (maybe the problem we are facing is preventing us to use the backend)
When my CTO asked to find a solution for that, I was pretty confident.
So let’s go, challenge accepted!
What I would have done in 2002
In 2002, for a similar need, the typical approach was to make your Apache vhost to check if a file named maintenance.html exists in the root of your web server and if so, redirect to it.
To make the file avaialble, you had to connect in SSH to your server, or, worst case scenario if the CEO has to do it, it could be done with an SFTP client.
So, okay, today, it is not about pushing an HTML file, probably I would prefer a JSON file that can fetch from my frontend app, but a file is file, we just want to publish it.
How could it be significantly more difficult 20 years later, right?
So now, what?
So first thing first, the 2002 solution does not work anymore because it is not like people have an actual server you can ssh to (hmmm, wait a minute, was I really using SSH in 2002? I guess it was more likely to be Telnet or FTP, but whatever) and change files manually. Nowadays, servers are in containers, and obviously you cannot modify anything manually (that is considered a major crime in the DevOps law).
But that’s fine, I love containers, it makes my life much easier in so many ways. So no problem here, I just want to publish a file, let’s see my other options.
Because, yes, that’s 2022, I have options! File clouds, GitHub, third-party APIs, you name it!
My first idea was to use a Gist, it is super simple to create and modify and simple enough to access from a frontend app. But, guess what? Gist are not supported for organizations (and even more surprising: you can create a gist in a personal account, then turn it into an organization and the gist will still be available but you cannt create new ones…).
Next idea: PostHog. We use PostHog for usage analytics and also for feature flagging. I think it is a great tool. The warning message feature is kind of close to the feature flagging principle so maybe they are offering such an option. Well, they are not. But I though maybe I could create a feature flag named warning and store the warning message as description. I definitely can do that, but no luck again, the public API only retrieves the flag id, not the description…
Ok, let’s come back to the original need: I want to store a file and have it publicly available. That’s exactly what a CDN is for! We have Google Cloud, let’s create a bucket with a short max-age and we are good.
Yeah, so would definitely work but the Google Console is a bit more complicated than I expected. It does not let you edit a file online, you have to download it, modify it locally and upload it back to the bucket. That is super error prone.
If I want to make sure this warning message is easy to modify, maybe I need a tool dedicated to managing online content: that’s right, a CMS! We use WordPress, I am definitely not an expert but I am pretty sure WordPress can publish a file named /warning.json in the root of our website and make it easily editable! As I said, I am not an expert, and contrary to what I thought, the answer is no…
This is starting to be ridiculous.
What I finally ended up doing
I loved the Gist idea because editing a file on GitHub UI is easy. So finally, I decided to create a new public repository, having just one single JSON file and which contains the message and a boolean to enable/disable it.
Simple enough, right?
And my frontend app access it through its https://raw.githubusercontent.com URL. max-age header is 300 seconds, so that’s perfect.
Even though I felt it was a bit overkill (one repository just to publish a file), I was quite happy with my solution, and I proudly describe it on our Discord channel.
That’s where my CTO had this remark:
“Maybe rate limit on GitHub may be a problem”
Whaaat??!!! Don’t we live in a world where all the good things are free, unrestricted and unlimited forever???? Did Mom and Dad lied to me since the begining????
You bet, moron! Unauthenticated requests to raw.githubusercontent.com are limited to 60 requests per hour!!!! That’s so frustrating…
But (yes, fortunately there is a but), I gave it a quick though, and I figured out I just have to make my main branch the default branch for GitHub Pages.
That’s even better! As I can publish my JSON file without restriction and I can also publish a proper HTML file so Nuclia status can also be read by humans! I have a proper status page!
The only downside is max-age is 600 seconds on GitHub Pages, but I can live with that.
