Infrastructure (not only) as code?

I’m more the visual type of guy. When it comes to deploying my app to the cloud, I prefer to “see” what’s happening. I want to change a setting, hit a button and see the changes be applied in seconds.

But I know there are other types of engineers (one is sitting across from me in my office), who’d never touch a visual tool to update a critical system.

The clicker’s way

For deploying my platform Ken, I used Render.com. With a few clicks and adjustments I set up a web service to run a Node.js application as well as a Postgres database. In a matter of an hour or two I had it all working.

The downside is that I can’t easily replicate that setup. E.g. if I accidentally delete my services I have to remember all my clicks to recreate them.

The coder’s way

Render offers Blueprints to solve that issue. I can specify my infrastructure in code, so I could literally wipe all the services and recreate them based on the Blueprint, stored in a file called render.yaml.

What’s great about this? I’ll get one single-source of truth to instruct the cloud infrastructure. An update is triggered automatically when the file has changed.

This works fantastic, until I use Render’s UI in addition to changing render.yaml. Then it could easily happen that on the next deploy, I accidentally overwrite settings I’ve set via the UI.

Click and code

If I could wish something from Santa next Christmas, it’d be a cloud hosting platform, that gives me the best of both worlds. To me this would mean:

• All infrastructure is specified in code: deploy.yaml is the single source of truth

• A UI that lets me edit the settings visually: deploy.yaml is updated on each save

Consequently I’d not be able to deploy anything without a deploy.yaml file. Probably there are reasons why this may be a bad idea, but I’ll make this assumption now to evaluate a puristic proposal.

Mental model

Cloud computing lingo tends to be hard to understand. Particularly if you don’t have a history in devops. Here’s an attempt to create a mental abstraction I can easily wrap my head around.

Apps

Apps are the main entities in the system. An app is represented as a repo which holds a configuration file (deploy.yaml) describing the infrastructure. Often this repo would also hold the application code, but does not have to.

Services

An app can contain any number of services. Services can talk to each other via private networking. However, an App defines a sandbox, so a service of one app can not access a service of another app (unless exposed via a public API of course). There are different types of services such as Web Service, Private Service, Background Worker, Cron Job, Postgres DB, etc.

Environments

I love Render’s “Shared environments”, so I’d make that the only concept and just call these Environments. Each service must have one Environment assigned, which exposes a set of environment variables.

Previews

With the help of Previews I can test new features of my application in a safe way. Since the services of an app can co-depend it makes sense to use the preview concept on the app level. To preview new app functionality, I can create a Pull Request in the main repository (where deploy.yaml resides). Conceptually I’m not previewing individual services, but my entire app. In that Pull Request, representing the preview, the deploy.yaml file references other services and points to the Git branches of the services I want to test.

Clones

A service can be cloned based on its current state. This can be very useful for previews. Let’s say I want to test a new feature, then all my services including the production data are cloned. Now the preview can operate on quasi-live data, without any additional setup efforts.

First deploy

This would work just the way I used the Render.com interface to deploy letsken.com. The only difference would be that now as I change the settings, I get a confirmation that the updated settings will be written to deploy.yaml before coming to effect.

Preview a new version

Let’s assume that my deploy.yaml references three services (a web service and a Postgres Database, as well as a little private service I’ve set up for some anonymous usage tracking involving a Redis Database). Let’s assume that the code resides in two repositories (ken, ken-analytics).

Now I want to develop a new feature, which involves changing the main application as well as the analytics service. I create a branch ‘article-stats’ in both repos and start developing locally. After a long day of hacking I’m ready to preview that functionality and show it to friendly customers.

I create a new preview for that app using the visual control panel. I choose ‘article-stats’ as the branch to be used for this preview. Now I’m presented some settings for each service used in the app. I check the “Use a clone for preview” box for the analytics service, and select a the environment “testing” which will be used for the preview.

Clicking save writes an update to deploy.yaml and opens a Pull Request. Seconds later I share a link to that preview with my friendly customers. 🍾

These are just some rough ideas, which I may explore more and update this post accordingly.