formae vs. Terraform

Infrastructure code is the source of truth

There is a reason why it is called Infrastructure as code. You should be able to see your current infrastructure as code at any time, always up-to-date, without having to manually update it.

On Day 1, everything is simple, but a properly structured approach with IaC is needed - ClickOps doesn't work on Day 1 at scale, for example. Repeatability of initial rollout into different environments always plays an important role.

Day 2+ operation

Starting on Day 2, the reality is complicated, and the so-called "drift" is inevitable. Small changes are common, and also different tools will almost certainly be in use to manage infrastructure. Even ClickOps is a legitimate tool in some cases - there is no reason to avoid powerful tools such as Cloud Web Consoles just to satisfy a rigid process that is implemented for Day 1 operations.

Cloud providers make updates without updating or even knowing about your infrastructure code. All sorts of autoscaling rules and technologies constantly change the topology. Security tools and teams make urgent and planned changes often bypassing infrastructure code as well. Different teams follow different processes and implement different workflows for maximum speed. It is necessary to stay in control of drift and automate your response to it, not only be able to detect it.

Support for every engineer's use cases

Empowering every engineer to work with the tool enables faster change at lower cost. Experts can focus on low-level detail such as networks, while less experienced engineers can still make targeted/small changes without going too deep. Ultimately, developers can consume or request infrastructure without knowing any low-level detail. All these roles have their own nuances, too, depending on the team setup, skills and situations. For example, being an expert and having to fix an urgent problem at 2a.m. is a different use case than preparing a new team environment, and requires different focus, blast radius control and sometimes even tools.

Simple configuration language

The language of the infrastructure code needs to be simple. Of course, "simple" is highly subjective. But for example, Ops teams usually don’t do full-blown software development, which most ecosystems such as TypeScript, Java or Python require. Scripting is as far as people go in Ops. So far, YAML is the widely used common denominator that every engineer accepts, but it requires extra tools for templating, conditionals, functions and other means borrowed from programming. And these are needed for flexibility and DRY (don't repeat yourself).

Schema safety

Schema safety is essential to avoid human error. Understanding the schema without having to navigate through each provider's source code makes configuration of resources simple and allows team-specific derivatives. On the other hand, using such low-level languages such as YAML currently comes at the cost of not having any schema safety at all, with a high risk of human error—sometimes catastrophic.

Schema-based resource and dependency declaration

When all data-specific, mechanical aspects, as well as dependencies of the resource are defined through schema, target infrastructure is easy to maintain and adjust, and it allows you to keep the infrastructure code clear and the providers / plugins lean and simple to maintain. Too often, relationship and dependency logic is hiding inside already complicated provider code, which unnecessarily complicates the resource lifecycle management and the usage of the tool.

Low provider complexity

Low provider (or plugin) complexity means easy update cycle, reliability and ability for anyone in the community to quickly implement their own plugins and provide bugfixes to existing ones. In the classic IaC, providers are often full-blown programs with workflow and lifecycle logic which can usually only be adjusted by its creators. There are also big differences in provider quality that are difficult to deal with. Lightweight plugins also relax that issue.

Active agent

Engineers shouldn’t need extra tools to compensate for the inability of their IaC tool to support automation. Classic IaC tools attempt to work exclusively on the local machine, following the minimalistic philosophy of the original UNIX tools. This hinders the ability to work in teams or multiple teams, and puts the user in the middle of the process as yet another tool. True automation is only possible by handing off work to an active agent, removing toil for users.

Asynchronous change execution

Applying infrastructure changes asynchronously makes them reliable and repeatable. Some infrastructure modifications in some technologies can take a long period of time and follow a complicated workflow. This doesn't mean that a full-blown workflow engine is the solution though - it only means that infrastructure changes need to run asynchronously, without the initiating user process having to wait for the result. And that unfinished work can be picked up after crashes without the risk of state corruption.

Automatic state management

State management is a highly manual process in the classic IaC, and constantly requires human attention and intervention. Even if some tools manage the state data in cloud offerings, the point in time when the state is updated is still up to the engineer, making them part of the tool. Engineers should not have to tell their IaC tool when to persist or update its state. Having to manually maintain state introduces an additional layer of complexity and possible human and technical error.

Minimal locking

Global locking is how classic IaC tools make sure their state doesn't get corrupted by parallel mutations, but only a single resource can be locked in the majority of cases. Global locking is causing slowdowns and waterfall-like team workflows, while minimal locking makes sure teams and engineers can move independently and fast.

Arbitrary change granularity

Being able to apply changes in any granularity allows you to massively reduce risks, limit the blast radius of those changes and maintain focus, resulting in much less human error. Holistic Day-1-style rollouts on Day 2 create cognitive overload and more toil in cases where small changes are made, for example urgent fixes. Changing a single property of a single resource shouldn't feel like touching the whole infrastructure code.

Patching

Being able to incrementally apply changes to single resources without dealing with irrelevant details is crucial for safety and speed. Blast radius control is a big and central topic in IaC. The less detail is needed to make an infrastructure change, the more confident you can be. Also, incremental changes are very common in many environments starting on Day 2. There are many situations where a patch is the simplest, quickest solution, and the IaC tool needs to support these.

Automatic drift detection

The term "drift" has been massively bloated in the classic IaC world. Since classic IaC tools are not able to automatically catch up on changes happening outside or their own idea of state, these changes have been declared as something very bad. Some tools even go so far that they automatically reconcile from their own state, undoing changes that happened outside of the tool. The reality in any reasonably complex environment is much more nuanced though. Multiple tools will be used, and changes of all kinds happen through all kinds of actors. Enforcing a single process and a single tool is often impossible, and also not reasonable. Drift as such should not be a concern - state and real infrastructure need to be always in perfect sync, with changes coming from both ends.

Automatic drift correction

Once drift has been automatically detected, it is crucial to also automatically correct it. It doesn't mean to restore infrastructure from the state one way - this would be only one of the possible strategies, and it shouldn't be applied uniformly. A more reality-friendly strategy is to accept changes coming from both ends. After all, when a change already happened in the target infrastructure, the state needs to be first updated, so there is no drift. After that, the user is free to decide if they want to undo the change. Or the change can be undone automatically according to some policy, but even after it has been registered and versioned.

Automatic resource discovery

Automatic discovery of new resources makes use cases possible that are unthinkable or difficult to implement with the classic IaC tools. For example, you are able to analyse existing infrastructure immediately, even when there is no infrastructure code available. Also, patches on existing resources can be made quickly when these are discovered and turned into code - completely without the need to look at the rest of the infrastructure. On the other hand, classic IaC tools would send you through a manual process of importing existing resources that is prone to error. Automatic resource discovery also allows you to have a full inventory of existing resources for further processing entirely without manual work.

Automatic change synchronization

Changes happen all the time outside of the IaC tool in a reasonably complex environment. It is a good idea to capture them automatically, not manually. Classic IaC tools require the user to update their state explicitly and manually, which inevitably leads to drift. On the other hand, state that is automatically up-to-date with the infrastructure makes changes predictable and reliable, and allows you to make explicit decisions to keep or to undo changes that happened outside of the tool.

Automatic change versioning

When every change is automatically versioned, no matter where it came from, infrastructure is fully auditable. Also, versioning of changes allows you to review and to restore previous states at any time. Git is usually used for this in GitOps, but it comes at the price of code in Git being decoupled from state and from infrastructure. If the IaC tool takes care of change versioning, it is also one tool less to deal with in situations where change history needs to be analysed or used.

Automatic change codification

Users should communicate with the IaC tool exclusively through code. This requires every change to be codified automatically, no matter where it came from, so that the user can retrieve any current or past resource state as code. Classic IaC tools on the other hands Classic IaC tools, on the other hand, make users absorb their highly technical, low-level state format such as JSON and go through a complicated, manual procedure to turn it into actual infrastructure code.