The API Illusion: Why We Blame the Wrong Tools
I originally intended to write about the limitations of custom providers within the OpenTofu and Terraform ecosystems. While researching the topic, however, I realized that focusing on engine limitations misses a much larger architectural reality.
Infrastructure as Code engines are not magic. They are translation layers.
When a deployment lacks a feature or fails to support a particular workflow, the limitation is often not Terraform or OpenTofu itself. More commonly, the constraint comes from the maturity, consistency, and capabilities of the target platform’s API.
If a cloud service, network appliance, storage platform, or internal system does not expose a clean API—or at least a reliable CLI—there is very little any provider framework can do. Infrastructure as Code depends entirely on the ability to translate declarations into actions, and that translation requires something capable of receiving instructions.
Once you view infrastructure management as an API translation problem, your focus shifts. Rather than obsessing over the IaC engine, you begin examining the quality of the interfaces beneath it.
This is where the concept of text-based optimization becomes interesting.
Declarative Text vs. Imperative Execution
To understand text-based optimization, we first need to look at how infrastructure systems interpret text.
Historically, infrastructure was managed through imperative interfaces. Traditional shell scripts, PowerShell automation, and command-line driven network operating systems such as Cisco IOS focused on how a task should be performed.
In an imperative model, sequence matters.
Running Command A before Command B may produce a different result than executing them in reverse order. The system follows instructions exactly as written, without understanding the intended final outcome.
Modern infrastructure platforms increasingly favor declarative approaches.
Kubernetes manifests written in YAML and OpenTofu or Terraform configurations written in HCL describe a desired state rather than a sequence of actions. Instead of specifying every step, the operator defines the end result, and the platform determines how to achieve it.
| System Type | Core Characteristic | Examples |
|---|---|---|
| Imperative | Focuses on how state changes occur | Bash Scripts, PowerShell Scripts, Cisco IOS CLI |
| Declarative | Focuses on what the final state should be | OpenTofu/Terraform (HCL), Kubernetes (YAML) |
The distinction is important because declarative systems move complexity away from the user and into the engine itself. The operator describes the destination, while the platform calculates the route.
The Orchestration Paradox
A paradox appears when declarative tools enter enterprise automation environments.
Although HCL is declarative, the systems orchestrating it are often highly imperative.
CI/CD pipelines, Git event triggers, shell wrappers, scheduled jobs, and deployment scripts all execute tasks in a specific sequence. The order of operations becomes critical.
Consider a deployment pipeline that executes a database migration before the infrastructure platform has completed provisioning network connectivity. The migration fails—not because the HCL was incorrect, but because the surrounding orchestration executed steps in the wrong order.
This is where text-based optimization becomes more than a configuration exercise.
Infrastructure definitions cannot be evaluated in isolation. Module boundaries, dependency declarations, outputs, state management, and orchestration workflows must all align with the real-world dependencies of the platform being managed.
The more accurately those relationships are represented in text, the more reliably automation can execute them.
The Architecture of Translation
Understanding text-based optimization requires understanding the translation layer itself.
A common misconception is that Terraform or OpenTofu contain built-in logic for communicating directly with AWS, Azure, Google Cloud, Kubernetes, or other platforms.
They do not.
The core engine is responsible for parsing configuration files, constructing dependency graphs, maintaining state, and coordinating execution. Communication with external systems is delegated to providers.
This separation exists through a plugin architecture.
The engine itself is a compiled Go binary. Providers are separate executables that are launched by the engine during execution. Communication occurs through a provider protocol implemented over gRPC, allowing the engine and providers to operate independently while exchanging structured data.
This architecture is one of the reasons Terraform and OpenTofu can support such a wide range of platforms without embedding vendor-specific logic into the core binary.
Providers: The Real Intermediaries
When discussing API limitations, it is important to recognize that providers form a critical intermediary layer.
Platform API <-> Provider <->Terraform/OpenTofu
Even a powerful and well-designed API can produce a poor Infrastructure as Code experience if the provider implementation is incomplete or poorly maintained.
Likewise, a well-designed provider can often simplify complex APIs and present them through a cleaner declarative interface.
For most practitioners, the quality of the provider becomes just as important as the quality of the platform API itself.
The Evolution of Provider Development
The provider ecosystem has evolved significantly over time.
Early provider development required substantial amounts of boilerplate Go code. Developers were responsible for implementing API clients, handling HTTP communication, parsing responses, managing retries, validating data, and mapping resource state manually.
Much of the effort involved infrastructure plumbing rather than infrastructure logic.
Modern provider frameworks have reduced that burden considerably.
Frameworks originating from the Terraform ecosystem—and now widely supported across both Terraform and OpenTofu-compatible providers—allow developers to describe resources through schemas and standardized lifecycle patterns.
A schema defines:
- Required attributes
- Optional attributes
- Data types
- Validation rules
- Relationships between fields
From these definitions, the framework can automatically perform many common tasks:
- Converting HCL values into Go types
- Validating user input
- Managing state transitions
- Standardizing CRUD operations
- Producing consistent diagnostics
Provider development remains software engineering. Developers still write API integrations, business logic, state mapping, and lifecycle handling.
However, modern frameworks allow them to focus more on the platform being integrated and less on repetitive infrastructure code.
Why Text Continues to Win
The evolution of Infrastructure as Code reveals an interesting pattern.
As systems mature, more complexity moves into reusable frameworks and abstraction layers, while users interact with increasingly expressive text-based definitions.
Terraform modules, Kubernetes manifests, provider schemas, policy definitions, and CI/CD pipelines all follow the same trajectory: describe intent in text and allow automation layers to translate that intent into execution.
The success of modern infrastructure platforms is not merely a story about APIs or cloud services.
It is a story about abstraction.
The industry continues to move away from manually orchestrating individual commands and toward describing desired outcomes through structured text. The engines, providers, and APIs exist primarily to translate those declarations into reality.
In that sense, Infrastructure as Code is not really about code at all.
It is about translation.