Kubernetes & Kubernetes API

TL;DR: Kubernetes is driven entirely by its API in every component communicates through it to manage the cluster. This post focuses on Kubernetes internal communication. Key technical terms will be highlighted in code blocks as we go. What is Kubernetes? In short, Kubernetes is an orchestrator for containerized workloads. Before diving into orchestration, we need to clarify what containerized systems actually are. Unlike virtual machines (which virtualize full hardware), containers isolate processes at the OS level. They package an application and its dependencies together so it runs consistently across almost any Linux environment. ...

August 24, 2026 · rathwjj

Gospel of Matthew

In case you don’t know the “Serenity Prayer”: “God, grant me the serenity to accept the things I cannot change, courage to change the things I can, and wisdom to know the difference.” Also, don’t confuse it with Matthew’s prayer: “When you pray, you shall not be as the hypocrites, for they love to stand and pray in the synagogues and in the corners of the streets, that they may be seen by men. Most certainly, I tell you, they have received their reward.” ...

July 5, 2026 · rathwjj

Infrastructure as Translation: Connecting Proxmox, TrueNAS, and MariaDB

This article demonstrates how to use Terraform to orchestrate dependencies across independent infrastructure components: storage, databases, and compute. By focusing on a multi-provider workflow, we will examine how Terraform constructs an execution graph to ensure resources are provisioned in the correct sequence, while noting how to handle out-of-band application logic. The Architecture of the App To understand the data flow, our application architecture connects three distinct systems across the network: ...

June 17, 2026 · rathwjj

Infrastructure as Translation: The Provider Boundary

The API Illusion: Why We Blame the Wrong Tools I originally intended to write about the limitations of custom providers within the OpenTofu and Terraform ecosystems. While researching the topic, however, I realized that focusing on engine limitations misses a much larger architectural reality. Infrastructure as Code engines are not magic. They are translation layers. When a deployment lacks a feature or fails to support a particular workflow, the limitation is often not Terraform or OpenTofu itself. More commonly, the constraint comes from the maturity, consistency, and capabilities of the target platform’s API. ...

June 12, 2026 · rathwjj

Terraform Plan and .tfstate

In my previous articles covering HashiCorp Terraform and OpenTofu, we focused on the architecture around security and state isolation. But we haven’t yet addressed the actual execution engine’s primary feedback mechanism: the plan command. If you have ever run a complex deployment, you are familiar with the wall of text it generates—and the frustration of seeing the phrase (known after apply) repeated across dozens of resource attributes. At first glance, it feels like the engine is hiding the most important data. Dynamic attributes like cloud-generated instance IDs, auto-assigned IP addresses, and generated security certificates cannot be evaluated until the cloud provider actually creates the resource. Consequently, the plan output splits your infrastructure into two distinct categories: the computed variables it must wait for, and the explicit configurations you provided. ...

June 8, 2026 · rathwjj

Securing IaC State: Vault, OpenBao, and KMS

The Flaw in Traditional State Security In Infrastructure as Code (IaC), the state file (.tfstate) is the key to the treasure room. It stores plain-text passwords, API tokens, and private keys. Historically, safeguarding this file meant delegating security entirely to third-party remote storage backends like AWS S3 or Google Cloud Storage (GCS). In many deployments, Terraform state is stored in remote object storage backends such as AWS S3 or Google Cloud Storage (GCS), where encryption is typically handled through Server-Side Encryption (SSE). This introduces a major architectural vulnerability: read access to the bucket equals read access to the raw secrets. If an engineer or a compromised CI/CD worker fetches the state object, the storage layer automatically decrypts it, exposing the entire infrastructure state in plain-text JSON. ...

May 29, 2026 · rathwjj

Proxmox Introduction

We have previously explored how to automate cloud-like infrastructure patterns locally using emulation layers. Now, we will dig deeper into building true, bare-metal local infrastructure. Proxmox Virtual Environment (VE) is a complete, enterprise-grade virtualization solution. Operating as a Type-1 (bare-metal) hypervisor, it installs directly onto raw hardware and comes bundled with its own operating system based on Debian Linux. Out of the box, Proxmox provides a powerful web interface and robust APIs to orchestrate full Virtual Machines (via KVM) and lightweight, isolated system containers (via LXC). ...

May 24, 2026 · rathwjj

Terraform-Proxmox Phoenix Cycle

Note: This is the same Terraform Phoenix Cycle, but replace miniblue.io with Proxmox. This article focuses on transitioning our infrastructure design from local emulation layers to dedicated bare-metal server hardware using Proxmox VE. Unlike localized development tools, Proxmox allows your automated infrastructure to scale seamlessly across physical hardware via Proxmox clustering. By utilizing the native Proxmox Terraform provider (bpg/proxmox), we can replace hacky shell scripts with structured API calls to dynamically spin up, orchestrate, and collapse lightweight Linux Containers (LXC) or full Virtual Machines (VMs). ...

May 24, 2026 · rathwjj

OpenTofu and OpenBao

We talk about Hashicorp has been aquired by IBM. The history after that was HashiCorp Abandoned Open Source (August 2023). In August 2023, HashiCorp shifted its core products (including Terraform) from the open-source Mozilla Public License (MPL 2.0) to the restrictive Business Source License (BSL/BUSL 1.1). Under this license, the source code remained viewable, but competitors were legally prohibited from using it to provide competing commercial services. Because the final open-source versions under the MPL license could not be retroactively revoked, the community and competing vendors revolted. They immediately took the last open-source codebases and forked them: ...

May 18, 2026 · rathwjj

OpenTofu Phoenix Cycle

From the article that we use OpenTofu, this one will replace the code in OpenTofu. This article covers the transition from raw hardware to a functional orchestrator and back to zero. We call this the Provision-Orchestrate-Collapse-Destroy cycle. 1. The Provisioning Phase We start at the VM level. OpenTofu interacts with your provider to reserve CPU, RAM, and storage. Create a Virtual Machine. Use cloud-init to bootstrap the environment. Install the K8s Control Plane (e.g., K3s for efficiency). 2. The Orchestration Phase Once the cluster is “Healthy,” OpenTofu switches to the Kubernetes Provider. ...

May 18, 2026 · rathwjj