Joe Attwood

Joe Attwood

DevOps, Platform & infrastructure engineering, London

Open to Senior / Staff
Notice 3 months
CV (PDF)

/projects/gopg

GoPG

A Go templating engine that replaced a pile of Python and Jinja tooling for generating platform configuration.

  • Go
  • text/template
  • YAML

The problem

Configuration generation had accreted. Several teams each had their own Python-and-Jinja scripts producing manifests and config files, with the template logic, the data, and the orchestration all tangled together. The practical symptoms were the usual ones: no way to validate output before it shipped, no consistent error reporting, and a dependency on whoever wrote the script originally.

I wanted a single binary that could be dropped into a CI job with no runtime to install, that failed loudly on bad input rather than silently rendering an empty string.

Design decisions

Go’s text/template rather than a custom parser. Writing a template language is a tempting mistake. text/template is in the standard library, is well understood, and its strictness — particularly missingkey=error — gives exactly the failure behaviour I wanted. The cost is that its syntax is less ergonomic than Jinja’s, which is a real trade-off I accepted in exchange for zero parser maintenance.

Fail on missing keys by default. Jinja renders a missing variable as an empty string. That behaviour is why several of the original scripts produced subtly wrong config that nobody noticed until deploy. Defaulting to an error means the failure happens in CI, where it’s cheap.

Single static binary, no plugin system. A plugin interface would have made the tool more general. It would also have re-created the original problem — arbitrary per-team logic in an unversioned place — one layer down.

What I rejected

  • Extending the existing Python tooling. Faster in the short term, but it left the distribution problem unsolved: every consumer needed a matching Python environment.
  • Helm for everything. Helm is the right tool inside Kubernetes and we use it there. A lot of what needed generating wasn’t Kubernetes manifests, and bending Helm around that would have been worse than a small dedicated tool.
  • CUE. Genuinely a better fit for the type-safety goal, and I’d look at it again. The barrier was adoption: asking teams to learn a new configuration language to fix a tooling problem is a hard sell.

What I’d do differently

The error messages point at the template but not at the input that caused the failure, which makes debugging a large data file slower than it should be. Threading source position through the render path is the next change.

I’d also have drawn a harder line earlier between “data the tool owns” and “data the caller passes in.” Once callers start stuffing logic into the data file to work around missing template helpers, you are halfway back to the scripting mess you left.

Status

In active use for generating CI and deployment configuration from shared profiles. Distributed as a static binary into pipeline jobs; profiles live in versioned YAML rather than per-team scripts.

My cat, extremely close to the camera, unimpressed

resource "cat" "whiskers" {
  mood    = "unbothered"
  consent = false
}
Apply complete. 1 added, 0 changed, 0 destroyed.