howardjohn's Blog

While typical HTTP request routing happens based on the request headers/path, sometimes routing based on the body is useful. In particular, as AI use cases become more prevalent, the need to route based on the model field of the JSON request body is pretty handy to serve multiple models or route to different external LLMs. The Gateway API Inference Extension solution to this is to deploy an "external processor" that reads the body, and writes out a header; the header can then be matched as usual. An external processor is basically a service independent of the actual gateway proxy that processes the request. It does this by streaming the request headers/body between the proxy and the external processor. ...

How I run Agentgateway on my homelab setup.

Common Expression Language (CEL) is a great little language for embedding users' custom logic into an application. However, typically when I discuss it, I get a response something along the lines of "AHHHH!! I HATE CEL!!!" I, too, have been in that position. However, when building Agentgateway I decided to fully embrace CEL throughout the entire stack, and the results have been great. The problems with CEL are not about CEL itself, but in its usage -- and a lot of usages are poor, giving CEL a bad reputation. ...

I've spent the majority of my career building Istio's control plane, Istiod, with an emphasis on making it highly performant and scalable. And while it has come a very (very) long way, it's still a long way off from what a control plane could be. It's not alone. When I worked on building an open benchmark of Kubernetes Gateway control planes, I was surprised to find that no implementation met what I felt was a reasonable bar for a "highly scalable and performant control plane". ...

An often overlooked part of the Rust license is that within a year of usage, users are required to make a comparison to their (previously) favorite language. While I am a bit late, I am ready to pay my dues. Given the overabundance of Rust vs X content, I'll try to cover only areas that haven't been discussed to death. Lifetimes and Borrows Did I say I wouldn't cover areas everyone has already talked about? I lied! ...

In the past few years, CPUs have gotten really fast. Shockingly fast! Yet most people are stuck on previous generation mobile chips (whether by choice, or by their companies choice), at a huge detriment to their productivity. Meanwhile, AI coding subscriptions like Cursor are all the rage these days. I'll skip the debate on exactly how useful these tools are, and focus on the pricing. Cursor is $480/year for the team plan (the cheapest corporate plan), and other providers are around the same, setting a clear price point: engineering productivity is worth at least $500/year. ...

AI workloads introduce new requirements on networking infrastructure, but the same core requirements that service meshes solve not only remain, but are exacerbated. Your AI workloads still need a service mesh - they just need a better one

Gloo Mesh’s ambient multi-cluster mode sets a new benchmark for scalability.

Go 1.24 introduces new support for "Tools", which allows easy consumption of tools (which are written in Go) as a dependency for a project. This could be anything from golangci-lint to protoc-gen-go. In this post, I will cover usage and limitations. Basic usage Adding a tool to a project is nearly the same as a standard runtime dependency, with the additional -tool flag: $ goimports # I don't have goimports yet! zsh: command not found: goimports $ go get -tool golang.org/x/tools/cmd/goimports go: added golang.org/x/mod v0.22.0 go: added golang.org/x/sync v0.10.0 go: added golang.org/x/tools v0.29.0 $ go tool goimports --help usage: goimports [flags] [path ...] Once we add a tool, we can access it by go tool <name>. ...

Is it a per-node proxy? Is it a sidecar? The reality is somewhere in between