Kubernetes

While typical HTTP request routing happens based on the request headers/path, sometimes routing based on the body is useful. In particular, as AI use cases become more prevalent, the need to route based on the model field of the JSON request body is pretty handy to serve multiple models or route to different external LLMs. The Gateway API Inference Extension solution to this is to deploy an "external processor" that reads the body, and writes out a header; the header can then be matched as usual. An external processor is basically a service independent of the actual gateway proxy that processes the request. It does this by streaming the request headers/body between the proxy and the external processor. ...

A deep dive into the termination behavior of Istio ambient mode and how it impacts your applications.

Over 2 years ago, I started working on some ideas to build better Kubernetes controllers. In this post, I wanted to give a bit of a retrospective on how things have gone since then. Over the years working on Istio and other projects, I observed a number of major issues with controllers: Most code was about error-prone event handling and state reconciliation, rather than business logic. Most tests, in turn, were about the same. This, in turn, made the code extremely complex, brittle, and often incorrect. This complexity lead to user facing compromise: incorrectness and performance issues. You might argue I should just write a better controller that is faster and without bugs. Maybe, but probably not. ...

JetBrains IDEs (IntelliJ, GoLand, etc) have a nifty feature called Language Injection that lets you get full language features when a language is embedded within another. For example, a SQL query within a string within a Go file. A few of these come out of the box, but they are pretty limited -- I only had some XML ones prior to enabling the Databases plugin which added a few SQL ones. Fortunately, there is the ability to add custom ones. Unfortunately, this is expressed in a proprietary language with, as far as I can tell, zero documentation. ...

An overview of options to deploy mTLS on Kubernetes

Core problems with the API make it a challenging to use in a secure, scalable manner.

Zero trust architectures should not treat nodes as highly privileged components.

Sidecarless? Why not podless?

How Istio tests its networking proxy without Kubernetes, Docker, or root.

Like most other Kubernetes controllers in, Istio is written in Go and relies on the client-go library. While this provides an excellent low-level building block, usage in higher level code in Istio led to a variety of issues that led us to develop our own higher level, opinionated client for Istio. This post covers the issues we faced and how we incrementally solved them. Background knowledge At a high level, client-go provides a few layers for interactions with the API server: ...