Posts

In my last post, The hidden cost of mpsc channels, I discussed a common pitfall with Rust mpsc channels using more memory than intuitive (and subsequently got roasted by people pretending they read the footnotes of the documentation of every transitive dependency they use 🙂). I recently stumbled upon a similar surprising cause of memory utilization in Go, this time in sync.Pool. sync.Pool's "purpose is to cache allocated but unused items for later reuse, relieving pressure on the garbage collector." Notice the goal is not to reduce memory usage, but to reduce allocations. However, without being familiar with the implementation details, it's easy to assume that it doesn't increase memory usage much. ...

Recently I have been spending a lot of time analyzing and optimizing memory usage in our Rust reverse-proxy, agentgateway. One thing that repeatedly came up was a surprisingly large amount of memory allocated to innocent-looking Tokio mpsc channels. In my naive understanding, I would have assumed the following allocation pattern: struct BigStruct { data: [u8; 1024], } fn main() { // Allocates ~1024 bytes let _ = tokio::sync::mpsc::channel::<BigStruct>(1); // Allocates ~1024*1024 bytes let _ = tokio::sync::mpsc::channel::<BigStruct>(1024); } However, in practice both of these are wrong: they each allocate 32kb! ...

When trying to use any Kubernetes CRD (or even core types) I am pretty much always pulling up a reference doc of some sort to see what fields are available and what they do. However, I have pretty much always found the options for this to be pretty bad. To remedy this, I have been working on a new tool that automatically generates documentation for Kubernetes CRDs, and I wanted to share it here. ...

This is part two of a series. Envoy Quirks Part 1: Clear Route Cache Envoy Quirks Part 2: Filter Chain Match (this post) One of Envoy's core features is, of course, its ability to match traffic and route it to the appropriate destination. This is done at two levels generally: Filter Chain Matches define the top level matching of traffic, matching on attributes of the TCP and TLS handshake like port and SNI. Route Matches define the matching of HTTP traffic, matching on attributes of the HTTP request like path and headers. Filter chain matchers are notoriously tricky to get right, and have become a sort of rite of passage for Envoy users to learn how to use them correctly. Lets dig in. ...

After years of working on a project running our own testing infrastructure, I have been working on Agentgateway utilizing the free GitHub Actions runners. While the free tier is quite generous, the performance is... not. As someone who has spent a lot of time optimizing build times, I found myself quite disappointed that after going through considerable lengths to optimize our workflows, our builds were still severely bottlenecked by the performance of the GitHub Actions runners. ...

The go.mod file contains a mandatory go <version number> directive. Since Go 1.21, when a change was introduced to make this include a full patch number (1.21.0 instead of 1.21), a number of projects have started using this wrong, hurting everyone. The version is the minimum version your project can be compiled with. It is not the version you use to compile your project, but the minimum version that anyone can use to compile your project. ...

With the retirement of ingress-nginx I've seen a lot of frustrations with Gateway API migrations due to differences in some of the resource models. The common problem is that Ingress users are often running self-service models, where application teams fully own their ingress configuration, including TLS certificates. This would look something like so: apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: cert-manager.io/cluster-issuer: my-issuer name: app-a-routes namespace: app-a spec: rules: - host: app-a.example.com http: paths: - pathType: Prefix path: / backend: service: name: myservice port: number: 80 tls: - hosts: - app-a.example.com secretName: myingress-cert Combined with cert-manager a certificate would automatically be provisioned for app-a.example.com and linked up to the shared Nginx instance. DNS would then be handled with wildcard entries or external-dns. ...

This is part one of a series. Envoy Quirks Part 1: Clear Route Cache (this post) Envoy Quirks Part 2: Filter Chain Match Over the years working with Envoy (via Istio), I've come across quite a few quirks and gotchas. I thought it would be fun to share some of them, and how to work around them. Many of these surprise even Envoy experts! To start things off, lets talk about the "Clear route cache" option present on a number of Envoy filters. ...

Recently I had a need to run a bunch of steps in a bash script, with interweaved dependencies for a CI pipeline. While a simpler approach would do a linear flow like: Setup Kubernetes cluster. Deploy dependencies to Kubernetes. Build image 1. Build image 2. Run tests. could work, its slow; we don't take advantage that we could build our images while the cluster is setting up, for instance. Essentially what we want is to be able to run a bunch of steps, declare their dependencies, and have them run as soon as their dependencies are met. This is essentially a DAG (directed acyclic graph) execution. ...

When building out Agentgateway, we had a desire to introduce an embedded expression language to allow users to write custom logic to be evaluated at runtime. This is tremendously useful for a variety of use cases, such as: Extracting fields to log (request.headers["user-agent"]). Evaluating authorization conditions (jwt.sub == "admin" || request.path == "/public"). Manipulating fields in requests/responses (x-llm-model: 'json(request.body).model'). and so on. This provides a powerful way to allow users to customize behavior without needing custom compile-time extensions, external callouts, or complex YAML-based configuration. ...