Most people using Kubernetes extensive have already defined alias k=kubectl and are using tools like kubectx. As someone really lazy though, I have found/developed a few less common tools to help work with Kubernetes efficiently. kubectl apply from clipboard This relies on zsh, and uses the zle to define a custom command. function zle_apply { LBUFFER=" cat <<EOF | kubectl apply -f - $(xclip -se c -o) EOF" CURSOR=31 } zle -N zle_apply; bindkey "^k" zle_apply This defines a function and binds it to Ctrl+k. ...

A little bit of OpenTelemetry, a lot of shell hackery

In the past, Istio has suffered from performance issues from OpenCensus, which was used for metrics reporting. At extremes, we saw up to 20% of CPU spent just on incrementing various metrics. This was mitigated to some extent by batching metrics updates, optimizing OpenCensus itself, and caching parts of our OpenCensus usage. At best, we got down to roughly 600ns and 3 allocations per metric update. As OpenCensus is now deprecated, I have been looking into migration to OpenTelemetry - and hoping to avoid these issues this time around. ...

Go is often praised for its fast build times. While they are pretty quick, they are slow enough that I spend a lot of time waiting for them, enough that it prompted me to go down the rabbit hole of thoroughly analyzing them. This post covers all aspects of what makes Go builds fast or slow. Throughout this blog, we will use Istio as an example of real-world codebase. For reference on its size: ...

How versioning works in Kubernetes, especially with CustomResourceDefinitions, is a common source of confusion. The documentation is pretty comprehensive but a bit complicated. This post aims to give a simple description of how versioning works and dispel some misunderstandings. There is only one primary version Consider a CRD with versions alpha and beta. A user can create and view either resource version. Intuitively, they must be distinct things -- they are not. ...

Across the ecosystem, a variety of software support policies can be found, where "support" can mean, bug fixes, security patches, and sometimes technical support. However, they can be roughly categorized into two types: Rolling release: only latest release supported. At an extreme, this is the HEAD git commit. LTS (Long Term Support): Support latest N versions. Sometimes specific versions have extended support. "Long" is relative: Kubernetes supports a version for around a year, while RHEL does for 10 years. Enterprises love LTS Historically, enterprise users have favored -- if not demanded -- LTS software. ...

It is a very common misconception that egress policies in Istio can be used for security purposes. This is not true. Despite repeatedly explaining this (and documenting it), I still often see people that do not believe it, and that they can just add one more check to lock things down. In this post, I will show a variety of ways to bypass any possible check, and prove that these policies cannot be used as secure policies. ...

In many languages, on of the things I find myself doing is maping over a list to extract some field. For example, coverting a []Person to []Name. Most languages these days have ways to do this pretty easily: Kotlin: people.map { it.Name } JavaScript: people.map(p => p.Name) Rust: people.map(|p| p.Name) Scala: people.map(_.Name) With generics, Go finally can do this in a type safe manner: Map(people, func(t Person) string { return t.Name }) ... but we immediately stand out amongst other languages as having ugly, verbose syntax. ...

This post follows the journey Istio has taken over the years to optimize our docker image builds. While there is some useful tips to take out of this, this is not intended to be a guide on how a project should build images - the steps taken here go far past the needs of a typical project, optimizing exclusively for speed (and fun) regardless of the complexity to maintain. For background, over the years Istio has consisted of ~10-20 docker images (many are for tests only) made up of ~10-15 Go binaries and various static files. We also have a few variants (debug and distroless) and architectures (amd64 and arm64). Aside from CI which is building thousands of these images daily, building images quickly is important for the inner development loop. While I try to run things locally where possible, in many cases each minor code change is built and loaded into a local Kubernetes cluster to more closely resemble a real world deployment. This makes image build time critical for efficient development ...

Kubernetes Operators are powerful tools when used right, pushing complex operations from human operators to code. This sounds great - and is in some cases - but often the tradeoffs operators expose are not taken into account, both by operator developers and users. This article goes over my take on when operators are useful and not, and what makes a good operator. A basic installation operator A common feature amongst almost all operators is the ability to deploy Kubernetes resources. For example, lets consider a hypothetical nginx operator. ...