Useful tools for Kubernetes

Most people using Kubernetes extensive have already defined alias k=kubectl and are using tools like kubectx. As someone really lazy though, I have found/developed a few less common tools to help work with Kubernetes efficiently. kubectl apply from clipboard This relies on zsh, and uses the zle to define a custom command. function zle_apply { LBUFFER=" cat <<EOF | kubectl apply -f - $(xclip -se c -o) EOF" CURSOR=31 } zle -N zle_apply; bindkey "^k" zle_apply This defines a function and binds it to Ctrl+k....

July 12, 2023 · 3 min

Tracing shell scripts with OpenTelemetry

A little bit of OpenTelemetry, a lot of shell hackery

July 11, 2023 · 6 min

Zero allocations metrics with opentelemetry-go

In the past, Istio has suffered from performance issues from OpenCensus, which was used for metrics reporting. At extremes, we saw up to 20% of CPU spent just on incrementing various metrics. This was mitigated to some extent by batching metrics updates, optimizing OpenCensus itself, and caching parts of our OpenCensus usage. At best, we got down to roughly 600ns and 3 allocations per metric update. As OpenCensus is now deprecated, I have been looking into migration to OpenTelemetry - and hoping to avoid these issues this time around....

July 11, 2023 · 2 min

Analyzing Go Build Times

Go is often praised for its fast build times. While they are pretty quick, they are slow enough that I spend a lot of time waiting for them, enough that it prompted me to go down the rabbit hole of thoroughly analyzing them. This post covers all aspects of what makes Go builds fast or slow. Throughout this blog, we will use Istio as an example of real-world codebase. For reference on its size:...

 · June 24, 2023 · 26 min

CRD Versioning

How versioning works in Kubernetes, especially with CustomResourceDefinitions, is a common source of confusion. The documentation is pretty comprehensive but a bit complicated. This post aims to give a simple description of how versioning works and dispel some misunderstandings. There is only one primary version Consider a CRD with versions alpha and beta. A user can create and view either resource version. Intuitively, they must be distinct things -- they are not....

June 7, 2023 · 2 min

LTS and Rolling Releases

Across the ecosystem, a variety of software support policies can be found, where "support" can mean, bug fixes, security patches, and sometimes technical support. However, they can be roughly categorized into two types: Rolling release: only latest release supported. At an extreme, this is the HEAD git commit. LTS (Long Term Support): Support latest N versions. Sometimes specific versions have extended support. "Long" is relative: Kubernetes supports a version for around a year, while RHEL does for 10 years....

May 26, 2023 · 8 min

Outbound sidecars are not secure enforcement points

It is a very common misconception that egress policies in Istio can be used for security purposes. This is not true. Despite repeatedly explaining this (and documenting it), I still often see people that do not believe it, and that they can just add one more check to lock things down. In this post, I will show a variety of ways to bypass any possible check, and prove that these policies cannot be used as secure policies....

May 22, 2023 · 4 min

Ergonomic Map in Go

In many languages, on of the things I find myself doing is maping over a list to extract some field. For example, coverting a []Person to []Name. Most languages these days have ways to do this pretty easily: Kotlin: { it.Name } JavaScript: => p.Name) Rust:|p| p.Name) Scala: With generics, Go finally can do this in a type safe manner: Map(people, func(t Person) string { return t....

January 31, 2023 · 5 min

Building Docker Images Fast (by not using Docker)

This post follows the journey Istio has taken over the years to optimize our docker image builds. While there is some useful tips to take out of this, this is not intended to be a guide on how a project should build images - the steps taken here go far past the needs of a typical project, optimizing exclusively for speed (and fun) regardless of the complexity to maintain. For background, over the years Istio has consisted of ~10-20 docker images (many are for tests only) made up of ~10-15 Go binaries and various static files....

January 23, 2023 · 8 min

Kubernetes Operators

Kubernetes Operators are powerful tools when used right, pushing complex operations from human operators to code. This sounds great - and is in some cases - but often the tradeoffs operators expose are not taken into account, both by operator developers and users. This article goes over my take on when operators are useful and not, and what makes a good operator. A basic installation operator A common feature amongst almost all operators is the ability to deploy Kubernetes resources....

December 19, 2022 · 7 min