Istio

Exploring an extreme service mesh architecture to maximize extensibility.

The OSI model attempts to build a model for network communications, where increasingly high level layers are built upon lower layers. This is only slightly useful in practice, as the real world is not so simple. In service mesh, generally discussion is reduced to L4 and L7, or TCP and HTTP. This oversimplifies the problem, leading to some confusion. Thinking in terms of termination Simply saying "HTTP" is not really clear about what is going on. Instead, I think its more useful to think about what layer we terminate. ...

When looking at service mesh (or even general networking) architectures, the basic idea is to send network traffic through some component, which handles various functionality. This could be authentication, authorization, encryption, observability, reliability, networking, etc. There are a few different classes of components that can do this, though: Different types of proxy deployments Native application enhancement. The application itself is compiled in with functionality. This could be something like gRPC (or, even more "meshy", gRPC with xDS), Finagle, Hystrix, etc. Even simply instrumenting your application with metrics and traces could be classified here. "Sidecar", or running a proxy-per-application, is probably the most common service mesh deployment pattern today, used by Istio, Linkerd, and more. Per-node proxy; like sidecar, but instead of per-application the proxy is per-node. Each node contains multiple unique workloads, so the proxy is multi-tenant. Remote proxy. A completely standalone proxy deployment we send some traffic through. This could be correlated to one or many service(s), one or many workload(s), etc -- the correlation between proxies and other infrastructure components is flexible here. Within each of these, there is 2 actors: a client and a server. This gives us 8 points to insert functionality. Presumably, all 8 will not be used at once -- but its possible. If we are willing to blur the lines a bit, even a traditional sidecar based service mesh utilizes 6 of these! The most rich "service mesh" functionality may exist in the sidecar, but the application itself has some functionality (even if its not terribly rich), and the node does as well (again, this may not be terribly rich -- kube-proxy, for example, has very minimal functionality). And the same is mirror on the client and server side. ...

The common messaging around Istio Ambient Mesh is that is a "node proxy." For example, from The New Stack ... architecture that moves the proxy functionality from the pod-level to the node-level. While this is technically accurate, it is misleading and really missing the point and benefits of Ambient. A brief history of service mesh architectures This skips quite a bit of information, but is close enough. One of the earlier service meshes on the market was Linkerd 1 - not to be confused with Linkerd 2, which most people just call "Linkerd" today. Linkerd 1 was a per-node proxy that did all the service mesh functionality we know and love, at the node level. ...

How to achieve an architecture similar to "Waypoint Proxies" without ambient mesh, or even Istio.

It is a very common misconception that egress policies in Istio can be used for security purposes. This is not true. Despite repeatedly explaining this (and documenting it), I still often see people that do not believe it, and that they can just add one more check to lock things down. In this post, I will show a variety of ways to bypass any possible check, and prove that these policies cannot be used as secure policies. ...

When deploying a proxy outside of Kubernetes, users typically reach for solutions like nginx or Traefik; it's pretty rare to see Kubernetes-native load balancers, like Istio, used outside of the Kubernetes context. While it is not documented anywhere on istio.io, it is completely possible to set up an Istio Gateway outside of Kubernetes - not just running out of the cluster, but not connected to any Kubernetes api-server whatsoever. In this post, we will set up Istiod, and Istio ingress gateway, and a test application using docker-compose. The full configuration can be found in howardjohn/local-istio-gateway. Setting up a full mesh outside of Kubernetes is possible, but outside of the scope of this post. ...

The following graph shows PR approvals for istio/istio since 2020. DevA 100 <-> DevB 50 indicates DevB approved 100 PRs from DevA, and DevA approved 50 PRs from DevB. The data was collected from our policy bot data and graphed using flourish: gcloud spanner databases execute-sql main --instance istio-policy-bot --project istio-testing --format=json --sql='select prr.Author as Approver, pr.Author as Author, count(*) from PullRequestReviews as prr join PullRequests as pr on pr.PullRequestNumber = prr.PullRequestNumber where SubmittedAt > "2020-01-01T00:00:00.00Z" and prr.state = "APPROVED" and prr.RepoName = "istio" group by Approver, Author' \ | jq '.rows[] | select((.[2]|tonumber) > 5) | .[1] + "," + .[0] + "," + .[2]' -r If the dynamic graph doesn't work a static version can be found here. ...