Gateway API Cannot Retry. So You Will Write Both.

The standard channel of Gateway API v1.4 has no retry field, and the API server will tell you so rudely — which is why VirtualService is not nostalgia, it is load-bearing. Here is the split nobody explains: what happens to a request on the way, and how you talk to the destination once it is chosen.

You want the mesh to retry a failed request. This is not an exotic requirement. It is possibly the single most common thing anybody has ever asked a service mesh to do.

So you open the HTTPRoute you already have — the one whose parent is the catalog Service, the one the foundations book used to set a timeout — and you add the obvious block in the obvious place. And the API server refuses it:

$ kubectl apply -f catalog-retry.yaml
Error: HTTPRoute in version "v1" cannot be handled as a HTTPRoute: strict decoding error:
unknown field "spec.rules[0].retry"

Not “you spelled it wrong”. Not “that field moved”. The field does not exist. Gateway API ships its surface in channels — standard is the stable, GA API, and experimental is where fields live while they are still moving — and we installed Gateway API 1.4.0 standard, which is what a sane person installs in a cluster they intend to keep. In the standard channel of v1.4, retries have not landed.

So here is the fact that this chapter is built on, and it is worth writing on a sticky note before somebody senior tells you otherwise in a design review:

If you want retries in ambient today, you write a VirtualService.

That is not nostalgia and it is not a preference. It is a gap in the standard, and no amount of enthusiasm for Gateway API closes it. The foundations book flagged this in passing on its way to a resilience lesson. This chapter picks it up and takes it seriously, because the moment you accept that you are going to write a VirtualService, you have to actually understand the classic API — and the classic API is two objects, not one, and almost everybody conflates them.

The honest status of the classic API

Let me set the framing carefully, because there is a lot of bad writing about this and most of it is bad in one of two opposite directions.

Gateway API is where the project is going. That is real. It is a Kubernetes-wide standard rather than an Istio invention, it is implemented by a dozen controllers, the foundations book used it throughout for exactly that reason, and Istio’s own documentation now leads with it. If you are writing a route today, and HTTPRoute can express it, write the HTTPRoute. There is no prize for being the last team on a vendor API.

And VirtualService is not deprecated. It is the older API. Those are different words and the difference matters. It is fully supported, it is what istiod’s configuration model was built around, it does several things the standard channel cannot express, and — as the error message above demonstrates without any need for interpretation — you will be running both, side by side, in the same mesh, for years.

Anybody who tells you Gateway API has replaced VirtualService has not tried to configure a retry. Anybody who tells you Gateway API is a fad has not read the release notes. The correct posture is the boring one: use the standard where it reaches, use the classic API where it does not, know which is which, and re-check the boundary every release, because the boundary moves.

The split: on the way, and on arrival

Now the actual subject, which is the thing I most often see engineers get wrong — including engineers who have been running Istio in production for years and have simply never had cause to say it out loud.

VirtualService is about what happens to a request on the way. It is the itinerary. It sits between “a client asked for this host” and “a destination was chosen”, and everything it can do is a decision about this request:

  • match — on path, on headers, on method, on query parameters
  • route — to a destination, or to several destinations with weights
  • retry — attempt the request again, on these conditions, this many times
  • timeout — give up on it after this long
  • fault — inject an abort or a delay, on purpose
  • rewrite and redirect — change the path or the authority before forwarding it
  • mirror — send a copy somewhere and ignore the response

DestinationRule is about how you talk to a destination once it has been chosen. It is the terms of carriage. It says nothing about any individual request. It describes the relationship between callers and a service:

  • load balancer — round robin, least request, random, consistent hash
  • connection pool — how many connections, how many pending requests, how many requests per connection
  • outlier detection — eject a misbehaving host from the pool
  • subsets — named groups of the service’s pods, selected by label
  • TLS settings — how to originate TLS to this destination

Here is the test that will keep you straight forever, and I want you to apply it the next time you cannot remember which object a field lives in:

Ask whether the setting could be different for two requests in flight at the same instant. If it could, it belongs in a VirtualService. If it could not, it belongs in a DestinationRule.

A retry is a property of a request — this one failed, try it again. A load-balancing algorithm is not a property of a request; it is a property of the pool, and every request being routed at that moment is subject to the same one. A timeout is a request’s own budget. Outlier detection is a running opinion about a set of hosts, formed over many requests, belonging to none of them.

That is the split. It is not arbitrary and it is not historical accident. Once you see it, you stop reaching for the wrong object.

The VirtualService, and the retry that proves it is load-bearing

Here is the object the API server will accept:

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: catalog-retries
  namespace: bookshop
spec:
  hosts:
    - catalog
  http:
    - route:
        - destination:
            host: catalog
      retries:
        attempts: 3
        retryOn: 5xx

Ten lines. And to show it does something, we need an upstream that fails: /debug/fail?pct=50 makes catalog fail half of its requests, interleaved, with no pattern to get lucky against. Forty requests, no VirtualService:

40 requests: 21 x 200, 19 x 500

Nineteen failures out of forty. Exactly what a coin flip predicts, and exactly what a customer sees. Now apply the VirtualService above and change nothing else — the catalog is still broken, the fail switch is still on, the same forty requests:

40 requests: 40 x 200, 0 x 500

Zero failures, against a service that is failing half of everything it is asked to do, at the very moment those forty 200s are being counted. The foundations book used that number to make an argument about resilience, and it spent a long time afterwards explaining why the number is dangerous — retry storms, non-idempotent operations, and the fact that you have just made a broken service invisible to yourself. All of that still holds and I am not going to repeat it.

The reason it is here again is different. This is what the API gap costs you in practice. Those two numbers are separated by ten lines of YAML that Gateway API cannot express in its stable channel. Not “cannot express elegantly” — cannot express. If you had decided, on principle, to use only standard-channel Gateway API in your mesh, you would be looking at nineteen 500s and you would have no way to make them stop.

Note the two fields that people mix up, because they are both spelled host.

spec.hosts is the match: it is compared against the destination the client asked for — the authority of the request, the name it dialled. hosts: [catalog] means this VirtualService applies to requests addressed to catalog. It is not a statement about where the request ends up.

http[].route[].destination.host is the destination: where the request is actually sent. It happens to be catalog here too, which is why the distinction is invisible in this example and lethal in the next one. These two fields can name different services — that is precisely how you route requests addressed to one name into another, and it is the mechanism behind every migration, every rename, and every strangler-fig cutover anybody has ever run through a mesh.

I ran the short-name form above, inside the destination Service’s own namespace, and it behaved. Istio also documents FQDN hosts, cross-namespace resolution rules, and an exportTo field that controls which namespaces a VirtualService is visible in. I did not exercise any of those, and the namespace-resolution rules for short names are exactly the sort of thing that is easy to get subtly wrong and hard to notice — so use FQDNs when you are being careful, and verify the behaviour in your own mesh rather than taking mine.

The DestinationRule, and the thing only it can do

The flagship DestinationRule is outlier detection, and it is the cleanest possible illustration of why the split exists.

Poison one catalog pod out of three — make it return a 500 to everything, while its readiness probe keeps passing. Kubernetes is perfectly happy:

catalog-6c4f598bdb-48wb4  READY=1/1  Running
catalog-6c4f598bdb-ftbgd  READY=1/1  Running
catalog-6c4f598bdb-jxpmc  READY=1/1  Running

Three of three ready, one of them useless. Thirty requests through the Service:

30 requests: 20 x 200, 10 x 500   (~1 in 3 hits the poisoned pod)

And here is the object that fixes it:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: catalog-outlier
  namespace: bookshop
spec:
  host: catalog
  trafficPolicy:
    outlierDetection:
      consecutive5xxErrors: 3
      interval: 5s
      baseEjectionTime: 60s
      maxEjectionPercent: 50

Give it a warm-up — it has to watch the host fail three times before it can act, and those three failures are real requests belonging to real callers — and then run thirty more:

30 requests: 30 x 200, 0 x 500

The pod is still Running. Still Ready. Still in the EndpointSlice. Kubernetes never removed it and does not believe there is anything wrong with it. The mesh routed around it anyway.

Now look at what that policy is, in the terms of the split. It is not a decision about a request. No request carries it, no request is matched by it, and you could not write it into an itinerary if you tried — because it is an opinion about a host, formed by watching many requests, held between them, and applied to whichever request happens along next. It is a property of the pool. That is why it is a DestinationRule and not a VirtualService, and if you understand that one sentence you will never again have to look up which object a field goes in.

The same reasoning covers the rest of the trafficPolicy block, all of which I am going to name and none of which I ran:

  • loadBalancer — ROUND_ROBIN, LEAST_REQUEST, RANDOM, or a consistent hash on a header, a cookie, or the source IP. The consistent-hash mode is the one worth knowing about, because it is how you get session affinity in a mesh — the thing the canary chapter told you a weighted split fundamentally cannot give you.
  • connectionPool — maximum connections, maximum pending requests, maximum requests per connection. This is the circuit-breaker half of the classic API: it is how you stop a caller from opening ten thousand connections to a service that has stopped answering, which is how a slow dependency becomes a dead caller.
  • tls — how to originate TLS to the destination, which matters for external services and for the ones that terminate their own.

I ran outlierDetection. I did not run the other three. Their shapes are from Istio’s documentation, and I am telling you they exist rather than showing you output, because the whole premise of this book is that you can tell the difference between those two things by reading it.

Subsets: the field that makes the two objects need each other

There is one part of the classic API where the split becomes a hard dependency rather than a tidy separation, and it is the part people copy from a blog post without understanding.

A subset is a named group of a Service’s pods, selected by label, and it is declared in the DestinationRule:

spec:
  host: catalog
  subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2

That does nothing on its own. It defines two names — v1 and v2 — and defines what they mean. Then the VirtualService uses them:

  http:
    - route:
        - destination:
            host: catalog
            subset: v1
          weight: 90
        - destination:
            host: catalog
            subset: v2
          weight: 10

Neither object works without the other. The VirtualService routes to a subset it cannot define; the DestinationRule defines a subset that nothing routes to. And the failure mode when you get it wrong is a favourite of Istio support threads: a VirtualService naming a subset that no DestinationRule declares produces requests that go nowhere at all, because the route resolves to a cluster that does not exist.

I did not run subsets in this lab. The canary in the foundations book was a weighted split across two Services through an HTTPRoute, and the header-based canary in the next chapter is a VirtualService matching on a header and routing to a different Service. Both work, both are measured, and neither uses a subset. The YAML above is the documented shape and I am flagging it as such — but the architectural point stands on its own, and it is the reason I am showing it to you: subsets are the clearest evidence that these two objects are two halves of one API. One says which pods count as v2. The other says how much traffic v2 gets.

The rest of the family, and a name collision that will cost you an hour

VirtualService and DestinationRule are the two you will write, but they have siblings in networking.istio.io, and you should at least be able to recognise them when they turn up in somebody else’s repository.

ServiceEntry adds a destination to the mesh’s service registry that Kubernetes does not know about — a database in another VPC, a payment provider, an S3 endpoint. It is the object that makes an external host a mesh service, which is the precondition for doing anything mesh-shaped to it. It is also half of the egress story, and chapter seven is where it belongs — I did not run one, and that chapter says so.

Sidecar is the object that trims what a sidecar proxy is told about, and it exists entirely because of the cost problem from chapter one: by default every Envoy is handed the config for every destination in the mesh, so a large mesh makes every proxy fat. In ambient it is irrelevant — there is no per-pod proxy to trim. It is a fine example of an API that exists to manage a self-inflicted wound.

And Gateway. Here is the collision, and it is genuinely nasty. Istio has a Gateway in networking.istio.io. Gateway API has a Gateway in gateway.networking.k8s.io. Same kind name, different API group, different object, different semantics — and every search result, every Stack Overflow answer, and every LLM you ask will cheerfully mix them. The classic Istio Gateway describes ports and hosts on a gateway deployment and is paired with a VirtualService to actually route anything; the Gateway API Gateway is the one the foundations book used, paired with an HTTPRoute. If somebody hands you YAML with kind: Gateway in it, read the apiVersion before you read anything else. I ran only the Gateway API one, and the sentence you should carry away is not a fact about the classic object’s fields — it is the habit of looking at the API group first.

Who actually enforces any of this

In ambient, the answer is the same as it was for every L7 feature in the foundations book, and by now it should be reflexive: the waypoint.

Both experiments above ran with a waypoint bound to the catalog Service. The waypoint is the thing that retried the 500s — it is an Envoy, it can see the response code, it can decide to try again. The waypoint is the thing that watched the poisoned host fail three times and took it out of the pool. ztunnel could do neither, because ztunnel is L4 and never sees a response code at all.

Which means a VirtualService bound to a Service with no waypoint is in the same family as the fail-open authorization policy — an L7 rule with nothing in the path that can execute it. I want to be careful here, because it would be easy to overclaim: I did not apply a VirtualService to a Service with no waypoint and inspect what Istio reports. I am not going to invent a status condition. What I can tell you, because I watched it, is where the enforcement lives — bind the waypoint, and the rules take effect. Deploy the waypoint first, then write the policy, in that order, every time.

And note what changes when you come from a sidecar mesh, because this catches people mid-migration. In a sidecar mesh, VirtualService and DestinationRule are largely enforced by the caller’s Envoy — the retry happens in the client’s pod, the connection pool is the client’s pool, the load-balancing decision is made at the source. In ambient, the L7 proxy is a waypoint bound to the destination Service. The same YAML, the same intent, and a different machine doing the work in a different place in the network. I ran these against a waypoint and I did not re-run them against a sidecar, so treat that last sentence as reasoning from the architecture rather than a measurement — but it explains a great deal about why a policy that worked in your old mesh appears to do nothing in your new one.

So when is HTTPRoute enough?

Most of the time. That is the honest answer and it deserves to be said before the caveats.

Reach for HTTPRoute — the standard, portable, Kubernetes-native one — when you want path matching, header matching, weighted backends across Services, timeouts, or header rewrites and redirects. The foundations book ran path matching, ran weights (and got a gloriously ugly 93/7 out of a 90/10 split), and ran timeouts, all through HTTPRoute, all without a line of Istio-specific YAML. If that is your requirement, you are done, and you have written configuration that would survive swapping Istio out for another implementation entirely — which is a real and underrated form of insurance.

Reach for VirtualService when you want a retry (verified missing above), fault injection (the next chapter’s whole subject), or the richer matching and rewriting that the classic API has accumulated over the better part of a decade.

Reach for DestinationRule for anything about the connection rather than the request: outlier detection, load-balancing algorithm, connection pools, TLS origination, subsets. Gateway API’s story for these lives in its policy-attachment machinery, which is still forming — I did not survey which of its extension resources have graduated to standard, and I am not going to tell you the state of an API I did not install. Check it yourself for your version, and expect the answer to be different in a year.

Two more things I want to flag as unknown rather than paper over, because they are exactly the questions you will have:

What happens if an HTTPRoute and a VirtualService both apply to the same Service? I did not test it. I do not know the precedence, and I am not going to guess at it in print. Until you have tested it on your own mesh, the safe discipline is one API per Service — if a Service needs a retry, put all of that Service’s routing in the VirtualService, rather than splitting it across two objects and hoping they compose.

What happens if two VirtualServices claim the same host? Also not tested here. It is a well-known source of confusion in Istio deployments and it has bitten a great many people. istioctl analyze is your friend — it found real defects in my own manifests, it exits non-zero, and it costs nothing to run in CI.

Final thoughts

There is a particular failure of engineering culture that this chapter is really about, and Istio is only its current venue.

A standard gets announced. It is genuinely better — more portable, better designed, owned by a community rather than a vendor. And within about a week, the older API is being described as legacy in design documents, deprecated in code review comments, and embarrassing in conference talks, by people who have read the announcement and not the API reference. Then somebody actually tries to ship something, and discovers that the standard cannot do a thing their service needs — something as unglamorous, as universally required, as retrying a failed request — and now they are in the worst possible position: using the old API, in a codebase whose conventions call it forbidden, apologising for it in every review.

None of that had to happen. The old API is not a moral failing. It is the API that has the field in it.

So write both, deliberately, and document which is which. Leave a comment at the top of the VirtualService that says: this is here because HTTPRoute v1.4 standard has no retry field, and when it does, delete this. Then, at every Istio and Gateway API upgrade, go and check whether that comment has become false — because the boundary is genuinely moving, in your favour, and one day the standard will have retries and this object will be pure debt. That day is not today. Today the standard channel will reject your YAML with a strict decoding error, and the mesh will happily fail nineteen requests out of forty while you look for a field that is not there.

The next chapter is about the things you can do with a VirtualService that no standard has any intention of giving you — starting with breaking your own service on purpose, at a percentage you specify, and watching the numbers come back almost exactly where you asked for them.

Next: The 503 That catalog Never Sent

Comments