Feature Flag Outages: The Hard Part Is Recovery, Not Downtime

A feature flag outage should be survivable: SDKs evaluate from cached values and fall back to your defaults. The costly failure is an SDK that never reconnects after the service returns.

  • reliability
  • sdks

Every team plans for their feature-flag service going down. Far fewer plan for it coming back. That asymmetry is where the real cost of an outage hides.

A well-built feature-flag SDK makes a control-plane outage close to a non-event for reads. It evaluates flags from an in-memory copy of the last configuration it saw, and anything it cannot resolve falls back to the default you passed in code. Your users keep getting served. Surviving the outage is the easy half.

The expensive half is recovery. When the service comes back, does your SDK reconnect and pick up where it left off on its own, or does it sit in a failed state until someone restarts the process? On 2026-07-10, a major feature-flag provider, LaunchDarkly, had a platform outage of roughly an hour (status page, incident chywwz01ptb0). Flags mostly kept evaluating during it. For some teams the pain came after, when the service was healthy again but their SDKs were not.

This post draws the line between surviving an outage and recovering from one, walks through the three ways an SDK gets stuck, and gives you a four-point checklist for judging any flag SDK’s recovery behavior. Then it shows how Featureflip’s SDKs meet that bar.

Key Takeaways

  • A control-plane outage should be a non-event for reads. A server SDK evaluates flags from an in-memory cache of the last config it saw, so evaluation keeps working while the service is unreachable, and anything it cannot resolve returns the fail-safe default you passed in code.
  • The real failure is not the outage, it is recovery. An SDK that treats a transient error during the outage as fatal stops reconnecting, so the service comes back but the SDK does not, and the only fix is restarting the process fleet-wide.
  • Resilient recovery is one invariant: a running SDK must never permanently stop reconnecting, and must re-sync its full state on every recovery, with no operator intervention.
  • Four properties deliver it: never give up, treat a mid-outage 401/403 as transient rather than fatal, fall back to polling if streaming keeps failing, and re-sync a full snapshot on reconnect so flags changed or deleted during the outage are corrected.
  • Featureflip’s SDKs are built to that invariant, so a Featureflip outage never requires a customer restart. Reads stay on cached last-known-good, reconnects retry forever with backoff, and every reconnect pulls a full config snapshot.

1. Surviving an outage is the easy half

With a server SDK, evaluating a flag is a local, in-memory lookup against a ruleset the SDK already holds. It is not a network call. So when your flag provider’s control plane goes offline, evaluation does not stop: the SDK keeps serving the last configuration it received, and any flag it cannot resolve returns the default you passed at the call site.

Two safety nets make that true. The first is the local cache of last-known-good config. A server SDK streams configuration in the background and keeps a copy in memory, so reads on the hot path never touch the network and never depend on the service being up at that instant. The second is fail-safe defaults: every evaluation takes a default, and a missing flag, a type mismatch, or an evaluation error returns that default rather than throwing. Between the two, a brief outage is usually invisible to your users.

This is table stakes, and it is also where most reliability discussions stop. “We serve cached values during an outage” is true of every mature SDK, and it is not the interesting property. Surviving the outage was never the hard part. Coming back is.

2. The 2026-07-10 outage, and the part that actually hurt

On 2026-07-10, LaunchDarkly had a platform outage of roughly an hour (status.launchdarkly.com, incident chywwz01ptb0). For many teams the survivable half worked as designed: flags kept evaluating on cached values while the service was unreachable, and applications stayed up.

The cost for affected teams landed on the other side of the incident. When the service returned, SDKs that had logged an authentication or connection error during the blip did not all reconnect on their own. The service was healthy, but those SDKs were still sitting in a stopped state, and the fix was a manual application restart, done across the fleet.

That pattern is not specific to one vendor. It is a failure class: any SDK that treats a temporary error during an outage as a terminal condition can land there, and the anti-pattern is common enough that the incident is worth studying rather than dunking on. We are not here to litigate anyone’s internals. The design lesson is what matters, and it applies to every provider, including us. It is the reason teams weighing Featureflip against LaunchDarkly, or scanning LaunchDarkly alternatives, increasingly ask not just “what is your uptime” but “what does your SDK do when the service comes back.”

3. Why an SDK gets stuck

An SDK fails to recover in one of three ways. All three are the same mistake wearing different clothes: something temporary was treated as permanent.

It gives up on a transient error. A 401 or 403 received mid-session during an outage is indistinguishable from a real key revocation. It can be an auth-proxy blip, or the configuration service failing its own key validation closed because a dependency is down. An SDK that reacts by stopping has converted a 60-second wobble into a manual restart. The tell is a log line shaped like this:

[flags] auth error (401): giving up, will not retry

That “will not retry” is the whole bug. The word “permanently” in a client library’s log is almost always an operator’s cue to go restart something.

It bricks on a failed startup. If the process starts during the outage, its first config fetch fails. An SDK that makes a failed initial fetch terminal, rejecting initialization and caching the rejection, serves defaults forever, even after the service recovers, because it never started its background data source at all. Cold-start-during-outage is the nastiest variant, because the SDK looks initialized to your code while being permanently blind.

It reconnects but never re-syncs. This is the subtle one. The SDK reconnects fine, but only resubscribes to future changes. Anything that changed while it was disconnected, or was deleted, stays stale until it happens to change again. The stream shows green and the data is quietly wrong. A kill switch you flipped during the outage might not take effect on a reconnected client that never asked for the current state.

All three are recoverable by design. The next section is the design.

4. What resilient recovery looks like

Resilient recovery is one rule, and everything else follows from it: a running SDK must never permanently stop trying to reconnect, and must re-sync its full state on every recovery, with no operator intervention.

Four properties deliver that invariant. Together they are a checklist you can hold up against any flag SDK, ours included.

  1. Never give up. Reconnection backs off exponentially with jitter, capped at a maximum interval (around 30 seconds), never a maximum retry count. While the client object is alive there is no terminal state.
  2. Transient, not fatal. A 401 or 403 on a client that has run before is treated as a transient auth blip: keep serving last-known-good, log at warning level with a “retrying” framing, and keep retrying. If the key really was revoked, an SDK that harmlessly retries and serves defaults is strictly better than one that goes dark and forces a restart.
  3. Fall back to polling. If streaming keeps failing past a threshold, degrade to a periodic full re-fetch, which itself retries forever. A long outage still gets a corrective refresh instead of an endless reconnect loop.
  4. Re-sync the full state on reconnect. On every reconnect the SDK pulls a full snapshot of config and applies it as a replace, not a merge, so a flag changed or deleted during the disconnect is corrected the moment the stream is back. This is what fixes the silent-staleness failure from the previous section.
Two SDKs through the same outage: one gives up, one self-heals A shared timeline runs left to right through steady state, an outage, and the service returning. The top lane, an SDK that gives up, serves cache during the outage, hits an error, stops, and stays stopped after the service returns, requiring a manual restart. The bottom lane, a self-healing SDK, serves cache and keeps retrying during the outage, then re-syncs a full snapshot the moment the service returns and is healthy again with no restart. Two SDKs through the same outage Illustrative timeline, not measured durations outage begins service returns SDK that gives up × stopped: manual restart required Self-healing SDK serving cache, retrying recovered: no restart healthy, live config serving cached last-known-good stopped full snapshot re-sync on reconnect (replace, not merge)
Both SDKs survive the outage by serving cached values. Only the bottom one recovers on its own: it keeps retrying, then re-syncs a full snapshot the moment the service returns. The top one treated a transient error as terminal and needs a human to restart it.

There is one more property, for the startup case. A non-terminal initialization closes the cold-start-during-outage hole: a failed first fetch still starts the background data source, serves your defaults meanwhile, and self-heals when the service returns, instead of caching the failure and going blind.

5. How Featureflip’s SDKs recover

Featureflip’s SDKs are built to that invariant, so a Featureflip control-plane outage never requires a customer restart. Mapping the four properties to what the SDKs actually do:

Reads stay up. Server SDKs evaluate locally from cached config, so an outage never blocks evaluation, and fail-safe defaults cover anything the SDK cannot resolve. Nothing on your request path waits on the control plane.

Reconnects retry forever. A dropped stream reconnects with capped exponential backoff and jitter, and if streaming keeps failing the SDK degrades to polling, which also retries indefinitely. There is no give-up state and no log line telling an operator to restart.

A transient 401/403 is treated as transient. During an outage, an auth error on a client that was previously healthy does not end the client. It keeps serving last-known-good and keeps retrying, on the assumption that a running client hitting a sudden auth failure is far more likely to be seeing a blip than a real revocation.

Every reconnect re-syncs a full snapshot. On each connect the SDK receives a complete config snapshot and applies it as a replace, so a flag toggled or removed during the outage is correct the instant the stream is back. This holds across the whole server-SDK lineup (JavaScript, Node, Python, Go, Java, C#, and Ruby; PHP polls), and the client SDKs (React, Browser, Swift, Android, and Flutter) get the same full-snapshot treatment on reconnect.

A process that starts mid-outage still comes up. Initialization is non-terminal: the SDK starts, serves your defaults, and self-heals to live config when the service returns, rather than bricking on the first failed fetch.

We did not take this for granted. The 2026-07-10 incident was a useful prompt to re-audit our own SDKs against this exact failure class and confirm they meet the bar, because the “safer than an SDK that needs a restart” claim is only worth making if it is backed by shipped behavior. If you want the mechanics per language, the SDK overview is the place to start, and the SDK feature page covers local evaluation, live streaming, and the direct HTTP fallback.

6. What resilience does not do

Recovery keeps you serving and gets you back to correct state on your own. It is not magic, and three limits are worth stating plainly so you design around them rather than assuming them away.

It cannot serve a flag it has never seen. A brand-new process that has never once reached the service has only your in-code defaults to serve. That is exactly why the default you pass is your real outage behavior for a cold start: choose it as the value you would be comfortable serving to everyone, because during a cold-start outage that is what happens. Last-known-good requires having once been good.

No new configuration propagates during the outage. You cannot flip a flag while the control plane is down. Resilience is about not going dark and recovering cleanly, not about pushing config changes mid-outage. The kill switch you rely on works because the last state you set is already cached in every SDK, not because you can reach the dashboard while it is offline.

It is not a substitute for your own monitoring. An SDK recovering silently is the goal, but your observability should still record that the stream was down and came back. Resilience hides the blast radius from your users. It should not hide the event from your dashboards.

The shorter version

Surviving a feature-flag outage is the easy half: a server SDK evaluates from an in-memory cache and falls back to your defaults, so a brief control-plane outage is usually invisible. Recovering without a restart is the hard half, and it is where the 2026-07-10 LaunchDarkly outage cost some teams the most, when the service returned but their SDKs stayed stopped. An SDK gets stuck three ways: it gives up on a transient error, it bricks on a failed startup, or it reconnects without re-syncing and serves stale data. Resilient recovery is one invariant with four properties: never stop reconnecting, treat a mid-outage auth error as transient, fall back to polling, and re-sync a full snapshot on every reconnect. Featureflip’s SDKs are built to it, so a Featureflip outage never needs a customer restart.

Frequently asked questions

What happens to my app when a feature flag service goes down?

With a server SDK, evaluation is a local in-memory read from cached config, so flags keep evaluating at their last-known-good values while the service is unreachable, and anything the SDK cannot resolve returns the in-code default you passed. A brief control-plane outage is normally invisible to your users, because reads never depend on the service being reachable at that instant.

Do feature flags keep working during an outage?

Reads do: local evaluation plus fail-safe defaults keep serving. What pauses is config changes, because you cannot propagate a new flag value while the control plane is down. The SDK keeps serving the last values it saw and resumes live updates when the service returns.

Why do some feature flag SDKs need an app restart after an outage?

Because they treat something temporary as terminal. A transient 401/403 from an auth blip, or a failed initial fetch at startup, makes the SDK stop reconnecting. The service recovers but the SDK does not, so the only fix is restarting the process. A resilient SDK treats those conditions as transient and keeps retrying instead.

How should a feature flag SDK recover from an outage?

It should never stop reconnecting (exponential backoff with a capped interval, never a capped retry count), fall back to polling if streaming keeps failing, and re-sync its full state on every reconnect so flags changed or deleted during the outage are corrected, all with no operator intervention. A non-terminal initialization closes the cold-start case, where the process started while the service was down.

Is LaunchDarkly reliable?

LaunchDarkly is a mature, widely used platform, and like any cloud service it has occasional outages, for example the roughly one-hour incident on 2026-07-10 reported on its status page. For any provider, the reliability question that matters most is recovery: when the service returns, does your SDK reconnect and re-sync on its own, or does it need a fleet restart? Evaluate that behavior for whatever platform you run. Featureflip’s SDKs are built to recover without a restart.

Featureflip is a focused, flat-priced feature flag platform, and its SDKs are built so a control-plane outage is a non-event: reads keep evaluating from a local cache, fail-safe defaults cover the rest, reconnects retry until they succeed, and every reconnect pulls a full config snapshot so nothing stays stale, all without a restart. Because it uses flat pricing with unlimited evaluations, that local-evaluation model also means an outage or a traffic spike never turns into a surprise bill. Read the SDK overview to see how it works in your language, learn how the SDKs are built for fail-safe evaluation, and start on the free Solo plan without a credit card.