When Reliability Layers Hide a Weak Vendor

Reliable systems expect dependencies to fail. Timeouts, retries, circuit breakers, and fallbacks are standard engineering tools. But there is a point where resilience stops protecting the product and starts compensating for a supplier that should be replaced.

That point is easy to miss because every individual fix looks reasonable.

The reliability tax

Imagine an API that occasionally times out. The team adds a retry. Requests then arrive twice, so it adds idempotency. Longer outages create a backlog, so it adds a queue. Support needs answers, so it builds a delivery log and replay tool.

The vendor bill has not changed. Your internal cost has.

The reliability tax includes engineering time spent around the dependency, additional infrastructure, customer support caused by ambiguous failures, and slower feature delivery.

Resilience or compensation?

A circuit breaker used across external APIs is an architectural capability. A bespoke reconciliation service required to make one vendor usable is a purchasing problem.

Measure the whole dependency

Vendor reviews often focus on price and uptime. Add four internal measures:

1. incidents attributed to the dependency;
2. engineering hours spent on workarounds;
3. support cases caused by its behavior;
4. complexity that would disappear after a migration.

These numbers turn a vague frustration into a decision. A more expensive supplier can be cheaper if it removes a layer your team currently owns.

Keep an exit path

Wrap important dependencies behind a narrow interface, store your own identifiers, and document what a migration would require. This is not an argument for abstracting every service. It preserves leverage where failure would materially affect the product.

Resilience is valuable. But when the resilience layer becomes a permanent product of its own, review the vendor before adding another feature to the workaround.