LLM Workflows Need Failure-Aware Circuit Breakers, Not Just Retries

A common first response to a failing LLM workflow is simple:

The call failed. Retry later.

That is a reasonable pattern for temporary failures. Rate limits, network timeouts, provider overload, and transient API errors often recover after a cooldown.

But LLM systems introduce another category of failures where retrying does not improve reliability.

Examples include:

• malformed model output
• prompt and schema mismatch
• parser contract failure
• wrong model configuration
• missing source data
• unsupported workflow path
• report formatting failure
• persistence failure after generation

When these failures repeat, more retries do not make the system safer. They consume more tokens, create messy job states, frustrate users, and may charge users for a flow that cannot complete.

For LLM products, especially paid workflows, failure handling is not only a technical reliability problem. It is also a cost, trust, and operations problem.

Context and operating problem

Consider a paid report generation flow.

A user clicks Generate Report. The backend calls the LLM. Tokens are consumed. The model may even return a response.

Then something fails after the model call:

• parsing fails
• report formatting fails
• final output cannot be saved
• required source data is missing
• the workflow reaches an unsupported branch

From the user's point of view, the report was not delivered.

From the system's point of view, the LLM cost has already been paid.

Now the product may need to refund the user, restore their credit, or preserve partial output for support review. If the user keeps retrying the same broken flow, the business keeps losing money while the user still does not receive a reliable result.

That is why retry alone is not enough.

Common mistake or failure mode

The mistake is treating all LLM failures as transient failures.

A normal retry strategy usually assumes that the same request may succeed later if the system waits long enough. That works for temporary conditions:

• rate limit exceeded
• provider timeout
• network failure
• temporary service overload
• short-lived dependency issue

But many LLM workflow failures are structural. The same input, prompt, schema, model, parser, and downstream workflow will likely fail again until something changes.

Examples:

• the prompt does not produce the schema the parser expects
• the selected model is not suitable for the requested output format
• source data is incomplete
• the workflow does not support this report type
• formatting logic cannot handle a valid but unexpected response shape
• the final persistence step fails after tokens have already been consumed

Retrying structural failures can make the incident worse. It increases token spend, duplicates failed jobs, creates unclear user state, and makes refund or credit logic harder to reason about.

Better architecture direction

LLM workflows need a failure-aware circuit breaker.

The circuit breaker should not only count failures. It should classify failures and decide the next safe action.

A useful circuit breaker should be able to answer:

• Can this failure be retried automatically?
• Does this failure need a cooldown window?
• Should this user, report, or job be locked after repeated attempts?
• Should this report type or feature be disabled globally until reviewed?
• Should the user receive a credit reversal or refund path?
• Is there completed output that should be preserved for user access or admin review?

This is especially important when the workflow involves credits, subscriptions, paid reports, or real money.

The goal is not only safe retry. The goal is controlled recovery.

Failure classification model

A simple starting point is to divide failures into categories.

| Failure type | Example | Suggested action | | --- | --- | --- | | Transient provider failure | Timeout, rate limit, overload | Retry with cooldown and attempt limit | | Contract failure | Output does not match schema | Stop automatic retry, mark for review | | Configuration failure | Wrong model, wrong prompt version, wrong parser | Disable affected workflow until fixed | | Data failure | Missing required source data | Stop job and show actionable state | | Downstream failure | Formatting or save failed after LLM response | Preserve generated output if possible, retry only safe downstream step | | Repeated user/job failure | Same report fails multiple times | Lock job/report state and require admin review | | Cross-user feature failure | Same report type fails for multiple users | Disable feature globally for investigation |

This classification matters because each category has a different recovery path.

A timeout can be retried. A schema mismatch should usually not be retried endlessly. A missing source-data problem should be shown clearly. A repeated report-type failure across multiple users should become an operational alert, not a silent retry loop.

Reference implementation notes

One practical approach is to treat every generation as a stateful job.

The job should track:

• current status
• failure category
• retry count
• token cost or provider request metadata
• user credit transaction
• report type
• prompt version
• model configuration
• parser version
• source-data snapshot or reference
• generated output, if any
• refund or credit-reversal status
• admin review status

This gives the system enough context to make safe decisions after failure.

Example job states:

Queued
Running
LLMCompleted
ParsingFailed
FormattingFailed
SaveFailed
Completed
RetryScheduled
LockedForReview
CreditReversed
FeatureDisabled

The important detail is that LLMCompleted and Completed are not the same thing.

The LLM may return a response, but the user has not received value until the report is parsed, formatted, saved, and made available.

Example decision policy

A failure-aware circuit breaker can be implemented as a policy layer around the workflow.

public enum FailureCategory
{
    TransientProviderFailure,
    ContractFailure,
    ConfigurationFailure,
    SourceDataFailure,
    DownstreamPersistenceFailure,
    UnsupportedWorkflow,
    Unknown
}

public enum RecoveryAction
{
    RetryWithCooldown,
    StopAndMarkFailed,
    LockJobForReview,
    DisableFeatureForReview,
    ReverseCredit,
    PreserveOutputAndRetryDownstreamStep
}

public sealed record FailureDecision(
    FailureCategory Category,
    RecoveryAction Action,
    string Reason);

public sealed class LlmFailurePolicy
{
    public FailureDecision Decide(GenerationJob job, WorkflowFailure failure)
    {
        if (failure.IsRateLimit || failure.IsTimeout)
        {
            if (job.AttemptCount < 3)
            {
                return new FailureDecision(
                    FailureCategory.TransientProviderFailure,
                    RecoveryAction.RetryWithCooldown,
                    "Temporary provider failure within retry limit.");
            }

            return new FailureDecision(
                FailureCategory.TransientProviderFailure,
                RecoveryAction.LockJobForReview,
                "Temporary failure repeated beyond retry limit.");
        }

        if (failure.IsParserContractFailure)
        {
            return new FailureDecision(
                FailureCategory.ContractFailure,
                RecoveryAction.LockJobForReview,
                "Model output does not match parser contract.");
        }

        if (failure.IsMissingSourceData)
        {
            return new FailureDecision(
                FailureCategory.SourceDataFailure,
                RecoveryAction.ReverseCredit,
                "Required source data was missing before report delivery.");
        }

        if (failure.IsSaveFailure && job.HasGeneratedOutput)
        {
            return new FailureDecision(
                FailureCategory.DownstreamPersistenceFailure,
                RecoveryAction.PreserveOutputAndRetryDownstreamStep,
                "LLM output exists; avoid another model call and retry safe downstream persistence.");
        }

        if (failure.IsUnsupportedWorkflow)
        {
            return new FailureDecision(
                FailureCategory.UnsupportedWorkflow,
                RecoveryAction.DisableFeatureForReview,
                "Workflow path is not supported for this report type.");
        }

        return new FailureDecision(
            FailureCategory.Unknown,
            RecoveryAction.LockJobForReview,
            "Unknown failure should not retry endlessly.");
    }
}

This is only a reference pattern, not a complete implementation. The main idea is to avoid using one retry strategy for every failure mode.

Cost and credit handling

Paid LLM workflows need a clear boundary between provider cost and user value.

The system may spend tokens before the user receives a completed report. That means credit handling should be tied to delivery state, not only to provider invocation.

A practical credit model can separate these events:

• credit reserved when the job starts
• LLM cost recorded when the provider call completes
• credit captured only when the report is delivered
• credit reversed when the report cannot be delivered because of system failure
• generated output preserved when it exists and can still be recovered

This makes the product easier to support. It also avoids charging the user for a broken internal workflow.

Operational controls

A failure-aware circuit breaker should also work above the individual job level.

For example:

• if one user's report fails repeatedly, lock that job or report for review
• if one report type fails across multiple users, disable that report type globally
• if one prompt version causes parser failures, roll back or block that prompt version
• if one model configuration causes repeated malformed output, remove it from routing
• if source-data issues affect a workflow, stop accepting new jobs until the dependency is fixed

This is where the circuit breaker becomes an operations tool, not just a retry utility.

Tradeoffs and constraints

A failure-aware circuit breaker adds complexity. It requires better failure classification, clearer job states, admin visibility, and careful credit handling.

But the alternative is worse: endless retries, unclear user state, repeated token spend, and support teams trying to reconstruct what happened after the fact.

There are also product decisions to make:

• Which failures deserve automatic credit reversal?
• Which failures should preserve partial or completed output?
• Which failures should be visible to the user?
• Which failures should only be visible to admins?
• When should a feature be globally disabled instead of letting more users hit the same broken path?

These decisions should be designed before the product reaches real payment flows.

Checklist for review

When reviewing an LLM workflow that involves credits, reports, or subscriptions, I would check:

• Are transient failures separated from structural failures?
• Is there an attempt limit for automatic retries?
• Does the system avoid another LLM call when only a downstream step needs retry?
• Are job states explicit enough for support and admin review?
• Is user credit captured only after successful delivery?
• Can the system reverse or restore credit when the product fails to deliver?
• Are prompt version, model configuration, parser version, and report type stored with the job?
• Can repeated failures lock a specific job, report, or user flow?
• Can repeated cross-user failures disable a report type or feature globally?
• Is partial or completed LLM output preserved when it can help recovery?
• Are admin review screens designed for failure investigation, not just log browsing?

Closing thought

Retries are useful, but they are not enough for production LLM workflows.

A retry policy answers one question:

Should we try the same thing again?

A failure-aware circuit breaker asks better questions:

What failed?

Is retry safe?

Has the user already paid?

Did we already spend tokens?

Can we recover without another model call?

Should this workflow be stopped until someone reviews it?

That distinction matters.

In paid LLM systems, reliability is not only about getting the model call to succeed. It is about protecting cost, user trust, and operational control across the full workflow.