Reliable AI Agent Workflows: Retries, Idempotency and Business Value

Define the state machine before the conversation

The client’s service desk turns its exception-handling problem into explicit states and transitions.

A chat transcript is not a durable workflow model. It does not reliably tell a recovering worker whether a credit was merely proposed, approved, submitted, or confirmed. Define those states explicitly and store transitions transactionally. For this example, a request might move through received, gathering, proposed, awaiting approval, executing, and completed, with separate rejected, expired, and needs-review outcomes.

Persist a workflow identifier, input version, current state, proposal hash, policy version, and references to tool results. Use optimistic concurrency or an equivalent transactional mechanism so two workers cannot independently advance the same revision. Keep large or sensitive payloads in access-controlled storage rather than copying every result into the conversation and event history.

The model can classify a request or draft a proposal, but ordinary application logic checks whether the proposed transition is legal. A generated statement that approval was granted is not approval. Durable workflow engines can help with timers and recovery; they do not replace an authorization model or turn an unreliable external side effect into an exactly-once operation.

Treat retries as a side-effect problem

The timeout arrives after a possible side effect. Retrying safely requires more than asking the model to try again.

A timeout creates uncertainty, not evidence of failure. The adjustment API might have accepted a write before the connection broke. Blindly retrying with a new identifier can duplicate the credit. Derive an idempotency key from the stable business operation and approved proposal version, then reuse it for every attempt. The receiving system must enforce that key atomically with the effect.

Store the request fingerprint and result against the key, reject reuse with a different payload, and retain deduplication records for the entire retry horizon. A local completed flag is insufficient: a worker can crash between the external write and the flag update. If the downstream API offers no idempotency support, use a lookup or reconciliation path; do not claim exactly-once behavior from a local database alone.

Temporal documents this distinction clearly: an activity may execute multiple times even when its completion is recorded once. Bound retries by attempt count and deadline, distinguish transient failures from permanent validation errors, and send uncertain results to reconciliation. Cancellation must also define what happens to an already submitted operation.

Put authority in tools and approval records

The team preserves exact action authority while asking which workflow failures cause avoidable handling effort.

Expose narrow tools such as read-account-summary or propose-credit rather than unrestricted shell, SQL, or HTTP access. Validate arguments against a schema and authorize the target account using trusted request context. Model-supplied account identifiers are inputs to check, not proof of access. Limit result sizes, destinations, timeouts, and cumulative tool calls so one task cannot consume unbounded resources.

Bind human approval to the exact proposed action, target, amount, currency, policy context, and expiry. If the proposal changes, require new approval. Revalidate permissions and relevant account state immediately before execution to prevent a stale approval from authorizing a different situation. The following record illustrates those bindings; it is not a complete authorization implementation.

Separate read credentials from write credentials. Release a narrowly scoped execution capability only after approval checks pass, and avoid giving secrets to the model. Log the authorization decision and external operation identifier without retaining unnecessary customer text.

{
  "workflow": "service-credit-request-42",
  "state": "awaiting_approval",
  "proposalVersion": 3,
  "action": "issue_service_credit",
  "approvalBinding": [
    "proposalHash", "accountId", "amount",
    "currency", "approverId", "expiresAt"
  ]
}

Treat retrieved text as potentially hostile

Retrieved text joins the context, including instructions it has no authority to give. Prompt injection belongs in the threat model from the start.

Support tickets, documents, and tool responses are untrusted data even when they are useful evidence. A sentence inside a ticket telling the assistant to bypass approval must not become a workflow instruction. Keep retrieved content separate from privileged policy, preserve provenance, and make permission checks outside the model. Prompt wording alone is not a security boundary.

Constrain outbound destinations and redact sensitive fields before model submission. A read-only tool can still leak data if its output is forwarded to an arbitrary URL or copied into a public response. Apply tenant isolation to retrieval, verify source references, and require the draft to distinguish known facts from missing information.

Bound reasoning and execution independently. Exhausting a token or tool budget should lead to a clear incomplete outcome, not a fabricated success message. If a dependency is unavailable, preserve the work already gathered and explain the missing prerequisite. Do not repeatedly ask the model to solve a permissions failure that requires an authorized human decision.

Evaluate transitions and effects, not eloquence

Only accepted outcomes, adverse-path tests and observed exception effort can validate the 100-hour capacity difference.

Build evaluation cases around observable outcomes: duplicated requests, changed proposals after approval, cross-account access, hostile retrieved instructions, timeouts after accepted writes, and expired approvals. Check that unauthorized writes never occur, legitimate approved operations complete, and uncertain operations enter reconciliation. A convincing explanation attached to the wrong account is still a failure.

Use controlled fixtures for repeatable fault injection, then shadow the read-and-propose path against representative authorized inputs before permitting writes. Track task correctness, unnecessary approval requests, escaped policy violations, repeated effects, review burden, latency, and cost per successful workflow. Retain versioned prompts, models, tools, and evaluation evidence so a regression can be attributed and reversed. Reliability comes from the combined system boundary, not from a promise that the model will always follow instructions.