Two agents can work in parallel and still corrupt a workflow. One writes a new answer while another works from an older snapshot; both save to the same state field; the later write silently erases the earlier one. An external send can also complete just before its worker crashes, then happen again on retry. These are state ownership problems, not arguments for a particular model or a larger tool catalog. F3Fundit’s MCP introduction covers connecting tools, while its orchestration-versus-agentic guide discusses control flow and briefly names state machines; this guide narrows the lens to versioning, handoff, and recoverable effects.[4][5]
Give every piece of state an owner
Take a workflow that drafts a customer-facing report from two independent inputs. The research worker owns research_artifact; the calculation worker owns calculation_artifact. Neither may overwrite the other’s output. A coordinator owns run_status, version, and the join of those artifacts. A human reviewer approves an immutable candidate snapshot of that join, not a mutable working copy or a future version that does not yet exist. Each artifact carries its source references, input-version reference, timestamp, and a content hash. These are proposed records, not a tested production implementation.
Store the canonical run record in a durable database. Store bulky documents separately, with access control and a stable reference in the record. Make transitions explicit: received → researching/calculating → ready_for_review → approved → dispatching → sent, with failed, revoked, and needs_reconciliation branches. A stale worker may submit its own immutable artifact, but it must not claim the coordinator’s version has advanced without a conditional update. Define expiry, cancellation, and manual repair before adding autonomous retries.
A graph checkpointer and a business ledger are different things. LangGraph describes checkpoints for thread-scoped graph state and a store for application-defined cross-thread data. A resumed conversation or checkpoint does not automatically prove that a customer was billed or a report was sent; your application must record those facts independently.[18]
Freeze a v7 candidate before approving v8
Imagine run R-42 at version 7, status ready_for_review. Research and calculation both start from the expected v7 inputs and save separate, immutable artifacts: for example research_artifact_id=R-42:research:7:a with research_hash=sha256:<research-digest>, and calculation_artifact_id=R-42:calculation:7:b with calculation_hash=sha256:<calculation-digest>. The digest placeholders illustrate fields, not computed hashes. Under a coordinator transaction, lock R-42, verify each artifact’s input-version references and expected inputs, and freeze review_snapshot_id=R-42:review:v7:c1: a manifest containing both exact IDs, both content hashes, both input-version references, and the intended destination/payload reference (or its digest). Store its own canonical manifest hash. The run’s active snapshot pointer and v7/ready status must be committed together; do not overwrite that snapshot or a referenced blob in place. A changed byte, destination, or input requires a new snapshot ID/hash and review. PostgreSQL’s SELECT ... FOR UPDATE blocks competing updates to a locked row until the transaction ends.[20]
The reviewer decision names R-42:review:v7:c1 and its manifest hash, with reviewer identity, decision time, and an active/not-revoked state. It approves this frozen v7 candidate for a proposed v8 send; it does not claim that v8 already exists. Give approval and revocation serialized database transitions against the run/snapshot so a late approval cannot silently supersede a replacement. At the v8 commit, lock the run, require version=7, status=ready_for_review, active_snapshot_id=R-42:review:v7:c1, and the exact manifest and artifact hashes; check that the approval for that same ID/hash is active and that the referenced objects still match. If anything differs, abort and require a fresh snapshot/review. If another coordinator has advanced the run, reload and reconcile rather than doing last-write-wins.
A compact transaction outline—not copy-paste SQL for an unspecified schema—is:
BEGIN
lock run R-42; require version = 7 and status = ready_for_review
require active snapshot = R-42:review:v7:c1 and manifest hash matches
require immutable artifact IDs, content hashes, input refs and destination/payload
match the frozen manifest; require active approval of this exact ID + hash
set version = 8, status = approved, approved_snapshot = R-42:review:v7:c1
insert outbox(operation_id = "R-42:send:v8", snapshot_id, payload_hash,
destination, provider_idempotency_key = "R-42:send:v8",
state = pending) unless that operation_id already exists
COMMIT
Back operation_id with a unique constraint, and require a duplicate to match the original snapshot, payload hash, destination, and provider key rather than treating any conflict as permission to proceed. PostgreSQL documents ON CONFLICT DO NOTHING for duplicate inserts, but the application still has to inspect conflicts and enforce those invariants.[21] The v8 transition and one outbox row are atomic within this database; neither means one remote effect. A database transaction cannot atomically commit an unrelated provider’s send.
Claim, send, and reconcile the effect
Workers atomically claim a pending or expired-lease outbox row in a short database transaction, setting a lease owner/token, expiry, and attempt count with a conditional update (or row-lock equivalent); only the successful claimant proceeds. The claim must check the authorized snapshot, approval state, and operation status before moving the run to dispatching. Do not hold a database lock across a network call. A lease prevents ordinary concurrent claims, but an expired lease or a crash after remote success can allow another worker to send the same request. A fencing token can keep an old worker from recording a stale local result; it cannot retract a provider-side action already made.
Send the exact frozen payload to the exact destination with the same provider idempotency key R-42:send:v8 on every retry if the provider supports an appropriate idempotency contract. Verify its scope, retention window, and payload-matching rules for that provider; do not assume the key works forever or across destinations. Persist provider request/result identifiers and acknowledgement against the operation. On timeout or a crash between provider success and local acknowledgement, first consult a provider lookup keyed by the operation or provider request ID when available. Retry only when the provider’s idempotency semantics make a replay safe for this operation; otherwise put the row in needs_reconciliation and have an operator establish whether the remote effect occurred before any resend. If there is neither suitable idempotency nor reliable lookup, ambiguity remains: do not promise automatic exactly-once delivery or blindly send again. Temporal likewise recommends idempotent Activities because retries may repeat external work when completion was not recorded; it does not supply exactly-once remote effects.[19]
There is a real revocation boundary. Before dispatch authorization, revoke the approval under the same run lock used by the claim, cancel the pending outbox row, and block new claims; a late worker must fail its authorization check. Once a claim has committed dispatching, even if its HTTP call has not visibly returned, a concurrent revocation cannot guarantee the request will not reach the provider. Mark the operation in flight/uncertain and reconcile it; after a confirmed send, revocation is post-send remediation (for example, request a provider cancellation if available, notify the recipient, or issue a corrected report), not an “unsend.” The reviewer gate and effect boundary must be legible in the ledger. An instruction inside a retrieved document is input data, not permission to advance run_status or bypass a reviewer.
Checkpoints, retries, and cost
Temporal describes a durable Workflow Execution with local state and external interaction through Activities. That is one viable orchestration route, not a promise of exactly-once delivery to an external API.[19] A smaller team can start with a database, an outbox, and an explicit worker; a durable workflow engine may earn its operational overhead when steps last a long time, require many retries, or span multiple systems.
Budget for database writes/checkpoint retention, queue or workflow hosting, storage and egress for artifacts, model calls on retries, observability, and time spent reconciling partial failures. Persist only what is needed: prompts, tool results, and checkpoints may contain customer data or credentials. Limit readers, redact secrets, set retention, encrypt and protect backups, and inspect each provider boundary. “Durable” means recoverable execution state; it does not mean confidential storage by default.
Recovery drill before launch
Run the same workflow twice with the same run ID; race two v8 coordinator commits; attempt to replace an input or referenced artifact after freezing v7 and verify the old approval cannot authorize the replacement; race two senders for one outbox row; expire a lease while its former owner is still running; and stop the sender after a remote request but before local acknowledgement. Compare the canonical version, snapshot IDs/hashes, approvals, outbox rows/claims, provider-side action, and human-review queue after each drill. Test revocation before dispatch authorization (no new send permitted), then revocation after dispatch begins (uncertain/confirmed effect goes to reconciliation or remediation, never an implied unsend). Resume from a checkpoint with an intentionally stale artifact. If you cannot explain the next legal transition from those records, adding more agents will make diagnosis harder, not easier. This is a proposed drill, not a report of tests performed here.
Sources
- [4] https://f3fundit.com/the-model-context-protocol-mcp-how-solopreneurs-can-build-interoperable-ai-agent-stacks-that-actually-work-together
- [5] https://f3fundit.com/ai-workflow-orchestration-vs-agentic-execution
- [18] https://docs.langchain.com/oss/python/langgraph/persistence
- [19] https://docs.temporal.io/activity-definition
- [20] https://www.postgresql.org/docs/current/explicit-locking.html
- [21] https://www.postgresql.org/docs/current/sql-insert.html

