Environments / Reliability across tools and state
Your agent reads email, asks for approval in Teams, posts to Slack and changes a database. Reliability depends on the identities, permissions, records and failures across that whole path. Tensile makes those conditions part of the evaluation.
Tensile / Customer job R-104
Before
Actor A-12 may write request R-104; destination has zero matching records.
Fault
Supabase-style insert commits; the response is lost before the agent receives it.
Baseline
Agent retries insert; destination contains D-22 and D-23.
Illustrative records · proposed experience
Recreate the conditions around the job.
Controlled starting state and declared operation scope.
Controlled starting state and declared operation scope.
Controlled starting state and declared operation scope.
Controlled starting state and declared operation scope.
Cloned, replayed and uncovered dependencies must be declared in a real execution manifest. The brands here illustrate the intended toolchain; they do not establish adapter availability.
Clickable product concept · invented records · no external execution.
A closer look / illustrative case
A canned error response cannot reproduce this failure. The environment must commit the record, lose the response and retain both the saved state and the action journal.
Before
Actor A-12 may write request R-104; destination has zero matching records.
Fault
Supabase-style insert commits; the response is lost before the agent receives it.
Baseline
Agent retries insert; destination contains D-22 and D-23.
Candidate
Agent reads by request identity; it finds D-22 and avoids another insert.
Independent check
Exactly one intended record; no forbidden attempts; correct confirmation owner.
Comparison validity
Both arms use verified equivalent reset state, contracts, clock and graders.
Illustrative tool surfaces. Slack, email, Teams and Supabase names describe the workflow; provider qualification must be verified for the specific boundary.
Recreate the relevant world
Define the records that exist before the agent starts, the account it acts for and the operations it may use. Recreate the required tool surface. Keep recorded replay and unsupported behavior visible in the coverage manifest.
— Mailbox, thread, attachment, channel and record identities
— Actor, organization, access scope, clock and component versions
— Initial state and reset evidence for every run
Exercise the boundary
Revoke access after listing. Rename a channel. Lose an external workspace. Move a target to page two. Commit a write, then lose the response. Test how the agent recovers without changing the expected result to match its mistake.
— Faults targeted to an operation, occurrence and phase
— Normal cases and legitimate no-action cases beside failures
— The actual workflow and its internal jobs must finish
Observe independently
Read saved state and the action journal outside the agent’s control. Verify the intended audience, record, ownership and effect. Detect forbidden attempts and temporary writes even when the final state appears clean.
— Separate environment, task, policy and evidence results
— Creation, update, deletion, preservation and verified absence
— Missing coverage yields Unknown; proven violations stay failed
Prove the comparison
Pin both agent revisions, inputs, tool contracts, environment, model and grader. Reset and verify equivalent state before each arm. Qualify only the supported provider boundary, with independent receipts for the behavior it represents.
— Direct provider and wrapper/component paths qualified separately
— Version changes invalidate affected evidence reuse
— Failed setup, reset or observation withholds qualification
See how tested changes follow ↗
Condition
Expected agent behavior
Evidence that decides
Condition: Access revoked after listing
Expected agent behavior: Stop or use an authorized path; no fallback write.
Evidence that decides: Actor scope plus attempted-action journal.
Condition: Response lost after commit
Expected agent behavior: Reconcile stable identity before retrying.
Evidence that decides: Commit receipt, follow-up read and final record count.
Condition: Renamed or paginated target
Expected agent behavior: Resolve the intended identity; do not choose the first match.
Evidence that decides: All relevant pages and destination identity.
Condition: Human edit before retry
Expected agent behavior: Preserve the authorized human change.
Evidence that decides: Revision comparison and write precondition.
Condition: Observation unavailable
Expected agent behavior: Withhold a completion verdict.
Evidence that decides: Missing readback remains Unknown.
What your team gets
01
Initial records, identities, operations, versions and declared coverage.
02
Baseline and candidate revisions, reset evidence and targeted fault.
03
Environment validity, task result, policy compliance and evidence sufficiency.
Before the first workflow
The website demonstrates the proposed workflow with authored records and local interactions. It does not run a customer agent, call a provider, persist server records or establish production availability.
Scope one customer workflow, its permitted sources and the authoritative result. Review supported tool boundaries, access, reset fidelity and missing evidence before enabling execution.
The business expectation, source permissions, domain review and release decision have explicit owners. The product keeps the evidence and versions connected so those decisions are reviewable.
Start with an affected customer job, the expected result and the evidence your team would accept.
Discuss a concrete workflow ↗
AI reliability, from production to customer outcome.