Research direction
Introducing AI into a business workflow changes more than the way a task is performed. It changes who decides what happens next, what counts as sufficient evidence, and who is responsible when an operation fails.
My interests connect systems architecture with the way organizations operate: reliable execution across services, useful human oversight, and the transition from established software workflows to AI-assisted systems. I want to distinguish the value of AI from improvements that better data, integration, or conventional automation can already provide.
Current study · Design stage
Execution evidence and recovery decisions in AI business workflows
Consider a workflow that updates a customer record in two services. One confirms the change; the other times out. A timeout does not tell the agent whether the second update failed or completed without a response. Retrying, waiting, checking the state, and requesting help each have different consequences.
How do execution evidence, recovery policy, and model choice affect correct completion, unintended side effects, human-review demand, and cost?
I’m developing an experimental design around this question. Experiments have not yet been run.
Planned approach
- Build a controlled workflow.
Use synthetic records, two mock services, and a SQL event ledger that independently records what actually happened. The ledger supplies evaluation ground truth and is not exposed to the agent.
- Vary the evidence and failures.
Introduce timeouts, partial updates, and stale reads. Vary the information available for verification, while keeping each comparison’s tools and permissions consistent.
- Compare credible alternatives.
Test conventional recovery rules and verification-based retries alongside model-driven decisions. Include idempotency protections where supported, so AI is compared against a well-engineered baseline.
- Measure the whole workflow.
Track final correctness, harmful duplicate effects, unresolved updates, recovery time, requests for human review, and API and model costs. A handoff will remain unresolved unless a reviewer actually completes it.
The aim is to find where AI helps, where conventional software is sufficient, and what evidence is needed to make that decision.
Connection to practice
My work at AHRC includes integrations, shared infrastructure, and a reporting workflow in which Claude produces cited drafts for staff review and approval. At Equity Residential, I worked on authentication migration, acceptance testing, and release decisions. Both experiences make recovery and human control concrete engineering concerns for me.
The proposed experiments will begin with synthetic data. The AHRC case provides practical context; it is not a completed experimental evaluation.
Selected reading
These works help frame the questions I’m exploring and the comparisons a useful study should make.
- WorkArena — evaluating language agents on business tasks in enterprise software environments.
- TheAgentCompany — evaluating agents on tasks in a simulated company environment.
- τ²-bench — evaluating agents in environments where users and agents both act.
- Building effective agents — practical distinctions between predefined workflows and agents that direct their own actions.