Pick agent pilots by what a wrong action costs and who can check the result, not by how impressive the workflow sounds. Here are five to run, three to hold, and a way to tell the difference.
The agent pilots worth running this quarter read and propose while a person acts: ticket triage, contract renewal watch, CRM clean-up, competitor watch and new-hire paperwork. Three aren’t ready: customer-facing agents that commit on their own, agents that set prices or pay, and agents that decide about people. Score any candidate with the six-question test, write a permissions sheet, and run a 30-day pilot.
Agent pilots often get chosen by the demo, and the demo is a poor judge of what to run first. The workflow that sounds most impressive in a leadership meeting is often the one where a wrong action costs the most and the fewest people can check it. It tends to lose the quarter to something quieter that nobody presented.
Gartner has predicted that more than 40% of agentic AI projects will be canceled by the end of 2027, and it names three causes: escalating costs, unclear business value and inadequate risk controls.[1] That’s a forecast, not a measurement. The three causes still make a useful design brief for a pilot: cap what it can cost, pick work where the value is visible, and settle the controls before the agent runs.
Here, “agent” means software that takes several steps across your tools on its own, such as reading a ticket, looking up the account and drafting a reply, instead of answering one prompt and stopping. For definitions, see our plain-English guide to AI agents and our piece on assistants, copilots and agents. This one is about what to pilot, and it pairs with our test for choosing between AI pilots in operations. The placements below are our judgment unless a source is named.
The companies building these systems are cautious too. Anthropic’s engineering team recommends the simplest solution possible, which “might mean not building agentic systems at all,” and notes that agent autonomy brings higher costs and the potential for compounding errors.[2] OpenAI’s guide says that if a use case doesn’t clearly need an agent, a deterministic solution may suffice.[3] Some pilots below may turn out to need ordinary automation, and a pilot that tells you so has done its job.
The five pilots share one shape. The agent reads, then proposes, and a person acts. Each is written from the seat of the role that would own it.
| Owner | Workflow | What the agent does | Why it’s a good first pilot |
|---|---|---|---|
| Support lead | Ticket triage and draft replies | Tags each ticket by issue and urgency, attaches recent history, drafts a reply from the matching help article | Output goes to a colleague, not the customer, and the help article is a source to check against |
| Procurement or contracts lead | Contract renewal watch | Reads each agreement, lists the renewal date and notice period, drafts a heads-up to the contract owner | The answer sits in the document, so every item can be traced back, and a missed notice date is expensive |
| RevOps lead | CRM clean-up queue | Finds likely duplicates, stale deals and missing fields, and queues each proposed fix | Nothing is written until a person approves a batch |
| Marketing or strategy lead | Weekly competitor and market watch | Visits a fixed list of pages and lists what changed since last week, with links | Read-only, contacts no one, and judged by whether anyone acted on it |
| HR coordinator | New-hire paperwork chase | Checks each new hire’s checklist, flags what’s missing, drafts reminders to the hire, manager and IT | Colleagues are the audience, and the checklist is the source of truth |
Future Factors working view of five starting pilots. Permissions for all five start at read-only plus proposals.
A support lead opens Monday’s queue and finds the weekend’s tickets in the order they arrived, not sorted by what they’re about. A triage agent changes that morning: tickets arrive tagged, with history attached and a draft reply ready, and the lead’s job becomes reading drafts and sending or rewriting them. Anthropic’s guide calls customer support a natural fit for agents, partly because resolutions make success measurable.[2] This version keeps the agent away from the customer and the refund button. Expect early misses on tickets that raise two problems at once. Each one exposes a rule that lives only in someone’s head.
The contract renewal watch suits a procurement or contracts lead because the answer sits in the document. A frequent trap is amendments: the date in the original agreement isn’t always the one that applies, so the first month is mostly about finding which contracts the agent reads wrongly. Renewals arrive slowly, so test it first on contracts that have already come up.
A CRM clean-up queue fits a RevOps lead. Some of it is rule-based work that ordinary automation handles well, so decide which parts need judgment (is this the same company, or a subsidiary?) before deciding what the agent is for. On Salesforce AI Research’s CRMArena-Pro benchmark, leading agents reached only around 58% success on single-turn tasks and around 35% in multi-turn ones.[4] Benchmarks aren’t your CRM and models improve quickly, but the direction holds: the more back-and-forth a task needs, the more often it goes wrong. So the agent writes proposals to a queue and a person applies them in batches.
The competitor and market watch is the most forgiving pilot on the list. The likely failure is noise, such as a site redesign reported as a pricing change, and the test of value is blunt: did anyone act on something it surfaced.
The paperwork chase belongs to an HR coordinator, who reads the drafts before they go out for the first few weeks. Anything touching pay, contract terms or someone’s personal circumstances stays with a person.
An agent earns the right to act by being right first, while it only proposes.
Not ready is a judgment about the setup around the agent more than about the model. All three holds fail the same two questions: can you undo what the agent does, and can a named person check it before it lands? A hold can lift, and the table in the next section says what would change our mind.
A customer-facing agent that answers policy questions and makes commitments on its own looks like the natural step after support triage, and for some teams it eventually will be. The case to keep on your desk is Moffatt v. Air Canada. A customer asked the airline’s website chatbot about bereavement fares and was told he could request the reduced rate within 90 days of the ticket being issued, while the airline’s own page said the policy didn’t apply to requests made after travel. Air Canada argued, in effect, that the chatbot was responsible for its own statements. The tribunal called that “a remarkable submission” and held the airline responsible for what its website says.[5] The award was small, CA$812.02. The principle isn’t: what your agent tells a customer is something your company said.
Project Vend is the cleaner illustration of why an agent shouldn’t set prices, buy stock or spend your budget yet. In 2025, Anthropic and Andon Labs let Claude run a small shop in Anthropic’s San Francisco office for about a month. It offered prices without researching them, was talked into discount codes over Slack, told customers for a time to pay an account it had made up, and the shop didn’t make money. Its authors wrote that they would not hire it for the job.[6] That was an earlier model and the tools have moved on, so read it as a list of failure patterns, not a current scorecard: invented details, pressure from the person on the other end, and no sense that some actions are hard to reverse. OpenAI’s guide puts canceling orders, authorizing large refunds and making payments in the category that should trigger human oversight until confidence in the agent grows.[3]
If you can’t undo what an agent did, a person approves it first.
A hiring manager gets a ranked shortlist from an agent, with a short note on each candidate. The ranking looks reasoned, which is the difficulty: the reasons are hers to defend, and she can’t see what the agent saw. That’s the third hold. It covers screening candidates, ranking staff, or recommending who is promoted or let go, and here regulation adds to the risk.
Annex III of the EU AI Act lists AI used to filter job applications and evaluate candidates, to make decisions on promotion or termination, to allocate tasks based on individual behaviour, and to monitor and evaluate performance as high-risk.[7] Article 14 expects the people overseeing such a system to be able to watch for automation bias, override or reverse its output, and stop it.[8] Separately, Article 22 of the GDPR gives people the right not to be subject to a decision based solely on automated processing that significantly affects them, with safeguards that include the right to obtain human intervention.[9] I’m not a lawyer and haven’t confirmed how these apply where you operate, so take any pilot in this area to counsel first.
For a first quarter, our own rule is clearer to apply than the law. An agent can prepare for a decision about a person, by collecting documents or assembling a timeline, and a person makes the decision and can explain it in their own words. Two other kinds of work belong on the same list:
| Use case | Why it isn’t ready | What would change our mind |
|---|---|---|
| Agent that answers customers and makes commitments on its own | Your company answers for what it says, and nobody reads the reply first | A few weeks of replies a person reads and sends, a measured error rate on policy and money answers, and a written list of what it may never promise |
| Agent that sets prices, buys or pays | A wrong action is hard to undo, and the other party can talk the agent into it | Spending limits it can’t override, human approval above a small amount, and a log someone reads weekly |
| Agent that screens, ranks or decides about people | Regulators treat it as high-risk, and the signer can’t explain what they can’t see | A named reviewer with authority to override, the source documents in view, and legal sign-off on your use |
Placement is Future Factors’ judgment. The sources behind each reason are cited in the text above.
The five pilots and three holds come out of the same six questions, and you can run them on any candidate. Two go beyond the general test in our operations guide, and both matter only once software can act: the worst case, and whether you can undo it. Score each 0 (no), 1 (partly) or 2 (yes), on the work as it runs in a normal week.
| # | Question | A 2 looks like | A 0 looks like |
|---|---|---|---|
| 1 | What’s the worst thing it could do with the access it has? | A wrong draft a colleague sees first | A customer message, a payment, or a change to a person’s record |
| 2 | Can you undo what it does? | Nothing is written until a person approves it | Changes stick the moment they’re made |
| 3 | Is there a written right answer to check against? | A policy, contract, checklist or help article | Judgment calls with no reference |
| 4 | Is checking quicker than doing the work? | Scanning the output takes a minute | You’d have to redo the work to trust it |
| 5 | Will a named person read the output anyway? | One role who uses it every week | “The team”, or nobody in particular |
| 6 | Does enough of this work arrive in 30 days to learn from? | Daily or weekly, in volume | A handful of cases a quarter |
Maximum 12. 10 to 12: pilot this quarter. 7 to 9: raise the lowest-scoring line first, or narrow what the agent may do. 0 to 6: hold. Bands are a Future Factors working guide based on our judgment, not a benchmark.
Run it on the support team’s two ideas. Triage with drafts for a person to send scores 11, losing a point only because checking isn’t quicker than writing on tangled tickets. An agent that issues refunds on its own scores 4: the worst case reaches a customer and a payment (0), a refund that has gone out is hard to pull back (0), the policy exists but its exceptions live in people’s heads (1), the check would come after the money moves (0), the lead reads it only when something goes wrong (1), and volume is plenty (2). The first is a pilot. The second is a hold, and the score shows which lines to fix before revisiting it.
A score is only as honest as the person giving it, so have the person who would review the output do the scoring, not the person pitching the pilot. And a high score doesn’t replace the pilot. It means the pilot is safe to start.
Each pilot needs its split written down before it starts, because “who checked this?” gets asked after the fact, usually by someone senior.
| Pilot (owner) | What AI does | What the human still owns | How it gets checked |
|---|---|---|---|
| Ticket triage (support lead) | Tags tickets, attaches history, drafts a reply from the help article | Sending or rewriting every reply, deciding what is urgent, handling the unusual ticket | Lead compares a random handful of drafts with what was sent, weekly |
| Renewal watch (procurement lead) | Extracts renewal dates and notice periods, drafts the heads-up | Deciding to renew, renegotiate or leave, and the supplier relationship | Each date traced to its clause before the first reminder, then a monthly sample |
| CRM clean-up (RevOps lead) | Proposes merges, closures and filled fields to a queue | Approving each batch, and defining a duplicate or a stale deal | Only approved rows applied, every change logged so it can be reversed, a sample reviewed next week |
| Competitor watch (strategy lead) | Lists what changed on a fixed set of pages, with links | Deciding which changes matter and what to do about them | Every item opened at its source before it is repeated to anyone else |
| Paperwork chase (HR coordinator) | Finds missing items, drafts reminders | Sending reminders at first, and anything touching pay, terms or personal circumstances | Coordinator compares the agent’s list with the checklist for a weekly sample |
A Future Factors working view. The split changes with the workflow; a named person and a source to check against stay constant.
The check differs by seat: reading drafts for a support lead, tracing a date to a clause for procurement, approving a batch for RevOps. Pick the one that is quick enough to survive a busy week.
Pilot only what a person can check faster than they could do the work.
A random spot-check every week beats a thorough review that stops in the third week. Write down what each check finds, because that is how the agent’s rules improve. Our human-in-the-loop guidelines cover how to write a reviewer’s job down.
One pilot, one named owner, and a written permissions sheet before the agent touches anything. (Our agent boss guide covers the levels of authority an agent can hold.) Here is the sheet filled in for support triage; swap the right-hand column for your own workflow.
| Field | Support ticket triage |
|---|---|
| Job | Tag new tickets by issue and urgency, attach recent account history, draft a reply from the matching help article for a person to send |
| Can read | Ticket text, the customer’s last few tickets, the help centre articles |
| Can write | Tags and a draft inside the ticket. Nothing else |
| Must never | Send anything to a customer, issue a refund or credit, change an account record, or mark a ticket solved |
| Stops and hands to a person when | The ticket mentions legal action, a safety issue, cancellation or a refund, or comes from a named key account |
| Owner and reviewer | The support lead owns it and reads a random handful of drafts every week against what was actually sent |
| Pause rule | Pause the pilot if a draft turns up that would have been wrong to send and nobody can say how it got through |
The two rows people tend to leave blank are Must never and Pause rule. Fill them in first.
Then run it in four steps, with the permissions unchanged throughout. Widening them is a separate decision.
Nobody acts on the agent’s output. Compare its proposals with what people did.
The owner edits and acts on outputs. Log every edit and miss.
A larger slice of the work, same permissions. Start the weekly check.
Answer the three questions below, then keep, narrow or stop.
Future Factors’ working view of a 30-day pilot, not a validated method.
Judge the pilot on use, persistence and impact, not on how many times the agent ran.
| Question | In support triage | Where the evidence is |
|---|---|---|
| Use | Did the lead read the drafts, including on tangled tickets and not only the routine ones? | Draft views and edits in the ticket tool, split by ticket type |
| Persistence | By week four, were drafts still being read and used without a reminder? | The same counts, week four against week two |
| Impact | Did something change: a faster first reply, fewer retags, a policy gap found? | What the weekly checks found, plus the lead’s own account |
Usage counts alone measure access. Persistence and impact are what tell you the pilot earned a second quarter.
A month shows use and the start of persistence. Put a second review in the diary for 60 or 90 days.
Teams can run this internally, and many do. Structured training with an experienced facilitator shortens the learning curve and spreads the skill of reviewing an agent’s work across a team instead of leaving it with one enthusiastic person. In the terms we use at Future Factors, training builds the capability to review an agent’s work, workflow design gives it somewhere to be used (the permissions sheet and weekly check), and reinforcement, such as a manager who asks what was caught, turns it into behavior. Adoption is what you see when that holds, as in our guide on moving from AI awareness to fluency.
To stress-test a candidate before you commit, paste this into your AI tool.
| I own [FUNCTION] and I’m considering a pilot of an AI agent for [WORKFLOW]. List every system it would need to read and everything it could write or send. For each write action, tell me the worst plausible wrong action, whether it can be undone, and who would notice first. Then recommend a starting permission level (read only, propose, or act with approval) and name the weakest of these six areas: worst case, undo, a written right answer, check speed, a named reader, volume. Do not invent statistics, vendor capabilities or case studies. If you aren’t sure how a tool behaves, say so. |
Treat the answer as a first pass. The model doesn’t know your systems, so correct what it gets wrong first.
This week:
Anthropic’s engineering team draws the line this way: workflows run models and tools through predefined code paths, while in agents the model directs its own process and tool use. If the steps are predictable, try a workflow first. An agent earns its place where the number of steps can’t be predicted, as with the ticket that raises two problems at once. Our guide to building an agent has a longer test.
Long enough to see whether people use the output and to catch the first round of mistakes, which is why the plan above stops at 30 days. Extend it if the work arrives slowly, like contract renewals, by running the agent over past cases first. End it early if the pause rule triggers.
It needs to read whatever the work depends on, and that is a decision to make on purpose. Start read-only, ask your IT or security team about a narrow, separate account, and keep write permissions off until the proposals have held up. Check with them before connecting customer or employee data to any tool your organization hasn’t approved.
Treat that as a vendor claim and ask to see it on your own data, messy cases included. Gartner has said some products sold as agentic are rebranded assistants, automation or chatbots, as we cover in our piece on agent washing. Even if the claim holds, start in proposal mode, because the pilot tests your setup as well as the product.
The person in the function who would read the output anyway: the support lead for triage, the RevOps lead for the CRM queue. IT or security owns the access, and a sponsor owns the decision to widen it. If nobody in the function wants it, that tells you something about the pilot.
Sources were read on October 15, 2026: Gartner’s June 2025 press release on agentic AI projects, Anthropic’s “Building effective agents” (December 2024) and its Project Vend write-up (June 2025), OpenAI’s “A practical guide to building agents” (undated), the abstract and first page of the CRMArena-Pro paper from Salesforce AI Research, the Moffatt v. Air Canada tribunal decision, and the text of Annex III and Article 14 of the EU AI Act and Article 22 of the GDPR. Anthropic and OpenAI write about agents they sell, so their guidance is vendor guidance, and Gartner’s 40% is a forecast, not a measurement. Project Vend used an earlier model than is available now. The five pilots, the three holds, the fit test and its score bands, the ownership table, the permissions sheet and the 30-day plan are Future Factors working views based on our own judgment, not research-validated models. The scenes in this guide are illustrations, not client stories. We left out ROI, hours-saved and adoption-rate figures because we could not confirm any against a primary source. This is not legal advice.
Weekly tips. Real examples. Practical help for busy professionals.
We care about your data, check out our privacy policy.