AI Usage Metrics: What to Track Instead of Tokens and Logins

Why counting AI use feels like progress

An AI usage dashboard is usually the most cheerful page in the monthly pack. Active users are up, prompts are up, and the line only goes one way. It stays cheerful until someone senior asks the obvious follow-up, which is what got better in the work, and the room goes quiet because the dashboard was never built to answer that.

Some companies went a step further and started rewarding the line. A token is roughly a small chunk of text a model reads or writes (our tokenomics explainer covers what they are and what they cost), and in 2026 counting them per employee picked up a name: tokenmaxxing. As Fortune reported on 28 May 2026, some firms set up formal or informal leaderboards of token use and encouraged engineers to compete on them, and Meta took down an informal leaderboard its own employees had created.^[1] Fortune’s verdict was that the practice is over.

I get why it happened. Usage is the first number any AI tool hands you, and it moves the right way as soon as people are told to use the tool. A usage count tells you whether people touched the tool, and nothing about whether the work got better. And once you turn it into a target, people start optimising the count instead of the work.

So here’s the argument of this piece: keep usage numbers as diagnostics, put your goals on workflow outcomes with a quality check beside them, and review both with the team.

This is for whoever sets or receives AI goals: L&D and HR leads, people managers, team leads. It doesn’t cover how to prove AI caused a result (that’s in how to prove your AI actually worked), how to time your savings (see how to track your time savings from AI) or ROI maths (the guide to measuring AI ROI). It’s about one narrower decision: which numbers are allowed to be targets.

What usage numbers can tell you, and what they can't

Usage numbers aren’t useless. They’re the fastest way to find a team where nobody has opened the tool since launch. Read as questions, they’re helpful. Read as answers about value, they mislead, mostly because of what sits inside them.

Microsoft’s documentation for its Copilot Dashboard is a clear example, because it spells its definitions out (page checked 7 October 2026, last updated 29 September).^[2] An active user there is someone who performed at least one activity in the tool in the previous 28 days. Assisted hours are described as an estimate, built from people’s actions and multipliers derived from Microsoft’s own research. And the page itself says the tool is not likely to be solely responsible for any change the dashboard shows.

None of that is hidden, and the vendor is clear about what it counts. The trouble starts when one action in four weeks gets read as adopted, or an estimate built on someone else’s multipliers gets pasted into a business case as hours saved. Before you quote any dashboard figure, open the vendor’s own definition page and read what the number counts.

Asking people how it’s going has a gap of its own. In a 2025 randomised trial, METR found experienced open-source developers took 19% longer on tasks when AI tools were allowed, while afterwards believing AI had sped them up by about 20%.^[3] That’s one small group in one setting, and METR says plainly it doesn’t show AI fails to help most developers or other kinds of work. What it does show is that how fast work feels and how fast it goes can come apart, so a survey asking whether AI is helping belongs in the diagnostic pile too.

Here’s the audit I’d run on whatever your team currently tracks.

The measurement lens we use across this site runs use, then persistence, then impact. Usage dashboards live almost entirely in the first stage, and targets belong in the third. For what healthy use looks like at the first stage, see our piece on adoption benchmarks

How usage targets get gamed without anyone meaning to

An L&D coordinator hears on Monday that the team goal is everyone active on the AI assistant each week. On Friday afternoon she realises she hasn’t opened it, so she asks it to reword a sentence she’d already written, accepts the change and closes the tab. The dashboard counts her as active. Nothing about her week changed, and nobody did anything dishonest. She met the target the way it was written.

That’s an old pattern with a well-known name. The anthropologist Marilyn Strathern gave the version of Goodhart’s law most people quote: “When a measure becomes a target, it ceases to be a good measure.”^[4] A 2009 review of goal-setting research, Goals Gone Wild, listed the side effects of goals set too narrowly, among them a focus that neglects everything outside the goal, distorted risk-taking, a rise in unethical behaviour and reduced intrinsic motivation.^[5] Neither was written about AI. Both describe what a usage target invites.

Gaming rarely looks like cheating. It looks like ordinary people being reasonable about a number they’ve been handed. By role, it tends to show up like this:

• L&D: every learner follow-up runs through AI so the count rises, and goes to everyone whether or not they needed one.
• HR: policy questions get pasted in and the answer forwarded unchecked, because checking takes longer and doesn’t count twice.
• Marketing: five versions of every headline get generated and four thrown away, which multiplies prompts without improving the one that runs.
• Sales: proposals get regenerated from scratch rather than edited, piling up activity while last month’s pricing errors get copied forward.
• People managers: every one-to-one gets an AI summary, including the ones that needed a follow-up conversation rather than a document.

There’s a cost beyond a misleading chart, too. A usage target teaches people that being seen to use AI is what counts, when the lesson you wanted was to use it where it helps and skip it where it doesn’t. Some of a team’s most sensible AI decisions are decisions not to use it, and a usage target tends to penalise each one.

The diagnostic-or-target test for any AI metric

Banning usage numbers would be an overcorrection. You need them to spot rollouts that have stalled. What you need instead is a way to sort every metric into one of two piles before anyone attaches a goal to it: diagnostics, which you look at and ask questions about, and targets, which someone is accountable for moving.

I call this the diagnostic-or-target test. Ask three questions of any AI metric:

1. Can this number go up while the work gets worse?
2. Is there a quality check sitting beside it that a named person reads?
3. Do we know what decision we’d make if it moved?

If the answer to the first is yes, or the answer to the second is no, the metric stays a diagnostic. You look at it, you ask why it moved, and you don’t set a target on it. Only a measure that passes all three becomes a target. The third question catches something quieter: numbers tracked out of habit that nobody would act on whichever way they went.

Here it is applied to six metrics teams commonly track.

A sales operations lead brings the scorecard to a planning meeting, and it starts an argument about the estimated hours row. Someone points out that people rate the tool, so there’s a quality check of sorts. Someone else asks who reads those ratings. Nobody can answer, which is the test working. The number stays on the dashboard as a diagnostic, and the team’s target moves to proposal turnaround, with the sales manager checking pricing on a sample of proposals each week.

Put the goal on the workflow, with a quality check beside it

Once usage is a diagnostic, the goal needs somewhere to go. The unit that works is one recurring workflow: a specific piece of work, with a start and an end, that AI now helps with. Use AI more in HR can’t be measured. First replies to routine benefits questions can.

Every outcome goal also needs a guard beside it, because faster work that has to be redone hasn’t saved anything. In a Workday survey of 3,200 employees who use AI, run by Hanover Research and published in January 2026, nearly 40% of AI time savings were reported as lost to rework such as correcting errors, rewriting content and verifying outputs.^[6] It’s vendor-commissioned and self-reported, so hold the figure loosely. The direction fits a familiar pattern: measure only speed, and the checking moves somewhere nobody counts.

So each workflow gets a one-page outcome card. Here’s one filled in for an HR team.

The row people tend to leave blank is the last one. Deciding what you’ll do before the number moves stops the review becoming a debate about whether the result is good news.

The outcome measure changes with the role, which is why one company-wide AI target rarely fits anyone well. These are examples to adapt, not benchmarks:

Start with one workflow per team. Take on more in the first month and the quality checks tend to go unread.

Run the review, and know what to say when someone wants a usage target

Targets on their own drift. What keeps them useful is a short, regular conversation where the diagnostics and the outcomes sit side by side, and in our judgment twenty minutes a month is enough for most teams if it follows the same shape each time.

This review is the reinforcement part of the job. Training builds the capability, workflow design gives it a recurring task to land in, and the review turns it into habit. Adoption is what you see afterwards, when the new way of working holds without chasing. If the review keeps finding people can’t get good output in a workflow, that’s a training gap, and structured training closes it faster than trial and error while giving the whole team the same footing.

Managers carry more of this than the dashboard does. Gallup’s AI indicator (figures as of May 2026) found employees whose managers actively support AI use were 1.7 times as likely to use AI a few times a week or more.^[7] That’s a correlation from a survey, not proof that support causes use. It fits what this review is for, though: a manager asking good questions about the work sends a different signal from a manager watching a chart.

AI can do real work in the review itself, as long as the split is clear. A model can summarise a dashboard export confidently and still misread what a column counts, treating a 28-day active flag as daily use, for example. So the manager checks every metric definition against the vendor’s own page before the summary goes anywhere. Our human-in-the-loop guide covers how to write that kind of rule down.

When leadership asks for a usage number

The request usually arrives as one line from someone senior: every team at a set level of weekly active use by the end of the quarter. It’s well meant, since it signals that AI matters. It’s also the easiest goal in the building to hit without changing anything. Here’s a reply you can adapt.

If you’re the one being measured on usage

If the target has already landed on you, there’s no need to fight the number in public. Hit it honestly on work that’s genuinely yours, note one workflow where AI changed the result and how you checked quality, and bring that note to your next one-to-one. A manager handed a usage target usually wants something better to send upward, and a specific outcome with a check beside it gives them that.

One boundary: monitoring individual employees’ tool use is governed by local employment and privacy rules, so if individual-level tracking comes up, check with HR or counsel before anyone builds on it.

If you do one thing this month, take the AI number your team is judged on and run it through the three questions. If it fails, move it to the diagnostics column, pick one workflow, fill in an outcome card with a named quality reader, and put the first review in the diary.

Hina Mian, Co-Founder of Future Factors AI

Hina is a marketing strategist with over a decade of hands-on campaign experience across B2B and consumer brands. She writes about using AI to run leaner, sharper marketing without losing the human touch. Future Factors helps professionals and teams build practical AI capability through role-based training, workflow design, and hands-on adoption.

More about Hina →