What to Measure in the First 90 Days of an AI Rollout

Most Australian businesses that try AI and give up on it never actually failed at AI. They failed at measurement. They picked a tool, used it for a few weeks, felt vaguely better or vaguely annoyed, and made a call based on gut feel rather than numbers. Ninety days later, "it didn't really work" becomes the verdict, and the subscription gets cancelled.

That verdict is almost always wrong, or at least unproven. You cannot know whether something worked if you never wrote down what "before" looked like. The good news is that proving it doesn't require a dashboard, a data analyst, or a single dollar of new software. You already have everything you need sitting in your calendar, your inbox, your invoicing system and your own memory. This piece sets out four things worth tracking in the first 90 days, all measurable with what you've already got.

Why "it didn't work" is usually a measurement failure, not an AI failure

Uptake of AI among Australian small businesses is still relatively low compared with larger firms, and that gap is telling. According to the Australian Bureau of Statistics, around 35 per cent of large businesses reported using AI, up from 9 per cent in 2021 to 2022, while 22 per cent of medium sized businesses had adopted AI, compared with 3 per cent previously, while uptake among small and micro businesses was lower, at around 11 per cent. Interestingly, the ABS also found that the AI adoption rate for innovation active small businesses was 19 per cent, almost five times greater than the rate for those that did not undertake any innovation activity. Businesses that treat AI as a structured project, not a novelty, stick with it at far higher rates.

Part of the reason smaller operators drop off is that they never set up a way to judge success. Research tracking SME AI use found that you can't optimise what you don't measure, and the companies building a real AI advantage aren't necessarily using more sophisticated tools, they're measuring what's working and iterating on it, while most businesses are still skipping that step. In other words, the businesses getting genuine value aren't smarter or better resourced. They just kept score.

Without a baseline, every outcome looks ambiguous. Did the quote turnaround improve because of the new AI drafting tool, or because it was a quiet month? Did errors drop, or did you just get lucky with a run of simple jobs? Ninety days is long enough to see a real pattern, but only if you captured the starting point and kept checking in along the way.

The four things you can measure without buying anything

The first is time on task. Before you touch any AI tool, pick one recurring job, writing quotes, drafting job descriptions, summarising meeting notes, answering common customer emails, and time yourself doing it the old way for a week. A stopwatch on your phone or a note in your calendar app is enough. Once the AI tool is in use, time the same task again at the 30, 60 and 90 day marks. If the number hasn't moved, that's real information, not a failure of the exercise.

The second is error and rework rate. Count how many quotes get sent back for correction, how many invoices need reissuing, or how many drafts need a full rewrite before they go out. You're probably already tracking some of this informally through email threads or a shared folder of "final_v3" documents. Formalise it into a simple tally for a fortnight before you start, then repeat the tally periodically afterwards. A drop in rework is often worth more than a drop in time, because rework eats trust with customers as well as hours.

The third is turnaround time, meaning the gap between a request coming in and a response going out. Your email and your invoicing or job management software already timestamp everything. Pull ten recent quote requests and check how long each took from enquiry to sent quote. Do the same after the rollout. This measure matters because customers rarely notice how long you spent on a task, but they absolutely notice how long they waited.

The fourth is output volume per person, counted from records you already keep. How many quotes went out this month, how many invoices were raised, how many job files were closed. If a team member is producing the same or more output in the same hours, or the same output in fewer hours, that's a real productivity signal you can point to without any new tracking system.

Setting the baseline before you start

None of this works if you skip the baseline. The temptation with any new tool is to start using it immediately and worry about proof later, but "later" is exactly when the comparison becomes impossible. Give yourself one week, before the AI tool goes live, to record the four numbers above for the task you've chosen. Write them on a whiteboard, in a spreadsheet you already have open, or in the notes app on your phone. It doesn't need to be pretty. It needs to exist.

Then check in at day 30, day 60 and day 90. Thirty days tells you whether the tool is being used at all. Sixty days tells you whether the team has adjusted their workflow around it. Ninety days is when the real pattern shows up, because early enthusiasm and early frustration have both worn off by then and what's left is the actual habit.

Turning the numbers into a decision

At the 90 day mark, you should have four simple before and after comparisons, not a spreadsheet full of vanity metrics. If time on task, error rate, turnaround and output have all moved in the right direction, you have a genuine business case for expanding the tool to other tasks or other team members. If only one or two measures improved, you've still learned something specific and can decide whether that improvement alone justifies the cost and the change management. If nothing moved, you've saved yourself from quietly paying for a tool nobody actually uses, which is a far more common outcome than most business owners want to admit.

The one thing you can do today, before reading anything else about AI tools, is pick a single task and start the stopwatch. Time it five times this week. That's your baseline, and it costs nothing.

If you want a structured version of this, a clear 90 day plan with the right task chosen, the right metrics tracked, and a proper decision point at the end, that's exactly what our AI Roadmap Sprint is built to deliver.