Businesses choosing AI tools for employees need to know what each one can actually do. Buying licenses is the kind of expense that’s costing tens to hundreds of thousands of dollars on the basis of “we must enable our employees with AI”. But what are employees supposed to do with that AI?
Excel is a useful test because people across every business use it to build formulas, create charts, record data, and collate/share information, and many would gladly spend less time doing those jobs by hand.
We wanted to know which platforms were better at different kinds of spreadsheet work, so I put five AI assistant setups through eight Excel tasks. The workbooks used finance scenarios, but these can easily be extrapolated to general Excel tasks. The Excel skills being tested were completing a template, writing formulas that update, building charts and finding faults in a complex workbook.
How we set this up
The tasks came from SpreadsheetBench 2, a benchmark for end-to-end business spreadsheet work. We selected eight that covered different Excel skills.
We tested five assistant setups:
- CoPilot Basic in Desktop Excel
- ChatGPT plugin in Desktop Excel
- Claude plugin in Desktop Excel
- Claude Code, using a headless harness to drive Excel
- CoPilot Business in Excel Online
This let us compare three assistants used in Desktop Excel with Claude Code and Excel Online. Three tasks covered formula construction and template completion using bond schedules. Three involved charts. The final two asked the assistants to debug a large leveraged-buyout workbook, first with a planted double-count and then with deleted rows and broken references.
We scored each finished workbook on four questions:
- Accuracy: Did it produce the right answer?
- Formulas: Did the calculations use formulas that update when the inputs change?
- Formatting: Was the result clear and readable?
- Useful additions: Did the assistant add checks, labels or anything else that improved the workbook?
We started by using a cell-by-cell comparison to find differences from the reference “gold standard” workbook, but the percentage match was a terrible predictor of how well the AI performed. A tool could easily match nearly every cell and still miss the point of the task. It could also disagree with the reference because it had made a more sensible modelling choice.
The grid below summarises the results. Hover over a cell for a short explanation, or expand it to read the full version.
The whole study on one grid: eight tasks down, five tools across, each cell our plain-language verdict. Hover a cell for the one-line read; expand for the full grid.
across all 40 runs
| Task | CoPilot BasicAuto · Auto | ChatGPTGPT-5.6 · High | ClaudeFable 5.1 · High | CoPilot BusinessAuto · Auto | Claude CodeFable 5.1 · Medium |
|---|---|---|---|---|---|
| 3.0 The formula and template tasks | |||||
| 3.1Bond schedule | |||||
| 3.2Zero-coupon bond | |||||
| 3.3Bond model | |||||
| 4.0 The chart tasks | |||||
| 4.1Sunburst chart | |||||
| 4.2Bubble chart | |||||
| 4.3Line burndown | |||||
| 5.0 The debugging tasks | |||||
| 5.1Audit and fix a broken model | |||||
| 5.2Repair the broken references | |||||
Verdicts are qualitative, judged on output quality, not cell-match. One run per tool. CoPilot Business could not complete either debugging task; those cells read no result.
This was a small test: eight tasks, with one run for most combinations of task and assistant. CoPilot Business in Excel Online received a second attempt at the first debugging task after it failed to complete, which subsequently also failed to complete.
The formula builds exposed shared mistakes
The build tasks tested whether each assistant could complete a template without breaking its calculations. It was interesting to see the models all falling into similar traps.
The clearest example was a bond issued at a $480 premium. All five tools used the $12,480 issue proceeds as the tax basis instead of the $12,000 face value. Four created and taxed a phantom $480 loss. Claude Code used the same wrong basis but treated the difference as a gain. To be fair, the source sheet wasn’t exactly clearly labelled which is what ultimately misled the models.
Another test showed how a workbook can reconcile while one of its labelled lines is still wrong. CoPilot Basic reported amortisation that was $1,000 too high because it omitted the annual $1,000 cash coupon from that row. Its loan roll still reconciled because the coupon subtraction appeared elsewhere, which is an important lesson. You need to pay close attention to the way AI is making changes to your sheets.
Bond schedule
Several tools made an error in the maturity year. CoPilot Basic's loan roll still reconciles even though its labelled amortisation row is $1,000 too high.
The brief
Complete the six-year accounting schedule for a 4% coupon discount bond.
Good looks like Every empty cell a live formula, and the schedule ties: the loan accretes to par at maturity.
Choose a tool
CoPilot Basic's schedule is structurally sound and the best-presented of the three plugins. It is the only tool that added colour, with navy section banners and light row shading over a template that shipped with none. Row 7, labelled amortisation, shows the full effective interest instead of interest minus the coupon, so it overstates by exactly the $1,000 annual coupon every year. It also hardcoded the IRR helper column, so the schedule will not update when the inputs change.
Row 7 overstates amortization by the $1,000 coupon; the rest reconciles.
Live formulas, but the IRR helper column is hardcoded.
The only arm to add colour: banded, scannable and easy to read.
Added colour to a plain template, the only tool that did.
ChatGPT wrote clean, live formulas and made the same row-7 amortization slip as CoPilot Basic, off by the $1,000 coupon. It added one of its own: the loan's carrying value nets to zero at maturity instead of holding at the $25,000 par, because it booked the redemption inside the accretion block, when it belongs in the cash section below. On screen the final column reads 0.0 against a stated $25,000 face value, which a reader would catch.
Shared $1,000 slip, and the terminal balance collapses to 0 instead of $25,000 par.
Live formulas throughout; the errors are logic, not hardcodes.
Clean, professional, one-decimal formats.
Nothing beyond the ask.
Claude is the one plugin tool whose terminal year is right, with the loan accreting to exactly $25,000 at maturity where CoPilot Basic and ChatGPT both mishandled it. Every value is a live formula. The only divergence is row 29, where Claude modelled the full cash to debt-holders, coupons plus principal, against the reference's blanks and a lone principal line. It is a defensible reading, and still a difference from the reference.
Terminal rollforward correct to par; only row 29 differs, defensibly.
Live formulas throughout, black text on blue inputs.
Clean accounting formats, faithful to the template.
Nothing added, nothing needed.
CoPilot Business in Excel Online gets the pretax profit, loan roll, deferred-tax build and reversal right. Its labelled amortisation row carries the same $1,000 error as CoPilot Basic, although the calculations that follow still reconcile. It also added live tie-out checks, a derived inputs block and restrained colour.
Economically correct; the $1,000 slip sits only in the display row.
Live formulas, plus reconciliation checks.
Section labels and tasteful banker colour.
Live tie-out checks that prove the schedule reconciles.
Claude Code built the schedule with live formulas through Excel COM. The IRR, P&L and deferred-tax accounts are correct, but it booked the $25,000 redemption inside the GAAP carrying-value roll, so the ending balance is 0 instead of par. Its result differs from the Claude plugin by one cell.
P&L and tax correct; the terminal balance ends at 0, not $25,000 par.
All-live formulas via Excel COM.
Clean, correct accounting formats.
Nothing beyond the ask.
Zero-coupon bond
Four of the five results pass. ChatGPT and Claude use a more rigorous maturity-year tax treatment than the benchmark reference.
The brief
Fill every empty cell in a four-year zero-coupon bond's statements and balance-sheet schedules.
Good looks like Live formulas throughout, and accretion and tax schedules that tie out.
Choose a tool
CoPilot Basic shares the maturity-year tax treatment the others reasoned their way to, but it also broke the retained-earnings rollforward: the formula drops net income, which zeroes the schedule and cascades up the chain. It special-cased the redemption to zero as well. Fully formula-driven, and over-engineered in a way that is what broke it, with array formulas doing work that simpler links would have done correctly.
A genuine retained-earnings bug on top of the shared maturity-year divergence.
Formula-driven, but over-engineered array formulas caused the break.
Clean, template-faithful, one-decimal formats.
Nothing beyond the ask.
ChatGPT is fully formula-driven, and its only divergence from the reference is a deliberate one: it deducts the accumulated original-issue discount at maturity and unwinds the deferred-tax asset, the economically realistic treatment of a zero-coupon redemption. The reference simply repeats the annual pattern and never takes the deduction. The benchmark marks the more rigorous answer as wrong; on our standard, a tool that is more correct is not marked down for it.
Correct; the one divergence is a more rigorous tax treatment than the reference's.
Formula-driven throughout; it even computes a cell the reference hardcodes.
Clean, consistent formats.
Nothing beyond the ask.
Claude models the redemption as a taxable extinguishment, books the loss, treats it as deductible and reverses the deferred-tax asset, the same realistic treatment ChatGPT reached and the same divergence from the reference's simpler pattern. All formulas are live. It left a comment on the maturity cell explaining the extinguishment reasoning, openly documenting the very logic that differs from the reference.
Correct; a more rigorous maturity-year reading than the reference.
All-live formulas, internally coherent.
Clean, template-faithful.
A cell comment documenting the divergent reasoning.
CoPilot Business in Excel Online matches the reference across the full graded range. It uses the reference's simpler assumption: no discount deduction at maturity and no reversal of the deferred-tax asset. ChatGPT and Claude differ from the reference because they model that maturity-year tax event. The CoPilot Business workbook is formula-driven and clean throughout.
Exact match across the graded range.
Formula-driven, live links.
Red-parenthesis negatives, consistent formats.
Nothing beyond the ask.
Claude Code matches every graded cell and uses live formulas throughout. It follows the reference's treatment of the maturity year, while the Claude plugin models the extinguishment. Both are defensible interpretations of an ambiguous part of the task.
Cell-perfect against the reference.
Fully formula-driven.
Clean formats.
Nothing beyond the ask.
Bond model
All five tools use proceeds instead of face value for the $480 bond premium. Four record a phantom loss, while Claude Code records a phantom gain.
The brief
Complete a four-year bond accounting model, filling every empty cell.
Good looks like Live formulas and a model that closes to zero at maturity, on the right tax basis.
Choose a tool
CoPilot Basic gets the income statement, cash flows and IRR right. It then books the tax basis at the $12,480 proceeds instead of the $12,000 face value, creating a $480 loss and $144 of tax. Both amortisation rollforwards also shift by one period. The workbook uses formulas throughout, including one hand-patched term that does not follow the pattern in the rest of its row.
Income statement and IRR right; the $480 premium trap and a one-period shift.
Formula-driven, but one cell is hand-patched off the row pattern.
Clean, correct accounting formats.
Nothing beyond the ask.
ChatGPT hits the same premium-basis trap as the rest and adds a cosmetic one of its own: it flipped the sign of the entire IRR cash-flow column. The yield still computes to 3.90% because IRR is invariant to flipping every sign, so it is wrong on the page but harmless to the number the column exists to produce. Consistently formula-driven; the chat panel confidently documents its own mistake.
The shared premium trap; the sign-flipped column is cosmetic, IRR still 3.90%.
Formula-driven, more so than the reference in places.
Clean, correct formats.
Nothing beyond the ask.
Claude has the highest cell match of the three plugins and adds no separate errors. Its cash flows have the correct signs, but it makes the shared premium-basis error and shifts the amortisation schedule by one period. Almost every calculation uses a live formula. Its tie-out passes because both sides use the same wrong basis.
Only the shared premium trap; no tool-specific errors.
Almost entirely live formulas.
Clean, template-faithful.
Nothing beyond the ask.
CoPilot Business in Excel Online makes the identical premium error and the one-period shift. The notable part is the self-check: its panel reports a clean tie-out, GAAP tax equal to cash tax, no formula errors, all of it internally consistent because both sides carry the same bad basis and agree with each other. The verification reports clean and misses the error.
The shared premium trap; the self-check confirms the bad basis without catching it.
Formula-driven, polished.
Red-parenthesis negatives, correct formats.
Nothing beyond the ask.
Claude Code uses the same wrong tax basis, records a phantom $480 gain in 2021 and shifts the GAAP and deferred-tax rollforwards by one column. The workbook contains 95 live formulas and 18 literal zero seeds. Its result differs from the Claude plugin by one cell.
Same premium trap; GAAP and deferred-tax rollforwards shifted one column.
95 live formulas over 18 literal zero seeds.
Clean, correct formats.
Nothing beyond the ask.
The chart tasks produced very different results
Charts test a different set of Excel skills. The assistant has to interpret the brief, create the right chart object, connect it to the source data and format it correctly. Those tasks separated the tools more sharply than the formula builds did.
On the sunburst task, Claude and Claude Code created real charts linked to the source data. ChatGPT created a static image and pasted it over a bar chart, so the picture would not update when the numbers changed. Both CoPilot versions built genuine charts but assigned the colours to the wrong departments. ChatGPT’s result looked correct at first, but we only found the problem by opening the file and inspecting the chart object.
Sunburst chart
The tools produce three outcomes from the same brief: two correct charts, a picture of one, and two charts with the colours on the wrong departments.
The brief
Build a sunburst titled "Expense Breakdown": a three-level hierarchy of department into category into subcategory, sized by spend, with IT orange, HR blue and Sales grey, every label inside its segment.
Good looks like A live, data-linked chart that refreshes when the numbers change, with the three colours on the right departments.
Choose a tool
CoPilot Basic created a genuine sunburst with the correct title, hierarchy, sizing and labels. The colour mapping is wrong: IT is grey instead of orange, and two HR segments, Managers and Training, are orange. Its summary claims "HR blue, IT orange, Sales grey", which does not match the chart.
The chart is right; the colour key is on the wrong departments.
A live chartEx object, not an image.
Tidy and contained, but the colour mapping is wrong.
Nothing beyond the ask.
ChatGPT produced the best-looking result on screen: a donut with a "Total Spend $250k" centre, a legend, and the colours right. It is not a chart. The workbook holds a bar chart with a static image pasted over it, so nothing updates when the data changes. ChatGPT said as much itself, noting the runtime had no native sunburst. Judged on the screenshot alone it passes, which is the argument for not judging charts by eye.
The requested deliverable, a working sunburst object, is not in the file.
A static image sitting over a bar chart.
As a picture it is the most polished of the five.
It disclosed the chart was a static image, so a reviewer knows not to rely on it to refresh.
Claude built a genuine sunburst with the title, the three-ring hierarchy sized by spend, and the colours on the right departments: IT orange, HR blue, Sales grey. All fifteen labels sit inside their segments, including the two smallest. Each branch shades dark to light toward the rim, which reads as an intentional choice and helps you follow a parent ring to its children.
Right chart, right colours, right hierarchy.
A live chartEx object sourced from the data.
Contained labels and a considered shade gradient.
The per-branch shading aids parent-to-child tracking.
CoPilot Business in Excel Online built a real native sunburst, not a picture, with the title, hierarchy, spend-sizing and contained labels all on-spec. It then made the identical colour error as CoPilot Basic: IT grey, orange bleeding onto the HR segments. Both surfaces produced the same bug.
Right chart, colours on the wrong departments.
A live native sunburst.
Clean layout, wrong colour key.
Nothing beyond the ask.
The Claude Code run got to the same result through the Excel COM interface: a data-linked sunburst, exact title, the three colours mapped per department, all fifteen labels contained. Getting there cost twenty-one turns, seventeen of them spent on a wrong chart-type constant before the right one worked, which is why it ran six minutes against the plugin's fifty seconds.
Right chart, right colours, right hierarchy.
A live chartEx object, coloured per data point.
Contained labels, right department mapping.
None, and the turn count is the cost of this surface.
Bubble chart
Everyone can build a bubble chart, so the separation moves to axis honesty: CoPilot Basic truncates the revenue axis to $24,000, distorting every position, while Claude starts at zero and formats it in dollars.
The brief
Build a bubble chart titled "Product Portfolio Analysis": revenue on X, growth on Y, market size as bubble size, each product labelled, the largest bubble green and the smallest red, no legend.
Good looks like A native bubble chart that meets all of it, on an honest axis that starts at zero.
Choose a tool
CoPilot Basic meets all eight stated requirements, then undercuts them with the axis: the revenue axis starts at $24,000 instead of zero, which stretches the horizontal spread and jams the low-revenue product hard against the left edge. It does add dollar formatting, the format we prefer over the reference's plain numbers, but pairs it with a misleading scale, so a good format choice sits on a distorted axis.
Meets all eight requirements, but the truncated X-axis distorts positions.
A native bubble chart, one series per product.
Dollar formatting, undercut by the $24,000 axis start.
Nothing beyond the ask.
ChatGPT meets all eight requirements and starts the axis at zero, unlike CoPilot Basic. The revenue axis uses plain numbers rather than dollars, as the reference does. ChatGPT also adds axis titles that are not in the reference.
All eight met, the axis correctly starts at zero.
A native bubble chart.
Right scale, but plain numbers, no dollar format.
Added axis titles the reference omits.
Claude meets all eight requirements. Its revenue axis starts at zero and uses dollar formatting, while the reference uses plain numbers. The bubble colours match the reference. It completed the task in twenty-eight seconds, the fastest result in the study.
All eight met, axis starts at zero with dollar formatting.
A native bubble chart, exact reference colours.
Best axis of the field; added axis titles.
Nothing beyond the ask.
CoPilot Business in Excel Online meets all eight requirements and starts the axis at zero, although it does not use dollar formatting. One mid-size bubble is orange because a seed-series colour was not changed to the default blue. It also adds axis titles.
All eight met, the X-axis starts at zero.
A native bubble chart.
Honest axis, but one stray orange bubble and no dollar format.
Added axis titles.
The harness run meets all eight requirements with the exact reference reds and greens and cell-linked product labels. The polish misses are small: the chart sits over the data table, the Y-axis auto-drops to minus two percent, and the revenue axis stays plain numbers with no dollar sign.
All eight met, exact reference colours.
A native bubble chart, cell-linked labels.
Anchored over the data; Y-axis dips to -2%; plain numbers.
Nothing beyond the ask.
Line burndown
All five tools build the line chart correctly. Each follows the brief's date format and axis requirements more closely than the benchmark reference.
The brief
Build a line chart titled "Invoice Count Burndown August 2023": daily dates as dd/mm/yyyy at an angle, a 0 to 800 y-axis, a grey "Count to be Collected" series and a blue "Goal" series.
Good looks like All of it, on a native chart sourced from the data.
Choose a tool
CoPilot Basic builds the line chart to spec: exact title, the two dated series, angled dd/mm/yyyy labels, and an honest 0 to 800 axis it set explicitly. It is more prompt-faithful than the reference, which used a looser date format and an auto axis. The zero-padded dates are unambiguous where the reference's single-digit days are not.
All eight met; more faithful to the brief than the reference.
A native line chart; axis hard-set to 0 to 800.
Angled dd/mm/yyyy dates, explicit axis.
Nothing beyond the ask.
ChatGPT meets all eight requirements as rendered, with leading-zero dd/mm/yyyy dates that out-format the reference's single-digit days. It did not hard-set the axis; Excel's auto-range happens to top out at 800 for this data, so it renders 0 to 800 correctly but would drift if the data grew past 800. It added axis titles.
All eight met as rendered.
A native line chart; axis auto-ranges to 0 to 800, not locked.
Clean angled dates, added axis titles.
Nothing beyond the ask.
Claude builds the chart in a single clean run: exact title, honest 0 to 800 axis set explicitly, angled dd/mm/yyyy dates, grey and blue series. It is larger and more legible than the reference and, like the others, more faithful to the brief's date and axis spec than the reference is.
All eight met; more prompt-faithful than the reference.
A native line chart; axis hard-set to 0 to 800.
Clean single run, angled dates, legible.
Nothing beyond the ask.
CoPilot Business in Excel Online meets all eight requirements: the line chart, exact title, angled dd/mm/yyyy dates, and a 0 to 800 axis that comes from Excel's auto-scaling, with no explicit bound set. It added Date and Invoice Count axis titles the reference lacks, arguably clearer.
All eight met.
A native line chart; axis auto-scales to 0 to 800.
Angled dates, added axis titles.
Nothing beyond the ask.
The harness run meets all eight requirements: a data-sourced 41-day series, dd/mm/yyyy at an explicit minus 45 degrees, a hard 0 to 800 axis, grey and blue series. The chart is smaller with a right-hand legend, and the y-axis zero renders as a dash from an inherited accounting format, a cosmetic quirk.
All eight met from the full 41-day series.
A native line chart; axis hard-set to 0 to 800.
Smaller chart; the y-zero shows as a dash from an inherited format.
Nothing beyond the ask.
The debugging tasks were harder to verify
The debugging tasks asked each assistant to find a hidden fault, fix it and show that the workbook worked afterwards. A clean-looking result did not mean the repair was correct.
CoPilot said an unfixed workbook reconciled
At the end of the first debugging task, CoPilot reported that “Sources/uses, both balance-sheet years, and both cash-flow years reconcile exactly.” The planted double-count was still in the workbook, and it had introduced another one. Its checks did not test the parts of the workbook that were still wrong.
ChatGPT did something similar on the second debugging task. It cleared every error on screen but switched the workbook to the wrong scenario, which left some headline numbers up to eight times off. Its checks confirmed that the edited workbook was internally consistent; they did not catch that it was calculating the wrong case.
Audit and fix a broken model
CoPilot Basic left the planted double-count in place and added another. Claude was the only plugin that fixed all sixteen affected cells.
The brief
Audit and fix the workbook thoroughly: formula-logic errors, hardcodes that should be formulas, broken references, and anything affecting accuracy or maintainability.
Good looks like The planted double-count found and fixed, the model still live, nothing new broken.
Choose a tool
CoPilot Basic left the planted double-count in place and added a second one, rewriting a previously-correct EBITDA row to be cumulative. It also froze about eighty live sensitivity cells into static text, the opposite of the task. Its chat lists verified outputs that reconcile exactly, none of which match the reference and none of which mention a double-count. A reviewer who trusted it would ship an overstated model.
Left the planted double-count and introduced a new one.
Froze about eighty live sensitivity cells to static text.
Clean and professionally formatted.
Its self-check did not identify either double-count.
ChatGPT identified the double-count and edited the relevant parts of the workbook, but its revised formulas produce different entry values and returns from the reference. It also replaced the live sensitivity grids with hardcoded strings and added error guards that are not in the reference. It found the problem but did not repair it correctly.
Diagnosed the double-count; the re-derivation diverges from the reference.
Froze the live sensitivity grids to hardcoded strings.
Clean, professional.
Nothing beyond the ask.
Claude removed the cumulative term that caused the double-count and matched the reference on all sixteen affected cells. It is the only plugin that completed that repair correctly. Its overall cell match is lower because a small number of upstream changes affected cells across seven sheets. Claude also disclosed that it had cleared the sensitivity tables because it could not rebuild them.
The only plugin tool that fixed the planted double-count, exactly.
Leaned hardcode-to-formula, but cleared the tables it could not rebuild.
Clean; the changes are logic, not cosmetics.
Openly flagged the tables it had to clear.
CoPilot Business returned no result.
CoPilot Business in Excel Online failed to complete the edits after two attempts and returned no result. The web surface could not finish the largest model in the study.
The harness run fixed the planted double-count exactly, sixteen of sixteen, and kept all four sensitivity tables live where the plugin run had to delete them. It reproduced the reference's gross-debt fix, hit the same cohort relink cascade, and left zero errors. Its entire reply to the user was one word: DONE.
Fixed the planted double-count exactly, sixteen of sixteen.
Kept all four sensitivity tables live.
Zero residual errors.
Terse: it replied only DONE.
Repair the broken references
ChatGPT cleared every visible error but calculated the wrong scenario. The prompt and reference disagree about which scenario to use, so only tools that followed the reference produced its numbers.
The brief
Audit and fix the workbook, restoring structure before dependent formulas: deleted rows throwing reference errors, broken cross-sheet references, sheet-name typos, and cascading failures.
Good looks like Every reference error cleared to the reference's numbers, the structure restored, the tables still live.
Choose a tool
CoPilot Basic fixed two whole error classes, the broken selector references and the mistyped sheet names, and broke nothing. It then disclosed precisely what remained, the exact count of recalc-pending cells and the manual steps to clear them, matching the judge's own count to the cell. It skipped the structural row-restore the prompt asked for first, and followed the prompt's ambiguous case selector into the Downside scenario.
Fixed the reference errors and typos; skipped the row-restore, followed the prompt into Downside.
No broken formulas left; the residuals are recalc-pending, not broken.
Broke nothing; the untouched sheets are identical to the reference.
Disclosed the exact residual count and the recalc steps.
ChatGPT cleared every reference error, kept the sensitivity tables live and had the highest cell match. It also forced the operating case to Downside while the reference uses Base. As a result, the entry multiple, valuation bridge and exit equity are up to eight times away from the reference values. The workbook has no visible errors, but it calculates the wrong scenario.
Zero errors, but the operating case is forced to Downside; headline numbers up to 8x off.
Structurally clean; kept the data tables live.
No residual errors.
The clean metrics mislead.
Claude cleared every reference error, and each sampled formula matches the reference exactly. It also kept the intended scenario. Its numbers are about four percent low because the headless run could not complete the circular-reference recalculation described in the prompt. Claude replaced the sensitivity tables with static values, so they no longer update.
Closest to the reference; formulas byte-identical, scenario kept correct.
Byte-identical formulas, but hardcoded the sensitivity tables.
Zero residual errors, cleaner than the reference.
Nothing beyond the ask.
CoPilot Business returned no result.
CoPilot Business in Excel Online was unable to complete the repair and returned no result, the second debugging task the web surface could not finish.
The harness run restored the deleted row, cleared every reference error, purged the external links and actually converged the circular reference in live Excel with the data tables kept live, formulas equivalent to the reference in all but one of 1,933 cells. It set the operating case to 3 as the prompt states, where the reference keeps it at 1, so its returns read 1.4x against the reference's 2.4x, following the prompt over the answer key.
Most complete repair; followed the prompt's case selector, so returns diverge from the reference.
Converged the circular reference live; data tables kept live.
Zero residual errors; the deleted row restored.
The only arm to converge the circular model in live Excel.
Four checks mattered across all eight tasks
These are the four checks I would use on any workbook edited by an AI assistant.
The confident wrong answer
The hardest errors to spot were in workbooks that looked finished. We had to double check all of the claimed repairs to establish whether it could be trusted, which requires a reviewer who understands the workbook.
Live formulas
A correct number is less useful if it has been typed into a cell and no longer updates. CoPilot Basic hardcoded the yield helper column on the first build, so its current numbers matched but would not change with the inputs. On the debugging tasks, several tools replaced live sensitivity tables with plain text.
The extras you can’t count on
Some assistants added useful work that the prompt had not requested. CoPilot Business in Excel Online added live tie-out checks, an organised ‘inputs’ block and coloured table headers. Several tools added axis titles that made their charts easier to read. The additions varied from task to task, so they are worth checking for rather than assuming they will be there.
The interface changed the result
We tested Claude and CoPilot through two alternative ways of accessing Excel. CoPilot Business (using Excel online) reproduced several errors from CoPilot Basic (using Excel desktop) and could not finish either debugging task. It made two attempts at the first one, and still returned no result. Claude Code often produced similar results to the Claude plugin, but it needed twenty-one turns to build the sunburst because seventeen were spent trying the wrong chart-type constant.
How long you wait, and why it matters
The fastest task took twenty-eight seconds and the slowest took nearly nine minutes. Among the three plugins, Claude had a median time of 1:15. ChatGPT’s median was 3:04, about two and a half times as long, and its two debugging runs took between eight and nine minutes. The interface also affected the time: Claude Code needed 6:22 for a sunburst that the Claude plugin completed in fifty-one seconds.
How the assistants compare on speed, ranked fastest to slowest by median. Each dot is one task; the bold tick is the tool's median. Expand for a full-size view.
View as a table
| Tool | Configuration | Median | Range | Tasks |
|---|---|---|---|---|
| Claude | Fable 5.1 · High | 1:15 | 0:28 to 7:04 | 8 |
| CoPilot Basic | Auto · Auto | 1:50 | 1:00 to 6:42 | 8 |
| Claude Code | Fable 5.1 · Medium | 1:56 | 1:06 to 6:30 | 8 |
| CoPilot Business | Auto · Auto | 2:42 | 1:45 to 7:10 | 6 |
| ChatGPT | GPT-5.6 · High | 3:04 | 1:55 to 8:57 | 8 |
One run per tool, wall-clock to a usable answer. CoPilot Business is measured over six tasks, not eight: it returned no result on the two debugging workbooks, which are the slowest, so its median understates where it would land on the full set.
How I’d use these tools in practice
A few simple checks would have caught most of the errors we found.
Know what to check
Four problems kept recurring:
- a wrong answer accompanied by a clean self-check
- a chart that is actually a pasted image
- an assistant that has answered a different question
- a hardcoded number where a formula should be
These are all typical errors that you can get ahead of.
Be specific about what you want
One debugging prompt said the scenario selector was “3”, which two tools interpreted one way and the others interpreted another. The chart brief named the type, colours, axis format and labels, which made every deviation easy to identify. Give the assistant something specific that you can check.
Ask it to show its checks
Several assistants said “verified, no errors” while important defects remained. CoPilot Business added live tie-out formulas to one schedule, which let us test its claim. Ask the assistant to add checks you can inspect, then verify them yourself.
Give it one job at a time
A single chart or schedule was easier to check. Broad instructions to audit a whole workbook led to unnecessary edits, the wrong scenario or a missed planted bug. Give it one change and check that change before asking for the next.
Test performance on your common tasks
The setup mattered. One model created a real sunburst while another supplied an image of one, and CoPilot Business could not finish the two debugging workbooks. Test the actual tool and interface on the kind of Excel work you expect employees to give it.
Where I would use them
These tools can save time on specific Excel tasks, but someone who understands the workbook still needs to review the result. That review should include:
- checking the formulas behind the numbers
- confirming which scenario the workbook calculated
- verifying that charts are linked to the source data
- running checks that do not depend on the assistant’s own calculations
I’d start with a chart that has a clear brief, a schedule with defined inputs and outputs, or one specific repair. Someone should still inspect the result before using it. A broad workbook audit needs closer review because the assistant may change more than you asked or miss the fault entirely.
Before rolling out one of these tools, decide which Excel tasks employees can use it for and how the results will be checked. Based on this test, I would not let any of them make important spreadsheet changes without review.
How we ran it
We ran the study on 10 September 2026 from identical starting workbooks. Most task and tool combinations ran once; CoPilot Business in Excel Online received a second attempt on the first debugging task after it failed to complete. In Desktop Excel, we tested CoPilot Basic, the ChatGPT plugin on GPT-5.6 Sol at high reasoning, and the Claude plugin on Fable 5.1 at high reasoning. We also tested Claude Code on Fable 5.1 at medium reasoning through a headless harness, and CoPilot Business in Excel Online on its Auto setting. Microsoft doesn’t expose the underlying CoPilot model or setting used in Desktop Excel.
We reviewed each workbook manually and used the golden reference to locate differences. The cell comparison did not determine whether a result passed. Where the reference used a weaker interpretation of the prompt, we did not mark an assistant down for producing the more sensible answer. Every verdict reflects the quality of the finished workbook.
If you’re choosing AI tools for employees and want to test them against real work, get in touch.
David has spent ten years on the inside of big technology programmes, five consulting to some of Australia's largest businesses and five before that at one of the world's biggest FMCG firms. He co-founded Cassidium, where he runs delivery on CRM implementations, RevOps builds and AI projects. He is an optimist about where AI is heading, and writes about how it shows up at work: the good, the bad and the occasionally ridiculous.
David on LinkedIn