Less than 10%. That is how often two leading AI models agreed when a research studio asked each to name one candidate in nearly 500 US midterm races. Same prompt and same races, yet one leaned one way in 92% of answers and the other the opposite way in 97%. Leave the politics aside: for a finance function the useful number is the agreement rate, because the same mechanism already runs inside tools that score your suppliers.
Like most of us in audit, I spent years treating system output as neutral once the IT general controls passed. If the report came from the system, the system had no opinion. That assumption was already shaky for spreadsheets. For models it does not hold at all.
Picture the onboarding queue
Picture an ordinary Zoho implementation at a trading group with entities in Jakarta and Coimbatore. The finance team connects a large language model to vendor onboarding through an API. It reads registration documents, scans news and returns one word per supplier: approve, review or reject. Queue time drops, and the controller stops chasing.
Now picture the integration vendor upgrading the model in a routine release, or switching providers because the other one is cheaper per token. Nobody in finance is told, because nothing on the screen changed. A supplier that cleared last month is now flagged, or a flagged one now clears. The approval matrix still shows the controller’s name against every decision.
This scene is hypothetical. The mechanism is not, and this week’s study is the cleanest public evidence of it that I have read.
What the study shows, and what it does not
The study was run by Convovo, a conversational AI research and design studio led by Cornell professor Tapan Parikh, and reported by TechCrunch on 24 October 2026. The researchers sent one prompt through the developer APIs of SpaceXAI’s Grok 4.7 and OpenAI’s GPT-6.1 Sol, demanding a single named candidate with no hedging. As reported, Grok picked the Republican in 92% of its answers and GPT picked the Democrat in 97%. Across nearly 500 races they agreed less than 10% of the time.
Convovo has published the race-by-race answers, three per model per race, so the figures can be checked.
Now the limits, because a sceptical CFO will ask. The consumer apps refused the same question, and an OpenAI spokesperson said the API test does not reflect how people use ChatGPT. SpaceXAI had not commented when TechCrunch published. I have read the coverage and Convovo’s results page, not an independent check of the data, so treat every figure here as reported.
The follow-up matters more to us than the headline. With web search switched off for a 30-race sample, GPT still picked the Democrat in 24 races, while Grok declined to choose in 27, according to TechCrunch. So part of the lean came from what each model retrieved and part from what it carried in. Two different roots, one output that looks equally confident.
Why the API detail decides it
The consumer app said no. Your accounts payable tool does not use the consumer app. Invoice coding, credit scoring, vendor risk: every enterprise integration calls the model through the same kind of developer interface this study tested, usually with a system prompt telling it to decide. The guardrails that make headlines live in the chat window, and the judgment that reaches your ledger lives behind the API.
For an auditor this is familiar ground. ISA 500 paragraph 9, in the version most current audits still apply, requires us to evaluate whether information produced by the entity is sufficiently reliable for our purpose, including its accuracy and completeness. Across the Indonesia–India–Singapore corridor the equivalent sits in each jurisdiction’s own SA 500 or SSA 500, so check your local standard for the exact wording. When the information is a model’s judgment, reliability carries a dimension we rarely test: would the same input give the same answer tomorrow, or from a different model?
On the management side, ISO/IEC 42001:2023 already points here. Its Annex A control A.6.2.4 covers verification and validation of AI systems, so a certified organisation should be able to show what it validated and against what. In many corridor groups, my field judgment is that validation still means a demo before go-live and nothing after.
The study also shows where to look. The two models leaned on very different sources when they searched. In a finance workflow, the retrieval list is an input like any master file. If a vendor-risk model reads one set of news sites and its replacement reads another, the supplier file has not moved, yet the verdict can.
The number worth reporting upward
The usual advice is to pick the best model: read the benchmarks, run a bake-off, sign the contract. That treats model choice as procurement. I think the more useful number is the disagreement rate between two competent models on your own cases, because it tells you how much of the judgment belongs to the vendor and how much to the facts.
A low disagreement rate means the facts are doing the work and the model is a fast clerk. A high one means the model is exercising discretion on your behalf, and discretion is exactly what an audit committee exists to oversee.
If two models given the same question agree less than one time in ten, then the model your vendor picked is making the judgment, and your team is signing it. Switching that model behaves like a change in accounting estimate that arrives wearing an IT change ticket. Nobody takes it to the audit committee. Somebody should.
None of this argues against using models. Medicine runs on instruments that disagree and handles them with calibration, second readings and written protocols. Our profession can borrow those habits now, without waiting for a standard to force them.
The Monday test
Inventory. Ask your financial controller for a list of every workflow where a model output feeds a judgment: vendor risk rating, expected credit loss staging, journal anomaly flags. For each, record the model name, version, and whether it is called by app or API.
Disagreement test. Take 50 closed historical cases from one workflow. Run them twice through the current model and once through an alternative model, and record the agreement rate. File it as validation evidence under ISO/IEC 42001:2023 A.6.2.4, or in your ISA 500 paragraph 9 reliability work paper.
Change control. Ask your IT head to show where a model or version switch appears in the change log. If it does not, add model version and retrieval source list to the scope of your ISO/IEC 27001:2022 A.8.32 change management test, with CFO sign-off for judgment workflows.
Also during last week
Agents logging in for you. 1Password launched 1Password for Agents, letting AI agents sign in to personal accounts without seeing the password, with the user confirming each request. The same report cites Axios saying US banks now detect agent activity in accounts and most cannot tell if the user asked for it. User access reviews will need a new question: which logins were human?
Personal AI features and ads. Google’s privacy policy, as of 16 October, adds an exception: Gmail, Drive and Photos data may shape personalised ads if users connect those services to its Personal Intelligence features, according to The Verge. Advertisers do not receive the content itself, per the same report.
If staff move work files through personal Google accounts, my inference is that those files now sit under that policy. Worth one line in your acceptable use review.
UPI concentration in India. NPCI app data for September 2026 shows PhonePe at 45.57% and Google Pay at 32.23% of listed-app UPI volume, 77.8% combined. Both sit above the 30% level in NPCI’s volume-cap circular, whose deadline for existing providers runs to 31 December 2026. The cap is tested on rolling three-month volumes, so one month’s share is not the compliance measure.
One last thing
Any Tamil household already knows this. Give two aunts the same recipe and you get two sambars, each served with full confidence, and nobody calls the recipe neutral. We taste before we serve.
Reply and tell me: which judgment in your month-end close already runs through a model, and who picked that model?
Lift as you Rise.