Technology Solutions & Practices Shared Interest Group

The one thing I've always been wary about is trusting technology with my numbers.

When I was younger, it was a fancy Excel formula that could do a lot of magical things, but if that formula was wrong, it would mess up a lot of cells at once. Then I discovered Power Query and Power BI. I could do the math at scale, hundreds of thousands of lines instead of a handful of cells. The risk was the same shape, just bigger: if I made a mistake centrally, in the model itself, it would mess up everything downstream.

But with that older technology, you could search through and figure out exactly where the mistake was, fix one formula or one measure, and everything downstream would correct itself in one fell swoop. That magic doesn't exist with AI. It's non-deterministic, and it doesn't necessarily keep a clean audit trail unless you specifically instruct it to. There's no single fix that ripples back and cleans everything up.

Traditional technology excels at the mathematical work itself. What I think AI does well is the first cut, maybe the second cut, of analysis. But I don't feel comfortable with it crunching my numbers yet.

How do you all feel about AI actually doing the math? Preparing a P&L? Preparing a footnote?

------------------------------
Daniel Harley CMA,CSCA
Director/Manager
Newtown PA
https://www.linkedin.com/in/danieljharley/
------------------------------

First, let me address the obvious elephant in the room that will pop up. I am replying to my own comment lol. 

So my thoughts on this are that I'm okay with AI doing math for me on a very simplistic level with data that I'm really familiar with. I know the rough number, and I'm trying to get a more precise number without putting in much effort. Let's take a good example. I connected Apple Health to my ChatGPT account, and I'm analyzing a little running program that I'm doing with my daughter. I really don't want to spin up a spreadsheet and track it manually and so on and so forth. That's worked out really well, but I remember the last five runs roughly, so when it's way off, I know that's way off. When it's not updated, I know that it's not updated. The error rate for this endeavor is actually remarkably high, so it gives me pause. 

That being said, I'm 100% comfortable with AI creating my Excel formulas. The reason that I'm comfortable with that is because I can audit it. When I ask AI to calculate whatever analysis for me and I have them do it in Excel, I'm comfortable with that because I can go back and try and understand that. I don't rely on AI for anything where it gives me a number and not a concrete, auditable way for me to verify it. 

I can be a little skeptical at times, and my skepticism leads me to places where I don't know how many things are actually wrong. I think it's perhaps tainted a little bit by the fact that I've had employees who use AI, don't check it, and their math just doesn't add up. Funny thing is, that happens without AI as well, so I'm not saying AI is creating a new problem. It's just not solving for the old one. 

------------------------------
Daniel Harley CMA,CSCA
Director/Manager
Newtown PA
https://www.linkedin.com/in/danieljharley/
------------------------------