Discussion about this post

User's avatar
Laurie Stark's avatar

I spent some time doing contract work with a team of analysts that almost exclusively used AI for analysis. I don't mean that they asked AI to write complex SQL queries or occasionally asked it to label unstructured data. I mean that they uploaded the dataset and told it to "act as a senior analyst" and write the report, and then they copy and pasted the report into a doc and sent it.

They weren't being lazy or irresponsible, that's quite literally what we were instructed to do, and the workload didn't allow for anything much more 'bespoke' than that. I used AI to write Python scripts or Google Apps scripts or to help me finesse the wording of a tricky paragraph, but I refused to use it for wholesale analysis because every time I tried, the result was absolute nonsense. Or at least it was like 30% nonsense, which I personally believe is too much nonsense.

But what I realized over time is that it actually didn't matter. The people charged with reading the reports were mostly not reading them anyway, and the ones that did mostly didn't care if the data was even accurate, let alone if it was statistically significant or used a "rigorous methodology." They just wanted a stat they could bring to their boss to say "something I did worked" or to say "here's why we should do this idea I have."

8Lee's avatar
1dEdited

As you said: “Claude is a bunch of loops.”

Isn’t that everything in life and business? A loop of gathering information to guide the next loop (that gathers more info)?

Analysis is literally this. And as these things become more widely adopted the tentacles of those loops will reach into every single department, regardless of our concern.

The loops don’t scare me. The people who don’t want this do. They concern me greatly. How did they learn anything? A feedback loop. An imperfect one that was refined over time that eventually built habits, beliefs, and intuition (we call this “taste” now).

It’s how any child learns anything.

AI won’t eat your lunch; it’ll just make it far less interesting than the kid who brought a full course meal to their grade school cafeteria and then shared it with others.

And we all know what it’s like to show up with a soggy ham sandwich.

19 more comments...

No posts

Ready for more?