I spent some time doing contract work with a team of analysts that almost exclusively used AI for analysis. I don't mean that they asked AI to write complex SQL queries or occasionally asked it to label unstructured data. I mean that they uploaded the dataset and told it to "act as a senior analyst" and write the report, and then they copy and pasted the report into a doc and sent it.
They weren't being lazy or irresponsible, that's quite literally what we were instructed to do, and the workload didn't allow for anything much more 'bespoke' than that. I used AI to write Python scripts or Google Apps scripts or to help me finesse the wording of a tricky paragraph, but I refused to use it for wholesale analysis because every time I tried, the result was absolute nonsense. Or at least it was like 30% nonsense, which I personally believe is too much nonsense.
But what I realized over time is that it actually didn't matter. The people charged with reading the reports were mostly not reading them anyway, and the ones that did mostly didn't care if the data was even accurate, let alone if it was statistically significant or used a "rigorous methodology." They just wanted a stat they could bring to their boss to say "something I did worked" or to say "here's why we should do this idea I have."
This reminds me of when I started in consulting and a senior manager had to send a status report each Friday changing the date in the filename. He knew nobody read it so he just sent the same file every week rather than sweat over every line.
What are you talking about? Data analysts are cautious because if you’re wrong once, stakeholders stop trusting you. When I’ve tried to let copilot act as a senior analyst, it spits out nonsense so I use it to write subqueries and individual cells, which it’s quite good at.
Also, like there’s real world consequences to being wrong? Just on a daily basis, things stop adding up. And, since I work in product growth, the product stops growing lol
It sounds like your work is much more directly connected to outcomes than the work on this team was, which is enviable! I suspect most analysts on that team would have loved for their work to have more real world impact.
Apologies — I meant to reply to Ben, not you. But, yeah, I mean, sadly, lots of consultants bullshit their way through life. A long term embedded data team can’t plausibly get things wrong all the time and not see consequences
What you said though really hits the nail on the head for me. The reason I care about data accuracy is that I think data should be (one of the things) used to make actual decisions that have actual real world consequences. When it's just window dressing, it genuinely kind of doesn't matter if it's accurate or not, which I find depressing.
I think the distinct I'd make here is on reporting that's presented as fact, vs "research" or things that are in effect more like brainstorming. If we ask AI to replace data teams by automating their reporting work - and to do so with the same implicit guarantees of accuracy that that comes with today - for sure, that probably creates all sorts of problems.
But does that change if you're asking it to look for compelling moonshot ideas (which is often the sort of work that data teams aspire to do, and say they're bogged down in the reporting work). Even if it's not always accurate, that email - "Ten hypotheses about big things we could do" - is one that I'd open every week.
Hmm, I don't know about this one. There is the implementation-focused techbro viewpoint on analytics: "I just build dashboards when asked, I don't know who uses them, and I don't care - probably they don't either", and then there's actual analytics. Say, in a manufacturing setting, for example, an analytical solution that optimizes the various inputs of a chemical process and provides the operator of a honking big machine with instructions and warnings is *hardly* the same as the stereotypical concept of a monthly board report that no-one reads.
It's fun to attack the latter as pointless busywork, and such busywork could (and maybe should) be easily replaced by an army of bots fishing for "insight", but that's not nearly the full picture. In real life the numbers and trends and metrics do actually often matter, some analytics teams do actually work together with their business counterparts, and if you (or your bot army) screw these up you lose their confidence in you and probably your job as well (or you end up assigned to dashboard busywork duties).
On the manufacturing plant problem, for sure, I tend to separate "analytics via recommendations" from true operational stuff that tells you if machines are running correctly or optimizes your delivery fleet or whatever. To me, those aren't really problems where you're looking to discovery anything, and they're more akin to outright engineering problems than data problem (eg, they are programs with numbers as much as they're analytics). And business dashboards seem generally similar to me, except, because they're usually much cheaper to build than the operational stuff, there are a lot of useless ones. But, in both cases, those aren't really quite "analytics" to me, and to the extent that they are, it's because "useful insights" are often hard to find, so we make dashboards, because we need to do something to make ourselves feel useful.
On the point about trust though, that's certainly true today. I'm not convinced it stays that way, though. People care a lot about trust in analysis today because it's slow; people think about it fairly slowly; etc. If analysis becomes much more about rapid brainstorming - here are my ten ideas today, what do think? - mistakes are just part of the process. You say, "these two sound really interesting, go look into them" and then you find out one is wrong, and that's ok. I think trust matters to data teams because they've sold themselves as a team you're supposed to trust. The could also become a team with wild ideas that you might want to double check, and I think that's a perfectly coherent approach too.
Isn’t that everything in life and business? A loop of gathering information to guide the next loop (that gathers more info)?
Analysis is literally this. And as these things become more widely adopted the tentacles of those loops will reach into every single department, regardless of our concern.
The loops don’t scare me. The people who don’t want this do. They concern me greatly. How did they learn anything? A feedback loop. An imperfect one that was refined over time that eventually built habits, beliefs, and intuition (we call this “taste” now).
It’s how any child learns anything.
AI won’t eat your lunch; it’ll just make it far less interesting than the kid who brought a full course meal to their grade school cafeteria and then shared it with others.
And we all know what it’s like to show up with a soggy ham sandwich.
I wasn't really a believer in this for a while, but more and more, I struggle coming up good counter-arguments. To your point, it's less about how AI is doing something divine; it's more that I don't think I do anything all that special. I ask myself questions, I look stuff up, I repeat.
People periodically say stuff like, ah, but, the AI doesn't know. Which is true, but that feels very stuck in the "AI is ChatGPT" mindset, where it's just prompt and response. But we seem pretty far beyond that at this point. Once a model gets good enough to deal with fairly ambiguous stuff, and once a harness gets good enough at looping itself without spiraling out of control, it's hard to see where that stops.
I think about chaos and how a lot of people see that as scary; they avoid anything that feels, smells, and tastes like it.
But then the few say: Creativity and innovation require the context of chaos to even begin. Out of nothing we create something. Out of the storm delivers a real gem.
The question is one of control. Anyone trying to "control" AI has already taken a backseat. This is so counter-intuitive for most that they won't even consider their built-in bias to control because it's based on fear.
But the best entrepreneurs and inventors across time and space and history never really saw it that way. They saw it as part of their canvas. Their plaything. I think AI is the same. Let the mf just go.
I'm not sure I'm quite so optimistic about it, but I'm also pretty sure there's nothing to do to stop it at this point. And at least as far as creativity goes (to the extent that a blog about dumb tech nonsense is "creative"), smashing a bunch of weird chaotic ideas together in your head is the only reliable way I've found to come up with anything. So, we at least have a lot of that these days.
"Once a model gets good enough to deal with fairly ambiguous stuff..."
I'm not sure there is any reason to believe this will happen. An LLM is not good at ambiguity. It is not good at nuance. This isn't surprising because their very architecture is not built in a way to handle this. It cannot reason.
Genuine question - why do you say that? At least in some domains, I find them very good at that.
Like, one of the things I've realized as I use them is I talk to them at the wrong level. It's very hard not to give them specific instructions about what you want and what they're supposed to do, but increasingly, I've realized you're better off just telling them what you're thinking. If you're stuck, say that. If you have a problem and don't know how to solve it, say that. If someone's complaining about this thing and you don't know what to do about it, say that. Trying to translate your ideas into instructions - ie, trying to resolve the ambiguity, rather than giving it to it - seems like what you're supposed to do. But, man, they seem to do just fine when I don't do that.
I guess it depends on what you're trying to accomplish (and, maybe what you mean by ambiguity). If you're brainstorming, sure, be general and ambiguous. That will probably lead to less predictable inference.
I'm thinking about trying to get valid answers from an LLM, where the words I use in my question / prompt are intentional. In my experience so far, the LLM doesn't process them very well. It ignores some or makes assumptions about something. It confidently gives me false answers; it does not ask for clarification before answering. I can't imagine how being less precise will help. If I ask a question that is more ambiguous, it's still going to process my prompt into tokens and do its inference thing. They don't seem to ask clarifying questions before answering, like a thoughtful human might (no, not all humans ask clarifying questions before answering questions).
Maybe I'm not using the right model, someone will say. Maybe.
Yeah, that makes sense. I think that was a bit of the turning point for me on these things. As answer machines where I have a question and ask for a factual answer, they're not great. They are like people in that way, where you can only kinda sorta trust it. But if you ask them to help work through something (and often, i just say that - "talk through this with me") they can be quite good. They can still be a little over eager, but they often just ask questions back, and they often either say something useful, or their questions make me realize something useful on my own.
I think that our current framing for junior analyst positions will go away once the right scaffolding is in place. I'm less confident that Sr+ goes away, except for roles that didn't really need effective analysts.
And so, if you work in credit analytics, finance, internal consulting, or something where accuracy is a critical to quality value, then I expect the need to not change very quickly.
yeah, I think that's fair. The argument here definitely isn't that they are all that accurate. It's much more that, for certain types of problems, volume - and especially creative volume - can sometimes be more useful than accuracy.
I think saying OpenAI and Anthropic are two of the world's most valuable companies needs a huge caveat; this is only because a relatively tiny number of a certain class of investors. A class that is wrong far more often than it is right. We'll see.
Fair, though, if someone gave you chance to take a big short on either of them, would you do it? The smart answer may well be yes, but, I'm not sure I'm taking that bet.
Probably not, because I don't like risks like that. Not to pick on them specifically either. Other pre-IPO companies are / have been similar. Arguably Databricks is overvalued. Snowflake came out of the gate way too hot when it IPOd (I was there).
Sure, yeah, and I certainly wouldn't claim to have any particular takes on how much these companies should be worth, or if something like Databricks is overvalued or not. But OpenAI and Anthropic definitely seem like *could* be enormously valuable; they both do make tons of money (in revenue, obviously); and they bot do seem to be almost impossibly influential. Of course, they could also implode, though I don't think that changes the underlying point, which is that you can get a lot of attention and traction by doing what seems like the extreme thing.
This is a useful provocation. The obstacle to greater autonomy may be less the agents themselves than the absence of an analytical equivalent to tests and CI.
The broader direction seems to be AI checking AI, but that only becomes practical when the checker is grounded in something external: benchmark cases, semantic definitions, reconciliations, domain invariants, or observed outcomes.
Perhaps the path forward is broad autonomy in exploration, paired with much stronger verification infrastructure?
After reading your other article, I think we may have slightly different takes on this.
I agree that an artifact cannot be verified without access to the underlying data. Even when the data is public, an independent tool would still need to know which datasets, definitions, and methodology were used.
What I had in mind was something closer to an internal auditor. The AI data scientist would provide the artifact, identify the underlying data, and explain the methodology. There might be some back-and-forth before the auditor either rejected the work or approved it, possibly with qualifications.
The external benchmarking would test the auditor itself against known good and bad artifacts, along with their underlying data. I am imagining something analogous to testing coding agents against historical GitHub issues and PRs. The goal would be to establish that the auditor is generally effective at reviewing analytical work.
I am not sure whether this would work, or what the benchmark dataset would look like. It also would not help us trust artifacts produced by other organizations. But it might make us more comfortable giving AI greater autonomy to perform analytical work within our own.
I work as a data scientist and my work is some combination of model development, ML pipelines, and analytics infrastructure and analysis. I use AI near constantly in the form of both a coding agents and chatbots to consult with.
The difference between using an LLM for coding and for analysis is stark. I can use an LLM generated python pipeline basically out of the box as long as I was careful in my initial prompting. LLM SQL queries and data analysis are bad though; even with careful explanation of the levels of aggregation and ways to approach joins and the full context of the repo generating analytics databases, the outputs are often completely unusable.
I don't think I'm just being cautious; I use LLMs constantly in other areas of my work including for production code. When I say unusable I mean "either unable to run or visibly incorrect to the point where an executive would notice," not just "doesn't meet my carefully honed spidey sense for data accuracy and completeness."
How much do you think that's the harness, versus the nature of the work? My theory on this is basically this:
1. Part of the reason coding tools started to work is because they weren't conservative about costs. They basically said, "loop until you get this right," and built in various testing mechanisms to do that.
2. Admittedly, that's easier for coding tools because you can see if the thing compiles or tests pass. For data work, there isn't as clear of check.
3. But there are *some* checks. People do them, after all. They check if the query works; they compare it to other numbers; etc. There are "tests" in analytical work; they're just a lot messier.
4. LLMs can probably do those too. It's not easy, and it's expensive, but it's not impossible. Still, most analytics vendors don't do that, because it's kinda hard, and because it seems like it would be expensive.
5. But eventually, someone will. And while it will be expensive, it'll still be a lot cheaper than having a person do it.
Which is to say, it would surprise me if the models aren't there already. And what they need isn't necessarily capital C-Context - a perfect semantic ontology and just the right skills - but more like, a lot of access to go sniffing around, and a big budget to spend doing it.
It seems you’re describing the most depressing conclusion of our current economy. Winning is equal to popularity. Accuracy doesn’t matter because no one really cares. Art is defined by output rather than impact. I still believe real art takes time, speed has a cost, and people who care make better stuff even when it may not “win”
On winning being equal to popularity, unfortunately, yeah, that kinda seems to be the world now. (Which, again, I wouldn't say is *good*, but does seem to be how it goes.) On the other stuff though, I'm not sure I'm quite so cynical about it. My point is less that accuracy doesn't matter; it's more that for some type of work - brainstorming, prototyping, searching for valuable analytical needles - accuracy isn't as useful as exploring a bunch of creative ideas. It's less about nobody caring about something being good, and more about finding ways to point the machine that sometimes gets things wrong at stuff that it's ok to now always get entirely right.
Is the valuation, and the hype, coming from the fact that no one has a sane counterpoint that is able to scale the way these cowboy companies are? In the absence of which, FOMO wins.
They are winning until a valid counterpoint arrives. Just like all the banks in the old, "wild" west, these companies will collapse into a heap of dust.
Analysis is testable in the same way that programs are testable. You can test that you are using the statistics correctly. But which statistics are you using and why?
AI isn't good at figuring out what program needs to be written in the same way that it isn't good at encapsulating all the knowledge that goes into data analysis.
Sure, insofar as analysis is the math. But I think analysis is a lot messier than that in two ways? First, it's really hard to judge what is "right." Is playing the numbers right if the result is bad? In cases where you don't really know the numbers (which is most situations), how do you even know if you're playing the numbers? And second, even if you can say if the math is right, a lot of analysis is just judgement. It's closer to a legal argument than a mathematical proof. And what makes a good legal argument? There's some amount of symbolic logic, but a lot of it is fuzzy grey areas.
Essentially open the Pandora's box of "give it full control" and let the end users handle the rest.
It's true to a point. How many flawed analyses and massaged metrics and nonsense graphs have business users generated and accepted over the years before the AI slop came for us all? Why wouldn't they want to pay the low low cost of having AI spit out some stuff so you can call yourself "data-driven"?
If the bar is in hell, why the hell not? No need to clutch pearls in the attention economy.
Does that change if we know that's what it's trying to create though? I agree that a bunch of robots that crank out "look, I made you a yet another sales tracking report!" isn't useful at best, and drowns you in untrustworthy slop at worst. But it's more of a brainstorming type thing, where it attempts to propose big ideas (which is often what data work aspires to be, and gets stuck in the reporting prerequisites), I'm not sure it has the same issue. To me, slop is slop because it's trying to pretend it's something else, or hide itself among more valuable things. So what if it's more direct about what it is?
Ok, first, this is pretty cool. Second, I do wonder what the right balance is here. On one hand, all the semantics and guardrails seem like they should be useful, and you do want these sorts of analyses to be right (like, obviously). On the other hand, it does seem possible to put too many guardrails in place. In one way, where it constraints the creativity of the model, and nudges it to much towards reporting and whatnot (that seems resolvable though). And in another way where, if it's accurate enough, you might start to treat it like a BI tool and always want the right answers, rather than a brainstorming tool that can be wildly off, but interesting.
True. It depends on what your expectations are, as a user. What I'm noticing is that even though people know "AI is prone to errors" and "you need to validate whatever AI tells you", nobody does. Whatever comes from ChatGPT, people tend to treat as true and reliable. They rarely validate it.
So, if you build a solution that has some level of high potential error rate, you are likely to have errors flow into the decision-making, because people won't validate it.
That said, humans have error rates too in their analyses... so...
Yeah, and, part of it is that this sort of problem seems ok to be error prone? It's analytical prototyping, and exploring ideas. Which seems to be the work that data teams always claim to want to do - "finding strategic insights," or whatever - but never is, because they're always stuck on the reporting. And the irony is, we now seem to have a tool that maybe can help with that former thing, but everyone's stockholm syndromed themselves into thinking their job is careful reporting, and that's exactly what the tool *isn't* that good at.
I spent some time doing contract work with a team of analysts that almost exclusively used AI for analysis. I don't mean that they asked AI to write complex SQL queries or occasionally asked it to label unstructured data. I mean that they uploaded the dataset and told it to "act as a senior analyst" and write the report, and then they copy and pasted the report into a doc and sent it.
They weren't being lazy or irresponsible, that's quite literally what we were instructed to do, and the workload didn't allow for anything much more 'bespoke' than that. I used AI to write Python scripts or Google Apps scripts or to help me finesse the wording of a tricky paragraph, but I refused to use it for wholesale analysis because every time I tried, the result was absolute nonsense. Or at least it was like 30% nonsense, which I personally believe is too much nonsense.
But what I realized over time is that it actually didn't matter. The people charged with reading the reports were mostly not reading them anyway, and the ones that did mostly didn't care if the data was even accurate, let alone if it was statistically significant or used a "rigorous methodology." They just wanted a stat they could bring to their boss to say "something I did worked" or to say "here's why we should do this idea I have."
This reminds me of when I started in consulting and a senior manager had to send a status report each Friday changing the date in the filename. He knew nobody read it so he just sent the same file every week rather than sweat over every line.
What are you talking about? Data analysts are cautious because if you’re wrong once, stakeholders stop trusting you. When I’ve tried to let copilot act as a senior analyst, it spits out nonsense so I use it to write subqueries and individual cells, which it’s quite good at.
Also, like there’s real world consequences to being wrong? Just on a daily basis, things stop adding up. And, since I work in product growth, the product stops growing lol
It sounds like your work is much more directly connected to outcomes than the work on this team was, which is enviable! I suspect most analysts on that team would have loved for their work to have more real world impact.
Apologies — I meant to reply to Ben, not you. But, yeah, I mean, sadly, lots of consultants bullshit their way through life. A long term embedded data team can’t plausibly get things wrong all the time and not see consequences
What you said though really hits the nail on the head for me. The reason I care about data accuracy is that I think data should be (one of the things) used to make actual decisions that have actual real world consequences. When it's just window dressing, it genuinely kind of doesn't matter if it's accurate or not, which I find depressing.
I think the distinct I'd make here is on reporting that's presented as fact, vs "research" or things that are in effect more like brainstorming. If we ask AI to replace data teams by automating their reporting work - and to do so with the same implicit guarantees of accuracy that that comes with today - for sure, that probably creates all sorts of problems.
But does that change if you're asking it to look for compelling moonshot ideas (which is often the sort of work that data teams aspire to do, and say they're bogged down in the reporting work). Even if it's not always accurate, that email - "Ten hypotheses about big things we could do" - is one that I'd open every week.
True!!
Hmm, I don't know about this one. There is the implementation-focused techbro viewpoint on analytics: "I just build dashboards when asked, I don't know who uses them, and I don't care - probably they don't either", and then there's actual analytics. Say, in a manufacturing setting, for example, an analytical solution that optimizes the various inputs of a chemical process and provides the operator of a honking big machine with instructions and warnings is *hardly* the same as the stereotypical concept of a monthly board report that no-one reads.
It's fun to attack the latter as pointless busywork, and such busywork could (and maybe should) be easily replaced by an army of bots fishing for "insight", but that's not nearly the full picture. In real life the numbers and trends and metrics do actually often matter, some analytics teams do actually work together with their business counterparts, and if you (or your bot army) screw these up you lose their confidence in you and probably your job as well (or you end up assigned to dashboard busywork duties).
On the manufacturing plant problem, for sure, I tend to separate "analytics via recommendations" from true operational stuff that tells you if machines are running correctly or optimizes your delivery fleet or whatever. To me, those aren't really problems where you're looking to discovery anything, and they're more akin to outright engineering problems than data problem (eg, they are programs with numbers as much as they're analytics). And business dashboards seem generally similar to me, except, because they're usually much cheaper to build than the operational stuff, there are a lot of useless ones. But, in both cases, those aren't really quite "analytics" to me, and to the extent that they are, it's because "useful insights" are often hard to find, so we make dashboards, because we need to do something to make ourselves feel useful.
On the point about trust though, that's certainly true today. I'm not convinced it stays that way, though. People care a lot about trust in analysis today because it's slow; people think about it fairly slowly; etc. If analysis becomes much more about rapid brainstorming - here are my ten ideas today, what do think? - mistakes are just part of the process. You say, "these two sound really interesting, go look into them" and then you find out one is wrong, and that's ok. I think trust matters to data teams because they've sold themselves as a team you're supposed to trust. The could also become a team with wild ideas that you might want to double check, and I think that's a perfectly coherent approach too.
As you said: “Claude is a bunch of loops.”
Isn’t that everything in life and business? A loop of gathering information to guide the next loop (that gathers more info)?
Analysis is literally this. And as these things become more widely adopted the tentacles of those loops will reach into every single department, regardless of our concern.
The loops don’t scare me. The people who don’t want this do. They concern me greatly. How did they learn anything? A feedback loop. An imperfect one that was refined over time that eventually built habits, beliefs, and intuition (we call this “taste” now).
It’s how any child learns anything.
AI won’t eat your lunch; it’ll just make it far less interesting than the kid who brought a full course meal to their grade school cafeteria and then shared it with others.
And we all know what it’s like to show up with a soggy ham sandwich.
I wasn't really a believer in this for a while, but more and more, I struggle coming up good counter-arguments. To your point, it's less about how AI is doing something divine; it's more that I don't think I do anything all that special. I ask myself questions, I look stuff up, I repeat.
People periodically say stuff like, ah, but, the AI doesn't know. Which is true, but that feels very stuck in the "AI is ChatGPT" mindset, where it's just prompt and response. But we seem pretty far beyond that at this point. Once a model gets good enough to deal with fairly ambiguous stuff, and once a harness gets good enough at looping itself without spiraling out of control, it's hard to see where that stops.
I think about chaos and how a lot of people see that as scary; they avoid anything that feels, smells, and tastes like it.
But then the few say: Creativity and innovation require the context of chaos to even begin. Out of nothing we create something. Out of the storm delivers a real gem.
The question is one of control. Anyone trying to "control" AI has already taken a backseat. This is so counter-intuitive for most that they won't even consider their built-in bias to control because it's based on fear.
But the best entrepreneurs and inventors across time and space and history never really saw it that way. They saw it as part of their canvas. Their plaything. I think AI is the same. Let the mf just go.
I'm not sure I'm quite so optimistic about it, but I'm also pretty sure there's nothing to do to stop it at this point. And at least as far as creativity goes (to the extent that a blog about dumb tech nonsense is "creative"), smashing a bunch of weird chaotic ideas together in your head is the only reliable way I've found to come up with anything. So, we at least have a lot of that these days.
"Once a model gets good enough to deal with fairly ambiguous stuff..."
I'm not sure there is any reason to believe this will happen. An LLM is not good at ambiguity. It is not good at nuance. This isn't surprising because their very architecture is not built in a way to handle this. It cannot reason.
Genuine question - why do you say that? At least in some domains, I find them very good at that.
Like, one of the things I've realized as I use them is I talk to them at the wrong level. It's very hard not to give them specific instructions about what you want and what they're supposed to do, but increasingly, I've realized you're better off just telling them what you're thinking. If you're stuck, say that. If you have a problem and don't know how to solve it, say that. If someone's complaining about this thing and you don't know what to do about it, say that. Trying to translate your ideas into instructions - ie, trying to resolve the ambiguity, rather than giving it to it - seems like what you're supposed to do. But, man, they seem to do just fine when I don't do that.
I guess it depends on what you're trying to accomplish (and, maybe what you mean by ambiguity). If you're brainstorming, sure, be general and ambiguous. That will probably lead to less predictable inference.
I'm thinking about trying to get valid answers from an LLM, where the words I use in my question / prompt are intentional. In my experience so far, the LLM doesn't process them very well. It ignores some or makes assumptions about something. It confidently gives me false answers; it does not ask for clarification before answering. I can't imagine how being less precise will help. If I ask a question that is more ambiguous, it's still going to process my prompt into tokens and do its inference thing. They don't seem to ask clarifying questions before answering, like a thoughtful human might (no, not all humans ask clarifying questions before answering questions).
Maybe I'm not using the right model, someone will say. Maybe.
Yeah, that makes sense. I think that was a bit of the turning point for me on these things. As answer machines where I have a question and ask for a factual answer, they're not great. They are like people in that way, where you can only kinda sorta trust it. But if you ask them to help work through something (and often, i just say that - "talk through this with me") they can be quite good. They can still be a little over eager, but they often just ask questions back, and they often either say something useful, or their questions make me realize something useful on my own.
Sounds like you're describing Rubber Duck v2.0
I think that our current framing for junior analyst positions will go away once the right scaffolding is in place. I'm less confident that Sr+ goes away, except for roles that didn't really need effective analysts.
And so, if you work in credit analytics, finance, internal consulting, or something where accuracy is a critical to quality value, then I expect the need to not change very quickly.
yeah, I think that's fair. The argument here definitely isn't that they are all that accurate. It's much more that, for certain types of problems, volume - and especially creative volume - can sometimes be more useful than accuracy.
I think saying OpenAI and Anthropic are two of the world's most valuable companies needs a huge caveat; this is only because a relatively tiny number of a certain class of investors. A class that is wrong far more often than it is right. We'll see.
Fair, though, if someone gave you chance to take a big short on either of them, would you do it? The smart answer may well be yes, but, I'm not sure I'm taking that bet.
Probably not, because I don't like risks like that. Not to pick on them specifically either. Other pre-IPO companies are / have been similar. Arguably Databricks is overvalued. Snowflake came out of the gate way too hot when it IPOd (I was there).
Sure, yeah, and I certainly wouldn't claim to have any particular takes on how much these companies should be worth, or if something like Databricks is overvalued or not. But OpenAI and Anthropic definitely seem like *could* be enormously valuable; they both do make tons of money (in revenue, obviously); and they bot do seem to be almost impossibly influential. Of course, they could also implode, though I don't think that changes the underlying point, which is that you can get a lot of attention and traction by doing what seems like the extreme thing.
Raise your hand if you knew one of the 3 "soundtracks to my life" links would open Griff on Spotify.
a man must have a code
This is a useful provocation. The obstacle to greater autonomy may be less the agents themselves than the absence of an analytical equivalent to tests and CI.
The broader direction seems to be AI checking AI, but that only becomes practical when the checker is grounded in something external: benchmark cases, semantic definitions, reconciliations, domain invariants, or observed outcomes.
Perhaps the path forward is broad autonomy in exploration, paired with much stronger verification infrastructure?
I agree with that concept, but I have no idea how you actually test it. I talked about this some here, and still don't really have any answers: https://benn.substack.com/p/can-analysis-ever-be-automated
After reading your other article, I think we may have slightly different takes on this.
I agree that an artifact cannot be verified without access to the underlying data. Even when the data is public, an independent tool would still need to know which datasets, definitions, and methodology were used.
What I had in mind was something closer to an internal auditor. The AI data scientist would provide the artifact, identify the underlying data, and explain the methodology. There might be some back-and-forth before the auditor either rejected the work or approved it, possibly with qualifications.
The external benchmarking would test the auditor itself against known good and bad artifacts, along with their underlying data. I am imagining something analogous to testing coding agents against historical GitHub issues and PRs. The goal would be to establish that the auditor is generally effective at reviewing analytical work.
I am not sure whether this would work, or what the benchmark dataset would look like. It also would not help us trust artifacts produced by other organizations. But it might make us more comfortable giving AI greater autonomy to perform analytical work within our own.
BTW, I loved your take on crypto hype. :)
I work as a data scientist and my work is some combination of model development, ML pipelines, and analytics infrastructure and analysis. I use AI near constantly in the form of both a coding agents and chatbots to consult with.
The difference between using an LLM for coding and for analysis is stark. I can use an LLM generated python pipeline basically out of the box as long as I was careful in my initial prompting. LLM SQL queries and data analysis are bad though; even with careful explanation of the levels of aggregation and ways to approach joins and the full context of the repo generating analytics databases, the outputs are often completely unusable.
I don't think I'm just being cautious; I use LLMs constantly in other areas of my work including for production code. When I say unusable I mean "either unable to run or visibly incorrect to the point where an executive would notice," not just "doesn't meet my carefully honed spidey sense for data accuracy and completeness."
How much do you think that's the harness, versus the nature of the work? My theory on this is basically this:
1. Part of the reason coding tools started to work is because they weren't conservative about costs. They basically said, "loop until you get this right," and built in various testing mechanisms to do that.
2. Admittedly, that's easier for coding tools because you can see if the thing compiles or tests pass. For data work, there isn't as clear of check.
3. But there are *some* checks. People do them, after all. They check if the query works; they compare it to other numbers; etc. There are "tests" in analytical work; they're just a lot messier.
4. LLMs can probably do those too. It's not easy, and it's expensive, but it's not impossible. Still, most analytics vendors don't do that, because it's kinda hard, and because it seems like it would be expensive.
5. But eventually, someone will. And while it will be expensive, it'll still be a lot cheaper than having a person do it.
Which is to say, it would surprise me if the models aren't there already. And what they need isn't necessarily capital C-Context - a perfect semantic ontology and just the right skills - but more like, a lot of access to go sniffing around, and a big budget to spend doing it.
It seems you’re describing the most depressing conclusion of our current economy. Winning is equal to popularity. Accuracy doesn’t matter because no one really cares. Art is defined by output rather than impact. I still believe real art takes time, speed has a cost, and people who care make better stuff even when it may not “win”
On winning being equal to popularity, unfortunately, yeah, that kinda seems to be the world now. (Which, again, I wouldn't say is *good*, but does seem to be how it goes.) On the other stuff though, I'm not sure I'm quite so cynical about it. My point is less that accuracy doesn't matter; it's more that for some type of work - brainstorming, prototyping, searching for valuable analytical needles - accuracy isn't as useful as exploring a bunch of creative ideas. It's less about nobody caring about something being good, and more about finding ways to point the machine that sometimes gets things wrong at stuff that it's ok to now always get entirely right.
Is the valuation, and the hype, coming from the fact that no one has a sane counterpoint that is able to scale the way these cowboy companies are? In the absence of which, FOMO wins.
I have no idea, but that still says something? Like, maybe they're winning the "wrong" way, but winning is still winning.
They are winning until a valid counterpoint arrives. Just like all the banks in the old, "wild" west, these companies will collapse into a heap of dust.
true, though, also, "the market can stay irrational longer than you can stay solvent," as they say.
Analysis is testable in the same way that programs are testable. You can test that you are using the statistics correctly. But which statistics are you using and why?
AI isn't good at figuring out what program needs to be written in the same way that it isn't good at encapsulating all the knowledge that goes into data analysis.
Sure, insofar as analysis is the math. But I think analysis is a lot messier than that in two ways? First, it's really hard to judge what is "right." Is playing the numbers right if the result is bad? In cases where you don't really know the numbers (which is most situations), how do you even know if you're playing the numbers? And second, even if you can say if the math is right, a lot of analysis is just judgement. It's closer to a legal argument than a mathematical proof. And what makes a good legal argument? There's some amount of symbolic logic, but a lot of it is fuzzy grey areas.
https://open.spotify.com/track/6FMIVQPZg9cmMY8hPxAacD?si=FB1fKzPNRjGmvqRnshCteQ
Essentially open the Pandora's box of "give it full control" and let the end users handle the rest.
It's true to a point. How many flawed analyses and massaged metrics and nonsense graphs have business users generated and accepted over the years before the AI slop came for us all? Why wouldn't they want to pay the low low cost of having AI spit out some stuff so you can call yourself "data-driven"?
If the bar is in hell, why the hell not? No need to clutch pearls in the attention economy.
Does that change if we know that's what it's trying to create though? I agree that a bunch of robots that crank out "look, I made you a yet another sales tracking report!" isn't useful at best, and drowns you in untrustworthy slop at worst. But it's more of a brainstorming type thing, where it attempts to propose big ideas (which is often what data work aspires to be, and gets stuck in the reporting prerequisites), I'm not sure it has the same issue. To me, slop is slop because it's trying to pretend it's something else, or hide itself among more valuable things. So what if it's more direct about what it is?
https://www.youtube.com/watch?v=-1F7vaNP9w0
it's not looking good out there: https://theshamblog.com/an-ai-agent-published-a-hit-piece-on-me/
OMG... the time to enshittification - breathtaking.
This car is going very fast
I dunno man... I feel like there is a balance to be found between good structure and YOLOing with AI: https://www.loom.com/share/a6e7051bb22642e5b9de4642b60fe67a (shameless plug, sorry)
And speaking with analysts myself, more and more are picking up on this direction.
Ok, first, this is pretty cool. Second, I do wonder what the right balance is here. On one hand, all the semantics and guardrails seem like they should be useful, and you do want these sorts of analyses to be right (like, obviously). On the other hand, it does seem possible to put too many guardrails in place. In one way, where it constraints the creativity of the model, and nudges it to much towards reporting and whatnot (that seems resolvable though). And in another way where, if it's accurate enough, you might start to treat it like a BI tool and always want the right answers, rather than a brainstorming tool that can be wildly off, but interesting.
True. It depends on what your expectations are, as a user. What I'm noticing is that even though people know "AI is prone to errors" and "you need to validate whatever AI tells you", nobody does. Whatever comes from ChatGPT, people tend to treat as true and reliable. They rarely validate it.
So, if you build a solution that has some level of high potential error rate, you are likely to have errors flow into the decision-making, because people won't validate it.
That said, humans have error rates too in their analyses... so...
Yeah, and, part of it is that this sort of problem seems ok to be error prone? It's analytical prototyping, and exploring ideas. Which seems to be the work that data teams always claim to want to do - "finding strategic insights," or whatever - but never is, because they're always stuck on the reporting. And the irony is, we now seem to have a tool that maybe can help with that former thing, but everyone's stockholm syndromed themselves into thinking their job is careful reporting, and that's exactly what the tool *isn't* that good at.