Per usual, to what end? Why would any organization "need" a decision to trace back to the LLMWHY? How will those LLMWHYs be interrogated? Who has the authority to usurp LLMWHY? Dear user, here is WHY:
- the board said so
- an executive said so
- someone's boss said so
- the client said so
- an intern changed the setting at it worked fine so we kept it
- we were past deadline so we shipped it and no one complained
- it was a full moon at the vernal equinox
The text box is as ever, a solution in search of a problem. A visualized, organization-wide decision matrix on its surface is an idea with no merit. This industry is full of people who need hobbies.
I'm admittedly out of my depth here, but I'm...optimistic? Though that's probably not quite the right word?...that LLMs can essentially "learn" to reason, in ways that they end up making better decisions than most people do. Like, to take a simple example, if I wanted AI to write a blog post, the "simple" way that it would do it is:
1. Read stuff I wrote
2. Do a bunch of statistical predictions on how I talk, and try to mimic it.
That works ok, but it's fairly hollow. It just sounds like something I might say, but there's nothing behind it.
But what if, in addition to giving it a bunch of stuff I wrote, I also gave it my internal thought process for how I thought about it? "Ah, that doesn't make sense; I don't like that point; that phrasing is off; etc etc etc." And then, when it tried to write something, it did something like this:
1. Read stuff I wrote and why I wrote it.
2. Do a bunch of statistical predictions on how I talk.
3. Evaluate that against statistical predictions on how I think.
4. Repeat.
How would that go? I mean, I have no idea. But at some point, if I'm complete enough in documenting my reasoning, and it loops in this way enough, I'm not sure what it does and what I do is all that different. Reasoning about writing a blog post is basically a form of next work prediction in my head, combined with a bit of randomness. And at some level of depth, an LLM starts to basically do that exact same thing.
Though it's up to philosophers and people much smarter than me to decide if a computer is a conscious, reasoning thing or not, at some point, the distinction is essentially academic. The models can't do that now, because they neither have the reasoning inputs, nor the computational speed or context windows to loop at that depth. But could they? Will they? It's hard for me to say that they won't.
You don't really think computers are or ever could be conscious, do you? If that's the case, let me kindly prescribe a vacation outside of SF where you will definitely not run into a potential future employer who requires you to harbor or say or pretend to leave room for such things.
What you essentially do when writing and what a theoretically, better, LLM could do are NOT the same thing. You're forgetting the transformer architecture treats context and prompt as the same input, which is a feature not a bug and why they will never be secure enough for widespread corporate usage on all the entire corpus of institutional knowledge.
I know you have to pretend like you don't know these things. But that speaks more to the moment than any of potential uses of the lossy next-word-predictor.
Eh, I certainly don't feel any pressure from anywhere to say (or not say) any of this. (I'm also in NY, though, SF is as much a place on the internet as is it a physical place at this point.)
On the point of "conscious computers," no, I don't really believe in that in a philosophical sense, though I do think it gets pretty hard to tell the difference? And it doesn't seem at all crazy to me to think that an LLM could work across an entire corpus of knowledge. Not in their memory, but if they can loop fast enough, could it read things, ask itself "what do I need to know more about," read things, and so on, and do that as effectively as I could? That seems quite possible to me.
In some ways, my broader point is less about LLMs being some divine technology; it's more about my skepticism of how special *we* are. If you go a few levels of reasoning down, I'm not sure I'm anything more than a lossy, slightly random next-word-predictor that 1) is nudged in inscrutable ways by the various things I've seen in my life, and 2) has the ability to go look up stuff when I hit the end of what I know. And a sufficiently prompted and looped LLM starts to look a lot like that to me.
One of the many things I’ve appreciated about fully remote work is that you have a record of nearly every conversation, every decision, every everything. That can also sometimes be a drag though, like when it turns out you were wrong about something.
I remember having a conversation about this with someone a while back, and we were entertaining an idea that was basically, "What if we tried to run a company entirely online (slack, zoom, etc) and never did anything that was recorded and transcribed, so that there is no "unrecorded context." If you did that and stuffed it into some LLM, what kind of decisions would it make?" I'm not sure the answer is good, but I'm not sure it's that much worse than what people would do either, tbh.
yeah, this isn't a blog with any real point to it, but if it were, that post would probably be one of the few that I find myself repeating over and over again.
Not really? Though I don't actually think it'll get solved directly. The automatically generated "reasons" for a decision seem difficult to get right, because those reasons often aren't recorded at all. They're just in someone's head. And it's next to impossible to get people to write stuff down for the sake of writing it down. That's like documentation - everyone knows you should do it; most people still don't.
So if we do solve this, I think it'll be through something like granola, which starts solving one problem, but then realizes they've sort of accidentally collected the data to make this possible. Or, put differently, it'll be a product that convinces everyone to use it for one reason, but becomes very valuable for this other reason.
Per usual, to what end? Why would any organization "need" a decision to trace back to the LLMWHY? How will those LLMWHYs be interrogated? Who has the authority to usurp LLMWHY? Dear user, here is WHY:
- the board said so
- an executive said so
- someone's boss said so
- the client said so
- an intern changed the setting at it worked fine so we kept it
- we were past deadline so we shipped it and no one complained
- it was a full moon at the vernal equinox
The text box is as ever, a solution in search of a problem. A visualized, organization-wide decision matrix on its surface is an idea with no merit. This industry is full of people who need hobbies.
I'm admittedly out of my depth here, but I'm...optimistic? Though that's probably not quite the right word?...that LLMs can essentially "learn" to reason, in ways that they end up making better decisions than most people do. Like, to take a simple example, if I wanted AI to write a blog post, the "simple" way that it would do it is:
1. Read stuff I wrote
2. Do a bunch of statistical predictions on how I talk, and try to mimic it.
That works ok, but it's fairly hollow. It just sounds like something I might say, but there's nothing behind it.
But what if, in addition to giving it a bunch of stuff I wrote, I also gave it my internal thought process for how I thought about it? "Ah, that doesn't make sense; I don't like that point; that phrasing is off; etc etc etc." And then, when it tried to write something, it did something like this:
1. Read stuff I wrote and why I wrote it.
2. Do a bunch of statistical predictions on how I talk.
3. Evaluate that against statistical predictions on how I think.
4. Repeat.
How would that go? I mean, I have no idea. But at some point, if I'm complete enough in documenting my reasoning, and it loops in this way enough, I'm not sure what it does and what I do is all that different. Reasoning about writing a blog post is basically a form of next work prediction in my head, combined with a bit of randomness. And at some level of depth, an LLM starts to basically do that exact same thing.
Though it's up to philosophers and people much smarter than me to decide if a computer is a conscious, reasoning thing or not, at some point, the distinction is essentially academic. The models can't do that now, because they neither have the reasoning inputs, nor the computational speed or context windows to loop at that depth. But could they? Will they? It's hard for me to say that they won't.
You don't really think computers are or ever could be conscious, do you? If that's the case, let me kindly prescribe a vacation outside of SF where you will definitely not run into a potential future employer who requires you to harbor or say or pretend to leave room for such things.
What you essentially do when writing and what a theoretically, better, LLM could do are NOT the same thing. You're forgetting the transformer architecture treats context and prompt as the same input, which is a feature not a bug and why they will never be secure enough for widespread corporate usage on all the entire corpus of institutional knowledge.
I know you have to pretend like you don't know these things. But that speaks more to the moment than any of potential uses of the lossy next-word-predictor.
Eh, I certainly don't feel any pressure from anywhere to say (or not say) any of this. (I'm also in NY, though, SF is as much a place on the internet as is it a physical place at this point.)
On the point of "conscious computers," no, I don't really believe in that in a philosophical sense, though I do think it gets pretty hard to tell the difference? And it doesn't seem at all crazy to me to think that an LLM could work across an entire corpus of knowledge. Not in their memory, but if they can loop fast enough, could it read things, ask itself "what do I need to know more about," read things, and so on, and do that as effectively as I could? That seems quite possible to me.
In some ways, my broader point is less about LLMs being some divine technology; it's more about my skepticism of how special *we* are. If you go a few levels of reasoning down, I'm not sure I'm anything more than a lossy, slightly random next-word-predictor that 1) is nudged in inscrutable ways by the various things I've seen in my life, and 2) has the ability to go look up stuff when I hit the end of what I know. And a sufficiently prompted and looped LLM starts to look a lot like that to me.
One of the many things I’ve appreciated about fully remote work is that you have a record of nearly every conversation, every decision, every everything. That can also sometimes be a drag though, like when it turns out you were wrong about something.
I remember having a conversation about this with someone a while back, and we were entertaining an idea that was basically, "What if we tried to run a company entirely online (slack, zoom, etc) and never did anything that was recorded and transcribed, so that there is no "unrecorded context." If you did that and stuffed it into some LLM, what kind of decisions would it make?" I'm not sure the answer is good, but I'm not sure it's that much worse than what people would do either, tbh.
LLMs are the new turtles!
all the way down..
for better or for worse, I think that's quickly becoming a lot more true than people realize.
Reminds of AVG(text)
yeah, this isn't a blog with any real point to it, but if it were, that post would probably be one of the few that I find myself repeating over and over again.
Have you seen any good attempts to solve this?
Not really? Though I don't actually think it'll get solved directly. The automatically generated "reasons" for a decision seem difficult to get right, because those reasons often aren't recorded at all. They're just in someone's head. And it's next to impossible to get people to write stuff down for the sake of writing it down. That's like documentation - everyone knows you should do it; most people still don't.
So if we do solve this, I think it'll be through something like granola, which starts solving one problem, but then realizes they've sort of accidentally collected the data to make this possible. Or, put differently, it'll be a product that convinces everyone to use it for one reason, but becomes very valuable for this other reason.
These days, when someone says something in a meeting I just say "@grok is this true?"
and what grok says is true is more and more true, because grok (et al) said it's true.