I like how the English speaking world came up with Legalese because English is so ambiguous, and now a whole bunch of people pretend that they can program computers with natural language. 😅
After reading the first half of this I was going to make the point that humans who had “natural language” at their disposal choose to use SQL instead of natural language to interact with data…. But you pretty much made that point already.
Secondly - everyone think back to your analyst days… When is the last time you got a coherent English paragraph as a request for an analysis/dashboard/report? Mine were always the results of a hand waving conversations with torn off sheets from yellow legal pads with unreadable diagrams and handwriting. So if AI can deal with that - sounds awesome!
Tell your brother to check out Change Research for a modern polling / data analytics firm that works with state legislature candidates all over the country (disclosure: I used to run data science there)
Stephen Wolfram's point has always been that a computational language (read: Mathematica, of course) cannot be displaced by natural language. Computational language differs from natural language precisely insofar as it is constrained and unambiguous: one thing cannot mean another thing. There is no "but I actually meant this." The logic and reasoning of computational language is reproducible - you can always follow the same inputs and get the same outputs.
Natural language is not as precise nor as constrained. Words have "fuzzy borders." Interpretation is ambiguous and not even reproducible person-to-person (i.e. people will have different interpretations of the same words).
For example, if I asked ChatGPT for "the daily average temperature over the last month", am I asking for the average temperature each day over the month (line chart), or an average of that line (a number)? What if I wrote "the average daily temperature over the last month" instead? Not clear! But in SQL of course it is very clear.
However, just like with other people, we can have a discourse in natural language to arrive at computational language. This works from business people to programmers, so it is possible that one day it works from business people to ChatGPT as well.
I absolutely love your thoughts and fresh takes regarding Data, AI, all things "Tech". A few points stuck out to me about today's blog that I would like to comment on:
-I've never learned SQL, but I do know python very well (especially data frames, data analysis, etc). Your description of SQL as a cockroach curiously makes me want to learn it now, haha. Thought that was such a great descriptor. Also your knowledge and passion for SQL also makes me want to learn it, if for no other reason that to see what I am missing.
-Strangely enough (and especially after admitting my lack of experience with SQL) I think that what AI does best--better than any other task I have found--is the technical translation from Natural Language (plain english) into workable code. For Example, I spent about 3 months writing a ML algo stock options trading bot based upon a random forest algorithm, and last night I decided to "try again" rebuilding a ML Algo stock options bot from scratch using AI, and asking the AI for specific recommendations and to write the code itself, and "we" decided on a Gradient Regression, which works perfectly and I used today for the first time on Apple Stock (worked great). On this vein, I've recently heard about the lack of COBOL users, legacy code, etc, and actually I bet that AI is able to fill in as COBOL programmer for legacy code / fix issues that many people are unaware of (a bit of my own side project). So I find it very...strange (?) that AI is unable to work with SQL, and I know that AI is powerful at data analysis tasks--again this is another reason I am curious to explore SQL a bit further.
Anyways just wanted to say thanks for your blog, writing, and thoughts. You have a new perspective on something that I think has too many thinkers thinking in the same direction ("AI WILL REVOLUTIONIZE THE WORLD"), but yours is a new and refreshing take. One idea that I loved during the data camp podcast was your idea of "teleportation" in technology, that instead of going one direction quickly (which is the way I always thought of it as), we are "teleporting" to an entirely new, and unexpected area. I will definitely be listening to that podcast a few more times!
Yes there is, but in order for it to be reliable and high-performance, it cannot rely on LLMs alone. Liquify Analytics is building it. It’s coming along great.
I like how the English speaking world came up with Legalese because English is so ambiguous, and now a whole bunch of people pretend that they can program computers with natural language. 😅
After reading the first half of this I was going to make the point that humans who had “natural language” at their disposal choose to use SQL instead of natural language to interact with data…. But you pretty much made that point already.
Secondly - everyone think back to your analyst days… When is the last time you got a coherent English paragraph as a request for an analysis/dashboard/report? Mine were always the results of a hand waving conversations with torn off sheets from yellow legal pads with unreadable diagrams and handwriting. So if AI can deal with that - sounds awesome!
Can't agree more with you Benn. Resonates a lot with recent writing (https://medium.pimpaudben.fr/sql-is-not-designed-for-analytics-079fc97b139c): SQL wasn't design for Analytics in the first place.
Still not mature enough, but I definitely think that the idea behind Malloy (aka. "data is not rectangular") is something too not overlook.
Any views on Malloy on your side ?
Tell your brother to check out Change Research for a modern polling / data analytics firm that works with state legislature candidates all over the country (disclosure: I used to run data science there)
I'm still upset the name "draggy-droppy BI interface" didn't make it into the Mode product. I fought so hard for that.
Stephen Wolfram's point has always been that a computational language (read: Mathematica, of course) cannot be displaced by natural language. Computational language differs from natural language precisely insofar as it is constrained and unambiguous: one thing cannot mean another thing. There is no "but I actually meant this." The logic and reasoning of computational language is reproducible - you can always follow the same inputs and get the same outputs.
Natural language is not as precise nor as constrained. Words have "fuzzy borders." Interpretation is ambiguous and not even reproducible person-to-person (i.e. people will have different interpretations of the same words).
For example, if I asked ChatGPT for "the daily average temperature over the last month", am I asking for the average temperature each day over the month (line chart), or an average of that line (a number)? What if I wrote "the average daily temperature over the last month" instead? Not clear! But in SQL of course it is very clear.
However, just like with other people, we can have a discourse in natural language to arrive at computational language. This works from business people to programmers, so it is possible that one day it works from business people to ChatGPT as well.
Benn,
I absolutely love your thoughts and fresh takes regarding Data, AI, all things "Tech". A few points stuck out to me about today's blog that I would like to comment on:
-I've never learned SQL, but I do know python very well (especially data frames, data analysis, etc). Your description of SQL as a cockroach curiously makes me want to learn it now, haha. Thought that was such a great descriptor. Also your knowledge and passion for SQL also makes me want to learn it, if for no other reason that to see what I am missing.
-Strangely enough (and especially after admitting my lack of experience with SQL) I think that what AI does best--better than any other task I have found--is the technical translation from Natural Language (plain english) into workable code. For Example, I spent about 3 months writing a ML algo stock options trading bot based upon a random forest algorithm, and last night I decided to "try again" rebuilding a ML Algo stock options bot from scratch using AI, and asking the AI for specific recommendations and to write the code itself, and "we" decided on a Gradient Regression, which works perfectly and I used today for the first time on Apple Stock (worked great). On this vein, I've recently heard about the lack of COBOL users, legacy code, etc, and actually I bet that AI is able to fill in as COBOL programmer for legacy code / fix issues that many people are unaware of (a bit of my own side project). So I find it very...strange (?) that AI is unable to work with SQL, and I know that AI is powerful at data analysis tasks--again this is another reason I am curious to explore SQL a bit further.
Anyways just wanted to say thanks for your blog, writing, and thoughts. You have a new perspective on something that I think has too many thinkers thinking in the same direction ("AI WILL REVOLUTIONIZE THE WORLD"), but yours is a new and refreshing take. One idea that I loved during the data camp podcast was your idea of "teleportation" in technology, that instead of going one direction quickly (which is the way I always thought of it as), we are "teleporting" to an entirely new, and unexpected area. I will definitely be listening to that podcast a few more times!
this assumes that we have business questions that AI hasn't already answered for us!
Yes there is, but in order for it to be reliable and high-performance, it cannot rely on LLMs alone. Liquify Analytics is building it. It’s coming along great.