Yet another reason why data insights is a good use case for LLMs
Getting insights out of data is not something humans are particularly good at, and there is no notion of "correctness" in this. Hence it's a good use case for LLMs
A few months back I came across this talk by Martin Casado of a16z where he talks about the “economics case for generative AI”. Actually I’d recommend you to watch this video in full:
I’ve screenshotted what I think is the key slide in his presentation, and this talks about the necessary conditions for generative AI to succeed:
Now consider the problem of generating insights from data.
That it is applicable to many markets is trivial. In fact I wrote the other day in terms of our difficulty in narrowing down our ideal customer profile
The second is to do with correctness. If you are attacking a problem where humans can very easily get it right, that raises the benchmark that the AI has to hit in order to be acceptable.
Generating insight from data, though, is more or an art than a science (check the URL of this blog), and often there is no “correct answer”. And this means that the AI’s analysis is largely acceptable (and adding a human in the loop, to “verify” the answer, doesn’t necessarily improve the quality of insight)Related to this is the fourth point - analysing data is an art, and prone to silly mistakes. It is fairly easy to make small errors in a piece of complex analysis, and generate the wrong business recommendations based on that (I’ve been guilty of this several times, and have seen lots of others commit such mistakes as well). If not anything, AI is unlikely to make such silly mistakes, especially since it doesn’t have emotion.
Moreover, AI doesn’t have bias (I’m not talking about THAT kind of bias, which happens due to garbage training data) in terms of what the analysis ought to be. And this lack of bias can itself result, on average, in far superior analysis than what a human can offer.
Finally, we can (hopefully) program LLMs to not fall prey to cognitive biases, which human data analysis is rather rife with.The third point, of course, is trivial when it comes to data analysis - it is by definition a software product.
Around the same time I watched this lecture, I came across another post from another a16z partner Vineeta Agarwala. In this, she draws out a simple 2x2 on tasks that are easy and hard for humans and AI.
Analysing data is hard for humans, and not that hard for AI (or so I hope, based on the hypothesis that we have started off with), and that implies again that data insights is a good use case for generative AI.