AI's conversion problem
The paradox of infinite choice that has ruined online dating is coming to GenAI based projects as well.
Earlier this week, the Times of India carried an article that IT firms are facing a “GenAI fatigue and revenue challenge”.
Most seem to be grappling with the challenge of transitioning from GenAI proof-of-concepts (PoCs) to creating a revenue upside from these projects. “For many players working on GenAI projects, it's an efficiency play now. There aren’t many use cases that demonstrate a revenue upside for GenAI projects. And when that happens, it would be the tipping point," said LTIMindtree CEO & MD Debashis Chatterjee.
Basically lots of people (including IT services firms) are building apps on top of LLMs, and offering them to clients. Clients want to try before they buy, and so far they have been extremely hesitant in terms of buying.
I did a very unscientific dipstick survey of people I know who are building around AI, and who have been doing PoCs, and my findings there corroborate the ToI story (at Babbage, we are still in the early stages of building, and yet to get to the POC stage). So what is happening here?
I can think of a few reasons on why GenAI projects are failing to produce revenue for the vendors.
LLMs are arbit
I wrote an entire post on this a couple of weeks ago:
Basically LLMs are nothing like what we’ve seen before. They succeed and fail in mysterious ways. When they succeed, we apply our prior models of what success in a particular problem means, and that leads to overinflated expectations. When they fail, again we apply our (wrong) prior models of what failure in a problem means, and our expectations crash.
When you do a full-fledged POC, you are likely to encounter a fair number of spectacular failures, which are spectacular enough that you don’t trust the system any more (based on your prior mental models). And so don’t want to spend money on it.
Uncreative products
It has almost become a meme nowadays to call GenAI-based products as “thin wrappers around LLMs”. Given that LLMs have become popular thanks to the chat interface of ChatGPT, a lot of the products built around it also come with a chat interface, but for a specific issue.
For example, you can use LLMs for efficient “enterprise search” (where a lot of the company documentation is placed as part of the LLM’s context, or in fine tuning, and you can ask questions about it). Or you have the copilots, where include (among other things) text to SQL engines, coding assistants, etc.
The problem with the chat interface is not only the hallucination but also that users need to learn to prompt engineer effectively. And this introduces friction into effective usage.
Cost
Most POCs based around GenAI typically use cloud-hosted APIs of models such as GPT or Claude or Gemini or … . All of these come with a per-token (approximately, a token is “almost” a word) cost. At the POC stage, it is unlikely for vendors to have optimised sufficiently in terms of cost.
And anything that goes beyond a simple question and answer system can see costs blow up - if you think of a grammar / tone checking system, all the text you input is cost. If you want to upload reams of data for data analysis, that is again a lot of tokens and high cost.
ChatGPT comes at a fixed cost, but the moment you start using it via API, you suddenly start seeing your costs blow up. And clients start thinking twice.
Privacy
For understandable reasons, companies are paranoid about sending out their data out of their clouds. There was an early report on an engineer in Samsung using ChatGPT to optimise his code, and the code getting “leaked” to ChatGPT. This has meant InfoSec and Legal departments are putting additional scrutiny into any product that has a remote mention of “GenAI”.
Congestion
Congestion is a concept in economics where you have so many options that you cannot fairly evaluate all of them. I even have a chapter on this subject in my book, and happily enough, that one was excerpted by Mint.
In other words, dating platforms in India have a significant congestion problem. Women have to wade through a large number of profiles before they can find men who they might want to date. Men, on the other hand, need to stand out in the crowd and ensure they get noticed by women, apart from whatever else they have to convey. Thus, both sides of the market face significant transaction costs, resulting in low market liquidity.
Now, GenAI based projects are not a two-sided market, but you can see this concept playing out on one side. CIOs (and other buyers) nowadays are faced with so many proposals for GenAI related projects they basically “end up putting NED” for all of them.
Yesterday I read a post by a VC on LinkedIn about how this is causing sales cycles to get longer. And when you have lots of people proposing to you, even if you have done a few POCs, you are indecisive because you want to see what else is there in the market. The paradox of infinite choice that has ruined online dating is coming to GenAI based projects as well.
Peer Pressure, or lack thereof
Finally it comes down to this. Right now the market is in a Nash equilibrium where nobody is really investing in GenAI related projects. Everyone is doing POCs (because everyone else is doing POCs) but no one is buying (because no one else is buying). The market needs some kind of perturbation (I don’t know what that is) that can snap it to the other Nash equilibrium of everyone actually buying GenAI.
So what can we do about this?
Given we are yet to hit the POC stage, we haven’t thought about this seriously, but off the top of my head, there are a few things we can do.
Differentiate: In a generally crowded market, we need to stand out, and do so in a visible manner.
Hide the AI: We have consciously decided to not have a “.ai” domain. One possible strategy here is to make the AI inconspicuous, like Apple has announced as part of its recent WWDC announcements. This article by Ben Evans talks about this nicely.
On Prem Deployments: This means using an open source LLM so that the data doesn’t leave the customer’s cloud. This can take care of both cost and privacy issues. The downside is that open source LLMs will always lag proprietary LLMs in terms of capabilities and never be “state of the art” (I frankly get irritated by people constantly playing “mine’s bigger than yours” games in terms of LLM benchmarks). However, if your product can work well even without the “latest and best LLM”, you will be able to do well here.
Product design: This again points to the way Apple is implementing AI, but perhaps the “conventional” chat interface is not the best way to implement GenAI? Maybe other kinds of interfaces might work better?
Again I haven’t given this too much thought. I better do better!
I first published this on LinkedIn. Cross posting here for posterity.
Hard to keep up with so much good writing from you. But keep going because you're currently in very good form.