Announcing Bito’s free open-source sponsorship program. Apply now

Get high quality AI code reviews

Are LLMs Commoditizing?

Table of Contents

One of my friends is looking at an investment in Anthropic. He asked me what I thought – are the LLMs going to get commoditized? I don’t have THE answer, but here are a few thoughts: 

  1. There are a lot of data points saying LLMs are getting more expensive to train. Witness the huge rounds of funding Anthropic and OpenAI are raising, mostly for GPUs for model training.  
  1. Dario, CEO of Anthropic, has said that it costs $100M to train a model today. In a few years he said it will cost $1B. And in 5 years it could cost up to $10B.  
  1. But… every day, a new LLM seems to come to market. If you look at the leaderboards, you have decent to best in class LLMs from OpenAI, Anthropic, Meta, Google, Mistral, Cohere, X, Cohesity, Inflection, and Databricks. Amazon is now working on one too.  
  1. Our experience at Bito, while using a number of the LLMs mentioned above, is that there are significant differences when you are trying to do complicated use cases. For example, if you try to get structured output with complex reasoning required, GPT-4 and Claude 3 far exceed the capabilities of other models such as Llama 2 and related models like Mistral. If you want to write a paragraph about Paris, the models are similar.  
  1. Interestingly, most of these model companies are not that big in terms of headcount. Mistral launched its first model when there were 10 people. Anthropic said Claude 3 was trained by 60 people.  
  1. I was talking to a top 5 exec at a top 5 cloud company, and he said he is training a 500B parameter model for $3M. And most of the money goes into failed training runs where you find out after 4 weeks you made a mistake. The successful run takes 2 months to run.  

So, is a lot of capital going to be required to build models? Or is it mostly about the people?  

If it’s capital, then we are probably headed for a world where there will be a few LLMs that matter. And even if there are just a few, and they are similar, they will be an oligopoly and capture a lot of profits. As an example, the 3 major cloud vendors (AWS, Azure, and GCP) are similar in capabilities, but have cornered the market.  

If it’s cheap to train an LLM and the gating factor is people, then we will probably end up with many LLMs. People are relatively scarce right now, but that will change as the field grows, and more and more people learn how to train a model. There are probably 10x the number of people that have trained a model today than 18 months ago.  

One other point I’ll make is around data. If your LLM is truly leveraging a data set that is hard to get or not readily available then you could be the LLM for that vertical. Like Hippocratic AI is focused on healthcare, and is doing some unique stuff there leveraging actual patient voice conversations and encryption. That’s pretty different from OpenAI.  

So, is Anthropic a good investment? My gut tells me that the cost of training models has been going up. And will continue to go up. So capital will become a more defining feature. I think we will see a few big LLMs over time. So seems reasonable that Anthropic could be a winner. Is the $18B valuation at a reasonable price… well, that’s a different question for a different post!

Picture of Amar Goel

Amar Goel

Bito’s Co-founder and CEO. Dedicated to helping developers innovate to lead the future. A serial entrepreneur, Amar previously founded PubMatic, a leading infrastructure provider for the digital advertising industry, in 2006, serving as the company’s first CEO. PubMatic went public in 2020 (NASDAQ: PUBM). He holds a master’s degree in Computer Science and a bachelor’s degree in Economics from Harvard University.

Picture of Amar Goel

Amar Goel

Amar is the Co-founder and CEO of Bito. With a background in software engineering and economics, Amar is a serial entrepreneur and has founded multiple companies including the publicly traded PubMatic and Komli Media.

Written by developers for developers

This article was handcrafted with by the Bito team.

Latest posts

Recent releases: Pick your AI model, create PR from IDE, integrated Linter feedback, and more

PEER REVIEW: Shubham Gupta, Chief Technology Officer at ToolJet

Ultimate Java Code Review Checklist

Ultimate Python Code Review Checklist

13 Best Java AI Coding Tools 2024 [Free & Paid]

Top posts

Recent releases: Pick your AI model, create PR from IDE, integrated Linter feedback, and more

PEER REVIEW: Shubham Gupta, Chief Technology Officer at ToolJet

Ultimate Java Code Review Checklist

Ultimate Python Code Review Checklist

13 Best Java AI Coding Tools 2024 [Free & Paid]

From the blog

The latest industry news, interviews, technologies, and resources.

Get Bito for IDE of your choice